E: jguyett@andrew.cmu.edu
D: via-rhine net driver hacking
+N: Nitin Gupta
+E: ngupta@vflare.org
+D: zsmalloc memory allocator and zram block device driver
+
N: Danny ter Haar
E: dth@cistron.nl
D: /proc/cpuinfo, reboot on panic , kernel pre-patch tester ;)
KernelVersion: 5.6.0
Contact: dmaengine@vger.kernel.org
Description: The largest number of work descriptors in a batch.
+ It's not visible when the device does not support batch.
What: /sys/bus/dsa/devices/dsa<m>/max_work_queues_size
Date: Oct 25, 2019
The read buffers represent resources within the DSA
implementation, and these resources are allocated by engines to
support operations. See DSA spec v1.2 9.2.4 Total Read Buffers.
+ It's not visible when the device does not support Read Buffer
+ allocation control.
What: /sys/bus/dsa/devices/dsa<m>/max_transfer_size
Date: Oct 25, 2019
Description: The maximum number of read buffers that may be in use at
one time by operations that access low bandwidth memory in the
device. See DSA spec v1.2 9.2.8 GENCFG on Global Read Buffer Limit.
+ It's not visible when the device does not support Read Buffer
+ allocation control.
What: /sys/bus/dsa/devices/dsa<m>/cmd_status
Date: Aug 28, 2020
Contact: dmaengine@vger.kernel.org
Description: The max batch size for this workqueue. Cannot exceed device
max batch size. Configurable parameter.
+ It's not visible when the device does not support batch.
What: /sys/bus/dsa/devices/wq<m>.<n>/ats_disable
Date: Nov 13, 2020
Contact: dmaengine@vger.kernel.org
Description: Enable the use of global read buffer limit for the group. See DSA
spec v1.2 9.2.18 GRPCFG Use Global Read Buffer Limit.
+ It's not visible when the device does not support Read Buffer
+ allocation control.
What: /sys/bus/dsa/devices/group<m>.<n>/read_buffers_allowed
Date: Dec 10, 2021
Description: Indicates max number of read buffers that may be in use at one time
by all engines in the group. See DSA spec v1.2 9.2.18 GRPCFG Read
Buffers Allowed.
+ It's not visible when the device does not support Read Buffer
+ allocation control.
What: /sys/bus/dsa/devices/group<m>.<n>/read_buffers_reserved
Date: Dec 10, 2021
Description: Indicates the number of Read Buffers reserved for the use of
engines in the group. See DSA spec v1.2 9.2.18 GRPCFG Read Buffers
Reserved.
+ It's not visible when the device does not support Read Buffer
+ allocation control.
What: /sys/bus/dsa/devices/group<m>.<n>/desc_progress_limit
Date: Sept 14, 2022
characters. Set this to a higher value to adjust for the delay
and better synchronisation between cursor position and speech.
+What: /sys/accessibility/speakup/cur_phonetic
+KernelVersion: 6.2
+Contact: speakup@linux-speakup.org
+Description: This allows speakup to speak letters phoneticaly when arrowing through
+ a word letter by letter. This doesn't affect the spelling when typing
+ the characters. When cur_phonetic=1, speakup will speak characters
+ phoneticaly when arrowing over a letter. When cur_phonetic=0, speakup
+ will speak letters as normally.
+
What: /sys/accessibility/speakup/delimiters
KernelVersion: 2.6
Contact: speakup@linux-speakup.org
read-only
bmaControls this format's data for bmaControls in
the streaming header
- bmInterfaceFlags specifies interlace information,
+ bmInterlaceFlags specifies interlace information,
read-only
bAspectRatioY the X dimension of the picture aspect
ratio, read-only
read-only
bmaControls this format's data for bmaControls in
the streaming header
- bmInterfaceFlags specifies interlace information,
+ bmInterlaceFlags specifies interlace information,
read-only
bAspectRatioY the X dimension of the picture aspect
ratio, read-only
Valid values are "disable", "enable", "suspend", "resume".
User can read this property to see the valid values
+What: /sys/kernel/debug/habanalabs/hl<n>/device_release_watchdog_timeout
+Date: Oct 2022
+KernelVersion: 6.2
+Contact: ttayar@habana.ai
+Description: The watchdog timeout value in seconds for a device relese upon
+ certain error cases, after which the device is reset.
+
What: /sys/kernel/debug/habanalabs/hl<n>/dma_size
Date: Apr 2021
KernelVersion: 5.13
--- /dev/null
+What: /sys/bus/coreboot
+Date: August 2022
+Contact: Jack Rosenthal <jrosenth@chromium.org>
+Description:
+ The coreboot bus provides a variety of virtual devices used to
+ access data structures created by the Coreboot BIOS.
+
+What: /sys/bus/coreboot/devices/cbmem-<id>
+Date: August 2022
+Contact: Jack Rosenthal <jrosenth@chromium.org>
+Description:
+ CBMEM is a downwards-growing memory region created by Coreboot,
+ and contains tagged data structures to be shared with payloads
+ in the boot process and the OS. Each CBMEM entry is given a
+ directory in /sys/bus/coreboot/devices based on its id.
+ A list of ids known to Coreboot can be found in the coreboot
+ source tree at
+ ``src/commonlib/bsd/include/commonlib/bsd/cbmem_id.h``.
+
+What: /sys/bus/coreboot/devices/cbmem-<id>/address
+Date: August 2022
+Contact: Jack Rosenthal <jrosenth@chromium.org>
+Description:
+ This is the pyhsical memory address that the CBMEM entry's data
+ begins at, in hexadecimal (e.g., ``0x76ffe000``).
+
+What: /sys/bus/coreboot/devices/cbmem-<id>/size
+Date: August 2022
+Contact: Jack Rosenthal <jrosenth@chromium.org>
+Description:
+ This is the size of the CBMEM entry's data, in hexadecimal
+ (e.g., ``0x1234``).
+
+What: /sys/bus/coreboot/devices/cbmem-<id>/mem
+Date: August 2022
+Contact: Jack Rosenthal <jrosenth@chromium.org>
+Description:
+ A file exposing read/write access to the entry's data. Note
+ that this file does not support mmap(), as coreboot
+ does not guarantee that the data will be page-aligned.
+
+ The mode of this file is 0600. While there shouldn't be
+ anything security-sensitive contained in CBMEM, read access
+ requires root privileges given this is exposing a small subset
+ of physical memory.
--- /dev/null
+What: /sys/bus/iio/devices/iio:deviceX/in_voltage-voltage_filter_mode_available
+KernelVersion: 6.2
+Contact: linux-iio@vger.kernel.org
+Description:
+ Reading returns a list with the possible filter modes.
+
+ * "sinc4" - Sinc 4. Excellent noise performance. Long
+ 1st conversion time. No natural 50/60Hz rejection.
+
+ * "sinc4+sinc1" - Sinc4 + averaging by 8. Low 1st conversion
+ time.
+
+ * "sinc3" - Sinc3. Moderate 1st conversion time.
+ Good noise performance.
+
+ * "sinc3+rej60" - Sinc3 + 60Hz rejection. At a sampling
+ frequency of 50Hz, achieves simultaneous 50Hz and 60Hz
+ rejection.
+
+ * "sinc3+sinc1" - Sinc3 + averaging by 8. Low 1st conversion
+ time. Best used with a sampling frequency of at least
+ 216.19Hz.
+
+ * "sinc3+pf1" - Sinc3 + Post Filter 1. 53dB rejection @
+ 50Hz, 58dB rejection @ 60Hz.
+
+ * "sinc3+pf2" - Sinc3 + Post Filter 2. 70dB rejection @
+ 50Hz, 70dB rejection @ 60Hz.
+
+ * "sinc3+pf3" - Sinc3 + Post Filter 3. 99dB rejection @
+ 50Hz, 103dB rejection @ 60Hz.
+
+ * "sinc3+pf4" - Sinc3 + Post Filter 4. 103dB rejection @
+ 50Hz, 109dB rejection @ 60Hz.
+
+What: /sys/bus/iio/devices/iio:deviceX/in_voltageY-voltageZ_filter_mode
+KernelVersion: 6.2
+Contact: linux-iio@vger.kernel.org
+Description:
+ Set the filter mode of the differential channel. When the filter
+ mode changes, the in_voltageY-voltageZ_sampling_frequency and
+ in_voltageY-voltageZ_sampling_frequency_available attributes
+ might also change to accommodate the new filter mode.
+ If the current sampling frequency is out of range for the new
+ filter mode, the sampling frequency will be changed to the
+ closest valid one.
--- /dev/null
+What: /sys/bus/iio/devices/iio:deviceX/in_voltage_filterY_notch_en
+Date: September 2022
+KernelVersion: 6.0
+Contact: linux-iio@vger.kernel.org
+Description:
+ Enable or disable a notch filter.
+
+What: /sys/bus/iio/devices/iio:deviceX/in_voltage_filterY_notch_center
+Date: September 2022
+KernelVersion: 6.0
+Contact: linux-iio@vger.kernel.org
+Description:
+ Center frequency of the notch filter in Hz.
--- /dev/null
+What: /sys/bus/platform/devices/smpro-errmon.*/error_[core|mem|pcie|other]_[ce|ue]
+KernelVersion: 6.1
+Contact: Quan Nguyen <quan@os.amperecomputing.com>
+Description:
+ (RO) Contains the 48-byte Ampere (Vendor-Specific) Error Record printed
+ in hex format according to the table below:
+
+ +--------+---------------+-------------+------------------------------------------------------------+
+ | Offset | Field | Size (byte) | Description |
+ +--------+---------------+-------------+------------------------------------------------------------+
+ | 00 | Error Type | 1 | See :ref:`the table below <smpro-error-types>` for details |
+ +--------+---------------+-------------+------------------------------------------------------------+
+ | 01 | Subtype | 1 | See :ref:`the table below <smpro-error-types>` for details |
+ +--------+---------------+-------------+------------------------------------------------------------+
+ | 02 | Instance | 2 | See :ref:`the table below <smpro-error-types>` for details |
+ +--------+---------------+-------------+------------------------------------------------------------+
+ | 04 | Error status | 4 | See ARM RAS specification for details |
+ +--------+---------------+-------------+------------------------------------------------------------+
+ | 08 | Error Address | 8 | See ARM RAS specification for details |
+ +--------+---------------+-------------+------------------------------------------------------------+
+ | 16 | Error Misc 0 | 8 | See ARM RAS specification for details |
+ +--------+---------------+-------------+------------------------------------------------------------+
+ | 24 | Error Misc 1 | 8 | See ARM RAS specification for details |
+ +--------+---------------+-------------+------------------------------------------------------------+
+ | 32 | Error Misc 2 | 8 | See ARM RAS specification for details |
+ +--------+---------------+-------------+------------------------------------------------------------+
+ | 40 | Error Misc 3 | 8 | See ARM RAS specification for details |
+ +--------+---------------+-------------+------------------------------------------------------------+
+
+ The table below defines the value of error types, their subtype, subcomponent and instance:
+
+ .. _smpro-error-types:
+
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | Error Group | Error Type | Sub type | Sub component | Instance |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | CPM (core) | 0 | 0 | Snoop-Logic | CPM # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | CPM (core) | 0 | 2 | Armv8 Core 1 | CPM # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | MCU (mem) | 1 | 1 | ERR1 | MCU # \| SLOT << 11 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | MCU (mem) | 1 | 2 | ERR2 | MCU # \| SLOT << 11 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | MCU (mem) | 1 | 3 | ERR3 | MCU # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | MCU (mem) | 1 | 4 | ERR4 | MCU # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | MCU (mem) | 1 | 5 | ERR5 | MCU # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | MCU (mem) | 1 | 6 | ERR6 | MCU # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | MCU (mem) | 1 | 7 | Link Error | MCU # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | Mesh (other) | 2 | 0 | Cross Point | X \| (Y << 5) \| NS <<11 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | Mesh (other) | 2 | 1 | Home Node(IO) | X \| (Y << 5) \| NS <<11 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | Mesh (other) | 2 | 2 | Home Node(Mem) | X \| (Y << 5) \| NS <<11 \| device<<12 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | Mesh (other) | 2 | 4 | CCIX Node | X \| (Y << 5) \| NS <<11 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | 2P Link (other) | 3 | 0 | N/A | Altra 2P Link # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 0 | ERR0 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 1 | ERR1 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 2 | ERR2 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 3 | ERR3 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 4 | ERR4 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 5 | ERR5 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 6 | ERR6 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 7 | ERR7 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 8 | ERR8 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 9 | ERR9 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 10 | ERR10 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 11 | ERR11 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 12 | ERR12 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | GIC (other) | 5 | 13-21 | ERR13 | RC # + 1 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TCU | 100 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TBU0 | 0 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TBU1 | 1 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TBU2 | 2 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TBU3 | 3 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TBU4 | 4 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TBU5 | 5 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TBU6 | 6 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TBU7 | 7 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TBU8 | 8 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMMU (other) | 6 | TBU9 | 9 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | PCIe AER (pcie) | 7 | Root | 0 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | PCIe AER (pcie) | 7 | Device | 1 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | PCIe RC (pcie) | 8 | RCA HB | 0 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | PCIe RC (pcie) | 8 | RCB HB | 1 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | PCIe RC (pcie) | 8 | RASDP | 8 | RC # |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | OCM (other) | 9 | ERR0 | 0 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | OCM (other) | 9 | ERR1 | 1 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | OCM (other) | 9 | ERR2 | 2 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMpro (other) | 10 | ERR0 | 0 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMpro (other) | 10 | ERR1 | 1 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | SMpro (other) | 10 | MPA_ERR | 2 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | PMpro (other) | 11 | ERR0 | 0 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | PMpro (other) | 11 | ERR1 | 1 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+ | PMpro (other) | 11 | MPA_ERR | 2 | 0 |
+ +-----------------+------------+----------+----------------+----------------------------------------+
+
+ Example::
+
+ # cat error_other_ue
+ 880807001e004010401040101500000001004010401040100c0000000000000000000000000000000000000000000000
+
+ The detail of each sysfs entries is as below:
+
+ +-------------+---------------------------------------------------------+----------------------------------+
+ | Error | Sysfs entry | Description (when triggered) |
+ +-------------+---------------------------------------------------------+----------------------------------+
+ | Core's CE | /sys/bus/platform/devices/smpro-errmon.*/error_core_ce | Core has CE error |
+ +-------------+---------------------------------------------------------+----------------------------------+
+ | Core's UE | /sys/bus/platform/devices/smpro-errmon.*/error_core_ue | Core has UE error |
+ +-------------+---------------------------------------------------------+----------------------------------+
+ | Memory's CE | /sys/bus/platform/devices/smpro-errmon.*/error_mem_ce | Memory has CE error |
+ +-------------+---------------------------------------------------------+----------------------------------+
+ | Memory's UE | /sys/bus/platform/devices/smpro-errmon.*/error_mem_ue | Memory has UE error |
+ +-------------+---------------------------------------------------------+----------------------------------+
+ | PCIe's CE | /sys/bus/platform/devices/smpro-errmon.*/error_pcie_ce | any PCIe controller has CE error |
+ +-------------+---------------------------------------------------------+----------------------------------+
+ | PCIe's UE | /sys/bus/platform/devices/smpro-errmon.*/error_pcie_ue | any PCIe controller has UE error |
+ +-------------+---------------------------------------------------------+----------------------------------+
+ | Other's CE | /sys/bus/platform/devices/smpro-errmon.*/error_other_ce | any other CE error |
+ +-------------+---------------------------------------------------------+----------------------------------+
+ | Other's UE | /sys/bus/platform/devices/smpro-errmon.*/error_other_ue | any other UE error |
+ +-------------+---------------------------------------------------------+----------------------------------+
+
+ UE: Uncorrect-able Error
+ CE: Correct-able Error
+
+ For details, see section `3.3 Ampere (Vendor-Specific) Error Record Formats,
+ Altra Family RAS Supplement`.
+
+
+What: /sys/bus/platform/devices/smpro-errmon.*/overflow_[core|mem|pcie|other]_[ce|ue]
+KernelVersion: 6.1
+Contact: Quan Nguyen <quan@os.amperecomputing.com>
+Description:
+ (RO) Return the overflow status of each type HW error reported:
+
+ - 0 : No overflow
+ - 1 : There is an overflow and the oldest HW errors are dropped
+
+ The detail of each sysfs entries is as below:
+
+ +-------------+-----------------------------------------------------------+---------------------------------------+
+ | Overflow | Sysfs entry | Description |
+ +-------------+-----------------------------------------------------------+---------------------------------------+
+ | Core's CE | /sys/bus/platform/devices/smpro-errmon.*/overflow_core_ce | Core CE error overflow |
+ +-------------+-----------------------------------------------------------+---------------------------------------+
+ | Core's UE | /sys/bus/platform/devices/smpro-errmon.*/overflow_core_ue | Core UE error overflow |
+ +-------------+-----------------------------------------------------------+---------------------------------------+
+ | Memory's CE | /sys/bus/platform/devices/smpro-errmon.*/overflow_mem_ce | Memory CE error overflow |
+ +-------------+-----------------------------------------------------------+---------------------------------------+
+ | Memory's UE | /sys/bus/platform/devices/smpro-errmon.*/overflow_mem_ue | Memory UE error overflow |
+ +-------------+-----------------------------------------------------------+---------------------------------------+
+ | PCIe's CE | /sys/bus/platform/devices/smpro-errmon.*/overflow_pcie_ce | any PCIe controller CE error overflow |
+ +-------------+-----------------------------------------------------------+---------------------------------------+
+ | PCIe's UE | /sys/bus/platform/devices/smpro-errmon.*/overflow_pcie_ue | any PCIe controller UE error overflow |
+ +-------------+-----------------------------------------------------------+---------------------------------------+
+ | Other's CE | /sys/bus/platform/devices/smpro-errmon.*/overflow_other_ce| any other CE error overflow |
+ +-------------+-----------------------------------------------------------+---------------------------------------+
+ | Other's UE | /sys/bus/platform/devices/smpro-errmon.*/overflow_other_ue| other UE error overflow |
+ +-------------+-----------------------------------------------------------+---------------------------------------+
+
+ where:
+
+ - UE: Uncorrect-able Error
+ - CE: Correct-able Error
+
+What: /sys/bus/platform/devices/smpro-errmon.*/[error|warn]_[smpro|pmpro]
+KernelVersion: 6.1
+Contact: Quan Nguyen <quan@os.amperecomputing.com>
+Description:
+ (RO) Contains the internal firmware error/warning printed as hex format.
+
+ The detail of each sysfs entries is as below:
+
+ +---------------+------------------------------------------------------+--------------------------+
+ | Error | Sysfs entry | Description |
+ +---------------+------------------------------------------------------+--------------------------+
+ | SMpro error | /sys/bus/platform/devices/smpro-errmon.*/error_smpro | system has SMpro error |
+ +---------------+------------------------------------------------------+--------------------------+
+ | SMpro warning | /sys/bus/platform/devices/smpro-errmon.*/warn_smpro | system has SMpro warning |
+ +---------------+------------------------------------------------------+--------------------------+
+ | PMpro error | /sys/bus/platform/devices/smpro-errmon.*/error_pmpro | system has PMpro error |
+ +---------------+------------------------------------------------------+--------------------------+
+ | PMpro warning | /sys/bus/platform/devices/smpro-errmon.*/warn_pmpro | system has PMpro warning |
+ +---------------+------------------------------------------------------+--------------------------+
+
+ For details, see section `5.10 RAS Internal Error Register Definitions,
+ Altra Family Soc BMC Interface Specification`.
+
+What: /sys/bus/platform/devices/smpro-errmon.*/event_[vrd_warn_fault|vrd_hot|dimm_hot]
+KernelVersion: 6.1
+Contact: Quan Nguyen <quan@os.amperecomputing.com>
+Description:
+ (RO) Contains the detail information in case of VRD/DIMM warning/hot events
+ in hex format as below::
+
+ AAAA
+
+ where:
+
+ - ``AAAA``: The event detail information data
+
+ The detail of each sysfs entries is as below:
+
+ +---------------+---------------------------------------------------------------+---------------------+
+ | Event | Sysfs entry | Description |
+ +---------------+---------------------------------------------------------------+---------------------+
+ | VRD HOT | /sys/bus/platform/devices/smpro-errmon.*/event_vrd_hot | VRD Hot |
+ +---------------+---------------------------------------------------------------+---------------------+
+ | VR Warn/Fault | /sys/bus/platform/devices/smpro-errmon.*/event_vrd_warn_fault | VR Warning or Fault |
+ +---------------+---------------------------------------------------------------+---------------------+
+ | DIMM HOT | /sys/bus/platform/devices/smpro-errmon.*/event_dimm_hot | DIMM Hot |
+ +---------------+---------------------------------------------------------------+---------------------+
+
+ For more details, see section `5.7 GPI Status Registers,
+ Altra Family Soc BMC Interface Specification`.
+
+What: /sys/bus/platform/devices/smpro-misc.*/boot_progress
+KernelVersion: 6.1
+Contact: Quan Nguyen <quan@os.amperecomputing.com>
+Description:
+ (RO) Contains the boot stages information in hex as format below::
+
+ AABBCCCCCCCC
+
+ where:
+
+ - ``AA`` : The boot stages
+
+ - 00: SMpro firmware booting
+ - 01: PMpro firmware booting
+ - 02: ATF BL1 firmware booting
+ - 03: DDR initialization
+ - 04: DDR training report status
+ - 05: ATF BL2 firmware booting
+ - 06: ATF BL31 firmware booting
+ - 07: ATF BL32 firmware booting
+ - 08: UEFI firmware booting
+ - 09: OS booting
+
+ - ``BB`` : Boot status
+
+ - 00: Not started
+ - 01: Started
+ - 02: Completed without error
+ - 03: Failed.
+
+ - ``CCCCCCCC``: Boot status information defined for each boot stages
+
+ For details, see section `5.11 Boot Stage Register Definitions`
+ and section `6. Processor Boot Progress Codes, Altra Family Soc BMC
+ Interface Specification`.
+
+
+What: /sys/bus/platform/devices/smpro-misc*/soc_power_limit
+KernelVersion: 6.1
+Contact: Quan Nguyen <quan@os.amperecomputing.com>
+Description:
+ (RW) Contains the desired SoC power limit in Watt.
+ Writes to this sysfs set the desired SoC power limit (W).
+ Reads from this register return the current SoC power limit (W).
+ The value ranges:
+
+ - Minimum: 120 W
+ - Maximum: Socket TDP power
attached to the port will not be detected, initialized,
or enumerated.
+What: /sys/bus/usb/devices/.../<hub_interface>/port<X>/early_stop
+Date: Sep 2022
+Contact: Ray Chi <raychi@google.com>
+Description:
+ Some USB hosts have some watchdog mechanisms so that the device
+ may enter ramdump if it takes a long time during port initialization.
+ This attribute allows each port just has two attempts so that the
+ port initialization will be failed quickly. In addition, if a port
+ which is marked with early_stop has failed to initialize, it will ignore
+ all future connections until this attribute is clear.
+
What: /sys/bus/usb/devices/.../power/usb2_lpm_l1_timeout
Date: May 2013
Contact: Mathias Nyman <mathias.nyman@linux.intel.com>
--- /dev/null
+What: /sys/kernel/cpu_byteorder
+Date: February 2023
+KernelVersion: 6.2
+Contact: Thomas Weißschuh <linux@weissschuh.net>
+Description:
+ The endianness of the running kernel.
+
+ Access: Read
+
+ Valid values:
+ "little", "big"
+Users: util-linux
them frequently to increase the rate of SLB faults
on kernel addresses.
+ stress_hpt [PPC]
+ Limits the number of kernel HPT entries in the hash
+ page table to increase the rate of hash page table
+ faults on kernel addresses.
+
disable= [IPV6]
See Documentation/networking/ipv6.rst.
Provide an override to the IOAPIC-ID<->DEVICE-ID
mapping provided in the IVRS ACPI table.
By default, PCI segment is 0, and can be omitted.
- For example:
+
+ For example, to map IOAPIC-ID decimal 10 to
+ PCI segment 0x1 and PCI device 00:14.0,
+ write the parameter as:
+ ivrs_ioapic=10@0001:00:14.0
+
+ Deprecated formats:
* To map IOAPIC-ID decimal 10 to PCI device 00:14.0
write the parameter as:
ivrs_ioapic[10]=00:14.0
Provide an override to the HPET-ID<->DEVICE-ID
mapping provided in the IVRS ACPI table.
By default, PCI segment is 0, and can be omitted.
- For example:
+
+ For example, to map HPET-ID decimal 10 to
+ PCI segment 0x1 and PCI device 00:14.0,
+ write the parameter as:
+ ivrs_hpet=10@0001:00:14.0
+
+ Deprecated formats:
* To map HPET-ID decimal 0 to PCI device 00:14.0
write the parameter as:
ivrs_hpet[0]=00:14.0
ivrs_acpihid [HW,X86-64]
Provide an override to the ACPI-HID:UID<->DEVICE-ID
mapping provided in the IVRS ACPI table.
+ By default, PCI segment is 0, and can be omitted.
For example, to map UART-HID:UID AMD0020:0 to
PCI segment 0x1 and PCI device ID 00:14.5,
write the parameter as:
- ivrs_acpihid[0001:00:14.5]=AMD0020:0
+ ivrs_acpihid=AMD0020:0@0001:00:14.5
- By default, PCI segment is 0, and can be omitted.
- For example, PCI device 00:14.5 write the parameter as:
+ Deprecated formats:
+ * To map UART-HID:UID AMD0020:0 to PCI segment is 0,
+ PCI device ID 00:14.5, write the parameter as:
ivrs_acpihid[00:14.5]=AMD0020:0
+ * To map UART-HID:UID AMD0020:0 to PCI segment 0x1 and
+ PCI device ID 00:14.5, write the parameter as:
+ ivrs_acpihid[0001:00:14.5]=AMD0020:0
js= [HW,JOY] Analog joystick
See Documentation/input/joydev/joystick.rst.
See also Documentation/trace/ftrace.rst "trace options"
section.
+ trace_trigger=[trigger-list]
+ [FTRACE] Add a event trigger on specific events.
+ Set a trigger on top of a specific event, with an optional
+ filter.
+
+ The format is is "trace_trigger=<event>.<trigger>[ if <filter>],..."
+ Where more than one trigger may be specified that are comma deliminated.
+
+ For example:
+
+ trace_trigger="sched_switch.stacktrace if prev_state == 2"
+
+ The above will enable the "stacktrace" trigger on the "sched_switch"
+ event but only trigger it if the "prev_state" of the "sched_switch"
+ event is "2" (TASK_UNINTERUPTIBLE).
+
+ See also "Event triggers" in Documentation/trace/events.rst
+
+
traceoff_on_warning
[FTRACE] enable this option to disable tracing when a
warning is hit. This turns off "tracing_on". Tracing can
On architectures where unaligned accesses cause traps, and where this
feature is supported (``CONFIG_SYSCTL_ARCH_UNALIGN_NO_WARN``;
-currently, ``arc`` and ``ia64``), controls whether all unaligned traps
-are logged.
+currently, ``arc``, ``ia64`` and ``loongarch``), controls whether all
+unaligned traps are logged.
= =============================================================
0 Log all unaligned accesses.
On architectures where unaligned accesses cause traps, and where this
feature is supported (``CONFIG_SYSCTL_ARCH_UNALIGN_ALLOW``; currently,
-``arc`` and ``parisc``), controls whether unaligned traps are caught
-and emulated (instead of failing).
+``arc``, ``parisc`` and ``loongarch``), controls whether unaligned traps
+are caught and emulated (instead of failing).
= ========================================================
0 Do not emulate unaligned accesses.
+----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-A710 | #2224489 | ARM64_ERRATUM_2224489 |
+----------------+-----------------+-----------------+-----------------------------+
-| ARM | Cortex-A715 | #2645198 | ARM64_ERRATUM_2645198 |
-+----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-X2 | #2119858 | ARM64_ERRATUM_2119858 |
+----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-X2 | #2224489 | ARM64_ERRATUM_2224489 |
void *bpf_sk_storage_get(struct bpf_map *map, void *sk, void *value, u64 flags)
-Socket-local storage can be retrieved using the ``bpf_sk_storage_get()``
-helper. The helper gets the storage from ``sk`` that is associated with ``map``.
-If the ``BPF_LOCAL_STORAGE_GET_F_CREATE`` flag is used then
-``bpf_sk_storage_get()`` will create the storage for ``sk`` if it does not
-already exist. ``value`` can be used together with
-``BPF_LOCAL_STORAGE_GET_F_CREATE`` to initialize the storage value, otherwise it
-will be zero initialized. Returns a pointer to the storage on success, or
+Socket-local storage for ``map`` can be retrieved from socket ``sk`` using the
+``bpf_sk_storage_get()`` helper. If the ``BPF_LOCAL_STORAGE_GET_F_CREATE``
+flag is used then ``bpf_sk_storage_get()`` will create the storage for ``sk``
+if it does not already exist. ``value`` can be used together with
+``BPF_LOCAL_STORAGE_GET_F_CREATE`` to initialize the storage value, otherwise
+it will be zero initialized. Returns a pointer to the storage on success, or
``NULL`` in case of failure.
.. note::
long bpf_sk_storage_delete(struct bpf_map *map, void *sk)
-Socket-local storage can be deleted using the ``bpf_sk_storage_delete()``
-helper. The helper deletes the storage from ``sk`` that is identified by
-``map``. Returns ``0`` on success, or negative error in case of failure.
+Socket-local storage for ``map`` can be deleted from socket ``sk`` using the
+``bpf_sk_storage_delete()`` helper. Returns ``0`` on success, or negative
+error in case of failure.
User space
----------
int bpf_map_update_elem(int map_fd, const void *key, const void *value, __u64 flags)
-Socket-local storage for the socket identified by ``key`` belonging to
-``map_fd`` can be added or updated using the ``bpf_map_update_elem()`` libbpf
-function. ``key`` must be a pointer to a valid ``fd`` in the user space
-program. The ``flags`` parameter can be used to control the update behaviour:
+Socket-local storage for map ``map_fd`` can be added or updated locally to a
+socket using the ``bpf_map_update_elem()`` libbpf function. The socket is
+identified by a `socket` ``fd`` stored in the pointer ``key``. The pointer
+``value`` has the data to be added or updated to the socket ``fd``. The type
+and size of ``value`` should be the same as the value type of the map
+definition.
-- ``BPF_ANY`` will create storage for ``fd`` or update existing storage.
-- ``BPF_NOEXIST`` will create storage for ``fd`` only if it did not already
- exist, otherwise the call will fail with ``-EEXIST``.
-- ``BPF_EXIST`` will update existing storage for ``fd`` if it already exists,
- otherwise the call will fail with ``-ENOENT``.
+The ``flags`` parameter can be used to control the update behaviour:
+
+- ``BPF_ANY`` will create storage for `socket` ``fd`` or update existing storage.
+- ``BPF_NOEXIST`` will create storage for `socket` ``fd`` only if it did not
+ already exist, otherwise the call will fail with ``-EEXIST``.
+- ``BPF_EXIST`` will update existing storage for `socket` ``fd`` if it already
+ exists, otherwise the call will fail with ``-ENOENT``.
Returns ``0`` on success, or negative error in case of failure.
int bpf_map_lookup_elem(int map_fd, const void *key, void *value)
-Socket-local storage for the socket identified by ``key`` belonging to
-``map_fd`` can be retrieved using the ``bpf_map_lookup_elem()`` libbpf
-function. ``key`` must be a pointer to a valid ``fd`` in the user space
-program. Returns ``0`` on success, or negative error in case of failure.
+Socket-local storage for map ``map_fd`` can be retrieved from a socket using
+the ``bpf_map_lookup_elem()`` libbpf function. The storage is retrieved from
+the socket identified by a `socket` ``fd`` stored in the pointer
+``key``. Returns ``0`` on success, or negative error in case of failure.
bpf_map_delete_elem()
~~~~~~~~~~~~~~~~~~~~~
int bpf_map_delete_elem(int map_fd, const void *key)
-Socket-local storage for the socket identified by ``key`` belonging to
-``map_fd`` can be deleted using the ``bpf_map_delete_elem()`` libbpf
-function. Returns ``0`` on success, or negative error in case of failure.
+Socket-local storage for map ``map_fd`` can be deleted from a socket using the
+``bpf_map_delete_elem()`` libbpf function. The storage is deleted from the
+socket identified by a `socket` ``fd`` stored in the pointer ``key``. Returns
+``0`` on success, or negative error in case of failure.
Examples
========
$id: http://devicetree.org/schemas/arm/bcm/bcm2835.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom BCM2711/BCM2835 Platforms Device Tree Bindings
+title: Broadcom BCM2711/BCM2835 Platforms
maintainers:
- Eric Anholt <eric@anholt.net>
$id: http://devicetree.org/schemas/arm/bcm/brcm,bcm11351.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom BCM11351 device tree bindings
+title: Broadcom BCM11351
maintainers:
- Florian Fainelli <f.fainelli@gmail.com>
$id: http://devicetree.org/schemas/arm/bcm/brcm,bcm21664.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom BCM21664 device tree bindings
+title: Broadcom BCM21664
maintainers:
- Florian Fainelli <f.fainelli@gmail.com>
$id: http://devicetree.org/schemas/arm/bcm/brcm,bcm23550.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom BCM23550 device tree bindings
+title: Broadcom BCM23550
maintainers:
- Florian Fainelli <f.fainelli@gmail.com>
$id: http://devicetree.org/schemas/arm/bcm/brcm,bcm4708.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom BCM4708 device tree bindings
+title: Broadcom BCM4708
description:
Broadcom BCM4708/47081/4709/47094/53012 Wi-Fi/network SoCs based
$id: http://devicetree.org/schemas/arm/bcm/brcm,bcmbca.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom Broadband SoC device tree bindings
+title: Broadcom Broadband SoC
description:
Broadcom Broadband SoCs include family of high performance DSL/PON/Wireless
$id: http://devicetree.org/schemas/arm/bcm/brcm,cygnus.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom Cygnus device tree bindings
+title: Broadcom Cygnus
maintainers:
- Ray Jui <rjui@broadcom.com>
$id: http://devicetree.org/schemas/arm/bcm/brcm,hr2.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom Hurricane 2 device tree bindings
+title: Broadcom Hurricane 2
description:
Broadcom Hurricane 2 family of SoCs are used for switching control. These SoCs
$id: http://devicetree.org/schemas/arm/bcm/brcm,ns2.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom North Star 2 (NS2) device tree bindings
+title: Broadcom North Star 2 (NS2)
maintainers:
- Ray Jui <rjui@broadcom.com>
$id: http://devicetree.org/schemas/arm/bcm/brcm,nsp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom Northstar Plus device tree bindings
+title: Broadcom Northstar Plus
description:
Broadcom Northstar Plus family of SoCs are used for switching control
$id: http://devicetree.org/schemas/arm/bcm/brcm,stingray.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom Stingray device tree bindings
+title: Broadcom Stingray
maintainers:
- Ray Jui <rjui@broadcom.com>
$id: http://devicetree.org/schemas/arm/bcm/brcm,vulcan-soc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom Vulcan device tree bindings
+title: Broadcom Vulcan
maintainers:
- Robert Richter <rrichter@marvell.com>
$id: http://devicetree.org/schemas/arm/cci-control-port.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: CCI Interconnect Bus Masters binding
+title: CCI Interconnect Bus Masters
maintainers:
- Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
$id: http://devicetree.org/schemas/arm/cpus.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ARM CPUs bindings
+title: ARM CPUs
maintainers:
- Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
$id: http://devicetree.org/schemas/arm/firmware/linaro,optee-tz.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: OP-TEE Device Tree Bindings
+title: OP-TEE
maintainers:
- Jens Wiklander <jens.wiklander@linaro.org>
$id: http://devicetree.org/schemas/arm/hisilicon/hisilicon.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Hisilicon Platforms Device Tree Bindings
+title: Hisilicon Platforms
maintainers:
- Wei Xu <xuwei5@hisilicon.com>
$id: http://devicetree.org/schemas/arm/keystone/ti,k3-sci-common.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Common K3 TI-SCI bindings
+title: Common K3 TI-SCI
maintainers:
- Nishanth Menon <nm@ti.com>
$id: http://devicetree.org/schemas/arm/keystone/ti,sci.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: TI-SCI controller device node bindings
+title: TI-SCI controller
maintainers:
- Nishanth Menon <nm@ti.com>
$id: http://devicetree.org/schemas/arm/marvell/armada-7k-8k.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell Armada 7K/8K Platforms Device Tree Bindings
+title: Marvell Armada 7K/8K Platforms
maintainers:
- Gregory CLEMENT <gregory.clement@bootlin.com>
$id: http://devicetree.org/schemas/arm/mrvl/mrvl.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell Platforms Device Tree Bindings
+title: Marvell Platforms
maintainers:
- Lubomir Rintel <lkundrak@v3.sk>
$id: http://devicetree.org/schemas/arm/mstar/mstar.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MStar platforms device tree bindings
+title: MStar platforms
maintainers:
- Daniel Palmer <daniel@thingy.jp>
$id: http://devicetree.org/schemas/arm/npcm/npcm.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NPCM Platforms Device Tree Bindings
+title: NPCM Platforms
maintainers:
- Jonathan Neuschäfer <j.neuschaefer@gmx.net>
$id: http://devicetree.org/schemas/arm/nxp/lpc32xx.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP LPC32xx Platforms Device Tree Bindings
+title: NXP LPC32xx Platforms
maintainers:
- Roland Stigge <stigge@antcom.de>
$id: http://devicetree.org/schemas/arm/socionext/milbeaut.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Milbeaut platforms device tree bindings
+title: Milbeaut platforms
maintainers:
- Taichi Sugaya <sugaya.taichi@socionext.com>
$id: http://devicetree.org/schemas/arm/socionext/uniphier.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Socionext UniPhier platform device tree bindings
+title: Socionext UniPhier platform
maintainers:
- Masahiro Yamada <yamada.masahiro@socionext.com>
$id: http://devicetree.org/schemas/arm/sp810.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ARM Versatile Express SP810 System Controller bindings
+title: ARM Versatile Express SP810 System Controller
maintainers:
- Andre Przywara <andre.przywara@arm.com>
$id: http://devicetree.org/schemas/arm/sprd/sprd.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Unisoc platforms device tree bindings
+title: Unisoc platforms
maintainers:
- Orson Zhai <orsonzhai@gmail.com>
$id: "http://devicetree.org/schemas/arm/stm32/st,mlahb.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: STMicroelectronics STM32 ML-AHB interconnect bindings
+title: STMicroelectronics STM32 ML-AHB interconnect
maintainers:
- Fabien Dessenne <fabien.dessenne@foss.st.com>
$id: "http://devicetree.org/schemas/arm/stm32/st,stm32-syscon.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: STMicroelectronics STM32 Platforms System Controller bindings
+title: STMicroelectronics STM32 Platforms System Controller
maintainers:
- Alexandre Torgue <alexandre.torgue@foss.st.com>
$id: http://devicetree.org/schemas/arm/stm32/stm32.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Platforms Device Tree Bindings
+title: STMicroelectronics STM32 Platforms
maintainers:
- Alexandre Torgue <alexandre.torgue@foss.st.com>
$id: http://devicetree.org/schemas/arm/sunxi/allwinner,sun6i-a31-cpuconfig.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Allwinner CPU Configuration Controller Device Tree Bindings
+title: Allwinner CPU Configuration Controller
maintainers:
- Chen-Yu Tsai <wens@csie.org>
$id: http://devicetree.org/schemas/arm/sunxi/allwinner,sun9i-a80-prcm.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Allwinner A80 PRCM Device Tree Bindings
+title: Allwinner A80 PRCM
maintainers:
- Chen-Yu Tsai <wens@csie.org>
$id: "http://devicetree.org/schemas/arm/tegra/nvidia,tegra-ccplex-cluster.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: NVIDIA Tegra CPU COMPLEX CLUSTER area device tree bindings
+title: NVIDIA Tegra CPU COMPLEX CLUSTER area
maintainers:
- Sumit Gupta <sumitg@nvidia.com>
$id: "http://devicetree.org/schemas/arm/tegra/nvidia,tegra194-cbb.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: NVIDIA Tegra194 CBB 1.0 bindings
+title: NVIDIA Tegra194 CBB 1.0
maintainers:
- Sumit Gupta <sumitg@nvidia.com>
$id: "http://devicetree.org/schemas/arm/tegra/nvidia,tegra234-cbb.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: NVIDIA Tegra CBB 2.0 bindings
+title: NVIDIA Tegra CBB 2.0
maintainers:
- Sumit Gupta <sumitg@nvidia.com>
$id: http://devicetree.org/schemas/arm/ti/k3.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments K3 Multicore SoC architecture device tree bindings
+title: Texas Instruments K3 Multicore SoC architecture
maintainers:
- Nishanth Menon <nm@ti.com>
$id: http://devicetree.org/schemas/arm/ti/ti,davinci.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments DaVinci Platforms Device Tree Bindings
+title: Texas Instruments DaVinci Platforms
maintainers:
- Sekhar Nori <nsekhar@ti.com>
$id: http://devicetree.org/schemas/arm/vexpress-config.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ARM Versatile Express configuration bus bindings
+title: ARM Versatile Express configuration bus
maintainers:
- Andre Przywara <andre.przywara@arm.com>
$id: http://devicetree.org/schemas/arm/vexpress-sysreg.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ARM Versatile Express system registers bindings
+title: ARM Versatile Express system registers
maintainers:
- Andre Przywara <andre.przywara@arm.com>
$id: http://devicetree.org/schemas/ata/allwinner,sun4i-a10-ahci.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Allwinner A10 AHCI SATA Controller bindings
+title: Allwinner A10 AHCI SATA Controller
maintainers:
- Chen-Yu Tsai <wens@csie.org>
$id: http://devicetree.org/schemas/ata/allwinner,sun8i-r40-ahci.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Allwinner R40 AHCI SATA Controller bindings
+title: Allwinner R40 AHCI SATA Controller
maintainers:
- Chen-Yu Tsai <wens@csie.org>
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/bus/fsl,imx8qxp-pixel-link-msi-bus.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Freescale i.MX8qxp Pixel Link Medium Speed Interconnect (MSI) Bus
+
+maintainers:
+ - Liu Ying <victor.liu@nxp.com>
+
+description: |
+ i.MX8qxp pixel link MSI bus is used to control settings of PHYs, I/Os
+ sitting together with the PHYs. It is not the same as the MSI bus coming
+ from i.MX8 System Controller Unit (SCU) which is used to control power,
+ clock and reset through the i.MX8 Distributed Slave System Controller (DSC).
+
+ i.MX8qxp pixel link MSI bus is a simple memory-mapped bus. Two input clocks,
+ that is, MSI clock and AHB clock, need to be enabled so that peripherals
+ connected to the bus can be accessed. Also, the bus is part of a power
+ domain. The power domain needs to be enabled before the peripherals can
+ be accessed.
+
+ Peripherals in i.MX8qm/qxp imaging, LVDS, MIPI DSI and HDMI TX subsystems,
+ like I2C controller, PWM controller, MIPI DSI controller and Control and
+ Status Registers (CSR) module, are accessed through the bus.
+
+ The i.MX System Controller Firmware (SCFW) owns and uses the i.MX8qm/qxp
+ pixel link MSI bus controller and does not allow SCFW user to control it.
+ So, the controller's registers cannot be accessed by SCFW user. Hence,
+ the interrupts generated by the controller don't make any sense from SCFW
+ user's point of view.
+
+allOf:
+ - $ref: simple-pm-bus.yaml#
+
+# We need a select here so we don't match all nodes with 'simple-pm-bus'.
+select:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - fsl,imx8qxp-display-pixel-link-msi-bus
+ - fsl,imx8qm-display-pixel-link-msi-bus
+ required:
+ - compatible
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - fsl,imx8qxp-display-pixel-link-msi-bus
+ - fsl,imx8qm-display-pixel-link-msi-bus
+ - const: simple-pm-bus
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: master gated clock from system
+ - description: AHB clock
+
+ clock-names:
+ items:
+ - const: msi
+ - const: ahb
+
+patternProperties:
+ "^.*@[0-9a-f]+$":
+ description: Devices attached to the bus
+ type: object
+ properties:
+ reg:
+ maxItems: 1
+
+ required:
+ - reg
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - power-domains
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/imx8-lpcg.h>
+ #include <dt-bindings/firmware/imx/rsrc.h>
+ bus@56200000 {
+ compatible = "fsl,imx8qxp-display-pixel-link-msi-bus", "simple-pm-bus";
+ reg = <0x56200000 0x20000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ interrupt-parent = <&dc0_irqsteer>;
+ interrupts = <320>;
+ ranges;
+ clocks = <&dc0_disp_ctrl_link_mst0_lpcg IMX_LPCG_CLK_4>,
+ <&dc0_disp_ctrl_link_mst0_lpcg IMX_LPCG_CLK_4>;
+ clock-names = "msi", "ahb";
+ power-domains = <&pd IMX_SC_R_DC_0>;
+
+ syscon@56221000 {
+ compatible = "fsl,imx8qxp-mipi-lvds-csr", "syscon", "simple-mfd";
+ reg = <0x56221000 0x1000>;
+ clocks = <&mipi_lvds_0_di_mipi_lvds_regs_lpcg IMX_LPCG_CLK_4>;
+ clock-names = "ipg";
+
+ pxl2dpi {
+ compatible = "fsl,imx8qxp-pxl2dpi";
+ fsl,sc-resource = <IMX_SC_R_MIPI_0>;
+ power-domains = <&pd IMX_SC_R_MIPI_0>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0>;
+
+ mipi_lvds_0_pxl2dpi_dc0_pixel_link0: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&dc0_pixel_link0_mipi_lvds_0_pxl2dpi>;
+ };
+
+ mipi_lvds_0_pxl2dpi_dc0_pixel_link1: endpoint@1 {
+ reg = <1>;
+ remote-endpoint = <&dc0_pixel_link1_mipi_lvds_0_pxl2dpi>;
+ };
+ };
+
+ port@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <1>;
+
+ mipi_lvds_0_pxl2dpi_mipi_lvds_0_ldb_ch0: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&mipi_lvds_0_ldb_ch0_mipi_lvds_0_pxl2dpi>;
+ };
+
+ mipi_lvds_0_pxl2dpi_mipi_lvds_0_ldb_ch1: endpoint@1 {
+ reg = <1>;
+ remote-endpoint = <&mipi_lvds_0_ldb_ch1_mipi_lvds_0_pxl2dpi>;
+ };
+ };
+ };
+ };
+
+ ldb {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ compatible = "fsl,imx8qxp-ldb";
+ clocks = <&clk IMX_SC_R_LVDS_0 IMX_SC_PM_CLK_MISC2>,
+ <&clk IMX_SC_R_LVDS_0 IMX_SC_PM_CLK_BYPASS>;
+ clock-names = "pixel", "bypass";
+ power-domains = <&pd IMX_SC_R_LVDS_0>;
+
+ channel@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0>;
+ phys = <&mipi_lvds_0_phy>;
+ phy-names = "lvds_phy";
+
+ port@0 {
+ reg = <0>;
+
+ mipi_lvds_0_ldb_ch0_mipi_lvds_0_pxl2dpi: endpoint {
+ remote-endpoint = <&mipi_lvds_0_pxl2dpi_mipi_lvds_0_ldb_ch0>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ /* ... */
+ };
+ };
+
+ channel@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <1>;
+ phys = <&mipi_lvds_0_phy>;
+ phy-names = "lvds_phy";
+
+ port@0 {
+ reg = <0>;
+
+ mipi_lvds_0_ldb_ch1_mipi_lvds_0_pxl2dpi: endpoint {
+ remote-endpoint = <&mipi_lvds_0_pxl2dpi_mipi_lvds_0_ldb_ch1>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ /* ... */
+ };
+ };
+ };
+ };
+
+ clock-controller@56223004 {
+ compatible = "fsl,imx8qxp-lpcg";
+ reg = <0x56223004 0x4>;
+ #clock-cells = <1>;
+ clocks = <&mipi_lvds_0_ipg_clk>;
+ clock-indices = <IMX_LPCG_CLK_4>;
+ clock-output-names = "mipi_lvds_0_di_mipi_lvds_regs_lpcg_ipg_clk";
+ power-domains = <&pd IMX_SC_R_MIPI_0>;
+ };
+
+ phy@56228300 {
+ compatible = "fsl,imx8qxp-mipi-dphy";
+ reg = <0x56228300 0x100>;
+ clocks = <&clk IMX_SC_R_LVDS_0 IMX_SC_PM_CLK_PHY>;
+ clock-names = "phy_ref";
+ #phy-cells = <0>;
+ fsl,syscon = <&mipi_lvds_0_csr>;
+ power-domains = <&pd IMX_SC_R_MIPI_0>;
+ };
+ };
$id: http://devicetree.org/schemas/bus/ti-sysc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments interconnect target module binding
+title: Texas Instruments interconnect target module
maintainers:
- Tony Lindgren <tony@atomide.com>
$id: http://devicetree.org/schemas/clock/adi,axi-clkgen.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for Analog Devices AXI clkgen pcore clock generator
+title: Analog Devices AXI clkgen pcore clock generator
maintainers:
- Lars-Peter Clausen <lars@metafoo.de>
$id: http://devicetree.org/schemas/clock/calxeda.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Device Tree Clock bindings for Calxeda highbank platform
+title: Calxeda highbank platform Clock Controller
description: |
This binding covers the Calxeda SoC internal peripheral and bus clocks
$id: http://devicetree.org/schemas/clock/cirrus,cs2000-cp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding CIRRUS LOGIC Fractional-N Clock Synthesizer & Clock Multiplier
+title: CIRRUS LOGIC Fractional-N Clock Synthesizer & Clock Multiplier
maintainers:
- Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
$id: http://devicetree.org/schemas/clock/fixed-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for simple fixed-rate clock sources
+title: Simple fixed-rate clock sources
maintainers:
- Michael Turquette <mturquette@baylibre.com>
$id: http://devicetree.org/schemas/clock/fixed-factor-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for simple fixed factor rate clock sources
+title: Simple fixed factor rate clock sources
maintainers:
- Michael Turquette <mturquette@baylibre.com>
$id: http://devicetree.org/schemas/clock/fixed-mmio-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for simple memory mapped IO fixed-rate clock sources
+title: Simple memory mapped IO fixed-rate clock sources
description:
This binding describes a fixed-rate clock for which the frequency can
$id: http://devicetree.org/schemas/clock/fsl,plldig.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP QorIQ Layerscape LS1028A Display PIXEL Clock Binding
+title: NXP QorIQ Layerscape LS1028A Display PIXEL Clock
maintainers:
- Wen He <wen.he_1@nxp.com>
$id: http://devicetree.org/schemas/clock/fsl,sai-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Freescale SAI bitclock-as-a-clock binding
+title: Freescale SAI bitclock-as-a-clock
maintainers:
- Michael Walle <michael@walle.cc>
$id: http://devicetree.org/schemas/clock/fsl,scu-clk.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: i.MX SCU Client Device Node - Clock bindings based on SCU Message Protocol
+title: i.MX SCU Client Device Node - Clock Controller Based on SCU Message Protocol
maintainers:
- Abel Vesa <abel.vesa@nxp.com>
$id: http://devicetree.org/schemas/clock/idt,versaclock5.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for IDT VersaClock 5 and 6 programmable I2C clock generators
+title: IDT VersaClock 5 and 6 programmable I2C clock generators
description: |
The IDT VersaClock 5 and VersaClock 6 are programmable I2C
$id: http://devicetree.org/schemas/clock/imx1-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX1 CPUs
+title: Freescale i.MX1 CPUs Clock Controller
maintainers:
- Alexander Shiyan <shc_work@mail.ru>
$id: http://devicetree.org/schemas/clock/imx21-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX21
+title: Freescale i.MX21 Clock Controller
maintainers:
- Alexander Shiyan <shc_work@mail.ru>
$id: http://devicetree.org/schemas/clock/imx23-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX23
+title: Freescale i.MX23 Clock Controller
maintainers:
- Shawn Guo <shawnguo@kernel.org>
$id: http://devicetree.org/schemas/clock/imx25-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX25
+title: Freescale i.MX25 Clock Controller
maintainers:
- Sascha Hauer <s.hauer@pengutronix.de>
$id: http://devicetree.org/schemas/clock/imx27-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX27
+title: Freescale i.MX27 Clock Controller
maintainers:
- Fabio Estevam <festevam@gmail.com>
$id: http://devicetree.org/schemas/clock/imx28-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX28
+title: Freescale i.MX28 Clock Controller
maintainers:
- Shawn Guo <shawnguo@kernel.org>
$id: http://devicetree.org/schemas/clock/imx31-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX31
+title: Freescale i.MX31 Clock Controller
maintainers:
- Fabio Estevam <festevam@gmail.com>
$id: http://devicetree.org/schemas/clock/imx35-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX35
+title: Freescale i.MX35 Clock Controller
maintainers:
- Steffen Trumtrar <s.trumtrar@pengutronix.de>
$id: http://devicetree.org/schemas/clock/imx5-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX5
+title: Freescale i.MX5 Clock Controller
maintainers:
- Fabio Estevam <festevam@gmail.com>
$id: http://devicetree.org/schemas/clock/imx6q-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX6 Quad
+title: Freescale i.MX6 Quad Clock Controller
maintainers:
- Anson Huang <Anson.Huang@nxp.com>
$id: http://devicetree.org/schemas/clock/imx6sl-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX6 SoloLite
+title: Freescale i.MX6 SoloLite Clock Controller
maintainers:
- Anson Huang <Anson.Huang@nxp.com>
$id: http://devicetree.org/schemas/clock/imx6sll-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX6 SLL
+title: Freescale i.MX6 SLL Clock Controller
maintainers:
- Anson Huang <Anson.Huang@nxp.com>
$id: http://devicetree.org/schemas/clock/imx6sx-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX6 SoloX
+title: Freescale i.MX6 SoloX Clock Controller
maintainers:
- Anson Huang <Anson.Huang@nxp.com>
$id: http://devicetree.org/schemas/clock/imx6ul-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX6 UltraLite
+title: Freescale i.MX6 UltraLite Clock Controller
maintainers:
- Anson Huang <Anson.Huang@nxp.com>
$id: http://devicetree.org/schemas/clock/imx7d-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX7 Dual
+title: Freescale i.MX7 Dual Clock Controller
maintainers:
- Frank Li <Frank.Li@nxp.com>
$id: http://devicetree.org/schemas/clock/imx7ulp-pcc-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX7ULP Peripheral Clock Control (PCC) modules
+title: Freescale i.MX7ULP Peripheral Clock Control (PCC) modules Clock Controller
maintainers:
- A.s. Dong <aisheng.dong@nxp.com>
$id: http://devicetree.org/schemas/clock/imx7ulp-scg-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MX7ULP System Clock Generation (SCG) modules
+title: Freescale i.MX7ULP System Clock Generation (SCG) modules Clock Controller
maintainers:
- A.s. Dong <aisheng.dong@nxp.com>
$id: http://devicetree.org/schemas/clock/imx8m-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP i.MX8M Family Clock Control Module Binding
+title: NXP i.MX8M Family Clock Control Module
maintainers:
- Anson Huang <Anson.Huang@nxp.com>
$id: http://devicetree.org/schemas/clock/imx8qxp-lpcg.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP i.MX8QXP LPCG (Low-Power Clock Gating) Clock bindings
+title: NXP i.MX8QXP LPCG (Low-Power Clock Gating) Clock
maintainers:
- Aisheng Dong <aisheng.dong@nxp.com>
$id: http://devicetree.org/schemas/clock/imx8ulp-cgc-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP i.MX8ULP Clock Generation & Control(CGC) Module Binding
+title: NXP i.MX8ULP Clock Generation & Control(CGC) Module
maintainers:
- Jacky Bai <ping.bai@nxp.com>
$id: http://devicetree.org/schemas/clock/imx8ulp-pcc-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP i.MX8ULP Peripheral Clock Controller(PCC) Module Binding
+title: NXP i.MX8ULP Peripheral Clock Controller(PCC) Module
maintainers:
- Jacky Bai <ping.bai@nxp.com>
$id: http://devicetree.org/schemas/clock/imx93-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP i.MX93 Clock Control Module Binding
+title: NXP i.MX93 Clock Control Module
maintainers:
- Peng Fan <peng.fan@nxp.com>
$id: http://devicetree.org/schemas/clock/imxrt1050-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for Freescale i.MXRT
+title: Freescale i.MXRT Clock Controller
maintainers:
- Giulio Benetti <giulio.benetti@benettiengineering.com>
$id: http://devicetree.org/schemas/clock/ingenic,cgu.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs CGU devicetree bindings
+title: Ingenic SoCs CGU
description: |
The CGU in an Ingenic SoC provides all the clocks generated on-chip. It
$id: http://devicetree.org/schemas/clock/intel,agilex.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Intel SoCFPGA Agilex platform clock controller binding
+title: Intel SoCFPGA Agilex platform clock controller
maintainers:
- Dinh Nguyen <dinguyen@kernel.org>
$id: http://devicetree.org/schemas/clock/intel,cgu-lgm.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Intel Lightning Mountain SoC's Clock Controller(CGU) Binding
+title: Intel Lightning Mountain SoC's Clock Controller(CGU)
maintainers:
- Rahul Tanwar <rahul.tanwar@linux.intel.com>
$id: http://devicetree.org/schemas/clock/intel,easic-n5x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Intel SoCFPGA eASIC N5X platform clock controller binding
+title: Intel SoCFPGA eASIC N5X platform clock controller
maintainers:
- Dinh Nguyen <dinguyen@kernel.org>
$id: http://devicetree.org/schemas/clock/intel,stratix10.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Intel SoCFPGA Stratix10 platform clock controller binding
+title: Intel SoCFPGA Stratix10 platform clock controller
maintainers:
- Dinh Nguyen <dinguyen@kernel.org>
$id: http://devicetree.org/schemas/clock/microchip,mpfs-clkcfg.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Microchip PolarFire Clock Control Module Binding
+title: Microchip PolarFire Clock Control Module
maintainers:
- Daire McNamara <daire.mcnamara@microchip.com>
$id: http://devicetree.org/schemas/clock/milbeaut-clock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Milbeaut SoCs Clock Controller Binding
+title: Milbeaut SoCs Clock Controller
maintainers:
- Taichi Sugaya <sugaya.taichi@socionext.com>
$id: http://devicetree.org/schemas/clock/nuvoton,npcm845-clk.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Nuvoton NPCM8XX Clock Controller Binding
+title: Nuvoton NPCM8XX Clock Controller
maintainers:
- Tomer Maimon <tmaimon77@gmail.com>
$id: http://devicetree.org/schemas/clock/qcom,dispcc-sc8280xp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Display Clock & Reset Controller Binding for SC8280XP
+title: Qualcomm Display Clock & Reset Controller on SC8280XP
maintainers:
- Bjorn Andersson <bjorn.andersson@linaro.org>
$id: http://devicetree.org/schemas/clock/qcom,gcc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Global Clock & Reset Controller Common Bindings
+title: Qualcomm Global Clock & Reset Controller Common Properties
maintainers:
- Stephen Boyd <sboyd@kernel.org>
$id: http://devicetree.org/schemas/clock/qcom,rpmhcc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Technologies, Inc. RPMh Clocks Bindings
+title: Qualcomm Technologies, Inc. RPMh Clocks
maintainers:
- Taniya Das <tdas@codeaurora.org>
$id: http://devicetree.org/schemas/clock/renesas,9series.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for Renesas 9-series I2C PCIe clock generators
+title: Renesas 9-series I2C PCIe clock generators
description: |
The Renesas 9-series are I2C PCIe clock generators providing
$id: http://devicetree.org/schemas/clock/renesas,versaclock7.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Renesas Versaclock7 Programmable Clock Device Tree Bindings
+title: Renesas Versaclock7 Programmable Clock
maintainers:
- Alex Helms <alexander.helms.jy@renesas.com>
$id: http://devicetree.org/schemas/clock/rockchip,rk3568-cru.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ROCKCHIP rk3568 Family Clock Control Module Binding
+title: ROCKCHIP rk3568 Family Clock Control Module
maintainers:
- Elaine Zhang <zhangqing@rock-chips.com>
$id: http://devicetree.org/schemas/clock/st,stm32mp1-rcc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Reset Clock Controller Binding
+title: STMicroelectronics STM32MP1 Reset Clock Controller
maintainers:
- Gabriel Fernandez <gabriel.fernandez@foss.st.com>
$id: http://devicetree.org/schemas/clock/ti,lmk04832.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Clock bindings for the Texas Instruments LMK04832
+title: Texas Instruments LMK04832 Clock Controller
maintainers:
- Liam Beguin <liambeguin@gmail.com>
$id: http://devicetree.org/schemas/clock/ti,sci-clk.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: TI-SCI clock controller node bindings
+title: TI-SCI clock controller
maintainers:
- Nishanth Menon <nm@ti.com>
$id: http://devicetree.org/schemas/clock/ti/ti,clksel.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for TI clksel clock
+title: TI clksel clock
maintainers:
- Tony Lindgren <tony@atomide.com>
$id: http://devicetree.org/schemas/cpu/idle-states.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Idle states binding description
+title: Idle states
maintainers:
- Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
$id: http://devicetree.org/schemas/cpufreq/cpufreq-mediatek-hw.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MediaTek's CPUFREQ Bindings
+title: MediaTek's CPUFREQ
maintainers:
- Hector Yuan <hector.yuan@mediatek.com>
$id: http://devicetree.org/schemas/cpufreq/qcom-cpufreq-nvmem.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Technologies, Inc. NVMEM CPUFreq bindings
+title: Qualcomm Technologies, Inc. NVMEM CPUFreq
maintainers:
- Ilia Lin <ilia.lin@kernel.org>
$id: http://devicetree.org/schemas/crypto/st,stm32-crc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 CRC bindings
+title: STMicroelectronics STM32 CRC
maintainers:
- Lionel Debieve <lionel.debieve@foss.st.com>
$id: http://devicetree.org/schemas/crypto/st,stm32-cryp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 CRYP bindings
+title: STMicroelectronics STM32 CRYP
description: The STM32 CRYP block is built on the CRYP block found in
the STn8820 SoC introduced in 2007, and subsequently used in the U8500
$id: http://devicetree.org/schemas/crypto/st,stm32-hash.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 HASH bindings
+title: STMicroelectronics STM32 HASH
maintainers:
- Lionel Debieve <lionel.debieve@foss.st.com>
$id: http://devicetree.org/schemas/display/arm,hdlcd.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Arm HDLCD display controller binding
+title: Arm HDLCD display controller
maintainers:
- Liviu Dudau <Liviu.Dudau@arm.com>
$id: http://devicetree.org/schemas/display/arm,malidp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Arm Mali Display Processor (Mali-DP) binding
+title: Arm Mali Display Processor (Mali-DP)
maintainers:
- Liviu Dudau <Liviu.Dudau@arm.com>
$id: http://devicetree.org/schemas/display/bridge/anx6345.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Analogix ANX6345 eDP Transmitter Device Tree Bindings
+title: Analogix ANX6345 eDP Transmitter
maintainers:
- Torsten Duwe <duwe@lst.de>
$id: http://devicetree.org/schemas/display/bridge/chrontel,ch7033.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Chrontel CH7033 Video Encoder Device Tree Bindings
+title: Chrontel CH7033 Video Encoder
maintainers:
- Lubomir Rintel <lkundrak@v3.sk>
$id: http://devicetree.org/schemas/display/bridge/ingenic,jz4780-hdmi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for Ingenic JZ4780 HDMI Transmitter
+title: Ingenic JZ4780 HDMI Transmitter
maintainers:
- H. Nikolaus Schaller <hns@goldelico.com>
$id: http://devicetree.org/schemas/display/bridge/intel,keembay-dsi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Devicetree bindings for Intel Keem Bay mipi dsi controller
+title: Intel Keem Bay mipi dsi controller
maintainers:
- Anitha Chrisanthus <anitha.chrisanthus@intel.com>
$id: http://devicetree.org/schemas/display/bridge/ite,it6505.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ITE it6505 Device Tree Bindings
+title: ITE it6505
maintainers:
- Allen Chen <allen.chen@ite.com.tw>
$id: http://devicetree.org/schemas/display/bridge/ite,it66121.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ITE it66121 HDMI bridge Device Tree Bindings
+title: ITE it66121 HDMI bridge
maintainers:
- Phong LE <ple@baylibre.com>
$id: http://devicetree.org/schemas/display/bridge/ps8640.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MIPI DSI to eDP Video Format Converter Device Tree Bindings
+title: MIPI DSI to eDP Video Format Converter
maintainers:
- Nicolas Boichat <drinkcat@chromium.org>
$id: http://devicetree.org/schemas/display/bridge/toshiba,tc358767.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Toshiba TC358767 eDP bridge bindings
+title: Toshiba TC358767 eDP bridge
maintainers:
- Andrey Gusakov <andrey.gusakov@cogentembedded.com>
$id: http://devicetree.org/schemas/display/bridge/toshiba,tc358775.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Toshiba TC358775 DSI to LVDS bridge bindings
+title: Toshiba TC358775 DSI to LVDS bridge
maintainers:
- Vinay Simha BN <simhavcs@gmail.com>
maxItems: 3
required:
- clock-names
- - power-domains
- if:
not:
properties:
maxItems: 1
clock-names:
maxItems: 1
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - fsl,imx6sl-lcdif
+ - fsl,imx6sx-lcdif
+ - fsl,imx8mm-lcdif
+ - fsl,imx8mn-lcdif
+ - fsl,imx8mp-lcdif
+ then:
+ required:
+ - power-domains
examples:
- |
<&clks IMX6SX_CLK_LCDIF_APB>,
<&clks IMX6SX_CLK_DISPLAY_AXI>;
clock-names = "pix", "axi", "disp_axi";
+ power-domains = <&pd_disp>;
port {
endpoint {
$id: http://devicetree.org/schemas/display/ingenic,ipu.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs Image Processing Unit (IPU) devicetree bindings
+title: Ingenic SoCs Image Processing Unit (IPU)
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/display/ingenic,lcd.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs LCD controller devicetree bindings
+title: Ingenic SoCs LCD controller
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/display/intel,keembay-display.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Devicetree bindings for Intel Keem Bay display controller
+title: Intel Keem Bay display controller
maintainers:
- Anitha Chrisanthus <anitha.chrisanthus@intel.com>
$id: http://devicetree.org/schemas/display/intel,keembay-msscam.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Devicetree bindings for Intel Keem Bay MSSCAM
+title: Intel Keem Bay MSSCAM
maintainers:
- Anitha Chrisanthus <anitha.chrisanthus@intel.com>
$id: http://devicetree.org/schemas/display/mediatek/mediatek,cec.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Mediatek HDMI CEC Controller Device Tree Bindings
+title: Mediatek HDMI CEC Controller
maintainers:
- CK Hu <ck.hu@mediatek.com>
$id: http://devicetree.org/schemas/display/mediatek/mediatek,dsi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MediaTek DSI Controller Device Tree Bindings
+title: MediaTek DSI Controller
maintainers:
- Chun-Kuang Hu <chunkuang.hu@kernel.org>
$id: http://devicetree.org/schemas/display/mediatek/mediatek,hdmi-ddc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Mediatek HDMI DDC Device Tree Bindings
+title: Mediatek HDMI DDC
maintainers:
- CK Hu <ck.hu@mediatek.com>
$id: http://devicetree.org/schemas/display/mediatek/mediatek,hdmi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Mediatek HDMI Encoder Device Tree Bindings
+title: Mediatek HDMI Encoder
maintainers:
- CK Hu <ck.hu@mediatek.com>
$id: "http://devicetree.org/schemas/display/msm/gmu.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Devicetree bindings for the GMU attached to certain Adreno GPUs
+title: GMU attached to certain Adreno GPUs
maintainers:
- Rob Clark <robdclark@gmail.com>
$id: "http://devicetree.org/schemas/display/msm/gpu.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Devicetree bindings for the Adreno or Snapdragon GPUs
+title: Adreno or Snapdragon GPUs
maintainers:
- Rob Clark <robdclark@gmail.com>
$id: http://devicetree.org/schemas/display/panel/display-timings.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: display timings bindings
+title: display timings
maintainers:
- Thierry Reding <thierry.reding@gmail.com>
$id: http://devicetree.org/schemas/display/panel/ilitek,ili9163.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ilitek ILI9163 display panels device tree bindings
+title: Ilitek ILI9163 display panels
maintainers:
- Daniel Mack <daniel@zonque.org>
$id: http://devicetree.org/schemas/display/panel/olimex,lcd-olinuxino.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for Olimex Ltd. LCD-OLinuXino bridge panel.
+title: Olimex Ltd. LCD-OLinuXino bridge panel.
maintainers:
- Stefan Mavrodiev <stefan@olimex.com>
$id: http://devicetree.org/schemas/display/panel/panel-lvds.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Generic LVDS Display Panel Device Tree Bindings
+title: Generic LVDS Display Panel
maintainers:
- Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
$id: http://devicetree.org/schemas/display/panel/panel-timing.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: panel timing bindings
+title: panel timing
maintainers:
- Thierry Reding <thierry.reding@gmail.com>
$id: http://devicetree.org/schemas/display/panel/visionox,rm69299.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Visionox model RM69299 Panels Device Tree Bindings.
+title: Visionox model RM69299 Panels
maintainers:
- Harigovindan P <harigovi@codeaurora.org>
$id: http://devicetree.org/schemas/dma/dma-common.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: DMA Engine Generic Binding
+title: DMA Engine Common Properties
maintainers:
- Vinod Koul <vkoul@kernel.org>
$id: http://devicetree.org/schemas/dma/dma-controller.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: DMA Controller Generic Binding
+title: DMA Controller Common Properties
maintainers:
- Vinod Koul <vkoul@kernel.org>
$id: http://devicetree.org/schemas/dma/dma-router.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: DMA Router Generic Binding
+title: DMA Router Common Properties
maintainers:
- Vinod Koul <vkoul@kernel.org>
$id: http://devicetree.org/schemas/dma/ingenic,dma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs DMA Controller DT bindings
+title: Ingenic SoCs DMA Controller
maintainers:
- Paul Cercueil <paul@crapouillou.net>
- enum:
- ingenic,jz4740-dma
- ingenic,jz4725b-dma
+ - ingenic,jz4755-dma
- ingenic,jz4760-dma
- ingenic,jz4760-bdma
- ingenic,jz4760-mdma
Should contain all of the per-channel DMA interrupts in
ascending order with respect to the DMA channel index.
minItems: 1
- maxItems: 31
+ maxItems: 32
resets:
maxItems: 1
dma-coherent: true
+ dma-channel-mask:
+ maxItems: 1
+
required:
- compatible
- reg
- reset-names
- "#dma-cells"
- iommus
+ - dma-channel-mask
additionalProperties: false
#dma-cells = <1>;
iommus = <&smmu TEGRA186_SID_GPCDMA_0>;
dma-coherent;
+ dma-channel-mask = <0xfffffffe>;
};
...
properties:
compatible:
- enum:
- - qcom,sc7280-gpi-dma
- - qcom,sdm845-gpi-dma
- - qcom,sm6350-gpi-dma
- - qcom,sm8150-gpi-dma
- - qcom,sm8250-gpi-dma
- - qcom,sm8350-gpi-dma
- - qcom,sm8450-gpi-dma
+ oneOf:
+ - enum:
+ - qcom,sdm845-gpi-dma
+ - qcom,sm6350-gpi-dma
+ - items:
+ - enum:
+ - qcom,sc7280-gpi-dma
+ - qcom,sm6115-gpi-dma
+ - qcom,sm6375-gpi-dma
+ - qcom,sm8350-gpi-dma
+ - qcom,sm8450-gpi-dma
+ - const: qcom,sm6350-gpi-dma
+ - items:
+ - enum:
+ - qcom,sdm670-gpi-dma
+ - qcom,sm8150-gpi-dma
+ - qcom,sm8250-gpi-dma
+ - const: qcom,sdm845-gpi-dma
reg:
maxItems: 1
$id: http://devicetree.org/schemas/dma/st,stm32-dma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 DMA Controller bindings
+title: STMicroelectronics STM32 DMA Controller
description: |
The STM32 DMA is a general-purpose direct memory access controller capable of
$id: http://devicetree.org/schemas/dma/st,stm32-dmamux.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 DMA MUX (DMA request router) bindings
+title: STMicroelectronics STM32 DMA MUX (DMA request router)
maintainers:
- Amelie Delaunay <amelie.delaunay@foss.st.com>
$id: http://devicetree.org/schemas/dma/st,stm32-mdma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 MDMA Controller bindings
+title: STMicroelectronics STM32 MDMA Controller
description: |
The STM32 MDMA is a general-purpose direct memory access controller capable of
$id: http://devicetree.org/schemas/dma/ti/k3-bcdma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments K3 DMSS BCDMA Device Tree Bindings
+title: Texas Instruments K3 DMSS BCDMA
maintainers:
- Peter Ujfalusi <peter.ujfalusi@gmail.com>
$id: http://devicetree.org/schemas/dma/ti/k3-pktdma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments K3 DMSS PKTDMA Device Tree Bindings
+title: Texas Instruments K3 DMSS PKTDMA
maintainers:
- Peter Ujfalusi <peter.ujfalusi@gmail.com>
$id: http://devicetree.org/schemas/dma/ti/k3-udma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments K3 NAVSS Unified DMA Device Tree Bindings
+title: Texas Instruments K3 NAVSS Unified DMA
maintainers:
- Peter Ujfalusi <peter.ujfalusi@gmail.com>
$id: http://devicetree.org/schemas/dma/xilinx/xlnx,zynqmp-dpdma.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Xilinx ZynqMP DisplayPort DMA Controller Device Tree Bindings
+title: Xilinx ZynqMP DisplayPort DMA Controller
description: |
These bindings describe the DMA engine included in the Xilinx ZynqMP
$id: http://devicetree.org/schemas/edac/dmc-520.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ARM DMC-520 EDAC bindings
+title: ARM DMC-520 EDAC
maintainers:
- Lei Wang <lewan@microsoft.com>
maintainers:
- Bartosz Golaszewski <bgolaszewski@baylibre.com>
+allOf:
+ - $ref: /schemas/nvmem/nvmem.yaml
+
select:
properties:
compatible:
- compatible
- reg
-additionalProperties: false
+unevaluatedProperties: false
examples:
- |
allOf:
- $ref: /schemas/spi/spi-peripheral-props.yaml#
+ - $ref: /schemas/nvmem/nvmem.yaml
- if:
properties:
compatible:
$id: http://devicetree.org/schemas/eeprom/microchip,93lc46b.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Microchip 93xx46 SPI compatible EEPROM family dt bindings
+title: Microchip 93xx46 SPI compatible EEPROM family
maintainers:
- Cory Tusar <cory.tusar@pid1solutions.com>
allOf:
- $ref: /schemas/spi/spi-peripheral-props.yaml#
+ - $ref: /schemas/nvmem/nvmem.yaml
unevaluatedProperties: false
# $schema is the meta-schema this schema should be validated with.
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: An example schema annotated with jsonschema details
+title: An Example Device
maintainers:
- Rob Herring <robh@kernel.org>
$id: http://devicetree.org/schemas/firmware/arm,scmi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: System Control and Management Interface (SCMI) Message Protocol bindings
+title: System Control and Management Interface (SCMI) Message Protocol
maintainers:
- Sudeep Holla <sudeep.holla@arm.com>
$id: http://devicetree.org/schemas/firmware/arm,scpi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: System Control and Power Interface (SCPI) Message Protocol bindings
+title: System Control and Power Interface (SCPI) Message Protocol
maintainers:
- Sudeep Holla <sudeep.holla@arm.com>
$id: http://devicetree.org/schemas/firmware/qemu,fw-cfg-mmio.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: QEMU Firmware Configuration bindings
+title: QEMU Firmware Configuration
maintainers:
- Rob Herring <robh@kernel.org>
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/fpga/lattice,sysconfig.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Lattice Slave SPI sysCONFIG FPGA manager
+
+maintainers:
+ - Ivan Bornyakov <i.bornyakov@metrotek.ru>
+
+description: |
+ Lattice sysCONFIG port, which is used for FPGA configuration, among others,
+ have Slave Serial Peripheral Interface. Only full reconfiguration is
+ supported.
+
+ Programming of ECP5 is done by writing uncompressed bitstream image in .bit
+ format into FPGA's SRAM configuration memory.
+
+properties:
+ compatible:
+ enum:
+ - lattice,sysconfig-ecp5
+
+ reg:
+ maxItems: 1
+
+ program-gpios:
+ description:
+ A GPIO line connected to PROGRAMN (active low) pin of the device.
+ Initiates configuration sequence.
+ maxItems: 1
+
+ init-gpios:
+ description:
+ A GPIO line connected to INITN (active low) pin of the device.
+ Indicates that the FPGA is ready to be configured.
+ maxItems: 1
+
+ done-gpios:
+ description:
+ A GPIO line connected to DONE (active high) pin of the device.
+ Indicates that the configuration sequence is complete.
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: lattice,sysconfig-ecp5
+ then:
+ properties:
+ spi-max-frequency:
+ maximum: 60000000
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+
+ spi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ fpga-mgr@0 {
+ compatible = "lattice,sysconfig-ecp5";
+ reg = <0>;
+ spi-max-frequency = <20000000>;
+ program-gpios = <&gpio3 4 GPIO_ACTIVE_LOW>;
+ init-gpios = <&gpio3 3 GPIO_ACTIVE_LOW>;
+ done-gpios = <&gpio3 2 GPIO_ACTIVE_HIGH>;
+ };
+ };
$id: http://devicetree.org/schemas/gpio/gpio-tpic2810.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: TPIC2810 GPIO controller bindings
+title: TPIC2810 GPIO controller
maintainers:
- Aswath Govindraju <a-govindraju@ti.com>
$id: http://devicetree.org/schemas/gpio/ti,omap-gpio.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: OMAP GPIO controller bindings
+title: OMAP GPIO controller
maintainers:
- Grygorii Strashko <grygorii.strashko@ti.com>
$id: http://devicetree.org/schemas/gpu/brcm,bcm-v3d.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom V3D GPU Bindings
+title: Broadcom V3D GPU
maintainers:
- Eric Anholt <eric@anholt.net>
$id: "http://devicetree.org/schemas/gpu/host1x/nvidia,tegra210-nvdec.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Device tree binding for NVIDIA Tegra NVDEC
+title: NVIDIA Tegra NVDEC
description: |
NVDEC is the hardware video decoder present on NVIDIA Tegra210
$id: "http://devicetree.org/schemas/gpu/host1x/nvidia,tegra210-nvenc.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Device tree binding for NVIDIA Tegra NVENC
+title: NVIDIA Tegra NVENC
description: |
NVENC is the hardware video encoder present on NVIDIA Tegra210
$id: "http://devicetree.org/schemas/gpu/host1x/nvidia,tegra210-nvjpg.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Device tree binding for NVIDIA Tegra NVJPG
+title: NVIDIA Tegra NVJPG
description: |
NVJPG is the hardware JPEG decoder and encoder present on NVIDIA Tegra210
$id: "http://devicetree.org/schemas/gpu/host1x/nvidia,tegra234-nvdec.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Device tree binding for NVIDIA Tegra234 NVDEC
+title: NVIDIA Tegra234 NVDEC
description: |
NVDEC is the hardware video decoder present on NVIDIA Tegra210
$id: http://devicetree.org/schemas/gpu/vivante,gc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Vivante GPU Bindings
+title: Vivante GPU
description: Vivante GPU core devices
$id: http://devicetree.org/schemas/hwlock/st,stm32-hwspinlock.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Hardware Spinlock bindings
+title: STMicroelectronics STM32 Hardware Spinlock
maintainers:
- Fabien Dessenne <fabien.dessenne@foss.st.com>
$id: http://devicetree.org/schemas/hwmon/moortec,mr75203.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Moortec Semiconductor MR75203 PVT Controller bindings
+title: Moortec Semiconductor MR75203 PVT Controller
maintainers:
- Rahul Tanwar <rtanwar@maxlinear.com>
title: NTC thermistor temperature sensors
maintainers:
- - Naveen Krishna Chatradhi <ch.naveen@samsung.com>
- Linus Walleij <linus.walleij@linaro.org>
description: |
$id: http://devicetree.org/schemas/i2c/i2c-gpio.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for GPIO bitbanged I2C
+title: GPIO bitbanged I2C
maintainers:
- Wolfram Sang <wsa@kernel.org>
$id: http://devicetree.org/schemas/i2c/i2c-pxa.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell MMP I2C controller bindings
+title: Marvell MMP I2C controller
maintainers:
- Rob Herring <robh+dt@kernel.org>
$id: http://devicetree.org/schemas/i2c/ingenic,i2c.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs I2C controller devicetree bindings
+title: Ingenic SoCs I2C controller
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/i2c/st,nomadik-i2c.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ST Microelectronics Nomadik I2C Bindings
+title: ST Microelectronics Nomadik I2C
description: The Nomadik I2C host controller began its life in the ST
Microelectronics STn8800 SoC, and was then inherited into STn8810 and
$id: http://devicetree.org/schemas/i2c/ti,omap4-i2c.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for I2C controllers on TI's OMAP and K3 SoCs
+title: I2C controllers on TI's OMAP and K3 SoCs
maintainers:
- Vignesh Raghavendra <vigneshr@ti.com>
$id: http://devicetree.org/schemas/i3c/i3c.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: I3C bus binding
+title: I3C bus
maintainers:
- Alexandre Belloni <alexandre.belloni@bootlin.com>
$id: http://devicetree.org/schemas/iio/accel/adi,adxl355.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Analog Devices ADXL355 3-Axis, Low noise MEMS Accelerometer
+title: Analog Devices ADXL355 and ADXL359 3-Axis, Low noise MEMS Accelerometers
maintainers:
- Puranjay Mohan <puranjay12@gmail.com>
description: |
- Analog Devices ADXL355 3-Axis, Low noise MEMS Accelerometer that supports
- both I2C & SPI interfaces
+ Analog Devices ADXL355 and ADXL359 3-Axis, Low noise MEMS Accelerometers that
+ support both I2C & SPI interfaces
https://www.analog.com/en/products/adxl355.html
+ https://www.analog.com/en/products/adxl359.html
properties:
compatible:
enum:
- adi,adxl355
+ - adi,adxl359
reg:
maxItems: 1
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/iio/accel/kionix,kx022a.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: ROHM/Kionix KX022A Accelerometer
+
+maintainers:
+ - Matti Vaittinen <mazziesaccount@gmail.com>
+
+description: |
+ KX022A is a 3-axis accelerometer supporting +/- 2G, 4G, 8G and 16G ranges,
+ output data-rates from 0.78Hz to 1600Hz and a hardware-fifo buffering.
+ KX022A can be accessed either via I2C or SPI.
+
+properties:
+ compatible:
+ const: kionix,kx022a
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ minItems: 1
+ maxItems: 2
+
+ interrupt-names:
+ minItems: 1
+ items:
+ - enum: [INT1, INT2]
+ - const: INT2
+
+ vdd-supply: true
+ io-vdd-supply: true
+
+ mount-matrix:
+ description: |
+ an optional 3x3 mounting rotation matrix.
+
+required:
+ - compatible
+ - reg
+ - interrupts
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/irq.h>
+ i2c {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ accel@1f {
+ compatible = "kionix,kx022a";
+ reg = <0x1f>;
+
+ interrupt-parent = <&gpio1>;
+ interrupts = <29 IRQ_TYPE_LEVEL_LOW>;
+ interrupt-names = "INT1";
+
+ io-vdd-supply = <&iovdd>;
+ vdd-supply = <&vdd>;
+ };
+ };
$id: http://devicetree.org/schemas/iio/adc/adc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Generic IIO bindings for ADC channels
+title: IIO Common Properties for ADC Channels
maintainers:
- Jonathan Cameron <jic23@kernel.org>
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+# Copyright 2022 Analog Devices Inc.
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/iio/adc/adi,ad4130.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Analog Devices AD4130 ADC device driver
+
+maintainers:
+ - Cosmin Tanislav <cosmin.tanislav@analog.com>
+
+description: |
+ Bindings for the Analog Devices AD4130 ADC. Datasheet can be found here:
+ https://www.analog.com/media/en/technical-documentation/data-sheets/AD4130-8.pdf
+
+properties:
+ compatible:
+ enum:
+ - adi,ad4130
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+ description: phandle to the master clock (mclk)
+
+ clock-names:
+ items:
+ - const: mclk
+
+ interrupts:
+ maxItems: 1
+
+ interrupt-names:
+ description: |
+ Specify which interrupt pin should be configured as Data Ready / FIFO
+ interrupt.
+ Default if not supplied is int.
+ enum:
+ - int
+ - clk
+ - p2
+ - dout
+
+ '#address-cells':
+ const: 1
+
+ '#size-cells':
+ const: 0
+
+ '#clock-cells':
+ const: 0
+
+ clock-output-names:
+ maxItems: 1
+
+ refin1-supply:
+ description: refin1 supply. Can be used as reference for conversion.
+
+ refin2-supply:
+ description: refin2 supply. Can be used as reference for conversion.
+
+ avdd-supply:
+ description: AVDD voltage supply. Can be used as reference for conversion.
+
+ iovdd-supply:
+ description: IOVDD voltage supply. Used for the chip interface.
+
+ spi-max-frequency:
+ maximum: 5000000
+
+ adi,ext-clk-freq-hz:
+ description: Specify the frequency of the external clock.
+ enum: [76800, 153600]
+ default: 76800
+
+ adi,bipolar:
+ description: Specify if the device should be used in bipolar mode.
+ type: boolean
+
+ adi,vbias-pins:
+ description: Analog inputs to apply a voltage bias of (AVDD − AVSS) / 2 to.
+ $ref: /schemas/types.yaml#/definitions/uint32-array
+ minItems: 1
+ maxItems: 16
+ items:
+ minimum: 0
+ maximum: 15
+
+required:
+ - compatible
+ - reg
+ - interrupts
+
+patternProperties:
+ "^channel@([0-9a-f])$":
+ type: object
+ $ref: adc.yaml
+ unevaluatedProperties: false
+
+ properties:
+ reg:
+ description: The channel number.
+ minimum: 0
+ maximum: 15
+
+ diff-channels:
+ description: |
+ Besides the analog inputs available, internal inputs can be used.
+ 16: Internal temperature sensor.
+ 17: AVSS
+ 18: Internal reference
+ 19: DGND
+ 20: (AVDD − AVSS)/6+
+ 21: (AVDD − AVSS)/6-
+ 22: (IOVDD − DGND)/6+
+ 23: (IOVDD − DGND)/6-
+ 24: (ALDO − AVSS)/6+
+ 25: (ALDO − AVSS)/6-
+ 26: (DLDO − DGND)/6+
+ 27: (DLDO − DGND)/6-
+ 28: V_MV_P
+ 29: V_MV_M
+ items:
+ minimum: 0
+ maximum: 29
+
+ adi,reference-select:
+ description: |
+ Select the reference source to use when converting on the
+ specific channel. Valid values are:
+ 0: REFIN1(+)/REFIN1(−)
+ 1: REFIN2(+)/REFIN2(−)
+ 2: REFOUT/AVSS (Internal reference)
+ 3: AVDD/AVSS
+ If not specified, REFIN1 is used.
+ $ref: /schemas/types.yaml#/definitions/uint32
+ enum: [0, 1, 2, 3]
+ default: 0
+
+ adi,excitation-pin-0:
+ description: |
+ Analog input to apply excitation current to while the channel
+ is active.
+ $ref: /schemas/types.yaml#/definitions/uint32
+ minimum: 0
+ maximum: 15
+ default: 0
+
+ adi,excitation-pin-1:
+ description: |
+ Analog input to apply excitation current to while this channel
+ is active.
+ $ref: /schemas/types.yaml#/definitions/uint32
+ minimum: 0
+ maximum: 15
+ default: 0
+
+ adi,excitation-current-0-nanoamp:
+ description: |
+ Excitation current in nanoamps to be applied to pin specified in
+ adi,excitation-pin-0 while this channel is active.
+ enum: [0, 100, 10000, 20000, 50000, 100000, 150000, 200000]
+ default: 0
+
+ adi,excitation-current-1-nanoamp:
+ description: |
+ Excitation current in nanoamps to be applied to pin specified in
+ adi,excitation-pin-1 while this channel is active.
+ enum: [0, 100, 10000, 20000, 50000, 100000, 150000, 200000]
+ default: 0
+
+ adi,burnout-current-nanoamp:
+ description: |
+ Burnout current in nanoamps to be applied for this channel.
+ enum: [0, 500, 2000, 4000]
+ default: 0
+
+ adi,buffered-positive:
+ description: Enable buffered mode for positive input.
+ type: boolean
+
+ adi,buffered-negative:
+ description: Enable buffered mode for negative input.
+ type: boolean
+
+ required:
+ - reg
+ - diff-channels
+
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/irq.h>
+
+ spi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ adc@0 {
+ compatible = "adi,ad4130";
+ reg = <0>;
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ spi-max-frequency = <5000000>;
+ interrupts = <27 IRQ_TYPE_EDGE_FALLING>;
+ interrupt-parent = <&gpio>;
+
+ channel@0 {
+ reg = <0>;
+
+ adi,reference-select = <2>;
+
+ /* AIN8, AIN9 */
+ diff-channels = <8 9>;
+ };
+
+ channel@1 {
+ reg = <1>;
+
+ adi,reference-select = <2>;
+
+ /* AIN10, AIN11 */
+ diff-channels = <10 11>;
+ };
+
+ channel@2 {
+ reg = <2>;
+
+ adi,reference-select = <2>;
+
+ /* Temperature Sensor, DGND */
+ diff-channels = <16 19>;
+ };
+
+ channel@3 {
+ reg = <3>;
+
+ adi,reference-select = <2>;
+
+ /* Internal reference, DGND */
+ diff-channels = <18 19>;
+ };
+
+ channel@4 {
+ reg = <4>;
+
+ adi,reference-select = <2>;
+
+ /* DGND, DGND */
+ diff-channels = <19 19>;
+ };
+ };
+ };
description: |
Analog Devices AD7904, AD7914, AD7923, AD7924 4 Channel ADCs, and AD7908,
- AD7918, AD7928 8 Channels ADCs.
+ AD7918, AD7927, AD7928 8 Channels ADCs.
Specifications about the part can be found at:
https://www.analog.com/media/en/technical-documentation/data-sheets/AD7923.pdf
properties:
compatible:
- enum:
- - adi,ad7904
- - adi,ad7914
- - adi,ad7923
- - adi,ad7924
- - adi,ad7908
- - adi,ad7918
- - adi,ad7928
+ oneOf:
+ - enum:
+ - adi,ad7904
+ - adi,ad7908
+ - adi,ad7914
+ - adi,ad7918
+ - adi,ad7923
+ - adi,ad7928
+ - const: adi,ad7924
+ deprecated: true
+ - items:
+ - const: adi,ad7924
+ - const: adi,ad7923
+ - items:
+ - const: adi,ad7927
+ - const: adi,ad7928
reg:
maxItems: 1
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+# Copyright 2022 Analog Devices Inc.
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/iio/adc/adi,max11410.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Analog Devices MAX11410 ADC device driver
+
+maintainers:
+ - Ibrahim Tilki <Ibrahim.Tilki@analog.com>
+
+description: |
+ Bindings for the Analog Devices MAX11410 ADC device. Datasheet can be
+ found here:
+ https://datasheets.maximintegrated.com/en/ds/MAX11410.pdf
+
+properties:
+ compatible:
+ enum:
+ - adi,max11410
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ minItems: 1
+ maxItems: 2
+
+ interrupt-names:
+ description: Name of the gpio pin of max11410 used for IRQ
+ minItems: 1
+ items:
+ - enum: [gpio0, gpio1]
+ - const: gpio1
+
+ '#address-cells':
+ const: 1
+
+ '#size-cells':
+ const: 0
+
+ avdd-supply:
+ description: Optional avdd supply. Used as reference when no explicit reference supplied.
+
+ vref0p-supply:
+ description: vref0p supply can be used as reference for conversion.
+
+ vref1p-supply:
+ description: vref1p supply can be used as reference for conversion.
+
+ vref2p-supply:
+ description: vref2p supply can be used as reference for conversion.
+
+ vref0n-supply:
+ description: vref0n supply can be used as reference for conversion.
+
+ vref1n-supply:
+ description: vref1n supply can be used as reference for conversion.
+
+ vref2n-supply:
+ description: vref2n supply can be used as reference for conversion.
+
+ spi-max-frequency:
+ maximum: 8000000
+
+patternProperties:
+ "^channel(@[0-9])?$":
+ $ref: adc.yaml
+ type: object
+ description: Represents the external channels which are connected to the ADC.
+
+ properties:
+ reg:
+ description: The channel number in single-ended mode.
+ minimum: 0
+ maximum: 9
+
+ adi,reference:
+ description: |
+ Select the reference source to use when converting on
+ the specific channel. Valid values are:
+ 0: VREF0P/VREF0N
+ 1: VREF1P/VREF1N
+ 2: VREF2P/VREF2N
+ 3: AVDD/AGND
+ 4: VREF0P/AGND
+ 5: VREF1P/AGND
+ 6: VREF2P/AGND
+ If this field is left empty, AVDD/AGND is selected.
+ $ref: /schemas/types.yaml#/definitions/uint32
+ enum: [0, 1, 2, 3, 4, 5, 6]
+ default: 3
+
+ adi,input-mode:
+ description: |
+ Select signal path of input channels. Valid values are:
+ 0: Buffered, low-power, unity-gain path (default)
+ 1: Bypass path
+ 2: PGA path
+ $ref: /schemas/types.yaml#/definitions/uint32
+ enum: [0, 1, 2]
+ default: 0
+
+ diff-channels: true
+
+ bipolar: true
+
+ settling-time-us: true
+
+ adi,buffered-vrefp:
+ description: Enable buffered mode for positive reference.
+ type: boolean
+
+ adi,buffered-vrefn:
+ description: Enable buffered mode for negative reference.
+ type: boolean
+
+ required:
+ - reg
+
+ additionalProperties: false
+
+required:
+ - compatible
+ - reg
+
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/irq.h>
+ spi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ adc@0 {
+ reg = <0>;
+ compatible = "adi,max11410";
+ spi-max-frequency = <8000000>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <25 IRQ_TYPE_EDGE_FALLING>;
+ interrupt-names = "gpio1";
+
+ avdd-supply = <&adc_avdd>;
+
+ vref1p-supply = <&adc_vref1p>;
+ vref1n-supply = <&adc_vref1n>;
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ channel@0 {
+ reg = <0>;
+ };
+
+ channel@1 {
+ reg = <1>;
+ diff-channels = <2 3>;
+ adi,reference = <1>;
+ bipolar;
+ settling-time-us = <100000>;
+ };
+
+ channel@2 {
+ reg = <2>;
+ diff-channels = <7 9>;
+ adi,reference = <5>;
+ adi,input-mode = <2>;
+ settling-time-us = <50000>;
+ };
+ };
+ };
$id: http://devicetree.org/schemas/iio/adc/allwinner,sun8i-a33-ths.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Allwinner A33 Thermal Sensor Device Tree Bindings
+title: Allwinner A33 Thermal Sensor
maintainers:
- Chen-Yu Tsai <wens@csie.org>
$id: "http://devicetree.org/schemas/iio/adc/ingenic,adc.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Ingenic JZ47xx ADC controller IIO bindings
+title: Ingenic JZ47xx ADC controller IIO
maintainers:
- Artur Rojek <contact@artur-rojek.eu>
$id: http://devicetree.org/schemas/iio/adc/motorola,cpcap-adc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Motorola CPCAP PMIC ADC binding
+title: Motorola CPCAP PMIC ADC
maintainers:
- Tony Lindgren <tony@atomide.com>
$id: http://devicetree.org/schemas/iio/adc/nxp,imx8qxp-adc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP IMX8QXP ADC bindings
+title: NXP IMX8QXP ADC
maintainers:
- Cai Huoqing <caihuoqing@baidu.com>
$id: http://devicetree.org/schemas/iio/adc/nxp,lpc1850-adc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP LPC1850 ADC bindings
+title: NXP LPC1850 ADC
maintainers:
- Jonathan Cameron <jic23@kernel.org>
properties:
compatible:
- const: qcom,spmi-iadc
+ items:
+ - enum:
+ - qcom,pm8941-iadc
+ - const: qcom,spmi-iadc
reg:
description: IADC base address and length in the SPMI PMIC register map
#address-cells = <1>;
#size-cells = <0>;
pmic_iadc: adc@3600 {
- compatible = "qcom,spmi-iadc";
+ compatible = "qcom,pm8941-iadc", "qcom,spmi-iadc";
reg = <0x3600>;
interrupts = <0x0 0x36 0x0 IRQ_TYPE_EDGE_RISING>;
qcom,external-resistor-micro-ohms = <10000>;
- items:
- const: qcom,pms405-adc
- const: qcom,spmi-adc-rev2
-
- - items:
- - enum:
- - qcom,spmi-vadc
- - qcom,spmi-adc5
- - qcom,spmi-adc-rev2
- - qcom,spmi-adc7
+ - enum:
+ - qcom,spmi-vadc
+ - qcom,spmi-adc5
+ - qcom,spmi-adc-rev2
+ - qcom,spmi-adc7
reg:
description: VADC base address in the SPMI PMIC register map
compatible:
items:
- enum:
- - renesas,r9a07g043-adc # RZ/G2UL
+ - renesas,r9a07g043-adc # RZ/G2UL and RZ/Five
- renesas,r9a07g044-adc # RZ/G2L
- renesas,r9a07g054-adc # RZ/V2L
- const: renesas,rzg2l-adc
- rockchip,rk3328-saradc
- rockchip,rk3568-saradc
- rockchip,rv1108-saradc
+ - rockchip,rv1126-saradc
- const: rockchip,rk3399-saradc
reg:
$id: http://devicetree.org/schemas/iio/adc/sigma-delta-modulator.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Device-Tree bindings for sigma delta modulator
+title: Sigma delta modulator
maintainers:
- Arnaud Pouliquen <arnaud.pouliquen@foss.st.com>
$id: http://devicetree.org/schemas/iio/adc/sprd,sc2720-adc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Spreadtrum SC27XX series PMICs ADC binding
+title: Spreadtrum SC27XX series PMICs ADC
maintainers:
- Baolin Wang <baolin.wang7@gmail.com>
$id: "http://devicetree.org/schemas/iio/adc/st,stm32-adc.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: STMicroelectronics STM32 ADC bindings
+title: STMicroelectronics STM32 ADC
description: |
STM32 ADC is a successive approximation analog-to-digital converter.
- st,stm32f4-adc-core
- st,stm32h7-adc-core
- st,stm32mp1-adc-core
+ - st,stm32mp13-adc-core
reg:
maxItems: 1
- stm32f4 and stm32h7 share a common ADC interrupt line.
- stm32mp1 has two separate interrupt lines, one for each ADC within
ADC block.
+ - stm32mp13 has an interrupt line per ADC block.
minItems: 1
maxItems: 2
maximum: 36000000
default: 36000000
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: st,stm32mp13-adc-core
+
+ then:
+ properties:
+ clocks:
+ minItems: 1
+ maxItems: 2
+
+ clock-names:
+ items:
+ - const: bus
+ - const: adc
+ minItems: 1
+
+ interrupts:
+ items:
+ - description: ADC interrupt line
+
+ st,max-clk-rate-hz:
+ minimum: 150000
+ maximum: 75000000
+ default: 75000000
+
additionalProperties: false
required:
- st,stm32f4-adc
- st,stm32h7-adc
- st,stm32mp1-adc
+ - st,stm32mp13-adc
reg:
description: |
interrupts:
description: |
IRQ Line for the ADC instance. Valid values are:
- - 0 for adc@0
+ - 0 for adc@0 (single adc for stm32mp13)
- 1 for adc@100
- 2 for adc@200 (stm32f4 only)
maxItems: 1
assigned-resolution-bits:
description: |
Resolution (bits) to use for conversions:
- - can be 6, 8, 10 or 12 on stm32f4
+ - can be 6, 8, 10 or 12 on stm32f4 and stm32mp13
- can be 8, 10, 12, 14 or 16 on stm32h7 and stm32mp1
st,adc-channels:
description: |
List of single-ended channels muxed for this ADC. It can have up to:
- 16 channels, numbered from 0 to 15 (for in0..in15) on stm32f4
+ - 19 channels, numbered from 0 to 18 (for in0..in18) on stm32mp13.
- 20 channels, numbered from 0 to 19 (for in0..in19) on stm32h7 and
stm32mp1.
$ref: /schemas/types.yaml#/definitions/uint32-array
label:
description: |
Unique name to identify which channel this is.
- Reserved label names "vddcore", "vrefint" and "vbat"
+ Reserved label names "vddcore", "vddcpu", "vddq_ddr", "vrefint" and "vbat"
are used to identify internal channels with matching names.
diff-channels:
items:
minimum: 40
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: st,stm32mp13-adc
+
+ then:
+ properties:
+ reg:
+ const: 0x0
+
+ interrupts:
+ const: 0
+
+ assigned-resolution-bits:
+ enum: [6, 8, 10, 12]
+ default: 12
+
+ st,adc-channels:
+ minItems: 1
+ maxItems: 19
+ items:
+ minimum: 0
+ maximum: 18
+
+ st,min-sample-time-nsecs:
+ minItems: 1
+ maxItems: 19
+ items:
+ minimum: 40
additionalProperties: false
required:
$id: http://devicetree.org/schemas/iio/adc/ti,palmas-gpadc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Palmas general purpose ADC IP block devicetree bindings
+title: Palmas general purpose ADC IP block
maintainers:
- Tony Lindgren <tony@atomide.com>
$id: http://devicetree.org/schemas/iio/adc/x-powers,axp209-adc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: X-Powers AXP ADC bindings
+title: X-Powers AXP ADC
maintainers:
- Chen-Yu Tsai <wens@csie.org>
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/iio/addac/adi,ad74115.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Analog Devices AD74115H device
+
+maintainers:
+ - Cosmin Tanislav <cosmin.tanislav@analog.com>
+
+description: |
+ The AD74115H is a single-channel software configurable input/output
+ device for industrial control applications. It contains functionality for
+ analog output, analog input, digital output, digital input, resistance
+ temperature detector, and thermocouple measurements integrated into a single
+ chip solution with an SPI interface. The device features a 16-bit ADC and a
+ 14-bit DAC.
+
+ https://www.analog.com/en/products/ad74115h.html
+
+properties:
+ compatible:
+ enum:
+ - adi,ad74115h
+
+ reg:
+ maxItems: 1
+
+ spi-max-frequency:
+ maximum: 24000000
+
+ spi-cpol: true
+
+ reset-gpios: true
+
+ interrupts:
+ minItems: 1
+ maxItems: 2
+
+ interrupt-names:
+ minItems: 1
+ maxItems: 2
+ items:
+ enum:
+ - adc_rdy
+ - alert
+
+ avdd-supply: true
+ avcc-supply: true
+ dvcc-supply: true
+ dovdd-supply: true
+ refin-supply: true
+
+ adi,ch-func:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: |
+ Channel function.
+ 0 - High impedance
+ 1 - Voltage output
+ 2 - Current output
+ 3 - Voltage input
+ 4 - Current input, externally-powered
+ 5 - Current input, loop-powered
+ 6 - Resistance input
+ 7 - RTD measure
+ 8 - Digital input logic
+ 9 - Digital input, loop-powered
+ 10 - Current output with HART
+ 11 - Current input, externally-powered, with HART
+ 12 - Current input, loop-powered, with HART
+ minimum: 0
+ maximum: 12
+ default: 0
+
+ adi,conv2-mux:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: |
+ Input node for ADC conversion 2.
+ 0 - SENSE_EXT1 to AGND_SENSE
+ 1 - SENSE_EXT2 to AGND_SENSE
+ 2 - SENSE_EXT2 to SENSE_EXT1
+ 3 - AGND to AGND
+ minimum: 0
+ maximum: 3
+ default: 0
+
+ adi,conv2-range-microvolt:
+ description: Conversion range for ADC conversion 2.
+ oneOf:
+ - items:
+ - enum: [-2500000, 0]
+ - const: 2500000
+ - items:
+ - enum: [-12000000, 0]
+ - const: 12000000
+ - items:
+ - const: -2500000
+ - const: 0
+ - items:
+ - const: -104000
+ - const: 104000
+ - items:
+ - const: 0
+ - const: 625000
+
+ adi,sense-agnd-buffer-low-power:
+ type: boolean
+ description:
+ Whether to enable low-power buffered mode for the AGND sense pin.
+
+ adi,lf-buffer-low-power:
+ type: boolean
+ description:
+ Whether to enable low-power buffered mode for the low-side filtered
+ sense pin.
+
+ adi,hf-buffer-low-power:
+ type: boolean
+ description:
+ Whether to enable low-power buffered mode for the high-side filtered
+ sense pin.
+
+ adi,ext2-buffer-low-power:
+ type: boolean
+ description: Whether to enable low-power buffered mode for the EXT2 pin.
+
+ adi,ext1-buffer-low-power:
+ type: boolean
+ description: Whether to enable low-power buffered mode for the EXT1 pin.
+
+ adi,comparator-invert:
+ type: boolean
+ description: Whether to invert the comparator output.
+
+ adi,digital-input-sink-range-high:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: |
+ When not present, the digital input range is from 0 to 3700uA in steps
+ of 120uA, with a ~2k series resistance.
+ When present, the digital input range is from 0 to 7400uA in steps
+ of 240uA, with a ~1k series resistance.
+
+ adi,digital-input-sink-microamp:
+ description: Sink current in digital input mode.
+ minimum: 0
+ maximum: 3700
+ default: 0
+
+ adi,digital-input-debounce-mode-counter-reset:
+ type: boolean
+ description: |
+ When not present, a counter increments when the signal is asserted
+ and decrements when the signal is de-asserted.
+ When present, a counter increments while the signal is asserted and
+ resets when the signal de-asserts
+
+ adi,digital-input-unbuffered:
+ type: boolean
+ description: Whether to buffer digital input signals.
+
+ adi,digital-input-short-circuit-detection:
+ type: boolean
+ description: Whether to detect digital input short circuits.
+
+ adi,digital-input-open-circuit-detection:
+ type: boolean
+ description: Whether to detect digital input open circuits.
+
+ adi,digital-input-threshold-mode-fixed:
+ type: boolean
+ description: |
+ When not present, the digital input threshold range is -0.96 * AVDD
+ to AVDD.
+ When present, the threshold range is fixed from -19V to 30V.
+
+ adi,dac-bipolar:
+ type: boolean
+ description: |
+ When not present, the DAC operates in the 0V to 12V range.
+ When present, the DAC operates in the -12V to 12V range.
+
+ adi,charge-pump:
+ type: boolean
+ description: Whether to enable the internal charge pump.
+
+ adi,dac-hart-slew:
+ type: boolean
+ description: Whether to use a HART-compatible slew rate.
+
+ adi,dac-current-limit-low:
+ type: boolean
+ description: |
+ When not present, the DAC short-circuit current limit is 32mA in
+ either source or sink for VOUT and 4mA sink for IOUT.
+ When present, the limit is 16mA in either source or sink for VOUT,
+ 1mA sink for IOUT.
+
+ adi,4-wire-rtd:
+ type: boolean
+ description: |
+ When not present, the ADC should be used for measuring 3-wire RTDs.
+ When present, the ADC should be used for measuring 4-wire RTDs.
+
+ adi,3-wire-rtd-excitation-swap:
+ type: boolean
+ description: Whether to swap the excitation for 3-wire RTD.
+
+ adi,rtd-excitation-current-microamp:
+ description: Excitation current to apply to RTD.
+ enum: [250, 500, 750, 1000]
+ default: 250
+
+ adi,ext1-burnout:
+ type: boolean
+ description: Whether to enable burnout current for EXT1.
+
+ adi,ext1-burnout-current-nanoamp:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description:
+ Burnout current in nanoamps to be applied to EXT1.
+ enum: [0, 50, 500, 1000, 10000]
+ default: 0
+
+ adi,ext1-burnout-current-polarity-sourcing:
+ type: boolean
+ description: |
+ When not present, the burnout current polarity for EXT1 is sinking.
+ When present, the burnout current polarity for EXT1 is sourcing.
+
+ adi,ext2-burnout:
+ type: boolean
+ description: Whether to enable burnout current for EXT2.
+
+ adi,ext2-burnout-current-nanoamp:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: Burnout current in nanoamps to be applied to EXT2.
+ enum: [0, 50, 500, 1000, 10000]
+ default: 0
+
+ adi,ext2-burnout-current-polarity-sourcing:
+ type: boolean
+ description: |
+ When not present, the burnout current polarity for EXT2 is sinking.
+ When present, the burnout current polarity for EXT2 is sourcing.
+
+ adi,viout-burnout:
+ type: boolean
+ description: Whether to enable burnout current for VIOUT.
+
+ adi,viout-burnout-current-nanoamp:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: Burnout current in nanoamps to be applied to VIOUT.
+ enum: [0, 1000, 10000]
+ default: 0
+
+ adi,viout-burnout-current-polarity-sourcing:
+ type: boolean
+ description: |
+ When not present, the burnout current polarity for VIOUT is sinking.
+ When present, the burnout current polarity for VIOUT is sourcing.
+
+ adi,gpio0-mode:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: |
+ GPIO functions.
+ 0 - Disabled
+ 1 - Logic I/O
+ 2 - Comparator output
+ 3 - Control HART CD
+ 4 - Monitor HART CD
+ 5 - Monitor HART EOM status
+ minimum: 0
+ maximum: 5
+ default: 0
+
+ adi,gpio1-mode:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: |
+ GPIO functions.
+ 0 - Disabled
+ 1 - Logic I/O
+ 2 - Drive external digital output FET
+ 3 - Control HART RXD
+ 4 - Monitor HART RXD
+ 5 - Monitor HART SOM status
+ minimum: 0
+ maximum: 5
+ default: 0
+
+ adi,gpio2-mode:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: |
+ GPIO functions.
+ 0 - Disabled
+ 1 - Logic I/O
+ 2 - Drive internal digital output FET
+ 3 - Control HART TXD
+ 4 - Monitor HART TXD
+ 5 - Monitor HART TX complete status
+ minimum: 0
+ maximum: 5
+ default: 0
+
+ adi,gpio3-mode:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: |
+ GPIO functions.
+ 0 - Disabled
+ 1 - Logic I/O
+ 2 - High impedance
+ 3 - Control HART RTS
+ 4 - Monitor HART RTS
+ 5 - Monitor HART CD complete status
+ minimum: 0
+ maximum: 5
+ default: 0
+
+required:
+ - compatible
+ - reg
+ - spi-cpol
+ - avdd-supply
+
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+ - if:
+ required:
+ - adi,digital-input-sink-range-high
+ then:
+ properties:
+ adi,digital-input-sink-microamp:
+ maximum: 7400
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+ #include <dt-bindings/interrupt-controller/irq.h>
+
+ spi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ addac@0 {
+ compatible = "adi,ad74115h";
+ reg = <0>;
+
+ spi-max-frequency = <12000000>;
+ spi-cpol;
+
+ reset-gpios = <&gpio 27 GPIO_ACTIVE_LOW>;
+
+ interrupt-parent = <&gpio>;
+ interrupts = <26 IRQ_TYPE_EDGE_FALLING>;
+ interrupt-names = "adc_rdy";
+
+ avdd-supply = <&ad74115_avdd>;
+
+ adi,ch-func = <1>;
+ adi,conv2-mux = <2>;
+ adi,conv2-range-microvolt = <(-12000000) 12000000>;
+
+ adi,gpio0-mode = <1>;
+ adi,gpio1-mode = <1>;
+ adi,gpio2-mode = <1>;
+ adi,gpio3-mode = <1>;
+
+ adi,dac-bipolar;
+ };
+ };
+...
Shunt (sense) resistor value in micro-Ohms.
default: 100000000
+ reset-gpios:
+ maxItems: 1
+
required:
- compatible
- reg
- spi-cpol
- refin-supply
-additionalProperties: false
-
patternProperties:
"^channel@[0-3]$":
type: object
required:
- reg
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
+
examples:
- |
#include <dt-bindings/gpio/gpio.h>
#address-cells = <1>;
#size-cells = <0>;
- cs-gpios = <&gpio 17 GPIO_ACTIVE_LOW>;
-
- ad74413r@0 {
+ addac@0 {
compatible = "adi,ad74413r";
reg = <0>;
spi-max-frequency = <1000000>;
interrupts = <26 IRQ_TYPE_EDGE_FALLING>;
refin-supply = <&ad74413r_refin>;
+ reset-gpios = <&gpio2 6 GPIO_ACTIVE_LOW>;
channel@0 {
reg = <0>;
- if:
properties:
adi,dc-dc-mode:
- contains:
- enum: [1, 3]
+ enum: [1, 3]
then:
properties:
adi,range-microvolt: false
title: Analog Devices AD5766 DAC device driver
maintainers:
- - Cristian Pop <cristian.pop@analog.com>
+ - Nuno Sá <nuno.sa@analog.com>
description: |
Bindings for the Analog Devices AD5766 current DAC device. Datasheet can be
$id: http://devicetree.org/schemas/iio/dac/nxp,lpc1850-dac.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP LPC1850 DAC bindings
+title: NXP LPC1850 DAC
maintainers:
- Jonathan Cameron <jic23@kernel.org>
$id: "http://devicetree.org/schemas/iio/dac/st,stm32-dac.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: STMicroelectronics STM32 DAC bindings
+title: STMicroelectronics STM32 DAC
description: |
The STM32 DAC is a 12-bit voltage output digital-to-analog converter. The DAC
2: +2dBm
3: +5dBm
-additionalProperties: false
-
required:
- compatible
- reg
- clocks
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
+
examples:
- |
spi {
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/iio/frequency/adi,adf4377.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: ADF4377 Microwave Wideband Synthesizer with Integrated VCO
+
+maintainers:
+ - Antoniu Miclaus <antoniu.miclaus@analog.com>
+ - Dragos Bogdan <dragos.bogdan@analog.com>
+
+description: |
+ The ADF4377 is a high performance, ultralow jitter, dual output integer-N
+ phased locked loop (PLL) with integrated voltage controlled oscillator (VCO)
+ ideally suited for data converter and mixed signal front end (MxFE) clock
+ applications.
+
+ https://www.analog.com/en/products/adf4377.html
+
+properties:
+ compatible:
+ enum:
+ - adi,adf4377
+ - adi,adf4378
+
+ reg:
+ maxItems: 1
+
+ spi-max-frequency:
+ maximum: 10000000
+
+ clocks:
+ maxItems: 1
+
+ clock-names:
+ description:
+ External clock that provides reference input frequency.
+ items:
+ - const: ref_in
+
+ chip-enable-gpios:
+ description:
+ GPIO that controls the Chip Enable Pin.
+ maxItems: 1
+
+ clk1-enable-gpios:
+ description:
+ GPIO that controls the Enable Clock 1 Output Buffer Pin.
+ maxItems: 1
+
+ clk2-enable-gpios:
+ description:
+ GPIO that controls the Enable Clock 2 Output Buffer Pin.
+ maxItems: 1
+
+ adi,muxout-select:
+ description:
+ On chip multiplexer output selection.
+ high_z - MUXOUT Pin set to high-Z.
+ lock_detect - MUXOUT Pin set to lock detector output.
+ muxout_low - MUXOUT Pin set to low.
+ f_div_rclk_2 - MUXOUT Pin set to fDIV_RCLK/2.
+ f_div_nclk_2 - MUXOUT Pin set to fDIV_NCLK/2.
+ muxout_high - MUXOUT Pin set to high.
+ enum: [high_z, lock_detect, muxout_low, f_div_rclk_2, f_div_nclk_2, muxout_high]
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ spi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ frequency@0 {
+ compatible = "adi,adf4377";
+ reg = <0>;
+ spi-max-frequency = <10000000>;
+ clocks = <&adf4377_ref_in>;
+ clock-names = "ref_in";
+ };
+ };
+...
- clock-names
- vcm-supply
-additionalProperties: false
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
examples:
- |
- clock-names
- vcm-supply
-additionalProperties: false
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
examples:
- |
title: ADMV4420 K Band Downconverter
maintainers:
- - Cristian Pop <cristian.pop@analog.com>
+ - Nuno Sá <nuno.sa@analog.com>
description:
The ADMV4420 is a highly integrated, double balanced, active
- compatible
- reg
-additionalProperties: false
+
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
examples:
- |
- clocks
- clock-names
-additionalProperties: false
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
examples:
- |
- spi-cpol
- spi-cpha
-additionalProperties: false
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
examples:
- |
- compatible
- reg
-additionalProperties: false
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
examples:
- |
- spi-cpol
allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
- if:
properties:
compatible:
dependencies:
adi,sync-mode: [ clocks ]
-additionalProperties: false
+unevaluatedProperties: false
examples:
- |
- invensense,icm42602
- invensense,icm42605
- invensense,icm42622
+ - invensense,icm42631
reg:
maxItems: 1
- st,lsm6dsrx
- st,lsm6dst
- st,lsm6dsop
+ - st,lsm6dsv
+ - st,lsm6dso16is
- items:
- const: st,asm330lhhx
- const: st,lsm6dsr
- items:
- const: st,lsm6dstx
- const: st,lsm6dst
+ - items:
+ - const: st,lsm6dsv16x
+ - const: st,lsm6dsv
+ - items:
+ - const: st,ism330is
+ - const: st,lsm6dso16is
reg:
maxItems: 1
$id: http://devicetree.org/schemas/iio/multiplexer/io-channel-mux.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: I/O channel multiplexer bindings
+title: I/O channel multiplexer
maintainers:
- Peter Rosin <peda@axentia.se>
- compatible
- reg
-additionalProperties: false
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
examples:
- |
compatible = "meas,ms5611";
reg = <0>;
vdd-supply = <&ldo_3v3_gnss>;
+ spi-max-frequency = <20000000>;
};
};
...
- compatible
- reg
-additionalProperties: false
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
examples:
- |
- spi-cpha
- interrupts
-additionalProperties: false
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
examples:
- |
spi-cpha: true
-additionalProperties: false
-
required:
- compatible
- reg
spi-cpol: [ spi-cpha ]
spi-cpha: [ spi-cpol ]
+allOf:
+ - $ref: /schemas/spi/spi-peripheral-props.yaml#
+
+unevaluatedProperties: false
+
examples:
- |
spi {
$id: http://devicetree.org/schemas/iio/temperature/adi,ltc2983.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Analog Devices LTC2983 Multi-sensor Temperature system
+title: Analog Devices LTC2983, LTC2986, LTM2985 Multi-sensor Temperature system
maintainers:
- Nuno Sá <nuno.sa@analog.com>
description: |
- Analog Devices LTC2983 Multi-Sensor Digital Temperature Measurement System
+ Analog Devices LTC2983, LTC2984, LTC2986, LTM2985 Multi-Sensor Digital
+ Temperature Measurement Systems
+
https://www.analog.com/media/en/technical-documentation/data-sheets/2983fc.pdf
+ https://www.analog.com/media/en/technical-documentation/data-sheets/2984fb.pdf
+ https://www.analog.com/media/en/technical-documentation/data-sheets/29861fa.pdf
+ https://www.analog.com/media/en/technical-documentation/data-sheets/ltm2985.pdf
properties:
compatible:
- enum:
- - adi,ltc2983
+ oneOf:
+ - enum:
+ - adi,ltc2983
+ - adi,ltc2986
+ - adi,ltm2985
+ - items:
+ - const: adi,ltc2984
+ - const: adi,ltc2983
reg:
maxItems: 1
maxItems: 1
adi,mux-delay-config-us:
- description:
- The LTC2983 performs 2 or 3 internal conversion cycles per temperature
- result. Each conversion cycle is performed with different excitation and
- input multiplexer configurations. Prior to each conversion, these
- excitation circuits and input switch configurations are changed and an
- internal 1ms delay ensures settling prior to the conversion cycle in most
- cases. An extra delay can be configured using this property. The value is
- rounded to nearest 100us.
+ description: |
+ Extra delay prior to each conversion, in addition to the internal 1ms
+ delay, for the multiplexer to switch input configurations and
+ excitation values.
+
+ This property is supposed to be in microseconds, but to maintain
+ compatibility, this value will be multiplied by 100 before usage.
maximum: 255
+ default: 0
adi,filter-notch-freq:
description:
- Set's the default setting of the digital filter. The default is
- simultaneous 50/60Hz rejection.
+ Notch frequency of the digital filter.
0 - 50/60Hz rejection
1 - 60Hz rejection
2 - 50Hz rejection
$ref: /schemas/types.yaml#/definitions/uint32
minimum: 0
maximum: 2
+ default: 0
'#address-cells':
const: 1
const: 0
patternProperties:
- "@([1-9]|1[0-9]|20)$":
+ "@([0-9a-f]+)$":
type: object
+ description: Sensor.
properties:
reg:
description:
- The channel number. It can be connected to one of the 20 channels of
- the device.
+ Channel number. Connects the sensor to the channel with this number
+ of the device.
minimum: 1
maximum: 20
adi,sensor-type:
- description: Identifies the type of sensor connected to the device.
+ description: Type of sensor connected to the device.
$ref: /schemas/types.yaml#/definitions/uint32
required:
"^thermocouple@":
type: object
- description:
- Represents a thermocouple sensor which is connected to one of the device
- channels.
+ description: Thermocouple sensor.
properties:
adi,sensor-type:
maximum: 9
adi,single-ended:
- description:
- Boolean property which set's the thermocouple as single-ended.
+ description: Whether the sensor is single-ended.
type: boolean
adi,sensor-oc-current-microamp:
- description:
- This property set's the pulsed current value applied during
- open-circuit detect.
+ description: Pulsed current value applied during open-circuit detect.
enum: [10, 100, 500, 1000]
+ default: 10
adi,cold-junction-handle:
description:
- Phandle which points to a sensor object responsible for measuring
- the thermocouple cold junction temperature.
- $ref: "/schemas/types.yaml#/definitions/phandle"
+ Sensor responsible for measuring the thermocouple cold junction
+ temperature.
+ $ref: /schemas/types.yaml#/definitions/phandle
adi,custom-thermocouple:
description:
- This is a table, where each entry should be a pair of
- voltage(mv)-temperature(K). The entries must be given in nv and uK
- so that, the original values must be multiplied by 1000000. For
- more details look at table 69 and 70.
- Note should be signed, but dtc doesn't currently maintain the
- sign.
+ Used for digitizing custom thermocouples.
+ See Page 59 of the datasheet.
$ref: /schemas/types.yaml#/definitions/uint64-matrix
minItems: 3
maxItems: 64
items:
- minItems: 2
- maxItems: 2
+ items:
+ - description: Voltage point in nV, signed.
+ - description: Temperature point in uK.
+
+ allOf:
+ - if:
+ properties:
+ adi,sensor-type:
+ const: 9
+ then:
+ required:
+ - adi,custom-thermocouple
"^diode@":
type: object
- description:
- Represents a diode sensor which is connected to one of the device
- channels.
+ description: Diode sensor.
properties:
adi,sensor-type:
- description: Identifies the sensor as a diode.
+ description: Sensor type for diodes.
$ref: /schemas/types.yaml#/definitions/uint32
const: 28
adi,single-ended:
- description: Boolean property which set's the diode as single-ended.
+ description: Whether the sensor is single-ended.
type: boolean
adi,three-conversion-cycles:
description:
- Boolean property which set's three conversion cycles removing
- parasitic resistance effects between the LTC2983 and the diode.
+ Whether to use three conversion cycles to remove parasitic
+ resistance between the device and the diode.
type: boolean
adi,average-on:
description:
- Boolean property which enables a running average of the diode
- temperature reading. This reduces the noise when the diode is used
- as a cold junction temperature element on an isothermal block
- where temperatures change slowly.
+ Whether to use a running average of the diode temperature
+ reading to reduce the noise when the diode is used as a cold
+ junction temperature element on an isothermal block where
+ temperatures change slowly.
type: boolean
adi,excitation-current-microamp:
description:
- This property controls the magnitude of the excitation current
- applied to the diode. Depending on the number of conversions
- cycles, this property will assume different predefined values on
- each cycle. Just set the value of the first cycle (1l).
+ Magnitude of the 1l excitation current applied to the diode.
+ 4l excitation current will be 4 times this value, and 8l
+ excitation current will be 8 times value.
enum: [10, 20, 40, 80]
+ default: 10
adi,ideal-factor-value:
description:
- This property sets the diode ideality factor. The real value must
- be multiplied by 1000000 to remove the fractional part. For more
- information look at table 20 of the datasheet.
+ Diode ideality factor.
+ Set this property to 1000000 times the real value.
$ref: /schemas/types.yaml#/definitions/uint32
+ default: 0
"^rtd@":
type: object
- description:
- Represents a rtd sensor which is connected to one of the device channels.
+ description: RTD sensor.
properties:
reg:
maximum: 18
adi,rsense-handle:
- description:
- Phandle pointing to a rsense object associated with this RTD.
- $ref: "/schemas/types.yaml#/definitions/phandle"
+ description: Associated sense resistor sensor.
+ $ref: /schemas/types.yaml#/definitions/phandle
adi,number-of-wires:
description:
- Identifies the number of wires used by the RTD. Setting this
- property to 5 means 4 wires with Kelvin Rsense.
+ Number of wires used by the RTD.
+ 5 means 4 wires with Kelvin sense resistor.
$ref: /schemas/types.yaml#/definitions/uint32
enum: [2, 3, 4, 5]
+ default: 2
adi,rsense-share:
description:
- Boolean property which enables Rsense sharing, where one sense
- resistor is used for multiple 2-, 3-, and/or 4-wire RTDs.
- type: boolean
-
- adi,current-rotate:
- description:
- Boolean property which enables excitation current rotation to
- automatically remove parasitic thermocouple effects. Note that
- this property is not allowed for 2- and 3-wire RTDs.
+ Whether to enable sense resistor sharing, where one sense
+ resistor is used by multiple sensors.
type: boolean
adi,excitation-current-microamp:
- description:
- This property controls the magnitude of the excitation current
- applied to the RTD.
+ description: Excitation current applied to the RTD.
enum: [5, 10, 25, 50, 100, 250, 500, 1000]
+ default: 5
adi,rtd-curve:
- description:
- This property set the RTD curve used and the corresponding
- Callendar-VanDusen constants. Look at table 30 of the datasheet.
+ description: |
+ RTD curve and the corresponding Callendar-VanDusen constants.
+ 0 - European
+ 1 - American
+ 2 - Japanese
+ 3 - ITS-90
$ref: /schemas/types.yaml#/definitions/uint32
minimum: 0
maximum: 3
+ default: 0
adi,custom-rtd:
description:
- This is a table, where each entry should be a pair of
- resistance(ohm)-temperature(K). The entries added here are in uohm
- and uK. For more details values look at table 74 and 75.
+ Used for digitizing custom RTDs.
+ See Page 62 of the datasheet.
$ref: /schemas/types.yaml#/definitions/uint64-matrix
+ minItems: 3
+ maxItems: 64
items:
- minItems: 3
- maxItems: 64
items:
- minItems: 2
- maxItems: 2
+ - description: Resistance point in uOhms.
+ - description: Temperature point in uK.
required:
- adi,rsense-handle
- dependencies:
- adi,current-rotate: [ "adi,rsense-share" ]
+ allOf:
+ - if:
+ properties:
+ adi,number-of-wires:
+ const: 4
+ then:
+ properties:
+ adi,current-rotate:
+ description:
+ Whether to enable excitation current rotation to automatically
+ remove parasitic thermocouple effects.
+ type: boolean
+
+ dependencies:
+ adi,current-rotate: [ "adi,rsense-share" ]
+
+ - if:
+ properties:
+ adi,sensor-type:
+ const: 18
+ then:
+ required:
+ - adi,custom-rtd
"^thermistor@":
type: object
- description:
- Represents a thermistor sensor which is connected to one of the device
- channels.
+ description: Thermistor sensor.
properties:
adi,sensor-type:
maximum: 27
adi,rsense-handle:
- description:
- Phandle pointing to a rsense object associated with this
- thermistor.
- $ref: "/schemas/types.yaml#/definitions/phandle"
+ description: Associated sense resistor sensor.
+ $ref: /schemas/types.yaml#/definitions/phandle
adi,single-ended:
- description:
- Boolean property which set's the thermistor as single-ended.
+ description: Whether the sensor is single-ended.
type: boolean
adi,rsense-share:
description:
- Boolean property which enables Rsense sharing, where one sense
- resistor is used for multiple thermistors. Note that this property
- is ignored if adi,single-ended is set.
+ Whether to enable sense resistor sharing, where one sense
+ resistor is used by multiple sensors.
type: boolean
adi,current-rotate:
description:
- Boolean property which enables excitation current rotation to
- automatically remove parasitic thermocouple effects.
+ Whether to enable excitation current rotation to automatically
+ remove parasitic thermocouple effects.
type: boolean
adi,excitation-current-nanoamp:
description:
- This property controls the magnitude of the excitation current
- applied to the thermistor. Value 0 set's the sensor in auto-range
- mode.
- $ref: /schemas/types.yaml#/definitions/uint32
+ Excitation current applied to the thermistor.
+ 0 sets the sensor in auto-range mode.
enum: [0, 250, 500, 1000, 5000, 10000, 25000, 50000, 100000, 250000,
500000, 1000000]
+ default: 0
adi,custom-thermistor:
description:
- This is a table, where each entry should be a pair of
- resistance(ohm)-temperature(K). The entries added here are in uohm
- and uK only for custom thermistors. For more details look at table
- 78 and 79.
+ Used for digitizing custom thermistors.
+ See Page 65 of the datasheet.
$ref: /schemas/types.yaml#/definitions/uint64-matrix
minItems: 3
maxItems: 64
items:
- minItems: 2
- maxItems: 2
+ items:
+ - description: Resistance point in uOhms.
+ - description: Temperature point in uK.
adi,custom-steinhart:
description:
- Steinhart-Hart coefficients are also supported and can
- be programmed into the device memory using this property. For
- Steinhart sensors the coefficients are given in the raw
- format. Look at table 82 for more information.
+ Steinhart-Hart coefficients in raw format, used for digitizing
+ custom thermistors.
+ See Page 68 of the datasheet.
$ref: /schemas/types.yaml#/definitions/uint32-array
- items:
- minItems: 6
- maxItems: 6
+ minItems: 6
+ maxItems: 6
required:
- adi,rsense-handle
dependencies:
adi,current-rotate: [ "adi,rsense-share" ]
+ allOf:
+ - if:
+ properties:
+ adi,sensor-type:
+ const: 26
+ then:
+ properties:
+ adi,excitation-current-nanoamp:
+ enum: [250, 500, 1000, 5000, 10000, 25000, 50000, 100000,
+ 250000, 500000, 1000000]
+ default: 1000
+ required:
+ - adi,custom-steinhart
+ - if:
+ properties:
+ adi,sensor-type:
+ const: 27
+ then:
+ properties:
+ adi,excitation-current-nanoamp:
+ enum: [250, 500, 1000, 5000, 10000, 25000, 50000, 100000,
+ 250000, 500000, 1000000]
+ default: 1000
+ required:
+ - adi,custom-thermistor
+
"^adc@":
type: object
- description: Represents a channel which is being used as a direct adc.
+ description: Direct ADC sensor.
properties:
adi,sensor-type:
- description: Identifies the sensor as a direct adc.
+ description: Sensor type for direct ADC sensors.
$ref: /schemas/types.yaml#/definitions/uint32
const: 30
adi,single-ended:
- description: Boolean property which set's the adc as single-ended.
+ description: Whether the sensor is single-ended.
+ type: boolean
+
+ "^temp@":
+ type: object
+ description: Active analog temperature sensor.
+
+ properties:
+ adi,sensor-type:
+ description: Sensor type for active analog temperature sensors.
+ $ref: /schemas/types.yaml#/definitions/uint32
+ const: 31
+
+ adi,single-ended:
+ description: Whether the sensor is single-ended.
type: boolean
+ adi,custom-temp:
+ description:
+ Used for digitizing active analog temperature sensors.
+ See Page 67 of the LTM2985 datasheet.
+ $ref: /schemas/types.yaml#/definitions/uint64-matrix
+ minItems: 3
+ maxItems: 64
+ items:
+ items:
+ - description: Voltage point in nV, signed.
+ - description: Temperature point in uK.
+
+ required:
+ - adi,custom-temp
+
"^rsense@":
type: object
- description:
- Represents a rsense which is connected to one of the device channels.
- Rsense are used by thermistors and RTD's.
+ description: Sense resistor sensor.
properties:
reg:
maximum: 20
adi,sensor-type:
- description: Identifies the sensor as a rsense.
+ description: Sensor type sense resistor sensors.
$ref: /schemas/types.yaml#/definitions/uint32
const: 29
adi,rsense-val-milli-ohms:
- description:
- Sets the value of the sense resistor. Look at table 20 of the
- datasheet for information.
+ description: Value of the sense resistor.
required:
- adi,rsense-val-milli-ohms
additionalProperties: false
+allOf:
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - adi,ltc2983
+ - adi,ltc2984
+ then:
+ patternProperties:
+ "^temp@": false
+
examples:
- |
#include <dt-bindings/interrupt-controller/irq.h>
#address-cells = <1>;
#size-cells = <0>;
- sensor_ltc2983: ltc2983@0 {
+ temperature-sensor@0 {
compatible = "adi,ltc2983";
reg = <0>;
$id: http://devicetree.org/schemas/input/fsl,scu-key.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: i.MX SCU Client Device Node - SCU key bindings based on SCU Message Protocol
+title: i.MX SCU Client Device Node - SCU Key Based on SCU Message Protocol
maintainers:
- Dong Aisheng <aisheng.dong@nxp.com>
$id: http://devicetree.org/schemas/input/gpio-keys.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Device-Tree bindings for GPIO attached keys
+title: GPIO attached keys
maintainers:
- Rob Herring <robh@kernel.org>
$id: http://devicetree.org/schemas/input/input.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Common input schema binding
+title: Input Devices Common Properties
maintainers:
- Dmitry Torokhov <dmitry.torokhov@gmail.com>
$id: http://devicetree.org/schemas/input/matrix-keymap.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Common key matrices binding for matrix-connected key boards
+title: Common Key Matrices on Matrix-connected Key Boards
maintainers:
- Olof Johansson <olof@lixom.net>
$id: "http://devicetree.org/schemas/input/microchip,cap11xx.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Device tree bindings for Microchip CAP11xx based capacitive touch sensors
+title: Microchip CAP11xx based capacitive touch sensors
description: |
The Microchip CAP1xxx Family of RightTouchTM multiple-channel capacitive
$id: http://devicetree.org/schemas/input/pine64,pinephone-keyboard.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Pine64 PinePhone keyboard device tree bindings
+title: Pine64 PinePhone keyboard
maintainers:
- Samuel Holland <samuel@sholland.org>
$id: http://devicetree.org/schemas/input/touchscreen/chipone,icn8318.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ChipOne ICN8318 Touchscreen Controller Device Tree Bindings
+title: ChipOne ICN8318 Touchscreen Controller
maintainers:
- Dmitry Torokhov <dmitry.torokhov@gmail.com>
$id: http://devicetree.org/schemas/input/touchscreen/cypress,cy8ctma140.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Cypress CY8CTMA140 series touchscreen controller bindings
+title: Cypress CY8CTMA140 series touchscreen controller
maintainers:
- Linus Walleij <linus.walleij@linaro.org>
$id: http://devicetree.org/schemas/input/touchscreen/cypress,cy8ctma340.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Cypress CY8CTMA340 series touchscreen controller bindings
+title: Cypress CY8CTMA340 series touchscreen controller
description: The Cypress CY8CTMA340 series (also known as "CYTTSP" after
the marketing name Cypress TrueTouch Standard Product) touchscreens can
$id: http://devicetree.org/schemas/input/touchscreen/edt-ft5x06.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: FocalTech EDT-FT5x06 Polytouch Bindings
+title: FocalTech EDT-FT5x06 Polytouch
description: |
There are 5 variants of the chip for various touch panel sizes
$id: http://devicetree.org/schemas/input/touchscreen/goodix.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Goodix GT9xx series touchscreen controller Bindings
+title: Goodix GT9xx series touchscreen controller
maintainers:
- Dmitry Torokhov <dmitry.torokhov@gmail.com>
$id: http://devicetree.org/schemas/input/touchscreen/himax,hx83112b.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Himax hx83112b touchscreen controller bindings
+title: Himax hx83112b touchscreen controller
maintainers:
- Job Noorman <job@noorman.info>
$id: http://devicetree.org/schemas/input/touchscreen/hycon,hy46xx.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Hycon HY46XX series touchscreen controller bindings
+title: Hycon HY46XX series touchscreen controller
description: |
There are 6 variants of the chip for various touch panel sizes and cover lens material
$id: http://devicetree.org/schemas/input/touchscreen/imagis,ist3038c.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Imagis IST30XXC family touchscreen controller bindings
+title: Imagis IST30XXC family touchscreen controller
maintainers:
- Markuss Broks <markuss.broks@gmail.com>
$id: http://devicetree.org/schemas/input/touchscreen/melfas,mms114.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Melfas MMS114 family touchscreen controller bindings
+title: Melfas MMS114 family touchscreen controller
maintainers:
- Linus Walleij <linus.walleij@linaro.org>
$id: http://devicetree.org/schemas/input/touchscreen/mstar,msg2638.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MStar msg2638 touchscreen controller Bindings
+title: MStar msg2638 touchscreen controller
maintainers:
- Vincent Knecht <vincent.knecht@mailoo.org>
$id: http://devicetree.org/schemas/input/touchscreen/pixcir,pixcir_ts.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Pixcir Touchscreen Controller Device Tree Bindings
+title: Pixcir Touchscreen Controller
maintainers:
- Dmitry Torokhov <dmitry.torokhov@gmail.com>
$id: http://devicetree.org/schemas/input/touchscreen/silead,gsl1680.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Silead GSL1680 Touchscreen Controller Device Tree Bindings
+title: Silead GSL1680 Touchscreen Controller
maintainers:
- Dmitry Torokhov <dmitry.torokhov@gmail.com>
$id: http://devicetree.org/schemas/input/touchscreen/ti,tsc2005.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments TSC2004 and TSC2005 touchscreen controller bindings
+title: Texas Instruments TSC2004 and TSC2005 touchscreen controller
maintainers:
- Marek Vasut <marex@denx.de>
$id: http://devicetree.org/schemas/input/touchscreen/touchscreen.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Common touchscreen Bindings
+title: Common touchscreen
maintainers:
- Dmitry Torokhov <dmitry.torokhov@gmail.com>
$id: http://devicetree.org/schemas/input/touchscreen/zinitix,bt400.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Zinitix BT4xx and BT5xx series touchscreen controller bindings
+title: Zinitix BT4xx and BT5xx series touchscreen controller
description: The Zinitix BT4xx and BT5xx series of touchscreen controllers
are Korea-produced touchscreens with embedded microcontrollers. The
- items:
- enum:
- qcom,sc7280-cpu-bwmon
+ - qcom,sc8280xp-cpu-bwmon
- qcom,sdm845-bwmon
- const: qcom,msm8998-bwmon
- const: qcom,msm8998-bwmon # BWMON v4
+ - items:
+ - enum:
+ - qcom,sc8280xp-llcc-bwmon
+ - const: qcom,sc7280-llcc-bwmon
- const: qcom,sc7280-llcc-bwmon # BWMON v5
- const: qcom,sdm845-llcc-bwmon # BWMON v5
properties:
compatible:
- enum:
- - qcom,sc7180-osm-l3
- - qcom,sc7280-epss-l3
- - qcom,sc8180x-osm-l3
- - qcom,sdm845-osm-l3
- - qcom,sm8150-osm-l3
- - qcom,sm8250-epss-l3
+ oneOf:
+ - items:
+ - enum:
+ - qcom,sc7180-osm-l3
+ - qcom,sc8180x-osm-l3
+ - qcom,sdm845-osm-l3
+ - qcom,sm8150-osm-l3
+ - const: qcom,osm-l3
+ - items:
+ - enum:
+ - qcom,sc7280-epss-l3
+ - qcom,sc8280xp-epss-l3
+ - qcom,sm8250-epss-l3
+ - qcom,sm8350-epss-l3
+ - const: qcom,epss-l3
reg:
maxItems: 1
#define RPMH_CXO_CLK 0
osm_l3: interconnect@17d41000 {
- compatible = "qcom,sdm845-osm-l3";
+ compatible = "qcom,sdm845-osm-l3", "qcom,osm-l3";
reg = <0x17d41000 0x1400>;
clocks = <&rpmhcc RPMH_CXO_CLK>, <&gcc GPLL0>;
$id: http://devicetree.org/schemas/interrupt-controller/ingenic,intc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs interrupt controller devicetree bindings
+title: Ingenic SoCs interrupt controller
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/interrupt-controller/mrvl,intc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell MMP/Orion Interrupt controller bindings
+title: Marvell MMP/Orion Interrupt controller
maintainers:
- Andrew Lunn <andrew@lunn.ch>
$id: http://devicetree.org/schemas/interrupt-controller/nuvoton,wpcm450-aic.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Nuvoton WPCM450 Advanced Interrupt Controller bindings
+title: Nuvoton WPCM450 Advanced Interrupt Controller
maintainers:
- Jonathan Neuschäfer <j.neuschaefer@gmx.net>
$id: http://devicetree.org/schemas/interrupt-controller/realtek,rtl-intc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Realtek RTL SoC interrupt controller devicetree bindings
+title: Realtek RTL SoC interrupt controller
description:
Interrupt controller and router for Realtek MIPS SoCs, allowing each SoC
$id: http://devicetree.org/schemas/interrupt-controller/renesas,irqc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: DT bindings for the R-Mobile/R-Car/RZ/G interrupt controller
+title: R-Mobile/R-Car/RZ/G interrupt controller
maintainers:
- Geert Uytterhoeven <geert+renesas@glider.be>
- enum:
- qcom,msm8996-smmu-v2
- qcom,msm8998-smmu-v2
+ - qcom,sdm630-smmu-v2
- const: qcom,smmu-v2
- - description: Qcom SoCs implementing "arm,mmu-500"
+ - description: Qcom SoCs implementing "qcom,smmu-500" and "arm,mmu-500"
items:
- enum:
- qcom,qcm2290-smmu-500
+ - qcom,qdu1000-smmu-500
- qcom,sc7180-smmu-500
- qcom,sc7280-smmu-500
- qcom,sc8180x-smmu-500
- qcom,sc8280xp-smmu-500
+ - qcom,sdm670-smmu-500
- qcom,sdm845-smmu-500
+ - qcom,sm6115-smmu-500
+ - qcom,sm6350-smmu-500
+ - qcom,sm6375-smmu-500
+ - qcom,sm8150-smmu-500
+ - qcom,sm8250-smmu-500
+ - qcom,sm8350-smmu-500
+ - qcom,sm8450-smmu-500
+ - const: qcom,smmu-500
+ - const: arm,mmu-500
+
+ - description: Qcom SoCs implementing "arm,mmu-500" (non-qcom implementation)
+ deprecated: true
+ items:
+ - enum:
- qcom,sdx55-smmu-500
- qcom,sdx65-smmu-500
+ - const: arm,mmu-500
+
+ - description: Qcom SoCs implementing "arm,mmu-500" (legacy binding)
+ deprecated: true
+ items:
+ # Do not add additional SoC to this list. Instead use two previous lists.
+ - enum:
+ - qcom,qcm2290-smmu-500
+ - qcom,sc7180-smmu-500
+ - qcom,sc7280-smmu-500
+ - qcom,sc8180x-smmu-500
+ - qcom,sc8280xp-smmu-500
+ - qcom,sdm845-smmu-500
+ - qcom,sm6115-smmu-500
- qcom,sm6350-smmu-500
- qcom,sm6375-smmu-500
- qcom,sm8150-smmu-500
- qcom,sm8350-smmu-500
- qcom,sm8450-smmu-500
- const: arm,mmu-500
+
+ - description: Qcom Adreno GPUs implementing "arm,smmu-500"
+ items:
+ - enum:
+ - qcom,sc7280-smmu-500
+ - qcom,sm8250-smmu-500
+ - const: qcom,adreno-smmu
+ - const: arm,mmu-500
- description: Qcom Adreno GPUs implementing "arm,smmu-v2"
items:
- enum:
+ - qcom,msm8996-smmu-v2
- qcom,sc7180-smmu-v2
+ - qcom,sdm630-smmu-v2
- qcom,sdm845-smmu-v2
+ - qcom,sm6350-smmu-v2
- const: qcom,adreno-smmu
- const: qcom,smmu-v2
+ - description: Qcom Adreno GPUs on Google Cheza platform
+ items:
+ - const: qcom,sdm845-smmu-v2
+ - const: qcom,smmu-v2
- description: Marvell SoCs implementing "arm,mmu-500"
items:
- const: marvell,ap806-smmu-500
present in such cases.
clock-names:
- items:
- - const: bus
- - const: iface
+ minItems: 1
+ maxItems: 7
clocks:
- items:
- - description: bus clock required for downstream bus access and for the
- smmu ptw
- - description: interface clock required to access smmu's registers
- through the TCU's programming interface.
+ minItems: 1
+ maxItems: 7
power-domains:
maxItems: 1
reg:
maxItems: 1
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - qcom,msm8998-smmu-v2
+ - qcom,sdm630-smmu-v2
+ then:
+ anyOf:
+ - properties:
+ clock-names:
+ items:
+ - const: bus
+ clocks:
+ items:
+ - description: bus clock required for downstream bus access and for
+ the smmu ptw
+ - properties:
+ clock-names:
+ items:
+ - const: iface
+ - const: mem
+ - const: mem_iface
+ clocks:
+ items:
+ - description: interface clock required to access smmu's registers
+ through the TCU's programming interface.
+ - description: bus clock required for memory access
+ - description: bus clock required for GPU memory access
+ - properties:
+ clock-names:
+ items:
+ - const: iface-mm
+ - const: iface-smmu
+ - const: bus-mm
+ - const: bus-smmu
+ clocks:
+ items:
+ - description: interface clock required to access mnoc's registers
+ through the TCU's programming interface.
+ - description: interface clock required to access smmu's registers
+ through the TCU's programming interface.
+ - description: bus clock required for downstream bus access
+ - description: bus clock required for the smmu ptw
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - qcom,msm8996-smmu-v2
+ - qcom,sc7180-smmu-v2
+ - qcom,sdm845-smmu-v2
+ then:
+ properties:
+ clock-names:
+ items:
+ - const: bus
+ - const: iface
+
+ clocks:
+ items:
+ - description: bus clock required for downstream bus access and for
+ the smmu ptw
+ - description: interface clock required to access smmu's registers
+ through the TCU's programming interface.
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: qcom,sc7280-smmu-500
+ then:
+ properties:
+ clock-names:
+ items:
+ - const: gcc_gpu_memnoc_gfx_clk
+ - const: gcc_gpu_snoc_dvm_gfx_clk
+ - const: gpu_cc_ahb_clk
+ - const: gpu_cc_hlos1_vote_gpu_smmu_clk
+ - const: gpu_cc_cx_gmu_clk
+ - const: gpu_cc_hub_cx_int_clk
+ - const: gpu_cc_hub_aon_clk
+
+ clocks:
+ items:
+ - description: GPU memnoc_gfx clock
+ - description: GPU snoc_dvm_gfx clock
+ - description: GPU ahb clock
+ - description: GPU hlos1_vote_GPU smmu clock
+ - description: GPU cx_gmu clock
+ - description: GPU hub_cx_int clock
+ - description: GPU hub_aon clock
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - qcom,sm6350-smmu-v2
+ - qcom,sm8150-smmu-500
+ - qcom,sm8250-smmu-500
+ then:
+ properties:
+ clock-names:
+ items:
+ - const: ahb
+ - const: bus
+ - const: iface
+
+ clocks:
+ items:
+ - description: bus clock required for AHB bus access
+ - description: bus clock required for downstream bus access and for
+ the smmu ptw
+ - description: interface clock required to access smmu's registers
+ through the TCU's programming interface.
+
examples:
- |+
/* SMMU with stream matching or stream indexing */
- mediatek,mt8195-iommu-vdo # generation two
- mediatek,mt8195-iommu-vpp # generation two
- mediatek,mt8195-iommu-infra # generation two
+ - mediatek,mt8365-m4u # generation two
- description: mt7623 generation one
items:
dt-binding/memory/mt8186-memory-port.h for mt8186,
dt-binding/memory/mt8192-larb-port.h for mt8192.
dt-binding/memory/mt8195-memory-port.h for mt8195.
+ dt-binding/memory/mediatek,mt8365-larb-port.h for mt8365.
power-domains:
maxItems: 1
$id: http://devicetree.org/schemas/ipmi/ipmi-ipmb.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: IPMI IPMB device bindings
+title: IPMI IPMB device
description: IPMI IPMB device bindings
$id: http://devicetree.org/schemas/ipmi/ipmi-smic.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: IPMI device bindings
+title: IPMI device
description: IPMI device bindings
$id: http://devicetree.org/schemas/leds/backlight/gpio-backlight.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: gpio-backlight bindings
+title: gpio-backlight
maintainers:
- Lee Jones <lee@kernel.org>
$id: http://devicetree.org/schemas/leds/backlight/led-backlight.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: led-backlight bindings
+title: led-backlight
maintainers:
- Lee Jones <lee@kernel.org>
$id: http://devicetree.org/schemas/leds/backlight/pwm-backlight.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: pwm-backlight bindings
+title: pwm-backlight
maintainers:
- Lee Jones <lee@kernel.org>
$id: http://devicetree.org/schemas/leds/backlight/qcom-wled.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for Qualcomm Technologies, Inc. WLED driver
+title: Qualcomm Technologies, Inc. WLED driver
maintainers:
- Bjorn Andersson <bjorn.andersson@linaro.org>
- pattern
# LED is triggered by SD/MMC activity
- pattern: "^mmc[0-9]+$"
+ - pattern: "^cpu[0-9]*$"
led-pattern:
description: |
$id: http://devicetree.org/schemas/leds/issi,is31fl319x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ISSI LED controllers bindings for IS31FL319{0,1,3,6,9}
+title: ISSI LED Controllers for IS31FL319{0,1,3,6,9}
maintainers:
- Vincent Knecht <vincent.knecht@mailoo.org>
+++ /dev/null
-Qualcomm PM8058 LED driver
-
-The Qualcomm PM8058 is a multi-functional device which contains
-an LED driver block for up to six LEDs: three normal LEDs, two
-"flash" LEDs and one "keypad backlight" LED. The names are
-quoted because sometimes these LED drivers are used for wildly
-different things than flash or keypad backlight: their names
-are more of a suggestion than a hard-wired usecase.
-
-Hardware-wise the different LEDs support slightly different
-output currents. The "flash" LEDs do not need to charge nor
-do they support external triggers. They are just powerful LED
-drivers.
-
-The LEDs appear as children to the PM8058 device, with the
-proper compatible string. For the PM8058 bindings see:
-mfd/qcom-pm8xxx.txt.
-
-Each LED is represented as a sub-node of the syscon device. Each
-node's name represents the name of the corresponding LED.
-
-LED sub-node properties:
-
-Required properties:
-- compatible: one of
- "qcom,pm8058-led" (for the normal LEDs at 0x131, 0x132 and 0x133)
- "qcom,pm8058-keypad-led" (for the "keypad" LED at 0x48)
- "qcom,pm8058-flash-led" (for the "flash" LEDs at 0x49 and 0xFB)
-
-Optional properties:
-- label: see Documentation/devicetree/bindings/leds/common.txt
-- default-state: see Documentation/devicetree/bindings/leds/common.txt
-- linux,default-trigger: see Documentation/devicetree/bindings/leds/common.txt
-
-Example:
-
-qcom,ssbi@500000 {
- pmicintc: pmic@0 {
- compatible = "qcom,pm8058";
- led@48 {
- compatible = "qcom,pm8058-keypad-led";
- reg = <0x48>;
- label = "pm8050:white:keypad";
- default-state = "off";
- };
- led@131 {
- compatible = "qcom,pm8058-led";
- reg = <0x131>;
- label = "pm8058:red";
- default-state = "off";
- };
- led@132 {
- compatible = "qcom,pm8058-led";
- reg = <0x132>;
- label = "pm8058:yellow";
- default-state = "off";
- linux,default-trigger = "mmc0";
- };
- led@133 {
- compatible = "qcom,pm8058-led";
- reg = <0x133>;
- label = "pm8058:green";
- default-state = "on";
- linux,default-trigger = "heartbeat";
- };
- };
-};
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/leds/qcom,pm8058-led.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm PM8058 PMIC LED
+
+maintainers:
+ - Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
+
+description: |
+ The Qualcomm PM8058 contains an LED block for up to six LEDs:: three normal
+ LEDs, two "flash" LEDs and one "keypad backlight" LED. The names are quoted
+ because sometimes these LED drivers are used for wildly different things than
+ flash or keypad backlight:: their names are more of a suggestion than a
+ hard-wired usecase.
+
+ Hardware-wise the different LEDs support slightly different output currents.
+ The "flash" LEDs do not need to charge nor do they support external triggers.
+ They are just powerful LED drivers.
+
+allOf:
+ - $ref: common.yaml#
+
+properties:
+ compatible:
+ enum:
+ - qcom,pm8058-led
+ - qcom,pm8058-keypad-led
+ - qcom,pm8058-flash-led
+
+ reg:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/leds/common.h>
+
+ pmic {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@131 {
+ compatible = "qcom,pm8058-led";
+ reg = <0x131>;
+ label = "pm8058:red";
+ color = <LED_COLOR_ID_RED>;
+ default-state = "off";
+ };
+ };
$id: http://devicetree.org/schemas/leds/register-bit-led.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Device Tree Bindings for Register Bit LEDs
+title: Register Bit LEDs
maintainers:
- Linus Walleij <linus.walleij@linaro.org>
$id: http://devicetree.org/schemas/leds/regulator-led.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Device Tree Bindings for Regulator LEDs
+title: Regulator LEDs
maintainers:
- Linus Walleij <linus.walleij@linaro.org>
title: ROHM BD71828 Power Management Integrated Circuit LED driver
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
This module is part of the ROHM BD71828 MFD device. For more details
- mediatek,mt8173-gce
- mediatek,mt8183-gce
- mediatek,mt8186-gce
+ - mediatek,mt8188-gce
- mediatek,mt8192-gce
- mediatek,mt8195-gce
$id: "http://devicetree.org/schemas/mailbox/qcom,apcs-kpss-global.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Qualcomm APCS global block bindings
+title: Qualcomm APCS global block
description:
This binding describes the APCS "global" block found in various Qualcomm
- qcom,sc8180x-apss-shared
- qcom,sdm660-apcs-hmss-global
- qcom,sdm845-apss-shared
+ - qcom,sm4250-apcs-hmss-global
- qcom,sm6125-apcs-hmss-global
- qcom,sm6115-apcs-hmss-global
- qcom,sm8150-apss-shared
compatible:
items:
- enum:
+ - qcom,sc7280-ipcc
+ - qcom,sc8280xp-ipcc
- qcom,sm6350-ipcc
- qcom,sm6375-ipcc
- qcom,sm8250-ipcc
- qcom,sm8350-ipcc
- qcom,sm8450-ipcc
- - qcom,sc7280-ipcc
+ - qcom,sm8550-ipcc
- const: qcom,ipcc
reg:
$id: "http://devicetree.org/schemas/mailbox/sprd-mailbox.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Spreadtrum mailbox controller bindings
+title: Spreadtrum mailbox controller
maintainers:
- Orson Zhai <orsonzhai@gmail.com>
$id: "http://devicetree.org/schemas/mailbox/st,stm32-ipcc.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: STMicroelectronics STM32 IPC controller bindings
+title: STMicroelectronics STM32 IPC controller
description:
The IPCC block provides a non blocking signaling mechanism to post and
$id: http://devicetree.org/schemas/media/allwinner,sun6i-a31-isp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Allwinner A31 Image Signal Processor Driver (ISP) Device Tree Bindings
+title: Allwinner A31 Image Signal Processor Driver (ISP)
maintainers:
- Paul Kocialkowski <paul.kocialkowski@bootlin.com>
$id: http://devicetree.org/schemas/media/i2c/dongwoon,dw9768.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Dongwoon Anatech DW9768 Voice Coil Motor (VCM) Lens Device Tree Bindings
+title: Dongwoon Anatech DW9768 Voice Coil Motor (VCM) Lens
maintainers:
- Dongchun Zhu <dongchun.zhu@mediatek.com>
$id: http://devicetree.org/schemas/media/i2c/ov8856.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Omnivision OV8856 CMOS Sensor Device Tree Bindings
+title: Omnivision OV8856 CMOS Sensor
maintainers:
- Dongchun Zhu <dongchun.zhu@mediatek.com>
$id: http://devicetree.org/schemas/media/i2c/ovti,ov02a10.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Omnivision OV02A10 CMOS Sensor Device Tree Bindings
+title: Omnivision OV02A10 CMOS Sensor
maintainers:
- Dongchun Zhu <dongchun.zhu@mediatek.com>
$id: http://devicetree.org/schemas/media/i2c/ovti,ov5640.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: OmniVision OV5640 Image Sensor Device Tree Bindings
+title: OmniVision OV5640 Image Sensor
maintainers:
- Steve Longerbeam <slongerbeam@gmail.com>
$id: http://devicetree.org/schemas/media/i2c/ovti,ov5645.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: OmniVision OV5645 Image Sensor Device Tree Bindings
+title: OmniVision OV5645 Image Sensor
maintainers:
- Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
$id: http://devicetree.org/schemas/media/i2c/ovti,ov5648.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: OmniVision OV5648 Image Sensor Device Tree Bindings
+title: OmniVision OV5648 Image Sensor
maintainers:
- Paul Kocialkowski <paul.kocialkowski@bootlin.com>
$id: http://devicetree.org/schemas/media/i2c/ovti,ov8865.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: OmniVision OV8865 Image Sensor Device Tree Bindings
+title: OmniVision OV8865 Image Sensor
maintainers:
- Paul Kocialkowski <paul.kocialkowski@bootlin.com>
$id: http://devicetree.org/schemas/media/i2c/st,st-vgxy61.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics VGxy61 HDR Global Shutter Sensor Family Device Tree Bindings
+title: STMicroelectronics VGxy61 HDR Global Shutter Sensor Family
maintainers:
- Benjamin Mugnier <benjamin.mugnier@foss.st.com>
$id: http://devicetree.org/schemas/media/marvell,mmp2-ccic.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell MMP2 camera host interface bindings
+title: Marvell MMP2 camera host interface
maintainers:
- Lubomir Rintel <lkundrak@v3.sk>
$id: http://devicetree.org/schemas/media/renesas,ceu.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Renesas Capture Engine Unit (CEU) Bindings
+title: Renesas Capture Engine Unit (CEU)
maintainers:
- Jacopo Mondi <jacopo+renesas@jmondi.org>
$id: http://devicetree.org/schemas/media/st,stm32-cec.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 CEC bindings
+title: STMicroelectronics STM32 CEC
maintainers:
- Yannick Fertre <yannick.fertre@foss.st.com>
$id: http://devicetree.org/schemas/media/st,stm32-dcmi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Digital Camera Memory Interface (DCMI) binding
+title: STMicroelectronics STM32 Digital Camera Memory Interface (DCMI)
maintainers:
- Hugues Fruchet <hugues.fruchet@foss.st.com>
$id: http://devicetree.org/schemas/media/st,stm32-dma2d.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Chrom-Art Accelerator DMA2D binding
+title: STMicroelectronics STM32 Chrom-Art Accelerator DMA2D
description:
Chrom-ART Accelerator(DMA2D), graphical hardware accelerator
$id: http://devicetree.org/schemas/media/video-interface-devices.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Common bindings for video receiver and transmitter devices
+title: Common Properties for Video Receiver and Transmitter Devices
maintainers:
- Jacopo Mondi <jacopo@jmondi.org>
$id: http://devicetree.org/schemas/media/video-interfaces.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Common bindings for video receiver and transmitter interface endpoints
+title: Common Properties for Video Receiver and Transmitter Interface Endpoints
maintainers:
- Sakari Ailus <sakari.ailus@linux.intel.com>
$id: http://devicetree.org/schemas/memory-controllers/calxeda-ddr-ctrlr.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Calxeda DDR memory controller binding
+title: Calxeda DDR memory controller
description: |
The Calxeda DDR memory controller is initialised and programmed by the
$id: http://devicetree.org/schemas/memory-controllers/ingenic,nemc-peripherals.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs NAND / External Memory Controller (NEMC) devicetree bindings
+title: Ingenic SoCs NAND / External Memory Controller (NEMC)
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/memory-controllers/ingenic,nemc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs NAND / External Memory Controller (NEMC) devicetree bindings
+title: Ingenic SoCs NAND / External Memory Controller (NEMC)
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/memory-controllers/st,stm32-fmc2-ebi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics Flexible Memory Controller 2 (FMC2) Bindings
+title: STMicroelectronics Flexible Memory Controller 2 (FMC2)
description: |
The FMC2 functional block makes the interface with: synchronous and
$id: http://devicetree.org/schemas/memory-controllers/ti,gpmc-child.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: device tree bindings for children of the Texas Instruments GPMC
+title: Texas Instruments GPMC Bus Child Nodes
maintainers:
- Tony Lindgren <tony@atomide.com>
$id: http://devicetree.org/schemas/memory-controllers/ti,gpmc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments GPMC Memory Controller device-tree bindings
+title: Texas Instruments GPMC Memory Controller
maintainers:
- Tony Lindgren <tony@atomide.com>
$id: http://devicetree.org/schemas/mfd/actions,atc260x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Actions Semi ATC260x Power Management IC bindings
+title: Actions Semi ATC260x Power Management IC
maintainers:
- Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/mfd/ampere,smpro.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Ampere Altra SMPro firmware driver
+
+maintainers:
+ - Quan Nguyen <quan@os.amperecomputing.com>
+
+description: |
+ Ampere Altra SMPro firmware may contain different blocks like hardware
+ monitoring, error monitoring and other miscellaneous features.
+
+properties:
+ compatible:
+ enum:
+ - ampere,smpro
+
+ reg:
+ description:
+ I2C device address.
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+
+additionalProperties: false
+
+examples:
+ - |
+ i2c {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ smpro@4f {
+ compatible = "ampere,smpro";
+ reg = <0x4f>;
+ };
+ };
const: 1
patternProperties:
+ '^timer@[a-f0-9]+$':
+ $ref: /schemas/timer/brcm,bcmbca-timer.yaml
+
'^watchdog@[a-f0-9]+$':
$ref: /schemas/watchdog/brcm,bcm7038-wdt.yaml
#address-cells = <1>;
#size-cells = <1>;
+ timer@0 {
+ compatible = "brcm,bcm63138-timer";
+ reg = <0x0 0x28>;
+ };
+
watchdog@28 {
compatible = "brcm,bcm7038-wdt";
reg = <0x28 0x8>;
"dlg,da9061" for DA9061
- reg : Specifies the I2C slave address (this defaults to 0x58 but it can be
modified to match the chip's OTP settings).
-- interrupts : IRQ line information.
-- interrupt-controller
-
-See Documentation/devicetree/bindings/interrupt-controller/interrupts.txt for
-further information on IRQ bindings.
Optional properties:
See Documentation/devicetree/bindings/gpio/gpio.txt for further information on
GPIO bindings.
+- interrupts : IRQ line information.
+- interrupt-controller
+
+See Documentation/devicetree/bindings/interrupt-controller/interrupts.txt for
+further information on IRQ bindings.
+
Sub-nodes:
- regulators : This node defines the settings for the LDOs and BUCKs.
- onkey : See ../input/da9062-onkey.txt
-- watchdog: See ../watchdog/da9062-watchdog.txt
+- watchdog: See ../watchdog/da9062-wdt.txt
- thermal : See ../thermal/da9062-thermal.txt
$id: http://devicetree.org/schemas/mfd/ene-kb3930.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ENE KB3930 Embedded Controller bindings
+title: ENE KB3930 Embedded Controller
description: |
This binding describes the ENE KB3930 Embedded Controller attached to an
$id: http://devicetree.org/schemas/mfd/ene-kb930.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ENE KB930 Embedded Controller bindings
+title: ENE KB930 Embedded Controller
description: |
This binding describes the ENE KB930 Embedded Controller attached to an
maintainers:
- Dmitry Osipenko <digetx@gmail.com>
+$ref: /schemas/power/supply/power-supply.yaml
+
properties:
compatible:
items:
reg:
maxItems: 1
- monitored-battery: true
- power-supplies: true
system-power-controller: true
required:
- compatible
- reg
-additionalProperties: false
+unevaluatedProperties: false
examples:
- |
$id: http://devicetree.org/schemas/mfd/fsl,imx8qxp-csr.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Freescale i.MX8qm/qxp Control and Status Registers Module Bindings
+title: Freescale i.MX8qm/qxp Control and Status Registers Module
maintainers:
- Liu Ying <victor.liu@nxp.com>
description: |
The Ocelot ethernet switch family contains chips that have an internal CPU
(VSC7513, VSC7514) and chips that don't (VSC7511, VSC7512). All switches have
- the option to be controlled externally, which is the purpose of this driver.
+ the option to be controlled externally via external interfaces like SPI or
+ PCIe.
The switch family is a multi-port networking switch that supports many
interfaces. Additionally, the device can perform pin control, MDIO buses, and
- reg
- '#address-cells'
- '#size-cells'
- - spi-max-frequency
additionalProperties: false
compatible:
"mediatek,mt6323" for PMIC MT6323
"mediatek,mt6331" for PMIC MT6331 and MT6332
+ "mediatek,mt6357" for PMIC MT6357
"mediatek,mt6358" for PMIC MT6358 and MT6366
"mediatek,mt6359" for PMIC MT6359
"mediatek,mt6397" for PMIC MT6397
$id: http://devicetree.org/schemas/mfd/qcom,pm8008.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Technologies, Inc. PM8008 PMIC bindings
+title: Qualcomm Technologies, Inc. PM8008 PMIC
maintainers:
- Guru Das Srinagesh <gurus@codeaurora.org>
type: object
$ref: /schemas/regulator/qcom,spmi-regulator.yaml#
+ pwm:
+ type: object
+ $ref: /schemas/leds/leds-qcom-lpg.yaml#
+
patternProperties:
"^adc@[0-9a-f]+$":
type: object
- $ref: /schemas/iio/adc/qcom,spmi-vadc.yaml#
+ oneOf:
+ - $ref: /schemas/iio/adc/qcom,spmi-iadc.yaml#
+ - $ref: /schemas/iio/adc/qcom,spmi-vadc.yaml#
"^adc-tm@[0-9a-f]+$":
type: object
type: object
additionalProperties: true # FIXME qcom,pm8916-wcd-analog-codec binding not converted yet
- "extcon@[0-9a-f]+$":
+ "^charger@[0-9a-f]+$":
type: object
- $ref: /schemas/extcon/qcom,pm8941-misc.yaml#
+ oneOf:
+ - $ref: /schemas/power/supply/qcom,pm8941-charger.yaml#
+ - $ref: /schemas/power/supply/qcom,pm8941-coincell.yaml#
- "gpio(s)?@[0-9a-f]+$":
+ "gpio@[0-9a-f]+$":
type: object
$ref: /schemas/pinctrl/qcom,pmic-gpio.yaml#
type: object
$ref: /schemas/power/reset/qcom,pon.yaml#
- "pwm@[0-9a-f]+$":
- type: object
- $ref: /schemas/leds/leds-qcom-lpg.yaml#
-
"^rtc@[0-9a-f]+$":
type: object
$ref: /schemas/rtc/qcom-pm8xxx-rtc.yaml#
type: object
$ref: /schemas/thermal/qcom,spmi-temp-alarm.yaml#
+ "^usb-detect@[0-9a-f]+$":
+ type: object
+ $ref: /schemas/extcon/qcom,pm8941-misc.yaml#
+
+ "^usb-vbus-regulator@[0-9a-f]+$":
+ type: object
+ $ref: /schemas/regulator/qcom,usb-vbus-regulator.yaml#
+
"^vibrator@[0-9a-f]+$":
type: object
- additionalProperties: true # FIXME qcom,pm8916-vib binding not converted yet
+ $ref: /schemas/input/qcom,pm8xxx-vib.yaml#
"^mpps@[0-9a-f]+$":
type: object
#address-cells = <1>;
#size-cells = <0>;
- pmi8998_gpio: gpios@c000 {
+ pmi8998_gpio: gpio@c000 {
compatible = "qcom,pmi8998-gpio", "qcom,spmi-gpio";
reg = <0xc000>;
gpio-controller;
};
};
- pm6150_gpio: gpios@c000 {
+ pm6150_gpio: gpio@c000 {
compatible = "qcom,pm6150-gpio", "qcom,spmi-gpio";
reg = <0xc000>;
gpio-controller;
compatible:
items:
- enum:
+ - qcom,msm8976-tcsr
- qcom,msm8998-tcsr
- qcom,qcs404-tcsr
- qcom,sc7180-tcsr
- qcom,sc7280-tcsr
+ - qcom,sc8280xp-tcsr
- qcom,sdm630-tcsr
- qcom,sdm845-tcsr
- qcom,sm8150-tcsr
properties:
compatible:
- enum:
- - qcom,pm8018
- - qcom,pm8058
- - qcom,pm8821
- - qcom,pm8921
+ oneOf:
+ - enum:
+ - qcom,pm8058
+ - qcom,pm8821
+ - qcom,pm8921
+ - items:
+ - enum:
+ - qcom,pm8018
+ - const: qcom,pm8921
reg:
maxItems: 1
interrupt-controller: true
patternProperties:
+ "led@[0-9a-f]+$":
+ type: object
+ $ref: /schemas/leds/qcom,pm8058-led.yaml#
+
"rtc@[0-9a-f]+$":
type: object
$ref: "../rtc/qcom-pm8xxx-rtc.yaml"
- interrupt-controller
additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/irq.h>
+ ssbi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ pmic@0 {
+ compatible = "qcom,pm8921";
+ reg = <0>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+
+ interrupt-parent = <&tlmm>;
+ interrupts = <32 IRQ_TYPE_EDGE_RISING>;
+ };
+ };
...
The child node for the charger to hold additional properties. If a
battery is not in use, this node can be omitted.
type: object
+ $ref: /schemas/power/supply/power-supply.yaml
+
properties:
monitored-battery:
description: |
$id: http://devicetree.org/schemas/mfd/rohm,bd71815-pmic.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ROHM BD71815 Power Management Integrated Circuit bindings
+title: ROHM BD71815 Power Management Integrated Circuit
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
BD71815AGW is a single-chip power management ICs for battery-powered
$id: http://devicetree.org/schemas/mfd/rohm,bd71828-pmic.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ROHM BD71828 Power Management Integrated Circuit bindings
+title: ROHM BD71828 Power Management Integrated Circuit
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
BD71828GW is a single-chip power management IC for battery-powered portable
$id: http://devicetree.org/schemas/mfd/rohm,bd71837-pmic.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ROHM BD71837 Power Management Integrated Circuit bindings
+title: ROHM BD71837 Power Management Integrated Circuit
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
BD71837MWV is programmable Power Management ICs for powering single-core,
$id: http://devicetree.org/schemas/mfd/rohm,bd71847-pmic.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ROHM BD71847 and BD71850 Power Management Integrated Circuit bindings
+title: ROHM BD71847 and BD71850 Power Management Integrated Circuit
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
BD71847AMWV and BD71850MWV are programmable Power Management ICs for powering
$id: http://devicetree.org/schemas/mfd/rohm,bd9576-pmic.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: ROHM BD9576MUF and BD9573MUF Power Management Integrated Circuit bindings
+title: ROHM BD9576MUF and BD9573MUF Power Management Integrated Circuit
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
BD9576MUF and BD9573MUF are power management ICs primarily intended for
$id: http://devicetree.org/schemas/mfd/st,stm32-lptimer.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Low-Power Timers bindings
+title: STMicroelectronics STM32 Low-Power Timers
description: |
The STM32 Low-Power Timer (LPTIM) is a 16-bit timer that provides several
$id: http://devicetree.org/schemas/mfd/st,stm32-timers.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Timers bindings
+title: STMicroelectronics STM32 Timers
description: |
This hardware block provides 3 types of timer along with PWM functionality:
$id: http://devicetree.org/schemas/mfd/st,stmfx.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectonics Multi-Function eXpander (STMFX) bindings
+title: STMicroelectonics Multi-Function eXpander (STMFX)
description: ST Multi-Function eXpander (STMFX) is a slave controller using I2C for
communication with the main MCU. Its main features are GPIO expansion,
$id: http://devicetree.org/schemas/mfd/st,stpmic1.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectonics STPMIC1 Power Management IC bindings
+title: STMicroelectonics STPMIC1 Power Management IC
description: STMicroelectronics STPMIC1 Power Management IC
- microchip,lan966x-cpu-syscon
- microchip,sparx5-cpu-syscon
- mstar,msc313-pmsleep
+ - nuvoton,wpcm450-shm
- rockchip,px30-qos
- rockchip,rk3036-qos
- rockchip,rk3066-qos
type: object
description: Magnetic reader
+ power-domains:
+ maxItems: 1
+
required:
- compatible
- reg
compatible:
items:
- enum:
+ - ti,j7200-system-controller
- ti,j721e-system-controller
+ - ti,j721s2-system-controller
- const: syscon
- const: simple-mfd
$id: http://devicetree.org/schemas/mips/cpus.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MIPS CPUs bindings
+title: MIPS CPUs
maintainers:
- Thomas Bogendoerfer <tsbogend@alpha.franken.de>
$id: http://devicetree.org/schemas/mips/ingenic/devices.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic XBurst based Platforms Device Tree Bindings
+title: Ingenic XBurst based Platforms
maintainers:
- 周琰杰 (Zhou Yanjie) <zhouyanjie@wanyeetech.com>
$id: http://devicetree.org/schemas/mips/lantiq/lantiq,dma-xway.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Lantiq Xway SoCs DMA Controller DT bindings
+title: Lantiq Xway SoCs DMA Controller
maintainers:
- John Crispin <john@phrozen.org>
$id: http://devicetree.org/schemas/mips/loongson/devices.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Loongson based Platforms Device Tree Bindings
+title: Loongson based Platforms
maintainers:
- Jiaxun Yang <jiaxun.yang@flygoat.com>
$id: http://devicetree.org/schemas/misc/olpc,xo1.75-ec.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: OLPC XO-1.75 Embedded Controller bindings
+title: OLPC XO-1.75 Embedded Controller
description: |
This binding describes the Embedded Controller acting as a SPI bus master
iommus:
minItems: 1
- maxItems: 2
+ maxItems: 3
qcom,nsessions:
$ref: /schemas/types.yaml#/definitions/uint32
$id: "http://devicetree.org/schemas/mmc/arasan,sdhci.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Device Tree Bindings for the Arasan SDHCI Controller
+title: Arasan SDHCI Controller
maintainers:
- Adrian Hunter <adrian.hunter@intel.com>
$id: http://devicetree.org/schemas/mmc/brcm,sdhci-brcmstb.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom BRCMSTB/BMIPS SDHCI Controller binding
+title: Broadcom BRCMSTB/BMIPS SDHCI Controller
maintainers:
- Al Cooper <alcooperx@gmail.com>
$id: http://devicetree.org/schemas/mmc/ingenic,mmc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs MMC Controller DT bindings
+title: Ingenic SoCs MMC Controller
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/mmc/microchip,dw-sparx5-sdhci.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Microchip Sparx5 Mobile Storage Host Controller Binding
+title: Microchip Sparx5 Mobile Storage Host Controller
allOf:
- $ref: "mmc-controller.yaml"
$id: http://devicetree.org/schemas/mmc/mmc-controller.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MMC Controller Generic Binding
+title: MMC Controller Common Properties
maintainers:
- Ulf Hansson <ulf.hansson@linaro.org>
$id: http://devicetree.org/schemas/mmc/mmc-pwrseq-emmc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Simple eMMC hardware reset provider binding
+title: Simple eMMC hardware reset provider
maintainers:
- Ulf Hansson <ulf.hansson@linaro.org>
$id: http://devicetree.org/schemas/mmc/mmc-pwrseq-sd8787.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell SD8787 power sequence provider binding
+title: Marvell SD8787 power sequence provider
maintainers:
- Ulf Hansson <ulf.hansson@linaro.org>
$id: http://devicetree.org/schemas/mmc/mmc-pwrseq-simple.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Simple MMC power sequence provider binding
+title: Simple MMC power sequence provider
maintainers:
- Ulf Hansson <ulf.hansson@linaro.org>
$id: http://devicetree.org/schemas/mmc/mtk-sd.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MTK MSDC Storage Host Controller Binding
+title: MTK MSDC Storage Host Controller
maintainers:
- Chaotian Jing <chaotian.jing@mediatek.com>
$id: http://devicetree.org/schemas/mmc/sdhci-pxa.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell PXA SDHCI v2/v3 bindings
+title: Marvell PXA SDHCI v2/v3
maintainers:
- Ulf Hansson <ulf.hansson@linaro.org>
$id: http://devicetree.org/schemas/mmc/snps,dwcmshc-sdhci.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Synopsys Designware Mobile Storage Host Controller Binding
+title: Synopsys Designware Mobile Storage Host Controller
maintainers:
- Ulf Hansson <ulf.hansson@linaro.org>
$id: http://devicetree.org/schemas/mmc/synopsys-dw-mshc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Synopsys Designware Mobile Storage Host Controller Binding
+title: Synopsys Designware Mobile Storage Host Controller
maintainers:
- Ulf Hansson <ulf.hansson@linaro.org>
$id: http://devicetree.org/schemas/mtd/gpmi-nand.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Freescale General-Purpose Media Interface (GPMI) binding
+title: Freescale General-Purpose Media Interface (GPMI)
maintainers:
- Han Xu <han.xu@nxp.com>
$id: http://devicetree.org/schemas/mtd/ingenic,nand.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs NAND controller devicetree bindings
+title: Ingenic SoCs NAND controller
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: "http://devicetree.org/schemas/mtd/microchip,mchp48l640.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Microchip 48l640 (and similar) serial EERAM bindings
+title: Microchip 48l640 (and similar) serial EERAM
maintainers:
- Heiko Schocher <hs@denx.de>
$id: http://devicetree.org/schemas/mtd/mxc-nand.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Freescale's mxc_nand binding
+title: Freescale's mxc_nand
maintainers:
- Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
$id: http://devicetree.org/schemas/mtd/nand-chip.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NAND Chip and NAND Controller Generic Binding
+title: NAND Chip Common Properties
maintainers:
- Miquel Raynal <miquel.raynal@bootlin.com>
$id: http://devicetree.org/schemas/mtd/nand-controller.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NAND Chip and NAND Controller Generic Binding
+title: NAND Controller Common Properties
maintainers:
- Miquel Raynal <miquel.raynal@bootlin.com>
$id: http://devicetree.org/schemas/mtd/partitions/qcom,smem-part.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm SMEM NAND flash partition parser binding
+title: Qualcomm SMEM NAND flash partition parser
maintainers:
- Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
$id: http://devicetree.org/schemas/mtd/st,stm32-fmc2-nand.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics Flexible Memory Controller 2 (FMC2) Bindings
+title: STMicroelectronics Flexible Memory Controller 2 (FMC2)
maintainers:
- Christophe Kerello <christophe.kerello@foss.st.com>
$id: http://devicetree.org/schemas/mux/gpio-mux.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: GPIO-based multiplexer controller bindings
+title: GPIO-based multiplexer controller
maintainers:
- Peter Rosin <peda@axentia.se>
$id: http://devicetree.org/schemas/mux/mux-consumer.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Common multiplexer controller consumer bindings
+title: Common multiplexer controller consumer
maintainers:
- Peter Rosin <peda@axentia.se>
$id: http://devicetree.org/schemas/mux/mux-controller.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Common multiplexer controller provider bindings
+title: Common multiplexer controller provider
maintainers:
- Peter Rosin <peda@axentia.se>
$id: http://devicetree.org/schemas/mux/reg-mux.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Generic register bitfield-based multiplexer controller bindings
+title: Generic register bitfield-based multiplexer controller
maintainers:
- Peter Rosin <peda@axentia.se>
$id: http://devicetree.org/schemas/net/asix,ax88178.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: The device tree bindings for the USB Ethernet controllers
+title: ASIX AX88172/AX88772 USB Ethernet Controllers
maintainers:
- Oleksij Rempel <o.rempel@pengutronix.de>
$id: http://devicetree.org/schemas/net/bluetooth/bluetooth-controller.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bluetooth Controller Generic Binding
+title: Bluetooth Controller Common Properties
maintainers:
- Marcel Holtmann <marcel@holtmann.org>
$id: http://devicetree.org/schemas/net/brcm,bcmgenet.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom BCM7xxx Ethernet Controller (GENET) binding
+title: Broadcom BCM7xxx Ethernet Controller (GENET)
maintainers:
- Doug Berger <opendmb@gmail.com>
$id: http://devicetree.org/schemas/net/can/allwinner,sun4i-a10-can.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Allwinner A10 CAN Controller Device Tree Bindings
+title: Allwinner A10 CAN Controller
maintainers:
- Chen-Yu Tsai <wens@csie.org>
$id: http://devicetree.org/schemas/net/can/bosch,c_can.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bosch C_CAN/D_CAN controller Device Tree Bindings
+title: Bosch C_CAN/D_CAN controller
description: Bosch C_CAN/D_CAN controller for CAN bus
$id: http://devicetree.org/schemas/net/can/bosch,m_can.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bosch MCAN controller Bindings
+title: Bosch MCAN controller
description: Bosch MCAN controller for CAN bus
$id: http://devicetree.org/schemas/net/can/can-controller.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: CAN Controller Generic Binding
+title: CAN Controller Common Properties
maintainers:
- Marc Kleine-Budde <mkl@pengutronix.de>
$id: http://devicetree.org/schemas/net/can/can-transceiver.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: CAN transceiver Bindings
+title: CAN transceiver
description: CAN transceiver generic properties bindings
$id: http://devicetree.org/schemas/net/can/ctu,ctucanfd.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: CTU CAN FD Open-source IP Core Device Tree Bindings
+title: CTU CAN FD Open-source IP Core
description: |
Open-source CAN FD IP core developed at the Czech Technical University in Prague
$id: http://devicetree.org/schemas/net/can/microchip,mcp251xfd.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title:
- Microchip MCP2517FD, MCP2518FD and MCP251863 stand-alone CAN
- controller device tree bindings
+title: Microchip MCP2517FD, MCP2518FD and MCP251863 stand-alone CAN controller
maintainers:
- Marc Kleine-Budde <mkl@pengutronix.de>
$id: http://devicetree.org/schemas/net/dsa/arrow,xrs700x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Arrow SpeedChips XRS7000 Series Switch Device Tree Bindings
+title: Arrow SpeedChips XRS7000 Series Switch
allOf:
- $ref: dsa.yaml#
$id: http://devicetree.org/schemas/net/dsa/dsa-port.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ethernet Switch port Device Tree Bindings
+title: Ethernet Switch port
maintainers:
- Andrew Lunn <andrew@lunn.ch>
$id: http://devicetree.org/schemas/net/dsa/dsa.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ethernet Switch Device Tree Bindings
+title: Ethernet Switch
maintainers:
- Andrew Lunn <andrew@lunn.ch>
$id: http://devicetree.org/schemas/net/dsa/hirschmann,hellcreek.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Hirschmann Hellcreek TSN Switch Device Tree Bindings
+title: Hirschmann Hellcreek TSN Switch
allOf:
- $ref: dsa.yaml#
$id: http://devicetree.org/schemas/net/dsa/microchip,lan937x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: LAN937x Ethernet Switch Series Tree Bindings
+title: LAN937x Ethernet Switch Series
maintainers:
- UNGLinuxDriver@microchip.com
$id: http://devicetree.org/schemas/net/dsa/mscc,ocelot.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Microchip Ocelot Switch Family Device Tree Bindings
+title: Microchip Ocelot Switch Family
maintainers:
- Vladimir Oltean <vladimir.oltean@nxp.com>
$id: http://devicetree.org/schemas/net/dsa/nxp,sja1105.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP SJA1105 Automotive Ethernet Switch Family Device Tree Bindings
+title: NXP SJA1105 Automotive Ethernet Switch Family
description:
The SJA1105 SPI interface requires a CS-to-CLK time (t2 in UM10944.pdf) of at
$id: http://devicetree.org/schemas/net/engleder,tsnep.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: TSN endpoint Ethernet MAC binding
+title: TSN endpoint Ethernet MAC
maintainers:
- Gerhard Engleder <gerhard@engleder-embedded.com>
$id: http://devicetree.org/schemas/net/ethernet-controller.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ethernet Controller Generic Binding
+title: Ethernet Controller Common Properties
maintainers:
- David S. Miller <davem@davemloft.net>
$id: http://devicetree.org/schemas/net/ethernet-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ethernet PHY Generic Binding
+title: Ethernet PHY Common Properties
maintainers:
- Andrew Lunn <andrew@lunn.ch>
$id: http://devicetree.org/schemas/net/fsl,qoriq-mc-dpmac.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: DPAA2 MAC bindings
+title: DPAA2 MAC
maintainers:
- Ioana Ciornei <ioana.ciornei@nxp.com>
$id: http://devicetree.org/schemas/net/ingenic,mac.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for MAC in Ingenic SoCs
+title: MAC in Ingenic SoCs
maintainers:
- 周琰杰 (Zhou Yanjie) <zhouyanjie@wanyeetech.com>
$id: http://devicetree.org/schemas/net/mctp-i2c-controller.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MCTP I2C transport binding
+title: MCTP I2C transport
maintainers:
- Matt Johnston <matt@codeconstruct.com.au>
$id: http://devicetree.org/schemas/net/mdio.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MDIO Bus Generic Binding
+title: MDIO Bus Common Properties
maintainers:
- Andrew Lunn <andrew@lunn.ch>
$id: http://devicetree.org/schemas/net/microchip,lan95xx.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: The device tree bindings for the USB Ethernet controllers
+title: Microchip SMSC9500/LAN9530/LAN9730 USB Ethernet Controllers
maintainers:
- Oleksij Rempel <o.rempel@pengutronix.de>
$id: http://devicetree.org/schemas/net/wireless/esp,esp8089.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Espressif ESP8089 Device Tree Bindings
+title: Espressif ESP8089
maintainers:
- Hans de Goede <hdegoede@redhat.com>
$id: http://devicetree.org/schemas/net/wireless/ieee80211.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Common IEEE 802.11 Binding
+title: Common IEEE 802.11
maintainers:
- Lorenzo Bianconi <lorenzo@kernel.org>
$id: http://devicetree.org/schemas/net/wireless/mediatek,mt76.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MediaTek mt76 wireless devices Generic Binding
+title: MediaTek mt76 wireless devices
maintainers:
- Felix Fietkau <nbd@nbd.name>
$id: http://devicetree.org/schemas/net/wireless/microchip,wilc1000.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Microchip WILC wireless devicetree bindings
+title: Microchip WILC wireless
maintainers:
- Adham Abozaeid <adham.abozaeid@microchip.com>
$id: http://devicetree.org/schemas/net/wireless/qca,ath9k.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Atheros ath9k wireless devices Generic Binding
+title: Qualcomm Atheros ath9k wireless devices
maintainers:
- Toke Høiland-Jørgensen <toke@toke.dk>
$id: http://devicetree.org/schemas/net/wireless/qcom,ath11k.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Technologies ath11k wireless devices Generic Binding
+title: Qualcomm Technologies ath11k wireless devices
maintainers:
- Kalle Valo <kvalo@kernel.org>
$id: http://devicetree.org/schemas/net/wireless/silabs,wfx.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Silicon Labs WFxxx devicetree bindings
+title: Silicon Labs WFxxx
maintainers:
- Jérôme Pouiller <jerome.pouiller@silabs.com>
$id: http://devicetree.org/schemas/nvmem/fsl,scu-ocotp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: i.MX SCU Client Device Node - OCOTP bindings based on SCU Message Protocol
+title: i.MX SCU Client Device Node - OCOTP Based on SCU Message Protocol
maintainers:
- Dong Aisheng <aisheng.dong@nxp.com>
$id: http://devicetree.org/schemas/nvmem/ingenic,jz4780-efuse.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic JZ EFUSE driver bindings
+title: Ingenic JZ EFUSE driver
maintainers:
- PrasannaKumar Muralidharan <prasannatsmkumar@gmail.com>
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/nvmem/layouts/kontron,sl28-vpd.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVMEM layout of the Kontron SMARC-sAL28 vital product data
+
+maintainers:
+ - Michael Walle <michael@walle.cc>
+
+description:
+ The vital product data (VPD) of the sl28 boards contains a serial
+ number and a base MAC address. The actual MAC addresses for the
+ on-board ethernet devices are derived from this base MAC address by
+ adding an offset.
+
+select: false
+
+properties:
+ compatible:
+ const: kontron,sl28-vpd
+
+ serial-number:
+ type: object
+ description: The board's serial number
+
+ additionalProperties: false
+
+ base-mac-address:
+ type: object
+ description:
+ Base MAC address for all on-module network interfaces. The first
+ argument of the phandle will be treated as an offset.
+
+ properties:
+ "#nvmem-cell-cells":
+ const: 1
+
+ additionalProperties: false
+
+required:
+ - compatible
+
+additionalProperties: false
+
+examples:
+ - |
+ otp-1 {
+ compatible = "user-otp";
+
+ nvmem-layout {
+ compatible = "kontron,sl28-vpd";
+
+ serial_number: serial-number {
+ };
+
+ base_mac_address: base-mac-address {
+ #nvmem-cell-cells = <1>;
+ };
+ };
+ };
+
+...
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/nvmem/layouts/nvmem-layout.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVMEM (Non Volatile Memory) layouts
+
+maintainers:
+ - Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
+ - Michael Walle <michael@walle.cc>
+ - Miquel Raynal <miquel.raynal@bootlin.com>
+
+description: |
+ Most NVMEM layouts are static and thus do not require additional description
+ besides the bytes/bits offset and length. Other layouts can be less statically
+ define and might require dynamic reading of the NVMEM device in order to
+ perform their parsing. The nvmem-layout container is here to describe these.
+
+oneOf:
+ - $ref: kontron,sl28-vpd.yaml
+ - $ref: onie,tlv-layout.yaml
+
+properties:
+ compatible: true
+
+ '#address-cells': false
+
+ '#size-cells': false
+
+required:
+ - compatible
+
+unevaluatedProperties: false
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/nvmem/layouts/onie,tlv-layout.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NVMEM layout of the ONIE tlv table
+
+maintainers:
+ - Miquel Raynal <miquel.raynal@bootlin.com>
+
+description:
+ Modern networking hardware implementing the Open Compute Project ONIE
+ infrastructure shall provide a non-volatile memory with a table whose the
+ content is well specified and gives many information about the manufacturer
+ (name, country of manufacture, etc) as well as device caracteristics (serial
+ number, hardware version, mac addresses, etc). The underlaying device type
+ (flash, EEPROM,...) is not specified. The exact location of each value is also
+ dynamic and should be discovered at run time because it depends on the
+ parameters the manufacturer decided to embed.
+
+select: false
+
+properties:
+ compatible:
+ const: onie,tlv-layout
+
+ product-name:
+ type: object
+ additionalProperties: false
+
+ part-number:
+ type: object
+ additionalProperties: false
+
+ serial-number:
+ type: object
+ additionalProperties: false
+
+ mac-address:
+ type: object
+ description:
+ Base MAC address for all on-module network interfaces. The first
+ argument of the phandle will be treated as an offset.
+
+ properties:
+ "#nvmem-cell-cells":
+ const: 1
+
+ additionalProperties: false
+
+ manufacture-date:
+ type: object
+ additionalProperties: false
+
+ device-version:
+ type: object
+ additionalProperties: false
+
+ label-revision:
+ type: object
+ additionalProperties: false
+
+ platforn-name:
+ type: object
+ additionalProperties: false
+
+ onie-version:
+ type: object
+ additionalProperties: false
+
+ num-macs:
+ type: object
+ additionalProperties: false
+
+ manufacturer:
+ type: object
+ additionalProperties: false
+
+ country-code:
+ type: object
+ additionalProperties: false
+
+ vendor:
+ type: object
+ additionalProperties: false
+
+ diag-version:
+ type: object
+ additionalProperties: false
+
+ service-tag:
+ type: object
+ additionalProperties: false
+
+ vendor-extension:
+ type: object
+ additionalProperties: false
+
+required:
+ - compatible
+
+additionalProperties: false
+
+examples:
+ - |
+ spi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ eeprom@56 {
+ compatible = "atmel,24c64";
+ read-only;
+ reg = <0x56>;
+
+ nvmem-layout {
+ compatible = "onie,tlv-layout";
+
+ serial-number {
+ };
+ };
+ };
+ };
+
+ - |
+ spi {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ flash@0 {
+ compatible = "m25p80", "jedec,spi-nor";
+ reg = <0>;
+
+ otp {
+ compatible = "user-otp";
+
+ nvmem-layout {
+ compatible = "onie,tlv-layout";
+
+ mac-address {
+ #nvmem-cell-cells = <1>;
+ };
+ };
+ };
+ };
+ };
+...
when it's driven low (logical '0') to allow writing.
maxItems: 1
+ nvmem-layout:
+ $ref: /schemas/nvmem/layouts/nvmem-layout.yaml
+ description:
+ Alternative to the statically defined nvmem cells, this
+ container may reference more advanced (dynamic) layout
+ parsers.
+
patternProperties:
"@[0-9a-f]+(,[0-7])?$":
type: object
#include <dt-bindings/gpio/gpio.h>
qfprom: eeprom@700000 {
+ compatible = "qcom,msm8974-qfprom", "qcom,qfprom";
#address-cells = <1>;
#size-cells = <1>;
reg = <0x00700000 0x100000>;
$id: http://devicetree.org/schemas/nvmem/qcom,qfprom.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Technologies Inc, QFPROM Efuse bindings
+title: Qualcomm Technologies Inc, QFPROM Efuse
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/nvmem/qcom,spmi-sdam.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Technologies, Inc. SPMI SDAM DT bindings
+title: Qualcomm Technologies, Inc. SPMI SDAM
maintainers:
- Shyam Kumar Thella <sthella@codeaurora.org>
$id: http://devicetree.org/schemas/nvmem/socionext,uniphier-efuse.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Socionext UniPhier eFuse bindings
+title: Socionext UniPhier eFuse
maintainers:
- Keiji Hayashibara <hayashibara.keiji@socionext.com>
$id: http://devicetree.org/schemas/nvmem/st,stm32-romem.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Factory-programmed data bindings
+title: STMicroelectronics STM32 Factory-programmed data
description: |
This represents STM32 Factory-programmed read only non-volatile area: locked
compatible:
enum:
- st,stm32f4-otp
+ - st,stm32mp13-bsec
- st,stm32mp15-bsec
reg:
$id: http://devicetree.org/schemas/opp/opp-v1.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Generic OPP (Operating Performance Points) v1 Bindings
+title: Generic OPP (Operating Performance Points) v1
maintainers:
- Viresh Kumar <viresh.kumar@linaro.org>
$id: http://devicetree.org/schemas/opp/opp-v2-base.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Generic OPP (Operating Performance Points) Common Binding
+title: Generic OPP (Operating Performance Points) Common Properties
maintainers:
- Viresh Kumar <viresh.kumar@linaro.org>
$id: http://devicetree.org/schemas/opp/opp-v2-kryo-cpu.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Technologies, Inc. NVMEM OPP bindings
+title: Qualcomm Technologies, Inc. NVMEM OPP
maintainers:
- Ilia Lin <ilia.lin@kernel.org>
$id: http://devicetree.org/schemas/opp/opp-v2-qcom-level.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm OPP bindings to describe OPP nodes.
+title: Qualcomm OPP
maintainers:
- Niklas Cassel <nks@flawful.org>
$id: http://devicetree.org/schemas/opp/opp-v2.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Generic OPP (Operating Performance Points) Bindings
+title: Generic OPP (Operating Performance Points)
maintainers:
- Viresh Kumar <viresh.kumar@linaro.org>
items:
- const: pcie
- const: pcie_bus
- - const: pcie_phy
+ - enum: [ pcie_phy, pcie_aux ]
- enum: [ pcie_inbound_axi, pcie_aux ]
num-lanes:
required properties for imx7d-pcie and imx8mq-pcie.
power-domains:
+ minItems: 1
items:
- description: The phandle pointing to the DISPLAY domain for
imx6sx-pcie, to PCIE_PHY power domain for imx7d-pcie and
for imx6sx-pcie.
power-domain-names:
+ minItems: 1
items:
- const: pcie
- const: pcie_phy
resets:
+ minItems: 2
maxItems: 3
description: Phandles to PCIe-related reset lines exposed by SRC
IP block. Additional required by imx7d-pcie and imx8mq-pcie.
reset-names:
- items:
- - const: pciephy
- - const: apps
- - const: turnoff
+ minItems: 2
+ maxItems: 3
fsl,tx-deemph-gen1:
description: Gen1 De-emphasis value (optional required).
items:
- {}
- {}
- - {}
+ - const: pcie_phy
- const: pcie_inbound_axi
+ power-domains:
+ minItems: 2
+ power-domain-names:
+ minItems: 2
- if:
properties:
compatible:
items:
- {}
- {}
- - {}
+ - const: pcie_phy
- const: pcie_aux
- if:
properties:
- fsl,imx8mq-pcie
then:
properties:
+ clocks:
+ maxItems: 3
+ clock-names:
+ maxItems: 3
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - fsl,imx6q-pcie
+ - fsl,imx6qp-pcie
+ - fsl,imx7d-pcie
+ then:
+ properties:
+ clock-names:
+ maxItems: 3
+ contains:
+ const: pcie_phy
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - fsl,imx8mm-pcie
+ - fsl,imx8mp-pcie
+ then:
+ properties:
clock-names:
maxItems: 3
+ contains:
+ const: pcie_aux
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - fsl,imx6q-pcie
+ - fsl,imx6qp-pcie
+ then:
+ properties:
+ power-domains: false
+ power-domain-names: false
+
+ - if:
+ not:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - fsl,imx6sx-pcie
+ - fsl,imx6q-pcie
+ - fsl,imx6qp-pcie
+ then:
+ properties:
+ power-domains:
+ maxItems: 1
+ power-domain-names: false
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - fsl,imx6q-pcie
+ - fsl,imx6sx-pcie
+ - fsl,imx6qp-pcie
+ - fsl,imx7d-pcie
+ - fsl,imx8mq-pcie
+ then:
+ properties:
+ resets:
+ minItems: 3
+ reset-names:
+ items:
+ - const: pciephy
+ - const: apps
+ - const: turnoff
+ else:
+ properties:
+ resets:
+ maxItems: 2
+ reset-names:
+ items:
+ - const: apps
+ - const: turnoff
unevaluatedProperties: false
$id: http://devicetree.org/schemas/pci/pci-ep.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: PCI Endpoint Controller Schema
+title: PCI Endpoint Controller
description: |
Common properties for PCI Endpoint Controller Nodes.
$id: http://devicetree.org/schemas/pci/qcom,pcie-ep.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm PCIe Endpoint Controller binding
+title: Qualcomm PCIe Endpoint Controller
maintainers:
- Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
compatible:
oneOf:
- const: allwinner,sun6i-a31-mipi-dphy
+ - const: allwinner,sun50i-a100-mipi-dphy
- items:
- const: allwinner,sun50i-a64-mipi-dphy
- const: allwinner,sun6i-a31-mipi-dphy
+ - items:
+ - const: allwinner,sun20i-d1-mipi-dphy
+ - const: allwinner,sun50i-a100-mipi-dphy
reg:
maxItems: 1
+ interrupts:
+ maxItems: 1
+
clocks:
items:
- description: Bus Clock
- "#phy-cells"
- compatible
- reg
+ - interrupts
- clocks
- clock-names
- resets
examples:
- |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
dphy0: d-phy@1ca1000 {
compatible = "allwinner,sun6i-a31-mipi-dphy";
reg = <0x01ca1000 0x1000>;
+ interrupts = <GIC_SPI 89 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&ccu 23>, <&ccu 97>;
clock-names = "bus", "mod";
resets = <&ccu 4>;
- const: pmu3
clocks:
+ minItems: 4
items:
- description: USB OTG PHY bus clock
- description: USB Host 0 PHY bus clock
- description: USB Host 1 PHY bus clock
- description: USB Host 2 PHY bus clock
+ - description: PMU clock for host port 2
clock-names:
+ minItems: 4
items:
- const: usb0_phy
- const: usb1_phy
- const: usb2_phy
- const: usb3_phy
+ - const: pmu2_clk
resets:
items:
- resets
- reset-names
+allOf:
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - allwinner,sun50i-h616-usb-phy
+ then:
+ properties:
+ clocks:
+ minItems: 5
+
+ clock-names:
+ minItems: 5
+ else:
+ properties:
+ clocks:
+ maxItems: 4
+
+ clock-names:
+ maxItems: 4
+
additionalProperties: false
examples:
$id: http://devicetree.org/schemas/phy/brcm,ns2-pcie-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom NS2 PCIe PHY binding document
+title: Broadcom NS2 PCIe PHY
maintainers:
- Ray Jui <ray.jui@broadcom.com>
$id: http://devicetree.org/schemas/phy/calxeda-combophy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Calxeda Highbank Combination PHYs binding for SATA
+title: Calxeda Highbank Combination PHYs for SATA
description: |
The Calxeda Combination PHYs connect the SoC to the internal fabric
compatible:
enum:
- fsl,imx8mm-pcie-phy
+ - fsl,imx8mp-pcie-phy
reg:
maxItems: 1
- const: ref
resets:
- maxItems: 1
+ minItems: 1
+ maxItems: 2
reset-names:
- items:
- - const: pciephy
+ oneOf:
+ - items: # for iMX8MM
+ - const: pciephy
+ - items: # for IMX8MP
+ - const: pciephy
+ - const: perst
fsl,refclk-pad-mode:
description: |
description: A boolean property indicating the CLKREQ# signal is
not supported in the board design (optional)
+ power-domains:
+ description: PCIe PHY power domain (optional).
+ maxItems: 1
+
required:
- "#phy-cells"
- compatible
$id: http://devicetree.org/schemas/phy/fsl,imx8mq-usb-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Freescale i.MX8MQ USB3 PHY binding
+title: Freescale i.MX8MQ USB3 PHY
maintainers:
- Li Jun <jun.li@nxp.com>
items:
- const: phy
+ power-domains:
+ maxItems: 1
+
vbus-supply:
description:
A phandle to the regulator for USB VBUS.
$id: http://devicetree.org/schemas/phy/fsl,lynx-28g.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Freescale Lynx 28G SerDes PHY binding
+title: Freescale Lynx 28G SerDes PHY
maintainers:
- Ioana Ciornei <ioana.ciornei@nxp.com>
$id: http://devicetree.org/schemas/phy/ingenic,phy-usb.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs USB PHY devicetree bindings
+title: Ingenic SoCs USB PHY
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/phy/intel,keembay-phy-usb.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Intel Keem Bay USB PHY bindings
+title: Intel Keem Bay USB PHY
maintainers:
- Wan Ahmad Zainie <wan.ahmad.zainie.wan.mohamad@intel.com>
$id: http://devicetree.org/schemas/phy/intel,phy-thunderbay-emmc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Intel Thunder Bay eMMC PHY bindings
+title: Intel Thunder Bay eMMC PHY
maintainers:
- Srikandan Nandhini <nandhini.srikandan@intel.com>
$id: http://devicetree.org/schemas/phy/marvell,mmp3-usb-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell MMP3 USB PHY bindings
+title: Marvell MMP3 USB PHY
maintainers:
- Lubomir Rintel <lkundrak@v3.sk>
$id: http://devicetree.org/schemas/phy/mediatek,dsi-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MediaTek MIPI Display Serial Interface (DSI) PHY binding
+title: MediaTek MIPI Display Serial Interface (DSI) PHY
maintainers:
- Chun-Kuang Hu <chunkuang.hu@kernel.org>
$id: http://devicetree.org/schemas/phy/mediatek,hdmi-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MediaTek High Definition Multimedia Interface (HDMI) PHY binding
+title: MediaTek High Definition Multimedia Interface (HDMI) PHY
maintainers:
- Chun-Kuang Hu <chunkuang.hu@kernel.org>
$id: http://devicetree.org/schemas/phy/mediatek,ufs-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MediaTek Universal Flash Storage (UFS) M-PHY binding
+title: MediaTek Universal Flash Storage (UFS) M-PHY
maintainers:
- Stanley Chu <stanley.chu@mediatek.com>
$id: "http://devicetree.org/schemas/phy/phy-cadence-sierra.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Cadence Sierra PHY binding
+title: Cadence Sierra PHY
description:
This binding describes the Cadence Sierra PHY. Sierra PHY supports multilink
$id: "http://devicetree.org/schemas/phy/phy-cadence-torrent.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Cadence Torrent SD0801 PHY binding
+title: Cadence Torrent SD0801 PHY
description:
This binding describes the Cadence SD0801 PHY (also known as Torrent PHY)
$id: http://devicetree.org/schemas/phy/phy-stm32-usbphyc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 USB HS PHY controller binding
+title: STMicroelectronics STM32 USB HS PHY controller
description:
$id: "http://devicetree.org/schemas/phy/phy-tegra194-p2u.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: NVIDIA Tegra194 & Tegra234 P2U binding
+title: NVIDIA Tegra194 & Tegra234 P2U
maintainers:
- Thierry Reding <treding@nvidia.com>
# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
%YAML 1.2
---
-$id: http://devicetree.org/schemas/phy/qcom,qmp-pcie-phy.yaml#
+$id: http://devicetree.org/schemas/phy/qcom,ipq8074-qmp-pcie-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm QMP PHY controller (PCIe)
+title: Qualcomm QMP PHY controller (PCIe, IPQ8074)
maintainers:
- Vinod Koul <vkoul@kernel.org>
QMP PHY controller supports physical layer functionality for a number of
controllers on Qualcomm chipsets, such as, PCIe, UFS, and USB.
+ Note that these bindings are for SoCs up to SC8180X. For newer SoCs, see
+ qcom,sc8280xp-qmp-pcie-phy.yaml.
+
properties:
compatible:
enum:
# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
%YAML 1.2
---
-$id: http://devicetree.org/schemas/phy/qcom,qmp-ufs-phy.yaml#
+$id: http://devicetree.org/schemas/phy/qcom,msm8996-qmp-ufs-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm QMP PHY controller (UFS)
+title: Qualcomm QMP PHY controller (UFS, MSM8996)
maintainers:
- Vinod Koul <vkoul@kernel.org>
QMP PHY controller supports physical layer functionality for a number of
controllers on Qualcomm chipsets, such as, PCIe, UFS, and USB.
+ Note that these bindings are for SoCs up to SC8180X. For newer SoCs, see
+ qcom,sc8280xp-qmp-ufs-phy.yaml.
+
properties:
compatible:
enum:
- qcom,msm8996-qmp-ufs-phy
- qcom,msm8998-qmp-ufs-phy
- qcom,sc8180x-qmp-ufs-phy
- - qcom,sc8280xp-qmp-ufs-phy
- qcom,sdm845-qmp-ufs-phy
- qcom,sm6115-qmp-ufs-phy
- qcom,sm6350-qmp-ufs-phy
enum:
- qcom,msm8998-qmp-ufs-phy
- qcom,sc8180x-qmp-ufs-phy
- - qcom,sc8280xp-qmp-ufs-phy
- qcom,sdm845-qmp-ufs-phy
- qcom,sm6115-qmp-ufs-phy
- qcom,sm6350-qmp-ufs-phy
contains:
enum:
- qcom,msm8998-qmp-ufs-phy
- - qcom,sc8280xp-qmp-ufs-phy
- qcom,sdm845-qmp-ufs-phy
- qcom,sm6350-qmp-ufs-phy
- qcom,sm8150-qmp-ufs-phy
examples:
- |
- #include <dt-bindings/clock/qcom,gcc-sc8280xp.h>
+ #include <dt-bindings/clock/qcom,gcc-sm8250.h>
#include <dt-bindings/clock/qcom,rpmh.h>
+
phy-wrapper@1d87000 {
- compatible = "qcom,sc8280xp-qmp-ufs-phy";
- reg = <0x01d87000 0xe10>;
+ compatible = "qcom,sm8250-qmp-ufs-phy";
+ reg = <0x01d87000 0x1c0>;
#address-cells = <1>;
#size-cells = <1>;
ranges = <0x0 0x01d87000 0x1000>;
# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
%YAML 1.2
---
-$id: http://devicetree.org/schemas/phy/qcom,qmp-usb-phy.yaml#
+$id: http://devicetree.org/schemas/phy/qcom,msm8996-qmp-usb3-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm QMP PHY controller (USB)
+title: Qualcomm QMP PHY controller (USB, MSM8996)
maintainers:
- Vinod Koul <vkoul@kernel.org>
QMP PHY controller supports physical layer functionality for a number of
controllers on Qualcomm chipsets, such as, PCIe, UFS, and USB.
+ Note that these bindings are for SoCs up to SC8180X. For newer SoCs, see
+ qcom,sc8280xp-qmp-usb3-uni-phy.yaml.
+
properties:
compatible:
enum:
- qcom,qcm2290-qmp-usb3-phy
- qcom,sc7180-qmp-usb3-phy
- qcom,sc8180x-qmp-usb3-phy
- - qcom,sc8280xp-qmp-usb3-uni-phy
- qcom,sdm845-qmp-usb3-phy
- qcom,sdm845-qmp-usb3-uni-phy
- qcom,sdx55-qmp-usb3-uni-phy
compatible:
contains:
enum:
- - qcom,sc8280xp-qmp-usb3-uni-phy
- qcom,sm8150-qmp-usb3-phy
- qcom,sm8150-qmp-usb3-uni-phy
- qcom,sm8250-qmp-usb3-uni-phy
compatible:
contains:
enum:
- - qcom,sc8280xp-qmp-usb3-uni-phy
- then:
- required:
- - power-domains
-
- - if:
- properties:
- compatible:
- contains:
- enum:
- qcom,sdm845-qmp-usb3-phy
- qcom,sm8150-qmp-usb3-phy
- qcom,sm8350-qmp-usb3-phy
contains:
enum:
- qcom,msm8996-qmp-usb3-phy
- - qcom,sc8280xp-qmp-usb3-uni-phy
- qcom,sm8250-qmp-usb3-uni-phy
- qcom,sm8350-qmp-usb3-uni-phy
then:
%YAML 1.2
---
-$id: "http://devicetree.org/schemas/phy/qcom,qmp-usb3-dp-phy.yaml#"
-$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+$id: http://devicetree.org/schemas/phy/qcom,sc7180-qmp-usb3-dp-phy.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm QMP USB3 DP PHY controller
+title: Qualcomm QMP USB3 DP PHY controller (SC7180)
+
+description:
+ The QMP PHY controller supports physical layer functionality for a number of
+ controllers on Qualcomm chipsets, such as, PCIe, UFS and USB.
+
+ Note that these bindings are for SoCs up to SC8180X. For newer SoCs, see
+ qcom,sc8280xp-qmp-usb43dp-phy.yaml.
maintainers:
- Wesley Cheng <quic_wcheng@quicinc.com>
- qcom,sc7180-qmp-usb3-dp-phy
- qcom,sc7280-qmp-usb3-dp-phy
- qcom,sc8180x-qmp-usb3-dp-phy
- - qcom,sc8280xp-qmp-usb43dp-phy
- qcom,sdm845-qmp-usb3-dp-phy
- qcom,sm8250-qmp-usb3-dp-phy
reg:
additionalProperties: false
-allOf:
- - if:
- properties:
- compatible:
- contains:
- enum:
- - qcom,sc8280xp-qmp-usb43dp-phy
- then:
- required:
- - power-domains
-
examples:
- |
#include <dt-bindings/clock/qcom,gcc-sdm845.h>
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/phy/qcom,sc8280xp-qmp-pcie-phy.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm QMP PHY controller (PCIe, SC8280XP)
+
+maintainers:
+ - Vinod Koul <vkoul@kernel.org>
+
+description:
+ The QMP PHY controller supports physical layer functionality for a number of
+ controllers on Qualcomm chipsets, such as, PCIe, UFS, and USB.
+
+properties:
+ compatible:
+ enum:
+ - qcom,sc8280xp-qmp-gen3x1-pcie-phy
+ - qcom,sc8280xp-qmp-gen3x2-pcie-phy
+ - qcom,sc8280xp-qmp-gen3x4-pcie-phy
+
+ reg:
+ minItems: 1
+ maxItems: 2
+
+ clocks:
+ maxItems: 6
+
+ clock-names:
+ items:
+ - const: aux
+ - const: cfg_ahb
+ - const: ref
+ - const: rchng
+ - const: pipe
+ - const: pipediv2
+
+ power-domains:
+ maxItems: 1
+
+ resets:
+ maxItems: 1
+
+ reset-names:
+ items:
+ - const: phy
+
+ vdda-phy-supply: true
+
+ vdda-pll-supply: true
+
+ qcom,4ln-config-sel:
+ description: PCIe 4-lane configuration
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ items:
+ - items:
+ - description: phandle of TCSR syscon
+ - description: offset of PCIe 4-lane configuration register
+ - description: offset of configuration bit for this PHY
+
+ "#clock-cells":
+ const: 0
+
+ clock-output-names:
+ maxItems: 1
+
+ "#phy-cells":
+ const: 0
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - power-domains
+ - resets
+ - reset-names
+ - vdda-phy-supply
+ - vdda-pll-supply
+ - "#clock-cells"
+ - clock-output-names
+ - "#phy-cells"
+
+additionalProperties: false
+
+allOf:
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - qcom,sc8280xp-qmp-gen3x4-pcie-phy
+ then:
+ properties:
+ reg:
+ items:
+ - description: port a
+ - description: port b
+ required:
+ - qcom,4ln-config-sel
+ else:
+ properties:
+ reg:
+ maxItems: 1
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,gcc-sc8280xp.h>
+
+ pcie2b_phy: phy@1c18000 {
+ compatible = "qcom,sc8280xp-qmp-gen3x2-pcie-phy";
+ reg = <0x01c18000 0x2000>;
+
+ clocks = <&gcc GCC_PCIE_2B_AUX_CLK>,
+ <&gcc GCC_PCIE_2B_CFG_AHB_CLK>,
+ <&gcc GCC_PCIE_2A2B_CLKREF_CLK>,
+ <&gcc GCC_PCIE2B_PHY_RCHNG_CLK>,
+ <&gcc GCC_PCIE_2B_PIPE_CLK>,
+ <&gcc GCC_PCIE_2B_PIPEDIV2_CLK>;
+ clock-names = "aux", "cfg_ahb", "ref", "rchng",
+ "pipe", "pipediv2";
+
+ power-domains = <&gcc PCIE_2B_GDSC>;
+
+ resets = <&gcc GCC_PCIE_2B_PHY_BCR>;
+ reset-names = "phy";
+
+ vdda-phy-supply = <&vreg_l6d>;
+ vdda-pll-supply = <&vreg_l4d>;
+
+ #clock-cells = <0>;
+ clock-output-names = "pcie_2b_pipe_clk";
+
+ #phy-cells = <0>;
+ };
+
+ pcie2a_phy: phy@1c24000 {
+ compatible = "qcom,sc8280xp-qmp-gen3x4-pcie-phy";
+ reg = <0x01c24000 0x2000>, <0x01c26000 0x2000>;
+
+ clocks = <&gcc GCC_PCIE_2A_AUX_CLK>,
+ <&gcc GCC_PCIE_2A_CFG_AHB_CLK>,
+ <&gcc GCC_PCIE_2A2B_CLKREF_CLK>,
+ <&gcc GCC_PCIE2A_PHY_RCHNG_CLK>,
+ <&gcc GCC_PCIE_2A_PIPE_CLK>,
+ <&gcc GCC_PCIE_2A_PIPEDIV2_CLK>;
+ clock-names = "aux", "cfg_ahb", "ref", "rchng",
+ "pipe", "pipediv2";
+
+ power-domains = <&gcc PCIE_2A_GDSC>;
+
+ resets = <&gcc GCC_PCIE_2A_PHY_BCR>;
+ reset-names = "phy";
+
+ vdda-phy-supply = <&vreg_l6d>;
+ vdda-pll-supply = <&vreg_l4d>;
+
+ qcom,4ln-config-sel = <&tcsr 0xa044 0>;
+
+ #clock-cells = <0>;
+ clock-output-names = "pcie_2a_pipe_clk";
+
+ #phy-cells = <0>;
+ };
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/phy/qcom,sc8280xp-qmp-ufs-phy.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm QMP PHY controller (UFS, SC8280XP)
+
+maintainers:
+ - Vinod Koul <vkoul@kernel.org>
+
+description:
+ The QMP PHY controller supports physical layer functionality for a number of
+ controllers on Qualcomm chipsets, such as, PCIe, UFS, and USB.
+
+properties:
+ compatible:
+ enum:
+ - qcom,sc8280xp-qmp-ufs-phy
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 2
+
+ clock-names:
+ items:
+ - const: ref
+ - const: ref_aux
+
+ power-domains:
+ maxItems: 1
+
+ resets:
+ maxItems: 1
+
+ reset-names:
+ items:
+ - const: ufsphy
+
+ vdda-phy-supply: true
+
+ vdda-pll-supply: true
+
+ "#phy-cells":
+ const: 0
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - power-domains
+ - resets
+ - reset-names
+ - vdda-phy-supply
+ - vdda-pll-supply
+ - "#phy-cells"
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,gcc-sc8280xp.h>
+
+ ufs_mem_phy: phy@1d87000 {
+ compatible = "qcom,sc8280xp-qmp-ufs-phy";
+ reg = <0x01d87000 0x1000>;
+
+ clocks = <&gcc GCC_UFS_REF_CLKREF_CLK>, <&gcc GCC_UFS_PHY_PHY_AUX_CLK>;
+ clock-names = "ref", "ref_aux";
+
+ power-domains = <&gcc UFS_PHY_GDSC>;
+
+ resets = <&ufs_mem_hc 0>;
+ reset-names = "ufsphy";
+
+ vdda-phy-supply = <&vreg_l6b>;
+ vdda-pll-supply = <&vreg_l3b>;
+
+ #phy-cells = <0>;
+ };
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/phy/qcom,sc8280xp-qmp-usb3-uni-phy.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm QMP PHY controller (USB, SC8280XP)
+
+maintainers:
+ - Vinod Koul <vkoul@kernel.org>
+
+description:
+ The QMP PHY controller supports physical layer functionality for a number of
+ controllers on Qualcomm chipsets, such as, PCIe, UFS, and USB.
+
+properties:
+ compatible:
+ enum:
+ - qcom,sc8280xp-qmp-usb3-uni-phy
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 4
+
+ clock-names:
+ items:
+ - const: aux
+ - const: ref
+ - const: com_aux
+ - const: pipe
+
+ power-domains:
+ maxItems: 1
+
+ resets:
+ maxItems: 2
+
+ reset-names:
+ items:
+ - const: phy
+ - const: phy_phy
+
+ vdda-phy-supply: true
+
+ vdda-pll-supply: true
+
+ "#clock-cells":
+ const: 0
+
+ clock-output-names:
+ maxItems: 1
+
+ "#phy-cells":
+ const: 0
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - power-domains
+ - resets
+ - reset-names
+ - vdda-phy-supply
+ - vdda-pll-supply
+ - "#clock-cells"
+ - clock-output-names
+ - "#phy-cells"
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,gcc-sc8280xp.h>
+ #include <dt-bindings/clock/qcom,rpmh.h>
+
+ phy@88ef000 {
+ compatible = "qcom,sc8280xp-qmp-usb3-uni-phy";
+ reg = <0x088ef000 0x2000>;
+
+ clocks = <&gcc GCC_USB3_MP_PHY_AUX_CLK>,
+ <&gcc GCC_USB3_MP0_CLKREF_CLK>,
+ <&gcc GCC_USB3_MP_PHY_COM_AUX_CLK>,
+ <&gcc GCC_USB3_MP_PHY_PIPE_0_CLK>;
+ clock-names = "aux", "ref", "com_aux", "pipe";
+
+ power-domains = <&gcc USB30_MP_GDSC>;
+
+ resets = <&gcc GCC_USB3_UNIPHY_MP0_BCR>,
+ <&gcc GCC_USB3UNIPHY_PHY_MP0_BCR>;
+ reset-names = "phy", "phy_phy";
+
+ vdda-phy-supply = <&vreg_l3a>;
+ vdda-pll-supply = <&vreg_l5a>;
+
+ #clock-cells = <0>;
+ clock-output-names = "usb2_phy0_pipe_clk";
+
+ #phy-cells = <0>;
+ };
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/phy/qcom,sc8280xp-qmp-usb43dp-phy.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm QMP USB4-USB3-DP PHY controller (SC8280XP)
+
+maintainers:
+ - Vinod Koul <vkoul@kernel.org>
+
+description:
+ The QMP PHY controller supports physical layer functionality for a number of
+ controllers on Qualcomm chipsets, such as, PCIe, UFS and USB.
+
+properties:
+ compatible:
+ enum:
+ - qcom,sc8280xp-qmp-usb43dp-phy
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 4
+
+ clock-names:
+ items:
+ - const: aux
+ - const: ref
+ - const: com_aux
+ - const: usb3_pipe
+
+ power-domains:
+ maxItems: 1
+
+ resets:
+ maxItems: 2
+
+ reset-names:
+ items:
+ - const: phy
+ - const: common
+
+ vdda-phy-supply: true
+
+ vdda-pll-supply: true
+
+ "#clock-cells":
+ const: 1
+ description:
+ See include/dt-bindings/dt-bindings/phy/phy-qcom-qmp.h
+
+ "#phy-cells":
+ const: 1
+ description:
+ See include/dt-bindings/dt-bindings/phy/phy-qcom-qmp.h
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - power-domains
+ - resets
+ - reset-names
+ - vdda-phy-supply
+ - vdda-pll-supply
+ - "#clock-cells"
+ - "#phy-cells"
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,gcc-sc8280xp.h>
+
+ phy@88eb000 {
+ compatible = "qcom,sc8280xp-qmp-usb43dp-phy";
+ reg = <0x088eb000 0x4000>;
+
+ clocks = <&gcc GCC_USB3_PRIM_PHY_AUX_CLK>,
+ <&gcc GCC_USB4_EUD_CLKREF_CLK>,
+ <&gcc GCC_USB3_PRIM_PHY_COM_AUX_CLK>,
+ <&gcc GCC_USB3_PRIM_PHY_PIPE_CLK>;
+ clock-names = "aux", "ref", "com_aux", "usb3_pipe";
+
+ power-domains = <&gcc USB30_PRIM_GDSC>;
+
+ resets = <&gcc GCC_USB3_PHY_PRIM_BCR>,
+ <&gcc GCC_USB4_DP_PHY_PRIM_BCR>;
+ reset-names = "phy", "common";
+
+ vdda-phy-supply = <&vreg_l9d>;
+ vdda-pll-supply = <&vreg_l4d>;
+
+ #clock-cells = <1>;
+ #phy-cells = <1>;
+ };
$id: http://devicetree.org/schemas/phy/qcom,usb-hs-phy.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm's USB HS PHY binding description
+title: Qualcomm's USB HS PHY
maintainers:
- Bjorn Andersson <bjorn.andersson@linaro.org>
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/phy/renesas,r8a779f0-ether-serdes.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Renesas Ethernet SERDES
+
+maintainers:
+ - Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
+
+properties:
+ compatible:
+ const: renesas,r8a779f0-ether-serdes
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ resets:
+ maxItems: 1
+
+ power-domains:
+ maxItems: 1
+
+ '#phy-cells':
+ description: Port number of SERDES.
+ const: 1
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - resets
+ - power-domains
+ - '#phy-cells'
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/r8a779f0-cpg-mssr.h>
+ #include <dt-bindings/power/r8a779f0-sysc.h>
+
+ phy@e6444000 {
+ compatible = "renesas,r8a779f0-ether-serdes";
+ reg = <0xe6444000 0xc00>;
+ clocks = <&cpg CPG_MOD 1506>;
+ power-domains = <&sysc R8A779F0_PD_ALWAYS_ON>;
+ resets = <&cpg 1506>;
+ #phy-cells = <1>;
+ };
$id: http://devicetree.org/schemas/phy/ti,phy-am654-serdes.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: TI AM654 SERDES binding
+title: TI AM654 SERDES
description:
This binding describes the TI AM654 SERDES. AM654 SERDES can be configured
$id: "http://devicetree.org/schemas/phy/ti,phy-gmii-sel.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: CPSW Port's Interface Mode Selection PHY Tree Bindings
+title: CPSW Port's Interface Mode Selection PHY
maintainers:
- Kishon Vijay Abraham I <kishon@ti.com>
- ti,dm814-phy-gmii-sel
- ti,am654-phy-gmii-sel
- ti,j7200-cpsw5g-phy-gmii-sel
+ - ti,j721e-cpsw9g-phy-gmii-sel
reg:
maxItems: 1
ti,qsgmii-main-ports:
$ref: /schemas/types.yaml#/definitions/uint32-array
description: |
- Required only for QSGMII mode. Array to select the port for
- QSGMII main mode. Rest of the ports are selected as QSGMII_SUB
- ports automatically. Any one of the 4 CPSW5G ports can act as the
- main port with the rest of them being the QSGMII_SUB ports.
- maxItems: 1
+ Required only for QSGMII mode. Array to select the port/s for QSGMII
+ main mode. The size of the array corresponds to the number of QSGMII
+ interfaces and thus, the number of distinct QSGMII main ports,
+ supported by the device. If the device supports two QSGMII interfaces
+ but only one QSGMII interface is desired, repeat the QSGMII main port
+ value corresponding to the QSGMII interface in the array.
+ minItems: 1
+ maxItems: 2
items:
minimum: 1
- maximum: 4
+ maximum: 8
allOf:
- if:
- ti,dra7xx-phy-gmii-sel
- ti,dm814-phy-gmii-sel
- ti,am654-phy-gmii-sel
+ - ti,j7200-cpsw5g-phy-gmii-sel
+ - ti,j721e-cpsw9g-phy-gmii-sel
then:
properties:
'#phy-cells':
description: CPSW port number (starting from 1)
- if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - ti,j7200-cpsw5g-phy-gmii-sel
+ then:
+ properties:
+ ti,qsgmii-main-ports:
+ maxItems: 1
+ items:
+ minimum: 1
+ maximum: 4
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - ti,j721e-cpsw9g-phy-gmii-sel
+ then:
+ properties:
+ ti,qsgmii-main-ports:
+ minItems: 2
+ maxItems: 2
+ items:
+ minimum: 1
+ maximum: 8
+
+ - if:
not:
properties:
compatible:
contains:
enum:
- ti,j7200-cpsw5g-phy-gmii-sel
+ - ti,j721e-cpsw9g-phy-gmii-sel
then:
properties:
ti,qsgmii-main-ports: false
enum:
- ti,j721e-wiz-16g
- ti,j721e-wiz-10g
+ - ti,j721s2-wiz-10g
- ti,am64-wiz-10g
- ti,j7200-wiz-10g
+ - ti,j784s4-wiz-10g
power-domains:
maxItems: 1
$id: http://devicetree.org/schemas/phy/transmit-amplitude.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Common PHY and network PCS transmit amplitude property binding
+title: Common PHY and network PCS transmit amplitude property
description:
Binding describing the peak-to-peak transmit amplitude for common PHYs
$id: http://devicetree.org/schemas/pinctrl/fsl,scu-pinctrl.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: i.MX SCU Client Device Node - Pinctrl bindings based on SCU Message Protocol
+title: i.MX SCU Client Device Node - Pinctrl Based on SCU Message Protocol
maintainers:
- Dong Aisheng <aisheng.dong@nxp.com>
$id: http://devicetree.org/schemas/pinctrl/ingenic,pinctrl.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs pin controller devicetree bindings
+title: Ingenic SoCs pin controller
description: >
Please refer to pinctrl-bindings.txt in this directory for details of the
$id: http://devicetree.org/schemas/pinctrl/intel,lgm-io.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Intel Lightning Mountain SoC pinmux & GPIO controller binding
+title: Intel Lightning Mountain SoC pinmux & GPIO controller
maintainers:
- Rahul Tanwar <rahul.tanwar@linux.intel.com>
$id: http://devicetree.org/schemas/pinctrl/pincfg-node.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Generic pin configuration node schema
+title: Generic Pin Configuration Node
maintainers:
- Linus Walleij <linus.walleij@linaro.org>
$id: http://devicetree.org/schemas/pinctrl/pinmux-node.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Generic pin multiplexing node schema
+title: Generic Pin Multiplexing Node
maintainers:
- Linus Walleij <linus.walleij@linaro.org>
$id: http://devicetree.org/schemas/power/avs/qcom,cpr.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Core Power Reduction (CPR) bindings
+title: Qualcomm Core Power Reduction (CPR)
maintainers:
- Niklas Cassel <nks@flawful.org>
$id: http://devicetree.org/schemas/power/domain-idle-state.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: PM Domain Idle States binding description
+title: PM Domain Idle States
maintainers:
- Ulf Hansson <ulf.hansson@linaro.org>
$id: http://devicetree.org/schemas/power/fsl,scu-pd.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: i.MX SCU Client Device Node - Power domain bindings based on SCU Message Protocol
+title: i.MX SCU Client Device Node - Power Domain Based on SCU Message Protocol
maintainers:
- Dong Aisheng <aisheng.dong@nxp.com>
inactive-delay, the GPIO is driven active again. After a delay specified by wait-delay, the
restart handler completes allowing other restart handlers to be attempted.
+allOf:
+ - $ref: restart-handler.yaml#
+
properties:
compatible:
const: gpio-restart
in its inactive state.
priority:
- $ref: /schemas/types.yaml#/definitions/uint32
- description: |
- A priority ranging from 0 to 255 (default 129) according to the following guidelines:
-
- 0: Restart handler of last resort, with limited restart capabilities.
- 128: Default restart handler; use if no other restart handler is expected to be available,
- and/or if restart functionality is sufficient to restart the entire system.
- 255: Highest priority restart handler, will preempt all other restart handlers.
- minimum: 0
- maximum: 255
default: 129
active-delay:
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/power/reset/restart-handler.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Restart and shutdown handler generic binding
+
+maintainers:
+ - Sebastian Reichel <sre@kernel.org>
+
+description:
+ Restart and shutdown handler device is responsible for powering off the
+ system, e.g. my cutting off the power. System might have several restart
+ handlers, which usually are tried from most precise to last resort.
+
+properties:
+ priority:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: |
+ A priority ranging from 0 to 255 according to the following guidelines::
+ 0:: Restart handler of last resort, with limited restart capabilities.
+ 128:: Typical, default restart handler; use if no other restart handler
+ is expected to be available, and/or if restart functionality is
+ sufficient to restart the entire system.
+ 255:: Highest priority restart handler, will preempt all other restart handlers.
+ minimum: 0
+ maximum: 255
+
+additionalProperties: true
$id: http://devicetree.org/schemas/power/reset/xlnx,zynqmp-power.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Xilinx Zynq MPSoC Power Management Device Tree Bindings
+title: Xilinx Zynq MPSoC Power Management
maintainers:
- Michal Simek <michal.simek@xilinx.com>
$id: http://devicetree.org/schemas/power/supply/bq2415x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for TI bq2415x Li-Ion Charger
+title: TI bq2415x Li-Ion Charger
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/power/supply/bq24190.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for TI BQ2419x Li-Ion Battery Charger
+title: TI BQ2419x Li-Ion Battery Charger
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/power/supply/bq24257.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for bq24250, bq24251 and bq24257 Li-Ion Charger
+title: Bq24250, bq24251 and bq24257 Li-Ion Charger
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/power/supply/bq24735.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for TI BQ24735 Li-Ion Battery Charger
+title: TI BQ24735 Li-Ion Battery Charger
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/power/supply/bq25890.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for bq25890, bq25892, bq25895 and bq25896 Li-Ion Charger
+title: Bq25890, bq25892, bq25895 and bq25896 Li-Ion Charger
maintainers:
- Sebastian Reichel <sre@kernel.org>
properties:
compatible:
- enum:
- - ti,bq25890
- - ti,bq25892
- - ti,bq25895
- - ti,bq25896
+ oneOf:
+ - enum:
+ - ti,bq25890
+ - items:
+ - enum:
+ - ti,bq25892
+ - ti,bq25895
+ - ti,bq25896
+ - const: ti,bq25890
reg:
maxItems: 1
- ti,boost-voltage
- ti,boost-max-current
-additionalProperties: false
+unevaluatedProperties: false
examples:
- |
monitored-battery:
description: |
- phandle of battery characteristics node.
The fuel gauge uses the following battery properties:
- energy-full-design-microwatt-hours
- charge-full-design-microamp-hours
- voltage-min-design-microvolt
Both or neither of the *-full-design-*-hours properties must be set.
- See Documentation/devicetree/bindings/power/supply/battery.yaml
power-supplies: true
$id: http://devicetree.org/schemas/power/supply/dlg,da9150-charger.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Dialog Semiconductor DA9150 Charger Power Supply bindings
+title: Dialog Semiconductor DA9150 Charger Power Supply
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/power/supply/dlg,da9150-fuel-gauge.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Dialog Semiconductor DA9150 Fuel-Gauge Power Supply bindings
+title: Dialog Semiconductor DA9150 Fuel-Gauge Power Supply
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/power/supply/ingenic,battery.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic JZ47xx battery bindings
+title: Ingenic JZ47xx battery
maintainers:
- Artur Rojek <contact@artur-rojek.eu>
+$ref: power-supply.yaml#
+
properties:
compatible:
oneOf:
monitored-battery:
description: >
- phandle to a "simple-battery" compatible node.
-
This property must be a phandle to a node using the format described
in battery.yaml, with the following properties being required:
- voltage-min-design-microvolt: drained battery voltage,
$id: http://devicetree.org/schemas/power/supply/isp1704.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for NXP ISP1704 USB Charger Detection
+title: NXP ISP1704 USB Charger Detection
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/power/supply/lltc,lt3651-charger.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Analog Devices LT3651 Charger Power Supply bindings
+title: Analog Devices LT3651 Charger Power Supply
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/power/supply/lltc,ltc294x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for LTC2941, LTC2942, LTC2943 and LTC2944 battery fuel gauges
+title: LTC2941, LTC2942, LTC2943 and LTC2944 battery fuel gauges
description: |
All chips measure battery capacity.
$id: http://devicetree.org/schemas/power/supply/maxim,ds2760.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Maxim DS2760 DT bindings
+title: Maxim DS2760
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/power/supply/maxim,max14656.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Maxim MAX14656 DT bindings
+title: Maxim MAX14656
maintainers:
- Sebastian Reichel <sre@kernel.org>
Voltage threshold to report battery as over voltage (in mV).
Default is not to report over-voltage events.
+ power-supplies: true
+
required:
- compatible
- reg
This property is added to a supply in order to list the devices which
supply it power, referenced by their phandles.
+ monitored-battery:
+ $ref: /schemas/types.yaml#/definitions/phandle
+ description:
+ The battery (with "simple-battery" compatible) being monitored by this
+ power supply.
+
additionalProperties: true
$id: http://devicetree.org/schemas/power/supply/richtek,rt9455.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for Richtek rt9455 battery charger
+title: Richtek rt9455 battery charger
maintainers:
- Sebastian Reichel <sre@kernel.org>
title: ROHM BD99954 Battery charger
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
- Markus Laine <markus.laine@fi.rohmeurope.com>
- Mikko Mutanen <mikko.mutanen@fi.rohmeurope.com>
provides a Dual-source Battery Charger, two port BC1.2 detection and a
Battery Monitor.
+$ref: power-supply.yaml#
properties:
compatible:
$id: http://devicetree.org/schemas/power/supply/sc2731-charger.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Spreadtrum SC2731 PMICs battery charger binding
+title: Spreadtrum SC2731 PMICs battery charger
maintainers:
- Sebastian Reichel <sre@kernel.org>
The charger uses the following battery properties
- charge-term-current-microamp: current for charge termination phase.
- constant-charge-voltage-max-microvolt: maximum constant input voltage.
- See Documentation/devicetree/bindings/power/supply/battery.yaml
additionalProperties: false
$id: http://devicetree.org/schemas/power/supply/sc27xx-fg.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Spreadtrum SC27XX PMICs Fuel Gauge Unit Power Supply Bindings
+title: Spreadtrum SC27XX PMICs Fuel Gauge Unit Power Supply
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/power/supply/ti,lp8727.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for TI/National Semiconductor LP8727 Charger
+title: TI/National Semiconductor LP8727 Charger
maintainers:
- Sebastian Reichel <sre@kernel.org>
$id: http://devicetree.org/schemas/pwm/microchip,corepwm.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Microchip IP corePWM controller bindings
+title: Microchip IP corePWM controller
maintainers:
- Conor Dooley <conor.dooley@microchip.com>
- renesas,pwm-r8a77980 # R-Car V3H
- renesas,pwm-r8a77990 # R-Car E3
- renesas,pwm-r8a77995 # R-Car D3
+ - renesas,pwm-r8a779g0 # R-Car V4H
- const: renesas,pwm-rcar
reg:
- renesas,tpu-r8a77970 # R-Car V3M
- renesas,tpu-r8a77980 # R-Car V3H
- renesas,tpu-r8a779a0 # R-Car V3U
+ - renesas,tpu-r8a779g0 # R-Car V4H
- const: renesas,tpu
reg:
$id: http://devicetree.org/schemas/regulator/pwm-regulator.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for the Generic PWM Regulator
+title: Generic PWM Regulator
maintainers:
- Brian Norris <briannorris@chromium.org>
title: ROHM BD71815 Power Management Integrated Circuit regulators
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
This module is part of the ROHM BD718215 MFD device. For more details
title: ROHM BD71828 Power Management Integrated Circuit regulators
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
This module is part of the ROHM BD71828 MFD device. For more details
title: ROHM BD71837 Power Management Integrated Circuit regulators
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
List of regulators provided by this controller. BD71837 regulators node
title: ROHM BD71847 and BD71850 Power Management Integrated Circuit regulators
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
List of regulators provided by this controller. BD71847 regulators node
title: ROHM BD9576 and BD9573 Power Management Integrated Circuit regulators
maintainers:
- - Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ - Matti Vaittinen <mazziesaccount@gmail.com>
description: |
This module is part of the ROHM BD9576 MFD device. For more details
$id: http://devicetree.org/schemas/regulator/st,stm32-booster.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 booster for ADC analog input switches bindings
+title: STMicroelectronics STM32 booster for ADC analog input switches
maintainers:
- Fabrice Gasnier <fabrice.gasnier@foss.st.com>
$id: http://devicetree.org/schemas/regulator/st,stm32-vrefbuf.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Voltage reference buffer bindings
+title: STMicroelectronics STM32 Voltage reference buffer
description: |
Some STM32 devices embed a voltage reference buffer which can be used as
$id: "http://devicetree.org/schemas/remoteproc/amlogic,meson-mx-ao-arc.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Amlogic Meson AO ARC Remote Processor bindings
+title: Amlogic Meson AO ARC Remote Processor
description:
Amlogic Meson6, Meson8, Meson8b and Meson8m2 SoCs embed an ARC core
$id: "http://devicetree.org/schemas/remoteproc/fsl,imx-rproc.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: NXP i.MX Co-Processor Bindings
+title: NXP i.MX Co-Processor
description:
This binding provides support for ARM Cortex M4 Co-processor found on some NXP iMX SoCs.
- fsl,imx8mn-cm7
- fsl,imx8mp-cm7
- fsl,imx8mq-cm4
+ - fsl,imx8qm-cm4
+ - fsl,imx8qxp-cm4
- fsl,imx8ulp-cm33
- fsl,imx93-cm33
minItems: 1
maxItems: 32
+ power-domains:
+ maxItems: 8
+
fsl,auto-boot:
$ref: /schemas/types.yaml#/definitions/flag
description:
Indicate whether need to load the default firmware and start the remote
processor automatically.
+ fsl,entry-address:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description:
+ Specify CPU entry address for SCU enabled processor.
+
+ fsl,resource-id:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description:
+ This property is to specify the resource id of the remote processor in SoC
+ which supports SCFW
+
required:
- compatible
$id: "http://devicetree.org/schemas/remoteproc/ingenic,vpu.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Ingenic Video Processing Unit bindings
+title: Ingenic Video Processing Unit
description:
Inside the Video Processing Unit (VPU) of the recent JZ47xx SoCs from
$id: http://devicetree.org/schemas/remoteproc/mtk,scp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Mediatek SCP Bindings
+title: Mediatek SCP
maintainers:
- Tinghan Shen <tinghan.shen@mediatek.com>
$id: http://devicetree.org/schemas/remoteproc/qcom,adsp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm ADSP Peripheral Image Loader binding
+title: Qualcomm ADSP Peripheral Image Loader
maintainers:
- Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
$id: http://devicetree.org/schemas/remoteproc/qcom,pil-info.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm peripheral image loader relocation info binding
+title: Qualcomm peripheral image loader relocation info
maintainers:
- Bjorn Andersson <bjorn.andersson@linaro.org>
$id: "http://devicetree.org/schemas/remoteproc/renesas,rcar-rproc.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Renesas R-Car remote processor controller bindings
+title: Renesas R-Car remote processor controller
maintainers:
- Julien Massot <julien.massot@iot.bzh>
$id: "http://devicetree.org/schemas/remoteproc/st,stm32-rproc.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: STMicroelectronics STM32 remote processor controller bindings
+title: STMicroelectronics STM32 remote processor controller
description:
This document defines the binding for the remoteproc component that loads and
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only or BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/remoteproc/xlnx,zynqmp-r5fss.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Xilinx R5F processor subsystem
+
+maintainers:
+ - Ben Levinsky <ben.levinsky@amd.com>
+ - Tanmay Shah <tanmay.shah@amd.com>
+
+description: |
+ The Xilinx platforms include a pair of Cortex-R5F processors (RPU) for
+ real-time processing based on the Cortex-R5F processor core from ARM.
+ The Cortex-R5F processor implements the Arm v7-R architecture and includes a
+ floating-point unit that implements the Arm VFPv3 instruction set.
+
+properties:
+ compatible:
+ const: xlnx,zynqmp-r5fss
+
+ xlnx,cluster-mode:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ enum: [0, 1, 2]
+ description: |
+ The RPU MPCore can operate in split mode (Dual-processor performance), Safety
+ lock-step mode(Both RPU cores execute the same code in lock-step,
+ clock-for-clock) or Single CPU mode (RPU core 0 is held in reset while
+ core 1 runs normally). The processor does not support dynamic configuration.
+ Switching between modes is only permitted immediately after a processor reset.
+ If set to 1 then lockstep mode and if 0 then split mode.
+ If set to 2 then single CPU mode. When not defined, default will be lockstep mode.
+ In summary,
+ 0: split mode
+ 1: lockstep mode (default)
+ 2: single cpu mode
+
+patternProperties:
+ "^r5f-[a-f0-9]+$":
+ type: object
+ description: |
+ The RPU is located in the Low Power Domain of the Processor Subsystem.
+ Each processor includes separate L1 instruction and data caches and
+ tightly coupled memories (TCM). System memory is cacheable, but the TCM
+ memory space is non-cacheable.
+
+ Each RPU contains one 64KB memory and two 32KB memories that
+ are accessed via the TCM A and B port interfaces, for a total of 128KB
+ per processor. In lock-step mode, the processor has access to 256KB of
+ TCM memory.
+
+ properties:
+ compatible:
+ const: xlnx,zynqmp-r5f
+
+ power-domains:
+ maxItems: 1
+
+ mboxes:
+ minItems: 1
+ items:
+ - description: mailbox channel to send data to RPU
+ - description: mailbox channel to receive data from RPU
+
+ mbox-names:
+ minItems: 1
+ items:
+ - const: tx
+ - const: rx
+
+ sram:
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ minItems: 1
+ maxItems: 8
+ items:
+ maxItems: 1
+ description: |
+ phandles to one or more reserved on-chip SRAM regions. Other than TCM,
+ the RPU can execute instructions and access data from the OCM memory,
+ the main DDR memory, and other system memories.
+
+ The regions should be defined as child nodes of the respective SRAM
+ node, and should be defined as per the generic bindings in
+ Documentation/devicetree/bindings/sram/sram.yaml
+
+ memory-region:
+ description: |
+ List of phandles to the reserved memory regions associated with the
+ remoteproc device. This is variable and describes the memories shared with
+ the remote processor (e.g. remoteproc firmware and carveouts, rpmsg
+ vrings, ...). This reserved memory region will be allocated in DDR memory.
+ minItems: 1
+ maxItems: 8
+ items:
+ - description: region used for RPU firmware image section
+ - description: vdev buffer
+ - description: vring0
+ - description: vring1
+ additionalItems: true
+
+ required:
+ - compatible
+ - power-domains
+
+ unevaluatedProperties: false
+
+required:
+ - compatible
+
+additionalProperties: false
+
+examples:
+ - |
+ remoteproc {
+ compatible = "xlnx,zynqmp-r5fss";
+ xlnx,cluster-mode = <1>;
+
+ r5f-0 {
+ compatible = "xlnx,zynqmp-r5f";
+ power-domains = <&zynqmp_firmware 0x7>;
+ memory-region = <&rproc_0_fw_image>, <&rpu0vdev0buffer>, <&rpu0vdev0vring0>, <&rpu0vdev0vring1>;
+ mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
+ mbox-names = "tx", "rx";
+ };
+
+ r5f-1 {
+ compatible = "xlnx,zynqmp-r5f";
+ power-domains = <&zynqmp_firmware 0x8>;
+ memory-region = <&rproc_1_fw_image>, <&rpu1vdev0buffer>, <&rpu1vdev0vring0>, <&rpu1vdev0vring1>;
+ mboxes = <&ipi_mailbox_rpu1 0>, <&ipi_mailbox_rpu1 1>;
+ mbox-names = "tx", "rx";
+ };
+ };
+...
$id: http://devicetree.org/schemas/reserved-memory/shared-dma-pool.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: /reserved-memory DMA pool node bindings
+title: /reserved-memory DMA pool
maintainers:
- devicetree-spec@vger.kernel.org
$id: http://devicetree.org/schemas/reset/ti,sci-reset.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: TI-SCI reset controller node bindings
+title: TI-SCI reset controller
maintainers:
- Nishanth Menon <nm@ti.com>
$id: http://devicetree.org/schemas/reset/ti,tps380x-reset.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: TI TPS380x reset controller node bindings
+title: TI TPS380x reset controller
maintainers:
- Marco Felsch <kernel@pengutronix.de>
$id: http://devicetree.org/schemas/riscv/cpus.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: RISC-V bindings for 'cpus' DT nodes
+title: RISC-V CPUs
maintainers:
- Paul Walmsley <paul.walmsley@sifive.com>
$id: http://devicetree.org/schemas/rng/ingenic,rng.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for RNG in Ingenic SoCs
+title: RNG in Ingenic SoCs
maintainers:
- 周琰杰 (Zhou Yanjie) <zhouyanjie@wanyeetech.com>
$id: http://devicetree.org/schemas/rng/ingenic,trng.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for DTRNG in Ingenic SoCs
+title: DTRNG in Ingenic SoCs
maintainers:
- 周琰杰 (Zhou Yanjie) <zhouyanjie@wanyeetech.com>
$id: http://devicetree.org/schemas/rng/intel,ixp46x-rng.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Intel IXP46x RNG bindings
+title: Intel IXP46x RNG
description: |
The Intel IXP46x has a random number generator at a fixed physical
$id: http://devicetree.org/schemas/rng/silex-insight,ba431-rng.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Silex Insight BA431 RNG bindings
+title: Silex Insight BA431 RNG
description: |
The BA431 hardware random number generator is an IP that is FIPS-140-2/3
$id: http://devicetree.org/schemas/rng/st,stm32-rng.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 RNG bindings
+title: STMicroelectronics STM32 RNG
description: |
The STM32 hardware random number generator is a simple fixed purpose
$id: http://devicetree.org/schemas/rng/xiphera,xip8001b-trng.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Xiphera XIP8001B-trng bindings
+title: Xiphera XIP8001B-trng
maintainers:
- Atte Tommiska <atte.tommiska@xiphera.com>
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/rtc/amlogic,meson6-rtc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Amlogic Meson6, Meson8, Meson8b and Meson8m2 RTC
+
+maintainers:
+ - Neil Armstrong <neil.armstrong@linaro.org>
+ - Martin Blumenstingl <martin.blumenstingl@googlemail.com>
+
+allOf:
+ - $ref: rtc.yaml#
+ - $ref: /schemas/nvmem/nvmem.yaml#
+
+properties:
+ compatible:
+ enum:
+ - amlogic,meson6-rtc
+ - amlogic,meson8-rtc
+ - amlogic,meson8b-rtc
+ - amlogic,meson8m2-rtc
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ resets:
+ maxItems: 1
+
+ vdd-supply: true
+
+required:
+ - compatible
+ - reg
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/irq.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ rtc: rtc@740 {
+ compatible = "amlogic,meson6-rtc";
+ reg = <0x740 0x14>;
+ interrupts = <GIC_SPI 72 IRQ_TYPE_EDGE_RISING>;
+ clocks = <&rtc32k_xtal>;
+ vdd-supply = <&rtc_vdd>;
+ resets = <&reset_rtc>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ mac@0 {
+ reg = <0 6>;
+ };
+ };
$id: http://devicetree.org/schemas/rtc/epson,rx8900.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: EPSON RX8900 / Microcrystal RV8803 Real-Time Clock DT bindings
+title: EPSON RX8900 / Microcrystal RV8803 Real-Time Clock
maintainers:
- Marek Vasut <marex@denx.de>
$id: http://devicetree.org/schemas/rtc/fsl,scu-rtc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: i.MX SCU Client Device Node - RTC bindings based on SCU Message Protocol
+title: i.MX SCU Client Device Node - RTC Based on SCU Message Protocol
maintainers:
- Dong Aisheng <aisheng.dong@nxp.com>
+++ /dev/null
-Haoyu Microelectronics HYM8563 Real Time Clock
-
-The HYM8563 provides basic rtc and alarm functionality
-as well as a clock output of up to 32kHz.
-
-Required properties:
-- compatible: should be: "haoyu,hym8563"
-- reg: i2c address
-- #clock-cells: the value should be 0
-
-Optional properties:
-- clock-output-names: From common clock binding
-- interrupts: rtc alarm/event interrupt
-
-Example:
-
-hym8563: hym8563@51 {
- compatible = "haoyu,hym8563";
- reg = <0x51>;
-
- interrupts = <13 IRQ_TYPE_EDGE_FALLING>;
-
- #clock-cells = <0>;
-};
-
-device {
-...
- clocks = <&hym8563>;
-...
-};
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/rtc/haoyu,hym8563.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Haoyu Microelectronics HYM8563 RTC
+
+maintainers:
+ - Alexandre Belloni <alexandre.belloni@bootlin.com>
+
+properties:
+ compatible:
+ const: haoyu,hym8563
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ "#clock-cells":
+ const: 0
+
+ clock-output-names:
+ description: From common clock binding to override the default output clock name.
+ maxItems: 1
+
+ wakeup-source:
+ description: Enables wake up of host system on alarm.
+
+allOf:
+ - $ref: rtc.yaml
+
+unevaluatedProperties: false
+
+required:
+ - compatible
+ - reg
+ - "#clock-cells"
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/irq.h>
+
+ i2c {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ rtc@51 {
+ compatible = "haoyu,hym8563";
+ reg = <0x51>;
+ interrupts = <13 IRQ_TYPE_EDGE_FALLING>;
+ #clock-cells = <0>;
+ };
+ };
$id: http://devicetree.org/schemas/rtc/ingenic,rtc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs Real-Time Clock DT bindings
+title: Ingenic SoCs Real-Time Clock
maintainers:
- Paul Cercueil <paul@crapouillou.net>
properties:
compatible:
- enum:
- - qcom,pm8058-rtc
- - qcom,pm8921-rtc
- - qcom,pm8941-rtc
- - qcom,pm8018-rtc
- - qcom,pmk8350-rtc
+ oneOf:
+ - enum:
+ - qcom,pm8058-rtc
+ - qcom,pm8921-rtc
+ - qcom,pm8941-rtc
+ - qcom,pmk8350-rtc
+ - items:
+ - enum:
+ - qcom,pm8018-rtc
+ - const: qcom,pm8921-rtc
reg:
minItems: 1
$id: http://devicetree.org/schemas/rtc/renesas,rzn1-rtc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Renesas RZ/N1 SoCs Real-Time Clock DT bindings
+title: Renesas RZ/N1 SoCs Real-Time Clock
maintainers:
- Miquel Raynal <miquel.raynal@bootlin.com>
+++ /dev/null
-ST M41T80 family of RTC and compatible
-
-Required properties:
-- compatible: should be one of:
- "st,m41t62",
- "st,m41t65",
- "st,m41t80",
- "st,m41t81",
- "st,m41t81s",
- "st,m41t82",
- "st,m41t83",
- "st,m41t84",
- "st,m41t85",
- "st,m41t87",
- "microcrystal,rv4162",
-- reg: I2C bus address of the device
-
-Optional properties:
-- interrupts: rtc alarm interrupt.
-- clock-output-names: From common clock binding to override the default output
- clock name
-- wakeup-source: Enables wake up of host system on alarm
-
-Optional child node:
-- clock: Provide this if the square wave pin is used as boot-enabled fixed clock.
-
-Example:
- rtc@68 {
- compatible = "st,m41t80";
- reg = <0x68>;
- interrupt-parent = <&UIC0>;
- interrupts = <0x9 0x8>;
-
- clock {
- compatible = "fixed-clock";
- #clock-cells = <0>;
- clock-frequency = <32768>;
- };
- };
+++ /dev/null
-* Amlogic Meson6, Meson8, Meson8b and Meson8m2 RTC
-
-Required properties:
-- compatible: should be one of the following describing the hardware:
- * "amlogic,meson6-rtc"
- * "amlogic,meson8-rtc"
- * "amlogic,meson8b-rtc"
- * "amlogic,meson8m2-rtc"
-
-- reg: physical register space for the controller's memory mapped registers.
-- interrupts: the interrupt line of the RTC block.
-- clocks: reference to the external 32.768kHz crystal oscillator.
-- vdd-supply: reference to the power supply of the RTC block.
-- resets: reset controller reference to allow reset of the controller
-
-Optional properties for the battery-backed non-volatile memory:
-- #address-cells: should be 1 to address the battery-backed non-volatile memory
-- #size-cells: should be 1 to reference the battery-backed non-volatile memory
-
-Optional child nodes:
-- see ../nvmem/nvmem.txt
-
-Example:
-
- rtc: rtc@740 {
- compatible = "amlogic,meson6-rtc";
- reg = <0x740 0x14>;
- interrupts = <GIC_SPI 72 IRQ_TYPE_EDGE_RISING>;
- clocks = <&rtc32k_xtal>;
- vdd-supply = <&rtc_vdd>;
- resets = <&reset RESET_RTC>;
-
- #address-cells = <1>;
- #size-cells = <1>;
- };
$id: http://devicetree.org/schemas/rtc/rtc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: RTC Generic Binding
+title: Real Time Clock Common Properties
maintainers:
- Alexandre Belloni <alexandre.belloni@bootlin.com>
$id: http://devicetree.org/schemas/rtc/sa1100-rtc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell Real Time Clock controller bindings
+title: Marvell Real Time Clock controller
allOf:
- $ref: rtc.yaml#
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/rtc/st,m41t80.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: ST M41T80 family of RTC and compatible
+
+maintainers:
+ - Alexandre Belloni <alexandre.belloni@bootlin.com>
+
+properties:
+ compatible:
+ enum:
+ - st,m41t62
+ - st,m41t65
+ - st,m41t80
+ - st,m41t81
+ - st,m41t81s
+ - st,m41t82
+ - st,m41t83
+ - st,m41t84
+ - st,m41t85
+ - st,m41t87
+ - microcrystal,rv4162
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ "#clock-cells":
+ const: 1
+
+ clock-output-names:
+ maxItems: 1
+ description: From common clock binding to override the default output clock name.
+
+ clock:
+ type: object
+ $ref: /schemas/clock/fixed-clock.yaml#
+ properties:
+ clock-frequency:
+ const: 32768
+
+allOf:
+ - $ref: rtc.yaml
+
+unevaluatedProperties: false
+
+required:
+ - compatible
+ - reg
+
+examples:
+ - |
+ i2c {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ rtc@68 {
+ compatible = "st,m41t80";
+ reg = <0x68>;
+ interrupt-parent = <&UIC0>;
+ interrupts = <0x9 0x8>;
+
+ clock {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <32768>;
+ };
+ };
+ };
$id: http://devicetree.org/schemas/rtc/st,stm32-rtc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Real Time Clock Bindings
+title: STMicroelectronics STM32 Real Time Clock
maintainers:
- Gabriel Fernandez <gabriel.fernandez@foss.st.com>
$id: http://devicetree.org/schemas/serial/8250.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: UART (Universal Asynchronous Receiver/Transmitter) bindings
+title: UART (Universal Asynchronous Receiver/Transmitter)
maintainers:
- devicetree@vger.kernel.org
$id: http://devicetree.org/schemas/serial/8250_omap.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for 8250 compliant UARTs on TI's OMAP2+ and K3 SoCs
+title: 8250 compliant UARTs on TI's OMAP2+ and K3 SoCs
maintainers:
- Vignesh Raghavendra <vigneshr@ti.com>
$id: http://devicetree.org/schemas/serial/brcm,bcm7271-uart.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Broadcom 8250 based serial port devicetree bindings
+title: Broadcom 8250 based serial port
maintainers:
- Al Cooper <alcooperx@gmail.com>
$id: http://devicetree.org/schemas/serial/ingenic,uart.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs UART controller devicetree bindings
+title: Ingenic SoCs UART controller
maintainers:
- Paul Cercueil <paul@crapouillou.net>
oneOf:
- enum:
- ingenic,jz4740-uart
+ - ingenic,jz4750-uart
- ingenic,jz4760-uart
- ingenic,jz4780-uart
- ingenic,x1000-uart
- items:
- const: ingenic,jz4725b-uart
- const: ingenic,jz4740-uart
+ - items:
+ - const: ingenic,jz4755-uart
+ - const: ingenic,jz4750-uart
reg:
maxItems: 1
- enum:
- renesas,scif-r8a779a0 # R-Car V3U
- renesas,scif-r8a779f0 # R-Car S4-8
+ - renesas,scif-r8a779g0 # R-Car V4H
- const: renesas,rcar-gen4-scif # R-Car Gen4
- const: renesas,scif # generic SCIF compatible UART
$id: http://devicetree.org/schemas/serial/rs485.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: RS485 serial communications Bindings
+title: RS485 serial communications
description: The RTS signal is capable of automatically controlling line
direction for the built-in half-duplex mode. The properties described
$id: "http://devicetree.org/schemas/serial/serial.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Serial Interface Generic DT Bindings
+title: Serial Interface Generic
maintainers:
- Rob Herring <robh@kernel.org>
maintainers:
- Erwan Le Ray <erwan.leray@foss.st.com>
-title: STMicroelectronics STM32 USART bindings
+title: STMicroelectronics STM32 USART
properties:
compatible:
- if:
properties:
xlnx,use-parity:
- contains:
- const: 1
+ const: 1
then:
required:
- xlnx,odd-parity
$id: http://devicetree.org/schemas/serio/ps2-gpio.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for GPIO based PS/2
+title: GPIO based PS/2
maintainers:
- Danilo Krummrich <danilokrummrich@dk-develop.de>
+++ /dev/null
-SLIM(Serial Low Power Interchip Media Bus) bus
-
-SLIMbus is a 2-wire bus, and is used to communicate with peripheral
-components like audio-codec.
-
-Required property for SLIMbus controller node:
-- compatible - name of SLIMbus controller
-
-Child nodes:
-Every SLIMbus controller node can contain zero or more child nodes
-representing slave devices on the bus. Every SLIMbus slave device is
-uniquely determined by the enumeration address containing 4 fields:
-Manufacturer ID, Product code, Device index, and Instance value for
-the device.
-If child node is not present and it is instantiated after device
-discovery (slave device reporting itself present).
-
-In some cases it may be necessary to describe non-probeable device
-details such as non-standard ways of powering up a device. In
-such cases, child nodes for those devices will be present as
-slaves of the SLIMbus controller, as detailed below.
-
-Required property for SLIMbus child node if it is present:
-- reg - Should be ('Device index', 'Instance ID') from SLIMbus
- Enumeration Address.
- Device Index Uniquely identifies multiple Devices within
- a single Component.
- Instance ID Is for the cases where multiple Devices of the
- same type or Class are attached to the bus.
-
-- compatible -"slimMID,PID". The textual representation of Manufacturer ID,
- Product Code, shall be in lower case hexadecimal with leading
- zeroes suppressed
-
-Optional property for SLIMbus child node if it is present:
-- slim-ifc-dev - Should be phandle to SLIMBus Interface device.
- Required for devices which deal with streams.
-
-SLIMbus example for Qualcomm's slimbus manager component:
-
- slim@28080000 {
- compatible = "qcom,apq8064-slim", "qcom,slim";
- reg = <0x28080000 0x2000>,
- interrupts = <0 33 0>;
- clocks = <&lcc SLIMBUS_SRC>, <&lcc AUDIO_SLIMBUS_CLK>;
- clock-names = "iface", "core";
- #address-cells = <2>;
- #size-cell = <0>;
-
- codec_ifd: ifd@0,0{
- compatible = "slim217,60";
- reg = <0 0>;
- };
-
- codec: wcd9310@1,0{
- compatible = "slim217,60";
- reg = <1 0>;
- slim-ifc-dev = <&codec_ifd>;
- };
- };
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/slimbus/qcom,slim-ngd.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm SoC SLIMBus Non Generic Device (NGD) Controller
+
+maintainers:
+ - Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
+ - Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
+
+description:
+ SLIMBus NGD controller is a light-weight driver responsible for communicating
+ with SLIMBus slaves directly over the bus using messaging interface and
+ communicating with master component residing on ADSP for bandwidth and
+ data-channel management
+
+properties:
+ compatible:
+ enum:
+ - qcom,slim-ngd-v1.5.0 # for MSM8996
+ - qcom,slim-ngd-v2.1.0 # for SDM845
+
+ reg:
+ maxItems: 1
+
+ "#address-cells":
+ const: 1
+
+ "#size-cells":
+ const: 0
+
+ dmas:
+ maxItems: 2
+
+ dma-names:
+ items:
+ - const: rx
+ - const: tx
+
+ interrupts:
+ maxItems: 1
+
+ iommus:
+ maxItems: 1
+
+patternProperties:
+ "^slim@[0-9a-f]+$":
+ type: object
+ $ref: slimbus.yaml#
+ description:
+ Each subnode represents an instance of NGD
+
+ properties:
+ reg:
+ maxItems: 1
+
+ unevaluatedProperties: false
+
+required:
+ - compatible
+ - reg
+ - "#address-cells"
+ - "#size-cells"
+ - dmas
+ - dma-names
+ - interrupts
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ slim-ngd@171c0000 {
+ compatible = "qcom,slim-ngd-v2.1.0";
+ reg = <0x171c0000 0x2c000>;
+ interrupts = <GIC_SPI 163 IRQ_TYPE_LEVEL_HIGH>;
+
+ dmas = <&slimbam 3>, <&slimbam 4>;
+ dma-names = "rx", "tx";
+ iommus = <&apps_smmu 0x1806 0x0>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ slim@1 {
+ reg = <1>;
+ #address-cells = <2>;
+ #size-cells = <0>;
+
+ codec@1,0 {
+ compatible = "slim217,250";
+ reg = <1 0>;
+ slim-ifc-dev = <&wcd9340_ifd>;
+
+ #sound-dai-cells = <1>;
+
+ interrupts-extended = <&tlmm 54 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ #clock-cells = <0>;
+ clock-frequency = <9600000>;
+ clock-output-names = "mclk";
+ qcom,micbias1-microvolt = <1800000>;
+ qcom,micbias2-microvolt = <1800000>;
+ qcom,micbias3-microvolt = <1800000>;
+ qcom,micbias4-microvolt = <1800000>;
+
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ reset-gpios = <&tlmm 64 GPIO_ACTIVE_HIGH>;
+
+ /* Rest of the WCD9340 codec */
+ };
+ };
+ };
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/slimbus/qcom,slim.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm SoC SLIMbus controller
+
+maintainers:
+ - Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
+ - Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
+
+description:
+ SLIMbus controller used when applications processor controls SLIMbus master
+ component.
+
+allOf:
+ - $ref: slimbus.yaml#
+
+properties:
+ compatible:
+ items:
+ - enum:
+ - qcom,apq8064-slim
+ - const: qcom,slim
+
+ reg:
+ items:
+ - description: Physical address of controller register blocks
+ - description: SLEW RATE register
+
+ reg-names:
+ items:
+ - const: ctrl
+ - const: slew
+
+ clocks:
+ items:
+ - description: Interface clock for this controller
+ - description: Interrupt for controller core's BAM
+
+ clock-names:
+ items:
+ - const: iface
+ - const: core
+
+ interrupts:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - clocks
+ - clock-names
+ - interrupts
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,gcc-msm8960.h>
+ #include <dt-bindings/clock/qcom,lcc-msm8960.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ soc {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ slim@28080000 {
+ compatible = "qcom,apq8064-slim", "qcom,slim";
+ reg = <0x28080000 0x2000>, <0x80207c 4>;
+ reg-names = "ctrl", "slew";
+ interrupts = <GIC_SPI 33 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&lcc SLIMBUS_SRC>, <&lcc AUDIO_SLIMBUS_CLK>;
+ clock-names = "iface", "core";
+ #address-cells = <2>;
+ #size-cells = <0>;
+
+ audio-codec@1,0 {
+ compatible = "slim217,60";
+ reg = <1 0>;
+ };
+ };
+ };
+++ /dev/null
-Qualcomm SLIMBus Non Generic Device (NGD) Controller binding
-
-SLIMBus NGD controller is a light-weight driver responsible for communicating
-with SLIMBus slaves directly over the bus using messaging interface and
-communicating with master component residing on ADSP for bandwidth and
-data-channel management
-
-Please refer to slimbus/bus.txt for details of the common SLIMBus bindings.
-
-- compatible:
- Usage: required
- Value type: <stringlist>
- Definition: must be "qcom,slim-ngd-v<MAJOR>.<MINOR>.<STEP>"
- must be one of the following.
- "qcom,slim-ngd-v1.5.0" for MSM8996
- "qcom,slim-ngd-v2.1.0" for SDM845
-
-- reg:
- Usage: required
- Value type: <prop-encoded-array>
- Definition: must specify the base address and size of the controller
- register space.
-- dmas
- Usage: required
- Value type: <array of phandles>
- Definition: List of rx and tx dma channels
-
-- dma-names
- Usage: required
- Value type: <stringlist>
- Definition: must be "rx" and "tx".
-
-- interrupts:
- Usage: required
- Value type: <prop-encoded-array>
- Definition: must list controller IRQ.
-
-#address-cells
- Usage: required
- Value type: <u32>
- Definition: Should be 1, reflecting the instance id of ngd.
-
-#size-cells
- Usage: required
- Value type: <u32>
- Definition: Should be 0
-
-= NGD Devices
-Each subnode represents an instance of NGD, must contain the following
-properties:
-
-- reg:
- Usage: required
- Value type: <u32>
- Definition: Should be instance id of ngd.
-
-#address-cells
- Usage: required
- Refer to slimbus/bus.txt for details of the common SLIMBus bindings.
-
-#size-cells
- Usage: required
- Refer to slimbus/bus.txt for details of the common SLIMBus bindings.
-
-= EXAMPLE
-
-slim@91c0000 {
- compatible = "qcom,slim-ngd-v1.5.0";
- reg = <0x91c0000 0x2c000>;
- interrupts = <0 163 0>;
- dmas = <&slimbam 3>, <&slimbam 4>;
- dma-names = "rx", "tx";
- #address-cells = <1>;
- #size-cells = <0>;
- ngd@1 {
- reg = <1>;
- #address-cells = <1>;
- #size-cells = <1>;
- codec@1 {
- compatible = "slim217,1a0";
- reg = <1 0>;
- };
- };
-};
+++ /dev/null
-Qualcomm SLIMbus controller
-This controller is used if applications processor driver controls SLIMbus
-master component.
-
-Required properties:
-
- - #address-cells - refer to Documentation/devicetree/bindings/slimbus/bus.txt
- - #size-cells - refer to Documentation/devicetree/bindings/slimbus/bus.txt
-
- - reg : Offset and length of the register region(s) for the device
- - reg-names : Register region name(s) referenced in reg above
- Required register resource entries are:
- "ctrl": Physical address of controller register blocks
- "slew": required for "qcom,apq8064-slim" SOC.
- - compatible : should be "qcom,<SOC-NAME>-slim" for SOC specific compatible
- followed by "qcom,slim" for fallback.
- - interrupts : Interrupt number used by this controller
- - clocks : Interface and core clocks used by this SLIMbus controller
- - clock-names : Required clock-name entries are:
- "iface" : Interface clock for this controller
- "core" : Interrupt for controller core's BAM
-
-Example:
-
- slim@28080000 {
- compatible = "qcom,apq8064-slim", "qcom,slim";
- reg = <0x28080000 0x2000>, <0x80207C 4>;
- reg-names = "ctrl", "slew";
- interrupts = <0 33 0>;
- clocks = <&lcc SLIMBUS_SRC>, <&lcc AUDIO_SLIMBUS_CLK>;
- clock-names = "iface", "core";
- #address-cells = <2>;
- #size-cell = <0>;
-
- wcd9310: audio-codec@1,0{
- compatible = "slim217,60";
- reg = <1 0>;
- };
- };
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/slimbus/slimbus.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: SLIM (Serial Low Power Interchip Media) bus
+
+maintainers:
+ - Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
+
+description:
+ SLIMbus is a 2-wire bus, and is used to communicate with peripheral
+ components like audio-codec.
+
+properties:
+ $nodename:
+ pattern: "^slim(@.*|-[0-9a-f])*$"
+
+ "#address-cells":
+ const: 2
+
+ "#size-cells":
+ const: 0
+
+patternProperties:
+ "^.*@[0-9a-f]+,[0-9a-f]+$":
+ type: object
+ description: |
+ Every SLIMbus controller node can contain zero or more child nodes
+ representing slave devices on the bus. Every SLIMbus slave device is
+ uniquely determined by the enumeration address containing 4 fields::
+ Manufacturer ID, Product code, Device index, and Instance value for the
+ device.
+
+ If child node is not present and it is instantiated after device
+ discovery (slave device reporting itself present).
+
+ In some cases it may be necessary to describe non-probeable device
+ details such as non-standard ways of powering up a device. In such cases,
+ child nodes for those devices will be present as slaves of the SLIMbus
+ controller.
+
+ properties:
+ compatible:
+ pattern: "^slim[0-9a-f]+,[0-9a-f]+$"
+
+ reg:
+ maxItems: 1
+ description: |
+ Pair of (device index, instande ID), where::
+ - Device index, which uniquely identifies multiple devices within a
+ single component.
+ - Instance ID, can be used for the cases where multiple devices of
+ the same type or class are attached to the bus.
+
+ required:
+ - compatible
+ - reg
+
+ additionalProperties: true
+
+required:
+ - "#address-cells"
+ - "#size-cells"
+
+additionalProperties: true
+
+examples:
+ - |
+ #include <dt-bindings/clock/qcom,gcc-msm8960.h>
+ #include <dt-bindings/clock/qcom,lcc-msm8960.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ soc {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ slim@28080000 {
+ compatible = "qcom,apq8064-slim", "qcom,slim";
+ reg = <0x28080000 0x2000>, <0x80207c 4>;
+ reg-names = "ctrl", "slew";
+ interrupts = <GIC_SPI 33 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&lcc SLIMBUS_SRC>, <&lcc AUDIO_SLIMBUS_CLK>;
+ clock-names = "iface", "core";
+ #address-cells = <2>;
+ #size-cells = <0>;
+
+ audio-codec@1,0 {
+ compatible = "slim217,60";
+ reg = <1 0>;
+ };
+ };
+ };
$id: http://devicetree.org/schemas/soc/mediatek/mtk-svs.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: MediaTek Smart Voltage Scaling (SVS) Device Tree Bindings
+title: MediaTek Smart Voltage Scaling (SVS)
maintainers:
- Roger Lu <roger.lu@mediatek.com>
$id: http://devicetree.org/schemas/soc/qcom/qcom,aoss-qmp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Always-On Subsystem side channel binding
+title: Qualcomm Always-On Subsystem side channel
maintainers:
- Bjorn Andersson <bjorn.andersson@linaro.org>
$id: "http://devicetree.org/schemas/soc/qcom/qcom,apr.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Qualcomm APR/GPR (Asynchronous/Generic Packet Router) binding
+title: Qualcomm APR/GPR (Asynchronous/Generic Packet Router)
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: "http://devicetree.org/schemas/soc/qcom/qcom,smem.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Qualcomm Shared Memory Manager binding
+title: Qualcomm Shared Memory Manager
maintainers:
- Andy Gross <agross@kernel.org>
$id: "http://devicetree.org/schemas/soc/qcom/qcom,spm.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Qualcomm Subsystem Power Manager binding
+title: Qualcomm Subsystem Power Manager
maintainers:
- Andy Gross <agross@kernel.org>
$id: http://devicetree.org/schemas/soc/qcom/qcom-stats.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Technologies, Inc. (QTI) Stats bindings
+title: Qualcomm Technologies, Inc. (QTI) Stats
maintainers:
- Maulik Shah <mkshah@codeaurora.org>
$id: http://devicetree.org/schemas/soc/samsung/exynos-usi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Samsung's Exynos USI (Universal Serial Interface) binding
+title: Samsung's Exynos USI (Universal Serial Interface)
maintainers:
- Sam Protsenko <semen.protsenko@linaro.org>
$id: http://devicetree.org/schemas/soc/ti/sci-pm-domain.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: TI-SCI generic power domain node bindings
+title: TI-SCI generic power domain
maintainers:
- Nishanth Menon <nm@ti.com>
$id: http://devicetree.org/schemas/sound/audio-graph-port.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Audio Graph Card 'port' Node Bindings
+title: Audio Graph Card 'port'
maintainers:
- Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
$id: http://devicetree.org/schemas/sound/cirrus,cs42l51.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: CS42L51 audio codec DT bindings
+title: CS42L51 audio codec
maintainers:
- Olivier Moysan <olivier.moysan@foss.st.com>
$id: http://devicetree.org/schemas/sound/ingenic,aic.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs AC97 / I2S Controller (AIC) DT bindings
+title: Ingenic SoCs AC97 / I2S Controller (AIC)
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/sound/ingenic,codec.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic JZ47xx internal codec DT bindings
+title: Ingenic JZ47xx internal codec
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/sound/marvell,mmp-sspa.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvel SSPA Digital Audio Interface Bindings
+title: Marvel SSPA Digital Audio Interface
maintainers:
- Lubomir Rintel <lkundrak@v3.sk>
$id: http://devicetree.org/schemas/sound/qcom,lpass-cpu.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Qualcomm Technologies Inc. LPASS CPU dai driver bindings
+title: Qualcomm Technologies Inc. LPASS CPU dai driver
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/sound/qcom,lpass-rx-macro.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: LPASS(Low Power Audio Subsystem) RX Macro audio codec DT bindings
+title: LPASS(Low Power Audio Subsystem) RX Macro audio codec
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/sound/qcom,lpass-tx-macro.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: LPASS(Low Power Audio Subsystem) TX Macro audio codec DT bindings
+title: LPASS(Low Power Audio Subsystem) TX Macro audio codec
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/sound/qcom,lpass-va-macro.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: LPASS(Low Power Audio Subsystem) VA Macro audio codec DT bindings
+title: LPASS(Low Power Audio Subsystem) VA Macro audio codec
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/sound/qcom,lpass-wsa-macro.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: LPASS(Low Power Audio Subsystem) VA Macro audio codec DT bindings
+title: LPASS(Low Power Audio Subsystem) VA Macro audio codec
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: "http://devicetree.org/schemas/sound/qcom,q6apm-dai.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Qualcomm Audio Process Manager Digital Audio Interfaces binding
+title: Qualcomm Audio Process Manager Digital Audio Interfaces
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: "http://devicetree.org/schemas/sound/qcom,q6dsp-lpass-clocks.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Qualcomm DSP LPASS Clock Controller binding
+title: Qualcomm DSP LPASS Clock Controller
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: "http://devicetree.org/schemas/sound/qcom,q6dsp-lpass-ports.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Qualcomm DSP LPASS(Low Power Audio SubSystem) Audio Ports binding
+title: Qualcomm DSP LPASS(Low Power Audio SubSystem) Audio Ports
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/sound/qcom,wcd934x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for Qualcomm WCD9340/WCD9341 Audio Codec
+title: Qualcomm WCD9340/WCD9341 Audio Codec
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/sound/qcom,wcd938x-sdw.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for Qualcomm SoundWire Slave devices on WCD9380/WCD9385
+title: Qualcomm SoundWire Slave devices on WCD9380/WCD9385
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/sound/qcom,wcd938x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for Qualcomm WCD9380/WCD9385 Audio Codec
+title: Qualcomm WCD9380/WCD9385 Audio Codec
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/sound/qcom,wsa881x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for Qualcomm WSA8810/WSA8815 Class-D Smart Speaker Amplifier
+title: Qualcomm WSA8810/WSA8815 Class-D Smart Speaker Amplifier
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/sound/qcom,wsa883x.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for The Qualcomm WSA8830/WSA8832/WSA8835
+title: Qualcomm WSA8830/WSA8832/WSA8835
smart speaker amplifier
maintainers:
$id: http://devicetree.org/schemas/sound/realtek,rt1015p.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Realtek rt1015p codec devicetree bindings
+title: Realtek rt1015p codec
maintainers:
- Tzung-Bi Shih <tzungbi@kernel.org>
$id: http://devicetree.org/schemas/sound/realtek,rt5682s.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Realtek rt5682s codec devicetree bindings
+title: Realtek rt5682s codec
maintainers:
- Derek Fang <derek.fang@realtek.com>
$id: http://devicetree.org/schemas/sound/ti,src4xxx.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments SRC4392 Device Tree Bindings
+title: Texas Instruments SRC4392
description: |
The SRC4392 is a digital audio codec that can be connected via
+++ /dev/null
-Qualcomm SoundWire Controller Bindings
-
-
-This binding describes the Qualcomm SoundWire Controller along with its
-board specific bus parameters.
-
-- compatible:
- Usage: required
- Value type: <stringlist>
- Definition: must be "qcom,soundwire-v<MAJOR>.<MINOR>.<STEP>",
- Example:
- "qcom,soundwire-v1.3.0"
- "qcom,soundwire-v1.5.0"
- "qcom,soundwire-v1.5.1"
- "qcom,soundwire-v1.6.0"
-- reg:
- Usage: required
- Value type: <prop-encoded-array>
- Definition: the base address and size of SoundWire controller
- address space.
-
-- interrupts:
- Usage: required
- Value type: <prop-encoded-array>
- Definition: should specify the SoundWire Controller core and optional
- wake IRQ
-
-- interrupt-names:
- Usage: Optional
- Value type: boolean
- Value type: <stringlist>
- Definition: should be "core" for core and "wakeup" for wake interrupt.
-
-- wakeup-source:
- Usage: Optional
- Value type: boolean
- Definition: should specify if SoundWire Controller is wake up capable.
-
-- clock-names:
- Usage: required
- Value type: <stringlist>
- Definition: should be "iface" for SoundWire Controller interface clock
-
-- clocks:
- Usage: required
- Value type: <prop-encoded-array>
- Definition: should specify the SoundWire Controller interface clock
-
-- #sound-dai-cells:
- Usage: required
- Value type: <u32>
- Definition: must be 1 for digital audio interfaces on the controller.
-
-- qcom,dout-ports:
- Usage: required
- Value type: <u32>
- Definition: must be count of data out ports
-
-- qcom,din-ports:
- Usage: required
- Value type: <u32>
- Definition: must be count of data in ports
-
-- qcom,ports-offset1:
- Usage: required
- Value type: <prop-encoded-array>
- Definition: should specify payload transport window offset1 of each
- data port. Out ports followed by In ports.
- Value of 0xFF indicates that this option is not implemented
- or applicable for the respective data port.
- More info in MIPI Alliance SoundWire 1.0 Specifications.
-
-- qcom,ports-offset2:
- Usage: required
- Value type: <prop-encoded-array>
- Definition: should specify payload transport window offset2 of each
- data port. Out ports followed by In ports.
- Value of 0xFF indicates that this option is not implemented
- or applicable for the respective data port.
- More info in MIPI Alliance SoundWire 1.0 Specifications.
-
-- qcom,ports-sinterval-low:
- Usage: required
- Value type: <prop-encoded-array>
- Definition: should be sample interval low of each data port.
- Out ports followed by In ports. Used for Sample Interval
- calculation.
- Value of 0xFF indicates that this option is not implemented
- or applicable for the respective data port.
- More info in MIPI Alliance SoundWire 1.0 Specifications.
-
-- qcom,ports-word-length:
- Usage: optional
- Value type: <prop-encoded-array>
- Definition: should be size of payload channel sample.
- Value of 0xFF indicates that this option is not implemented
- or applicable for the respective data port.
- More info in MIPI Alliance SoundWire 1.0 Specifications.
-
-- qcom,ports-block-pack-mode:
- Usage: optional
- Value type: <prop-encoded-array>
- Definition: should be 0 or 1 to indicate the block packing mode.
- 0 to indicate Blocks are per Channel
- 1 to indicate Blocks are per Port.
- Out ports followed by In ports.
- Value of 0xFF indicates that this option is not implemented
- or applicable for the respective data port.
- More info in MIPI Alliance SoundWire 1.0 Specifications.
-
-- qcom,ports-block-group-count:
- Usage: optional
- Value type: <prop-encoded-array>
- Definition: should be in range 1 to 4 to indicate how many sample
- intervals are combined into a payload.
- Out ports followed by In ports.
- Value of 0xFF indicates that this option is not implemented
- or applicable for the respective data port.
- More info in MIPI Alliance SoundWire 1.0 Specifications.
-
-- qcom,ports-lane-control:
- Usage: optional
- Value type: <prop-encoded-array>
- Definition: should be in range 0 to 7 to identify which data lane
- the data port uses.
- Out ports followed by In ports.
- Value of 0xFF indicates that this option is not implemented
- or applicable for the respective data port.
- More info in MIPI Alliance SoundWire 1.0 Specifications.
-
-- qcom,ports-hstart:
- Usage: optional
- Value type: <prop-encoded-array>
- Definition: should be number identifying lowerst numbered coloum in
- SoundWire Frame, i.e. left edge of the Transport sub-frame
- for each port. Values between 0 and 15 are valid.
- Out ports followed by In ports.
- Value of 0xFF indicates that this option is not implemented
- or applicable for the respective data port.
- More info in MIPI Alliance SoundWire 1.0 Specifications.
-
-- qcom,ports-hstop:
- Usage: optional
- Value type: <prop-encoded-array>
- Definition: should be number identifying highest numbered coloum in
- SoundWire Frame, i.e. the right edge of the Transport
- sub-frame for each port. Values between 0 and 15 are valid.
- Out ports followed by In ports.
- Value of 0xFF indicates that this option is not implemented
- or applicable for the respective data port.
- More info in MIPI Alliance SoundWire 1.0 Specifications.
-
-- qcom,dports-type:
- Usage: optional
- Value type: <prop-encoded-array>
- Definition: should be one of the following types
- 0 for reduced port
- 1 for simple ports
- 2 for full port
- Out ports followed by In ports.
- Value of 0xFF indicates that this option is not implemented
- or applicable for the respective data port.
- More info in MIPI Alliance SoundWire 1.0 Specifications.
-
-- reset:
- Usage: optional
- Value type: <prop-encoded-array>
- Definition: Should specify the SoundWire audio CSR reset controller interface,
- which is required for SoundWire version 1.6.0 and above.
-
-- reset-names:
- Usage: optional
- Value type: <stringlist>
- Definition: should be "swr_audio_cgcr" for SoundWire audio CSR reset
- controller interface.
-
-Note:
- More Information on detail of encoding of these fields can be
-found in MIPI Alliance SoundWire 1.0 Specifications.
-
-= SoundWire devices
-Each subnode of the bus represents SoundWire device attached to it.
-The properties of these nodes are defined by the individual bindings.
-
-= EXAMPLE
-The following example represents a SoundWire controller on DB845c board
-which has controller integrated inside WCD934x codec on SDM845 SoC.
-
-soundwire: soundwire@c85 {
- compatible = "qcom,soundwire-v1.3.0";
- reg = <0xc85 0x20>;
- interrupts = <20 IRQ_TYPE_EDGE_RISING>;
- clocks = <&wcc>;
- clock-names = "iface";
- resets = <&lpass_audiocc LPASS_AUDIO_SWR_TX_CGCR>;
- reset-names = "swr_audio_cgcr";
- #sound-dai-cells = <1>;
- qcom,dports-type = <0>;
- qcom,dout-ports = <6>;
- qcom,din-ports = <2>;
- qcom,ports-sinterval-low = /bits/ 8 <0x07 0x1F 0x3F 0x7 0x1F 0x3F 0x0F 0x0F>;
- qcom,ports-offset1 = /bits/ 8 <0x01 0x02 0x0C 0x6 0x12 0x0D 0x07 0x0A >;
- qcom,ports-offset2 = /bits/ 8 <0x00 0x00 0x1F 0x00 0x00 0x1F 0x00 0x00>;
-
- /* Left Speaker */
- left{
- ....
- };
-
- /* Right Speaker */
- right{
- ....
- };
-};
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/soundwire/qcom,soundwire.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm SoundWire Controller
+
+maintainers:
+ - Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
+ - Srinivasa Rao Mandadapu <quic_srivasam@quicinc.com>
+
+description:
+ The Qualcomm SoundWire controller along with its board specific bus parameters.
+
+properties:
+ compatible:
+ enum:
+ - qcom,soundwire-v1.3.0
+ - qcom,soundwire-v1.5.0
+ - qcom,soundwire-v1.5.1
+ - qcom,soundwire-v1.6.0
+ - qcom,soundwire-v1.7.0
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ minItems: 1
+ items:
+ - description: specify the SoundWire controller core.
+ - description: specify the Soundwire controller wake IRQ.
+
+ interrupt-names:
+ minItems: 1
+ items:
+ - const: core
+ - const: wakeup
+
+ clocks:
+ items:
+ - description: iface clock
+
+ clock-names:
+ items:
+ - const: iface
+
+ resets:
+ items:
+ - description: SWR_AUDIO_CGCR RESET
+
+ reset-names:
+ items:
+ - const: swr_audio_cgcr
+
+ '#sound-dai-cells':
+ const: 1
+
+ '#address-cells':
+ const: 2
+
+ '#size-cells':
+ const: 0
+
+ wakeup-source: true
+
+ qcom,din-ports:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: count of data in ports
+
+ qcom,dout-ports:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description: count of data out ports
+
+ qcom,ports-word-length:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ Size of payload channel sample.
+ Value of 0xff indicates that this option is not implemented
+ or applicable for the respective data port.
+ More info in MIPI Alliance SoundWire 1.0 Specifications.
+ minItems: 3
+ maxItems: 5
+
+ qcom,ports-sinterval-low:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ Sample interval low of each data port.
+ Out ports followed by In ports. Used for Sample Interval calculation.
+ Value of 0xff indicates that this option is not implemented
+ or applicable for the respective data port.
+ More info in MIPI Alliance SoundWire 1.0 Specifications.
+ minItems: 3
+ maxItems: 8
+
+ qcom,ports-offset1:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ Payload transport window offset1 of each data port.
+ Out ports followed by In ports.
+ Value of 0xff indicates that this option is not implemented
+ or applicable for the respective data port.
+ More info in MIPI Alliance SoundWire 1.0 Specifications.
+ minItems: 3
+ maxItems: 8
+
+ qcom,ports-offset2:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ Payload transport window offset2 of each data port.
+ Out ports followed by In ports.
+ Value of 0xff indicates that this option is not implemented
+ or applicable for the respective data port.
+ More info in MIPI Alliance SoundWire 1.0 Specifications.
+ minItems: 3
+ maxItems: 8
+
+ qcom,ports-lane-control:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ Identify which data lane the data port uses.
+ Out ports followed by In ports.
+ Value of 0xff indicates that this option is not implemented
+ or applicable for the respective data port.
+ More info in MIPI Alliance SoundWire 1.0 Specifications.
+ minItems: 3
+ maxItems: 5
+
+ qcom,ports-block-pack-mode:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ Indicate the block packing mode.
+ 0 to indicate Blocks are per Channel
+ 1 to indicate Blocks are per Port.
+ Out ports followed by In ports.
+ Value of 0xff indicates that this option is not implemented
+ or applicable for the respective data port.
+ More info in MIPI Alliance SoundWire 1.0 Specifications.
+ minItems: 3
+ maxItems: 8
+ items:
+ oneOf:
+ - minimum: 0
+ maximum: 1
+ - const: 0xff
+
+ qcom,ports-hstart:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ Identifying lowerst numbered coloum in SoundWire Frame,
+ i.e. left edge of the Transport sub-frame for each port.
+ Out ports followed by In ports.
+ Value of 0xff indicates that this option is not implemented
+ or applicable for the respective data port.
+ More info in MIPI Alliance SoundWire 1.0 Specifications.
+ minItems: 3
+ maxItems: 5
+ items:
+ oneOf:
+ - minimum: 0
+ maximum: 15
+ - const: 0xff
+
+ qcom,ports-hstop:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ Identifying highest numbered coloum in SoundWire Frame,
+ i.e. the right edge of the Transport
+ sub-frame for each port. Out ports followed by In ports.
+ Value of 0xff indicates that this option is not implemented
+ or applicable for the respective data port.
+ More info in MIPI Alliance SoundWire 1.0 Specifications.
+ minItems: 3
+ maxItems: 5
+ items:
+ oneOf:
+ - minimum: 0
+ maximum: 15
+ - const: 0xff
+
+ qcom,ports-block-group-count:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description:
+ In range 1 to 4 to indicate how many sample intervals are combined
+ into a payload. Out ports followed by In ports.
+ Value of 0xff indicates that this option is not implemented
+ or applicable for the respective data port.
+ More info in MIPI Alliance SoundWire 1.0 Specifications.
+ minItems: 3
+ maxItems: 5
+ items:
+ oneOf:
+ - minimum: 0
+ maximum: 4
+ - const: 0xff
+
+ label:
+ maxItems: 1
+
+patternProperties:
+ "^.*@[0-9a-f],[0-9a-f]$":
+ type: object
+ description:
+ Child nodes for a standalone audio codec or speaker amplifier IC.
+ It has RX and TX Soundwire secondary devices.
+ properties:
+ compatible:
+ pattern: "^sdw[0-9a-f]{1}[0-9a-f]{4}[0-9a-f]{4}[0-9a-f]{2}$"
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+ - '#sound-dai-cells'
+ - '#address-cells'
+ - '#size-cells'
+ - qcom,dout-ports
+ - qcom,din-ports
+ - qcom,ports-sinterval-low
+ - qcom,ports-offset1
+ - qcom,ports-offset2
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interrupt-controller/irq.h>
+ #include <dt-bindings/clock/qcom,lpassaudiocc-sc7280.h>
+
+ soundwire@3210000 {
+ compatible = "qcom,soundwire-v1.6.0";
+ reg = <0x03210000 0x2000>;
+
+ interrupts = <GIC_SPI 155 IRQ_TYPE_LEVEL_HIGH>,
+ <&pdc 130 IRQ_TYPE_LEVEL_HIGH>;
+
+ interrupt-names = "core", "wakeup";
+
+ clocks = <&lpass_rx_macro>;
+ clock-names = "iface";
+
+ qcom,din-ports = <0>;
+ qcom,dout-ports = <5>;
+
+ resets = <&lpass_audiocc LPASS_AUDIO_SWR_RX_CGCR>;
+ reset-names = "swr_audio_cgcr";
+
+ qcom,ports-word-length = /bits/ 8 <0x01 0x07 0x04 0xff 0xff>;
+ qcom,ports-sinterval-low = /bits/ 8 <0x03 0x3f 0x1f 0x03 0x03>;
+ qcom,ports-offset1 = /bits/ 8 <0x00 0x00 0x0b 0x01 0x01>;
+ qcom,ports-offset2 = /bits/ 8 <0x00 0x00 0x0b 0x00 0x00>;
+ qcom,ports-lane-control = /bits/ 8 <0x01 0x00 0x00 0x00 0x00>;
+ qcom,ports-block-pack-mode = /bits/ 8 <0xff 0x00 0x01 0xff 0xff>;
+ qcom,ports-hstart = /bits/ 8 <0xff 0x03 0xff 0xff 0xff>;
+ qcom,ports-hstop = /bits/ 8 <0xff 0x06 0xff 0xff 0xff>;
+ qcom,ports-block-group-count = /bits/ 8 <0xff 0xff 0xff 0xff 0x00>;
+
+ #sound-dai-cells = <1>;
+ #address-cells = <2>;
+ #size-cells = <0>;
+
+ codec@0,4 {
+ compatible = "sdw20217010d00";
+ reg = <0 4>;
+ qcom,rx-port-mapping = <1 2 3 4 5>;
+ };
+ };
$id: http://devicetree.org/schemas/soundwire/soundwire-controller.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: SoundWire Controller Generic Binding
+title: SoundWire Controller Common Properties
maintainers:
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
$id: http://devicetree.org/schemas/spi/aspeed,ast2600-fmc.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Aspeed SMC controllers bindings
+title: Aspeed SMC controllers
maintainers:
- Chin-Ting Kuo <chin-ting_kuo@aspeedtech.com>
$id: http://devicetree.org/schemas/spi/ingenic,spi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs SPI controller devicetree bindings
+title: Ingenic SoCs SPI controller
maintainers:
- Artur Rojek <contact@artur-rojek.eu>
$id: http://devicetree.org/schemas/spi/marvell,mmp2-ssp.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: PXA2xx SSP SPI Controller bindings
+title: PXA2xx SSP SPI Controller
maintainers:
- Lubomir Rintel <lkundrak@v3.sk>
$id: http://devicetree.org/schemas/spi/omap-spi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: SPI controller bindings for OMAP and K3 SoCs
+title: SPI Controller on OMAP and K3 SoCs
maintainers:
- Aswath Govindraju <a-govindraju@ti.com>
$id: http://devicetree.org/schemas/spi/spi-controller.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: SPI Controller Generic Binding
+title: SPI Controller Common Properties
maintainers:
- Mark Brown <broonie@kernel.org>
$id: http://devicetree.org/schemas/spi/spi-gpio.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: SPI-GPIO devicetree bindings
+title: SPI-GPIO
maintainers:
- Rob Herring <robh@kernel.org>
$id: http://devicetree.org/schemas/spi/st,stm32-qspi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Quad Serial Peripheral Interface (QSPI) bindings
+title: STMicroelectronics STM32 Quad Serial Peripheral Interface (QSPI)
maintainers:
- Christophe Kerello <christophe.kerello@foss.st.com>
$id: http://devicetree.org/schemas/spi/st,stm32-spi.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 SPI Controller bindings
+title: STMicroelectronics STM32 SPI Controller
description: |
The STM32 SPI controller is used to communicate with external devices using
description: >
which of the PMIC Arb provided channels to use for accesses
+ qcom,bus-id:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ minimum: 0
+ maximum: 1
+ description: >
+ SPMI bus instance. only applicable to PMIC arbiter version 7 and beyond.
+ Supported values, 0 = primary bus, 1 = secondary bus
+
required:
- compatible
- reg-names
interrupt-controller;
#interrupt-cells = <4>;
+
+ qcom,bus-id = <0>;
};
$id: http://devicetree.org/schemas/thermal/fsl,scu-thermal.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: i.MX SCU Client Device Node - Thermal bindings based on SCU Message Protocol
+title: i.MX SCU Client Device Node - Thermal Based on SCU Message Protocol
maintainers:
- Dong Aisheng <aisheng.dong@nxp.com>
$id: http://devicetree.org/schemas/thermal/imx-thermal.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP i.MX Thermal Binding
+title: NXP i.MX Thermal
maintainers:
- Shawn Guo <shawnguo@kernel.org>
$id: http://devicetree.org/schemas/thermal/imx8mm-thermal.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP i.MX8M Mini Thermal Binding
+title: NXP i.MX8M Mini Thermal
maintainers:
- Anson Huang <Anson.Huang@nxp.com>
$id: http://devicetree.org/schemas/thermal/sprd-thermal.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Spreadtrum thermal sensor controller bindings
+title: Spreadtrum thermal sensor controller
maintainers:
- Orson Zhai <orsonzhai@gmail.com>
$id: http://devicetree.org/schemas/thermal/st,stm32-thermal.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 digital thermal sensor (DTS) binding
+title: STMicroelectronics STM32 digital thermal sensor (DTS)
maintainers:
- Pascal Paillet <p.paillet@foss.st.com>
$id: http://devicetree.org/schemas/thermal/thermal-cooling-devices.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Thermal cooling device binding
+title: Thermal cooling device
maintainers:
- Amit Kucheria <amitk@kernel.org>
$id: http://devicetree.org/schemas/thermal/thermal-idle.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Thermal idle cooling device binding
+title: Thermal idle cooling device
maintainers:
- Daniel Lezcano <daniel.lezcano@linaro.org>
$id: http://devicetree.org/schemas/thermal/thermal-sensor.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Thermal sensor binding
+title: Thermal sensor
maintainers:
- Amit Kucheria <amitk@kernel.org>
$id: http://devicetree.org/schemas/thermal/thermal-zones.yaml#
$schema: http://devicetree.org/meta-schemas/base.yaml#
-title: Thermal zone binding
+title: Thermal zone
maintainers:
- Amit Kucheria <amitk@kernel.org>
$id: http://devicetree.org/schemas/thermal/ti,am654-thermal.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments AM654 VTM (DTS) binding
+title: Texas Instruments AM654 VTM (DTS)
maintainers:
- Keerthy <j-keerthy@ti.com>
$id: http://devicetree.org/schemas/thermal/ti,j72xx-thermal.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Texas Instruments J72XX VTM (DTS) binding
+title: Texas Instruments J72XX VTM (DTS)
maintainers:
- Keerthy <j-keerthy@ti.com>
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/timer/brcm,bcmbca-timer.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Broadcom Broadband SoC timer
+
+maintainers:
+ - Rafał Miłecki <rafal@milecki.pl>
+
+properties:
+ compatible:
+ oneOf:
+ - const: brcm,bcm6345-timer
+ description: >
+ An old block with 3 timers.
+
+ It can be found in BCM6345, BCM6838 and BCM63268.
+ - const: brcm,bcm63138-timer
+ description: >
+ Updated block with 4 timers and control regs at the beginning.
+
+ It can be found in newer SoCs, e.g. BCM63138, BCM63148, BCM63381,
+ BCM68360, BCM6848, BCM6858, BCM4908.
+
+ reg:
+ maxItems: 1
+
+additionalProperties: false
+
+required:
+ - reg
+
+examples:
+ - |
+ timer@fffe0200 {
+ compatible = "brcm,bcm6345-timer";
+ reg = <0xfffe0200 0x1c>;
+ };
$id: http://devicetree.org/schemas/timer/ingenic,sysost.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Bindings for SYSOST in Ingenic XBurst family SoCs
+title: SYSOST in Ingenic XBurst family SoCs
maintainers:
- 周琰杰 (Zhou Yanjie) <zhouyanjie@wanyeetech.com>
$id: http://devicetree.org/schemas/timer/ingenic,tcu.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic SoCs Timer/Counter Unit (TCU) devicetree bindings
+title: Ingenic SoCs Timer/Counter Unit (TCU)
description: |
For a description of the TCU hardware and drivers, have a look at
$id: http://devicetree.org/schemas/timer/mrvl,mmp-timer.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell MMP Timer bindings
+title: Marvell MMP Timer
maintainers:
- Daniel Lezcano <daniel.lezcano@linaro.org>
$id: http://devicetree.org/schemas/timer/st,stm32-timer.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 general-purpose 16 and 32 bits timers bindings
+title: STMicroelectronics STM32 general-purpose 16 and 32 bits timers
maintainers:
- Fabrice Gasnier <fabrice.gasnier@foss.st.com>
$id: http://devicetree.org/schemas/trivial-devices.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Trivial I2C and SPI devices that have simple device tree bindings
+title: Trivial I2C and SPI devices
maintainers:
- Rob Herring <robh@kernel.org>
$id: http://devicetree.org/schemas/usb/analogix,anx7411.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Analogix ANX7411 Type-C controller bindings
+title: Analogix ANX7411 Type-C controller
maintainers:
- Xin Ji <xji@analogixsemi.com>
$id: http://devicetree.org/schemas/usb/cdns,usb3.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Cadence USBSS-DRD controller bindings
+title: Cadence USBSS-DRD controller
maintainers:
- Pawel Laszczak <pawell@cadence.com>
$id: http://devicetree.org/schemas/usb/dwc2.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: DesignWare HS OTG USB 2.0 controller Bindings
+title: DesignWare HS OTG USB 2.0 controller
maintainers:
- Rob Herring <robh@kernel.org>
- const: rockchip,rk3066-usb
- const: snps,dwc2
- const: lantiq,arx100-usb
+ - const: lantiq,ase-usb
+ - const: lantiq,danube-usb
- const: lantiq,xrx200-usb
+ - const: lantiq,xrx300-usb
- items:
- enum:
- amlogic,meson8-usb
$id: http://devicetree.org/schemas/usb/faraday,fotg210.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Faraday Technology FOTG210 HS OTG USB 2.0 controller Bindings
+title: Faraday Technology FOTG210 HS OTG USB 2.0 controller
maintainers:
- Linus Walleij <linus.walleij@linaro.org>
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0-only or BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/usb/genesys,gl850g.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Genesys Logic GL850G USB 2.0 hub controller
+
+maintainers:
+ - Icenowy Zheng <uwu@icenowy.me>
+
+allOf:
+ - $ref: usb-device.yaml#
+
+properties:
+ compatible:
+ enum:
+ - usb5e3,608
+
+ reg: true
+
+ reset-gpios:
+ description: GPIO controlling the RESET# pin.
+
+ vdd-supply:
+ description:
+ the regulator that provides 3.3V core power to the hub.
+
+required:
+ - compatible
+ - reg
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+ usb {
+ dr_mode = "host";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ hub: hub@1 {
+ compatible = "usb5e3,608";
+ reg = <1>;
+ reset-gpios = <&pio 7 2 GPIO_ACTIVE_LOW>;
+ };
+ };
$id: http://devicetree.org/schemas/usb/ingenic,musb.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Ingenic JZ47xx USB IP DT bindings
+title: Ingenic JZ47xx USB IP
maintainers:
- Paul Cercueil <paul@crapouillou.net>
$id: http://devicetree.org/schemas/usb/marvell,pxau2o-ehci.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Marvell PXA/MMP EHCI bindings
+title: Marvell PXA/MMP EHCI
maintainers:
- Lubomir Rintel <lkundrak@v3.sk>
$id: "http://devicetree.org/schemas/usb/maxim,max33359.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Maxim TCPCI Type-C PD controller DT bindings
+title: Maxim TCPCI Type-C PD controller
maintainers:
- Badhri Jagan Sridharan <badhri@google.com>
$id: "http://devicetree.org/schemas/usb/mediatek,mt6360-tcpc.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Mediatek MT6360 Type-C Port Switch and Power Delivery controller DT bindings
+title: Mediatek MT6360 Type-C Port Switch and Power Delivery controller
maintainers:
- ChiYuan Huang <cy_huang@richtek.com>
- mediatek,mt7622-xhci
- mediatek,mt7623-xhci
- mediatek,mt7629-xhci
+ - mediatek,mt7986-xhci
- mediatek,mt8173-xhci
- mediatek,mt8183-xhci
- mediatek,mt8186-xhci
- mediatek,mt2712-mtu3
- mediatek,mt8173-mtu3
- mediatek,mt8183-mtu3
+ - mediatek,mt8186-mtu3
- mediatek,mt8188-mtu3
- mediatek,mt8192-mtu3
- mediatek,mt8195-mtu3
$id: "http://devicetree.org/schemas/usb/nvidia,tegra-xudc.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Device tree binding for NVIDIA Tegra XUSB device mode controller (XUDC)
+title: NVIDIA Tegra XUSB device mode controller (XUDC)
description:
The Tegra XUDC controller supports both USB 2.0 HighSpeed/FullSpeed and
nvidia,xusb-padctl = <&padctl>;
- phys = <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-1}>, /* mini-PCIe USB */
- <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-2}>, /* USB A */
- <&{/padctl@0,7009f000/pads/pcie/lanes/pcie-0}>; /* USB A */
+ phys = <&phy_usb2_1>, <&phy_usb2_2>, <&phy_pcie_0>;
phy-names = "usb2-1", "usb2-2", "usb3-0";
avddio-pex-supply = <&vdd_1v05_run>;
#address-cells = <1>;
#size-cells = <0>;
- phys = <&{/padctl@3520000/pads/usb2/lanes/usb2-0}>,
- <&{/padctl@3520000/pads/usb2/lanes/usb2-1}>,
- <&{/padctl@3520000/pads/usb3/lanes/usb3-0}>;
+ phys = <&phy_usb2_0>, <&phy_usb2_1>, <&phy_usb3_0>;
phy-names = "usb2-0", "usb2-1", "usb3-0";
};
nvidia,xusb-padctl = <&xusb_padctl>;
- phys = <&{/bus@0/padctl@3520000/pads/usb2/lanes/usb2-0}>,
- <&{/bus@0/padctl@3520000/pads/usb2/lanes/usb2-1}>,
- <&{/bus@0/padctl@3520000/pads/usb2/lanes/usb2-3}>,
- <&{/bus@0/padctl@3520000/pads/usb3/lanes/usb3-0}>,
- <&{/bus@0/padctl@3520000/pads/usb3/lanes/usb3-2}>,
- <&{/bus@0/padctl@3520000/pads/usb3/lanes/usb3-3}>;
+ phys = <&phy_usb2_0>, <&phy_usb2_1>, <&phy_usb2_3>, <&phy_usb3_0>,
+ <&phy_usb3_2>, <&phy_usb3_3>;
phy-names = "usb2-0", "usb2-1", "usb2-3", "usb3-0", "usb3-2", "usb3-3";
};
nvidia,xusb-padctl = <&padctl>;
- phys = <&{/padctl@7009f000/pads/usb2/lanes/usb2-0}>,
- <&{/padctl@7009f000/pads/usb2/lanes/usb2-1}>,
- <&{/padctl@7009f000/pads/usb2/lanes/usb2-2}>,
- <&{/padctl@7009f000/pads/usb2/lanes/usb2-3}>,
- <&{/padctl@7009f000/pads/pcie/lanes/pcie-6}>,
- <&{/padctl@7009f000/pads/pcie/lanes/pcie-5}>;
+ phys = <&phy_usb2_0>, <&phy_usb2_1>, <&phy_usb2_2>, <&phy_usb2_3>,
+ <&phy_pcie_6>, <&phy_pcie_5>;
phy-names = "usb2-0", "usb2-1", "usb2-2", "usb2-3", "usb3-0",
"usb3-1";
dvddio-pex-supply = <&vdd_pex_1v05>;
$id: http://devicetree.org/schemas/usb/nxp,isp1760.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: NXP ISP1760 family controller bindings
+title: NXP ISP1760 family controller
maintainers:
- Sebastian Siewior <bigeasy@linutronix.de>
- qcom,sm8250-dwc3
- qcom,sm8350-dwc3
- qcom,sm8450-dwc3
+ - qcom,sm8550-dwc3
- const: qcom,dwc3
reg:
- qcom,sm8150-dwc3
- qcom,sm8250-dwc3
- qcom,sm8450-dwc3
+ - qcom,sm8550-dwc3
then:
properties:
clocks:
- qcom,sm8250-dwc3
- qcom,sm8350-dwc3
- qcom,sm8450-dwc3
+ - qcom,sm8550-dwc3
then:
properties:
interrupts:
$id: http://devicetree.org/schemas/usb/realtek,rts5411.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for the Realtek RTS5411 USB 3.0 hub controller
+title: Realtek RTS5411 USB 3.0 hub controller
maintainers:
- Matthias Kaehlcke <mka@chromium.org>
$id: "http://devicetree.org/schemas/usb/richtek,rt1719.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Richtek RT1719 sink-only Type-C PD controller bindings
+title: Richtek RT1719 sink-only Type-C PD controller
maintainers:
- ChiYuan Huang <cy_huang@richtek.com>
$id: "http://devicetree.org/schemas/usb/st,stusb160x.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: STMicroelectronics STUSB160x Type-C controller bindings
+title: STMicroelectronics STUSB160x Type-C controller
maintainers:
- Amelie Delaunay <amelie.delaunay@foss.st.com>
$id: "http://devicetree.org/schemas/usb/ti,j721e-usb.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Bindings for the TI wrapper module for the Cadence USBSS-DRD controller
+title: TI wrapper module for the Cadence USBSS-DRD controller
maintainers:
- Roger Quadros <rogerq@kernel.org>
$id: "http://devicetree.org/schemas/usb/ti,tps6598x.yaml#"
$schema: "http://devicetree.org/meta-schemas/core.yaml#"
-title: Texas Instruments 6598x Type-C Port Switch and Power Delivery controller DT bindings
+title: Texas Instruments 6598x Type-C Port Switch and Power Delivery controller
maintainers:
- Bryan O'Donoghue <bryan.odonoghue@linaro.org>
$id: http://devicetree.org/schemas/usb/ti,usb8041.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Binding for the TI USB8041 USB 3.0 hub controller
+title: TI USB8041 USB 3.0 hub controller
maintainers:
- Alexander Stein <alexander.stein@ew.tq-group.com>
$id: http://devicetree.org/schemas/usb/usb-device.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: The device tree bindings for the Generic USB Device
+title: Generic USB Device
maintainers:
- Greg Kroah-Hartman <gregkh@linuxfoundation.org>
should default to OTG.
$ref: /schemas/types.yaml#/definitions/string
enum: [host, peripheral, otg]
+ default: otg
hnp-disable:
description:
the VBus line.
$ref: /schemas/types.yaml#/definitions/phandle
+ wakeup-source:
+ description:
+ Specify if the USB phy can detect the remote wakeup signal
+ while the system sleep.
+
required:
- compatible
- '#phy-cells'
+++ /dev/null
-Microchip USB 2.0 Hi-Speed Hub Controller
-
-The device node for the configuration of a Microchip USB251x/xBi USB 2.0
-Hi-Speed Controller.
-
-Required properties :
- - compatible : Should be "microchip,usb251xb" or one of the specific types:
- "microchip,usb2512b", "microchip,usb2512bi", "microchip,usb2513b",
- "microchip,usb2513bi", "microchip,usb2514b", "microchip,usb2514bi",
- "microchip,usb2517", "microchip,usb2517i", "microchip,usb2422"
- - reg : I2C address on the selected bus (default is <0x2C>)
-
-Optional properties :
- - reset-gpios : Should specify the gpio for hub reset
- - vdd-supply : Should specify the phandle to the regulator supplying vdd
- - skip-config : Skip Hub configuration, but only send the USB-Attach command
- - vendor-id : Set USB Vendor ID of the hub (16 bit, default is 0x0424)
- - product-id : Set USB Product ID of the hub (16 bit, default depends on type)
- - device-id : Set USB Device ID of the hub (16 bit, default is 0x0bb3)
- - language-id : Set USB Language ID (16 bit, default is 0x0000)
- - manufacturer : Set USB Manufacturer string (max 31 characters long)
- - product : Set USB Product string (max 31 characters long)
- - serial : Set USB Serial string (max 31 characters long)
- - {bus,self}-powered : selects between self- and bus-powered operation
- (boolean, default is self-powered)
- - disable-hi-speed : disable USB Hi-Speed support (boolean)
- - {multi,single}-tt : selects between multi- and single-transaction-translator
- (boolean, default is multi-tt)
- - disable-eop : disable End of Packet generation in full-speed mode (boolean)
- - {ganged,individual}-sensing : select over-current sense type in self-powered
- mode (boolean, default is individual)
- - {ganged,individual}-port-switching : select port power switching mode
- (boolean, default is individual)
- - dynamic-power-switching : enable auto-switching from self- to bus-powered
- operation if the local power source is removed or unavailable (boolean)
- - oc-delay-us : Delay time (in microseconds) for filtering the over-current
- sense inputs. Valid values are 100, 4000, 8000 (default) and 16000. If
- an invalid value is given, the default is used instead.
- - compound-device : indicate the hub is part of a compound device (boolean)
- - port-mapping-mode : enable port mapping mode (boolean)
- - led-{usb,speed}-mode : led usb/speed indication mode selection
- (boolean, default is speed mode)
- - string-support : enable string descriptor support (required for manufacturer,
- product and serial string configuration)
- - non-removable-ports : Should specify the ports which have a non-removable
- device connected.
- - sp-disabled-ports : Specifies the ports which will be self-power disabled
- - bp-disabled-ports : Specifies the ports which will be bus-power disabled
- - sp-max-total-current-microamp: Specifies max current consumed by the hub
- from VBUS when operating in self-powered hub. It includes the hub
- silicon along with all associated circuitry including a permanently
- attached peripheral (range: 0 - 100000 uA, default 1000 uA)
- - bp-max-total-current-microamp: Specifies max current consumed by the hub
- from VBUS when operating in self-powered hub. It includes the hub
- silicon along with all associated circuitry including a permanently
- attached peripheral (range: 0 - 510000 uA, default 100000 uA)
- - sp-max-removable-current-microamp: Specifies max current consumed by the hub
- from VBUS when operating in self-powered hub. It includes the hub
- silicon along with all associated circuitry excluding a permanently
- attached peripheral (range: 0 - 100000 uA, default 1000 uA)
- - bp-max-removable-current-microamp: Specifies max current consumed by the hub
- from VBUS when operating in self-powered hub. It includes the hub
- silicon along with all associated circuitry excluding a permanently
- attached peripheral (range: 0 - 510000 uA, default 100000 uA)
- - power-on-time-ms : Specifies the time it takes from the time the host
- initiates the power-on sequence to a port until the port has adequate
- power. The value is given in ms in a 0 - 510 range (default is 100ms).
- - swap-dx-lanes : Specifies the ports which will swap the differential-pair
- (D+/D-), default is not-swapped.
-
-Examples:
- usb2512b@2c {
- compatible = "microchip,usb2512b";
- reg = <0x2c>;
- reset-gpios = <&gpio1 4 GPIO_ACTIVE_LOW>;
- };
-
- usb2514b@2c {
- compatible = "microchip,usb2514b";
- reg = <0x2c>;
- vendor-id = /bits/ 16 <0x0000>;
- product-id = /bits/ 16 <0x0000>;
- string-support;
- manufacturer = "Foo";
- product = "Foo-Bar";
- serial = "1234567890A";
- /* correct misplaced usb connectors on port 1,2 */
- swap-dx-lanes = <1 2>;
- };
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/usb/usb251xb.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Microchip USB 2.0 Hi-Speed Hub Controller
+
+maintainers:
+ - Richard Leitner <richard.leitner@skidata.com>
+
+properties:
+ compatible:
+ enum:
+ - microchip,usb2422
+ - microchip,usb2512b
+ - microchip,usb2512bi
+ - microchip,usb2513b
+ - microchip,usb2513bi
+ - microchip,usb2514b
+ - microchip,usb2514bi
+ - microchip,usb2517
+ - microchip,usb2517i
+ - microchip,usb251xb
+
+ reg:
+ maxItems: 1
+
+ reset-gpios:
+ description: |
+ Should specify the gpio for hub reset
+
+ vdd-supply:
+ description: |
+ Should specify the phandle to the regulator supplying vdd
+
+ skip-config:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ Skip Hub configuration, but only send the USB-Attach command
+
+ vendor-id:
+ $ref: /schemas/types.yaml#/definitions/uint16
+ default: 0x0424
+ description: |
+ Set USB Vendor ID of the hub
+
+ product-id:
+ $ref: /schemas/types.yaml#/definitions/uint16
+ description: |
+ Set USB Product ID of the hub
+
+ device-id:
+ $ref: /schemas/types.yaml#/definitions/uint16
+ default: 0x0bb3
+ description: |
+ Set USB Device ID of the hub
+
+ language-id:
+ $ref: /schemas/types.yaml#/definitions/uint16
+ default: 0x0000
+ description: |
+ Set USB Language ID
+
+ manufacturer:
+ $ref: /schemas/types.yaml#/definitions/string
+ description: |
+ Set USB Manufacturer string (max 31 characters long)
+
+ product:
+ $ref: /schemas/types.yaml#/definitions/string
+ description: |
+ Set USB Product string (max 31 characters long)
+
+ serial:
+ $ref: /schemas/types.yaml#/definitions/string
+ description: |
+ Set USB Serial string (max 31 characters long)
+
+ bus-powered:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ selects between self- and bus-powered operation
+ (boolean, default is self-powered)
+
+ self-powered:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ selects between self- and bus-powered operation
+ (boolean, default is self-powered)
+
+ disable-hi-speed:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ disable USB Hi-Speed support (boolean)
+
+ multi-tt:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ selects between multi- and single-transaction-translator
+ (boolean, default is multi-tt)
+
+ single-tt:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ selects between multi- and single-transaction-translator
+ (boolean, default is multi-tt)
+
+ disable-eop:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ disable End of Packet generation in full-speed mode (boolean)
+
+ ganged-sensing:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ select over-current sense type in self-powered mode
+ (boolean, default is individual)
+
+ individual-sensing:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ select over-current sense type in self-powered mode
+ (boolean, default is individual)
+
+ ganged-port-switching:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ select port power switching mode (boolean, default is individual)
+
+ individual-port-switching:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ select port power switching mode (boolean, default is individual)
+
+ dynamic-power-switching:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ enable auto-switching from self- to bus-powered operation if the
+ local power source is removed or unavailable (boolean)
+
+ oc-delay-us:
+ enum: [100, 4000, 8000, 16000]
+ default: 8000
+ description: |
+ Delay time (in microseconds) for filtering the over-current sense
+ inputs. If an invalid value is given, the default is used instead.
+
+ compound-device:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ indicate the hub is part of a compound device (boolean)
+
+ port-mapping-mode:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ enable port mapping mode (boolean)
+
+ led-usb-mode:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ led usb/speed indication mode selection (boolean, default is speed mode)
+
+ led-speed-mode:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ led usb/speed indication mode selection (boolean, default is speed mode)
+
+ string-support:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description: |
+ enable string descriptor support (required for manufacturer, product
+ and serial string configuration)
+
+ non-removable-ports:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description: |
+ Should specify the ports which have a non-removable device connected.
+
+ sp-disabled-ports:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description: |
+ Specifies the ports which will be self-power disabled
+
+ bp-disabled-ports:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description: |
+ Specifies the ports which will be bus-power disabled
+
+ sp-max-total-current-microamp:
+ maximum: 100000
+ default: 1000
+ description: |
+ Specifies max current consumed by the hub from VBUS when
+ operating in self-powered hub. It includes the hub silicon
+ along with all associated circuitry including a permanently
+ attached peripheral.
+
+ bp-max-total-current-microamp:
+ maximum: 510000
+ default: 100000
+ description: |
+ Specifies max current consumed by the hub from VBUS when
+ operating in self-powered hub. It includes the hub silicon
+ along with all associated circuitry including a permanently
+ attached peripheral.
+
+ sp-max-removable-current-microamp:
+ maximum: 100000
+ default: 1000
+ description: |
+ Specifies max current consumed by the hub from VBUS when
+ operating in self-powered hub. It includes the hub silicon
+ along with all associated circuitry excluding a permanently
+ attached peripheral.
+
+ bp-max-removable-current-microamp:
+ maximum: 510000
+ default: 100000
+ description: |
+ Specifies max current consumed by the hub from VBUS when
+ operating in self-powered hub. It includes the hub silicon
+ along with all associated circuitry excluding a permanently
+ attached peripheral.
+
+ power-on-time-ms:
+ maximum: 510
+ default: 100
+ description: |
+ Specifies the time it takes from the time the host initiates the
+ power-on sequence to a port until the port has adequate power.
+
+ swap-dx-lanes:
+ $ref: /schemas/types.yaml#/definitions/uint8-array
+ description: |
+ Specifies the ports which will swap the differential-pair (D+/D-),
+ default is not-swapped.
+
+additionalProperties: false
+
+required:
+ - compatible
+ - reg
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+
+ i2c {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ usb-hub@2c {
+ compatible = "microchip,usb2512b";
+ reg = <0x2c>;
+ reset-gpios = <&gpio1 4 GPIO_ACTIVE_LOW>;
+ };
+
+ usb-hub@2d {
+ compatible = "microchip,usb2514b";
+ reg = <0x2d>;
+ vendor-id = /bits/ 16 <0x0000>;
+ product-id = /bits/ 16 <0x0000>;
+ string-support;
+ manufacturer = "Foo";
+ product = "Foo-Bar";
+ serial = "1234567890A";
+ /* correct misplaced usb connectors on port 1,2 */
+ swap-dx-lanes = <1 2>;
+ };
+ };
$id: http://devicetree.org/schemas/usb/willsemi,wusb3801.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: WUSB3801 Type-C port controller DT bindings
+title: WUSB3801 Type-C port controller
description:
The Will Semiconductor WUSB3801 is a USB Type-C port controller which
description: Bosch Sensortec GmbH
"^boundary,.*":
description: Boundary Devices Inc.
- "^broadmobi,.*":
- description: Shanghai Broadmobi Communication Technology Co.,Ltd.
"^brcm,.*":
description: Broadcom Corporation
+ "^broadmobi,.*":
+ description: Shanghai Broadmobi Communication Technology Co.,Ltd.
"^bsh,.*":
description: BSH Hausgeraete GmbH
"^bticino,.*":
description: GE Fanuc Intelligent Platforms Embedded Systems, Inc.
"^gemei,.*":
description: Gemei Digital Technology Co., Ltd.
+ "^genesys,.*":
+ description: Genesys Logic, Inc.
"^geniatech,.*":
description: Geniatech, Inc.
"^giantec,.*":
description: Hitex Development Tools
"^holt,.*":
description: Holt Integrated Circuits, Inc.
+ "^holtek,.*":
+ description: Holtek Semiconductor, Inc.
"^honestar,.*":
description: Honestar Technologies Co., Ltd.
"^honeywell,.*":
description: Hewlett Packard Enterprise
"^hsg,.*":
description: HannStar Display Co.
- "^holtek,.*":
- description: Holtek Semiconductor, Inc.
"^huawei,.*":
description: Huawei Technologies Co., Ltd.
"^hugsun,.*":
description: Infineon Technologies
"^inforce,.*":
description: Inforce Computing
- "^ingrasys,.*":
- description: Ingrasys Technology Inc.
- "^ivo,.*":
- description: InfoVision Optoelectronics Kunshan Co. Ltd.
"^ingenic,.*":
description: Ingenic Semiconductor
+ "^ingrasys,.*":
+ description: Ingrasys Technology Inc.
"^injoinic,.*":
description: Injoinic Technology Corp.
"^innocomm,.*":
description: ITEAD Intelligent Systems Co.Ltd
"^itian,.*":
description: ITian Corporation
+ "^ivo,.*":
+ description: InfoVision Optoelectronics Kunshan Co. Ltd.
"^iwave,.*":
description: iWave Systems Technologies Pvt. Ltd.
"^jadard,.*":
description: Shenzhen Netxeon Technology CO., LTD
"^neweast,.*":
description: Guangdong Neweast Optoelectronics CO., LTD
+ "^newhaven,.*":
+ description: Newhaven Display International
"^newvision,.*":
description: New Vision Display (Shenzhen) Co., Ltd.
"^nexbox,.*":
description: Nexbox
"^nextthing,.*":
description: Next Thing Co.
- "^newhaven,.*":
- description: Newhaven Display International
"^ni,.*":
description: National Instruments
"^nintendo,.*":
description: One Laptop Per Child
"^oneplus,.*":
description: OnePlus Technology (Shenzhen) Co., Ltd.
+ "^onie,.*":
+ description: Open Network Install Environment group
"^onion,.*":
description: Onion Corporation
"^onnn,.*":
description: QEMU, a generic and open source machine emulator and virtualizer
"^qi,.*":
description: Qi Hardware
- "^qihua,.*":
- description: Chengdu Kaixuan Information Technology Co., Ltd.
"^qiaodian,.*":
description: QiaoDian XianShi Corporation
+ "^qihua,.*":
+ description: Chengdu Kaixuan Information Technology Co., Ltd.
"^qishenglong,.*":
description: Shenzhen QiShenglong Industrialist Co., Ltd.
"^qnap,.*":
description: reMarkable AS
"^renesas,.*":
description: Renesas Electronics Corporation
- "^rex,.*":
- description: iMX6 Rex Project
"^rervision,.*":
description: Shenzhen Rervision Technology Co., Ltd.
"^revotics,.*":
description: Revolution Robotics, Inc. (Revotics)
+ "^rex,.*":
+ description: iMX6 Rex Project
"^richtek,.*":
description: Richtek Technology Corporation
"^ricoh,.*":
description: Ricoh Co. Ltd.
"^rikomagic,.*":
description: Rikomagic Tech Corp. Ltd
- "^riscv,.*":
- description: RISC-V Foundation
"^riot,.*":
description: Embest RIoT
+ "^riscv,.*":
+ description: RISC-V Foundation
"^rockchip,.*":
description: Fuzhou Rockchip Electronics Co., Ltd
"^rocktech,.*":
description: Si-En Technology Ltd.
"^si-linux,.*":
description: Silicon Linux Corporation
+ "^siemens,.*":
+ description: Siemens AG
"^sifive,.*":
description: SiFive, Inc.
"^sigma,.*":
description: Siliconfile Technologies lnc.
"^siliconmitus,.*":
description: Silicon Mitus, Inc.
- "^siemens,.*":
- description: Siemens AG
+ "^silvaco,.*":
+ description: Silvaco, Inc.
"^simtek,.*":
description: Cypress Semiconductor Corporation (Simtek Corporation)
"^sinlinx,.*":
description: Sun Microsystems, Inc
"^supermicro,.*":
description: Super Micro Computer, Inc.
- "^silvaco,.*":
- description: Silvaco, Inc.
"^swir,.*":
description: Sierra Wireless
"^syna,.*":
description: Shenzhen City Tang Cheng Technology Co., Ltd.
"^tdo,.*":
description: Shangai Top Display Optoelectronics Co., Ltd
+ "^team-source-display,.*":
+ description: Shenzhen Team Source Display Technology Co., Ltd. (TSD)
"^technexion,.*":
description: TechNexion
"^technologic,.*":
description: Technologic Systems
+ "^techstar,.*":
+ description: Shenzhen Techstar Electronics Co., Ltd.
"^teltonika,.*":
description: Teltonika Networks
"^tempo,.*":
description: Tempo Semiconductor
- "^techstar,.*":
- description: Shenzhen Techstar Electronics Co., Ltd.
"^terasic,.*":
description: Terasic Inc.
"^tesla,.*":
description: Tronsmart
"^truly,.*":
description: Truly Semiconductors Limited
- "^visionox,.*":
- description: Visionox
- "^team-source-display,.*":
- description: Shenzhen Team Source Display Technology Co., Ltd. (TSD)
"^tsd,.*":
description: Theobroma Systems Design und Consulting GmbH
"^tyan,.*":
description: u-blox
"^u-boot,.*":
description: U-Boot bootloader
- "^ucrobotics,.*":
- description: uCRobotics
"^ubnt,.*":
description: Ubiquiti Networks
+ "^ucrobotics,.*":
+ description: uCRobotics
"^udoo,.*":
description: Udoo
"^ugoos,.*":
description: Used for virtual device without specific vendor.
"^vishay,.*":
description: Vishay Intertechnology, Inc
+ "^visionox,.*":
+ description: Visionox
"^vitesse,.*":
description: Vitesse Semiconductor Corporation
"^vivante,.*":
description: Vision Optical Technology Co., Ltd.
"^vxt,.*":
description: VXT Ltd
+ "^wanchanglong,.*":
+ description: Wanchanglong Electronics Technology（SHENZHEN）Co.，Ltd.
"^wand,.*":
description: Wandbord (Technexion)
"^waveshare,.*":
description: Wondermedia Technologies, Inc.
"^wobo,.*":
description: Wobo
- "^wanchanglong,.*":
- description: Wanchanglong Electronics Technology（SHENZHEN）Co.，Ltd.
"^x-powers,.*":
description: X-Powers
"^xen,.*":
description: Shenzhen Yashi Changhua Intelligent Technology Co., Ltd.
"^ysoft,.*":
description: Y Soft Corporation a.s.
- "^zealz,.*":
- description: Zealz
"^zarlink,.*":
description: Zarlink Semiconductor
+ "^zealz,.*":
+ description: Zealz
"^zeitec,.*":
description: ZEITEC Semiconductor Co., LTD.
"^zidoo,.*":
$id: http://devicetree.org/schemas/virtio/virtio-device.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Virtio device bindings
+title: Virtio device
maintainers:
- Viresh Kumar <viresh.kumar@linaro.org>
$id: http://devicetree.org/schemas/watchdog/fsl,scu-wdt.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: i.MX SCU Client Device Node - Watchdog bindings based on SCU Message Protocol
+title: i.MX SCU Client Device Node - Watchdog Based on SCU Message Protocol
maintainers:
- Dong Aisheng <aisheng.dong@nxp.com>
+++ /dev/null
-* GPIO-controlled Watchdog
-
-Required Properties:
-- compatible: Should contain "linux,wdt-gpio".
-- gpios: From common gpio binding; gpio connection to WDT reset pin.
-- hw_algo: The algorithm used by the driver. Should be one of the
- following values:
- - toggle: Either a high-to-low or a low-to-high transition clears
- the WDT counter. The watchdog timer is disabled when GPIO is
- left floating or connected to a three-state buffer.
- - level: Low or high level starts counting WDT timeout,
- the opposite level disables the WDT. Active level is determined
- by the GPIO flags.
-- hw_margin_ms: Maximum time to reset watchdog circuit (milliseconds).
-
-Optional Properties:
-- always-running: If the watchdog timer cannot be disabled, add this flag to
- have the driver keep toggling the signal without a client. It will only cease
- to toggle the signal when the device is open and the timeout elapsed.
-
-Example:
- watchdog: watchdog {
- /* ADM706 */
- compatible = "linux,wdt-gpio";
- gpios = <&gpio3 9 GPIO_ACTIVE_LOW>;
- hw_algo = "toggle";
- hw_margin_ms = <1600>;
- };
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/watchdog/linux,wdt-gpio.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: GPIO-controlled Watchdog
+
+maintainers:
+ - Guenter Roeck <linux@roeck-us.net>
+
+properties:
+ compatible:
+ const: linux,wdt-gpio
+
+ gpios:
+ description: gpio connection to WDT reset pin
+ maxItems: 1
+
+ hw_algo:
+ description: The algorithm used by the driver.
+ enum: [ level, toggle ]
+
+ hw_margin_ms:
+ description: Maximum time to reset watchdog circuit (milliseconds).
+ $ref: /schemas/types.yaml#/definitions/uint32
+
+ always-running:
+ type: boolean
+ description:
+ If the watchdog timer cannot be disabled, add this flag to have the driver
+ keep toggling the signal without a client.
+ It will only cease to toggle the signal when the device is open and the
+ timeout elapsed.
+
+required:
+ - compatible
+ - gpios
+ - hw_algo
+ - hw_margin_ms
+
+allOf:
+ - $ref: watchdog.yaml#
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+ watchdog {
+ compatible = "linux,wdt-gpio";
+ gpios = <&gpio3 9 GPIO_ACTIVE_LOW>;
+ hw_algo = "toggle";
+ hw_margin_ms = <1600>;
+ };
--- /dev/null
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/watchdog/mediatek,mtk-wdt.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: MediaTek SoCs Watchdog timer
+
+maintainers:
+ - Matthias Brugger <matthias.bgg@gmail.com>
+
+description:
+ The watchdog supports a pre-timeout interrupt that fires
+ timeout-sec/2 before the expiry.
+
+allOf:
+ - $ref: watchdog.yaml#
+
+properties:
+ compatible:
+ oneOf:
+ - enum:
+ - mediatek,mt2712-wdt
+ - mediatek,mt6589-wdt
+ - mediatek,mt6795-wdt
+ - mediatek,mt7986-wdt
+ - mediatek,mt8183-wdt
+ - mediatek,mt8186-wdt
+ - mediatek,mt8188-wdt
+ - mediatek,mt8192-wdt
+ - mediatek,mt8195-wdt
+ - items:
+ - enum:
+ - mediatek,mt2701-wdt
+ - mediatek,mt6582-wdt
+ - mediatek,mt6797-wdt
+ - mediatek,mt7622-wdt
+ - mediatek,mt7623-wdt
+ - mediatek,mt7629-wdt
+ - mediatek,mt8173-wdt
+ - mediatek,mt8516-wdt
+ - const: mediatek,mt6589-wdt
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ items:
+ - description: Watchdog pre-timeout (bark) interrupt
+
+ mediatek,disable-extrst:
+ description: Disable sending output reset signal
+ type: boolean
+
+ '#reset-cells':
+ const: 1
+
+required:
+ - compatible
+ - reg
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ watchdog: watchdog@10007000 {
+ compatible = "mediatek,mt8183-wdt";
+ reg = <0 0x10007000 0 0x100>;
+ interrupts = <GIC_SPI 139 IRQ_TYPE_LEVEL_LOW>;
+ mediatek,disable-extrst;
+ timeout-sec = <10>;
+ #reset-cells = <1>;
+ };
+ };
+++ /dev/null
-Mediatek SoCs Watchdog timer
-
-The watchdog supports a pre-timeout interrupt that fires timeout-sec/2
-before the expiry.
-
-Required properties:
-
-- compatible should contain:
- "mediatek,mt2701-wdt", "mediatek,mt6589-wdt": for MT2701
- "mediatek,mt2712-wdt": for MT2712
- "mediatek,mt6582-wdt", "mediatek,mt6589-wdt": for MT6582
- "mediatek,mt6589-wdt": for MT6589
- "mediatek,mt6797-wdt", "mediatek,mt6589-wdt": for MT6797
- "mediatek,mt7622-wdt", "mediatek,mt6589-wdt": for MT7622
- "mediatek,mt7623-wdt", "mediatek,mt6589-wdt": for MT7623
- "mediatek,mt7629-wdt", "mediatek,mt6589-wdt": for MT7629
- "mediatek,mt7986-wdt", "mediatek,mt6589-wdt": for MT7986
- "mediatek,mt8183-wdt": for MT8183
- "mediatek,mt8186-wdt", "mediatek,mt6589-wdt": for MT8186
- "mediatek,mt8516-wdt", "mediatek,mt6589-wdt": for MT8516
- "mediatek,mt8192-wdt": for MT8192
- "mediatek,mt8195-wdt", "mediatek,mt6589-wdt": for MT8195
-
-- reg : Specifies base physical address and size of the registers.
-
-Optional properties:
-- mediatek,disable-extrst: disable send output reset signal
-- interrupts: Watchdog pre-timeout (bark) interrupt.
-- timeout-sec: contains the watchdog timeout in seconds.
-- #reset-cells: Should be 1.
-
-Example:
-
-watchdog: watchdog@10007000 {
- compatible = "mediatek,mt8183-wdt",
- "mediatek,mt6589-wdt";
- mediatek,disable-extrst;
- reg = <0 0x10007000 0 0x100>;
- interrupts = <GIC_SPI 139 IRQ_TYPE_NONE>;
- timeout-sec = <10>;
- #reset-cells = <1>;
-};
$id: http://devicetree.org/schemas/watchdog/st,stm32-iwdg.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: STMicroelectronics STM32 Independent WatchDoG (IWDG) bindings
+title: STMicroelectronics STM32 Independent WatchDoG (IWDG)
maintainers:
- Yannick Fertre <yannick.fertre@foss.st.com>
$id: http://devicetree.org/schemas/watchdog/watchdog.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
-title: Watchdog Generic Bindings
+title: Watchdog Common Properties
maintainers:
- Guenter Roeck <linux@roeck-us.net>
devm_kmemdup()
devm_krealloc()
devm_kstrdup()
+ devm_kstrdup_const()
devm_kvasprintf()
devm_kzalloc()
PHY
devm_usb_get_phy()
+ devm_usb_get_phy_by_node()
+ devm_usb_get_phy_by_phandle()
devm_usb_put_phy()
PINCTRL
SLAVE DMA ENGINE
devm_acpi_dma_controller_register()
+ devm_acpi_dma_controller_free()
SPI
devm_spi_alloc_master()
can use phy_set_drvdata() after creating the PHY and use phy_get_drvdata() in
phy_ops to get back the private data.
-4. Getting a reference to the PHY
+Getting a reference to the PHY
+==============================
Before the controller can make use of the PHY, it has to get a reference to
it. This framework provides the following APIs to get a reference to the PHY.
phy_power_off() calls are all NOP when applied to a NULL phy. The NULL
phy is useful in devices for handling optional phy devices.
+Order of API calls
+==================
+
+The general order of calls should be::
+
+ [devm_][of_]phy_get()
+ phy_init()
+ phy_power_on()
+ [phy_set_mode[_ext]()]
+ ...
+ phy_power_off()
+ phy_exit()
+ [[of_]phy_put()]
+
+Some PHY drivers may not implement :c:func:`phy_init` or :c:func:`phy_power_on`,
+but controllers should always call these functions to be compatible with other
+PHYs. Some PHYs may require :c:func:`phy_set_mode <phy_set_mode_ext>`, while
+others may use a default mode (typically configured via devicetree or other
+firmware). For compatibility, you should always call this function if you know
+what mode you will be using. Generally, this function should be called after
+:c:func:`phy_power_on`, although some PHY drivers may allow it at any time.
+
Releasing a reference to the PHY
================================
uart_get_lsr_info uart_handle_dcd_change uart_handle_cts_change
uart_try_toggle_sysrq uart_get_console
+.. kernel-doc:: include/linux/serial_core.h
+ :identifiers: uart_port_tx_limited uart_port_tx
+
Other notes
-----------
3. Data Structures Already Available in the Kernel
==================================================
- The Linux kernel provides the serial_rs485 structure (see [1]) to handle
- RS485 communications. This data structure is used to set and configure RS485
+ The Linux kernel provides the struct serial_rs485 to handle RS485
+ communications. This data structure is used to set and configure RS485
parameters in the platform data and in ioctls.
- The device tree can also provide RS485 boot time parameters (see [2]
- for bindings). The driver is in charge of filling this data structure from
- the values given by the device tree.
+ The device tree can also provide RS485 boot time parameters
+ [#DT-bindings]_. The serial core fills the struct serial_rs485 from the
+ values given by the device tree when the driver calls
+ uart_get_rs485_mode().
Any driver for devices capable of working both as RS232 and RS485 should
- implement the rs485_config callback and provide rs485_supported in the
- uart_port structure. The serial core calls rs485_config to do the device
- specific part in response to TIOCSRS485 ioctl (see below). The rs485_config
- callback receives a pointer to a sanitizated serial_rs485 structure. The
- serial_rs485 userspace provides is sanitized before calling rs485_config
- using rs485_supported that indicates what RS485 features the driver supports
- for the uart_port. TIOCGRS485 ioctl can be used to read back the
- serial_rs485 structure matching to the current configuration.
+ implement the ``rs485_config`` callback and provide ``rs485_supported``
+ in the ``struct uart_port``. The serial core calls ``rs485_config`` to do
+ the device specific part in response to TIOCSRS485 ioctl (see below). The
+ ``rs485_config`` callback receives a pointer to a sanitizated struct
+ serial_rs485. The struct serial_rs485 userspace provides is sanitized
+ before calling ``rs485_config`` using ``rs485_supported`` that indicates
+ what RS485 features the driver supports for the ``struct uart_port``.
+ TIOCGRS485 ioctl can be used to read back the struct serial_rs485
+ matching to the current configuration.
+
+.. kernel-doc:: include/uapi/linux/serial.h
+ :identifiers: serial_rs485 uart_get_rs485_mode
4. Usage from user-level
========================
========================
The Linux kernel provides addressing mode for multipoint RS-485 serial
- communications line. The addressing mode is enabled with SER_RS485_ADDRB
- flag in serial_rs485. Struct serial_rs485 has two additional flags and
- fields for enabling receive and destination addresses.
+ communications line. The addressing mode is enabled with
+ ``SER_RS485_ADDRB`` flag in struct serial_rs485. The struct serial_rs485
+ has two additional flags and fields for enabling receive and destination
+ addresses.
Address mode flags:
- - SER_RS485_ADDRB: Enabled addressing mode (sets also ADDRB in termios).
- - SER_RS485_ADDR_RECV: Receive (filter) address enabled.
- - SER_RS485_ADDR_DEST: Set destination address.
+ - ``SER_RS485_ADDRB``: Enabled addressing mode (sets also ADDRB in termios).
+ - ``SER_RS485_ADDR_RECV``: Receive (filter) address enabled.
+ - ``SER_RS485_ADDR_DEST``: Set destination address.
- Address fields (enabled with corresponding SER_RS485_ADDR_* flag):
- - addr_recv: Receive address.
- - addr_dest: Destination address.
+ Address fields (enabled with corresponding ``SER_RS485_ADDR_*`` flag):
+ - ``addr_recv``: Receive address.
+ - ``addr_dest``: Destination address.
Once a receive address is set, the communication can occur only with the
particular device and other peers are filtered out. It is left up to the
receiver side to enforce the filtering. Receive address will be cleared
- if SER_RS485_ADDR_RECV is not set.
+ if ``SER_RS485_ADDR_RECV`` is not set.
Note: not all devices supporting RS485 support multipoint addressing.
6. References
=============
- [1] include/uapi/linux/serial.h
-
- [2] Documentation/devicetree/bindings/serial/rs485.txt
+.. [#DT-bindings] Documentation/devicetree/bindings/serial/rs485.txt
Note: Applied to empty files, this allows to switch type between
sparse(0x200), compressed(0x800) and normal.
+ - *system.ntfs_attrib_be* gets/sets ntfs file/dir attributes.
+
+ Same value as system.ntfs_attrib but always represent as big-endian
+ (endianness of system.ntfs_attrib is the same as of the CPU).
+
Mount Options
=============
- Files with the Windows-specific SYSTEM (FILE_ATTRIBUTE_SYSTEM) attribute
will be marked as system immutable files.
+ * - hide_dot_files
+ - Updates the Windows-specific HIDDEN (FILE_ATTRIBUTE_HIDDEN) attribute
+ when creating and moving or renaming files. Files whose names start
+ with a dot will have the HIDDEN attribute set and files whose names
+ do not start with a dot will have it unset.
+
+ * - windows_names
+ - Prevents the creation of files and directories with a name not allowed
+ by Windows, either because it contains some not allowed character (which
+ are the characters " * / : < > ? \\ | and those whose code is less than
+ 0x20), because the name (with or without extension) is a reserved file
+ name (CON, AUX, NUL, PRN, LPT1-9, COM1-9) or because the last character
+ is a space or a dot. Existing such files can still be read and renamed.
+
* - discard
- Enable support of the TRIM command for improved performance on delete
operations, which is recommended for use with the solid-state drives
:maxdepth: 1
bnxt
+ etas_es58x
hns3
ionic
ice
Enable connection tracking flow timestamping.
+nf_conntrack_sctp_timeout_closed - INTEGER (seconds)
+ default 10
+
+nf_conntrack_sctp_timeout_cookie_wait - INTEGER (seconds)
+ default 3
+
+nf_conntrack_sctp_timeout_cookie_echoed - INTEGER (seconds)
+ default 3
+
+nf_conntrack_sctp_timeout_established - INTEGER (seconds)
+ default 432000 (5 days)
+
+nf_conntrack_sctp_timeout_shutdown_sent - INTEGER (seconds)
+ default 0.3
+
+nf_conntrack_sctp_timeout_shutdown_recd - INTEGER (seconds)
+ default 0.3
+
+nf_conntrack_sctp_timeout_shutdown_ack_sent - INTEGER (seconds)
+ default 3
+
+nf_conntrack_sctp_timeout_heartbeat_sent - INTEGER (seconds)
+ default 30
+
+ This timeout is used to setup conntrack entry on secondary paths.
+ Default is set to hb_interval.
+
+nf_conntrack_sctp_timeout_heartbeat_acked - INTEGER (seconds)
+ default 210
+
+ This timeout is used to setup conntrack entry on secondary paths.
+ Default is set to (hb_interval * path_max_retrans + rto_max)
+
nf_conntrack_udp_timeout - INTEGER (seconds)
default 30
------------------
- Hash MMU (except 603 and e300)
+- Radix MMU (POWER9 and later)
- Software loaded TLB (603 and e300)
- Selectable Software loaded TLB in addition to hash MMU (755, 7450, e600)
- Mix of 32 & 64 bit::
+--------------+
| POWER8 |
+--------------+
+ |
+ |
+ v
+ +--------------+
+ | POWER9 |
+ +--------------+
+ |
+ |
+ v
+ +--------------+
+ | POWER10 |
+ +--------------+
+---------------+
bindgen (optional) 0.56.0 bindgen --version
GNU make 3.82 make --version
bash 4.2 bash --version
-binutils 2.23 ld -v
+binutils 2.25 ld -v
flex 2.5.35 flex --version
bison 2.0 bison --version
pahole 1.16 pahole --version
Binutils
--------
-Binutils 2.23 or newer is needed to build the kernel.
+Binutils 2.25 or newer is needed to build the kernel.
pkg-config
----------
hist:keys=<field1[,field2,...]>[:values=<field1[,field2,...]>]
[:sort=<field1[,field2,...]>][:size=#entries][:pause][:continue]
- [:clear][:name=histname1][:<handler>.<action>] [if <filter>]
+ [:clear][:name=histname1][:
nohitcount][:<handler>.<action>] [if <filter>]
When a matching event is hit, an entry is added to a hash table
using the key(s) and value(s) named. Keys and values correspond to
.log2 display log2 value rather than raw number
.buckets=size display grouping of values rather than raw number
.usecs display a common_timestamp in microseconds
+ .percent display a number of percentage value
+ .graph display a bar-graph of a value
============= =================================================
Note that in general the semantics of a given field aren't
existing trigger, rather than via the '>' operator, which will cause
the trigger to be removed through truncation.
+ The 'nohitcount' (or NOHC) parameter will suppress display of
+ raw hitcount in the histogram. This option requires at least one
+ value field which is not a 'raw hitcount'. For example,
+ 'hist:...:vals=hitcount:nohitcount' is rejected, but
+ 'hist:...:vals=hitcount.percent:nohitcount' is OK.
+
- enable_hist/disable_hist
The enable_hist and disable_hist triggers can be used to have one
The reason being is that this sample was taken on a virtual machine,
and the host interference is detected as a hardware interference.
-Tracer options
----------------------
+Tracer Configuration
+--------------------
The tracer has a set of options inside the osnoise directory, they are:
- tracing_threshold: the minimum delta between two time() reads to be
considered as noise, in us. When set to 0, the default value will
be used, which is currently 5 us.
+ - osnoise/options: a set of on/off options that can be enabled by
+ writing the option name to the file or disabled by writing the option
+ name preceded with the 'NO\_' prefix. For example, writing
+ NO_OSNOISE_WORKLOAD disables the OSNOISE_WORKLOAD option. The
+ special DEAFAULTS option resets all options to the default value.
+
+Tracer Options
+--------------
+
+The osnoise/options file exposes a set of on/off configuration options for
+the osnoise tracer. These options are:
+
+ - DEFAULTS: reset the options to the default value.
+ - OSNOISE_WORKLOAD: do not dispatch osnoise workload (see dedicated
+ section below).
+ - PANIC_ON_STOP: call panic() if the tracer stops. This option serves to
+ capture a vmcore.
+ - OSNOISE_PREEMPT_DISABLE: disable preemption while running the osnoise
+ workload, allowing only IRQ and hardware-related noise.
+ - OSNOISE_IRQ_DISABLE: disable IRQs while running the osnoise workload,
+ allowing only NMIs and hardware-related noise, like hwlat tracer.
Additional Tracing
------------------
The reason roots in the overhead of the entry and exit code that happens
before and after any interference execution. This justifies the dual
approach: measuring thread and tracing.
+
+Running osnoise tracer without workload
+---------------------------------------
+
+By enabling the osnoise tracer with the NO_OSNOISE_WORKLOAD option set,
+the osnoise: tracepoints serve to measure the execution time of
+any type of Linux task, free from the interference of other tasks.
--- /dev/null
+Valid-License-Identifier: copyleft-next-0.3.1
+SPDX-URL: https://spdx.org/licenses/copyleft-next-0.3.1
+Usage-Guide:
+ copyleft-next-0.3.1 is explicitly compatible with GPLv2 (or later) and
+ can therefore be used for kernel code. Though the best and recommended
+ practice is to express this in the SPDX license identifier by
+ licensing the code under both licenses expressed by the OR operator.
+ To use the copyleft-next-0.3.1 license put the following SPDX tag/value
+ pair into a comment according to the placement guidelines in the
+ licensing rules documentation:
+ SPDX-License-Identifier: GPL-2.0-only OR copyleft-next 0.3.1
+ SPDX-License-Identifier: GPL-2.0-or-later OR copyleft-next-0.3.1
+License-Text:
+
+=======================================================================
+
+ copyleft-next 0.3.1 ("this License")
+ Release date: 2016-04-29
+
+1. License Grants; No Trademark License
+
+ Subject to the terms of this License, I grant You:
+
+ a) A non-exclusive, worldwide, perpetual, royalty-free, irrevocable
+ copyright license, to reproduce, Distribute, prepare derivative works
+ of, publicly perform and publicly display My Work.
+
+ b) A non-exclusive, worldwide, perpetual, royalty-free, irrevocable
+ patent license under Licensed Patents to make, have made, use, sell,
+ offer for sale, and import Covered Works.
+
+ This License does not grant any rights in My name, trademarks, service
+ marks, or logos.
+
+2. Distribution: General Conditions
+
+ You may Distribute Covered Works, provided that You (i) inform
+ recipients how they can obtain a copy of this License; (ii) satisfy the
+ applicable conditions of sections 3 through 6; and (iii) preserve all
+ Legal Notices contained in My Work (to the extent they remain
+ pertinent). "Legal Notices" means copyright notices, license notices,
+ license texts, and author attributions, but does not include logos,
+ other graphical images, trademarks or trademark legends.
+
+3. Conditions for Distributing Derived Works; Outbound GPL Compatibility
+
+ If You Distribute a Derived Work, You must license the entire Derived
+ Work as a whole under this License, with prominent notice of such
+ licensing. This condition may not be avoided through such means as
+ separate Distribution of portions of the Derived Work.
+
+ If the Derived Work includes material licensed under the GPL, You may
+ instead license the Derived Work under the GPL.
+
+4. Condition Against Further Restrictions; Inbound License Compatibility
+
+ When Distributing a Covered Work, You may not impose further
+ restrictions on the exercise of rights in the Covered Work granted under
+ this License. This condition is not excused merely because such
+ restrictions result from Your compliance with conditions or obligations
+ extrinsic to this License (such as a court order or an agreement with a
+ third party).
+
+ However, You may Distribute a Covered Work incorporating material
+ governed by a license that is both OSI-Approved and FSF-Free as of the
+ release date of this License, provided that compliance with such
+ other license would not conflict with any conditions stated in other
+ sections of this License.
+
+5. Conditions for Distributing Object Code
+
+ You may Distribute an Object Code form of a Covered Work, provided that
+ you accompany the Object Code with a URL through which the Corresponding
+ Source is made available, at no charge, by some standard or customary
+ means of providing network access to source code.
+
+ If you Distribute the Object Code in a physical product or tangible
+ storage medium ("Product"), the Corresponding Source must be available
+ through such URL for two years from the date of Your most recent
+ Distribution of the Object Code in the Product. However, if the Product
+ itself contains or is accompanied by the Corresponding Source (made
+ available in a customarily accessible manner), You need not also comply
+ with the first paragraph of this section.
+
+ Each direct and indirect recipient of the Covered Work from You is an
+ intended third-party beneficiary of this License solely as to this
+ section 5, with the right to enforce its terms.
+
+6. Symmetrical Licensing Condition for Upstream Contributions
+
+ If You Distribute a work to Me specifically for inclusion in or
+ modification of a Covered Work (a "Patch"), and no explicit licensing
+ terms apply to the Patch, You license the Patch under this License, to
+ the extent of Your copyright in the Patch. This condition does not
+ negate the other conditions of this License, if applicable to the Patch.
+
+7. Nullification of Copyleft/Proprietary Dual Licensing
+
+ If I offer to license, for a fee, a Covered Work under terms other than
+ a license that is OSI-Approved or FSF-Free as of the release date of this
+ License or a numbered version of copyleft-next released by the
+ Copyleft-Next Project, then the license I grant You under section 1 is no
+ longer subject to the conditions in sections 3 through 5.
+
+8. Copyleft Sunset
+
+ The conditions in sections 3 through 5 no longer apply once fifteen
+ years have elapsed from the date of My first Distribution of My Work
+ under this License.
+
+9. Pass-Through
+
+ When You Distribute a Covered Work, the recipient automatically receives
+ a license to My Work from Me, subject to the terms of this License.
+
+10. Termination
+
+ Your license grants under section 1 are automatically terminated if You
+
+ a) fail to comply with the conditions of this License, unless You cure
+ such noncompliance within thirty days after becoming aware of it, or
+
+ b) initiate a patent infringement litigation claim (excluding
+ declaratory judgment actions, counterclaims, and cross-claims)
+ alleging that any part of My Work directly or indirectly infringes
+ any patent.
+
+ Termination of Your license grants extends to all copies of Covered
+ Works You subsequently obtain. Termination does not terminate the
+ rights of those who have received copies or rights from You subject to
+ this License.
+
+ To the extent permission to make copies of a Covered Work is necessary
+ merely for running it, such permission is not terminable.
+
+11. Later License Versions
+
+ The Copyleft-Next Project may release new versions of copyleft-next,
+ designated by a distinguishing version number ("Later Versions").
+ Unless I explicitly remove the option of Distributing Covered Works
+ under Later Versions, You may Distribute Covered Works under any Later
+ Version.
+
+** 12. No Warranty **
+** **
+** My Work is provided "as-is", without warranty. You bear the risk **
+** of using it. To the extent permitted by applicable law, each **
+** Distributor of My Work excludes the implied warranties of title, **
+** merchantability, fitness for a particular purpose and **
+** non-infringement. **
+
+** 13. Limitation of Liability **
+** **
+** To the extent permitted by applicable law, in no event will any **
+** Distributor of My Work be liable to You for any damages **
+** whatsoever, whether direct, indirect, special, incidental, or **
+** consequential damages, whether arising under contract, tort **
+** (including negligence), or otherwise, even where the Distributor **
+** knew or should have known about the possibility of such damages. **
+
+14. Severability
+
+ The invalidity or unenforceability of any provision of this License
+ does not affect the validity or enforceability of the remainder of
+ this License. Such provision is to be reformed to the minimum extent
+ necessary to make it valid and enforceable.
+
+15. Definitions
+
+ "Copyleft-Next Project" means the project that maintains the source
+ code repository at <https://github.com/copyleft-next/copyleft-next.git/>
+ as of the release date of this License.
+
+ "Corresponding Source" of a Covered Work in Object Code form means (i)
+ the Source Code form of the Covered Work; (ii) all scripts,
+ instructions and similar information that are reasonably necessary for
+ a skilled developer to generate such Object Code from the Source Code
+ provided under (i); and (iii) a list clearly identifying all Separate
+ Works (other than those provided in compliance with (ii)) that were
+ specifically used in building and (if applicable) installing the
+ Covered Work (for example, a specified proprietary compiler including
+ its version number). Corresponding Source must be machine-readable.
+
+ "Covered Work" means My Work or a Derived Work.
+
+ "Derived Work" means a work of authorship that copies from, modifies,
+ adapts, is based on, is a derivative work of, transforms, translates or
+ contains all or part of My Work, such that copyright permission is
+ required. The following are not Derived Works: (i) Mere Aggregation;
+ (ii) a mere reproduction of My Work; and (iii) if My Work fails to
+ explicitly state an expectation otherwise, a work that merely makes
+ reference to My Work.
+
+ "Distribute" means to distribute, transfer or make a copy available to
+ someone else, such that copyright permission is required.
+
+ "Distributor" means Me and anyone else who Distributes a Covered Work.
+
+ "FSF-Free" means classified as 'free' by the Free Software Foundation.
+
+ "GPL" means a version of the GNU General Public License or the GNU
+ Affero General Public License.
+
+ "I"/"Me"/"My" refers to the individual or legal entity that places My
+ Work under this License. "You"/"Your" refers to the individual or legal
+ entity exercising rights in My Work under this License. A legal entity
+ includes each entity that controls, is controlled by, or is under
+ common control with such legal entity. "Control" means (a) the power to
+ direct the actions of such legal entity, whether by contract or
+ otherwise, or (b) ownership of more than fifty percent of the
+ outstanding shares or beneficial ownership of such legal entity.
+
+ "Licensed Patents" means all patent claims licensable royalty-free by
+ Me, now or in the future, that are necessarily infringed by making,
+ using, or selling My Work, and excludes claims that would be infringed
+ only as a consequence of further modification of My Work.
+
+ "Mere Aggregation" means an aggregation of a Covered Work with a
+ Separate Work.
+
+ "My Work" means the particular work of authorship I license to You
+ under this License.
+
+ "Object Code" means any form of a work that is not Source Code.
+
+ "OSI-Approved" means approved as 'Open Source' by the Open Source
+ Initiative.
+
+ "Separate Work" means a work that is separate from and independent of a
+ particular Covered Work and is not by its nature an extension or
+ enhancement of the Covered Work, and/or a runtime library, standard
+ library or similar component that is used to generate an Object Code
+ form of a Covered Work.
+
+ "Source Code" means the preferred form of a work for making
+ modifications to it.
T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git
F: drivers/net/amt.c
+ANALOG DEVICES INC AD4130 DRIVER
+M: Cosmin Tanislav <cosmin.tanislav@analog.com>
+L: linux-iio@vger.kernel.org
+S: Supported
+W: http://ez.analog.com/community/linux-device-drivers
+F: Documentation/ABI/testing/sysfs-bus-iio-adc-ad4130
+F: Documentation/devicetree/bindings/iio/adc/adi,ad4130.yaml
+F: drivers/iio/adc/ad4130.c
+
ANALOG DEVICES INC AD7192 DRIVER
M: Alexandru Tachici <alexandru.tachici@analog.com>
L: linux-iio@vger.kernel.org
F: Documentation/devicetree/bindings/iio/adc/adi,ad7780.yaml
F: drivers/iio/adc/ad7780.c
+ANALOG DEVICES INC AD74115 DRIVER
+M: Cosmin Tanislav <cosmin.tanislav@analog.com>
+L: linux-iio@vger.kernel.org
+S: Supported
+W: http://ez.analog.com/community/linux-device-drivers
+F: Documentation/devicetree/bindings/iio/addac/adi,ad74115.yaml
+F: drivers/iio/addac/ad74115.c
+
ANALOG DEVICES INC AD74413R DRIVER
M: Cosmin Tanislav <cosmin.tanislav@analog.com>
L: linux-iio@vger.kernel.org
F: Documentation/devicetree/bindings/iio/amplifiers/adi,ada4250.yaml
F: drivers/iio/amplifiers/ada4250.c
+ANALOG DEVICES INC ADF4377 DRIVER
+M: Antoniu Miclaus <antoniu.miclaus@analog.com>
+L: linux-iio@vger.kernel.org
+S: Supported
+W: https://ez.analog.com/linux-software-drivers
+F: Documentation/devicetree/bindings/iio/frequency/adi,adf4377.yaml
+F: drivers/iio/frequency/adf4377.c
+
ANALOG DEVICES INC ADGS1408 DRIVER
M: Mircea Caprioru <mircea.caprioru@analog.com>
S: Supported
F: Documentation/devicetree/bindings/gpio/intel,ixp4xx-gpio.txt
F: Documentation/devicetree/bindings/interrupt-controller/intel,ixp4xx-interrupt.yaml
F: Documentation/devicetree/bindings/timer/intel,ixp4xx-timer.yaml
+F: arch/arm/boot/dts/intel-ixp*
F: arch/arm/mach-ixp4xx/
F: drivers/bus/intel-ixp4xx-eb.c
F: drivers/clocksource/timer-ixp4xx.c
F: include/linux/fanotify.h
F: include/uapi/linux/fanotify.h
+FARADAY FOTG210 USB2 DUAL-ROLE CONTROLLER
+M: Linus Walleij <linus.walleij@linaro.org>
+L: linux-usb@vger.kernel.org
+S: Maintained
+F: drivers/usb/fotg210/
+
FARSYNC SYNCHRONOUS DRIVER
M: Kevin Curtis <kevin.curtis@farsite.co.uk>
S: Supported
M: Steven Rostedt <rostedt@goodmis.org>
M: Masami Hiramatsu <mhiramat@kernel.org>
R: Mark Rutland <mark.rutland@arm.com>
+L: linux-kernel@vger.kernel.org
+L: linux-trace-kernel@vger.kernel.org
+Q: https://patchwork.kernel.org/project/linux-trace-kernel/list/
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
F: Documentation/trace/ftrace*
F: drivers/iommu/intel/
F: include/linux/intel-svm.h
-INTEL IOP-ADMA DMA DRIVER
-R: Dan Williams <dan.j.williams@intel.com>
-S: Odd fixes
-F: drivers/dma/iop-adma.c
-
INTEL IPU3 CSI-2 CIO2 DRIVER
M: Yong Zhi <yong.zhi@intel.com>
M: Sakari Ailus <sakari.ailus@linux.intel.com>
F: include/linux/mfd/khadas-mcu.h
F: drivers/thermal/khadas_mcu_fan.c
+KIONIX/ROHM KX022A ACCELEROMETER
+M: Matti Vaittinen <mazziesaccount@gmail.com>
+L: linux-iio@vger.kernel.org
+S: Supported
+F: drivers/iio/accel/kionix-kx022a*
+
KMEMLEAK
M: Catalin Marinas <catalin.marinas@arm.com>
S: Maintained
M: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
M: "David S. Miller" <davem@davemloft.net>
M: Masami Hiramatsu <mhiramat@kernel.org>
+L: linux-kernel@vger.kernel.org
+L: linux-trace-kernel@vger.kernel.org
+Q: https://patchwork.kernel.org/project/linux-trace-kernel/list/
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
F: Documentation/trace/kprobes.rst
LED SUBSYSTEM
M: Pavel Machek <pavel@ucw.cz>
+M: Lee Jones <lee@kernel.org>
L: linux-leds@vger.kernel.org
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/pavel/linux-leds.git
F: Documentation/devicetree/bindings/leds/
F: drivers/leds/
+F: include/dt-bindings/leds/
F: include/linux/leds.h
LEGACY EEPROM DRIVER
F: Documentation/devicetree/bindings/regulator/maxim,max20086.yaml
F: drivers/regulator/max20086-regulator.c
+MAXIM MAX30208 TEMPERATURE SENSOR DRIVER
+M: Rajat Khandelwal <rajat.khandelwal@linux.intel.com>
+L: linux-iio@vger.kernel.org
+S: Maintained
+F: drivers/iio/temperature/max30208.c
+
MAXIM MAX77650 PMIC MFD DRIVER
M: Bartosz Golaszewski <brgl@bgdev.pl>
L: linux-kernel@vger.kernel.org
MHI BUS
M: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
-R: Hemant Kumar <quic_hemantk@quicinc.com>
L: mhi@lists.linux.dev
L: linux-arm-msm@vger.kernel.org
S: Maintained
S: Supported
F: Documentation/devicetree/bindings/dma/atmel-dma.txt
F: drivers/dma/at_hdmac.c
-F: drivers/dma/at_hdmac_regs.h
F: drivers/dma/at_xdmac.c
F: include/dt-bindings/dma/at91.h
M: Richard Leitner <richard.leitner@skidata.com>
L: linux-usb@vger.kernel.org
S: Maintained
-F: Documentation/devicetree/bindings/usb/usb251xb.txt
+F: Documentation/devicetree/bindings/usb/usb251xb.yaml
F: drivers/usb/misc/usb251xb.c
MICROCHIP USBA UDC DRIVER
F: arch/nios2/
NITRO ENCLAVES (NE)
-M: Andra Paraschiv <andraprs@amazon.com>
-M: Alexandru Vasile <lexnv@amazon.com>
M: Alexandru Ciobotaru <alcioa@amazon.com>
L: linux-kernel@vger.kernel.org
+L: The AWS Nitro Enclaves Team <aws-nitro-enclaves-devel@amazon.com>
S: Supported
W: https://aws.amazon.com/ec2/nitro/nitro-enclaves/
F: Documentation/virt/ne_overview.rst
F: drivers/mfd/palmas.c
F: drivers/mfd/tps65217.c
F: drivers/mfd/tps65218.c
+F: drivers/mfd/tps65219.c
F: drivers/mfd/tps65910.c
F: drivers/mfd/twl-core.[ch]
F: drivers/mfd/twl4030*.c
F: tools/perf/
PERFORMANCE EVENTS TOOLING ARM64
-R: John Garry <john.garry@huawei.com>
+R: John Garry <john.g.garry@oracle.com>
R: Will Deacon <will@kernel.org>
R: James Clark <james.clark@arm.com>
R: Mike Leach <mike.leach@linaro.org>
TRACING
M: Steven Rostedt <rostedt@goodmis.org>
M: Masami Hiramatsu <mhiramat@kernel.org>
+L: linux-kernel@vger.kernel.org
+L: linux-trace-kernel@vger.kernel.org
+Q: https://patchwork.kernel.org/project/linux-trace-kernel/list/
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
F: Documentation/trace/*
ZRAM COMPRESSED RAM BLOCK DEVICE DRVIER
M: Minchan Kim <minchan@kernel.org>
-M: Nitin Gupta <ngupta@vflare.org>
-R: Sergey Senozhatsky <senozhatsky@chromium.org>
+M: Sergey Senozhatsky <senozhatsky@chromium.org>
L: linux-kernel@vger.kernel.org
S: Maintained
F: Documentation/admin-guide/blockdev/zram.rst
ZSMALLOC COMPRESSED SLAB MEMORY ALLOCATOR
M: Minchan Kim <minchan@kernel.org>
-M: Nitin Gupta <ngupta@vflare.org>
-R: Sergey Senozhatsky <senozhatsky@chromium.org>
+M: Sergey Senozhatsky <senozhatsky@chromium.org>
L: linux-mm@kvack.org
S: Maintained
F: Documentation/mm/zsmalloc.rst
M: Nick Terrell <terrelln@fb.com>
S: Maintained
B: https://github.com/facebook/zstd/issues
-T: git git://github.com/terrelln/linux.git
+T: git https://github.com/terrelln/linux.git
F: include/linux/zstd*
F: lib/zstd/
F: lib/decompress_unzstd.c
# Comments in this file are targeted only to the developer, do not
# expect to learn how to build the kernel reading this file.
+ifeq ($(filter undefine,$(.FEATURES)),)
+$(error GNU Make >= 3.82 is required. Your Make version is $(MAKE_VERSION))
+endif
+
$(if $(filter __%, $(MAKECMDGOALS)), \
$(error targets prefixed with '__' are only for internal use))
# If the user is running make -s (silent mode), suppress echoing of
# commands
+# make-4.0 (and later) keep single letter options in the 1st word of MAKEFLAGS.
-ifneq ($(findstring s,$(filter-out --%,$(MAKEFLAGS))),)
- quiet=silent_
- KBUILD_VERBOSE = 0
+ifeq ($(filter 3.%,$(MAKE_VERSION)),)
+silence:=$(findstring s,$(firstword -$(MAKEFLAGS)))
+else
+silence:=$(findstring s,$(filter-out --%,$(MAKEFLAGS)))
+endif
+
+ifeq ($(silence),s)
+quiet=silent_
+KBUILD_VERBOSE = 0
endif
export quiet Q KBUILD_VERBOSE
include $(srctree)/scripts/Kbuild.include
# Read KERNELRELEASE from include/config/kernel.release (if it exists)
-KERNELRELEASE = $(shell cat include/config/kernel.release 2> /dev/null)
+KERNELRELEASE = $(call read-file, include/config/kernel.release)
KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION)
export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
KBUILD_CFLAGS += $(stackp-flags-y)
-KBUILD_CFLAGS-$(CONFIG_WERROR) += -Werror
+KBUILD_CPPFLAGS-$(CONFIG_WERROR) += -Werror
+KBUILD_CPPFLAGS += $(KBUILD_CPPFLAGS-y)
KBUILD_CFLAGS-$(CONFIG_CC_NO_ARRAY_BOUNDS) += -Wno-array-bounds
KBUILD_RUSTFLAGS-$(CONFIG_WERROR) += -Dwarnings
endif
endif
ifdef CONFIG_FTRACE_MCOUNT_USE_OBJTOOL
- CC_FLAGS_USING += -DCC_USING_NOP_MCOUNT
+ ifdef CONFIG_HAVE_OBJTOOL_NOP_MCOUNT
+ CC_FLAGS_USING += -DCC_USING_NOP_MCOUNT
+ endif
endif
ifdef CONFIG_FTRACE_MCOUNT_USE_RECORDMCOUNT
ifdef CONFIG_HAVE_C_RECORDMCOUNT
# Check for frame size exceeding threshold during prolog/epilog insertion
# when using lld < 13.0.0.
ifneq ($(CONFIG_FRAME_WARN),0)
-ifeq ($(shell test $(CONFIG_LLD_VERSION) -lt 130000; echo $$?),0)
+ifeq ($(call test-lt, $(CONFIG_LLD_VERSION), 130000),y)
KBUILD_LDFLAGS += -plugin-opt=-warn-stack-size=$(CONFIG_FRAME_WARN)
endif
endif
rm -f $(MODLIB)/build ; \
ln -s $(CURDIR) $(MODLIB)/build ; \
fi
- @sed 's:^:kernel/:' modules.order > $(MODLIB)/modules.order
+ @sed 's:^\(.*\)\.o$$:kernel/\1.ko:' modules.order > $(MODLIB)/modules.order
@cp -f modules.builtin $(MODLIB)/
@cp -f $(objtree)/modules.builtin.modinfo $(MODLIB)/
config SHADOW_CALL_STACK
bool "Shadow Call Stack"
depends on ARCH_SUPPORTS_SHADOW_CALL_STACK
- depends on DYNAMIC_FTRACE_WITH_REGS || !FUNCTION_GRAPH_TRACER
+ depends on DYNAMIC_FTRACE_WITH_ARGS || DYNAMIC_FTRACE_WITH_REGS || !FUNCTION_GRAPH_TRACER
help
This option enables the compiler's Shadow Call Stack, which
uses a shadow stack to protect function return addresses from
config ARCH_MULTI_V4
bool "ARMv4 based platforms (FA526, StrongARM)"
depends on !ARCH_MULTI_V6_V7
+ depends on !LD_IS_LLD
select ARCH_MULTI_V4_V5
select CPU_FA526 if !(CPU_SA110 || CPU_SA1100)
config ARCH_MULTI_V4T
bool "ARMv4T based platforms (ARM720T, ARM920T, ...)"
depends on !ARCH_MULTI_V6_V7
+ depends on !LD_IS_LLD
select ARCH_MULTI_V4_V5
select CPU_ARM920T if !(CPU_ARM7TDMI || CPU_ARM720T || \
CPU_ARM740T || CPU_ARM9TDMI || CPU_ARM922T || \
status = "disabled";
};
- shirq: interrupt-controller@0x50000000 {
+ shirq: interrupt-controller@50000000 {
compatible = "st,spear300-shirq";
reg = <0x50000000 0x1000>;
interrupts = <28>;
status = "disabled";
};
- shirq: interrupt-controller@0xb4000000 {
+ shirq: interrupt-controller@b4000000 {
compatible = "st,spear310-shirq";
reg = <0xb4000000 0x1000>;
interrupts = <28 29 30 1>;
status = "disabled";
};
- shirq: interrupt-controller@0xb3000000 {
+ shirq: interrupt-controller@b3000000 {
compatible = "st,spear320-shirq";
reg = <0xb3000000 0x1000>;
interrupts = <30 28 29 1>;
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/io.h>
+#include <linux/set_memory.h>
#include <asm/fncpy.h>
#include <asm/tlb.h>
#include <asm/cacheflush.h>
-#include <asm/set_memory.h>
#include <asm/mach/map.h>
dst = fncpy(sram, funcp, size);
- set_memory_ro(base, pages);
- set_memory_x(base, pages);
+ set_memory_rox(base, pages);
return dst;
}
base = (unsigned long)omap_sram_base;
pages = PAGE_ALIGN(omap_sram_size) / PAGE_SIZE;
- set_memory_ro(base, pages);
- set_memory_x(base, pages);
+ set_memory_rox(base, pages);
}
static void (*_omap_sram_reprogram_clock)(u32 dpllctl, u32 ckctl);
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/io.h>
+#include <linux/set_memory.h>
#include <asm/fncpy.h>
#include <asm/tlb.h>
#include <asm/cacheflush.h>
-#include <asm/set_memory.h>
#include <asm/mach/map.h>
dst = fncpy(sram, funcp, size);
- set_memory_ro(base, pages);
- set_memory_x(base, pages);
+ set_memory_rox(base, pages);
return dst;
}
base = (unsigned long)omap_sram_base;
pages = PAGE_ALIGN(omap_sram_size) / PAGE_SIZE;
- set_memory_ro(base, pages);
- set_memory_x(base, pages);
+ set_memory_rox(base, pages);
}
static void (*_omap2_sram_ddr_init)(u32 *slow_dll_ctrl, u32 fast_dll_ctrl,
#ifndef CONFIG_IWMMXT
u64 acc0;
+#ifndef CONFIG_AS_IS_LLVM
asm volatile(".arch_extension xscale\n\t"
"mra %Q0, %R0, acc0" : "=r" (acc0));
+#else
+ asm volatile("mrrc p0, 0, %Q0, %R0, c0" : "=r" (acc0));
+#endif
#endif
/* ensure voltage-change sequencer not initiated, which hangs */
case PM_SUSPEND_MEM:
cpu_suspend(pwrmode, pxa27x_finish_suspend);
#ifndef CONFIG_IWMMXT
+#ifndef CONFIG_AS_IS_LLVM
asm volatile(".arch_extension xscale\n\t"
"mar acc0, %Q0, %R0" : "=r" (acc0));
+#else
+ asm volatile("mcrr p0, 0, %Q0, %R0, c0" :: "r" (acc0));
+#endif
#endif
break;
}
#ifndef CONFIG_IWMMXT
u64 acc0;
+#ifdef CONFIG_CC_IS_GCC
asm volatile(".arch_extension xscale\n\t"
"mra %Q0, %R0, acc0" : "=r" (acc0));
+#else
+ asm volatile("mrrc p0, 0, %Q0, %R0, c0" : "=r" (acc0));
+#endif
#endif
/* resuming from D2 requires the HSIO2/BOOT/TPM clocks enabled */
AD3ER = 0;
#ifndef CONFIG_IWMMXT
+#ifndef CONFIG_AS_IS_LLVM
asm volatile(".arch_extension xscale\n\t"
"mar acc0, %Q0, %R0" : "=r" (acc0));
+#else
+ asm volatile("mcrr p0, 0, %Q0, %R0, c0" :: "r" (acc0));
+#endif
#endif
}
If unsure, say Y.
-config ARM64_ERRATUM_2645198
- bool "Cortex-A715: 2645198: Workaround possible [ESR|FAR]_ELx corruption"
- default y
- help
- This option adds the workaround for ARM Cortex-A715 erratum 2645198.
-
- If a Cortex-A715 cpu sees a page mapping permissions change from executable
- to non-executable, it may corrupt the ESR_ELx and FAR_ELx registers on the
- next instruction abort caused by permission fault.
-
- Only user-space does executable to non-executable permission transition via
- mprotect() system call. Workaround the problem by doing a break-before-make
- TLB invalidation, for all changes to executable user space mappings.
-
- If unsure, say Y.
-
config CAVIUM_ERRATUM_22375
bool "Cavium erratum 22375, 24313"
default y
<GIC_SPI 278 IRQ_TYPE_LEVEL_LOW>;
interrupt-names = "job", "mmu", "gpu";
- clocks = <&topckgen CLK_TOP_MFGPLL_CK>;
+ clocks = <&mfgcfg CLK_MFG_BG3D>;
power-domains =
<&spm MT8183_POWER_DOMAIN_MFG_CORE0>,
#size-cells = <2>;
ranges;
- /* 192 KiB reserved for ARM Trusted Firmware (BL31) */
+ /* 2 MiB reserved for ARM Trusted Firmware (BL31) */
bl31_secmon_reserved: secmon@54600000 {
no-map;
- reg = <0 0x54600000 0x0 0x30000>;
+ reg = <0 0x54600000 0x0 0x200000>;
};
/* 12 MiB reserved for OP-TEE (BL32)
};
};
+ reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ rproc_0_fw_image: memory@3ed00000 {
+ no-map;
+ reg = <0x0 0x3ed00000 0x0 0x40000>;
+ };
+
+ rproc_1_fw_image: memory@3ef00000 {
+ no-map;
+ reg = <0x0 0x3ef00000 0x0 0x40000>;
+ };
+ };
+
zynqmp_ipi: zynqmp_ipi {
compatible = "xlnx,zynqmp-ipi-mailbox";
interrupt-parent = <&gic>;
ranges;
};
+ remoteproc {
+ compatible = "xlnx,zynqmp-r5fss";
+ xlnx,cluster-mode = <1>;
+
+ r5f-0 {
+ compatible = "xlnx,zynqmp-r5f";
+ power-domains = <&zynqmp_firmware PD_RPU_0>;
+ memory-region = <&rproc_0_fw_image>;
+ };
+
+ r5f-1 {
+ compatible = "xlnx,zynqmp-r5f";
+ power-domains = <&zynqmp_firmware PD_RPU_1>;
+ memory-region = <&rproc_1_fw_image>;
+ };
+ };
+
amba: axi {
compatible = "simple-bus";
#address-cells = <2>;
void __init arm64_hugetlb_cma_reserve(void);
-#define huge_ptep_modify_prot_start huge_ptep_modify_prot_start
-extern pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep);
-
-#define huge_ptep_modify_prot_commit huge_ptep_modify_prot_commit
-extern void huge_ptep_modify_prot_commit(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep,
- pte_t old_pte, pte_t new_pte);
-
#include <asm-generic/hugetlb.h>
#endif /* __ASM_HUGETLB_H */
}
-#define __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION
-#define ptep_modify_prot_start ptep_modify_prot_start
-extern pte_t ptep_modify_prot_start(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep);
-
-#define ptep_modify_prot_commit ptep_modify_prot_commit
-extern void ptep_modify_prot_commit(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep,
- pte_t old_pte, pte_t new_pte);
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_PGTABLE_H */
CAP_MIDR_RANGE_LIST(trbe_write_out_of_range_cpus),
},
#endif
-#ifdef CONFIG_ARM64_ERRATUM_2645198
- {
- .desc = "ARM erratum 2645198",
- .capability = ARM64_WORKAROUND_2645198,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A715)
- },
-#endif
#ifdef CONFIG_ARM64_ERRATUM_2077057
{
.desc = "ARM erratum 2077057",
{
return __hugetlb_valid_size(size);
}
-
-pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
-{
- if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198) &&
- cpus_have_const_cap(ARM64_WORKAROUND_2645198)) {
- /*
- * Break-before-make (BBM) is required for all user space mappings
- * when the permission changes from executable to non-executable
- * in cases where cpu is affected with errata #2645198.
- */
- if (pte_user_exec(READ_ONCE(*ptep)))
- return huge_ptep_clear_flush(vma, addr, ptep);
- }
- return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep);
-}
-
-void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep,
- pte_t old_pte, pte_t pte)
-{
- set_huge_pte_at(vma->vm_mm, addr, ptep, pte);
-}
}
early_initcall(prevent_bootmem_remove_init);
#endif
-
-pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
-{
- if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198) &&
- cpus_have_const_cap(ARM64_WORKAROUND_2645198)) {
- /*
- * Break-before-make (BBM) is required for all user space mappings
- * when the permission changes from executable to non-executable
- * in cases where cpu is affected with errata #2645198.
- */
- if (pte_user_exec(READ_ONCE(*ptep)))
- return ptep_clear_flush(vma, addr, ptep);
- }
- return ptep_get_and_clear(vma->vm_mm, addr, ptep);
-}
-
-void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep,
- pte_t old_pte, pte_t pte)
-{
- set_pte_at(vma->vm_mm, addr, ptep, pte);
-}
WORKAROUND_2064142
WORKAROUND_2077057
WORKAROUND_2457168
-WORKAROUND_2645198
WORKAROUND_2658417
WORKAROUND_TRBE_OVERWRITE_FILL_MODE
WORKAROUND_TSB_FLUSH_FAILURE
select ARCH_USE_BUILTIN_BSWAP
select ARCH_USE_QUEUED_RWLOCKS
select ARCH_USE_QUEUED_SPINLOCKS
+ select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_INLINE_READ_LOCK if !PREEMPTION
select ARCH_INLINE_READ_LOCK_BH if !PREEMPTION
select ARCH_INLINE_READ_LOCK_IRQ if !PREEMPTION
select HAVE_PERF_USER_STACK_DUMP
select HAVE_DMA_CONTIGUOUS
select HAVE_REGS_AND_STACK_ACCESS_API
- select HAVE_RSEQ
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select MAY_HAVE_SPARSE_IRQ
bool "Tightly-Coupled/Sram Memory"
depends on !COMPILE_TEST
help
- The implementation are not only used by TCM (Tightly-Coupled Meory)
+ The implementation are not only used by TCM (Tightly-Coupled Memory)
but also used by sram on SOC bus. It follow existed linux tcm
software interface, so that old tcm application codes could be
re-used directly.
#define cpu_relax() barrier()
+register unsigned long current_stack_pointer __asm__("sp");
+
#endif /* __ASM_CSKY_PROCESSOR_H */
lrw r9, __NR_syscalls
cmphs syscallid, r9 /* Check nr of syscall */
- bt 1f
+ bt ret_from_exception
lrw r9, sys_call_table
ixw r9, syscallid
jsr syscallid
#endif
stw a0, (sp, LSAVE_A0) /* Save return value */
-1:
-#ifdef CONFIG_DEBUG_RSEQ
- mov a0, sp
- jbsr rseq_syscall
-#endif
jmpi ret_from_exception
csky_syscall_trace:
stw a0, (sp, LSAVE_A0) /* Save return value */
1:
-#ifdef CONFIG_DEBUG_RSEQ
- mov a0, sp
- jbsr rseq_syscall
-#endif
mov a0, sp /* right now, sp --> pt_regs */
jbsr syscall_trace_exit
br ret_from_exception
sigset_t *oldset = sigmask_to_save();
int ret;
- rseq_signal_deliver(ksig, regs);
-
/* Are we from a system call? */
if (in_syscall(regs)) {
/* Avoid additional syscall restarting via ret_from_exception */
sp = user_stack_pointer(regs);
pc = instruction_pointer(regs);
} else if (task == NULL || task == current) {
- const register unsigned long current_sp __asm__ ("sp");
const register unsigned long current_fp __asm__ ("r8");
fp = current_fp;
- sp = current_sp;
+ sp = current_stack_pointer;
pc = (unsigned long)walk_stackframe;
} else {
/* task blocked in __switch_to */
sp = user_stack_pointer(regs);
pc = instruction_pointer(regs);
} else if (task == NULL || task == current) {
- const register unsigned long current_sp __asm__ ("sp");
- sp = current_sp;
+ sp = current_stack_pointer;
pc = (unsigned long)walk_stackframe;
} else {
/* task blocked in __switch_to */
select ARCH_WANTS_NO_INSTR
select BUILDTIME_TABLE_SORT
select COMMON_CLK
+ select CPU_PM
select EFI
select GENERIC_CLOCKEVENTS
select GENERIC_CMOS_UPDATE
select HAVE_ARCH_TRANSPARENT_HUGEPAGE
select HAVE_ASM_MODVERSIONS
select HAVE_CONTEXT_TRACKING_USER
+ select HAVE_C_RECORDMCOUNT
select HAVE_DEBUG_STACKOVERFLOW
select HAVE_DMA_CONTIGUOUS
+ select HAVE_DYNAMIC_FTRACE
+ select HAVE_DYNAMIC_FTRACE_WITH_ARGS
+ select HAVE_DYNAMIC_FTRACE_WITH_REGS
select HAVE_EBPF_JIT
select HAVE_EXIT_THREAD
select HAVE_FAST_GUP
+ select HAVE_FTRACE_MCOUNT_RECORD
+ select HAVE_FUNCTION_GRAPH_TRACER
+ select HAVE_FUNCTION_TRACER
select HAVE_GENERIC_VDSO
select HAVE_IOREMAP_PROT
select HAVE_IRQ_EXIT_ON_IRQ_STACK
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_RSEQ
select HAVE_SETUP_PER_CPU_AREA if NUMA
+ select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_TIF_NOHZ
select HAVE_VIRT_CPU_ACCOUNTING_GEN if !SMP
select MODULES_USE_ELF_RELA if MODULES
select NEED_PER_CPU_EMBED_FIRST_CHUNK
select NEED_PER_CPU_PAGE_FIRST_CHUNK
+ select OF
+ select OF_EARLY_FLATTREE
select PCI
select PCI_DOMAINS_GENERIC
select PCI_ECAM if ACPI
select RTC_LIB
select SMP
select SPARSE_IRQ
+ select SYSCTL_ARCH_UNALIGN_ALLOW
+ select SYSCTL_ARCH_UNALIGN_NO_WARN
select SYSCTL_EXCEPTION_TRACE
select SWIOTLB
select TRACE_IRQFLAGS_SUPPORT
menu "Power management options"
+config ARCH_SUSPEND_POSSIBLE
+ def_bool y
+
+config ARCH_HIBERNATION_POSSIBLE
+ def_bool y
+
+source "kernel/power/Kconfig"
source "drivers/acpi/Kconfig"
endmenu
32bit-emul = elf32loongarch
64bit-emul = elf64loongarch
+ifdef CONFIG_DYNAMIC_FTRACE
+KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
+CC_FLAGS_FTRACE := -fpatchable-function-entry=2
+endif
+
ifdef CONFIG_64BIT
tool-archpref = $(64bit-tool-archpref)
UTS_MACHINE := loongarch64
libs-y += arch/loongarch/lib/
libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
+# suspend and hibernation support
+drivers-$(CONFIG_PM) += arch/loongarch/power/
+
ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_EXPERT=y
-CONFIG_USERFAULTFD=y
+CONFIG_KALLSYMS_ALL=y
CONFIG_PERF_EVENTS=y
-# CONFIG_COMPAT_BRK is not set
CONFIG_LOONGARCH=y
CONFIG_64BIT=y
CONFIG_MACH_LOONGSON64=y
+CONFIG_PAGE_SIZE_16KB=y
+CONFIG_HZ_250=y
CONFIG_DMI=y
CONFIG_EFI=y
CONFIG_SMP=y
CONFIG_NR_CPUS=64
CONFIG_NUMA=y
CONFIG_KEXEC=y
-CONFIG_PAGE_SIZE_16KB=y
-CONFIG_HZ_250=y
+CONFIG_SUSPEND=y
+CONFIG_HIBERNATION=y
CONFIG_ACPI=y
CONFIG_ACPI_SPCR_TABLE=y
-CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_TAD=y
CONFIG_ACPI_DOCK=y
CONFIG_ACPI_IPMI=m
+CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_PCI_SLOT=y
CONFIG_ACPI_HOTPLUG_MEMORY=y
CONFIG_EFI_ZBOOT=y
CONFIG_IOSCHED_BFQ=y
CONFIG_BFQ_GROUP_IOSCHED=y
CONFIG_BINFMT_MISC=m
-CONFIG_MEMORY_HOTPLUG=y
-CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=y
-CONFIG_MEMORY_HOTREMOVE=y
-CONFIG_KSM=y
-CONFIG_TRANSPARENT_HUGEPAGE=y
+CONFIG_ZPOOL=y
CONFIG_ZSWAP=y
CONFIG_ZSWAP_COMPRESSOR_DEFAULT_ZSTD=y
-CONFIG_ZPOOL=y
CONFIG_ZBUD=y
CONFIG_Z3FOLD=y
CONFIG_ZSMALLOC=m
+# CONFIG_COMPAT_BRK is not set
+CONFIG_MEMORY_HOTPLUG=y
+CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=y
+CONFIG_MEMORY_HOTREMOVE=y
+CONFIG_KSM=y
+CONFIG_TRANSPARENT_HUGEPAGE=y
+CONFIG_USERFAULTFD=y
CONFIG_NET=y
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_BRIDGE_NETFILTER=m
CONFIG_NETFILTER_NETLINK_LOG=m
CONFIG_NF_CONNTRACK=m
-CONFIG_NF_LOG_NETDEV=m
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
CONFIG_ATA=y
CONFIG_SATA_AHCI=y
CONFIG_SATA_AHCI_PLATFORM=y
+CONFIG_AHCI_DWC=y
CONFIG_PATA_ATIIXP=y
CONFIG_PATA_PCMCIA=m
CONFIG_MD=y
# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_VENDOR_ATHEROS is not set
CONFIG_BNX2=y
-# CONFIG_NET_VENDOR_BROCADE is not set
# CONFIG_NET_VENDOR_CAVIUM is not set
CONFIG_CHELSIO_T1=m
CONFIG_CHELSIO_T1_1G=y
CONFIG_CHELSIO_T3=m
CONFIG_CHELSIO_T4=m
-# CONFIG_NET_VENDOR_CIRRUS is not set
# CONFIG_NET_VENDOR_CISCO is not set
# CONFIG_NET_VENDOR_DEC is not set
# CONFIG_NET_VENDOR_DLINK is not set
# CONFIG_NET_VENDOR_NVIDIA is not set
# CONFIG_NET_VENDOR_OKI is not set
# CONFIG_NET_VENDOR_QLOGIC is not set
+# CONFIG_NET_VENDOR_BROCADE is not set
# CONFIG_NET_VENDOR_QUALCOMM is not set
# CONFIG_NET_VENDOR_RDC is not set
CONFIG_8139CP=m
# CONFIG_NET_VENDOR_ROCKER is not set
# CONFIG_NET_VENDOR_SAMSUNG is not set
# CONFIG_NET_VENDOR_SEEQ is not set
-# CONFIG_NET_VENDOR_SOLARFLARE is not set
# CONFIG_NET_VENDOR_SILAN is not set
# CONFIG_NET_VENDOR_SIS is not set
+# CONFIG_NET_VENDOR_SOLARFLARE is not set
# CONFIG_NET_VENDOR_SMSC is not set
CONFIG_STMMAC_ETH=y
# CONFIG_NET_VENDOR_SUN is not set
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_SHARE_IRQ=y
CONFIG_SERIAL_8250_RSA=y
+CONFIG_SERIAL_OF_PLATFORM=y
CONFIG_SERIAL_NONSTANDARD=y
CONFIG_PRINTER=m
CONFIG_VIRTIO_CONSOLE=y
CONFIG_SPI=y
CONFIG_GPIO_SYSFS=y
CONFIG_GPIO_LOONGSON=y
+CONFIG_POWER_RESET=y
+CONFIG_POWER_RESET_RESTART=y
+CONFIG_POWER_RESET_SYSCON=y
+CONFIG_POWER_RESET_SYSCON_POWEROFF=y
+CONFIG_SYSCON_REBOOT_MODE=y
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM93=m
CONFIG_SENSORS_W83795=m
CONFIG_RC_CORE=m
CONFIG_LIRC=y
CONFIG_RC_DECODERS=y
+CONFIG_IR_IMON_DECODER=m
+CONFIG_IR_JVC_DECODER=m
+CONFIG_IR_MCE_KBD_DECODER=m
CONFIG_IR_NEC_DECODER=m
CONFIG_IR_RC5_DECODER=m
CONFIG_IR_RC6_DECODER=m
-CONFIG_IR_JVC_DECODER=m
-CONFIG_IR_SONY_DECODER=m
CONFIG_IR_SANYO_DECODER=m
CONFIG_IR_SHARP_DECODER=m
-CONFIG_IR_MCE_KBD_DECODER=m
+CONFIG_IR_SONY_DECODER=m
CONFIG_IR_XMP_DECODER=m
-CONFIG_IR_IMON_DECODER=m
CONFIG_MEDIA_SUPPORT=m
CONFIG_MEDIA_USB_SUPPORT=y
CONFIG_USB_VIDEO_CLASS=m
CONFIG_FB=y
CONFIG_FB_EFI=y
CONFIG_FB_RADEON=y
+CONFIG_LCD_CLASS_DEVICE=y
CONFIG_LCD_PLATFORM=m
# CONFIG_VGA_CONSOLE is not set
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_SND=y
CONFIG_SND_SEQUENCER=m
CONFIG_SND_SEQ_DUMMY=m
-# CONFIG_SND_ISA is not set
CONFIG_SND_BT87X=m
CONFIG_SND_BT87X_OVERCLOCK=y
CONFIG_SND_HDA_INTEL=y
# CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set
CONFIG_CRYPTO_PCRYPT=m
CONFIG_CRYPTO_CRYPTD=m
-CONFIG_CRYPTO_CHACHA20POLY1305=m
-CONFIG_CRYPTO_HMAC=y
-CONFIG_CRYPTO_VMAC=m
-CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_ANUBIS=m
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
+CONFIG_CRYPTO_CHACHA20POLY1305=m
+CONFIG_CRYPTO_VMAC=m
+CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_PRINTK_TIME=y
CONFIG_STRIP_ASM_SYMS=y
CONFIG_MAGIC_SYSRQ=y
+CONFIG_DEBUG_FS=y
# CONFIG_SCHED_DEBUG is not set
CONFIG_SCHEDSTATS=y
# CONFIG_DEBUG_PREEMPT is not set
#define ACPI_TABLE_UPGRADE_MAX_PHYS ARCH_LOW_ADDRESS_LIMIT
+extern int loongarch_acpi_suspend(void);
+extern int (*acpi_suspend_lowlevel)(void);
+extern void loongarch_suspend_enter(void);
+
+static inline unsigned long acpi_get_wakeup_address(void)
+{
+ extern void loongarch_wakeup_start(void);
+ return (unsigned long)loongarch_wakeup_start;
+}
+
#endif /* _ASM_LOONGARCH_ACPI_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_ALTERNATIVE_ASM_H
+#define _ASM_ALTERNATIVE_ASM_H
+
+#ifdef __ASSEMBLY__
+
+#include <asm/asm.h>
+
+/*
+ * Issue one struct alt_instr descriptor entry (need to put it into
+ * the section .altinstructions, see below). This entry contains
+ * enough information for the alternatives patching code to patch an
+ * instruction. See apply_alternatives().
+ */
+.macro altinstruction_entry orig alt feature orig_len alt_len
+ .long \orig - .
+ .long \alt - .
+ .short \feature
+ .byte \orig_len
+ .byte \alt_len
+.endm
+
+/*
+ * Define an alternative between two instructions. If @feature is
+ * present, early code in apply_alternatives() replaces @oldinstr with
+ * @newinstr. ".fill" directive takes care of proper instruction padding
+ * in case @newinstr is longer than @oldinstr.
+ */
+.macro ALTERNATIVE oldinstr, newinstr, feature
+140 :
+ \oldinstr
+141 :
+ .fill - (((144f-143f)-(141b-140b)) > 0) * ((144f-143f)-(141b-140b)) / 4, 4, 0x03400000
+142 :
+
+ .pushsection .altinstructions, "a"
+ altinstruction_entry 140b, 143f, \feature, 142b-140b, 144f-143f
+ .popsection
+
+ .subsection 1
+143 :
+ \newinstr
+144 :
+ .previous
+.endm
+
+#define old_len (141b-140b)
+#define new_len1 (144f-143f)
+#define new_len2 (145f-144f)
+
+#define alt_max_short(a, b) ((a) ^ (((a) ^ (b)) & -(-((a) < (b)))))
+
+/*
+ * Same as ALTERNATIVE macro above but for two alternatives. If CPU
+ * has @feature1, it replaces @oldinstr with @newinstr1. If CPU has
+ * @feature2, it replaces @oldinstr with @feature2.
+ */
+.macro ALTERNATIVE_2 oldinstr, newinstr1, feature1, newinstr2, feature2
+140 :
+ \oldinstr
+141 :
+ .fill - ((alt_max_short(new_len1, new_len2) - (old_len)) > 0) * \
+ (alt_max_short(new_len1, new_len2) - (old_len)) / 4, 4, 0x03400000
+142 :
+
+ .pushsection .altinstructions, "a"
+ altinstruction_entry 140b, 143f, \feature1, 142b-140b, 144f-143f, 142b-141b
+ altinstruction_entry 140b, 144f, \feature2, 142b-140b, 145f-144f, 142b-141b
+ .popsection
+
+ .subsection 1
+143 :
+ \newinstr1
+144 :
+ \newinstr2
+145 :
+ .previous
+.endm
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_ALTERNATIVE_ASM_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_ALTERNATIVE_H
+#define _ASM_ALTERNATIVE_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+#include <linux/stddef.h>
+#include <linux/stringify.h>
+#include <asm/asm.h>
+
+struct alt_instr {
+ s32 instr_offset; /* offset to original instruction */
+ s32 replace_offset; /* offset to replacement instruction */
+ u16 feature; /* feature bit set for replacement */
+ u8 instrlen; /* length of original instruction */
+ u8 replacementlen; /* length of new instruction */
+} __packed;
+
+/*
+ * Debug flag that can be tested to see whether alternative
+ * instructions were patched in already:
+ */
+extern int alternatives_patched;
+extern struct alt_instr __alt_instructions[], __alt_instructions_end[];
+
+extern void alternative_instructions(void);
+extern void apply_alternatives(struct alt_instr *start, struct alt_instr *end);
+
+#define b_replacement(num) "664"#num
+#define e_replacement(num) "665"#num
+
+#define alt_end_marker "663"
+#define alt_slen "662b-661b"
+#define alt_total_slen alt_end_marker"b-661b"
+#define alt_rlen(num) e_replacement(num)"f-"b_replacement(num)"f"
+
+#define __OLDINSTR(oldinstr, num) \
+ "661:\n\t" oldinstr "\n662:\n" \
+ ".fill -(((" alt_rlen(num) ")-(" alt_slen ")) > 0) * " \
+ "((" alt_rlen(num) ")-(" alt_slen ")) / 4, 4, 0x03400000\n"
+
+#define OLDINSTR(oldinstr, num) \
+ __OLDINSTR(oldinstr, num) \
+ alt_end_marker ":\n"
+
+#define alt_max_short(a, b) "((" a ") ^ (((" a ") ^ (" b ")) & -(-((" a ") < (" b ")))))"
+
+/*
+ * Pad the second replacement alternative with additional NOPs if it is
+ * additionally longer than the first replacement alternative.
+ */
+#define OLDINSTR_2(oldinstr, num1, num2) \
+ "661:\n\t" oldinstr "\n662:\n" \
+ ".fill -((" alt_max_short(alt_rlen(num1), alt_rlen(num2)) " - (" alt_slen ")) > 0) * " \
+ "(" alt_max_short(alt_rlen(num1), alt_rlen(num2)) " - (" alt_slen ")) / 4, " \
+ "4, 0x03400000\n" \
+ alt_end_marker ":\n"
+
+#define ALTINSTR_ENTRY(feature, num) \
+ " .long 661b - .\n" /* label */ \
+ " .long " b_replacement(num)"f - .\n" /* new instruction */ \
+ " .short " __stringify(feature) "\n" /* feature bit */ \
+ " .byte " alt_total_slen "\n" /* source len */ \
+ " .byte " alt_rlen(num) "\n" /* replacement len */
+
+#define ALTINSTR_REPLACEMENT(newinstr, feature, num) /* replacement */ \
+ b_replacement(num)":\n\t" newinstr "\n" e_replacement(num) ":\n\t"
+
+/* alternative assembly primitive: */
+#define ALTERNATIVE(oldinstr, newinstr, feature) \
+ OLDINSTR(oldinstr, 1) \
+ ".pushsection .altinstructions,\"a\"\n" \
+ ALTINSTR_ENTRY(feature, 1) \
+ ".popsection\n" \
+ ".subsection 1\n" \
+ ALTINSTR_REPLACEMENT(newinstr, feature, 1) \
+ ".previous\n"
+
+#define ALTERNATIVE_2(oldinstr, newinstr1, feature1, newinstr2, feature2)\
+ OLDINSTR_2(oldinstr, 1, 2) \
+ ".pushsection .altinstructions,\"a\"\n" \
+ ALTINSTR_ENTRY(feature1, 1) \
+ ALTINSTR_ENTRY(feature2, 2) \
+ ".popsection\n" \
+ ".subsection 1\n" \
+ ALTINSTR_REPLACEMENT(newinstr1, feature1, 1) \
+ ALTINSTR_REPLACEMENT(newinstr2, feature2, 2) \
+ ".previous\n"
+
+/*
+ * Alternative instructions for different CPU types or capabilities.
+ *
+ * This allows to use optimized instructions even on generic binary
+ * kernels.
+ *
+ * length of oldinstr must be longer or equal the length of newinstr
+ * It can be padded with nops as needed.
+ *
+ * For non barrier like inlines please define new variants
+ * without volatile and memory clobber.
+ */
+#define alternative(oldinstr, newinstr, feature) \
+ (asm volatile (ALTERNATIVE(oldinstr, newinstr, feature) : : : "memory"))
+
+#define alternative_2(oldinstr, newinstr1, feature1, newinstr2, feature2) \
+ (asm volatile(ALTERNATIVE_2(oldinstr, newinstr1, feature1, newinstr2, feature2) ::: "memory"))
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_ALTERNATIVE_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_ASM_EXTABLE_H
+#define __ASM_ASM_EXTABLE_H
+
+#define EX_TYPE_NONE 0
+#define EX_TYPE_FIXUP 1
+#define EX_TYPE_UACCESS_ERR_ZERO 2
+#define EX_TYPE_BPF 3
+
+#ifdef __ASSEMBLY__
+
+#define __ASM_EXTABLE_RAW(insn, fixup, type, data) \
+ .pushsection __ex_table, "a"; \
+ .balign 4; \
+ .long ((insn) - .); \
+ .long ((fixup) - .); \
+ .short (type); \
+ .short (data); \
+ .popsection;
+
+ .macro _asm_extable, insn, fixup
+ __ASM_EXTABLE_RAW(\insn, \fixup, EX_TYPE_FIXUP, 0)
+ .endm
+
+#else /* __ASSEMBLY__ */
+
+#include <linux/bits.h>
+#include <linux/stringify.h>
+#include <asm/gpr-num.h>
+
+#define __ASM_EXTABLE_RAW(insn, fixup, type, data) \
+ ".pushsection __ex_table, \"a\"\n" \
+ ".balign 4\n" \
+ ".long ((" insn ") - .)\n" \
+ ".long ((" fixup ") - .)\n" \
+ ".short (" type ")\n" \
+ ".short (" data ")\n" \
+ ".popsection\n"
+
+#define _ASM_EXTABLE(insn, fixup) \
+ __ASM_EXTABLE_RAW(#insn, #fixup, __stringify(EX_TYPE_FIXUP), "0")
+
+#define EX_DATA_REG_ERR_SHIFT 0
+#define EX_DATA_REG_ERR GENMASK(4, 0)
+#define EX_DATA_REG_ZERO_SHIFT 5
+#define EX_DATA_REG_ZERO GENMASK(9, 5)
+
+#define EX_DATA_REG(reg, gpr) \
+ "((.L__gpr_num_" #gpr ") << " __stringify(EX_DATA_REG_##reg##_SHIFT) ")"
+
+#define _ASM_EXTABLE_UACCESS_ERR_ZERO(insn, fixup, err, zero) \
+ __DEFINE_ASM_GPR_NUMS \
+ __ASM_EXTABLE_RAW(#insn, #fixup, \
+ __stringify(EX_TYPE_UACCESS_ERR_ZERO), \
+ "(" \
+ EX_DATA_REG(ERR, err) " | " \
+ EX_DATA_REG(ZERO, zero) \
+ ")")
+
+#define _ASM_EXTABLE_UACCESS_ERR(insn, fixup, err) \
+ _ASM_EXTABLE_UACCESS_ERR_ZERO(insn, fixup, err, zero)
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_ASM_EXTABLE_H */
int cores_per_node;
int cores_per_package;
unsigned long cores_io_master;
+ unsigned long suspend_addr;
const char *cpuname;
};
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * This is included by init/main.c to check for architecture-dependent bugs.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#ifndef _ASM_BUGS_H
+#define _ASM_BUGS_H
+
+#include <asm/cpu.h>
+#include <asm/cpu-info.h>
+
+extern void check_bugs(void);
+
+#endif /* _ASM_BUGS_H */
void __init efi_init(void);
void __init efi_runtime_init(void);
+void __init *efi_fdt_pointer(void);
void efifb_setup_from_dmi(struct screen_info *si, const char *opt);
#define ARCH_EFI_IRQ_FLAGS_MASK 0x00000004 /* Bit 2: CSR.CRMD.IE */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LOONGARCH_EXTABLE_H
+#define _ASM_LOONGARCH_EXTABLE_H
+
+/*
+ * The exception table consists of pairs of relative offsets: the first
+ * is the relative offset to an instruction that is allowed to fault,
+ * and the second is the relative offset at which the program should
+ * continue. No registers are modified, so it is entirely up to the
+ * continuation code to figure out what to do.
+ *
+ * All the routines below use bits of fixup code that are out of line
+ * with the main instruction path. This means when everything is well,
+ * we don't even have to jump over them. Further, they do not intrude
+ * on our cache or tlb entries.
+ */
+
+struct exception_table_entry {
+ int insn, fixup;
+ short type, data;
+};
+
+#define ARCH_HAS_RELATIVE_EXTABLE
+
+#define swap_ex_entry_fixup(a, b, tmp, delta) \
+do { \
+ (a)->fixup = (b)->fixup + (delta); \
+ (b)->fixup = (tmp).fixup - (delta); \
+ (a)->type = (b)->type; \
+ (b)->type = (tmp).type; \
+ (a)->data = (b)->data; \
+ (b)->data = (tmp).data; \
+} while (0)
+
+#ifdef CONFIG_BPF_JIT
+bool ex_handler_bpf(const struct exception_table_entry *ex, struct pt_regs *regs);
+#else
+static inline
+bool ex_handler_bpf(const struct exception_table_entry *ex, struct pt_regs *regs)
+{
+ return false;
+}
+#endif /* !CONFIG_BPF_JIT */
+
+bool fixup_exception(struct pt_regs *regs);
+
+#endif
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef _ASM_LOONGARCH_FTRACE_H
+#define _ASM_LOONGARCH_FTRACE_H
+
+#define FTRACE_PLT_IDX 0
+#define FTRACE_REGS_PLT_IDX 1
+#define NR_FTRACE_PLTS 2
+
+#define GRAPH_FAKE_OFFSET (sizeof(struct pt_regs) - offsetof(struct pt_regs, regs[1]))
+
+#ifdef CONFIG_FUNCTION_TRACER
+
+#define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
+
+#ifndef __ASSEMBLY__
+
+#ifndef CONFIG_DYNAMIC_FTRACE
+
+#define mcount _mcount
+extern void _mcount(void);
+extern void prepare_ftrace_return(unsigned long self_addr, unsigned long callsite_sp, unsigned long old);
+
+#else
+
+struct dyn_ftrace;
+struct dyn_arch_ftrace { };
+
+#define ARCH_SUPPORTS_FTRACE_OPS 1
+#define HAVE_FUNCTION_GRAPH_RET_ADDR_PTR
+
+#define ftrace_init_nop ftrace_init_nop
+int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
+
+static inline unsigned long ftrace_call_adjust(unsigned long addr)
+{
+ return addr;
+}
+
+void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent);
+
+#endif /* CONFIG_DYNAMIC_FTRACE */
+
+#ifdef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS
+struct ftrace_ops;
+
+struct ftrace_regs {
+ struct pt_regs regs;
+};
+
+static __always_inline struct pt_regs *arch_ftrace_get_regs(struct ftrace_regs *fregs)
+{
+ return &fregs->regs;
+}
+
+#define ftrace_graph_func ftrace_graph_func
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct ftrace_regs *fregs);
+#endif
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* CONFIG_FUNCTION_TRACER */
+
+#endif /* _ASM_LOONGARCH_FTRACE_H */
#include <linux/futex.h>
#include <linux/uaccess.h>
+#include <asm/asm-extable.h>
#include <asm/barrier.h>
#include <asm/errno.h>
"2: sc.w $t0, %2 \n" \
" beqz $t0, 1b \n" \
"3: \n" \
- " .section .fixup,\"ax\" \n" \
- "4: li.w %0, %6 \n" \
- " b 3b \n" \
- " .previous \n" \
- " .section __ex_table,\"a\" \n" \
- " "__UA_ADDR "\t1b, 4b \n" \
- " "__UA_ADDR "\t2b, 4b \n" \
- " .previous \n" \
+ _ASM_EXTABLE_UACCESS_ERR(1b, 3b, %0) \
+ _ASM_EXTABLE_UACCESS_ERR(2b, 3b, %0) \
: "=r" (ret), "=&r" (oldval), \
"=ZC" (*uaddr) \
- : "0" (0), "ZC" (*uaddr), "Jr" (oparg), \
- "i" (-EFAULT) \
+ : "0" (0), "ZC" (*uaddr), "Jr" (oparg) \
: "memory", "t0"); \
}
" beqz $t0, 1b \n"
"3: \n"
__WEAK_LLSC_MB
- " .section .fixup,\"ax\" \n"
- "4: li.d %0, %6 \n"
- " b 3b \n"
- " .previous \n"
- " .section __ex_table,\"a\" \n"
- " "__UA_ADDR "\t1b, 4b \n"
- " "__UA_ADDR "\t2b, 4b \n"
- " .previous \n"
+ _ASM_EXTABLE_UACCESS_ERR(1b, 3b, %0)
+ _ASM_EXTABLE_UACCESS_ERR(2b, 3b, %0)
: "+r" (ret), "=&r" (val), "=ZC" (*uaddr)
- : "ZC" (*uaddr), "Jr" (oldval), "Jr" (newval),
- "i" (-EFAULT)
+ : "ZC" (*uaddr), "Jr" (oldval), "Jr" (newval)
: "memory", "t0");
*uval = val;
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_GPR_NUM_H
+#define __ASM_GPR_NUM_H
+
+#ifdef __ASSEMBLY__
+
+ .equ .L__gpr_num_zero, 0
+ .irp num,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31
+ .equ .L__gpr_num_$r\num, \num
+ .endr
+
+#else /* __ASSEMBLY__ */
+
+#define __DEFINE_ASM_GPR_NUMS \
+" .equ .L__gpr_num_zero, 0\n" \
+" .irp num,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31\n" \
+" .equ .L__gpr_num_$r\\num, \\num\n" \
+" .endr\n" \
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_GPR_NUM_H */
#include <linux/types.h>
#include <asm/asm.h>
+#define INSN_NOP 0x03400000
#define INSN_BREAK 0x002a0000
#define ADDR_IMMMASK_LU52ID 0xFFF0000000000000
#define ADDR_IMMMASK_LU32ID 0x000FFFFF00000000
+#define ADDR_IMMMASK_LU12IW 0x00000000FFFFF000
#define ADDR_IMMMASK_ADDU16ID 0x00000000FFFF0000
#define ADDR_IMMSHIFT_LU52ID 52
#define ADDR_IMMSHIFT_LU32ID 32
+#define ADDR_IMMSHIFT_LU12IW 12
#define ADDR_IMMSHIFT_ADDU16ID 16
#define ADDR_IMM(addr, INSN) ((addr & ADDR_IMMMASK_##INSN) >> ADDR_IMMSHIFT_##INSN)
enum reg1i20_op {
lu12iw_op = 0x0a,
lu32id_op = 0x0b,
+ pcaddi_op = 0x0c,
pcaddu12i_op = 0x0e,
pcaddu18i_op = 0x0f,
};
enum reg1i21_op {
beqz_op = 0x10,
bnez_op = 0x11,
+ bceqz_op = 0x12, /* bits[9:8] = 0x00 */
+ bcnez_op = 0x12, /* bits[9:8] = 0x01 */
};
enum reg2_op {
ldbu_op = 0xa8,
ldhu_op = 0xa9,
ldwu_op = 0xaa,
+ flds_op = 0xac,
+ fsts_op = 0xad,
+ fldd_op = 0xae,
+ fstd_op = 0xaf,
};
enum reg2i14_op {
ldxbu_op = 0x7040,
ldxhu_op = 0x7048,
ldxwu_op = 0x7050,
+ fldxs_op = 0x7060,
+ fldxd_op = 0x7068,
+ fstxs_op = 0x7070,
+ fstxd_op = 0x7078,
amswapw_op = 0x70c0,
amswapd_op = 0x70c1,
amaddw_op = 0x70c2,
return val & (1UL << (bit - 1));
}
+static inline bool is_pc_ins(union loongarch_instruction *ip)
+{
+ return ip->reg1i20_format.opcode >= pcaddi_op &&
+ ip->reg1i20_format.opcode <= pcaddu18i_op;
+}
+
static inline bool is_branch_ins(union loongarch_instruction *ip)
{
return ip->reg1i21_format.opcode >= beqz_op &&
is_imm12_negative(ip->reg2i12_format.immediate);
}
+int larch_insn_read(void *addr, u32 *insnp);
+int larch_insn_write(void *addr, u32 insn);
+int larch_insn_patch_text(void *addr, u32 insn);
+
+u32 larch_insn_gen_nop(void);
+u32 larch_insn_gen_b(unsigned long pc, unsigned long dest);
+u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest);
+
+u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj, enum loongarch_gpr rk);
+u32 larch_insn_gen_move(enum loongarch_gpr rd, enum loongarch_gpr rj);
+
+u32 larch_insn_gen_lu12iw(enum loongarch_gpr rd, int imm);
u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm);
u32 larch_insn_gen_lu52id(enum loongarch_gpr rd, enum loongarch_gpr rj, int imm);
u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned long pc, unsigned long dest);
return val < (1UL << bit);
}
+static inline unsigned long sign_extend(unsigned long val, unsigned int idx)
+{
+ if (!is_imm_negative(val, idx + 1))
+ return ((1UL << idx) - 1) & val;
+ else
+ return ~((1UL << idx) - 1) | val;
+}
+
#define DEF_EMIT_REG0I26_FORMAT(NAME, OP) \
static inline void emit_##NAME(union loongarch_instruction *insn, \
int offset) \
DEF_EMIT_REG3SA2_FORMAT(alsld, alsld_op)
+struct pt_regs;
+
+void emulate_load_store_insn(struct pt_regs *regs, void __user *addr, unsigned int *pc);
+unsigned long unaligned_read(void __user *addr, void *value, unsigned long n, bool sign);
+unsigned long unaligned_write(void __user *addr, unsigned long value, unsigned long n);
+
#endif /* _ASM_INST_H */
#define ls7a_writel(val, addr) *(volatile unsigned int *)TO_UNCACHE(addr) = (val)
#define ls7a_writeq(val, addr) *(volatile unsigned long *)TO_UNCACHE(addr) = (val)
+void enable_gpe_wakeup(void);
+void enable_pci_wakeup(void);
+
#endif /* __ASM_LOONGSON_H */
#define RELA_STACK_DEPTH 16
struct mod_section {
- Elf_Shdr *shdr;
+ int shndx;
int num_entries;
int max_entries;
};
struct mod_section got;
struct mod_section plt;
struct mod_section plt_idx;
+
+ /* For CONFIG_DYNAMIC_FTRACE */
+ struct plt_entry *ftrace_trampolines;
};
struct got_entry {
Elf_Addr symbol_addr;
};
-Elf_Addr module_emit_got_entry(struct module *mod, Elf_Addr val);
-Elf_Addr module_emit_plt_entry(struct module *mod, Elf_Addr val);
+Elf_Addr module_emit_got_entry(struct module *mod, Elf_Shdr *sechdrs, Elf_Addr val);
+Elf_Addr module_emit_plt_entry(struct module *mod, Elf_Shdr *sechdrs, Elf_Addr val);
static inline struct got_entry emit_got_entry(Elf_Addr val)
{
{
u32 lu12iw, lu32id, lu52id, jirl;
- lu12iw = (lu12iw_op << 25 | (((val >> 12) & 0xfffff) << 5) | LOONGARCH_GPR_T1);
+ lu12iw = larch_insn_gen_lu12iw(LOONGARCH_GPR_T1, ADDR_IMM(val, LU12IW));
lu32id = larch_insn_gen_lu32id(LOONGARCH_GPR_T1, ADDR_IMM(val, LU32ID));
lu52id = larch_insn_gen_lu52id(LOONGARCH_GPR_T1, LOONGARCH_GPR_T1, ADDR_IMM(val, LU52ID));
jirl = larch_insn_gen_jirl(0, LOONGARCH_GPR_T1, 0, (val & 0xfff));
return (struct plt_idx_entry) { val };
}
-static inline int get_plt_idx(unsigned long val, const struct mod_section *sec)
+static inline int get_plt_idx(unsigned long val, Elf_Shdr *sechdrs, const struct mod_section *sec)
{
int i;
- struct plt_idx_entry *plt_idx = (struct plt_idx_entry *)sec->shdr->sh_addr;
+ struct plt_idx_entry *plt_idx = (struct plt_idx_entry *)sechdrs[sec->shndx].sh_addr;
for (i = 0; i < sec->num_entries; i++) {
if (plt_idx[i].symbol_addr == val)
}
static inline struct plt_entry *get_plt_entry(unsigned long val,
- const struct mod_section *sec_plt,
- const struct mod_section *sec_plt_idx)
+ Elf_Shdr *sechdrs,
+ const struct mod_section *sec_plt,
+ const struct mod_section *sec_plt_idx)
{
- int plt_idx = get_plt_idx(val, sec_plt_idx);
- struct plt_entry *plt = (struct plt_entry *)sec_plt->shdr->sh_addr;
+ int plt_idx = get_plt_idx(val, sechdrs, sec_plt_idx);
+ struct plt_entry *plt = (struct plt_entry *)sechdrs[sec_plt->shndx].sh_addr;
if (plt_idx < 0)
return NULL;
}
static inline struct got_entry *get_got_entry(Elf_Addr val,
+ Elf_Shdr *sechdrs,
const struct mod_section *sec)
{
- struct got_entry *got = (struct got_entry *)sec->shdr->sh_addr;
int i;
+ struct got_entry *got = (struct got_entry *)sechdrs[sec->shndx].sh_addr;
for (i = 0; i < sec->num_entries; i++)
if (got[i].symbol_addr == val)
.got : { BYTE(0) }
.plt : { BYTE(0) }
.plt.idx : { BYTE(0) }
+ .ftrace_trampoline : { BYTE(0) }
}
extern unsigned long eentry;
extern unsigned long tlbrentry;
+extern char init_command_line[COMMAND_LINE_SIZE];
extern void tlb_init(int cpu);
extern void cpu_cache_init(void);
extern void cache_error_setup(void);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * GCC stack protector support.
+ *
+ * Stack protector works by putting predefined pattern at the start of
+ * the stack frame and verifying that it hasn't been overwritten when
+ * returning from the function. The pattern is called stack canary and
+ * on LoongArch gcc expects it to be defined by a global variable called
+ * "__stack_chk_guard".
+ */
+
+#ifndef _ASM_STACKPROTECTOR_H
+#define _ASM_STACKPROTECTOR_H
+
+#include <linux/random.h>
+#include <linux/version.h>
+
+extern unsigned long __stack_chk_guard;
+
+/*
+ * Initialize the stackprotector canary value.
+ *
+ * NOTE: this must only be called from functions that never return,
+ * and it must always be inlined.
+ */
+static __always_inline void boot_init_stack_canary(void)
+{
+ unsigned long canary;
+
+ /* Try to get a semi random initial value. */
+ get_random_bytes(&canary, sizeof(canary));
+ canary ^= LINUX_VERSION_CODE;
+
+ current->stack_canary = canary;
+ __stack_chk_guard = current->stack_canary;
+}
+
+#endif /* _ASM_STACKPROTECTOR_H */
#ifndef _ASM_STRING_H
#define _ASM_STRING_H
+#define __HAVE_ARCH_MEMSET
extern void *memset(void *__s, int __c, size_t __count);
+
+#define __HAVE_ARCH_MEMCPY
extern void *memcpy(void *__to, __const__ void *__from, size_t __n);
+
+#define __HAVE_ARCH_MEMMOVE
extern void *memmove(void *__dest, __const__ void *__src, size_t __n);
#endif /* _ASM_STRING_H */
#define INIT_THREAD_INFO(tsk) \
{ \
.task = &tsk, \
- .flags = 0, \
+ .flags = _TIF_FIXADE, \
.cpu = 0, \
.preempt_count = INIT_PREEMPT_COUNT, \
}
extern u64 cpu_clock_freq;
extern u64 const_clock_freq;
+extern void save_counter(void);
extern void sync_counter(void);
static inline unsigned int calc_const_freq(void)
#include <linux/string.h>
#include <linux/extable.h>
#include <asm/pgtable.h>
-#include <asm-generic/extable.h>
+#include <asm/extable.h>
+#include <asm/asm-extable.h>
#include <asm-generic/access_ok.h>
extern u64 __ua_limit;
__asm__ __volatile__( \
"1: " insn " %1, %2 \n" \
"2: \n" \
- " .section .fixup,\"ax\" \n" \
- "3: li.w %0, %3 \n" \
- " move %1, $zero \n" \
- " b 2b \n" \
- " .previous \n" \
- " .section __ex_table,\"a\" \n" \
- " "__UA_ADDR "\t1b, 3b \n" \
- " .previous \n" \
+ _ASM_EXTABLE_UACCESS_ERR_ZERO(1b, 2b, %0, %1) \
: "+r" (__gu_err), "=r" (__gu_tmp) \
- : "m" (__m(ptr)), "i" (-EFAULT)); \
+ : "m" (__m(ptr))); \
\
(val) = (__typeof__(*(ptr))) __gu_tmp; \
}
__asm__ __volatile__( \
"1: " insn " %z2, %1 # __put_user_asm\n" \
"2: \n" \
- " .section .fixup,\"ax\" \n" \
- "3: li.w %0, %3 \n" \
- " b 2b \n" \
- " .previous \n" \
- " .section __ex_table,\"a\" \n" \
- " " __UA_ADDR " 1b, 3b \n" \
- " .previous \n" \
+ _ASM_EXTABLE_UACCESS_ERR(1b, 2b, %0) \
: "+r" (__pu_err), "=m" (__m(ptr)) \
- : "Jr" (__pu_val), "i" (-EFAULT)); \
+ : "Jr" (__pu_val)); \
}
#define __get_kernel_nofault(dst, src, type, err_label) \
char type; /* UNWINDER_XXX */
struct stack_info stack_info;
struct task_struct *task;
- bool first, error;
+ bool first, error, is_ftrace;
+ int graph_idx;
unsigned long sp, pc, ra;
};
obj-y += head.o cpu-probe.o cacheinfo.o env.o setup.o entry.o genex.o \
traps.o irq.o idle.o process.o dma.o mem.o io.o reset.o switch.o \
- elf.o syscall.o signal.o time.o topology.o inst.o ptrace.o vdso.o
+ elf.o syscall.o signal.o time.o topology.o inst.o ptrace.o vdso.o \
+ alternative.o unaligned.o
obj-$(CONFIG_ACPI) += acpi.o
obj-$(CONFIG_EFI) += efi.o
obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
+ifdef CONFIG_FUNCTION_TRACER
+ ifndef CONFIG_DYNAMIC_FTRACE
+ obj-y += mcount.o ftrace.o
+ CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
+ else
+ obj-y += mcount_dyn.o ftrace_dyn.o
+ CFLAGS_REMOVE_ftrace_dyn.o = $(CC_FLAGS_FTRACE)
+ endif
+ CFLAGS_REMOVE_inst.o = $(CC_FLAGS_FTRACE)
+ CFLAGS_REMOVE_time.o = $(CC_FLAGS_FTRACE)
+ CFLAGS_REMOVE_perf_event.o = $(CC_FLAGS_FTRACE)
+endif
+
obj-$(CONFIG_MODULES) += module.o module-sections.o
obj-$(CONFIG_STACKTRACE) += stacktrace.o
#include <linux/irq.h>
#include <linux/irqdomain.h>
#include <linux/memblock.h>
+#include <linux/of_fdt.h>
#include <linux/serial_core.h>
#include <asm/io.h>
#include <asm/numa.h>
loongson_sysconf.nr_cpus = num_processors;
}
+#ifndef CONFIG_SUSPEND
+int (*acpi_suspend_lowlevel)(void);
+#else
+int (*acpi_suspend_lowlevel)(void) = loongarch_acpi_suspend;
+#endif
+
void __init acpi_boot_table_init(void)
{
/*
* If acpi_disabled, bail out
*/
if (acpi_disabled)
- return;
+ goto fdt_earlycon;
/*
* Initialize the ACPI boot-time table parser.
*/
if (acpi_table_init()) {
disable_acpi();
- return;
+ goto fdt_earlycon;
}
loongson_sysconf.boot_cpu_id = read_csr_cpuid();
/* Do not enable ACPI SPCR console by default */
acpi_parse_spcr(earlycon_acpi_spcr_enable, false);
+
+ return;
+
+fdt_earlycon:
+ if (earlycon_acpi_spcr_enable)
+ early_init_dt_scan_chosen_stdout();
}
#ifdef CONFIG_ACPI_NUMA
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <asm/alternative.h>
+#include <asm/cacheflush.h>
+#include <asm/inst.h>
+#include <asm/sections.h>
+
+int __read_mostly alternatives_patched;
+
+EXPORT_SYMBOL_GPL(alternatives_patched);
+
+#define MAX_PATCH_SIZE (((u8)(-1)) / LOONGARCH_INSN_SIZE)
+
+static int __initdata_or_module debug_alternative;
+
+static int __init debug_alt(char *str)
+{
+ debug_alternative = 1;
+ return 1;
+}
+__setup("debug-alternative", debug_alt);
+
+#define DPRINTK(fmt, args...) \
+do { \
+ if (debug_alternative) \
+ printk(KERN_DEBUG "%s: " fmt "\n", __func__, ##args); \
+} while (0)
+
+#define DUMP_WORDS(buf, count, fmt, args...) \
+do { \
+ if (unlikely(debug_alternative)) { \
+ int _j; \
+ union loongarch_instruction *_buf = buf; \
+ \
+ if (!(count)) \
+ break; \
+ \
+ printk(KERN_DEBUG fmt, ##args); \
+ for (_j = 0; _j < count - 1; _j++) \
+ printk(KERN_CONT "<%08x> ", _buf[_j].word); \
+ printk(KERN_CONT "<%08x>\n", _buf[_j].word); \
+ } \
+} while (0)
+
+/* Use this to add nops to a buffer, then text_poke the whole buffer. */
+static void __init_or_module add_nops(union loongarch_instruction *insn, int count)
+{
+ while (count--) {
+ insn->word = INSN_NOP;
+ insn++;
+ }
+}
+
+/* Is the jump addr in local .altinstructions */
+static inline bool in_alt_jump(unsigned long jump, void *start, void *end)
+{
+ return jump >= (unsigned long)start && jump < (unsigned long)end;
+}
+
+static void __init_or_module recompute_jump(union loongarch_instruction *buf,
+ union loongarch_instruction *dest, union loongarch_instruction *src,
+ void *start, void *end)
+{
+ unsigned int si, si_l, si_h;
+ unsigned long cur_pc, jump_addr, pc;
+ long offset;
+
+ cur_pc = (unsigned long)src;
+ pc = (unsigned long)dest;
+
+ si_l = src->reg0i26_format.immediate_l;
+ si_h = src->reg0i26_format.immediate_h;
+ switch (src->reg0i26_format.opcode) {
+ case b_op:
+ case bl_op:
+ jump_addr = cur_pc + sign_extend((si_h << 16 | si_l) << 2, 27);
+ if (in_alt_jump(jump_addr, start, end))
+ return;
+ offset = jump_addr - pc;
+ BUG_ON(offset < -SZ_128M || offset >= SZ_128M);
+ offset >>= 2;
+ buf->reg0i26_format.immediate_h = offset >> 16;
+ buf->reg0i26_format.immediate_l = offset;
+ return;
+ }
+
+ si_l = src->reg1i21_format.immediate_l;
+ si_h = src->reg1i21_format.immediate_h;
+ switch (src->reg1i21_format.opcode) {
+ case bceqz_op: /* bceqz_op = bcnez_op */
+ BUG_ON(buf->reg1i21_format.rj & BIT(4));
+ fallthrough;
+ case beqz_op:
+ case bnez_op:
+ jump_addr = cur_pc + sign_extend((si_h << 16 | si_l) << 2, 22);
+ if (in_alt_jump(jump_addr, start, end))
+ return;
+ offset = jump_addr - pc;
+ BUG_ON(offset < -SZ_4M || offset >= SZ_4M);
+ offset >>= 2;
+ buf->reg1i21_format.immediate_h = offset >> 16;
+ buf->reg1i21_format.immediate_l = offset;
+ return;
+ }
+
+ si = src->reg2i16_format.immediate;
+ switch (src->reg2i16_format.opcode) {
+ case beq_op:
+ case bne_op:
+ case blt_op:
+ case bge_op:
+ case bltu_op:
+ case bgeu_op:
+ jump_addr = cur_pc + sign_extend(si << 2, 17);
+ if (in_alt_jump(jump_addr, start, end))
+ return;
+ offset = jump_addr - pc;
+ BUG_ON(offset < -SZ_128K || offset >= SZ_128K);
+ offset >>= 2;
+ buf->reg2i16_format.immediate = offset;
+ return;
+ }
+}
+
+static int __init_or_module copy_alt_insns(union loongarch_instruction *buf,
+ union loongarch_instruction *dest, union loongarch_instruction *src, int nr)
+{
+ int i;
+
+ for (i = 0; i < nr; i++) {
+ buf[i].word = src[i].word;
+
+ if (is_pc_ins(&src[i])) {
+ pr_err("Not support pcrel instruction at present!");
+ return -EINVAL;
+ }
+
+ if (is_branch_ins(&src[i]) &&
+ src[i].reg2i16_format.opcode != jirl_op) {
+ recompute_jump(&buf[i], &dest[i], &src[i], src, src + nr);
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * text_poke_early - Update instructions on a live kernel at boot time
+ *
+ * When you use this code to patch more than one byte of an instruction
+ * you need to make sure that other CPUs cannot execute this code in parallel.
+ * Also no thread must be currently preempted in the middle of these
+ * instructions. And on the local CPU you need to be protected again NMI or MCE
+ * handlers seeing an inconsistent instruction while you patch.
+ */
+static void *__init_or_module text_poke_early(union loongarch_instruction *insn,
+ union loongarch_instruction *buf, unsigned int nr)
+{
+ int i;
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ for (i = 0; i < nr; i++)
+ insn[i].word = buf[i].word;
+
+ local_irq_restore(flags);
+
+ wbflush();
+ flush_icache_range((unsigned long)insn, (unsigned long)(insn + nr));
+
+ return insn;
+}
+
+/*
+ * Replace instructions with better alternatives for this CPU type. This runs
+ * before SMP is initialized to avoid SMP problems with self modifying code.
+ * This implies that asymmetric systems where APs have less capabilities than
+ * the boot processor are not handled. Tough. Make sure you disable such
+ * features by hand.
+ */
+void __init_or_module apply_alternatives(struct alt_instr *start, struct alt_instr *end)
+{
+ struct alt_instr *a;
+ unsigned int nr_instr, nr_repl, nr_insnbuf;
+ union loongarch_instruction *instr, *replacement;
+ union loongarch_instruction insnbuf[MAX_PATCH_SIZE];
+
+ DPRINTK("alt table %px, -> %px", start, end);
+ /*
+ * The scan order should be from start to end. A later scanned
+ * alternative code can overwrite previously scanned alternative code.
+ * Some kernel functions (e.g. memcpy, memset, etc) use this order to
+ * patch code.
+ *
+ * So be careful if you want to change the scan order to any other
+ * order.
+ */
+ for (a = start; a < end; a++) {
+ nr_insnbuf = 0;
+
+ instr = (void *)&a->instr_offset + a->instr_offset;
+ replacement = (void *)&a->replace_offset + a->replace_offset;
+
+ BUG_ON(a->instrlen > sizeof(insnbuf));
+ BUG_ON(a->instrlen & 0x3);
+ BUG_ON(a->replacementlen & 0x3);
+
+ nr_instr = a->instrlen / LOONGARCH_INSN_SIZE;
+ nr_repl = a->replacementlen / LOONGARCH_INSN_SIZE;
+
+ if (!cpu_has(a->feature)) {
+ DPRINTK("feat not exist: %d, old: (%px len: %d), repl: (%px, len: %d)",
+ a->feature, instr, a->instrlen,
+ replacement, a->replacementlen);
+
+ continue;
+ }
+
+ DPRINTK("feat: %d, old: (%px len: %d), repl: (%px, len: %d)",
+ a->feature, instr, a->instrlen,
+ replacement, a->replacementlen);
+
+ DUMP_WORDS(instr, nr_instr, "%px: old_insn: ", instr);
+ DUMP_WORDS(replacement, nr_repl, "%px: rpl_insn: ", replacement);
+
+ copy_alt_insns(insnbuf, instr, replacement, nr_repl);
+ nr_insnbuf = nr_repl;
+
+ if (nr_instr > nr_repl) {
+ add_nops(insnbuf + nr_repl, nr_instr - nr_repl);
+ nr_insnbuf += nr_instr - nr_repl;
+ }
+ DUMP_WORDS(insnbuf, nr_insnbuf, "%px: final_insn: ", instr);
+
+ text_poke_early(instr, insnbuf, nr_insnbuf);
+ }
+}
+
+void __init alternative_instructions(void)
+{
+ apply_alternatives(__alt_instructions, __alt_instructions_end);
+
+ alternatives_patched = 1;
+}
OFFSET(TASK_FLAGS, task_struct, flags);
OFFSET(TASK_MM, task_struct, mm);
OFFSET(TASK_PID, task_struct, pid);
+#if defined(CONFIG_STACKPROTECTOR)
+ OFFSET(TASK_STACK_CANARY, task_struct, stack_canary);
+#endif
DEFINE(TASK_STRUCT_SIZE, sizeof(struct task_struct));
BLANK();
}
BLANK();
}
#endif
+
+#ifdef CONFIG_HIBERNATION
+void output_pbe_defines(void)
+{
+ COMMENT(" Linux struct pbe offsets. ");
+ OFFSET(PBE_ADDRESS, pbe, address);
+ OFFSET(PBE_ORIG_ADDRESS, pbe, orig_address);
+ OFFSET(PBE_NEXT, pbe, next);
+ DEFINE(PBE_SIZE, sizeof(struct pbe));
+ BLANK();
+}
+#endif
static unsigned long efi_config_table;
static unsigned long __initdata boot_memmap = EFI_INVALID_TABLE_ADDR;
+static unsigned long __initdata fdt_pointer = EFI_INVALID_TABLE_ADDR;
static efi_system_table_t *efi_systab;
static efi_config_table_type_t arch_tables[] __initdata = {
{LINUX_EFI_BOOT_MEMMAP_GUID, &boot_memmap, "MEMMAP" },
+ {DEVICE_TREE_GUID, &fdt_pointer, "FDTPTR" },
{},
};
+void __init *efi_fdt_pointer(void)
+{
+ if (!efi_systab)
+ return NULL;
+
+ if (fdt_pointer == EFI_INVALID_TABLE_ADDR)
+ return NULL;
+
+ return early_memremap_ro(fdt_pointer, SZ_64K);
+}
+
void __init efi_runtime_init(void)
{
- if (!efi_enabled(EFI_BOOT))
+ if (!efi_enabled(EFI_BOOT) || !efi_systab->runtime)
return;
if (efi_runtime_disabled()) {
#include <asm/early_ioremap.h>
#include <asm/bootinfo.h>
#include <asm/loongson.h>
+#include <asm/setup.h>
u64 efi_system_table;
struct loongson_system_configuration loongson_sysconf;
clear_bit(EFI_BOOT, &efi.flags);
strscpy(boot_command_line, cmdline, COMMAND_LINE_SIZE);
+ strscpy(init_command_line, cmdline, COMMAND_LINE_SIZE);
early_memunmap(cmdline, COMMAND_LINE_SIZE);
efi_system_table = fw_arg2;
*/
#include <asm/asm.h>
#include <asm/asmmacro.h>
+#include <asm/asm-extable.h>
#include <asm/asm-offsets.h>
#include <asm/errno.h>
#include <asm/export.h>
.macro EX insn, reg, src, offs
.ex\@: \insn \reg, \src, \offs
- .section __ex_table,"a"
- PTR .ex\@, fault
- .previous
+ _asm_extable .ex\@, fault
.endm
.macro sc_save_fp base
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/init.h>
+#include <linux/ftrace.h>
+#include <linux/syscalls.h>
+#include <linux/uaccess.h>
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/cacheflush.h>
+#include <asm/inst.h>
+#include <asm/loongarch.h>
+#include <asm/syscall.h>
+
+#include <asm-generic/sections.h>
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+
+/*
+ * As `call _mcount` follows LoongArch psABI, ra-saved operation and
+ * stack operation can be found before this insn.
+ */
+
+static int ftrace_get_parent_ra_addr(unsigned long insn_addr, int *ra_off)
+{
+ int limit = 32;
+ union loongarch_instruction *insn;
+
+ insn = (union loongarch_instruction *)insn_addr;
+
+ do {
+ insn--;
+ limit--;
+
+ if (is_ra_save_ins(insn))
+ *ra_off = -((1 << 12) - insn->reg2i12_format.immediate);
+
+ } while (!is_stack_alloc_ins(insn) && limit);
+
+ if (!limit)
+ return -EINVAL;
+
+ return 0;
+}
+
+void prepare_ftrace_return(unsigned long self_addr,
+ unsigned long callsite_sp, unsigned long old)
+{
+ int ra_off;
+ unsigned long return_hooker = (unsigned long)&return_to_handler;
+
+ if (unlikely(ftrace_graph_is_dead()))
+ return;
+
+ if (unlikely(atomic_read(¤t->tracing_graph_pause)))
+ return;
+
+ if (ftrace_get_parent_ra_addr(self_addr, &ra_off))
+ goto out;
+
+ if (!function_graph_enter(old, self_addr, 0, NULL))
+ *(unsigned long *)(callsite_sp + ra_off) = return_hooker;
+
+ return;
+
+out:
+ ftrace_graph_stop();
+ WARN_ON(1);
+}
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Based on arch/arm64/kernel/ftrace.c
+ *
+ * Copyright (C) 2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/ftrace.h>
+#include <linux/uaccess.h>
+
+#include <asm/inst.h>
+#include <asm/module.h>
+
+static int ftrace_modify_code(unsigned long pc, u32 old, u32 new, bool validate)
+{
+ u32 replaced;
+
+ if (validate) {
+ if (larch_insn_read((void *)pc, &replaced))
+ return -EFAULT;
+
+ if (replaced != old)
+ return -EINVAL;
+ }
+
+ if (larch_insn_patch_text((void *)pc, new))
+ return -EPERM;
+
+ return 0;
+}
+
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+
+#ifdef CONFIG_MODULES
+static inline int __get_mod(struct module **mod, unsigned long addr)
+{
+ preempt_disable();
+ *mod = __module_text_address(addr);
+ preempt_enable();
+
+ if (WARN_ON(!(*mod)))
+ return -EINVAL;
+
+ return 0;
+}
+
+static struct plt_entry *get_ftrace_plt(struct module *mod, unsigned long addr)
+{
+ struct plt_entry *plt = mod->arch.ftrace_trampolines;
+
+ if (addr == FTRACE_ADDR)
+ return &plt[FTRACE_PLT_IDX];
+ if (addr == FTRACE_REGS_ADDR &&
+ IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+ return &plt[FTRACE_REGS_PLT_IDX];
+
+ return NULL;
+}
+
+static unsigned long get_plt_addr(struct module *mod, unsigned long addr)
+{
+ struct plt_entry *plt;
+
+ plt = get_ftrace_plt(mod, addr);
+ if (!plt) {
+ pr_err("ftrace: no module PLT for %ps\n", (void *)addr);
+ return -EINVAL;
+ }
+
+ return (unsigned long)plt;
+}
+#endif
+
+int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
+{
+ u32 old, new;
+ unsigned long pc;
+ long offset __maybe_unused;
+
+ pc = rec->ip + LOONGARCH_INSN_SIZE;
+
+#ifdef CONFIG_MODULES
+ offset = (long)pc - (long)addr;
+
+ if (offset < -SZ_128M || offset >= SZ_128M) {
+ int ret;
+ struct module *mod;
+
+ ret = __get_mod(&mod, pc);
+ if (ret)
+ return ret;
+
+ addr = get_plt_addr(mod, addr);
+
+ old_addr = get_plt_addr(mod, old_addr);
+ }
+#endif
+
+ new = larch_insn_gen_bl(pc, addr);
+ old = larch_insn_gen_bl(pc, old_addr);
+
+ return ftrace_modify_code(pc, old, new, true);
+}
+
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
+
+int ftrace_update_ftrace_func(ftrace_func_t func)
+{
+ u32 new;
+ unsigned long pc;
+
+ pc = (unsigned long)&ftrace_call;
+ new = larch_insn_gen_bl(pc, (unsigned long)func);
+
+ return ftrace_modify_code(pc, 0, new, false);
+}
+
+/*
+ * The compiler has inserted 2 NOPs before the regular function prologue.
+ * T series registers are available and safe because of LoongArch's psABI.
+ *
+ * At runtime, we can replace nop with bl to enable ftrace call and replace bl
+ * with nop to disable ftrace call. The bl requires us to save the original RA
+ * value, so it saves RA at t0 here.
+ *
+ * Details are:
+ *
+ * | Compiled | Disabled | Enabled |
+ * +------------+------------------------+------------------------+
+ * | nop | move t0, ra | move t0, ra |
+ * | nop | nop | bl ftrace_caller |
+ * | func_body | func_body | func_body |
+ *
+ * The RA value will be recovered by ftrace_regs_entry, and restored into RA
+ * before returning to the regular function prologue. When a function is not
+ * being traced, the "move t0, ra" is not harmful.
+ */
+
+int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
+{
+ u32 old, new;
+ unsigned long pc;
+
+ pc = rec->ip;
+ old = larch_insn_gen_nop();
+ new = larch_insn_gen_move(LOONGARCH_GPR_T0, LOONGARCH_GPR_RA);
+
+ return ftrace_modify_code(pc, old, new, true);
+}
+
+int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+ u32 old, new;
+ unsigned long pc;
+ long offset __maybe_unused;
+
+ pc = rec->ip + LOONGARCH_INSN_SIZE;
+
+#ifdef CONFIG_MODULES
+ offset = (long)pc - (long)addr;
+
+ if (offset < -SZ_128M || offset >= SZ_128M) {
+ int ret;
+ struct module *mod;
+
+ ret = __get_mod(&mod, pc);
+ if (ret)
+ return ret;
+
+ addr = get_plt_addr(mod, addr);
+ }
+#endif
+
+ old = larch_insn_gen_nop();
+ new = larch_insn_gen_bl(pc, addr);
+
+ return ftrace_modify_code(pc, old, new, true);
+}
+
+int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
+{
+ u32 old, new;
+ unsigned long pc;
+ long offset __maybe_unused;
+
+ pc = rec->ip + LOONGARCH_INSN_SIZE;
+
+#ifdef CONFIG_MODULES
+ offset = (long)pc - (long)addr;
+
+ if (offset < -SZ_128M || offset >= SZ_128M) {
+ int ret;
+ struct module *mod;
+
+ ret = __get_mod(&mod, pc);
+ if (ret)
+ return ret;
+
+ addr = get_plt_addr(mod, addr);
+ }
+#endif
+
+ new = larch_insn_gen_nop();
+ old = larch_insn_gen_bl(pc, addr);
+
+ return ftrace_modify_code(pc, old, new, true);
+}
+
+void arch_ftrace_update_code(int command)
+{
+ command |= FTRACE_MAY_SLEEP;
+ ftrace_modify_all_code(command);
+}
+
+int __init ftrace_dyn_arch_init(void)
+{
+ return 0;
+}
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent)
+{
+ unsigned long old;
+ unsigned long return_hooker = (unsigned long)&return_to_handler;
+
+ if (unlikely(atomic_read(¤t->tracing_graph_pause)))
+ return;
+
+ old = *parent;
+
+ if (!function_graph_enter(old, self_addr, 0, parent))
+ *parent = return_hooker;
+}
+
+#ifdef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct ftrace_regs *fregs)
+{
+ struct pt_regs *regs = &fregs->regs;
+ unsigned long *parent = (unsigned long *)®s->regs[1];
+
+ prepare_ftrace_return(ip, (unsigned long *)parent);
+}
+#else
+static int ftrace_modify_graph_caller(bool enable)
+{
+ u32 branch, nop;
+ unsigned long pc, func;
+ extern void ftrace_graph_call(void);
+
+ pc = (unsigned long)&ftrace_graph_call;
+ func = (unsigned long)&ftrace_graph_caller;
+
+ nop = larch_insn_gen_nop();
+ branch = larch_insn_gen_b(pc, func);
+
+ if (enable)
+ return ftrace_modify_code(pc, nop, branch, true);
+ else
+ return ftrace_modify_code(pc, branch, nop, true);
+}
+
+int ftrace_enable_ftrace_graph_caller(void)
+{
+ return ftrace_modify_graph_caller(true);
+}
+
+int ftrace_disable_ftrace_graph_caller(void)
+{
+ return ftrace_modify_graph_caller(false);
+}
+#endif /* CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS */
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
/*
* Copyright (C) 2020-2022 Loongson Technology Corporation Limited
*/
+#include <linux/sizes.h>
+#include <linux/uaccess.h>
+
+#include <asm/cacheflush.h>
#include <asm/inst.h>
+static DEFINE_RAW_SPINLOCK(patch_lock);
+
+int larch_insn_read(void *addr, u32 *insnp)
+{
+ int ret;
+ u32 val;
+
+ ret = copy_from_kernel_nofault(&val, addr, LOONGARCH_INSN_SIZE);
+ if (!ret)
+ *insnp = val;
+
+ return ret;
+}
+
+int larch_insn_write(void *addr, u32 insn)
+{
+ int ret;
+ unsigned long flags = 0;
+
+ raw_spin_lock_irqsave(&patch_lock, flags);
+ ret = copy_to_kernel_nofault(addr, &insn, LOONGARCH_INSN_SIZE);
+ raw_spin_unlock_irqrestore(&patch_lock, flags);
+
+ return ret;
+}
+
+int larch_insn_patch_text(void *addr, u32 insn)
+{
+ int ret;
+ u32 *tp = addr;
+
+ if ((unsigned long)tp & 3)
+ return -EINVAL;
+
+ ret = larch_insn_write(tp, insn);
+ if (!ret)
+ flush_icache_range((unsigned long)tp,
+ (unsigned long)tp + LOONGARCH_INSN_SIZE);
+
+ return ret;
+}
+
+u32 larch_insn_gen_nop(void)
+{
+ return INSN_NOP;
+}
+
+u32 larch_insn_gen_b(unsigned long pc, unsigned long dest)
+{
+ long offset = dest - pc;
+ unsigned int immediate_l, immediate_h;
+ union loongarch_instruction insn;
+
+ if ((offset & 3) || offset < -SZ_128M || offset >= SZ_128M) {
+ pr_warn("The generated b instruction is out of range.\n");
+ return INSN_BREAK;
+ }
+
+ offset >>= 2;
+
+ immediate_l = offset & 0xffff;
+ offset >>= 16;
+ immediate_h = offset & 0x3ff;
+
+ insn.reg0i26_format.opcode = b_op;
+ insn.reg0i26_format.immediate_l = immediate_l;
+ insn.reg0i26_format.immediate_h = immediate_h;
+
+ return insn.word;
+}
+
+u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest)
+{
+ long offset = dest - pc;
+ unsigned int immediate_l, immediate_h;
+ union loongarch_instruction insn;
+
+ if ((offset & 3) || offset < -SZ_128M || offset >= SZ_128M) {
+ pr_warn("The generated bl instruction is out of range.\n");
+ return INSN_BREAK;
+ }
+
+ offset >>= 2;
+
+ immediate_l = offset & 0xffff;
+ offset >>= 16;
+ immediate_h = offset & 0x3ff;
+
+ insn.reg0i26_format.opcode = bl_op;
+ insn.reg0i26_format.immediate_l = immediate_l;
+ insn.reg0i26_format.immediate_h = immediate_h;
+
+ return insn.word;
+}
+
+u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj, enum loongarch_gpr rk)
+{
+ union loongarch_instruction insn;
+
+ insn.reg3_format.opcode = or_op;
+ insn.reg3_format.rd = rd;
+ insn.reg3_format.rj = rj;
+ insn.reg3_format.rk = rk;
+
+ return insn.word;
+}
+
+u32 larch_insn_gen_move(enum loongarch_gpr rd, enum loongarch_gpr rj)
+{
+ return larch_insn_gen_or(rd, rj, 0);
+}
+
+u32 larch_insn_gen_lu12iw(enum loongarch_gpr rd, int imm)
+{
+ union loongarch_instruction insn;
+
+ insn.reg1i20_format.opcode = lu12iw_op;
+ insn.reg1i20_format.rd = rd;
+ insn.reg1i20_format.immediate = imm;
+
+ return insn.word;
+}
+
u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm)
{
union loongarch_instruction insn;
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * LoongArch specific _mcount support
+ *
+ * Copyright (C) 2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/export.h>
+#include <asm/ftrace.h>
+#include <asm/regdef.h>
+#include <asm/stackframe.h>
+
+ .text
+
+#define MCOUNT_S0_OFFSET (0)
+#define MCOUNT_RA_OFFSET (SZREG)
+#define MCOUNT_STACK_SIZE (2 * SZREG)
+
+ .macro MCOUNT_SAVE_REGS
+ PTR_ADDI sp, sp, -MCOUNT_STACK_SIZE
+ PTR_S s0, sp, MCOUNT_S0_OFFSET
+ PTR_S ra, sp, MCOUNT_RA_OFFSET
+ move s0, a0
+ .endm
+
+ .macro MCOUNT_RESTORE_REGS
+ move a0, s0
+ PTR_L ra, sp, MCOUNT_RA_OFFSET
+ PTR_L s0, sp, MCOUNT_S0_OFFSET
+ PTR_ADDI sp, sp, MCOUNT_STACK_SIZE
+ .endm
+
+SYM_FUNC_START(_mcount)
+ la.pcrel t1, ftrace_stub
+ la.pcrel t2, ftrace_trace_function /* Prepare t2 for (1) */
+ PTR_L t2, t2, 0
+ beq t1, t2, fgraph_trace
+
+ MCOUNT_SAVE_REGS
+
+ move a0, ra /* arg0: self return address */
+ move a1, s0 /* arg1: parent's return address */
+ jirl ra, t2, 0 /* (1) call *ftrace_trace_function */
+
+ MCOUNT_RESTORE_REGS
+
+fgraph_trace:
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+ la.pcrel t1, ftrace_stub
+ la.pcrel t3, ftrace_graph_return
+ PTR_L t3, t3, 0
+ bne t1, t3, ftrace_graph_caller
+ la.pcrel t1, ftrace_graph_entry_stub
+ la.pcrel t3, ftrace_graph_entry
+ PTR_L t3, t3, 0
+ bne t1, t3, ftrace_graph_caller
+#endif
+
+SYM_INNER_LABEL(ftrace_stub, SYM_L_GLOBAL)
+ jr ra
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+SYM_INNER_LABEL(ftrace_graph_func, SYM_L_GLOBAL)
+ bl ftrace_stub
+#endif
+SYM_FUNC_END(_mcount)
+EXPORT_SYMBOL(_mcount)
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+SYM_FUNC_START(ftrace_graph_caller)
+ MCOUNT_SAVE_REGS
+
+ PTR_ADDI a0, ra, -4 /* arg0: Callsite self return addr */
+ PTR_ADDI a1, sp, MCOUNT_STACK_SIZE /* arg1: Callsite sp */
+ move a2, s0 /* arg2: Callsite parent ra */
+ bl prepare_ftrace_return
+
+ MCOUNT_RESTORE_REGS
+ jr ra
+SYM_FUNC_END(ftrace_graph_caller)
+
+SYM_FUNC_START(return_to_handler)
+ PTR_ADDI sp, sp, -2 * SZREG
+ PTR_S a0, sp, 0
+ PTR_S a1, sp, SZREG
+
+ bl ftrace_return_to_handler
+
+ /* Restore the real parent address: a0 -> ra */
+ move ra, a0
+
+ PTR_L a0, sp, 0
+ PTR_L a1, sp, SZREG
+ PTR_ADDI sp, sp, 2 * SZREG
+ jr ra
+SYM_FUNC_END(return_to_handler)
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/export.h>
+#include <asm/ftrace.h>
+#include <asm/regdef.h>
+#include <asm/stackframe.h>
+
+ .text
+/*
+ * Due to -fpatchable-function-entry=2: the compiler inserted 2 NOPs before the
+ * regular C function prologue. When PC arrived here, the last 2 instructions
+ * are as follows:
+ * move t0, ra
+ * bl callsite (for modules, callsite is a tramplione)
+ *
+ * modules trampoline is as follows:
+ * lu12i.w t1, callsite[31:12]
+ * lu32i.d t1, callsite[51:32]
+ * lu52i.d t1, t1, callsite[63:52]
+ * jirl zero, t1, callsite[11:0] >> 2
+ *
+ * See arch/loongarch/kernel/ftrace_dyn.c for details. Here, pay attention to
+ * that the T series regs are available and safe because each C functions
+ * follows the LoongArch's psABI as well.
+ */
+
+ .macro ftrace_regs_entry allregs=0
+ PTR_ADDI sp, sp, -PT_SIZE
+ PTR_S t0, sp, PT_R1 /* Save parent ra at PT_R1(RA) */
+ PTR_S a0, sp, PT_R4
+ PTR_S a1, sp, PT_R5
+ PTR_S a2, sp, PT_R6
+ PTR_S a3, sp, PT_R7
+ PTR_S a4, sp, PT_R8
+ PTR_S a5, sp, PT_R9
+ PTR_S a6, sp, PT_R10
+ PTR_S a7, sp, PT_R11
+ PTR_S fp, sp, PT_R22
+ .if \allregs
+ PTR_S tp, sp, PT_R2
+ PTR_S t0, sp, PT_R12
+ PTR_S t1, sp, PT_R13
+ PTR_S t2, sp, PT_R14
+ PTR_S t3, sp, PT_R15
+ PTR_S t4, sp, PT_R16
+ PTR_S t5, sp, PT_R17
+ PTR_S t6, sp, PT_R18
+ PTR_S t7, sp, PT_R19
+ PTR_S t8, sp, PT_R20
+ PTR_S u0, sp, PT_R21
+ PTR_S s0, sp, PT_R23
+ PTR_S s1, sp, PT_R24
+ PTR_S s2, sp, PT_R25
+ PTR_S s3, sp, PT_R26
+ PTR_S s4, sp, PT_R27
+ PTR_S s5, sp, PT_R28
+ PTR_S s6, sp, PT_R29
+ PTR_S s7, sp, PT_R30
+ PTR_S s8, sp, PT_R31
+ /* Clear it for later use as a flag sometimes. */
+ PTR_S zero, sp, PT_R0
+ .endif
+ PTR_S ra, sp, PT_ERA /* Save trace function ra at PT_ERA */
+ PTR_ADDI t8, sp, PT_SIZE
+ PTR_S t8, sp, PT_R3
+ .endm
+
+SYM_FUNC_START(ftrace_stub)
+ jr ra
+SYM_FUNC_END(ftrace_stub)
+
+SYM_CODE_START(ftrace_common)
+ PTR_ADDI a0, ra, -8 /* arg0: ip */
+ move a1, t0 /* arg1: parent_ip */
+ la.pcrel t1, function_trace_op
+ PTR_L a2, t1, 0 /* arg2: op */
+ move a3, sp /* arg3: regs */
+
+SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL)
+ bl ftrace_stub
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL)
+ nop /* b ftrace_graph_caller */
+#endif
+
+/*
+ * As we didn't use S series regs in this assmembly code and all calls
+ * are C function which will save S series regs by themselves, there is
+ * no need to restore S series regs. The T series is available and safe
+ * at the callsite, so there is no need to restore the T series regs.
+ */
+ftrace_common_return:
+ PTR_L ra, sp, PT_R1
+ PTR_L a0, sp, PT_R4
+ PTR_L a1, sp, PT_R5
+ PTR_L a2, sp, PT_R6
+ PTR_L a3, sp, PT_R7
+ PTR_L a4, sp, PT_R8
+ PTR_L a5, sp, PT_R9
+ PTR_L a6, sp, PT_R10
+ PTR_L a7, sp, PT_R11
+ PTR_L fp, sp, PT_R22
+ PTR_L t0, sp, PT_ERA
+ PTR_ADDI sp, sp, PT_SIZE
+ jr t0
+SYM_CODE_END(ftrace_common)
+
+SYM_CODE_START(ftrace_caller)
+ ftrace_regs_entry allregs=0
+ b ftrace_common
+SYM_CODE_END(ftrace_caller)
+
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+SYM_CODE_START(ftrace_regs_caller)
+ ftrace_regs_entry allregs=1
+ b ftrace_common
+SYM_CODE_END(ftrace_regs_caller)
+#endif
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+SYM_CODE_START(ftrace_graph_caller)
+ PTR_L a0, sp, PT_ERA
+ PTR_ADDI a0, a0, -8 /* arg0: self_addr */
+ PTR_ADDI a1, sp, PT_R1 /* arg1: parent */
+ bl prepare_ftrace_return
+ b ftrace_common_return
+SYM_CODE_END(ftrace_graph_caller)
+
+SYM_CODE_START(return_to_handler)
+ /* Save return value regs */
+ PTR_ADDI sp, sp, -2 * SZREG
+ PTR_S a0, sp, 0
+ PTR_S a1, sp, SZREG
+
+ move a0, zero
+ bl ftrace_return_to_handler
+ move ra, a0
+
+ /* Restore return value regs */
+ PTR_L a0, sp, 0
+ PTR_L a1, sp, SZREG
+ PTR_ADDI sp, sp, 2 * SZREG
+
+ jr ra
+SYM_CODE_END(return_to_handler)
+#endif
#include <linux/elf.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/ftrace.h>
-Elf_Addr module_emit_got_entry(struct module *mod, Elf_Addr val)
+Elf_Addr module_emit_got_entry(struct module *mod, Elf_Shdr *sechdrs, Elf_Addr val)
{
struct mod_section *got_sec = &mod->arch.got;
int i = got_sec->num_entries;
- struct got_entry *got = get_got_entry(val, got_sec);
+ struct got_entry *got = get_got_entry(val, sechdrs, got_sec);
if (got)
return (Elf_Addr)got;
/* There is no GOT entry for val yet, create a new one. */
- got = (struct got_entry *)got_sec->shdr->sh_addr;
+ got = (struct got_entry *)sechdrs[got_sec->shndx].sh_addr;
got[i] = emit_got_entry(val);
got_sec->num_entries++;
return (Elf_Addr)&got[i];
}
-Elf_Addr module_emit_plt_entry(struct module *mod, Elf_Addr val)
+Elf_Addr module_emit_plt_entry(struct module *mod, Elf_Shdr *sechdrs, Elf_Addr val)
{
int nr;
struct mod_section *plt_sec = &mod->arch.plt;
struct mod_section *plt_idx_sec = &mod->arch.plt_idx;
- struct plt_entry *plt = get_plt_entry(val, plt_sec, plt_idx_sec);
+ struct plt_entry *plt = get_plt_entry(val, sechdrs, plt_sec, plt_idx_sec);
struct plt_idx_entry *plt_idx;
if (plt)
nr = plt_sec->num_entries;
/* There is no duplicate entry, create a new one */
- plt = (struct plt_entry *)plt_sec->shdr->sh_addr;
+ plt = (struct plt_entry *)sechdrs[plt_sec->shndx].sh_addr;
plt[nr] = emit_plt_entry(val);
- plt_idx = (struct plt_idx_entry *)plt_idx_sec->shdr->sh_addr;
+ plt_idx = (struct plt_idx_entry *)sechdrs[plt_idx_sec->shndx].sh_addr;
plt_idx[nr] = emit_plt_idx_entry(val);
plt_sec->num_entries++;
char *secstrings, struct module *mod)
{
unsigned int i, num_plts = 0, num_gots = 0;
+ Elf_Shdr *got_sec, *plt_sec, *plt_idx_sec, *tramp = NULL;
/*
* Find the empty .plt sections.
*/
for (i = 0; i < ehdr->e_shnum; i++) {
if (!strcmp(secstrings + sechdrs[i].sh_name, ".got"))
- mod->arch.got.shdr = sechdrs + i;
+ mod->arch.got.shndx = i;
else if (!strcmp(secstrings + sechdrs[i].sh_name, ".plt"))
- mod->arch.plt.shdr = sechdrs + i;
+ mod->arch.plt.shndx = i;
else if (!strcmp(secstrings + sechdrs[i].sh_name, ".plt.idx"))
- mod->arch.plt_idx.shdr = sechdrs + i;
+ mod->arch.plt_idx.shndx = i;
+ else if (!strcmp(secstrings + sechdrs[i].sh_name, ".ftrace_trampoline"))
+ tramp = sechdrs + i;
}
- if (!mod->arch.got.shdr) {
+ if (!mod->arch.got.shndx) {
pr_err("%s: module GOT section(s) missing\n", mod->name);
return -ENOEXEC;
}
- if (!mod->arch.plt.shdr) {
+ if (!mod->arch.plt.shndx) {
pr_err("%s: module PLT section(s) missing\n", mod->name);
return -ENOEXEC;
}
- if (!mod->arch.plt_idx.shdr) {
+ if (!mod->arch.plt_idx.shndx) {
pr_err("%s: module PLT.IDX section(s) missing\n", mod->name);
return -ENOEXEC;
}
count_max_entries(relas, num_rela, &num_plts, &num_gots);
}
- mod->arch.got.shdr->sh_type = SHT_NOBITS;
- mod->arch.got.shdr->sh_flags = SHF_ALLOC;
- mod->arch.got.shdr->sh_addralign = L1_CACHE_BYTES;
- mod->arch.got.shdr->sh_size = (num_gots + 1) * sizeof(struct got_entry);
+ got_sec = sechdrs + mod->arch.got.shndx;
+ got_sec->sh_type = SHT_NOBITS;
+ got_sec->sh_flags = SHF_ALLOC;
+ got_sec->sh_addralign = L1_CACHE_BYTES;
+ got_sec->sh_size = (num_gots + 1) * sizeof(struct got_entry);
mod->arch.got.num_entries = 0;
mod->arch.got.max_entries = num_gots;
- mod->arch.plt.shdr->sh_type = SHT_NOBITS;
- mod->arch.plt.shdr->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
- mod->arch.plt.shdr->sh_addralign = L1_CACHE_BYTES;
- mod->arch.plt.shdr->sh_size = (num_plts + 1) * sizeof(struct plt_entry);
+ plt_sec = sechdrs + mod->arch.plt.shndx;
+ plt_sec->sh_type = SHT_NOBITS;
+ plt_sec->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
+ plt_sec->sh_addralign = L1_CACHE_BYTES;
+ plt_sec->sh_size = (num_plts + 1) * sizeof(struct plt_entry);
mod->arch.plt.num_entries = 0;
mod->arch.plt.max_entries = num_plts;
- mod->arch.plt_idx.shdr->sh_type = SHT_NOBITS;
- mod->arch.plt_idx.shdr->sh_flags = SHF_ALLOC;
- mod->arch.plt_idx.shdr->sh_addralign = L1_CACHE_BYTES;
- mod->arch.plt_idx.shdr->sh_size = (num_plts + 1) * sizeof(struct plt_idx_entry);
+ plt_idx_sec = sechdrs + mod->arch.plt_idx.shndx;
+ plt_idx_sec->sh_type = SHT_NOBITS;
+ plt_idx_sec->sh_flags = SHF_ALLOC;
+ plt_idx_sec->sh_addralign = L1_CACHE_BYTES;
+ plt_idx_sec->sh_size = (num_plts + 1) * sizeof(struct plt_idx_entry);
mod->arch.plt_idx.num_entries = 0;
mod->arch.plt_idx.max_entries = num_plts;
+ if (tramp) {
+ tramp->sh_type = SHT_NOBITS;
+ tramp->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
+ tramp->sh_addralign = __alignof__(struct plt_entry);
+ tramp->sh_size = NR_FTRACE_PLTS * sizeof(struct plt_entry);
+ }
+
return 0;
}
#include <linux/vmalloc.h>
#include <linux/slab.h>
#include <linux/fs.h>
+#include <linux/ftrace.h>
#include <linux/string.h>
#include <linux/kernel.h>
+#include <asm/alternative.h>
+#include <asm/inst.h>
static int rela_stack_push(s64 stack_value, s64 *rela_stack, size_t *rela_stack_top)
{
return 0;
}
-static int apply_r_larch_sop_push_plt_pcrel(struct module *mod, u32 *location, Elf_Addr v,
+static int apply_r_larch_sop_push_plt_pcrel(struct module *mod,
+ Elf_Shdr *sechdrs, u32 *location, Elf_Addr v,
s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
{
ptrdiff_t offset = (void *)v - (void *)location;
if (offset >= SZ_128M)
- v = module_emit_plt_entry(mod, v);
+ v = module_emit_plt_entry(mod, sechdrs, v);
if (offset < -SZ_128M)
- v = module_emit_plt_entry(mod, v);
+ v = module_emit_plt_entry(mod, sechdrs, v);
return apply_r_larch_sop_push_pcrel(mod, location, v, rela_stack, rela_stack_top, type);
}
}
}
-static int apply_r_larch_b26(struct module *mod, u32 *location, Elf_Addr v,
+static int apply_r_larch_b26(struct module *mod,
+ Elf_Shdr *sechdrs, u32 *location, Elf_Addr v,
s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
{
ptrdiff_t offset = (void *)v - (void *)location;
union loongarch_instruction *insn = (union loongarch_instruction *)location;
if (offset >= SZ_128M)
- v = module_emit_plt_entry(mod, v);
+ v = module_emit_plt_entry(mod, sechdrs, v);
if (offset < -SZ_128M)
- v = module_emit_plt_entry(mod, v);
+ v = module_emit_plt_entry(mod, sechdrs, v);
offset = (void *)v - (void *)location;
return 0;
}
-static int apply_r_larch_got_pc(struct module *mod, u32 *location, Elf_Addr v,
+static int apply_r_larch_got_pc(struct module *mod,
+ Elf_Shdr *sechdrs, u32 *location, Elf_Addr v,
s64 *rela_stack, size_t *rela_stack_top, unsigned int type)
{
- Elf_Addr got = module_emit_got_entry(mod, v);
+ Elf_Addr got = module_emit_got_entry(mod, sechdrs, v);
if (!got)
return -EINVAL;
[R_LARCH_SOP_PUSH_PCREL] = apply_r_larch_sop_push_pcrel,
[R_LARCH_SOP_PUSH_ABSOLUTE] = apply_r_larch_sop_push_absolute,
[R_LARCH_SOP_PUSH_DUP] = apply_r_larch_sop_push_dup,
- [R_LARCH_SOP_PUSH_PLT_PCREL] = apply_r_larch_sop_push_plt_pcrel,
[R_LARCH_SOP_SUB ... R_LARCH_SOP_IF_ELSE] = apply_r_larch_sop,
[R_LARCH_SOP_POP_32_S_10_5 ... R_LARCH_SOP_POP_32_U] = apply_r_larch_sop_imm_field,
[R_LARCH_ADD32 ... R_LARCH_SUB64] = apply_r_larch_add_sub,
- [R_LARCH_B26] = apply_r_larch_b26,
[R_LARCH_PCALA_HI20...R_LARCH_PCALA64_HI12] = apply_r_larch_pcala,
- [R_LARCH_GOT_PC_HI20...R_LARCH_GOT_PC_LO12] = apply_r_larch_got_pc,
};
int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab,
sym->st_value, rel[i].r_addend, (u64)location);
v = sym->st_value + rel[i].r_addend;
- err = handler(mod, location, v, rela_stack, &rela_stack_top, type);
+ switch (type) {
+ case R_LARCH_B26:
+ err = apply_r_larch_b26(mod, sechdrs, location,
+ v, rela_stack, &rela_stack_top, type);
+ break;
+ case R_LARCH_GOT_PC_HI20...R_LARCH_GOT_PC_LO12:
+ err = apply_r_larch_got_pc(mod, sechdrs, location,
+ v, rela_stack, &rela_stack_top, type);
+ break;
+ case R_LARCH_SOP_PUSH_PLT_PCREL:
+ err = apply_r_larch_sop_push_plt_pcrel(mod, sechdrs, location,
+ v, rela_stack, &rela_stack_top, type);
+ break;
+ default:
+ err = handler(mod, location, v, rela_stack, &rela_stack_top, type);
+ }
if (err)
return err;
}
return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END,
GFP_KERNEL, PAGE_KERNEL, 0, NUMA_NO_NODE, __builtin_return_address(0));
}
+
+static void module_init_ftrace_plt(const Elf_Ehdr *hdr,
+ const Elf_Shdr *sechdrs, struct module *mod)
+{
+#ifdef CONFIG_DYNAMIC_FTRACE
+ struct plt_entry *ftrace_plts;
+
+ ftrace_plts = (void *)sechdrs->sh_addr;
+
+ ftrace_plts[FTRACE_PLT_IDX] = emit_plt_entry(FTRACE_ADDR);
+
+ if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+ ftrace_plts[FTRACE_REGS_PLT_IDX] = emit_plt_entry(FTRACE_REGS_ADDR);
+
+ mod->arch.ftrace_trampolines = ftrace_plts;
+#endif
+}
+
+int module_finalize(const Elf_Ehdr *hdr,
+ const Elf_Shdr *sechdrs, struct module *mod)
+{
+ const Elf_Shdr *s, *se;
+ const char *secstrs = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
+
+ for (s = sechdrs, se = sechdrs + hdr->e_shnum; s < se; s++) {
+ if (!strcmp(".altinstructions", secstrs + s->sh_name))
+ apply_alternatives((void *)s->sh_addr, (void *)s->sh_addr + s->sh_size);
+ if (!strcmp(".ftrace_trampoline", secstrs + s->sh_name))
+ module_init_ftrace_plt(hdr, s, mod);
+ }
+
+ return 0;
+}
}
}
+/*
+ * fake_numa_init() - For Non-ACPI systems
+ * Return: 0 on success, -errno on failure.
+ */
+static int __init fake_numa_init(void)
+{
+ phys_addr_t start = memblock_start_of_DRAM();
+ phys_addr_t end = memblock_end_of_DRAM() - 1;
+
+ node_set(0, numa_nodes_parsed);
+ pr_info("Faking a node at [mem %pap-%pap]\n", &start, &end);
+
+ return numa_add_memblk(0, start, end + 1);
+}
+
int __init init_numa_memory(void)
{
int i;
memset(&numa_meminfo, 0, sizeof(numa_meminfo));
/* Parse SRAT and SLIT if provided by firmware. */
- ret = acpi_numa_init();
+ ret = acpi_disabled ? fake_numa_init() : acpi_numa_init();
if (ret < 0)
return ret;
#include <asm/unwind.h>
#include <asm/vdso.h>
+#ifdef CONFIG_STACKPROTECTOR
+#include <linux/stackprotector.h>
+unsigned long __stack_chk_guard __read_mostly;
+EXPORT_SYMBOL(__stack_chk_guard);
+#endif
+
/*
* Idle related variables and functions
*/
#include <acpi/reboot.h>
#include <asm/idle.h>
#include <asm/loongarch.h>
+#include <asm/loongson.h>
void (*pm_power_off)(void);
EXPORT_SYMBOL(pm_power_off);
preempt_disable();
smp_send_stop();
#endif
+#ifdef CONFIG_PM
+ if (!acpi_disabled)
+ enable_pci_wakeup();
+#endif
do_kernel_power_off();
#ifdef CONFIG_EFI
efi.reset_system(EFI_RESET_SHUTDOWN, EFI_SUCCESS, 0, NULL);
#include <linux/sizes.h>
#include <linux/device.h>
#include <linux/dma-map-ops.h>
+#include <linux/libfdt.h>
+#include <linux/of_fdt.h>
+#include <linux/of_address.h>
+#include <linux/suspend.h>
#include <linux/swiotlb.h>
#include <asm/addrspace.h>
+#include <asm/alternative.h>
#include <asm/bootinfo.h>
+#include <asm/bugs.h>
#include <asm/cache.h>
#include <asm/cpu.h>
#include <asm/dma.h>
*
* These are initialized so they are in the .data section
*/
+char init_command_line[COMMAND_LINE_SIZE] __initdata;
static int num_standard_resources;
static struct resource *standard_resources;
return "generic-loongson-machine";
}
+void __init check_bugs(void)
+{
+ alternative_instructions();
+}
+
static const char *dmi_string_parse(const struct dmi_header *dm, u8 s)
{
const u8 *bp = ((u8 *) dm) + dm->length;
#endif
}
+static void __init fdt_setup(void)
+{
+#ifdef CONFIG_OF_EARLY_FLATTREE
+ void *fdt_pointer;
+
+ /* ACPI-based systems do not require parsing fdt */
+ if (acpi_os_get_root_pointer())
+ return;
+
+ /* Look for a device tree configuration table entry */
+ fdt_pointer = efi_fdt_pointer();
+ if (!fdt_pointer || fdt_check_header(fdt_pointer))
+ return;
+
+ early_init_dt_scan(fdt_pointer);
+ early_init_fdt_reserve_self();
+
+ max_low_pfn = PFN_PHYS(memblock_end_of_DRAM());
+#endif
+}
+
+static void __init bootcmdline_init(char **cmdline_p)
+{
+ /*
+ * If CONFIG_CMDLINE_FORCE is enabled then initializing the command line
+ * is trivial - we simply use the built-in command line unconditionally &
+ * unmodified.
+ */
+ if (IS_ENABLED(CONFIG_CMDLINE_FORCE)) {
+ strscpy(boot_command_line, CONFIG_CMDLINE, COMMAND_LINE_SIZE);
+ goto out;
+ }
+
+#ifdef CONFIG_OF_FLATTREE
+ /*
+ * If CONFIG_CMDLINE_BOOTLOADER is enabled and we are in FDT-based system,
+ * the boot_command_line will be overwritten by early_init_dt_scan_chosen().
+ * So we need to append init_command_line (the original copy of boot_command_line)
+ * to boot_command_line.
+ */
+ if (initial_boot_params) {
+ if (boot_command_line[0])
+ strlcat(boot_command_line, " ", COMMAND_LINE_SIZE);
+
+ strlcat(boot_command_line, init_command_line, COMMAND_LINE_SIZE);
+ }
+#endif
+
+out:
+ *cmdline_p = boot_command_line;
+}
+
void __init platform_init(void)
{
arch_reserve_vmcore();
acpi_gbl_use_default_register_widths = false;
acpi_boot_table_init();
#endif
+ unflatten_and_copy_device_tree();
#ifdef CONFIG_NUMA
init_numa_memory();
check_kernel_sections_mem();
+ early_init_fdt_scan_reserved_mem();
+
/*
* In order to reduce the possibility of kernel panic when failed to
* get IO TLB memory under CONFIG_SWIOTLB, it is better to allocate
dma_contiguous_reserve(PFN_PHYS(max_low_pfn));
+ /* Reserve for hibernation. */
+ register_nosave_region(PFN_DOWN(__pa_symbol(&__nosave_begin)),
+ PFN_UP(__pa_symbol(&__nosave_end)));
+
memblock_dump_all();
early_memtest(PFN_PHYS(ARCH_PFN_OFFSET), PFN_PHYS(max_low_pfn));
#endif
}
+static int __init add_legacy_isa_io(struct fwnode_handle *fwnode,
+ resource_size_t hw_start, resource_size_t size)
+{
+ int ret = 0;
+ unsigned long vaddr;
+ struct logic_pio_hwaddr *range;
+
+ range = kzalloc(sizeof(*range), GFP_ATOMIC);
+ if (!range)
+ return -ENOMEM;
+
+ range->fwnode = fwnode;
+ range->size = size = round_up(size, PAGE_SIZE);
+ range->hw_start = hw_start;
+ range->flags = LOGIC_PIO_CPU_MMIO;
+
+ ret = logic_pio_register_range(range);
+ if (ret) {
+ kfree(range);
+ return ret;
+ }
+
+ /* Legacy ISA must placed at the start of PCI_IOBASE */
+ if (range->io_start != 0) {
+ logic_pio_unregister_range(range);
+ kfree(range);
+ return -EINVAL;
+ }
+
+ vaddr = (unsigned long)(PCI_IOBASE + range->io_start);
+ ioremap_page_range(vaddr, vaddr + size, hw_start, pgprot_device(PAGE_KERNEL));
+
+ return 0;
+}
+
+static __init int arch_reserve_pio_range(void)
+{
+ struct device_node *np;
+
+ for_each_node_by_name(np, "isa") {
+ struct of_range range;
+ struct of_range_parser parser;
+
+ pr_info("ISA Bridge: %pOF\n", np);
+
+ if (of_range_parser_init(&parser, np)) {
+ pr_info("Failed to parse resources.\n");
+ of_node_put(np);
+ break;
+ }
+
+ for_each_of_range(&parser, &range) {
+ switch (range.flags & IORESOURCE_TYPE_BITS) {
+ case IORESOURCE_IO:
+ pr_info(" IO 0x%016llx..0x%016llx -> 0x%016llx\n",
+ range.cpu_addr,
+ range.cpu_addr + range.size - 1,
+ range.bus_addr);
+ if (add_legacy_isa_io(&np->fwnode, range.cpu_addr, range.size))
+ pr_warn("Failed to reserve legacy IO in Logic PIO\n");
+ break;
+ case IORESOURCE_MEM:
+ pr_info(" MEM 0x%016llx..0x%016llx -> 0x%016llx\n",
+ range.cpu_addr,
+ range.cpu_addr + range.size - 1,
+ range.bus_addr);
+ break;
+ }
+ }
+ }
+
+ return 0;
+}
+arch_initcall(arch_reserve_pio_range);
+
static int __init reserve_memblock_reserved_regions(void)
{
u64 i, j;
void __init setup_arch(char **cmdline_p)
{
cpu_probe();
- *cmdline_p = boot_command_line;
init_environ();
efi_init();
+ fdt_setup();
memblock_init();
pagetable_init();
+ bootcmdline_init(cmdline_p);
parse_early_param();
reserve_initrd_mem();
#include <linux/smp.h>
#include <linux/threads.h>
#include <linux/export.h>
+#include <linux/syscore_ops.h>
#include <linux/time.h>
#include <linux/tracepoint.h>
#include <linux/sched/hotplug.h>
return IRQ_HANDLED;
}
+static void __init fdt_smp_setup(void)
+{
+#ifdef CONFIG_OF
+ unsigned int cpu, cpuid;
+ struct device_node *node = NULL;
+
+ for_each_of_cpu_node(node) {
+ if (!of_device_is_available(node))
+ continue;
+
+ cpuid = of_get_cpu_hwid(node, 0);
+ if (cpuid >= nr_cpu_ids)
+ continue;
+
+ if (cpuid == loongson_sysconf.boot_cpu_id) {
+ cpu = 0;
+ numa_add_cpu(cpu);
+ } else {
+ cpu = cpumask_next_zero(-1, cpu_present_mask);
+ }
+
+ num_processors++;
+ set_cpu_possible(cpu, true);
+ set_cpu_present(cpu, true);
+ __cpu_number_map[cpuid] = cpu;
+ __cpu_logical_map[cpu] = cpuid;
+ }
+
+ loongson_sysconf.nr_cpus = num_processors;
+#endif
+}
+
void __init loongson_smp_setup(void)
{
+ fdt_smp_setup();
+
cpu_data[0].core = cpu_logical_map(0) % loongson_sysconf.cores_per_package;
cpu_data[0].package = cpu_logical_map(0) / loongson_sysconf.cores_per_package;
stptr.d ra, a0, THREAD_REG01
stptr.d a3, a0, THREAD_SCHED_RA
stptr.d a4, a0, THREAD_SCHED_CFA
+#if defined(CONFIG_STACKPROTECTOR) && !defined(CONFIG_SMP)
+ la t7, __stack_chk_guard
+ LONG_L t8, a1, TASK_STACK_CANARY
+ LONG_S t8, t7, 0
+#endif
move tp, a2
cpu_restore_nonscratch a1
return lpj;
}
-static long init_timeval;
+static long init_offset __nosavedata;
+
+void save_counter(void)
+{
+ init_offset = drdtime();
+}
void sync_counter(void)
{
/* Ensure counter begin at 0 */
- csr_write64(-init_timeval, LOONGARCH_CSR_CNTC);
+ csr_write64(init_offset, LOONGARCH_CSR_CNTC);
}
static int get_timer_irq(void)
else
const_clock_freq = calc_const_freq();
- init_timeval = drdtime() - csr_read64(LOONGARCH_CSR_CNTC);
+ init_offset = -(drdtime() - csr_read64(LOONGARCH_CSR_CNTC));
constant_clockevent_init();
constant_clocksource_init();
irqentry_exit(regs, state);
}
+/* sysctl hooks */
+int unaligned_enabled __read_mostly = 1; /* Enabled by default */
+int no_unaligned_warning __read_mostly = 1; /* Only 1 warning by default */
+
asmlinkage void noinstr do_ale(struct pt_regs *regs)
{
+ unsigned int *pc;
irqentry_state_t state = irqentry_enter(regs);
+ perf_sw_event(PERF_COUNT_SW_ALIGNMENT_FAULTS, 1, regs, regs->csr_badvaddr);
+
+ /*
+ * Did we catch a fault trying to load an instruction?
+ */
+ if (regs->csr_badvaddr == regs->csr_era)
+ goto sigbus;
+ if (user_mode(regs) && !test_thread_flag(TIF_FIXADE))
+ goto sigbus;
+ if (!unaligned_enabled)
+ goto sigbus;
+ if (!no_unaligned_warning)
+ show_registers(regs);
+
+ pc = (unsigned int *)exception_era(regs);
+
+ emulate_load_store_insn(regs, (void __user *)regs->csr_badvaddr, pc);
+
+ goto out;
+
+sigbus:
die_if_kernel("Kernel ale access", regs);
force_sig_fault(SIGBUS, BUS_ADRALN, (void __user *)regs->csr_badvaddr);
+out:
irqentry_exit(regs, state);
}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Handle unaligned accesses by emulation.
+ *
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ *
+ * Derived from MIPS:
+ * Copyright (C) 1996, 1998, 1999, 2002 by Ralf Baechle
+ * Copyright (C) 1999 Silicon Graphics, Inc.
+ * Copyright (C) 2014 Imagination Technologies Ltd.
+ */
+#include <linux/mm.h>
+#include <linux/sched.h>
+#include <linux/signal.h>
+#include <linux/debugfs.h>
+#include <linux/perf_event.h>
+
+#include <asm/asm.h>
+#include <asm/branch.h>
+#include <asm/fpu.h>
+#include <asm/inst.h>
+
+#include "access-helper.h"
+
+#ifdef CONFIG_DEBUG_FS
+static u32 unaligned_instructions_user;
+static u32 unaligned_instructions_kernel;
+#endif
+
+static inline unsigned long read_fpr(unsigned int idx)
+{
+#define READ_FPR(idx, __value) \
+ __asm__ __volatile__("movfr2gr.d %0, $f"#idx"\n\t" : "=r"(__value));
+
+ unsigned long __value;
+
+ switch (idx) {
+ case 0:
+ READ_FPR(0, __value);
+ break;
+ case 1:
+ READ_FPR(1, __value);
+ break;
+ case 2:
+ READ_FPR(2, __value);
+ break;
+ case 3:
+ READ_FPR(3, __value);
+ break;
+ case 4:
+ READ_FPR(4, __value);
+ break;
+ case 5:
+ READ_FPR(5, __value);
+ break;
+ case 6:
+ READ_FPR(6, __value);
+ break;
+ case 7:
+ READ_FPR(7, __value);
+ break;
+ case 8:
+ READ_FPR(8, __value);
+ break;
+ case 9:
+ READ_FPR(9, __value);
+ break;
+ case 10:
+ READ_FPR(10, __value);
+ break;
+ case 11:
+ READ_FPR(11, __value);
+ break;
+ case 12:
+ READ_FPR(12, __value);
+ break;
+ case 13:
+ READ_FPR(13, __value);
+ break;
+ case 14:
+ READ_FPR(14, __value);
+ break;
+ case 15:
+ READ_FPR(15, __value);
+ break;
+ case 16:
+ READ_FPR(16, __value);
+ break;
+ case 17:
+ READ_FPR(17, __value);
+ break;
+ case 18:
+ READ_FPR(18, __value);
+ break;
+ case 19:
+ READ_FPR(19, __value);
+ break;
+ case 20:
+ READ_FPR(20, __value);
+ break;
+ case 21:
+ READ_FPR(21, __value);
+ break;
+ case 22:
+ READ_FPR(22, __value);
+ break;
+ case 23:
+ READ_FPR(23, __value);
+ break;
+ case 24:
+ READ_FPR(24, __value);
+ break;
+ case 25:
+ READ_FPR(25, __value);
+ break;
+ case 26:
+ READ_FPR(26, __value);
+ break;
+ case 27:
+ READ_FPR(27, __value);
+ break;
+ case 28:
+ READ_FPR(28, __value);
+ break;
+ case 29:
+ READ_FPR(29, __value);
+ break;
+ case 30:
+ READ_FPR(30, __value);
+ break;
+ case 31:
+ READ_FPR(31, __value);
+ break;
+ default:
+ panic("unexpected idx '%d'", idx);
+ }
+#undef READ_FPR
+ return __value;
+}
+
+static inline void write_fpr(unsigned int idx, unsigned long value)
+{
+#define WRITE_FPR(idx, value) \
+ __asm__ __volatile__("movgr2fr.d $f"#idx", %0\n\t" :: "r"(value));
+
+ switch (idx) {
+ case 0:
+ WRITE_FPR(0, value);
+ break;
+ case 1:
+ WRITE_FPR(1, value);
+ break;
+ case 2:
+ WRITE_FPR(2, value);
+ break;
+ case 3:
+ WRITE_FPR(3, value);
+ break;
+ case 4:
+ WRITE_FPR(4, value);
+ break;
+ case 5:
+ WRITE_FPR(5, value);
+ break;
+ case 6:
+ WRITE_FPR(6, value);
+ break;
+ case 7:
+ WRITE_FPR(7, value);
+ break;
+ case 8:
+ WRITE_FPR(8, value);
+ break;
+ case 9:
+ WRITE_FPR(9, value);
+ break;
+ case 10:
+ WRITE_FPR(10, value);
+ break;
+ case 11:
+ WRITE_FPR(11, value);
+ break;
+ case 12:
+ WRITE_FPR(12, value);
+ break;
+ case 13:
+ WRITE_FPR(13, value);
+ break;
+ case 14:
+ WRITE_FPR(14, value);
+ break;
+ case 15:
+ WRITE_FPR(15, value);
+ break;
+ case 16:
+ WRITE_FPR(16, value);
+ break;
+ case 17:
+ WRITE_FPR(17, value);
+ break;
+ case 18:
+ WRITE_FPR(18, value);
+ break;
+ case 19:
+ WRITE_FPR(19, value);
+ break;
+ case 20:
+ WRITE_FPR(20, value);
+ break;
+ case 21:
+ WRITE_FPR(21, value);
+ break;
+ case 22:
+ WRITE_FPR(22, value);
+ break;
+ case 23:
+ WRITE_FPR(23, value);
+ break;
+ case 24:
+ WRITE_FPR(24, value);
+ break;
+ case 25:
+ WRITE_FPR(25, value);
+ break;
+ case 26:
+ WRITE_FPR(26, value);
+ break;
+ case 27:
+ WRITE_FPR(27, value);
+ break;
+ case 28:
+ WRITE_FPR(28, value);
+ break;
+ case 29:
+ WRITE_FPR(29, value);
+ break;
+ case 30:
+ WRITE_FPR(30, value);
+ break;
+ case 31:
+ WRITE_FPR(31, value);
+ break;
+ default:
+ panic("unexpected idx '%d'", idx);
+ }
+#undef WRITE_FPR
+}
+
+void emulate_load_store_insn(struct pt_regs *regs, void __user *addr, unsigned int *pc)
+{
+ bool fp = false;
+ bool sign, write;
+ bool user = user_mode(regs);
+ unsigned int res, size = 0;
+ unsigned long value = 0;
+ union loongarch_instruction insn;
+
+ perf_sw_event(PERF_COUNT_SW_EMULATION_FAULTS, 1, regs, 0);
+
+ __get_inst(&insn.word, pc, user);
+
+ switch (insn.reg2i12_format.opcode) {
+ case ldh_op:
+ size = 2;
+ sign = true;
+ write = false;
+ break;
+ case ldhu_op:
+ size = 2;
+ sign = false;
+ write = false;
+ break;
+ case sth_op:
+ size = 2;
+ sign = true;
+ write = true;
+ break;
+ case ldw_op:
+ size = 4;
+ sign = true;
+ write = false;
+ break;
+ case ldwu_op:
+ size = 4;
+ sign = false;
+ write = false;
+ break;
+ case stw_op:
+ size = 4;
+ sign = true;
+ write = true;
+ break;
+ case ldd_op:
+ size = 8;
+ sign = true;
+ write = false;
+ break;
+ case std_op:
+ size = 8;
+ sign = true;
+ write = true;
+ break;
+ case flds_op:
+ size = 4;
+ fp = true;
+ sign = true;
+ write = false;
+ break;
+ case fsts_op:
+ size = 4;
+ fp = true;
+ sign = true;
+ write = true;
+ break;
+ case fldd_op:
+ size = 8;
+ fp = true;
+ sign = true;
+ write = false;
+ break;
+ case fstd_op:
+ size = 8;
+ fp = true;
+ sign = true;
+ write = true;
+ break;
+ }
+
+ switch (insn.reg2i14_format.opcode) {
+ case ldptrw_op:
+ size = 4;
+ sign = true;
+ write = false;
+ break;
+ case stptrw_op:
+ size = 4;
+ sign = true;
+ write = true;
+ break;
+ case ldptrd_op:
+ size = 8;
+ sign = true;
+ write = false;
+ break;
+ case stptrd_op:
+ size = 8;
+ sign = true;
+ write = true;
+ break;
+ }
+
+ switch (insn.reg3_format.opcode) {
+ case ldxh_op:
+ size = 2;
+ sign = true;
+ write = false;
+ break;
+ case ldxhu_op:
+ size = 2;
+ sign = false;
+ write = false;
+ break;
+ case stxh_op:
+ size = 2;
+ sign = true;
+ write = true;
+ break;
+ case ldxw_op:
+ size = 4;
+ sign = true;
+ write = false;
+ break;
+ case ldxwu_op:
+ size = 4;
+ sign = false;
+ write = false;
+ break;
+ case stxw_op:
+ size = 4;
+ sign = true;
+ write = true;
+ break;
+ case ldxd_op:
+ size = 8;
+ sign = true;
+ write = false;
+ break;
+ case stxd_op:
+ size = 8;
+ sign = true;
+ write = true;
+ break;
+ case fldxs_op:
+ size = 4;
+ fp = true;
+ sign = true;
+ write = false;
+ break;
+ case fstxs_op:
+ size = 4;
+ fp = true;
+ sign = true;
+ write = true;
+ break;
+ case fldxd_op:
+ size = 8;
+ fp = true;
+ sign = true;
+ write = false;
+ break;
+ case fstxd_op:
+ size = 8;
+ fp = true;
+ sign = true;
+ write = true;
+ break;
+ }
+
+ if (!size)
+ goto sigbus;
+ if (user && !access_ok(addr, size))
+ goto sigbus;
+
+ if (!write) {
+ res = unaligned_read(addr, &value, size, sign);
+ if (res)
+ goto fault;
+
+ /* Rd is the same field in any formats */
+ if (!fp)
+ regs->regs[insn.reg3_format.rd] = value;
+ else {
+ if (is_fpu_owner())
+ write_fpr(insn.reg3_format.rd, value);
+ else
+ set_fpr64(¤t->thread.fpu.fpr[insn.reg3_format.rd], 0, value);
+ }
+ } else {
+ /* Rd is the same field in any formats */
+ if (!fp)
+ value = regs->regs[insn.reg3_format.rd];
+ else {
+ if (is_fpu_owner())
+ value = read_fpr(insn.reg3_format.rd);
+ else
+ value = get_fpr64(¤t->thread.fpu.fpr[insn.reg3_format.rd], 0);
+ }
+
+ res = unaligned_write(addr, value, size);
+ if (res)
+ goto fault;
+ }
+
+#ifdef CONFIG_DEBUG_FS
+ if (user)
+ unaligned_instructions_user++;
+ else
+ unaligned_instructions_kernel++;
+#endif
+
+ compute_return_era(regs);
+
+ return;
+
+fault:
+ /* Did we have an exception handler installed? */
+ if (fixup_exception(regs))
+ return;
+
+ die_if_kernel("Unhandled kernel unaligned access", regs);
+ force_sig(SIGSEGV);
+
+ return;
+
+sigbus:
+ die_if_kernel("Unhandled kernel unaligned access", regs);
+ force_sig(SIGBUS);
+
+ return;
+}
+
+#ifdef CONFIG_DEBUG_FS
+static int __init debugfs_unaligned(void)
+{
+ struct dentry *d;
+
+ d = debugfs_create_dir("loongarch", NULL);
+ if (!d)
+ return -ENOMEM;
+
+ debugfs_create_u32("unaligned_instructions_user",
+ S_IRUGO, d, &unaligned_instructions_user);
+ debugfs_create_u32("unaligned_instructions_kernel",
+ S_IRUGO, d, &unaligned_instructions_kernel);
+
+ return 0;
+}
+arch_initcall(debugfs_unaligned);
+#endif
* Copyright (C) 2022 Loongson Technology Corporation Limited
*/
#include <linux/kernel.h>
+#include <linux/ftrace.h>
#include <asm/unwind.h>
state->sp < info->end;
state->sp += sizeof(unsigned long)) {
addr = *(unsigned long *)(state->sp);
-
+ state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx,
+ addr, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET));
if (__kernel_text_address(addr))
return true;
}
/*
* Copyright (C) 2022 Loongson Technology Corporation Limited
*/
+#include <linux/ftrace.h>
#include <linux/kallsyms.h>
#include <asm/inst.h>
#include <asm/ptrace.h>
#include <asm/unwind.h>
+static inline void unwind_state_fixup(struct unwind_state *state)
+{
+#ifdef CONFIG_DYNAMIC_FTRACE
+ static unsigned long ftrace = (unsigned long)ftrace_call + 4;
+
+ if (state->pc == ftrace)
+ state->is_ftrace = true;
+#endif
+}
+
unsigned long unwind_get_return_address(struct unwind_state *state)
{
state->sp < info->end;
state->sp += sizeof(unsigned long)) {
addr = *(unsigned long *)(state->sp);
+ state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx,
+ addr, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET));
if (__kernel_text_address(addr))
return true;
}
static bool unwind_by_prologue(struct unwind_state *state)
{
- struct stack_info *info = &state->stack_info;
- union loongarch_instruction *ip, *ip_end;
long frame_ra = -1;
unsigned long frame_size = 0;
unsigned long size, offset, pc = state->pc;
+ struct pt_regs *regs;
+ struct stack_info *info = &state->stack_info;
+ union loongarch_instruction *ip, *ip_end;
if (state->sp >= info->end || state->sp < info->begin)
return false;
+ if (state->is_ftrace) {
+ /*
+ * As we meet ftrace_regs_entry, reset first flag like first doing
+ * tracing. Prologue analysis will stop soon because PC is at entry.
+ */
+ regs = (struct pt_regs *)state->sp;
+ state->first = true;
+ state->is_ftrace = false;
+ state->pc = regs->csr_era;
+ state->ra = regs->regs[1];
+ state->sp = regs->regs[3];
+ return true;
+ }
+
if (!kallsyms_lookup_size_offset(pc, &size, &offset))
return false;
state->pc = *(unsigned long *)(state->sp + frame_ra);
state->sp = state->sp + frame_size;
- return !!__kernel_text_address(state->pc);
+ goto out;
first:
state->first = false;
state->pc = state->ra;
- return !!__kernel_text_address(state->ra);
+out:
+ unwind_state_fixup(state);
+ return !!__kernel_text_address(state->pc);
}
void unwind_start(struct unwind_state *state, struct task_struct *task,
break;
case UNWINDER_PROLOGUE:
- if (unwind_by_prologue(state))
+ if (unwind_by_prologue(state)) {
+ state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx,
+ state->pc, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET));
return true;
+ }
if (info->type == STACK_TYPE_IRQ &&
info->end == state->sp) {
if (user_mode(regs) || !__kernel_text_address(pc))
return false;
- state->pc = pc;
- state->sp = regs->regs[3];
- state->ra = regs->regs[1];
state->first = true;
+ state->ra = regs->regs[1];
+ state->sp = regs->regs[3];
+ state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx,
+ pc, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET));
get_stack_info(state->sp, state->task, info);
return true;
#include <asm/thread_info.h>
#define PAGE_SIZE _PAGE_SIZE
+#define RO_EXCEPTION_TABLE_ALIGN 4
/*
* Put .bss..swapper_pg_dir as the first thing in .bss. This will
. = ALIGN(PECOFF_SEGMENT_ALIGN);
_etext = .;
- EXCEPTION_TABLE(16)
+ /*
+ * struct alt_inst entries. From the header (alternative.h):
+ * "Alternative instructions for different CPU types or capabilities"
+ * Think locking instructions on spinlocks.
+ */
+ . = ALIGN(4);
+ .altinstructions : AT(ADDR(.altinstructions) - LOAD_OFFSET) {
+ __alt_instructions = .;
+ *(.altinstructions)
+ __alt_instructions_end = .;
+ }
.got : ALIGN(16) { *(.got) }
.plt : ALIGN(16) { *(.plt) }
# Makefile for LoongArch-specific library files.
#
-lib-y += delay.o clear_user.o copy_user.o dump_tlb.o
+lib-y += delay.o memset.o memcpy.o memmove.o \
+ clear_user.o copy_user.o dump_tlb.o unaligned.o
* Copyright (C) 2020-2022 Loongson Technology Corporation Limited
*/
+#include <asm/alternative-asm.h>
#include <asm/asm.h>
#include <asm/asmmacro.h>
+#include <asm/asm-extable.h>
+#include <asm/cpu.h>
#include <asm/export.h>
#include <asm/regdef.h>
-.macro fixup_ex from, to, offset, fix
-.if \fix
- .section .fixup, "ax"
-\to: addi.d a0, a1, \offset
+.irp to, 0, 1, 2, 3, 4, 5, 6, 7
+.L_fixup_handle_\to\():
+ addi.d a0, a1, (\to) * (-8)
jr ra
- .previous
-.endif
- .section __ex_table, "a"
- PTR \from\()b, \to\()b
- .previous
-.endm
+.endr
+
+SYM_FUNC_START(__clear_user)
+ /*
+ * Some CPUs support hardware unaligned access
+ */
+ ALTERNATIVE "b __clear_user_generic", \
+ "b __clear_user_fast", CPU_FEATURE_UAL
+SYM_FUNC_END(__clear_user)
+
+EXPORT_SYMBOL(__clear_user)
/*
- * unsigned long __clear_user(void *addr, size_t size)
+ * unsigned long __clear_user_generic(void *addr, size_t size)
*
* a0: addr
* a1: size
*/
-SYM_FUNC_START(__clear_user)
+SYM_FUNC_START(__clear_user_generic)
beqz a1, 2f
1: st.b zero, a0, 0
2: move a0, a1
jr ra
- fixup_ex 1, 3, 0, 1
-SYM_FUNC_END(__clear_user)
+ _asm_extable 1b, .L_fixup_handle_0
+SYM_FUNC_END(__clear_user_generic)
-EXPORT_SYMBOL(__clear_user)
+/*
+ * unsigned long __clear_user_fast(void *addr, unsigned long size)
+ *
+ * a0: addr
+ * a1: size
+ */
+SYM_FUNC_START(__clear_user_fast)
+ beqz a1, 10f
+
+ ori a2, zero, 64
+ blt a1, a2, 9f
+
+ /* set 64 bytes at a time */
+1: st.d zero, a0, 0
+2: st.d zero, a0, 8
+3: st.d zero, a0, 16
+4: st.d zero, a0, 24
+5: st.d zero, a0, 32
+6: st.d zero, a0, 40
+7: st.d zero, a0, 48
+8: st.d zero, a0, 56
+
+ addi.d a0, a0, 64
+ addi.d a1, a1, -64
+ bge a1, a2, 1b
+
+ beqz a1, 10f
+
+ /* set the remaining bytes */
+9: st.b zero, a0, 0
+ addi.d a0, a0, 1
+ addi.d a1, a1, -1
+ bgt a1, zero, 9b
+
+ /* return */
+10: move a0, a1
+ jr ra
+
+ /* fixup and ex_table */
+ _asm_extable 1b, .L_fixup_handle_0
+ _asm_extable 2b, .L_fixup_handle_1
+ _asm_extable 3b, .L_fixup_handle_2
+ _asm_extable 4b, .L_fixup_handle_3
+ _asm_extable 5b, .L_fixup_handle_4
+ _asm_extable 6b, .L_fixup_handle_5
+ _asm_extable 7b, .L_fixup_handle_6
+ _asm_extable 8b, .L_fixup_handle_7
+ _asm_extable 9b, .L_fixup_handle_0
+SYM_FUNC_END(__clear_user_fast)
* Copyright (C) 2020-2022 Loongson Technology Corporation Limited
*/
+#include <asm/alternative-asm.h>
#include <asm/asm.h>
#include <asm/asmmacro.h>
+#include <asm/asm-extable.h>
+#include <asm/cpu.h>
#include <asm/export.h>
#include <asm/regdef.h>
-.macro fixup_ex from, to, offset, fix
-.if \fix
- .section .fixup, "ax"
-\to: addi.d a0, a2, \offset
+.irp to, 0, 1, 2, 3, 4, 5, 6, 7
+.L_fixup_handle_\to\():
+ addi.d a0, a2, (\to) * (-8)
jr ra
- .previous
-.endif
- .section __ex_table, "a"
- PTR \from\()b, \to\()b
- .previous
-.endm
+.endr
+
+SYM_FUNC_START(__copy_user)
+ /*
+ * Some CPUs support hardware unaligned access
+ */
+ ALTERNATIVE "b __copy_user_generic", \
+ "b __copy_user_fast", CPU_FEATURE_UAL
+SYM_FUNC_END(__copy_user)
+
+EXPORT_SYMBOL(__copy_user)
/*
- * unsigned long __copy_user(void *to, const void *from, size_t n)
+ * unsigned long __copy_user_generic(void *to, const void *from, size_t n)
*
* a0: to
* a1: from
* a2: n
*/
-SYM_FUNC_START(__copy_user)
+SYM_FUNC_START(__copy_user_generic)
beqz a2, 3f
1: ld.b t0, a1, 0
3: move a0, a2
jr ra
- fixup_ex 1, 4, 0, 1
- fixup_ex 2, 4, 0, 0
-SYM_FUNC_END(__copy_user)
+ _asm_extable 1b, .L_fixup_handle_0
+ _asm_extable 2b, .L_fixup_handle_0
+SYM_FUNC_END(__copy_user_generic)
-EXPORT_SYMBOL(__copy_user)
+/*
+ * unsigned long __copy_user_fast(void *to, const void *from, unsigned long n)
+ *
+ * a0: to
+ * a1: from
+ * a2: n
+ */
+SYM_FUNC_START(__copy_user_fast)
+ beqz a2, 19f
+
+ ori a3, zero, 64
+ blt a2, a3, 17f
+
+ /* copy 64 bytes at a time */
+1: ld.d t0, a1, 0
+2: ld.d t1, a1, 8
+3: ld.d t2, a1, 16
+4: ld.d t3, a1, 24
+5: ld.d t4, a1, 32
+6: ld.d t5, a1, 40
+7: ld.d t6, a1, 48
+8: ld.d t7, a1, 56
+9: st.d t0, a0, 0
+10: st.d t1, a0, 8
+11: st.d t2, a0, 16
+12: st.d t3, a0, 24
+13: st.d t4, a0, 32
+14: st.d t5, a0, 40
+15: st.d t6, a0, 48
+16: st.d t7, a0, 56
+
+ addi.d a0, a0, 64
+ addi.d a1, a1, 64
+ addi.d a2, a2, -64
+ bge a2, a3, 1b
+
+ beqz a2, 19f
+
+ /* copy the remaining bytes */
+17: ld.b t0, a1, 0
+18: st.b t0, a0, 0
+ addi.d a0, a0, 1
+ addi.d a1, a1, 1
+ addi.d a2, a2, -1
+ bgt a2, zero, 17b
+
+ /* return */
+19: move a0, a2
+ jr ra
+
+ /* fixup and ex_table */
+ _asm_extable 1b, .L_fixup_handle_0
+ _asm_extable 2b, .L_fixup_handle_1
+ _asm_extable 3b, .L_fixup_handle_2
+ _asm_extable 4b, .L_fixup_handle_3
+ _asm_extable 5b, .L_fixup_handle_4
+ _asm_extable 6b, .L_fixup_handle_5
+ _asm_extable 7b, .L_fixup_handle_6
+ _asm_extable 8b, .L_fixup_handle_7
+ _asm_extable 9b, .L_fixup_handle_0
+ _asm_extable 10b, .L_fixup_handle_1
+ _asm_extable 11b, .L_fixup_handle_2
+ _asm_extable 12b, .L_fixup_handle_3
+ _asm_extable 13b, .L_fixup_handle_4
+ _asm_extable 14b, .L_fixup_handle_5
+ _asm_extable 15b, .L_fixup_handle_6
+ _asm_extable 16b, .L_fixup_handle_7
+ _asm_extable 17b, .L_fixup_handle_0
+ _asm_extable 18b, .L_fixup_handle_0
+SYM_FUNC_END(__copy_user_fast)
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/alternative-asm.h>
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/cpu.h>
+#include <asm/export.h>
+#include <asm/regdef.h>
+
+SYM_FUNC_START(memcpy)
+ /*
+ * Some CPUs support hardware unaligned access
+ */
+ ALTERNATIVE "b __memcpy_generic", \
+ "b __memcpy_fast", CPU_FEATURE_UAL
+SYM_FUNC_END(memcpy)
+
+EXPORT_SYMBOL(memcpy)
+
+/*
+ * void *__memcpy_generic(void *dst, const void *src, size_t n)
+ *
+ * a0: dst
+ * a1: src
+ * a2: n
+ */
+SYM_FUNC_START(__memcpy_generic)
+ move a3, a0
+ beqz a2, 2f
+
+1: ld.b t0, a1, 0
+ st.b t0, a0, 0
+ addi.d a0, a0, 1
+ addi.d a1, a1, 1
+ addi.d a2, a2, -1
+ bgt a2, zero, 1b
+
+2: move a0, a3
+ jr ra
+SYM_FUNC_END(__memcpy_generic)
+
+/*
+ * void *__memcpy_fast(void *dst, const void *src, size_t n)
+ *
+ * a0: dst
+ * a1: src
+ * a2: n
+ */
+SYM_FUNC_START(__memcpy_fast)
+ move a3, a0
+ beqz a2, 3f
+
+ ori a4, zero, 64
+ blt a2, a4, 2f
+
+ /* copy 64 bytes at a time */
+1: ld.d t0, a1, 0
+ ld.d t1, a1, 8
+ ld.d t2, a1, 16
+ ld.d t3, a1, 24
+ ld.d t4, a1, 32
+ ld.d t5, a1, 40
+ ld.d t6, a1, 48
+ ld.d t7, a1, 56
+ st.d t0, a0, 0
+ st.d t1, a0, 8
+ st.d t2, a0, 16
+ st.d t3, a0, 24
+ st.d t4, a0, 32
+ st.d t5, a0, 40
+ st.d t6, a0, 48
+ st.d t7, a0, 56
+
+ addi.d a0, a0, 64
+ addi.d a1, a1, 64
+ addi.d a2, a2, -64
+ bge a2, a4, 1b
+
+ beqz a2, 3f
+
+ /* copy the remaining bytes */
+2: ld.b t0, a1, 0
+ st.b t0, a0, 0
+ addi.d a0, a0, 1
+ addi.d a1, a1, 1
+ addi.d a2, a2, -1
+ bgt a2, zero, 2b
+
+ /* return */
+3: move a0, a3
+ jr ra
+SYM_FUNC_END(__memcpy_fast)
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/alternative-asm.h>
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/cpu.h>
+#include <asm/export.h>
+#include <asm/regdef.h>
+
+SYM_FUNC_START(memmove)
+ blt a0, a1, 1f /* dst < src, memcpy */
+ blt a1, a0, 3f /* src < dst, rmemcpy */
+ jr ra /* dst == src, return */
+
+ /* if (src - dst) < 64, copy 1 byte at a time */
+1: ori a3, zero, 64
+ sub.d t0, a1, a0
+ blt t0, a3, 2f
+ b memcpy
+2: b __memcpy_generic
+
+ /* if (dst - src) < 64, copy 1 byte at a time */
+3: ori a3, zero, 64
+ sub.d t0, a0, a1
+ blt t0, a3, 4f
+ b rmemcpy
+4: b __rmemcpy_generic
+SYM_FUNC_END(memmove)
+
+EXPORT_SYMBOL(memmove)
+
+SYM_FUNC_START(rmemcpy)
+ /*
+ * Some CPUs support hardware unaligned access
+ */
+ ALTERNATIVE "b __rmemcpy_generic", \
+ "b __rmemcpy_fast", CPU_FEATURE_UAL
+SYM_FUNC_END(rmemcpy)
+
+/*
+ * void *__rmemcpy_generic(void *dst, const void *src, size_t n)
+ *
+ * a0: dst
+ * a1: src
+ * a2: n
+ */
+SYM_FUNC_START(__rmemcpy_generic)
+ move a3, a0
+ beqz a2, 2f
+
+ add.d a0, a0, a2
+ add.d a1, a1, a2
+
+1: ld.b t0, a1, -1
+ st.b t0, a0, -1
+ addi.d a0, a0, -1
+ addi.d a1, a1, -1
+ addi.d a2, a2, -1
+ bgt a2, zero, 1b
+
+2: move a0, a3
+ jr ra
+SYM_FUNC_END(__rmemcpy_generic)
+
+/*
+ * void *__rmemcpy_fast(void *dst, const void *src, size_t n)
+ *
+ * a0: dst
+ * a1: src
+ * a2: n
+ */
+SYM_FUNC_START(__rmemcpy_fast)
+ move a3, a0
+ beqz a2, 3f
+
+ add.d a0, a0, a2
+ add.d a1, a1, a2
+
+ ori a4, zero, 64
+ blt a2, a4, 2f
+
+ /* copy 64 bytes at a time */
+1: ld.d t0, a1, -8
+ ld.d t1, a1, -16
+ ld.d t2, a1, -24
+ ld.d t3, a1, -32
+ ld.d t4, a1, -40
+ ld.d t5, a1, -48
+ ld.d t6, a1, -56
+ ld.d t7, a1, -64
+ st.d t0, a0, -8
+ st.d t1, a0, -16
+ st.d t2, a0, -24
+ st.d t3, a0, -32
+ st.d t4, a0, -40
+ st.d t5, a0, -48
+ st.d t6, a0, -56
+ st.d t7, a0, -64
+
+ addi.d a0, a0, -64
+ addi.d a1, a1, -64
+ addi.d a2, a2, -64
+ bge a2, a4, 1b
+
+ beqz a2, 3f
+
+ /* copy the remaining bytes */
+2: ld.b t0, a1, -1
+ st.b t0, a0, -1
+ addi.d a0, a0, -1
+ addi.d a1, a1, -1
+ addi.d a2, a2, -1
+ bgt a2, zero, 2b
+
+ /* return */
+3: move a0, a3
+ jr ra
+SYM_FUNC_END(__rmemcpy_fast)
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/alternative-asm.h>
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/cpu.h>
+#include <asm/export.h>
+#include <asm/regdef.h>
+
+.macro fill_to_64 r0
+ bstrins.d \r0, \r0, 15, 8
+ bstrins.d \r0, \r0, 31, 16
+ bstrins.d \r0, \r0, 63, 32
+.endm
+
+SYM_FUNC_START(memset)
+ /*
+ * Some CPUs support hardware unaligned access
+ */
+ ALTERNATIVE "b __memset_generic", \
+ "b __memset_fast", CPU_FEATURE_UAL
+SYM_FUNC_END(memset)
+
+EXPORT_SYMBOL(memset)
+
+/*
+ * void *__memset_generic(void *s, int c, size_t n)
+ *
+ * a0: s
+ * a1: c
+ * a2: n
+ */
+SYM_FUNC_START(__memset_generic)
+ move a3, a0
+ beqz a2, 2f
+
+1: st.b a1, a0, 0
+ addi.d a0, a0, 1
+ addi.d a2, a2, -1
+ bgt a2, zero, 1b
+
+2: move a0, a3
+ jr ra
+SYM_FUNC_END(__memset_generic)
+
+/*
+ * void *__memset_fast(void *s, int c, size_t n)
+ *
+ * a0: s
+ * a1: c
+ * a2: n
+ */
+SYM_FUNC_START(__memset_fast)
+ move a3, a0
+ beqz a2, 3f
+
+ ori a4, zero, 64
+ blt a2, a4, 2f
+
+ /* fill a1 to 64 bits */
+ fill_to_64 a1
+
+ /* set 64 bytes at a time */
+1: st.d a1, a0, 0
+ st.d a1, a0, 8
+ st.d a1, a0, 16
+ st.d a1, a0, 24
+ st.d a1, a0, 32
+ st.d a1, a0, 40
+ st.d a1, a0, 48
+ st.d a1, a0, 56
+
+ addi.d a0, a0, 64
+ addi.d a2, a2, -64
+ bge a2, a4, 1b
+
+ beqz a2, 3f
+
+ /* set the remaining bytes */
+2: st.b a1, a0, 0
+ addi.d a0, a0, 1
+ addi.d a2, a2, -1
+ bgt a2, zero, 2b
+
+ /* return */
+3: move a0, a3
+ jr ra
+SYM_FUNC_END(__memset_fast)
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/linkage.h>
+
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/asm-extable.h>
+#include <asm/errno.h>
+#include <asm/export.h>
+#include <asm/regdef.h>
+
+.L_fixup_handle_unaligned:
+ li.w a0, -EFAULT
+ jr ra
+
+/*
+ * unsigned long unaligned_read(void *addr, void *value, unsigned long n, bool sign)
+ *
+ * a0: addr
+ * a1: value
+ * a2: n
+ * a3: sign
+ */
+SYM_FUNC_START(unaligned_read)
+ beqz a2, 5f
+
+ li.w t2, 0
+ addi.d t0, a2, -1
+ slli.d t1, t0, 3
+ add.d a0, a0, t0
+
+ beqz a3, 2f
+1: ld.b t3, a0, 0
+ b 3f
+
+2: ld.bu t3, a0, 0
+3: sll.d t3, t3, t1
+ or t2, t2, t3
+ addi.d t1, t1, -8
+ addi.d a0, a0, -1
+ addi.d a2, a2, -1
+ bgtz a2, 2b
+4: st.d t2, a1, 0
+
+ move a0, a2
+ jr ra
+
+5: li.w a0, -EFAULT
+ jr ra
+
+ _asm_extable 1b, .L_fixup_handle_unaligned
+ _asm_extable 2b, .L_fixup_handle_unaligned
+ _asm_extable 4b, .L_fixup_handle_unaligned
+SYM_FUNC_END(unaligned_read)
+
+/*
+ * unsigned long unaligned_write(void *addr, unsigned long value, unsigned long n)
+ *
+ * a0: addr
+ * a1: value
+ * a2: n
+ */
+SYM_FUNC_START(unaligned_write)
+ beqz a2, 3f
+
+ li.w t0, 0
+1: srl.d t1, a1, t0
+2: st.b t1, a0, 0
+ addi.d t0, t0, 8
+ addi.d a2, a2, -1
+ addi.d a0, a0, 1
+ bgtz a2, 1b
+
+ move a0, a2
+ jr ra
+
+3: li.w a0, -EFAULT
+ jr ra
+
+ _asm_extable 2b, .L_fixup_handle_unaligned
+SYM_FUNC_END(unaligned_write)
/*
* Copyright (C) 2020-2022 Loongson Technology Corporation Limited
*/
+#include <linux/bitfield.h>
#include <linux/extable.h>
-#include <linux/spinlock.h>
-#include <asm/branch.h>
#include <linux/uaccess.h>
+#include <asm/asm-extable.h>
+#include <asm/branch.h>
+
+static inline unsigned long
+get_ex_fixup(const struct exception_table_entry *ex)
+{
+ return ((unsigned long)&ex->fixup + ex->fixup);
+}
+
+static inline void regs_set_gpr(struct pt_regs *regs,
+ unsigned int offset, unsigned long val)
+{
+ if (offset && offset <= MAX_REG_OFFSET)
+ *(unsigned long *)((unsigned long)regs + offset) = val;
+}
+
+static bool ex_handler_fixup(const struct exception_table_entry *ex,
+ struct pt_regs *regs)
+{
+ regs->csr_era = get_ex_fixup(ex);
+
+ return true;
+}
+
+static bool ex_handler_uaccess_err_zero(const struct exception_table_entry *ex,
+ struct pt_regs *regs)
+{
+ int reg_err = FIELD_GET(EX_DATA_REG_ERR, ex->data);
+ int reg_zero = FIELD_GET(EX_DATA_REG_ZERO, ex->data);
+
+ regs_set_gpr(regs, reg_err * sizeof(unsigned long), -EFAULT);
+ regs_set_gpr(regs, reg_zero * sizeof(unsigned long), 0);
+ regs->csr_era = get_ex_fixup(ex);
+
+ return true;
+}
-int fixup_exception(struct pt_regs *regs)
+bool fixup_exception(struct pt_regs *regs)
{
- const struct exception_table_entry *fixup;
+ const struct exception_table_entry *ex;
- fixup = search_exception_tables(exception_era(regs));
- if (fixup) {
- regs->csr_era = fixup->fixup;
+ ex = search_exception_tables(exception_era(regs));
+ if (!ex)
+ return false;
- return 1;
+ switch (ex->type) {
+ case EX_TYPE_FIXUP:
+ return ex_handler_fixup(ex, regs);
+ case EX_TYPE_UACCESS_ERR_ZERO:
+ return ex_handler_uaccess_err_zero(ex, regs);
+ case EX_TYPE_BPF:
+ return ex_handler_bpf(ex, regs);
}
- return 0;
+ BUG();
}
cond == BPF_JSGE || cond == BPF_JSLE;
}
+#define BPF_FIXUP_REG_MASK GENMASK(31, 27)
+#define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0)
+
+bool ex_handler_bpf(const struct exception_table_entry *ex,
+ struct pt_regs *regs)
+{
+ int dst_reg = FIELD_GET(BPF_FIXUP_REG_MASK, ex->fixup);
+ off_t offset = FIELD_GET(BPF_FIXUP_OFFSET_MASK, ex->fixup);
+
+ regs->regs[dst_reg] = 0;
+ regs->csr_era = (unsigned long)&ex->fixup - offset;
+
+ return true;
+}
+
+/* For accesses to BTF pointers, add an entry to the exception table */
+static int add_exception_handler(const struct bpf_insn *insn,
+ struct jit_ctx *ctx,
+ int dst_reg)
+{
+ unsigned long pc;
+ off_t offset;
+ struct exception_table_entry *ex;
+
+ if (!ctx->image || !ctx->prog->aux->extable || BPF_MODE(insn->code) != BPF_PROBE_MEM)
+ return 0;
+
+ if (WARN_ON_ONCE(ctx->num_exentries >= ctx->prog->aux->num_exentries))
+ return -EINVAL;
+
+ ex = &ctx->prog->aux->extable[ctx->num_exentries];
+ pc = (unsigned long)&ctx->image[ctx->idx - 1];
+
+ offset = pc - (long)&ex->insn;
+ if (WARN_ON_ONCE(offset >= 0 || offset < INT_MIN))
+ return -ERANGE;
+
+ ex->insn = offset;
+
+ /*
+ * Since the extable follows the program, the fixup offset is always
+ * negative and limited to BPF_JIT_REGION_SIZE. Store a positive value
+ * to keep things simple, and put the destination register in the upper
+ * bits. We don't need to worry about buildtime or runtime sort
+ * modifying the upper bits because the table is already sorted, and
+ * isn't part of the main exception table.
+ */
+ offset = (long)&ex->fixup - (pc + LOONGARCH_INSN_SIZE);
+ if (!FIELD_FIT(BPF_FIXUP_OFFSET_MASK, offset))
+ return -ERANGE;
+
+ ex->type = EX_TYPE_BPF;
+ ex->fixup = FIELD_PREP(BPF_FIXUP_OFFSET_MASK, offset) | FIELD_PREP(BPF_FIXUP_REG_MASK, dst_reg);
+
+ ctx->num_exentries++;
+
+ return 0;
+}
+
static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extra_pass)
{
u8 tm = -1;
case BPF_LDX | BPF_MEM | BPF_H:
case BPF_LDX | BPF_MEM | BPF_W:
case BPF_LDX | BPF_MEM | BPF_DW:
+ case BPF_LDX | BPF_PROBE_MEM | BPF_DW:
+ case BPF_LDX | BPF_PROBE_MEM | BPF_W:
+ case BPF_LDX | BPF_PROBE_MEM | BPF_H:
+ case BPF_LDX | BPF_PROBE_MEM | BPF_B:
switch (BPF_SIZE(code)) {
case BPF_B:
if (is_signed_imm12(off)) {
}
break;
}
+
+ ret = add_exception_handler(insn, ctx, dst);
+ if (ret)
+ return ret;
break;
/* *(size *)(dst + off) = imm */
return -1;
}
+ if (WARN_ON_ONCE(ctx->num_exentries != ctx->prog->aux->num_exentries))
+ return -1;
+
return 0;
}
{
bool tmp_blinded = false, extra_pass = false;
u8 *image_ptr;
- int image_size;
+ int image_size, prog_size, extable_size;
struct jit_ctx ctx;
struct jit_data *jit_data;
struct bpf_binary_header *header;
image_ptr = jit_data->image;
header = jit_data->header;
extra_pass = true;
- image_size = sizeof(u32) * ctx.idx;
+ prog_size = sizeof(u32) * ctx.idx;
goto skip_init_ctx;
}
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx);
+ extable_size = prog->aux->num_exentries * sizeof(struct exception_table_entry);
+
/* Now we know the actual image size.
* As each LoongArch instruction is of length 32bit,
* we are translating number of JITed intructions into
* the size required to store these JITed code.
*/
- image_size = sizeof(u32) * ctx.idx;
+ prog_size = sizeof(u32) * ctx.idx;
+ image_size = prog_size + extable_size;
/* Now we know the size of the structure to make */
header = bpf_jit_binary_alloc(image_size, &image_ptr,
sizeof(u32), jit_fill_hole);
/* 2. Now, the actual pass to generate final JIT code */
ctx.image = (union loongarch_instruction *)image_ptr;
+ if (extable_size)
+ prog->aux->extable = (void *)image_ptr + prog_size;
skip_init_ctx:
ctx.idx = 0;
+ ctx.num_exentries = 0;
build_prologue(&ctx);
if (build_body(&ctx, extra_pass)) {
/* And we're done */
if (bpf_jit_enable > 1)
- bpf_jit_dump(prog->len, image_size, 2, ctx.image);
+ bpf_jit_dump(prog->len, prog_size, 2, ctx.image);
/* Update the icache */
flush_icache_range((unsigned long)header, (unsigned long)(ctx.image + ctx.idx));
jit_data->header = header;
}
prog->jited = 1;
- prog->jited_len = image_size;
+ prog->jited_len = prog_size;
prog->bpf_func = (void *)ctx.image;
if (!prog->is_func || extra_pass) {
*
* Copyright (C) 2022 Loongson Technology Corporation Limited
*/
+#include <linux/bitfield.h>
#include <linux/bpf.h>
#include <linux/filter.h>
#include <asm/cacheflush.h>
unsigned int flags;
unsigned int epilogue_offset;
u32 *offset;
+ int num_exentries;
union loongarch_instruction *image;
u32 stack_size;
};
int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
{
- struct pci_config_window *cfg = bridge->bus->sysdata;
- struct acpi_device *adev = to_acpi_device(cfg->parent);
+ struct acpi_device *adev = NULL;
struct device *bus_dev = &bridge->bus->dev;
+ struct pci_config_window *cfg = bridge->bus->sysdata;
+
+ if (!acpi_disabled)
+ adev = to_acpi_device(cfg->parent);
ACPI_COMPANION_SET(&bridge->dev, adev);
set_dev_node(bus_dev, pa_to_nid(cfg->res.start));
--- /dev/null
+obj-y += platform.o
+
+obj-$(CONFIG_SUSPEND) += suspend.o suspend_asm.o
+obj-$(CONFIG_HIBERNATION) += hibernate.o hibernate_asm.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <asm/fpu.h>
+#include <asm/loongson.h>
+#include <asm/sections.h>
+#include <asm/tlbflush.h>
+#include <linux/suspend.h>
+
+static u32 saved_crmd;
+static u32 saved_prmd;
+static u32 saved_euen;
+static u32 saved_ecfg;
+static u64 saved_pcpu_base;
+struct pt_regs saved_regs;
+
+void save_processor_state(void)
+{
+ saved_crmd = csr_read32(LOONGARCH_CSR_CRMD);
+ saved_prmd = csr_read32(LOONGARCH_CSR_PRMD);
+ saved_euen = csr_read32(LOONGARCH_CSR_EUEN);
+ saved_ecfg = csr_read32(LOONGARCH_CSR_ECFG);
+ saved_pcpu_base = csr_read64(PERCPU_BASE_KS);
+
+ if (is_fpu_owner())
+ save_fp(current);
+}
+
+void restore_processor_state(void)
+{
+ csr_write32(saved_crmd, LOONGARCH_CSR_CRMD);
+ csr_write32(saved_prmd, LOONGARCH_CSR_PRMD);
+ csr_write32(saved_euen, LOONGARCH_CSR_EUEN);
+ csr_write32(saved_ecfg, LOONGARCH_CSR_ECFG);
+ csr_write64(saved_pcpu_base, PERCPU_BASE_KS);
+
+ if (is_fpu_owner())
+ restore_fp(current);
+}
+
+int pfn_is_nosave(unsigned long pfn)
+{
+ unsigned long nosave_begin_pfn = PFN_DOWN(__pa(&__nosave_begin));
+ unsigned long nosave_end_pfn = PFN_UP(__pa(&__nosave_end));
+
+ return (pfn >= nosave_begin_pfn) && (pfn < nosave_end_pfn);
+}
+
+extern int swsusp_asm_suspend(void);
+
+int swsusp_arch_suspend(void)
+{
+ enable_pci_wakeup();
+ return swsusp_asm_suspend();
+}
+
+extern int swsusp_asm_resume(void);
+
+int swsusp_arch_resume(void)
+{
+ /* Avoid TLB mismatch during and after kernel resume */
+ local_flush_tlb_all();
+ return swsusp_asm_resume();
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Hibernation support specific for LoongArch
+ *
+ * Author: Huacai Chen <chenhuacai@loongson.cn>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/linkage.h>
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/regdef.h>
+
+.text
+SYM_FUNC_START(swsusp_asm_suspend)
+ la.pcrel t0, saved_regs
+ PTR_S ra, t0, PT_R1
+ PTR_S tp, t0, PT_R2
+ PTR_S sp, t0, PT_R3
+ PTR_S u0, t0, PT_R21
+ PTR_S fp, t0, PT_R22
+ PTR_S s0, t0, PT_R23
+ PTR_S s1, t0, PT_R24
+ PTR_S s2, t0, PT_R25
+ PTR_S s3, t0, PT_R26
+ PTR_S s4, t0, PT_R27
+ PTR_S s5, t0, PT_R28
+ PTR_S s6, t0, PT_R29
+ PTR_S s7, t0, PT_R30
+ PTR_S s8, t0, PT_R31
+ b swsusp_save
+SYM_FUNC_END(swsusp_asm_suspend)
+
+SYM_FUNC_START(swsusp_asm_resume)
+ la.pcrel t0, restore_pblist
+ PTR_L t0, t0, 0
+0:
+ PTR_L t1, t0, PBE_ADDRESS /* source */
+ PTR_L t2, t0, PBE_ORIG_ADDRESS /* destination */
+ PTR_LI t3, _PAGE_SIZE
+ PTR_ADD t3, t3, t1
+1:
+ REG_L t8, t1, 0
+ REG_S t8, t2, 0
+ PTR_ADDI t1, t1, SZREG
+ PTR_ADDI t2, t2, SZREG
+ bne t1, t3, 1b
+ PTR_L t0, t0, PBE_NEXT
+ bnez t0, 0b
+ la.pcrel t0, saved_regs
+ PTR_L ra, t0, PT_R1
+ PTR_L tp, t0, PT_R2
+ PTR_L sp, t0, PT_R3
+ PTR_L u0, t0, PT_R21
+ PTR_L fp, t0, PT_R22
+ PTR_L s0, t0, PT_R23
+ PTR_L s1, t0, PT_R24
+ PTR_L s2, t0, PT_R25
+ PTR_L s3, t0, PT_R26
+ PTR_L s4, t0, PT_R27
+ PTR_L s5, t0, PT_R28
+ PTR_L s6, t0, PT_R29
+ PTR_L s7, t0, PT_R30
+ PTR_L s8, t0, PT_R31
+ PTR_LI a0, 0x0
+ jirl zero, ra, 0
+SYM_FUNC_END(swsusp_asm_resume)
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Author: Huacai Chen <chenhuacai@loongson.cn>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/acpi.h>
+#include <linux/platform_device.h>
+
+#include <asm/bootinfo.h>
+#include <asm/loongson.h>
+
+void enable_gpe_wakeup(void)
+{
+ if (acpi_disabled)
+ return;
+
+ if (acpi_gbl_reduced_hardware)
+ return;
+
+ acpi_enable_all_wakeup_gpes();
+}
+
+void enable_pci_wakeup(void)
+{
+ if (acpi_disabled)
+ return;
+
+ if (acpi_gbl_reduced_hardware)
+ return;
+
+ acpi_write_bit_register(ACPI_BITREG_PCIEXP_WAKE_STATUS, 1);
+
+ if (acpi_gbl_FADT.flags & ACPI_FADT_PCI_EXPRESS_WAKE)
+ acpi_write_bit_register(ACPI_BITREG_PCIEXP_WAKE_DISABLE, 0);
+}
+
+static int __init loongson3_acpi_suspend_init(void)
+{
+#ifdef CONFIG_ACPI
+ acpi_status status;
+ uint64_t suspend_addr = 0;
+
+ if (acpi_disabled || acpi_gbl_reduced_hardware)
+ return 0;
+
+ acpi_write_bit_register(ACPI_BITREG_SCI_ENABLE, 1);
+ status = acpi_evaluate_integer(NULL, "\\SADR", NULL, &suspend_addr);
+ if (ACPI_FAILURE(status) || !suspend_addr) {
+ pr_err("ACPI S3 is not support!\n");
+ return -1;
+ }
+ loongson_sysconf.suspend_addr = (u64)phys_to_virt(PHYSADDR(suspend_addr));
+#endif
+ return 0;
+}
+
+device_initcall(loongson3_acpi_suspend_init);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * loongson-specific suspend support
+ *
+ * Author: Huacai Chen <chenhuacai@loongson.cn>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+#include <linux/acpi.h>
+#include <linux/pm.h>
+#include <linux/suspend.h>
+
+#include <asm/loongarch.h>
+#include <asm/loongson.h>
+#include <asm/setup.h>
+#include <asm/time.h>
+#include <asm/tlbflush.h>
+
+u64 loongarch_suspend_addr;
+
+struct saved_registers {
+ u32 ecfg;
+ u32 euen;
+ u64 pgd;
+ u64 kpgd;
+ u32 pwctl0;
+ u32 pwctl1;
+};
+static struct saved_registers saved_regs;
+
+static void arch_common_suspend(void)
+{
+ save_counter();
+ saved_regs.pgd = csr_read64(LOONGARCH_CSR_PGDL);
+ saved_regs.kpgd = csr_read64(LOONGARCH_CSR_PGDH);
+ saved_regs.pwctl0 = csr_read32(LOONGARCH_CSR_PWCTL0);
+ saved_regs.pwctl1 = csr_read32(LOONGARCH_CSR_PWCTL1);
+ saved_regs.ecfg = csr_read32(LOONGARCH_CSR_ECFG);
+ saved_regs.euen = csr_read32(LOONGARCH_CSR_EUEN);
+
+ loongarch_suspend_addr = loongson_sysconf.suspend_addr;
+}
+
+static void arch_common_resume(void)
+{
+ sync_counter();
+ local_flush_tlb_all();
+ csr_write64(per_cpu_offset(0), PERCPU_BASE_KS);
+ csr_write64(eentry, LOONGARCH_CSR_EENTRY);
+ csr_write64(eentry, LOONGARCH_CSR_MERRENTRY);
+ csr_write64(tlbrentry, LOONGARCH_CSR_TLBRENTRY);
+
+ csr_write64(saved_regs.pgd, LOONGARCH_CSR_PGDL);
+ csr_write64(saved_regs.kpgd, LOONGARCH_CSR_PGDH);
+ csr_write32(saved_regs.pwctl0, LOONGARCH_CSR_PWCTL0);
+ csr_write32(saved_regs.pwctl1, LOONGARCH_CSR_PWCTL1);
+ csr_write32(saved_regs.ecfg, LOONGARCH_CSR_ECFG);
+ csr_write32(saved_regs.euen, LOONGARCH_CSR_EUEN);
+}
+
+int loongarch_acpi_suspend(void)
+{
+ enable_gpe_wakeup();
+ enable_pci_wakeup();
+
+ arch_common_suspend();
+
+ /* processor specific suspend */
+ loongarch_suspend_enter();
+
+ arch_common_resume();
+
+ return 0;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Sleep helper for Loongson-3 sleep mode.
+ *
+ * Author: Huacai Chen <chenhuacai@loongson.cn>
+ * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/addrspace.h>
+#include <asm/loongarch.h>
+#include <asm/stackframe.h>
+
+/* preparatory stuff */
+.macro SETUP_SLEEP
+ addi.d sp, sp, -PT_SIZE
+ st.d $r1, sp, PT_R1
+ st.d $r2, sp, PT_R2
+ st.d $r3, sp, PT_R3
+ st.d $r4, sp, PT_R4
+ st.d $r21, sp, PT_R21
+ st.d $r22, sp, PT_R22
+ st.d $r23, sp, PT_R23
+ st.d $r24, sp, PT_R24
+ st.d $r25, sp, PT_R25
+ st.d $r26, sp, PT_R26
+ st.d $r27, sp, PT_R27
+ st.d $r28, sp, PT_R28
+ st.d $r29, sp, PT_R29
+ st.d $r30, sp, PT_R30
+ st.d $r31, sp, PT_R31
+
+ la.pcrel t0, acpi_saved_sp
+ st.d sp, t0, 0
+.endm
+
+.macro SETUP_WAKEUP
+ ld.d $r1, sp, PT_R1
+ ld.d $r2, sp, PT_R2
+ ld.d $r3, sp, PT_R3
+ ld.d $r4, sp, PT_R4
+ ld.d $r21, sp, PT_R21
+ ld.d $r22, sp, PT_R22
+ ld.d $r23, sp, PT_R23
+ ld.d $r24, sp, PT_R24
+ ld.d $r25, sp, PT_R25
+ ld.d $r26, sp, PT_R26
+ ld.d $r27, sp, PT_R27
+ ld.d $r28, sp, PT_R28
+ ld.d $r29, sp, PT_R29
+ ld.d $r30, sp, PT_R30
+ ld.d $r31, sp, PT_R31
+.endm
+
+ .text
+ .align 12
+
+/* Sleep/wakeup code for Loongson-3 */
+SYM_FUNC_START(loongarch_suspend_enter)
+ SETUP_SLEEP
+ bl __flush_cache_all
+
+ /* Pass RA and SP to BIOS */
+ addi.d a1, sp, 0
+ la.pcrel a0, loongarch_wakeup_start
+ la.pcrel t0, loongarch_suspend_addr
+ ld.d t0, t0, 0
+ jirl a0, t0, 0 /* Call BIOS's STR sleep routine */
+
+ /*
+ * This is where we return upon wakeup.
+ * Reload all of the registers and return.
+ */
+SYM_INNER_LABEL(loongarch_wakeup_start, SYM_L_GLOBAL)
+ li.d t0, CSR_DMW0_INIT # UC, PLV0
+ csrwr t0, LOONGARCH_CSR_DMWIN0
+ li.d t0, CSR_DMW1_INIT # CA, PLV0
+ csrwr t0, LOONGARCH_CSR_DMWIN1
+
+ la.abs t0, 0f
+ jr t0
+0:
+ la.pcrel t0, acpi_saved_sp
+ ld.d sp, t0, 0
+ SETUP_WAKEUP
+ addi.d sp, sp, PT_SIZE
+ jr ra
+SYM_FUNC_END(loongarch_suspend_enter)
return xdest;
}
-#ifndef CONFIG_COLDFIRE
-#define __HAVE_ARCH_STRCMP
-static inline int strcmp(const char *cs, const char *ct)
-{
- char res;
-
- asm ("\n"
- "1: move.b (%0)+,%2\n" /* get *cs */
- " cmp.b (%1)+,%2\n" /* compare a byte */
- " jne 2f\n" /* not equal, break out */
- " tst.b %2\n" /* at end of cs? */
- " jne 1b\n" /* no, keep going */
- " jra 3f\n" /* strings are equal */
- "2: sub.b -(%1),%2\n" /* *cs - *ct */
- "3:"
- : "+a" (cs), "+a" (ct), "=d" (res));
- return res;
-}
-#endif /* CONFIG_COLDFIRE */
-
#define __HAVE_ARCH_MEMMOVE
extern void *memmove(void *, const void *, __kernel_size_t);
config_BSP(&command_line[0], sizeof(command_line));
#if defined(CONFIG_BOOTPARAM)
- strncpy(&command_line[0], CONFIG_BOOTPARAM_STRING, sizeof(command_line));
- command_line[sizeof(command_line) - 1] = 0;
+ strscpy(&command_line[0], CONFIG_BOOTPARAM_STRING, sizeof(command_line));
#endif /* CONFIG_BOOTPARAM */
process_uboot_commandline(&command_line[0], sizeof(command_line));
select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
- select GUP_GET_PTE_LOW_HIGH if CPU_MIPS32 && PHYS_ADDR_T_64BIT
+ select GUP_GET_PXX_LOW_HIGH if CPU_MIPS32 && PHYS_ADDR_T_64BIT
select HAVE_ARCH_COMPILER_H
select HAVE_ARCH_JUMP_LABEL
select HAVE_ARCH_KGDB if MIPS_FP_SUPPORT
/**
* Convert a memory pointer (void*) into a hardware compatible
- * memory address (uint64_t). Octeon hardware widgets don't
+ * memory address (phys_addr_t). Octeon hardware widgets don't
* understand logical addresses.
*
* @ptr: C style memory pointer
* Returns Hardware physical address
*/
-static inline uint64_t cvmx_ptr_to_phys(void *ptr)
+static inline phys_addr_t cvmx_ptr_to_phys(void *ptr)
{
if (sizeof(void *) == 8) {
/*
int pdc_system_map_find_addrs(struct pdc_system_map_addr_info *pdc_addr_info,
long mod_index, long addr_index);
int pdc_model_info(struct pdc_model *model);
-int pdc_model_sysmodel(char *name);
+int pdc_model_sysmodel(unsigned int os_id, char *name);
int pdc_model_cpuid(unsigned long *cpu_id);
int pdc_model_versions(unsigned long *versions, int id);
int pdc_model_capabilities(unsigned long *capabilities);
/* This calculates the number of initial pages we need for the initial
* page tables */
-#if (KERNEL_INITIAL_ORDER) >= (PMD_SHIFT)
-# define PT_INITIAL (1 << (KERNEL_INITIAL_ORDER - PMD_SHIFT))
+#if (KERNEL_INITIAL_ORDER) >= (PLD_SHIFT + BITS_PER_PTE)
+# define PT_INITIAL (1 << (KERNEL_INITIAL_ORDER - PLD_SHIFT - BITS_PER_PTE))
#else
# define PT_INITIAL (1) /* all initial PTEs fit into one page */
#endif
#define MADV_DONTFORK 10 /* don't inherit across fork */
#define MADV_DOFORK 11 /* do inherit across fork */
+#define MADV_MERGEABLE 12 /* KSM may merge identical pages */
+#define MADV_UNMERGEABLE 13 /* KSM may not merge identical pages */
+
+#define MADV_HUGEPAGE 14 /* Worth backing with hugepages */
+#define MADV_NOHUGEPAGE 15 /* Not worth backing with hugepages */
+
+#define MADV_DONTDUMP 16 /* Explicity exclude from the core dump,
+ overrides the coredump filter bits */
+#define MADV_DODUMP 17 /* Clear the MADV_NODUMP flag */
+
+#define MADV_WIPEONFORK 18 /* Zero memory on fork, child only */
+#define MADV_KEEPONFORK 19 /* Undo MADV_WIPEONFORK */
+
#define MADV_COLD 20 /* deactivate these pages */
#define MADV_PAGEOUT 21 /* reclaim these pages */
#define MADV_DONTNEED_LOCKED 24 /* like DONTNEED, but drop locked pages too */
-#define MADV_MERGEABLE 65 /* KSM may merge identical pages */
-#define MADV_UNMERGEABLE 66 /* KSM may not merge identical pages */
-
-#define MADV_HUGEPAGE 67 /* Worth backing with hugepages */
-#define MADV_NOHUGEPAGE 68 /* Not worth backing with hugepages */
-
-#define MADV_DONTDUMP 69 /* Explicity exclude from the core dump,
- overrides the coredump filter bits */
-#define MADV_DODUMP 70 /* Clear the MADV_NODUMP flag */
-
-#define MADV_WIPEONFORK 71 /* Zero memory on fork, child only */
-#define MADV_KEEPONFORK 72 /* Undo MADV_WIPEONFORK */
-
-#define MADV_COLLAPSE 73 /* Synchronous hugepage collapse */
+#define MADV_COLLAPSE 25 /* Synchronous hugepage collapse */
#define MADV_HWPOISON 100 /* poison a page for testing */
#define MADV_SOFT_OFFLINE 101 /* soft offline page for testing */
/* compatibility flags */
#define MAP_FILE 0
-#define MAP_VARIABLE 0
#define PKEY_DISABLE_ACCESS 0x1
#define PKEY_DISABLE_WRITE 0x2
static DEFINE_SPINLOCK(pdc_lock);
#endif
-extern unsigned long pdc_result[NUM_PDC_RESULT];
-extern unsigned long pdc_result2[NUM_PDC_RESULT];
+unsigned long pdc_result[NUM_PDC_RESULT] __aligned(8);
+unsigned long pdc_result2[NUM_PDC_RESULT] __aligned(8);
#ifdef CONFIG_64BIT
#define WIDE_FIRMWARE 0x1
* Using OS_ID_HPUX will return the equivalent of the 'modelname' command
* on HP/UX.
*/
-int pdc_model_sysmodel(char *name)
+int pdc_model_sysmodel(unsigned int os_id, char *name)
{
int retval;
unsigned long flags;
spin_lock_irqsave(&pdc_lock, flags);
retval = mem_pdc_call(PDC_MODEL, PDC_MODEL_SYSMODEL, __pa(pdc_result),
- OS_ID_HPUX, __pa(name));
+ os_id, __pa(name));
convert_to_wide(pdc_result);
if (retval == PDC_OK) {
#endif /* defined(BOOTLOADER) */
-/* locked by pdc_console_lock */
-static int __attribute__((aligned(8))) iodc_retbuf[32];
-static char __attribute__((aligned(64))) iodc_dbuf[4096];
+/* locked by pdc_lock */
+static char iodc_dbuf[4096] __page_aligned_bss;
/**
* pdc_iodc_print - Console print using IODC.
unsigned int i;
unsigned long flags;
+ count = min_t(unsigned int, count, sizeof(iodc_dbuf));
+
+ spin_lock_irqsave(&pdc_lock, flags);
for (i = 0; i < count;) {
switch(str[i]) {
case '\n':
}
print:
- spin_lock_irqsave(&pdc_lock, flags);
- real32_call(PAGE0->mem_cons.iodc_io,
- (unsigned long)PAGE0->mem_cons.hpa, ENTRY_IO_COUT,
- PAGE0->mem_cons.spa, __pa(PAGE0->mem_cons.dp.layers),
- __pa(iodc_retbuf), 0, __pa(iodc_dbuf), i, 0);
- spin_unlock_irqrestore(&pdc_lock, flags);
+ real32_call(PAGE0->mem_cons.iodc_io,
+ (unsigned long)PAGE0->mem_cons.hpa, ENTRY_IO_COUT,
+ PAGE0->mem_cons.spa, __pa(PAGE0->mem_cons.dp.layers),
+ __pa(pdc_result), 0, __pa(iodc_dbuf), i, 0);
+ spin_unlock_irqrestore(&pdc_lock, flags);
return i;
}
real32_call(PAGE0->mem_kbd.iodc_io,
(unsigned long)PAGE0->mem_kbd.hpa, ENTRY_IO_CIN,
PAGE0->mem_kbd.spa, __pa(PAGE0->mem_kbd.dp.layers),
- __pa(iodc_retbuf), 0, __pa(iodc_dbuf), 1, 0);
+ __pa(pdc_result), 0, __pa(iodc_dbuf), 1, 0);
ch = *iodc_dbuf;
- status = *iodc_retbuf;
+ /* like convert_to_wide() but for first return value only: */
+ status = *(int *)&pdc_result;
spin_unlock_irqrestore(&pdc_lock, flags);
if (status == 0)
}
return -1;
}
-
-/* KGDB console driver which uses PDC to read chars from keyboard */
-
-static void kgdb_pdc_write_char(u8 chr)
-{
- /* no need to print char. kgdb will do it. */
-}
-
-static struct kgdb_io kgdb_pdc_io_ops = {
- .name = "kgdb_pdc",
- .read_char = pdc_iodc_getc,
- .write_char = kgdb_pdc_write_char,
-};
-
-static int __init kgdb_pdc_init(void)
-{
- kgdb_register_io_module(&kgdb_pdc_io_ops);
- return 0;
-}
-early_initcall(kgdb_pdc_init);
#include <asm/page.h> /* for PAGE0 */
#include <asm/pdc.h> /* for iodc_call() proto and friends */
-static DEFINE_SPINLOCK(pdc_console_lock);
-
static void pdc_console_write(struct console *co, const char *s, unsigned count)
{
int i = 0;
- unsigned long flags;
- spin_lock_irqsave(&pdc_console_lock, flags);
do {
i += pdc_iodc_print(s + i, count - i);
} while (i < count);
- spin_unlock_irqrestore(&pdc_console_lock, flags);
}
#ifdef CONFIG_KGDB
static int kgdb_pdc_read_char(void)
{
- int c;
- unsigned long flags;
-
- spin_lock_irqsave(&pdc_console_lock, flags);
- c = pdc_iodc_getc();
- spin_unlock_irqrestore(&pdc_console_lock, flags);
+ int c = pdc_iodc_getc();
return (c <= 0) ? NO_POLL_CHAR : c;
}
static void kgdb_pdc_write_char(u8 chr)
{
- if (PAGE0->mem_cons.cl_class != CL_DUPLEX)
- pdc_console_write(NULL, &chr, 1);
+ /* no need to print char as it's shown on standard console */
+ /* pdc_iodc_print(&chr, 1); */
}
static struct kgdb_io kgdb_pdc_io_ops = {
printk(KERN_INFO "capabilities 0x%lx\n",
boot_cpu_data.pdc.capabilities);
- if (pdc_model_sysmodel(boot_cpu_data.pdc.sys_model_name) == PDC_OK)
- printk(KERN_INFO "model %s\n",
+ if (pdc_model_sysmodel(OS_ID_HPUX, boot_cpu_data.pdc.sys_model_name) == PDC_OK)
+ pr_info("HP-UX model name: %s\n",
boot_cpu_data.pdc.sys_model_name);
+ serial_no[0] = 0;
+ if (pdc_model_sysmodel(OS_ID_MPEXL, serial_no) == PDC_OK &&
+ serial_no[0])
+ pr_info("MPE/iX model name: %s\n", serial_no);
+
dump_stack_set_arch_desc("%s", boot_cpu_data.pdc.sys_model_name);
boot_cpu_data.hversion = boot_cpu_data.pdc.model.hversion;
#include <linux/linkage.h>
-
- .section .bss
-
- .export pdc_result
- .export pdc_result2
- .align 8
-pdc_result:
- .block ASM_PDC_RESULT_SIZE
-pdc_result2:
- .block ASM_PDC_RESULT_SIZE
-
.export real_stack
- .export real32_stack
.export real64_stack
- .align 64
+ __PAGE_ALIGNED_BSS
real_stack:
-real32_stack:
real64_stack:
.block 8192
#define N_SAVED_REGS 9
-
+ .section .bss
save_cr_space:
.block REG_SZ * N_SAVED_REGS
save_cr_end:
extern unsigned int boot_args[];
char *p;
- /* Collect stuff passed in from the boot loader */
+ *cmdline_p = command_line;
/* boot_args[0] is free-mem start, boot_args[1] is ptr to command line */
- if (boot_args[0] < 64) {
- /* called from hpux boot loader */
- boot_command_line[0] = '\0';
- } else {
- strscpy(boot_command_line, (char *)__va(boot_args[1]),
- COMMAND_LINE_SIZE);
+ if (boot_args[0] < 64)
+ return; /* return if called from hpux boot loader */
+
+ /* Collect stuff passed in from the boot loader */
+ strscpy(boot_command_line, (char *)__va(boot_args[1]),
+ COMMAND_LINE_SIZE);
/* autodetect console type (if not done by palo yet) */
p = boot_command_line;
strlcat(p, " earlycon=pdc", COMMAND_LINE_SIZE);
#ifdef CONFIG_BLK_DEV_INITRD
- if (boot_args[2] != 0) /* did palo pass us a ramdisk? */
- {
- initrd_start = (unsigned long)__va(boot_args[2]);
- initrd_end = (unsigned long)__va(boot_args[3]);
- }
-#endif
+ /* did palo pass us a ramdisk? */
+ if (boot_args[2] != 0) {
+ initrd_start = (unsigned long)__va(boot_args[2]);
+ initrd_end = (unsigned long)__va(boot_args[3]);
}
+#endif
strscpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
- *cmdline_p = command_line;
}
#ifdef CONFIG_PA11
flags = FIX_O_NONBLOCK(flags);
return sys_inotify_init1(flags);
}
+
+/*
+ * madvise() wrapper
+ *
+ * Up to kernel v6.1 parisc has different values than all other
+ * platforms for the MADV_xxx flags listed below.
+ * To keep binary compatibility with existing userspace programs
+ * translate the former values to the new values.
+ *
+ * XXX: Remove this wrapper in year 2025 (or later)
+ */
+
+asmlinkage notrace long parisc_madvise(unsigned long start, size_t len_in, int behavior)
+{
+ switch (behavior) {
+ case 65: behavior = MADV_MERGEABLE; break;
+ case 66: behavior = MADV_UNMERGEABLE; break;
+ case 67: behavior = MADV_HUGEPAGE; break;
+ case 68: behavior = MADV_NOHUGEPAGE; break;
+ case 69: behavior = MADV_DONTDUMP; break;
+ case 70: behavior = MADV_DODUMP; break;
+ case 71: behavior = MADV_WIPEONFORK; break;
+ case 72: behavior = MADV_KEEPONFORK; break;
+ case 73: behavior = MADV_COLLAPSE; break;
+ }
+
+ return sys_madvise(start, len_in, behavior);
+}
116 common sysinfo sys_sysinfo compat_sys_sysinfo
117 common shutdown sys_shutdown
118 common fsync sys_fsync
-119 common madvise sys_madvise
+119 common madvise parisc_madvise
120 common clone sys_clone_wrapper
121 common setdomainname sys_setdomainname
122 common sendfile sys_sendfile compat_sys_sendfile
# Force dependency (incbin is bad)
# link rule for the .so file, .lds has to be first
-$(obj)/vdso32.so: $(src)/vdso32.lds $(obj-vdso32) $(obj-cvdso32) $(VDSO_LIBGCC)
+$(obj)/vdso32.so: $(src)/vdso32.lds $(obj-vdso32) $(obj-cvdso32) $(VDSO_LIBGCC) FORCE
$(call if_changed,vdso32ld)
# assembly rules for the .S files
# actual build commands
quiet_cmd_vdso32ld = VDSO32L $@
- cmd_vdso32ld = $(CROSS32CC) $(c_flags) -Wl,-T $^ -o $@
+ cmd_vdso32ld = $(CROSS32CC) $(c_flags) -Wl,-T $(filter-out FORCE, $^) -o $@
quiet_cmd_vdso32as = VDSO32A $@
cmd_vdso32as = $(CROSS32CC) $(a_flags) -c -o $@ $<
quiet_cmd_vdso32cc = VDSO32C $@
# Force dependency (incbin is bad)
# link rule for the .so file, .lds has to be first
-$(obj)/vdso64.so: $(src)/vdso64.lds $(obj-vdso64) $(VDSO_LIBGCC)
+$(obj)/vdso64.so: $(src)/vdso64.lds $(obj-vdso64) $(VDSO_LIBGCC) FORCE
$(call if_changed,vdso64ld)
# assembly rules for the .S files
# actual build commands
quiet_cmd_vdso64ld = VDSO64L $@
- cmd_vdso64ld = $(CC) $(c_flags) -Wl,-T $^ -o $@
+ cmd_vdso64ld = $(CC) $(c_flags) -Wl,-T $(filter-out FORCE, $^) -o $@
quiet_cmd_vdso64as = VDSO64A $@
cmd_vdso64as = $(CC) $(a_flags) -c -o $@ $<
# SPDX-License-Identifier: GPL-2.0
source "arch/powerpc/platforms/Kconfig.cputype"
+config CC_HAS_ELFV2
+ def_bool PPC64 && $(cc-option, -mabi=elfv2)
+
config 32BIT
bool
default y if PPC32
config GENERIC_LOCKBREAK
bool
default y
- depends on SMP && PREEMPTION
+ depends on SMP && PREEMPTION && !PPC_QUEUED_SPINLOCKS
config GENERIC_HWEIGHT
bool
select ARCH_USE_CMPXCHG_LOCKREF if PPC64
select ARCH_USE_MEMTEST
select ARCH_USE_QUEUED_RWLOCKS if PPC_QUEUED_SPINLOCKS
- select ARCH_USE_QUEUED_SPINLOCKS if PPC_QUEUED_SPINLOCKS
select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT
select ARCH_WANT_IPC_PARSE_VERSION
select ARCH_WANT_IRQS_OFF_ACTIVATE_MM
select HAVE_MOD_ARCH_SPECIFIC
select HAVE_NMI if PERF_EVENTS || (PPC64 && PPC_BOOK3S)
select HAVE_OPTPROBES
+ select HAVE_OBJTOOL if PPC32 || MPROFILE_KERNEL
+ select HAVE_OBJTOOL_MCOUNT if HAVE_OBJTOOL
select HAVE_PERF_EVENTS
select HAVE_PERF_EVENTS_NMI if PPC64
select HAVE_PERF_REGS
default y
depends on PPC_BOOK3S_64 || PPC_E500
+config PPC_HAS_LBARX_LHARX
+ bool
+
config EARLY_PRINTK
bool
default y
Say N if you are unsure.
+config INTERRUPT_SANITIZE_REGISTERS
+ bool "Clear gprs on interrupt arrival"
+ depends on PPC64 && ARCH_HAS_SYSCALL_WRAPPER
+ default PPC_BOOK3E_64 || PPC_PSERIES || PPC_POWERNV
+ help
+ Reduce the influence of user register state on interrupt handlers and
+ syscalls through clearing user state from registers before handling
+ the exception.
+
config PPC_QUEUED_SPINLOCKS
bool "Queued spinlocks" if EXPERT
depends on SMP
config ARCH_HAS_KEXEC_PURGATORY
def_bool KEXEC_FILE
+config PPC64_BIG_ENDIAN_ELF_ABI_V2
+ bool "Build big-endian kernel using ELF ABI V2 (EXPERIMENTAL)"
+ depends on PPC64 && CPU_BIG_ENDIAN
+ depends on CC_HAS_ELFV2
+ depends on LD_IS_BFD && LD_VERSION >= 22400
+ default n
+ help
+ This builds the kernel image using the "Power Architecture 64-Bit ELF
+ V2 ABI Specification", which has a reduced stack overhead and faster
+ function calls. This internal kernel ABI option does not affect
+ userspace compatibility.
+
+ The V2 ABI is standard for 64-bit little-endian, but for big-endian
+ it is less well tested by kernel and toolchain. However some distros
+ build userspace this way, and it can produce a functioning kernel.
+
+ This requires GCC and binutils 2.24 or newer.
+
config RELOCATABLE
bool "Build a relocatable kernel"
depends on PPC64 || (FLATMEM && (44x || PPC_85xx))
read/write operations on these variables. Say Y if you have
secure boot enabled and want to expose variables to userspace.
-config PPC_RTAS_FILTER
- bool "Enable filtering of RTAS syscalls"
- default y
- depends on PPC_RTAS
- help
- The RTAS syscall API has security issues that could be used to
- compromise system integrity. This option enforces restrictions on the
- RTAS calls and arguments passed by userspace programs to mitigate
- these issues.
-
- Say Y unless you know what you are doing and the filter is causing
- problems for you.
-
endmenu
config ISA_DMA_API
};
i2c@118000 {
- pca9547@77 {
+ i2c-mux@77 {
compatible = "nxp,pca9547";
reg = <0x77>;
#address-cells = <1>;
};
i2c@118100 {
- pca9546@77 {
+ i2c-mux@77 {
compatible = "nxp,pca9546";
reg = <0x77>;
#address-cells = <1>;
};
i2c@118000 {
- pca9547@77 {
+ i2c-mux@77 {
compatible = "nxp,pca9547";
reg = <0x77>;
};
};
i2c@118100 {
- pca9546@77 {
+ i2c-mux@77 {
compatible = "nxp,pca9546";
reg = <0x77>;
#address-cells = <1>;
};
i2c@118000 {
- pca9547@77 {
+ i2c-mux@77 {
compatible = "nxp,pca9547";
reg = <0x77>;
#address-cells = <1>;
};
i2c@118100 {
- pca9546@77 {
+ i2c-mux@77 {
compatible = "nxp,pca9546";
reg = <0x77>;
};
reg = <0x00000000 0x00000000 0x00000000 0x10000000>;
};
+ clocks {
+ sys_clk: litex_sys_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <100000000>;
+ };
+ };
+
cpus {
#size-cells = <0x00>;
#address-cells = <0x01>;
litex,slot-size = <0x800>;
interrupts = <0x11 0x1>;
};
+
+ mmc@8040000 {
+ compatible = "litex,mmc";
+ reg = <0x8042800 0x800
+ 0x8041000 0x800
+ 0x8040800 0x800
+ 0x8042000 0x800
+ 0x8041800 0x800>;
+ reg-names = "phy", "core", "reader", "writer", "irq";
+ bus-width = <4>;
+ interrupts = <0x13 1>;
+ cap-sd-highspeed;
+ clocks = <&sys_clk>;
+ };
};
chosen {
interrupt-parent = <&gpio>;
interrupts = <12 IRQ_TYPE_LEVEL_LOW>, /* GPIO12 - ALERT pin */
<13 IRQ_TYPE_LEVEL_LOW>; /* GPIO13 - CRIT pin */
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* Local temperature sensor (SA56004ED internal) */
+ channel@0 {
+ reg = <0>;
+ label = "board";
+ };
+
+ /* Remote temperature sensor (D+/D- connected to P2020 CPU Temperature Diode) */
+ channel@1 {
+ reg = <1>;
+ label = "cpu";
+ };
};
/* DDR3 SPD/EEPROM */
};
power-leds {
- compatible = "gpio-leds";
+ compatible = "warp-power-leds";
green {
gpios = <&GPIO1 0 0>;
- default-state = "keep";
};
red {
gpios = <&GPIO1 1 0>;
- default-state = "keep";
};
};
}'
}
+ld_is_lld()
+{
+ ${CROSS}ld -V 2>&1 | grep -q LLD
+}
+
# Do not include PT_INTERP segment when linking pie. Non-pie linking
# just ignores this option.
LD_VERSION=$(${CROSS}ld --version | ld_version)
nodl="--no-dynamic-linker"
fi
+# suppress some warnings in recent ld versions
+nowarn="-z noexecstack"
+if ! ld_is_lld; then
+ if [ "$LD_VERSION" -ge "$(echo 2.39 | ld_version)" ]; then
+ nowarn="$nowarn --no-warn-rwx-segments"
+ fi
+fi
+
platformo=$object/"$platform".o
lds=$object/zImage.lds
ext=strip
text_start="-Ttext $link_address"
fi
#link everything
- ${CROSS}ld -m $format -T $lds $text_start $pie $nodl $rodynamic $notext -o "$ofile" $map \
+ ${CROSS}ld -m $format -T $lds $text_start $pie $nodl $nowarn $rodynamic $notext -o "$ofile" $map \
$platformo $tmp $object/wrapper.a
rm $tmp
fi
# reached, then enter the system reset vector of the partially decompressed
# image. No warning is issued.
rm -f "$odir"/{otheros,otheros-too-big}.bld
- size=$(${CROSS}nm --no-sort --radix=d "$ofile" | egrep ' _end$' | cut -d' ' -f1)
+ size=$(${CROSS}nm --no-sort --radix=d "$ofile" | grep -E ' _end$' | cut -d' ' -f1)
bld="otheros.bld"
if [ $size -gt $((0x1000000)) ]; then
bld="otheros-too-big.bld"
CONFIG_USB_FTDI_ELAN=m
CONFIG_USB_APPLEDISPLAY=m
CONFIG_USB_SISUSBVGA=m
-CONFIG_USB_SISUSBVGA_CON=y
CONFIG_USB_LD=m
CONFIG_USB_TRANCEVIBRATOR=m
CONFIG_USB_IOWARRIOR=m
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_POWERPC_ASM_H
+#define _ASM_POWERPC_ASM_H
+
+#define _ASM_PTR " .long "
+
+#endif /* _ASM_POWERPC_ASM_H */
#ifndef _ASM_POWERPC_BOOK3S_32_TLBFLUSH_H
#define _ASM_POWERPC_BOOK3S_32_TLBFLUSH_H
+#include <linux/build_bug.h>
+
#define MMU_NO_CONTEXT (0)
/*
* TLB flushing for "classic" hash-MMU 32-bit CPUs, 6xx, 7xx, 7xxx
{
flush_tlb_page(vma, vmaddr);
}
+
+static inline void local_flush_tlb_page_psize(struct mm_struct *mm,
+ unsigned long vmaddr, int psize)
+{
+ BUILD_BUG();
+}
+
static inline void local_flush_tlb_mm(struct mm_struct *mm)
{
flush_tlb_mm(mm);
extern void flush_hash_hugepage(unsigned long vsid, unsigned long addr,
pmd_t *pmdp, unsigned int psize, int ssize,
unsigned long flags);
-static inline void hash__local_flush_tlb_mm(struct mm_struct *mm)
-{
-}
-
-static inline void hash__flush_tlb_mm(struct mm_struct *mm)
-{
-}
-
-static inline void hash__local_flush_all_mm(struct mm_struct *mm)
-{
- /*
- * There's no Page Walk Cache for hash, so what is needed is
- * the same as flush_tlb_mm(), which doesn't really make sense
- * with hash. So the only thing we could do is flush the
- * entire LPID! Punt for now, as it's not being used.
- */
- WARN_ON_ONCE(1);
-}
-
-static inline void hash__flush_all_mm(struct mm_struct *mm)
-{
- /*
- * There's no Page Walk Cache for hash, so what is needed is
- * the same as flush_tlb_mm(), which doesn't really make sense
- * with hash. So the only thing we could do is flush the
- * entire LPID! Punt for now, as it's not being used.
- */
- WARN_ON_ONCE(1);
-}
-
-static inline void hash__local_flush_tlb_page(struct vm_area_struct *vma,
- unsigned long vmaddr)
-{
-}
-
-static inline void hash__flush_tlb_page(struct vm_area_struct *vma,
- unsigned long vmaddr)
-{
-}
-
-static inline void hash__flush_tlb_range(struct vm_area_struct *vma,
- unsigned long start, unsigned long end)
-{
-}
-
-static inline void hash__flush_tlb_kernel_range(unsigned long start,
- unsigned long end)
-{
-}
-
struct mmu_gather;
extern void hash__tlb_flush(struct mmu_gather *tlb);
unsigned long start, unsigned long end)
{
if (radix_enabled())
- return radix__flush_pmd_tlb_range(vma, start, end);
- return hash__flush_tlb_range(vma, start, end);
+ radix__flush_pmd_tlb_range(vma, start, end);
}
#define __HAVE_ARCH_FLUSH_HUGETLB_TLB_RANGE
unsigned long end)
{
if (radix_enabled())
- return radix__flush_hugetlb_tlb_range(vma, start, end);
- return hash__flush_tlb_range(vma, start, end);
+ radix__flush_hugetlb_tlb_range(vma, start, end);
}
static inline void flush_tlb_range(struct vm_area_struct *vma,
unsigned long start, unsigned long end)
{
if (radix_enabled())
- return radix__flush_tlb_range(vma, start, end);
- return hash__flush_tlb_range(vma, start, end);
+ radix__flush_tlb_range(vma, start, end);
}
static inline void flush_tlb_kernel_range(unsigned long start,
unsigned long end)
{
if (radix_enabled())
- return radix__flush_tlb_kernel_range(start, end);
- return hash__flush_tlb_kernel_range(start, end);
+ radix__flush_tlb_kernel_range(start, end);
}
static inline void local_flush_tlb_mm(struct mm_struct *mm)
{
if (radix_enabled())
- return radix__local_flush_tlb_mm(mm);
- return hash__local_flush_tlb_mm(mm);
+ radix__local_flush_tlb_mm(mm);
}
static inline void local_flush_tlb_page(struct vm_area_struct *vma,
unsigned long vmaddr)
{
if (radix_enabled())
- return radix__local_flush_tlb_page(vma, vmaddr);
- return hash__local_flush_tlb_page(vma, vmaddr);
+ radix__local_flush_tlb_page(vma, vmaddr);
}
-static inline void local_flush_all_mm(struct mm_struct *mm)
+static inline void local_flush_tlb_page_psize(struct mm_struct *mm,
+ unsigned long vmaddr, int psize)
{
if (radix_enabled())
- return radix__local_flush_all_mm(mm);
- return hash__local_flush_all_mm(mm);
+ radix__local_flush_tlb_page_psize(mm, vmaddr, psize);
}
static inline void tlb_flush(struct mmu_gather *tlb)
{
if (radix_enabled())
- return radix__tlb_flush(tlb);
- return hash__tlb_flush(tlb);
+ radix__tlb_flush(tlb);
}
#ifdef CONFIG_SMP
static inline void flush_tlb_mm(struct mm_struct *mm)
{
if (radix_enabled())
- return radix__flush_tlb_mm(mm);
- return hash__flush_tlb_mm(mm);
+ radix__flush_tlb_mm(mm);
}
static inline void flush_tlb_page(struct vm_area_struct *vma,
unsigned long vmaddr)
{
if (radix_enabled())
- return radix__flush_tlb_page(vma, vmaddr);
- return hash__flush_tlb_page(vma, vmaddr);
-}
-
-static inline void flush_all_mm(struct mm_struct *mm)
-{
- if (radix_enabled())
- return radix__flush_all_mm(mm);
- return hash__flush_all_mm(mm);
+ radix__flush_tlb_page(vma, vmaddr);
}
#else
#define flush_tlb_mm(mm) local_flush_tlb_mm(mm)
#define flush_tlb_page(vma, addr) local_flush_tlb_page(vma, addr)
-#define flush_all_mm(mm) local_flush_all_mm(mm)
#endif /* CONFIG_SMP */
#define flush_tlb_fix_spurious_fault flush_tlb_fix_spurious_fault
__label__ __label_warn_on; \
\
WARN_ENTRY("twi 31, 0, 0", BUGFLAG_WARNING | (flags), __label_warn_on); \
- unreachable(); \
+ barrier_before_unreachable(); \
+ __builtin_unreachable(); \
\
__label_warn_on: \
break; \
* the previous value stored there.
*/
+#ifndef CONFIG_PPC_HAS_LBARX_LHARX
XCHG_GEN(u8, _local, "memory");
XCHG_GEN(u8, _relaxed, "cc");
XCHG_GEN(u16, _local, "memory");
XCHG_GEN(u16, _relaxed, "cc");
+#else
+static __always_inline unsigned long
+__xchg_u8_local(volatile void *p, unsigned long val)
+{
+ unsigned long prev;
+
+ __asm__ __volatile__(
+"1: lbarx %0,0,%2 # __xchg_u8_local\n"
+" stbcx. %3,0,%2 \n"
+" bne- 1b"
+ : "=&r" (prev), "+m" (*(volatile unsigned char *)p)
+ : "r" (p), "r" (val)
+ : "cc", "memory");
+
+ return prev;
+}
+
+static __always_inline unsigned long
+__xchg_u8_relaxed(u8 *p, unsigned long val)
+{
+ unsigned long prev;
+
+ __asm__ __volatile__(
+"1: lbarx %0,0,%2 # __xchg_u8_relaxed\n"
+" stbcx. %3,0,%2\n"
+" bne- 1b"
+ : "=&r" (prev), "+m" (*p)
+ : "r" (p), "r" (val)
+ : "cc");
+
+ return prev;
+}
+
+static __always_inline unsigned long
+__xchg_u16_local(volatile void *p, unsigned long val)
+{
+ unsigned long prev;
+
+ __asm__ __volatile__(
+"1: lharx %0,0,%2 # __xchg_u16_local\n"
+" sthcx. %3,0,%2\n"
+" bne- 1b"
+ : "=&r" (prev), "+m" (*(volatile unsigned short *)p)
+ : "r" (p), "r" (val)
+ : "cc", "memory");
+
+ return prev;
+}
+
+static __always_inline unsigned long
+__xchg_u16_relaxed(u16 *p, unsigned long val)
+{
+ unsigned long prev;
+
+ __asm__ __volatile__(
+"1: lharx %0,0,%2 # __xchg_u16_relaxed\n"
+" sthcx. %3,0,%2\n"
+" bne- 1b"
+ : "=&r" (prev), "+m" (*p)
+ : "r" (p), "r" (val)
+ : "cc");
+
+ return prev;
+}
+#endif
static __always_inline unsigned long
__xchg_u32_local(volatile void *p, unsigned long val)
(__typeof__(*(ptr))) __xchg_relaxed((ptr), \
(unsigned long)_x_, sizeof(*(ptr))); \
})
+
/*
* Compare and exchange - if *p == old, set it to new,
* and return the old value of *p.
*/
-
+#ifndef CONFIG_PPC_HAS_LBARX_LHARX
CMPXCHG_GEN(u8, , PPC_ATOMIC_ENTRY_BARRIER, PPC_ATOMIC_EXIT_BARRIER, "memory");
CMPXCHG_GEN(u8, _local, , , "memory");
CMPXCHG_GEN(u8, _acquire, , PPC_ACQUIRE_BARRIER, "memory");
CMPXCHG_GEN(u16, _local, , , "memory");
CMPXCHG_GEN(u16, _acquire, , PPC_ACQUIRE_BARRIER, "memory");
CMPXCHG_GEN(u16, _relaxed, , , "cc");
+#else
+static __always_inline unsigned long
+__cmpxchg_u8(volatile unsigned char *p, unsigned long old, unsigned long new)
+{
+ unsigned int prev;
+
+ __asm__ __volatile__ (
+ PPC_ATOMIC_ENTRY_BARRIER
+"1: lbarx %0,0,%2 # __cmpxchg_u8\n"
+" cmpw 0,%0,%3\n"
+" bne- 2f\n"
+" stbcx. %4,0,%2\n"
+" bne- 1b"
+ PPC_ATOMIC_EXIT_BARRIER
+ "\n\
+2:"
+ : "=&r" (prev), "+m" (*p)
+ : "r" (p), "r" (old), "r" (new)
+ : "cc", "memory");
+
+ return prev;
+}
+
+static __always_inline unsigned long
+__cmpxchg_u8_local(volatile unsigned char *p, unsigned long old,
+ unsigned long new)
+{
+ unsigned int prev;
+
+ __asm__ __volatile__ (
+"1: lbarx %0,0,%2 # __cmpxchg_u8_local\n"
+" cmpw 0,%0,%3\n"
+" bne- 2f\n"
+" stbcx. %4,0,%2\n"
+" bne- 1b\n"
+"2:"
+ : "=&r" (prev), "+m" (*p)
+ : "r" (p), "r" (old), "r" (new)
+ : "cc", "memory");
+
+ return prev;
+}
+
+static __always_inline unsigned long
+__cmpxchg_u8_relaxed(u8 *p, unsigned long old, unsigned long new)
+{
+ unsigned long prev;
+
+ __asm__ __volatile__ (
+"1: lbarx %0,0,%2 # __cmpxchg_u8_relaxed\n"
+" cmpw 0,%0,%3\n"
+" bne- 2f\n"
+" stbcx. %4,0,%2\n"
+" bne- 1b\n"
+"2:"
+ : "=&r" (prev), "+m" (*p)
+ : "r" (p), "r" (old), "r" (new)
+ : "cc");
+
+ return prev;
+}
+
+static __always_inline unsigned long
+__cmpxchg_u8_acquire(u8 *p, unsigned long old, unsigned long new)
+{
+ unsigned long prev;
+
+ __asm__ __volatile__ (
+"1: lbarx %0,0,%2 # __cmpxchg_u8_acquire\n"
+" cmpw 0,%0,%3\n"
+" bne- 2f\n"
+" stbcx. %4,0,%2\n"
+" bne- 1b\n"
+ PPC_ACQUIRE_BARRIER
+"2:"
+ : "=&r" (prev), "+m" (*p)
+ : "r" (p), "r" (old), "r" (new)
+ : "cc", "memory");
+
+ return prev;
+}
+
+static __always_inline unsigned long
+__cmpxchg_u16(volatile unsigned short *p, unsigned long old, unsigned long new)
+{
+ unsigned int prev;
+
+ __asm__ __volatile__ (
+ PPC_ATOMIC_ENTRY_BARRIER
+"1: lharx %0,0,%2 # __cmpxchg_u16\n"
+" cmpw 0,%0,%3\n"
+" bne- 2f\n"
+" sthcx. %4,0,%2\n"
+" bne- 1b\n"
+ PPC_ATOMIC_EXIT_BARRIER
+"2:"
+ : "=&r" (prev), "+m" (*p)
+ : "r" (p), "r" (old), "r" (new)
+ : "cc", "memory");
+
+ return prev;
+}
+
+static __always_inline unsigned long
+__cmpxchg_u16_local(volatile unsigned short *p, unsigned long old,
+ unsigned long new)
+{
+ unsigned int prev;
+
+ __asm__ __volatile__ (
+"1: lharx %0,0,%2 # __cmpxchg_u16_local\n"
+" cmpw 0,%0,%3\n"
+" bne- 2f\n"
+" sthcx. %4,0,%2\n"
+" bne- 1b"
+"2:"
+ : "=&r" (prev), "+m" (*p)
+ : "r" (p), "r" (old), "r" (new)
+ : "cc", "memory");
+
+ return prev;
+}
+
+static __always_inline unsigned long
+__cmpxchg_u16_relaxed(u16 *p, unsigned long old, unsigned long new)
+{
+ unsigned long prev;
+
+ __asm__ __volatile__ (
+"1: lharx %0,0,%2 # __cmpxchg_u16_relaxed\n"
+" cmpw 0,%0,%3\n"
+" bne- 2f\n"
+" sthcx. %4,0,%2\n"
+" bne- 1b\n"
+"2:"
+ : "=&r" (prev), "+m" (*p)
+ : "r" (p), "r" (old), "r" (new)
+ : "cc");
+
+ return prev;
+}
+
+static __always_inline unsigned long
+__cmpxchg_u16_acquire(u16 *p, unsigned long old, unsigned long new)
+{
+ unsigned long prev;
+
+ __asm__ __volatile__ (
+"1: lharx %0,0,%2 # __cmpxchg_u16_acquire\n"
+" cmpw 0,%0,%3\n"
+" bne- 2f\n"
+" sthcx. %4,0,%2\n"
+" bne- 1b\n"
+ PPC_ACQUIRE_BARRIER
+"2:"
+ : "=&r" (prev), "+m" (*p)
+ : "r" (p), "r" (old), "r" (new)
+ : "cc", "memory");
+
+ return prev;
+}
+#endif
static __always_inline unsigned long
__cmpxchg_u32(volatile unsigned int *p, unsigned long old, unsigned long new)
#define BRANCH_SET_LINK 0x1
#define BRANCH_ABSOLUTE 0x2
-DECLARE_STATIC_KEY_FALSE(init_mem_is_free);
-
/*
* Powerpc branch instruction is :
*
#include <asm/param.h>
#include <asm/firmware.h>
-typedef u64 __nocast cputime_t;
-typedef u64 __nocast cputime64_t;
-
-#define cmpxchg_cputime(ptr, old, new) cmpxchg(ptr, old, new)
-
#ifdef __KERNEL__
-/*
- * Convert cputime <-> microseconds
- */
-extern u64 __cputime_usec_factor;
-
-static inline unsigned long cputime_to_usecs(const cputime_t ct)
-{
- return mulhdu((__force u64) ct, __cputime_usec_factor);
-}
-
-#define cputime_to_nsecs(cputime) tb_to_ns((__force u64)cputime)
+#define cputime_to_nsecs(cputime) tb_to_ns(cputime)
/*
* PPC64 uses PACA which is task independent for storing accounting data while
#endif
void __set_breakpoint(int nr, struct arch_hw_breakpoint *brk);
+void suspend_breakpoints(void);
+void restore_breakpoints(void);
bool ppc_breakpoint_available(void);
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
extern void do_send_trap(struct pt_regs *regs, unsigned long address,
#define HAVE_FUNCTION_GRAPH_RET_ADDR_PTR
+/* Ignore unused weak functions which will have larger offsets */
+#ifdef CONFIG_MPROFILE_KERNEL
+#define FTRACE_MCOUNT_MAX_OFFSET 12
+#elif defined(CONFIG_PPC32)
+#define FTRACE_MCOUNT_MAX_OFFSET 8
+#endif
+
#ifndef __ASSEMBLY__
extern void _mcount(void);
* those.
*/
#define ARCH_HAS_SYSCALL_MATCH_SYM_NAME
-#ifdef CONFIG_PPC64_ELF_ABI_V1
-static inline bool arch_syscall_match_sym_name(const char *sym, const char *name)
-{
- /* We need to skip past the initial dot, and the __se_sys alias */
- return !strcmp(sym + 1, name) ||
- (!strncmp(sym, ".__se_sys", 9) && !strcmp(sym + 6, name)) ||
- (!strncmp(sym, ".ppc_", 5) && !strcmp(sym + 5, name + 4)) ||
- (!strncmp(sym, ".ppc32_", 7) && !strcmp(sym + 7, name + 4)) ||
- (!strncmp(sym, ".ppc64_", 7) && !strcmp(sym + 7, name + 4));
-}
-#else
static inline bool arch_syscall_match_sym_name(const char *sym, const char *name)
{
return !strcmp(sym, name) ||
(!strncmp(sym, "ppc32_", 6) && !strcmp(sym + 6, name + 4)) ||
(!strncmp(sym, "ppc64_", 6) && !strcmp(sym + 6, name + 4));
}
-#endif /* CONFIG_PPC64_ELF_ABI_V1 */
#endif /* CONFIG_FTRACE_SYSCALLS */
#if defined(CONFIG_PPC64) && defined(CONFIG_FUNCTION_TRACER)
#define H_NOT_ENOUGH_RESOURCES -44
#define H_R_STATE -45
#define H_RESCINDED -46
-#define H_P1 -54
+#define H_ABORTED -54
#define H_P2 -55
#define H_P3 -56
#define H_P4 -57
#define H_COP_HW -74
#define H_STATE -75
#define H_IN_USE -77
-#define H_ABORTED -78
#define H_UNSUPPORTED_FLAG_START -256
#define H_UNSUPPORTED_FLAG_END -511
#define H_MULTI_THREADS_ACTIVE -9005
*/
#include <asm/hw_irq.h>
-#else
-#ifdef CONFIG_TRACE_IRQFLAGS
-#ifdef CONFIG_IRQSOFF_TRACER
-/*
- * Since the ftrace irqsoff latency trace checks CALLER_ADDR1,
- * which is the stack frame here, we need to force a stack frame
- * in case we came from user space.
- */
-#define TRACE_WITH_FRAME_BUFFER(func) \
- mflr r0; \
- stdu r1, -STACK_FRAME_OVERHEAD(r1); \
- std r0, 16(r1); \
- stdu r1, -STACK_FRAME_OVERHEAD(r1); \
- bl func; \
- ld r1, 0(r1); \
- ld r1, 0(r1);
-#else
-#define TRACE_WITH_FRAME_BUFFER(func) \
- bl func;
-#endif
-
-/*
- * These are calls to C code, so the caller must be prepared for volatiles to
- * be clobbered.
- */
-#define TRACE_ENABLE_INTS TRACE_WITH_FRAME_BUFFER(trace_hardirqs_on)
-#define TRACE_DISABLE_INTS TRACE_WITH_FRAME_BUFFER(trace_hardirqs_off)
-
-/*
- * This is used by assembly code to soft-disable interrupts first and
- * reconcile irq state.
- *
- * NB: This may call C code, so the caller must be prepared for volatiles to
- * be clobbered.
- */
-#define RECONCILE_IRQ_STATE(__rA, __rB) \
- lbz __rA,PACAIRQSOFTMASK(r13); \
- lbz __rB,PACAIRQHAPPENED(r13); \
- andi. __rA,__rA,IRQS_DISABLED; \
- li __rA,IRQS_DISABLED; \
- ori __rB,__rB,PACA_IRQ_HARD_DIS; \
- stb __rB,PACAIRQHAPPENED(r13); \
- bne 44f; \
- stb __rA,PACAIRQSOFTMASK(r13); \
- TRACE_DISABLE_INTS; \
-44:
-
-#else
-#define TRACE_ENABLE_INTS
-#define TRACE_DISABLE_INTS
-
-#define RECONCILE_IRQ_STATE(__rA, __rB) \
- lbz __rA,PACAIRQHAPPENED(r13); \
- li __rB,IRQS_DISABLED; \
- ori __rA,__rA,PACA_IRQ_HARD_DIS; \
- stb __rB,PACAIRQSOFTMASK(r13); \
- stb __rA,PACAIRQHAPPENED(r13)
-#endif
#endif
#endif
void __iomem *xive_tima_virt;
u32 saved_xirr;
u64 dabr;
- u64 host_mmcr[10]; /* MMCR 0,1,A, SIAR, SDAR, MMCR2, SIER, MMCR3, SIER2/3 */
+ u64 host_mmcr[7]; /* MMCR 0,1,A, SIAR, SDAR, MMCR2, SIER */
u32 host_pmc[8];
u64 host_purr;
u64 host_spurr;
#endif
}
+static inline void kvmppc_fix_ee_after_exit(void)
+{
+#ifdef CONFIG_PPC64
+ /* Only need to enable IRQs by hard enabling them after this */
+ local_paca->irq_happened = PACA_IRQ_HARD_DIS;
+ irq_soft_mask_set(IRQS_ALL_DISABLED);
+#endif
+
+ trace_hardirqs_off();
+}
+
+
static inline ulong kvmppc_get_ea_indexed(struct kvm_vcpu *vcpu, int ra, int rb)
{
ulong ea;
#include <asm/types.h>
+#define __ALIGN .align 2
+#define __ALIGN_STR ".align 2"
+
#ifdef CONFIG_PPC64_ELF_ABI_V1
#define cond_syscall(x) \
asm ("\t.weak " #x "\n\t.set " #x ", sys_ni_syscall\n" \
* nMMU and/or PSL need to be cleaned up.
*
* Both the 'copros' and 'active_cpus' counts are looked at in
- * flush_all_mm() to determine the scope (local/global) of the
- * TLBIs, so we need to flush first before decrementing
+ * radix__flush_all_mm() to determine the scope (local/global)
+ * of the TLBIs, so we need to flush first before decrementing
* 'copros'. If this API is used by several callers for the
* same context, it can lead to over-flushing. It's hopefully
* not common enough to be a problem.
* in-between.
*/
if (radix_enabled()) {
- flush_all_mm(mm);
+ radix__flush_all_mm(mm);
c = atomic_dec_if_positive(&mm->context.copros);
/* Detect imbalance between add and remove */
num = number_of_cells_per_pte(pmd, new, huge);
- for (i = 0; i < num; i++, entry++, new += SZ_4K)
- *entry = new;
+ for (i = 0; i < num; i += PAGE_SIZE / SZ_4K, new += PAGE_SIZE) {
+ *entry++ = new;
+ if (IS_ENABLED(CONFIG_PPC_16K_PAGES) && num != 1) {
+ *entry++ = new;
+ *entry++ = new;
+ *entry++ = new;
+ }
+ }
return old;
}
#ifdef CONFIG_PPC_16K_PAGES
-#define __HAVE_ARCH_PTEP_GET
+#define ptep_get ptep_get
static inline pte_t ptep_get(pte_t *ptep)
{
pte_basic_t val = READ_ONCE(ptep->pte);
* cases, and 32-bit non-hash with 32-bit PTEs.
*/
#if defined(CONFIG_PPC_8xx) && defined(CONFIG_PPC_16K_PAGES)
- ptep->pte = ptep->pte1 = ptep->pte2 = ptep->pte3 = pte_val(pte);
+ ptep->pte3 = ptep->pte2 = ptep->pte1 = ptep->pte = pte_val(pte);
#else
*ptep = pte;
#endif
asm volatile ("tlbie %0; sync" : : "r" (vmaddr) : "memory");
}
+static inline void local_flush_tlb_page_psize(struct mm_struct *mm,
+ unsigned long vmaddr, int psize)
+{
+ asm volatile ("tlbie %0; sync" : : "r" (vmaddr) : "memory");
+}
+
static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end)
{
start &= PAGE_MASK;
extern void flush_tlb_kernel_range(unsigned long start, unsigned long end);
extern void local_flush_tlb_mm(struct mm_struct *mm);
extern void local_flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr);
+void local_flush_tlb_page_psize(struct mm_struct *mm, unsigned long vmaddr, int psize);
extern void __local_flush_tlb_page(struct mm_struct *mm, unsigned long vmaddr,
int tsize, int ind);
#define SAVE_GPR(n, base) SAVE_GPRS(n, n, base)
#define REST_GPR(n, base) REST_GPRS(n, n, base)
+/* macros for handling user register sanitisation */
+#ifdef CONFIG_INTERRUPT_SANITIZE_REGISTERS
+#define SANITIZE_SYSCALL_GPRS() ZEROIZE_GPR(0); \
+ ZEROIZE_GPRS(5, 12); \
+ ZEROIZE_NVGPRS()
+#define SANITIZE_GPR(n) ZEROIZE_GPR(n)
+#define SANITIZE_GPRS(start, end) ZEROIZE_GPRS(start, end)
+#define SANITIZE_NVGPRS() ZEROIZE_NVGPRS()
+#define SANITIZE_RESTORE_NVGPRS() REST_NVGPRS(r1)
+#define HANDLER_RESTORE_NVGPRS()
+#else
+#define SANITIZE_SYSCALL_GPRS()
+#define SANITIZE_GPR(n)
+#define SANITIZE_GPRS(start, end)
+#define SANITIZE_NVGPRS()
+#define SANITIZE_RESTORE_NVGPRS()
+#define HANDLER_RESTORE_NVGPRS() REST_NVGPRS(r1)
+#endif /* CONFIG_INTERRUPT_SANITIZE_REGISTERS */
+
#define SAVE_FPR(n, base) stfd n,8*TS_FPRWIDTH*(n)(base)
#define SAVE_2FPRS(n, base) SAVE_FPR(n, base); SAVE_FPR(n+1, base)
#define SAVE_4FPRS(n, base) SAVE_2FPRS(n, base); SAVE_2FPRS(n+2, base)
#endif
-/* Check that a certain kernel stack pointer is valid in task_struct p */
-int validate_sp(unsigned long sp, struct task_struct *p,
- unsigned long nbytes);
+/*
+ * Check that a certain kernel stack pointer is a valid (minimum sized)
+ * stack frame in task_struct p.
+ */
+int validate_sp(unsigned long sp, struct task_struct *p);
+
+/*
+ * validate the stack frame of a particular minimum size, used for when we are
+ * looking at a certain object in the stack beyond the minimum.
+ */
+int validate_sp_size(unsigned long sp, struct task_struct *p,
+ unsigned long nbytes);
/*
* Prefetch macros.
extern int of_read_drc_info_cell(struct property **prop,
const __be32 **curval, struct of_drc_info *data);
+extern unsigned int boot_cpu_node_count;
/*
* There are two methods for telling firmware what our capabilities are.
return dev_get_drvdata(&dev->core);
}
-/* These two need global scope for get_arch_dma_ops(). */
-
-extern struct bus_type ps3_system_bus_type;
-
/* system manager */
struct ps3_sys_manager_ops {
return pa;
}
-/*
- * This is what we should always use. Any other lockless page table lookup needs
- * careful audit against THP split.
- */
-static inline pte_t *find_current_mm_pte(pgd_t *pgdir, unsigned long ea,
- bool *is_thp, unsigned *hshift)
-{
- pte_t *pte;
-
- VM_WARN(!arch_irqs_disabled(), "%s called with irq enabled\n", __func__);
- VM_WARN(pgdir != current->mm->pgd,
- "%s lock less page table lookup called on wrong mm\n", __func__);
- pte = __find_linux_pte(pgdir, ea, is_thp, hshift);
-
-#if defined(CONFIG_DEBUG_VM) && \
- !(defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE))
- /*
- * We should not find huge page if these configs are not enabled.
- */
- if (hshift)
- WARN_ON(*hshift);
-#endif
- return pte;
-}
-
#endif /* _ASM_POWERPC_PTE_WALK_H */
#endif
-#define STACK_FRAME_WITH_PT_REGS (STACK_FRAME_OVERHEAD + sizeof(struct pt_regs))
-
// Always displays as "REGS" in memory dumps
#ifdef CONFIG_CPU_BIG_ENDIAN
#define STACK_FRAME_REGS_MARKER ASM_CONST(0x52454753)
#define USER_REDZONE_SIZE 512
#define KERNEL_REDZONE_SIZE 288
-#define STACK_FRAME_OVERHEAD 112 /* size of minimum stack frame */
#define STACK_FRAME_LR_SAVE 2 /* Location of LR in stack frame */
-#define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + \
- STACK_FRAME_OVERHEAD + KERNEL_REDZONE_SIZE)
-#define STACK_FRAME_MARKER 12
#ifdef CONFIG_PPC64_ELF_ABI_V2
#define STACK_FRAME_MIN_SIZE 32
+#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE + 16)
+#define STACK_INT_FRAME_REGS (STACK_FRAME_MIN_SIZE + 16)
+#define STACK_INT_FRAME_MARKER STACK_FRAME_MIN_SIZE
+#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE + 16)
+#define STACK_SWITCH_FRAME_REGS (STACK_FRAME_MIN_SIZE + 16)
#else
-#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD
+/*
+ * The ELFv1 ABI specifies 48 bytes plus a minimum 64 byte parameter save
+ * area. This parameter area is not used by calls to C from interrupt entry,
+ * so the second from last one of those is used for the frame marker.
+ */
+#define STACK_FRAME_MIN_SIZE 112
+#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE)
+#define STACK_INT_FRAME_REGS STACK_FRAME_MIN_SIZE
+#define STACK_INT_FRAME_MARKER (STACK_FRAME_MIN_SIZE - 16)
+#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE)
+#define STACK_SWITCH_FRAME_REGS STACK_FRAME_MIN_SIZE
#endif
/* Size of dummy stack frame allocated when calling signal handler. */
#define USER_REDZONE_SIZE 0
#define KERNEL_REDZONE_SIZE 0
-#define STACK_FRAME_OVERHEAD 16 /* size of minimum stack frame */
+#define STACK_FRAME_MIN_SIZE 16
#define STACK_FRAME_LR_SAVE 1 /* Location of LR in stack frame */
-#define STACK_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_OVERHEAD)
-#define STACK_FRAME_MARKER 2
-#define STACK_FRAME_MIN_SIZE STACK_FRAME_OVERHEAD
+#define STACK_USER_INT_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE)
+#define STACK_INT_FRAME_REGS STACK_FRAME_MIN_SIZE
+#define STACK_INT_FRAME_MARKER (STACK_FRAME_MIN_SIZE - 8)
+#define STACK_SWITCH_FRAME_SIZE (sizeof(struct pt_regs) + STACK_FRAME_MIN_SIZE)
+#define STACK_SWITCH_FRAME_REGS STACK_FRAME_MIN_SIZE
/* Size of stack frame allocated when calling signal handler. */
#define __SIGNAL_FRAMESIZE 64
#endif /* __powerpc64__ */
+#define STACK_INT_FRAME_SIZE (KERNEL_REDZONE_SIZE + STACK_USER_INT_FRAME_SIZE)
+#define STACK_INT_FRAME_MARKER_LONGS (STACK_INT_FRAME_MARKER/sizeof(long))
+
#ifndef __ASSEMBLY__
#include <asm/paca.h>
#ifndef _ASM_POWERPC_QSPINLOCK_H
#define _ASM_POWERPC_QSPINLOCK_H
-#include <asm-generic/qspinlock_types.h>
+#include <linux/compiler.h>
+#include <asm/qspinlock_types.h>
#include <asm/paravirt.h>
-#define _Q_PENDING_LOOPS (1 << 9) /* not tuned */
+#ifdef CONFIG_PPC64
+/*
+ * Use the EH=1 hint for accesses that result in the lock being acquired.
+ * The hardware is supposed to optimise this pattern by holding the lock
+ * cacheline longer, and releasing when a store to the same memory (the
+ * unlock) is performed.
+ */
+#define _Q_SPIN_EH_HINT 1
+#else
+#define _Q_SPIN_EH_HINT 0
+#endif
-#ifdef CONFIG_PARAVIRT_SPINLOCKS
-extern void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
-extern void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
-extern void __pv_queued_spin_unlock(struct qspinlock *lock);
+/*
+ * The trylock itself may steal. This makes trylocks slightly stronger, and
+ * makes locks slightly more efficient when stealing.
+ *
+ * This is compile-time, so if true then there may always be stealers, so the
+ * nosteal paths become unused.
+ */
+#define _Q_SPIN_TRY_LOCK_STEAL 1
-static __always_inline void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
-{
- if (!is_shared_processor())
- native_queued_spin_lock_slowpath(lock, val);
- else
- __pv_queued_spin_lock_slowpath(lock, val);
-}
+/*
+ * Put a speculation barrier after testing the lock/node and finding it
+ * busy. Try to prevent pointless speculation in slow paths.
+ *
+ * Slows down the lockstorm microbenchmark with no stealing, where locking
+ * is purely FIFO through the queue. May have more benefit in real workload
+ * where speculating into the wrong place could have a greater cost.
+ */
+#define _Q_SPIN_SPEC_BARRIER 0
-#define queued_spin_unlock queued_spin_unlock
-static inline void queued_spin_unlock(struct qspinlock *lock)
-{
- if (!is_shared_processor())
- smp_store_release(&lock->locked, 0);
- else
- __pv_queued_spin_unlock(lock);
-}
+#ifdef CONFIG_PPC64
+/*
+ * Execute a miso instruction after passing the MCS lock ownership to the
+ * queue head. Miso is intended to make stores visible to other CPUs sooner.
+ *
+ * This seems to make the lockstorm microbenchmark nospin test go slightly
+ * faster on POWER10, but disable for now.
+ */
+#define _Q_SPIN_MISO 0
+#else
+#define _Q_SPIN_MISO 0
+#endif
+#ifdef CONFIG_PPC64
+/*
+ * This executes miso after an unlock of the lock word, having ownership
+ * pass to the next CPU sooner. This will slow the uncontended path to some
+ * degree. Not evidence it helps yet.
+ */
+#define _Q_SPIN_MISO_UNLOCK 0
#else
-extern void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
+#define _Q_SPIN_MISO_UNLOCK 0
#endif
-static __always_inline void queued_spin_lock(struct qspinlock *lock)
+/*
+ * Seems to slow down lockstorm microbenchmark, suspect queue node just
+ * has to become shared again right afterwards when its waiter spins on
+ * the lock field.
+ */
+#define _Q_SPIN_PREFETCH_NEXT 0
+
+static __always_inline int queued_spin_is_locked(struct qspinlock *lock)
{
- u32 val = 0;
+ return READ_ONCE(lock->val);
+}
- if (likely(arch_atomic_try_cmpxchg_lock(&lock->val, &val, _Q_LOCKED_VAL)))
- return;
+static __always_inline int queued_spin_value_unlocked(struct qspinlock lock)
+{
+ return !lock.val;
+}
- queued_spin_lock_slowpath(lock, val);
+static __always_inline int queued_spin_is_contended(struct qspinlock *lock)
+{
+ return !!(READ_ONCE(lock->val) & _Q_TAIL_CPU_MASK);
}
-#define queued_spin_lock queued_spin_lock
-#ifdef CONFIG_PARAVIRT_SPINLOCKS
-#define SPIN_THRESHOLD (1<<15) /* not tuned */
+static __always_inline u32 queued_spin_encode_locked_val(void)
+{
+ /* XXX: make this use lock value in paca like simple spinlocks? */
+ return _Q_LOCKED_VAL | (smp_processor_id() << _Q_OWNER_CPU_OFFSET);
+}
-static __always_inline void pv_wait(u8 *ptr, u8 val)
+static __always_inline int __queued_spin_trylock_nosteal(struct qspinlock *lock)
{
- if (*ptr != val)
- return;
- yield_to_any();
- /*
- * We could pass in a CPU here if waiting in the queue and yield to
- * the previous CPU in the queue.
- */
+ u32 new = queued_spin_encode_locked_val();
+ u32 prev;
+
+ /* Trylock succeeds only when unlocked and no queued nodes */
+ asm volatile(
+"1: lwarx %0,0,%1,%3 # __queued_spin_trylock_nosteal \n"
+" cmpwi 0,%0,0 \n"
+" bne- 2f \n"
+" stwcx. %2,0,%1 \n"
+" bne- 1b \n"
+"\t" PPC_ACQUIRE_BARRIER " \n"
+"2: \n"
+ : "=&r" (prev)
+ : "r" (&lock->val), "r" (new),
+ "i" (_Q_SPIN_EH_HINT)
+ : "cr0", "memory");
+
+ return likely(prev == 0);
}
-static __always_inline void pv_kick(int cpu)
+static __always_inline int __queued_spin_trylock_steal(struct qspinlock *lock)
{
- prod_cpu(cpu);
+ u32 new = queued_spin_encode_locked_val();
+ u32 prev, tmp;
+
+ /* Trylock may get ahead of queued nodes if it finds unlocked */
+ asm volatile(
+"1: lwarx %0,0,%2,%5 # __queued_spin_trylock_steal \n"
+" andc. %1,%0,%4 \n"
+" bne- 2f \n"
+" and %1,%0,%4 \n"
+" or %1,%1,%3 \n"
+" stwcx. %1,0,%2 \n"
+" bne- 1b \n"
+"\t" PPC_ACQUIRE_BARRIER " \n"
+"2: \n"
+ : "=&r" (prev), "=&r" (tmp)
+ : "r" (&lock->val), "r" (new), "r" (_Q_TAIL_CPU_MASK),
+ "i" (_Q_SPIN_EH_HINT)
+ : "cr0", "memory");
+
+ return likely(!(prev & ~_Q_TAIL_CPU_MASK));
}
-extern void __pv_init_lock_hash(void);
+static __always_inline int queued_spin_trylock(struct qspinlock *lock)
+{
+ if (!_Q_SPIN_TRY_LOCK_STEAL)
+ return __queued_spin_trylock_nosteal(lock);
+ else
+ return __queued_spin_trylock_steal(lock);
+}
-static inline void pv_spinlocks_init(void)
+void queued_spin_lock_slowpath(struct qspinlock *lock);
+
+static __always_inline void queued_spin_lock(struct qspinlock *lock)
{
- __pv_init_lock_hash();
+ if (!queued_spin_trylock(lock))
+ queued_spin_lock_slowpath(lock);
}
-#endif
+static inline void queued_spin_unlock(struct qspinlock *lock)
+{
+ smp_store_release(&lock->locked, 0);
+ if (_Q_SPIN_MISO_UNLOCK)
+ asm volatile("miso" ::: "memory");
+}
-/*
- * Queued spinlocks rely heavily on smp_cond_load_relaxed() to busy-wait,
- * which was found to have performance problems if implemented with
- * the preferred spin_begin()/spin_end() SMT priority pattern. Use the
- * generic version instead.
- */
+#define arch_spin_is_locked(l) queued_spin_is_locked(l)
+#define arch_spin_is_contended(l) queued_spin_is_contended(l)
+#define arch_spin_value_unlocked(l) queued_spin_value_unlocked(l)
+#define arch_spin_lock(l) queued_spin_lock(l)
+#define arch_spin_trylock(l) queued_spin_trylock(l)
+#define arch_spin_unlock(l) queued_spin_unlock(l)
-#include <asm-generic/qspinlock.h>
+#ifdef CONFIG_PARAVIRT_SPINLOCKS
+void pv_spinlocks_init(void);
+#else
+static inline void pv_spinlocks_init(void) { }
+#endif
#endif /* _ASM_POWERPC_QSPINLOCK_H */
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-#ifndef _ASM_POWERPC_QSPINLOCK_PARAVIRT_H
-#define _ASM_POWERPC_QSPINLOCK_PARAVIRT_H
-
-EXPORT_SYMBOL(__pv_queued_spin_unlock);
-
-#endif /* _ASM_POWERPC_QSPINLOCK_PARAVIRT_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef _ASM_POWERPC_QSPINLOCK_TYPES_H
+#define _ASM_POWERPC_QSPINLOCK_TYPES_H
+
+#include <linux/types.h>
+#include <asm/byteorder.h>
+
+typedef struct qspinlock {
+ union {
+ u32 val;
+
+#ifdef __LITTLE_ENDIAN
+ struct {
+ u16 locked;
+ u8 reserved[2];
+ };
+#else
+ struct {
+ u8 reserved[2];
+ u16 locked;
+ };
+#endif
+ };
+} arch_spinlock_t;
+
+#define __ARCH_SPIN_LOCK_UNLOCKED { { .val = 0 } }
+
+/*
+ * Bitfields in the lock word:
+ *
+ * 0: locked bit
+ * 1-14: lock holder cpu
+ * 15: lock owner or queuer vcpus observed to be preempted bit
+ * 16: must queue bit
+ * 17-31: tail cpu (+1)
+ */
+#define _Q_SET_MASK(type) (((1U << _Q_ ## type ## _BITS) - 1)\
+ << _Q_ ## type ## _OFFSET)
+/* 0x00000001 */
+#define _Q_LOCKED_OFFSET 0
+#define _Q_LOCKED_BITS 1
+#define _Q_LOCKED_VAL (1U << _Q_LOCKED_OFFSET)
+
+/* 0x00007ffe */
+#define _Q_OWNER_CPU_OFFSET 1
+#define _Q_OWNER_CPU_BITS 14
+#define _Q_OWNER_CPU_MASK _Q_SET_MASK(OWNER_CPU)
+
+#if CONFIG_NR_CPUS > (1U << _Q_OWNER_CPU_BITS)
+#error "qspinlock does not support such large CONFIG_NR_CPUS"
+#endif
+
+/* 0x00008000 */
+#define _Q_SLEEPY_OFFSET 15
+#define _Q_SLEEPY_BITS 1
+#define _Q_SLEEPY_VAL (1U << _Q_SLEEPY_OFFSET)
+
+/* 0x00010000 */
+#define _Q_MUST_Q_OFFSET 16
+#define _Q_MUST_Q_BITS 1
+#define _Q_MUST_Q_VAL (1U << _Q_MUST_Q_OFFSET)
+
+/* 0xfffe0000 */
+#define _Q_TAIL_CPU_OFFSET 17
+#define _Q_TAIL_CPU_BITS 15
+#define _Q_TAIL_CPU_MASK _Q_SET_MASK(TAIL_CPU)
+
+#if CONFIG_NR_CPUS >= (1U << _Q_TAIL_CPU_BITS)
+#error "qspinlock does not support such large CONFIG_NR_CPUS"
+#endif
+
+#endif /* _ASM_POWERPC_QSPINLOCK_TYPES_H */
#define RTAS_THREADS_ACTIVE -9005 /* Multiple processor threads active */
#define RTAS_OUTSTANDING_COPROC -9006 /* Outstanding coprocessor operations */
-/*
- * In general to call RTAS use rtas_token("string") to lookup
- * an RTAS token for the given string (e.g. "event-scan").
- * To actually perform the call use
- * ret = rtas_call(token, n_in, n_out, ...)
- * Where n_in is the number of input parameters and
- * n_out is the number of output parameters
- *
- * If the "string" is invalid on this system, RTAS_UNKNOWN_SERVICE
- * will be returned as a token. rtas_call() does look for this
- * token and error out gracefully so rtas_call(rtas_token("str"), ...)
- * may be safely used for one-shot calls to RTAS.
- *
- */
-
/* RTAS event classes */
#define RTAS_INTERNAL_ERROR 0x80000000 /* set bit 0 */
#define RTAS_EPOW_WARNING 0x40000000 /* set bit 1 */
/* See include/linux/spinlock.h */
#define smp_mb__after_spinlock() smp_mb()
-#ifndef CONFIG_PARAVIRT_SPINLOCKS
+#ifndef CONFIG_PPC_QUEUED_SPINLOCKS
static inline void pv_spinlocks_init(void) { }
#endif
#endif
#ifdef CONFIG_PPC_QUEUED_SPINLOCKS
-#include <asm-generic/qspinlock_types.h>
+#include <asm/qspinlock_types.h>
#include <asm-generic/qrwlock_types.h>
#else
#include <asm/simple_spinlock_types.h>
#endif
#define STACK_PT_REGS_OFFSET(sym, val) \
- DEFINE(sym, STACK_FRAME_OVERHEAD + offsetof(struct pt_regs, val))
+ DEFINE(sym, STACK_INT_FRAME_REGS + offsetof(struct pt_regs, val))
int main(void)
{
OFFSET(THREAD_CKVRSTATE, thread_struct, ckvr_state.vr);
OFFSET(THREAD_CKVRSAVE, thread_struct, ckvrsave);
OFFSET(THREAD_CKFPSTATE, thread_struct, ckfp_state.fpr);
- /* Local pt_regs on stack for Transactional Memory funcs. */
- DEFINE(TM_FRAME_SIZE, STACK_FRAME_OVERHEAD +
- sizeof(struct pt_regs) + 16);
+ /* Local pt_regs on stack in int frame form, plus 16 bytes for TM */
+ DEFINE(TM_FRAME_SIZE, STACK_INT_FRAME_SIZE + 16);
#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
OFFSET(TI_LOCAL_FLAGS, thread_info, local_flags);
/* Interrupt register frame */
DEFINE(INT_FRAME_SIZE, STACK_INT_FRAME_SIZE);
- DEFINE(SWITCH_FRAME_SIZE, STACK_FRAME_WITH_PT_REGS);
+ DEFINE(SWITCH_FRAME_SIZE, STACK_SWITCH_FRAME_SIZE);
STACK_PT_REGS_OFFSET(GPR0, gpr[0]);
STACK_PT_REGS_OFFSET(GPR1, gpr[1]);
STACK_PT_REGS_OFFSET(GPR2, gpr[2]);
/* book3s */
#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
- OFFSET(KVM_TLB_SETS, kvm, arch.tlb_sets);
OFFSET(KVM_SDR1, kvm, arch.sdr1);
OFFSET(KVM_HOST_LPID, kvm, arch.host_lpid);
OFFSET(KVM_HOST_LPCR, kvm, arch.host_lpcr);
OFFSET(KVM_HOST_SDR1, kvm, arch.host_sdr1);
OFFSET(KVM_ENABLED_HCALLS, kvm, arch.enabled_hcalls);
OFFSET(KVM_VRMA_SLB_V, kvm, arch.vrma_slb_v);
- OFFSET(KVM_RADIX, kvm, arch.radix);
OFFSET(KVM_SECURE_GUEST, kvm, arch.secure_guest);
OFFSET(VCPU_DSISR, kvm_vcpu, arch.shregs.dsisr);
OFFSET(VCPU_DAR, kvm_vcpu, arch.shregs.dar);
OFFSET(VCPU_VPA, kvm_vcpu, arch.vpa.pinned_addr);
OFFSET(VCPU_VPA_DIRTY, kvm_vcpu, arch.vpa.dirty);
OFFSET(VCPU_HEIR, kvm_vcpu, arch.emul_inst);
- OFFSET(VCPU_NESTED, kvm_vcpu, arch.nested);
OFFSET(VCPU_CPU, kvm_vcpu, cpu);
OFFSET(VCPU_THREAD_CPU, kvm_vcpu, arch.thread_cpu);
#endif
OFFSET(VCPU_DABRX, kvm_vcpu, arch.dabrx);
OFFSET(VCPU_DAWR0, kvm_vcpu, arch.dawr0);
OFFSET(VCPU_DAWRX0, kvm_vcpu, arch.dawrx0);
- OFFSET(VCPU_DAWR1, kvm_vcpu, arch.dawr1);
- OFFSET(VCPU_DAWRX1, kvm_vcpu, arch.dawrx1);
OFFSET(VCPU_CIABR, kvm_vcpu, arch.ciabr);
OFFSET(VCPU_HFLAGS, kvm_vcpu, arch.hflags);
OFFSET(VCPU_DEC_EXPIRES, kvm_vcpu, arch.dec_expires);
OFFSET(VCPU_PENDING_EXC, kvm_vcpu, arch.pending_exceptions);
OFFSET(VCPU_CEDED, kvm_vcpu, arch.ceded);
OFFSET(VCPU_PRODDED, kvm_vcpu, arch.prodded);
- OFFSET(VCPU_IRQ_PENDING, kvm_vcpu, arch.irq_pending);
- OFFSET(VCPU_DBELL_REQ, kvm_vcpu, arch.doorbell_request);
OFFSET(VCPU_MMCR, kvm_vcpu, arch.mmcr);
OFFSET(VCPU_MMCRA, kvm_vcpu, arch.mmcra);
OFFSET(VCPU_MMCRS, kvm_vcpu, arch.mmcrs);
OFFSET(VCPU_TCSCR, kvm_vcpu, arch.tcscr);
OFFSET(VCPU_ACOP, kvm_vcpu, arch.acop);
OFFSET(VCPU_WORT, kvm_vcpu, arch.wort);
- OFFSET(VCPU_TID, kvm_vcpu, arch.tid);
- OFFSET(VCPU_PSSCR, kvm_vcpu, arch.psscr);
OFFSET(VCPU_HFSCR, kvm_vcpu, arch.hfscr);
OFFSET(VCORE_ENTRY_EXIT, kvmppc_vcore, entry_exit_map);
OFFSET(VCORE_IN_GUEST, kvmppc_vcore, in_guest);
HSTATE_FIELD(HSTATE_HWTHREAD_STATE, hwthread_state);
HSTATE_FIELD(HSTATE_KVM_VCPU, kvm_vcpu);
HSTATE_FIELD(HSTATE_KVM_VCORE, kvm_vcore);
- HSTATE_FIELD(HSTATE_XIVE_TIMA_PHYS, xive_tima_phys);
- HSTATE_FIELD(HSTATE_XIVE_TIMA_VIRT, xive_tima_virt);
HSTATE_FIELD(HSTATE_HOST_IPI, host_ipi);
HSTATE_FIELD(HSTATE_PTID, ptid);
HSTATE_FIELD(HSTATE_FAKE_SUSPEND, fake_suspend);
HSTATE_FIELD(HSTATE_SDAR, host_mmcr[4]);
HSTATE_FIELD(HSTATE_MMCR2, host_mmcr[5]);
HSTATE_FIELD(HSTATE_SIER, host_mmcr[6]);
- HSTATE_FIELD(HSTATE_MMCR3, host_mmcr[7]);
- HSTATE_FIELD(HSTATE_SIER2, host_mmcr[8]);
- HSTATE_FIELD(HSTATE_SIER3, host_mmcr[9]);
HSTATE_FIELD(HSTATE_PMC1, host_pmc[0]);
HSTATE_FIELD(HSTATE_PMC2, host_pmc[1]);
HSTATE_FIELD(HSTATE_PMC3, host_pmc[2]);
OFFSET(VCPU_HOST_MAS6, kvm_vcpu, arch.host_mas6);
#endif
-#ifdef CONFIG_KVM_XICS
- DEFINE(VCPU_XIVE_SAVED_STATE, offsetof(struct kvm_vcpu,
- arch.xive_saved_state));
- DEFINE(VCPU_XIVE_CAM_WORD, offsetof(struct kvm_vcpu,
- arch.xive_cam_word));
- DEFINE(VCPU_XIVE_PUSHED, offsetof(struct kvm_vcpu, arch.xive_pushed));
- DEFINE(VCPU_XIVE_ESC_ON, offsetof(struct kvm_vcpu, arch.xive_esc_on));
- DEFINE(VCPU_XIVE_ESC_RADDR, offsetof(struct kvm_vcpu, arch.xive_esc_raddr));
- DEFINE(VCPU_XIVE_ESC_VADDR, offsetof(struct kvm_vcpu, arch.xive_esc_vaddr));
-#endif
-
#ifdef CONFIG_KVM_EXIT_TIMING
OFFSET(VCPU_TIMING_EXIT_TBU, kvm_vcpu, arch.timing_exit.tv32.tbu);
OFFSET(VCPU_TIMING_EXIT_TBL, kvm_vcpu, arch.timing_exit.tv32.tbl);
* Copyright (C) 2003 Benjamin Herrenschmidt (benh@kernel.crashing.org)
*/
+#include <linux/linkage.h>
+
#include <asm/processor.h>
#include <asm/page.h>
#include <asm/cputable.h>
blr
/* Enable caches for 603's, 604, 750 & 7400 */
-setup_common_caches:
+SYM_FUNC_START_LOCAL(setup_common_caches)
mfspr r11,SPRN_HID0
andi. r0,r11,HID0_DCE
ori r11,r11,HID0_ICE|HID0_DCE
sync
isync
blr
+SYM_FUNC_END(setup_common_caches)
/* 604, 604e, 604ev, ...
* Enable superscalar execution & branch history table
*/
-setup_604_hid0:
+SYM_FUNC_START_LOCAL(setup_604_hid0)
mfspr r11,SPRN_HID0
ori r11,r11,HID0_SIED|HID0_BHTE
ori r8,r11,HID0_BTCD
sync
isync
blr
+SYM_FUNC_END(setup_604_hid0)
/* 7400 <= rev 2.7 and 7410 rev = 1.0 suffer from some
* erratas we work around here.
* needed once we have applied workaround #5 (though it's
* not set by Apple's firmware at least).
*/
-setup_7400_workarounds:
+SYM_FUNC_START_LOCAL(setup_7400_workarounds)
mfpvr r3
rlwinm r3,r3,0,20,31
cmpwi 0,r3,0x0207
ble 1f
blr
-setup_7410_workarounds:
+SYM_FUNC_END(setup_7400_workarounds)
+SYM_FUNC_START_LOCAL(setup_7410_workarounds)
mfpvr r3
rlwinm r3,r3,0,20,31
cmpwi 0,r3,0x0100
sync
isync
blr
+SYM_FUNC_END(setup_7410_workarounds)
/* 740/750/7400/7410
* Enable Store Gathering (SGE), Address Broadcast (ABE),
* Dynamic Power Management (DPM), Speculative (SPD)
* Clear Instruction cache throttling (ICTC)
*/
-setup_750_7400_hid0:
+SYM_FUNC_START_LOCAL(setup_750_7400_hid0)
mfspr r11,SPRN_HID0
ori r11,r11,HID0_SGE | HID0_ABE | HID0_BHTE | HID0_BTIC
oris r11,r11,HID0_DPM@h
sync
isync
blr
+SYM_FUNC_END(setup_750_7400_hid0)
/* 750cx specific
* Looks like we have to disable NAP feature for some PLL settings...
* (waiting for confirmation)
*/
-setup_750cx:
+SYM_FUNC_START_LOCAL(setup_750cx)
mfspr r10, SPRN_HID1
rlwinm r10,r10,4,28,31
cmpwi cr0,r10,7
andc r6,r6,r7
stw r6,CPU_SPEC_FEATURES(r4)
blr
+SYM_FUNC_END(setup_750cx)
/* 750fx specific
*/
-setup_750fx:
+SYM_FUNC_START_LOCAL(setup_750fx)
blr
+SYM_FUNC_END(setup_750fx)
/* MPC 745x
* Enable Store Gathering (SGE), Branch Folding (FOLD)
* Clear Instruction cache throttling (ICTC)
* Enable L2 HW prefetch
*/
-setup_745x_specifics:
+SYM_FUNC_START_LOCAL(setup_745x_specifics)
/* We check for the presence of an L3 cache setup by
* the firmware. If any, we disable NAP capability as
* it's known to be bogus on rev 2.1 and earlier
sync
isync
blr
+SYM_FUNC_END(setup_745x_specifics)
/*
* Initialize the FPU registers. This is needed to work around an errata
* Benjamin Herrenschmidt <benh@kernel.crashing.org>
*/
+#include <linux/linkage.h>
+
#include <asm/page.h>
#include <asm/processor.h>
#include <asm/cputable.h>
blr
-has_L2_cache:
+SYM_FUNC_START_LOCAL(has_L2_cache)
/* skip L2 cache on P2040/P2040E as they have no L2 cache */
mfspr r3, SPRN_SVR
/* shift right by 8 bits and clear E bit of SVR */
1:
li r3, 0
blr
+SYM_FUNC_END(has_L2_cache)
/* flush backside L2 cache */
-flush_backside_L2_cache:
+SYM_FUNC_START_LOCAL(flush_backside_L2_cache)
mflr r10
bl has_L2_cache
mtlr r10
bne 1b
2:
blr
+SYM_FUNC_END(flush_backside_L2_cache)
_GLOBAL(cpu_down_flush_e500v2)
mflr r0
#include <linux/err.h>
#include <linux/sys.h>
#include <linux/threads.h>
+#include <linux/linkage.h>
+
#include <asm/reg.h>
#include <asm/page.h>
#include <asm/mmu.h>
#endif /* CONFIG_PPC_BOOK3S_32 || CONFIG_PPC_E500 */
#if defined(CONFIG_PPC_KUEP) && defined(CONFIG_PPC_BOOK3S_32)
- .globl __kuep_lock
-__kuep_lock:
+SYM_FUNC_START(__kuep_lock)
lwz r9, THREAD+THSR0(r2)
update_user_segments_by_4 r9, r10, r11, r12
blr
+SYM_FUNC_END(__kuep_lock)
-__kuep_unlock:
+SYM_FUNC_START_LOCAL(__kuep_unlock)
lwz r9, THREAD+THSR0(r2)
rlwinm r9,r9,0,~SR_NX
update_user_segments_by_4 r9, r10, r11, r12
blr
+SYM_FUNC_END(__kuep_unlock)
.macro kuep_lock
bl __kuep_lock
addi r12,r12,STACK_FRAME_REGS_MARKER@l
stw r9,_MSR(r1)
li r2, INTERRUPT_SYSCALL
- stw r12,8(r1)
+ stw r12,STACK_INT_FRAME_MARKER(r1)
stw r2,_TRAP(r1)
SAVE_GPR(0, r1)
SAVE_GPRS(3, 8, r1)
kuep_lock
/* Calling convention has r3 = regs, r4 = orig r0 */
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
mr r4,r0
bl system_call_exception
ret_from_syscall:
- addi r4,r1,STACK_FRAME_OVERHEAD
+ addi r4,r1,STACK_INT_FRAME_REGS
li r5,0
bl syscall_exit_prepare
#ifdef CONFIG_PPC_47x
* in arch/ppc/kernel/process.c
*/
_GLOBAL(_switch)
- stwu r1,-INT_FRAME_SIZE(r1)
+ stwu r1,-SWITCH_FRAME_SIZE(r1)
mflr r0
- stw r0,INT_FRAME_SIZE+4(r1)
+ stw r0,SWITCH_FRAME_SIZE+4(r1)
/* r3-r12 are caller saved -- Cort */
SAVE_NVGPRS(r1)
stw r0,_NIP(r1) /* Return to switch caller */
lwz r4,_NIP(r1) /* Return to _switch caller in new task */
mtlr r4
- addi r1,r1,INT_FRAME_SIZE
+ addi r1,r1,SWITCH_FRAME_SIZE
blr
.globl fast_exception_return
.globl interrupt_return
interrupt_return:
lwz r4,_MSR(r1)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
andi. r0,r4,MSR_PR
beq .Lkernel_interrupt_return
bl interrupt_exit_user_prepare
* code, and exception/interrupt return code for PowerPC.
*/
+#include <linux/objtool.h>
#include <linux/errno.h>
#include <linux/err.h>
#include <asm/cache.h>
// Flush the link stack
.rept 64
+ ANNOTATE_INTRA_FUNCTION_CALL
bl .+4
.endr
b 1f
std r14,PACA_EXMC+EX_R14(r13); \
std r15,PACA_EXMC+EX_R15(r13)
-
/* Core exception code for all exceptions except TLB misses. */
#define EXCEPTION_COMMON_LVL(n, scratch, excf) \
exc_##n##_common: \
std r10,_CCR(r1); /* store orig CR in stackframe */ \
std r9,GPR1(r1); /* store stack frame back link */ \
std r11,SOFTE(r1); /* and save it to stackframe */ \
- std r12,STACK_FRAME_OVERHEAD-16(r1); /* mark the frame */ \
+ std r12,STACK_INT_FRAME_MARKER(r1); /* mark the frame */ \
std r3,_TRAP(r1); /* set trap number */ \
std r0,RESULT(r1); /* clear regs->result */ \
- SAVE_NVGPRS(r1);
+ SAVE_NVGPRS(r1); \
+ SANITIZE_NVGPRS(); /* minimise speculation influence */
#define EXCEPTION_COMMON(n) \
EXCEPTION_COMMON_LVL(n, SPRN_SPRG_GEN_SCRATCH, PACA_EXGEN)
EXCEPTION_COMMON(trapnum) \
ack(r8); \
CHECK_NAPPING(); \
- addi r3,r1,STACK_FRAME_OVERHEAD; \
+ addi r3,r1,STACK_INT_FRAME_REGS; \
bl hdlr; \
b interrupt_return
EXCEPTION_COMMON_CRIT(0x100)
bl special_reg_save
CHECK_NAPPING();
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unknown_nmi_exception
b ret_from_crit_except
EXCEPTION_COMMON_MC(0x000)
bl special_reg_save
CHECK_NAPPING();
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl machine_check_exception
b ret_from_mc_except
std r14,_ESR(r1)
ld r14,PACA_EXGEN+EX_R14(r13)
EXCEPTION_COMMON(0x700)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl program_check_exception
REST_NVGPRS(r1)
b interrupt_return
beq- 1f
bl load_up_fpu
b fast_interrupt_return
-1: addi r3,r1,STACK_FRAME_OVERHEAD
+1: addi r3,r1,STACK_INT_FRAME_REGS
bl kernel_fp_unavailable_exception
b interrupt_return
1:
END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
#endif
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl altivec_unavailable_exception
b interrupt_return
BOOKE_INTERRUPT_ALTIVEC_ASSIST,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x220)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
#ifdef CONFIG_ALTIVEC
BEGIN_FTR_SECTION
bl altivec_assist_exception
EXCEPTION_COMMON_CRIT(0x9f0)
bl special_reg_save
CHECK_NAPPING();
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
#ifdef CONFIG_BOOKE_WDT
bl WatchdogException
#else
NORMAL_EXCEPTION_PROLOG(0xf20, BOOKE_INTERRUPT_AP_UNAVAIL,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0xf20)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unknown_exception
b interrupt_return
ld r14,PACA_EXCRIT+EX_R14(r13)
ld r15,PACA_EXCRIT+EX_R15(r13)
EXCEPTION_COMMON_CRIT(0xd00)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl DebugException
REST_NVGPRS(r1)
b interrupt_return
ld r14,PACA_EXDBG+EX_R14(r13)
ld r15,PACA_EXDBG+EX_R15(r13)
EXCEPTION_COMMON_DBG(0xd08)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl DebugException
REST_NVGPRS(r1)
b interrupt_return
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x260)
CHECK_NAPPING()
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
/*
* XXX: Returning from performance_monitor_exception taken as a
* soft-NMI (Linux irqs disabled) may be risky to use interrupt_return
EXCEPTION_COMMON_CRIT(0x2a0)
bl special_reg_save
CHECK_NAPPING();
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unknown_nmi_exception
b ret_from_crit_except
GDBELL_EXCEPTION_PROLOG(0x2c0, BOOKE_INTERRUPT_GUEST_DBELL,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x2c0)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unknown_exception
b interrupt_return
EXCEPTION_COMMON_CRIT(0x2e0)
bl special_reg_save
CHECK_NAPPING();
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unknown_nmi_exception
b ret_from_crit_except
NORMAL_EXCEPTION_PROLOG(0x310, BOOKE_INTERRUPT_HV_SYSCALL,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x310)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unknown_exception
b interrupt_return
NORMAL_EXCEPTION_PROLOG(0x320, BOOKE_INTERRUPT_HV_PRIV,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x320)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unknown_exception
b interrupt_return
NORMAL_EXCEPTION_PROLOG(0x340, BOOKE_INTERRUPT_LRAT_ERROR,
PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x340)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unknown_exception
b interrupt_return
* original values stashed away in the PACA
*/
storage_fault_common:
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl do_page_fault
b interrupt_return
* continues here.
*/
alignment_more:
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl alignment_exception
REST_NVGPRS(r1)
b interrupt_return
ZEROIZE_GPR(12)
std r12,0(r11)
LOAD_PACA_TOC()
-1: addi r3,r1,STACK_FRAME_OVERHEAD
+1: addi r3,r1,STACK_INT_FRAME_REGS
bl kernel_bad_stack
b 1b
*
*/
+#include <linux/linkage.h>
#include <asm/hw_irq.h>
#include <asm/exception-64s.h>
#include <asm/ptrace.h>
#define ISTACK .L_ISTACK_\name\() /* Set regular kernel stack */
#define __ISTACK(name) .L_ISTACK_ ## name
#define IKUAP .L_IKUAP_\name\() /* Do KUAP lock */
+#define IMSR_R12 .L_IMSR_R12_\name\() /* Assumes MSR saved to r12 */
#define INT_DEFINE_BEGIN(n) \
.macro int_define_ ## n name
.ifndef IKUAP
IKUAP=1
.endif
+ .ifndef IMSR_R12
+ IMSR_R12=0
+ .endif
.endm
/*
std r10,0(r1) /* make stack chain pointer */
std r0,GPR0(r1) /* save r0 in stackframe */
std r10,GPR1(r1) /* save r1 in stackframe */
+ SANITIZE_GPR(0)
/* Mark our [H]SRRs valid for return */
li r10,1
std r9,GPR11(r1)
std r10,GPR12(r1)
std r11,GPR13(r1)
+ .if !IMSR_R12
+ SANITIZE_GPRS(9, 12)
+ .else
+ SANITIZE_GPRS(9, 11)
+ .endif
SAVE_NVGPRS(r1)
+ SANITIZE_NVGPRS()
.if IDAR
.if IISIDE
END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
ld r10,IAREA+EX_CTR(r13)
std r10,_CTR(r1)
- std r2,GPR2(r1) /* save r2 in stackframe */
- SAVE_GPRS(3, 8, r1) /* save r3 - r8 in stackframe */
+ SAVE_GPRS(2, 8, r1) /* save r2 - r8 in stackframe */
+ SANITIZE_GPRS(2, 8)
mflr r9 /* Get LR, later save to stack */
LOAD_PACA_TOC() /* get kernel TOC into r2 */
std r9,_LINK(r1)
li r10,0
LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER)
std r10,RESULT(r1) /* clear regs->result */
- std r11,STACK_FRAME_OVERHEAD-16(r1) /* mark the frame */
+ std r11,STACK_INT_FRAME_MARKER(r1) /* mark the frame */
.endm
/*
mtlr r9
ld r9,_CCR(r1)
mtcr r9
+ SANITIZE_RESTORE_NVGPRS()
REST_GPRS(2, 13, r1)
REST_GPR(0, r1)
/* restore original r1. */
subi r1,r1,INT_FRAME_SIZE
__GEN_COMMON_BODY system_reset
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl system_reset_exception
/* Clear MSR_RI before setting SRR0 and SRR1. */
BEGIN_FTR_SECTION
bl enable_machine_check
END_FTR_SECTION_IFSET(CPU_FTR_HVMODE)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
BEGIN_FTR_SECTION
bl machine_check_early_boot
END_FTR_SECTION(0, 1) // nop out after boot
* save area: PACA_EXMC instead of PACA_EXGEN.
*/
GEN_COMMON machine_check
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl machine_check_exception_async
b interrupt_return_srr
* This is the NMI version of the handler because we are called from
* the early handler which is a true NMI.
*/
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl machine_check_exception
/*
* We will not reach here. Even if we did, there is no way out.
* Call unrecoverable_exception and die.
*/
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unrecoverable_exception
b .
EXC_COMMON_BEGIN(data_access_common)
GEN_COMMON data_access
ld r4,_DSISR(r1)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
andis. r0,r4,DSISR_DABRMATCH@h
bne- 1f
#ifdef CONFIG_PPC_64S_HASH_MMU
* do_break() may have changed the NV GPRS while handling a breakpoint.
* If so, we need to restore them with their updated values.
*/
- REST_NVGPRS(r1)
+ HANDLER_RESTORE_NVGPRS()
b interrupt_return_srr
#ifdef CONFIG_PPC_64S_HASH_MMU
BEGIN_MMU_FTR_SECTION
/* HPT case, do SLB fault */
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl do_slb_fault
cmpdi r3,0
bne- 1f
li r3,-EFAULT
#endif
std r3,RESULT(r1)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl do_bad_segment_interrupt
b interrupt_return_srr
EXC_VIRT_END(instruction_access, 0x4400, 0x80)
EXC_COMMON_BEGIN(instruction_access_common)
GEN_COMMON instruction_access
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
#ifdef CONFIG_PPC_64S_HASH_MMU
BEGIN_MMU_FTR_SECTION
bl do_hash_fault
#ifdef CONFIG_PPC_64S_HASH_MMU
BEGIN_MMU_FTR_SECTION
/* HPT case, do SLB fault */
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl do_slb_fault
cmpdi r3,0
bne- 1f
li r3,-EFAULT
#endif
std r3,RESULT(r1)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl do_bad_segment_interrupt
b interrupt_return_srr
EXC_VIRT_END(hardware_interrupt, 0x4500, 0x100)
EXC_COMMON_BEGIN(hardware_interrupt_common)
GEN_COMMON hardware_interrupt
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl do_IRQ
BEGIN_FTR_SECTION
b interrupt_return_hsrr
EXC_VIRT_END(alignment, 0x4600, 0x100)
EXC_COMMON_BEGIN(alignment_common)
GEN_COMMON alignment
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl alignment_exception
- REST_NVGPRS(r1) /* instruction emulation may change GPRs */
+ HANDLER_RESTORE_NVGPRS() /* instruction emulation may change GPRs */
b interrupt_return_srr
__GEN_COMMON_BODY program_check
.Ldo_program_check:
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl program_check_exception
- REST_NVGPRS(r1) /* instruction emulation may change GPRs */
+ HANDLER_RESTORE_NVGPRS() /* instruction emulation may change GPRs */
b interrupt_return_srr
#ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE
IKVM_REAL=1
#endif
+ IMSR_R12=1
INT_DEFINE_END(fp_unavailable)
EXC_REAL_BEGIN(fp_unavailable, 0x800, 0x100)
EXC_COMMON_BEGIN(fp_unavailable_common)
GEN_COMMON fp_unavailable
bne 1f /* if from user, just load it up */
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl kernel_fp_unavailable_exception
0: trap
EMIT_BUG_ENTRY 0b, __FILE__, __LINE__, 0
b fast_interrupt_return_srr
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
2: /* User process was in a transaction */
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl fp_unavailable_tm
b interrupt_return_srr
#endif
EXC_VIRT_END(decrementer, 0x4900, 0x80)
EXC_COMMON_BEGIN(decrementer_common)
GEN_COMMON decrementer
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl timer_interrupt
b interrupt_return_srr
EXC_VIRT_END(doorbell_super, 0x4a00, 0x100)
EXC_COMMON_BEGIN(doorbell_super_common)
GEN_COMMON doorbell_super
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
#ifdef CONFIG_PPC_DOORBELL
bl doorbell_exception
#else
EXC_VIRT_END(single_step, 0x4d00, 0x100)
EXC_COMMON_BEGIN(single_step_common)
GEN_COMMON single_step
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl single_step_exception
b interrupt_return_srr
EXC_VIRT_END(h_data_storage, 0x4e00, 0x20)
EXC_COMMON_BEGIN(h_data_storage_common)
GEN_COMMON h_data_storage
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
BEGIN_MMU_FTR_SECTION
bl do_bad_page_fault_segv
MMU_FTR_SECTION_ELSE
EXC_VIRT_END(h_instr_storage, 0x4e20, 0x20)
EXC_COMMON_BEGIN(h_instr_storage_common)
GEN_COMMON h_instr_storage
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unknown_exception
b interrupt_return_hsrr
EXC_VIRT_END(emulation_assist, 0x4e40, 0x20)
EXC_COMMON_BEGIN(emulation_assist_common)
GEN_COMMON emulation_assist
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl emulation_assist_interrupt
- REST_NVGPRS(r1) /* instruction emulation may change GPRs */
+ HANDLER_RESTORE_NVGPRS() /* instruction emulation may change GPRs */
b interrupt_return_hsrr
__GEN_COMMON_BODY hmi_exception_early
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl hmi_exception_realmode
cmpdi cr0,r3,0
bne 1f
EXC_COMMON_BEGIN(hmi_exception_common)
GEN_COMMON hmi_exception
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl handle_hmi_exception
b interrupt_return_hsrr
EXC_VIRT_END(h_doorbell, 0x4e80, 0x20)
EXC_COMMON_BEGIN(h_doorbell_common)
GEN_COMMON h_doorbell
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
#ifdef CONFIG_PPC_DOORBELL
bl doorbell_exception
#else
EXC_VIRT_END(h_virt_irq, 0x4ea0, 0x20)
EXC_COMMON_BEGIN(h_virt_irq_common)
GEN_COMMON h_virt_irq
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl do_IRQ
b interrupt_return_hsrr
EXC_VIRT_END(performance_monitor, 0x4f00, 0x20)
EXC_COMMON_BEGIN(performance_monitor_common)
GEN_COMMON performance_monitor
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
lbz r4,PACAIRQSOFTMASK(r13)
cmpdi r4,IRQS_ENABLED
bne 1f
#ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE
IKVM_REAL=1
#endif
+ IMSR_R12=1
INT_DEFINE_END(altivec_unavailable)
EXC_REAL_BEGIN(altivec_unavailable, 0xf20, 0x20)
b fast_interrupt_return_srr
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
2: /* User process was in a transaction */
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl altivec_unavailable_tm
b interrupt_return_srr
#endif
1:
END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
#endif
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl altivec_unavailable_exception
b interrupt_return_srr
#ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE
IKVM_REAL=1
#endif
+ IMSR_R12=1
INT_DEFINE_END(vsx_unavailable)
EXC_REAL_BEGIN(vsx_unavailable, 0xf40, 0x20)
b load_up_vsx
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
2: /* User process was in a transaction */
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl vsx_unavailable_tm
b interrupt_return_srr
#endif
1:
END_FTR_SECTION_IFSET(CPU_FTR_VSX)
#endif
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl vsx_unavailable_exception
b interrupt_return_srr
EXC_VIRT_END(facility_unavailable, 0x4f60, 0x20)
EXC_COMMON_BEGIN(facility_unavailable_common)
GEN_COMMON facility_unavailable
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl facility_unavailable_exception
- REST_NVGPRS(r1) /* instruction emulation may change GPRs */
+ HANDLER_RESTORE_NVGPRS() /* instruction emulation may change GPRs */
b interrupt_return_srr
EXC_VIRT_END(h_facility_unavailable, 0x4f80, 0x20)
EXC_COMMON_BEGIN(h_facility_unavailable_common)
GEN_COMMON h_facility_unavailable
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl facility_unavailable_exception
- REST_NVGPRS(r1) /* XXX Shouldn't be necessary in practice */
+ /* XXX Shouldn't be necessary in practice */
+ HANDLER_RESTORE_NVGPRS()
b interrupt_return_hsrr
EXC_VIRT_NONE(0x5200, 0x100)
EXC_COMMON_BEGIN(cbe_system_error_common)
GEN_COMMON cbe_system_error
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl cbe_system_error_exception
b interrupt_return_hsrr
EXC_VIRT_END(instruction_breakpoint, 0x5300, 0x100)
EXC_COMMON_BEGIN(instruction_breakpoint_common)
GEN_COMMON instruction_breakpoint
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl instruction_breakpoint_exception
b interrupt_return_srr
EXC_COMMON_BEGIN(denorm_exception_common)
GEN_COMMON denorm_exception
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unknown_exception
b interrupt_return_hsrr
EXC_VIRT_NONE(0x5600, 0x100)
EXC_COMMON_BEGIN(cbe_maintenance_common)
GEN_COMMON cbe_maintenance
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl cbe_maintenance_exception
b interrupt_return_hsrr
EXC_VIRT_END(altivec_assist, 0x5700, 0x100)
EXC_COMMON_BEGIN(altivec_assist_common)
GEN_COMMON altivec_assist
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
#ifdef CONFIG_ALTIVEC
bl altivec_assist_exception
- REST_NVGPRS(r1) /* instruction emulation may change GPRs */
+ HANDLER_RESTORE_NVGPRS() /* instruction emulation may change GPRs */
#else
bl unknown_exception
#endif
EXC_VIRT_NONE(0x5800, 0x100)
EXC_COMMON_BEGIN(cbe_thermal_common)
GEN_COMMON cbe_thermal
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl cbe_thermal_exception
b interrupt_return_hsrr
subi r1,r1,INT_FRAME_SIZE
__GEN_COMMON_BODY soft_nmi
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl soft_nmi_interrupt
/* Clear MSR_RI before setting SRR0 and SRR1. */
blr
/* MSR[RI] should be clear because this uses SRR[01] */
-disable_machine_check:
+SYM_FUNC_START_LOCAL(disable_machine_check)
mflr r0
bcl 20,31,$+4
0: mflr r3
RFI_TO_KERNEL
1: mtlr r0
blr
+SYM_FUNC_END(disable_machine_check)
stw r0,GPR0(r1)
lis r10,STACK_FRAME_REGS_MARKER@ha /* exception frame marker */
addi r10,r10,STACK_FRAME_REGS_MARKER@l
- stw r10,8(r1)
+ stw r10,STACK_INT_FRAME_MARKER(r1)
li r10, \trapno
stw r10,_TRAP(r1)
SAVE_GPRS(3, 8, r1)
mfspr r10,SPRN_XER
addi r2, r2, -THREAD
stw r10,_XER(r1)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
.endm
.macro prepare_transfer_to_handler
#include <linux/init.h>
#include <linux/pgtable.h>
#include <linux/sizes.h>
+#include <linux/linkage.h>
+
#include <asm/processor.h>
#include <asm/page.h>
#include <asm/mmu.h>
lis r1,init_thread_union@ha
addi r1,r1,init_thread_union@l
li r0,0
- stwu r0,THREAD_SIZE-STACK_FRAME_OVERHEAD(r1)
+ stwu r0,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r1)
bl early_init /* We have to do this with MMU on */
* kernel initialization. This maps the first 32 MBytes of memory 1:1
* virtual to physical and more importantly sets the cache mode.
*/
-initial_mmu:
+SYM_FUNC_START_LOCAL(initial_mmu)
tlbia /* Invalidate all TLB entries */
isync
mtspr SPRN_EVPR,r0
blr
+SYM_FUNC_END(initial_mmu)
_GLOBAL(abort)
mfspr r13,SPRN_DBCR0
lis r1,init_thread_union@h
ori r1,r1,init_thread_union@l
li r0,0
- stwu r0,THREAD_SIZE-STACK_FRAME_OVERHEAD(r1)
+ stwu r0,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r1)
bl early_init
*/
lis r1,temp_boot_stack@h
ori r1,r1,temp_boot_stack@l
- addi r1,r1,1024-STACK_FRAME_OVERHEAD
+ addi r1,r1,1024-STACK_FRAME_MIN_SIZE
li r0,0
stw r0,0(r1)
bl mmu_init_secondary
lwz r1,TASK_STACK(r2)
/* Current stack pointer */
- addi r1,r1,THREAD_SIZE-STACK_FRAME_OVERHEAD
+ addi r1,r1,THREAD_SIZE-STACK_FRAME_MIN_SIZE
li r0,0
stw r0,0(r1)
* variants.
*/
+#include <linux/linkage.h>
#include <linux/threads.h>
#include <linux/init.h>
#include <asm/reg.h>
/* Create a temp kernel stack for use before relocation is on. */
ld r1,PACAEMERGSP(r13)
- subi r1,r1,STACK_FRAME_OVERHEAD
+ subi r1,r1,STACK_FRAME_MIN_SIZE
/* See if we need to call a cpu state restore handler */
LOAD_REG_ADDR(r23, cur_cpu_spec)
* Assumes we're mapped EA == RA if the MMU is on.
*/
#ifdef CONFIG_PPC_BOOK3S
-__mmu_off:
+SYM_FUNC_START_LOCAL(__mmu_off)
mfmsr r3
andi. r0,r3,MSR_IR|MSR_DR
beqlr
sync
rfid
b . /* prevent speculative execution */
+SYM_FUNC_END(__mmu_off)
#endif
/* Create a temp kernel stack for use before relocation is on. */
ld r1,PACAEMERGSP(r13)
- subi r1,r1,STACK_FRAME_OVERHEAD
+ subi r1,r1,STACK_FRAME_MIN_SIZE
b __secondary_start
/*
* This subroutine clobbers r11 and r12
*/
-enable_64b_mode:
+SYM_FUNC_START_LOCAL(enable_64b_mode)
mfmsr r11 /* grab the current MSR */
#ifdef CONFIG_PPC_BOOK3E_64
oris r11,r11,0x8000 /* CM bit set, we'll set ICM later */
isync
#endif
blr
+SYM_FUNC_END(enable_64b_mode)
/*
* This puts the TOC pointer into r2, offset by 0x8000 (as expected
LOAD_REG_IMMEDIATE(r1,THREAD_SIZE)
add r1,r3,r1
li r0,0
- stdu r0,-STACK_FRAME_OVERHEAD(r1)
+ stdu r0,-STACK_FRAME_MIN_SIZE(r1)
/*
* Do very early kernel initializations, including initial hash table
#include <linux/init.h>
#include <linux/threads.h>
#include <linux/pgtable.h>
+#include <linux/linkage.h>
+
#include <asm/processor.h>
#include <asm/page.h>
#include <asm/mmu.h>
lis r1,init_thread_union@h
ori r1,r1,init_thread_union@l
li r0,0
- stwu r0,THREAD_SIZE-STACK_FRAME_OVERHEAD(r1)
+ stwu r0,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r1)
#ifdef CONFIG_SMP
stw r24, TASK_CPU(r2)
* Translate the effec addr in r3 to phys addr. The phys addr will be put
* into r3(higher 32bit) and r4(lower 32bit)
*/
-get_phys_addr:
+SYM_FUNC_START_LOCAL(get_phys_addr)
mfmsr r8
mfspr r9,SPRN_PID
rlwinm r9,r9,16,0x3fff0000 /* turn PID into MAS6[SPID] */
mfspr r3,SPRN_MAS7
#endif
blr
+SYM_FUNC_END(get_phys_addr)
/*
* Global functions
li r4,THREAD_ACC
evstddx evr6, r4, r3 /* save off accumulator */
beq 1f
- lwz r4,_MSR-STACK_FRAME_OVERHEAD(r5)
+ lwz r4,_MSR-STACK_INT_FRAME_REGS(r5)
lis r3,MSR_SPE@h
andc r4,r4,r3 /* disable SPE for previous task */
- stw r4,_MSR-STACK_FRAME_OVERHEAD(r5)
+ stw r4,_MSR-STACK_INT_FRAME_REGS(r5)
1:
blr
#endif /* CONFIG_SPE */
lwz r1,TASK_STACK(r2)
/* stack */
- addi r1,r1,THREAD_SIZE-STACK_FRAME_OVERHEAD
+ addi r1,r1,THREAD_SIZE-STACK_FRAME_MIN_SIZE
li r0,0
stw r0,0(r1)
#include <linux/magic.h>
#include <linux/pgtable.h>
#include <linux/sizes.h>
+#include <linux/linkage.h>
+
#include <asm/processor.h>
#include <asm/page.h>
#include <asm/mmu.h>
ori r0, r0, STACK_END_MAGIC@l
stw r0, 0(r1)
li r0,0
- stwu r0,THREAD_SIZE-STACK_FRAME_OVERHEAD(r1)
+ stwu r0,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r1)
lis r6, swapper_pg_dir@ha
tophys(r6,r6)
* 24 Mbytes of data, and the 512k IMMR space. Anything not covered by
* these mappings is mapped by page tables.
*/
-initial_mmu:
+SYM_FUNC_START_LOCAL(initial_mmu)
li r8, 0
mtspr SPRN_MI_CTR, r8 /* remove PINNED ITLB entries */
lis r10, MD_TWAM@h
#endif
mtspr SPRN_DER, r8
blr
+SYM_FUNC_END(initial_mmu)
_GLOBAL(mmu_pin_tlb)
lis r9, (1f - PAGE_OFFSET)@h
#include <linux/init.h>
#include <linux/pgtable.h>
+#include <linux/linkage.h>
+
#include <asm/reg.h>
#include <asm/page.h>
#include <asm/mmu.h>
lwz r1,TASK_STACK(r1)
/* stack */
- addi r1,r1,THREAD_SIZE-STACK_FRAME_OVERHEAD
+ addi r1,r1,THREAD_SIZE-STACK_FRAME_MIN_SIZE
li r0,0
tophys(r3,r1)
stw r0,0(r3)
* Load stuff into the MMU. Intended to be called with
* IR=0 and DR=0.
*/
-early_hash_table:
+SYM_FUNC_START_LOCAL(early_hash_table)
sync /* Force all PTE updates to finish */
isync
tlbia /* Clear all TLB entries */
ori r6, r6, 3 /* 256kB table */
mtspr SPRN_SDR1, r6
blr
+SYM_FUNC_END(early_hash_table)
-load_up_mmu:
+SYM_FUNC_START_LOCAL(load_up_mmu)
sync /* Force all PTE updates to finish */
isync
tlbia /* Clear all TLB entries */
LOAD_BAT(7,r3,r4,r5)
END_MMU_FTR_SECTION_IFSET(MMU_FTR_USE_HIGH_BATS)
blr
+SYM_FUNC_END(load_up_mmu)
_GLOBAL(load_segment_registers)
li r0, NUM_USER_SEGMENTS /* load up user segment register values */
lis r1,init_thread_union@ha
addi r1,r1,init_thread_union@l
li r0,0
- stwu r0,THREAD_SIZE-STACK_FRAME_OVERHEAD(r1)
+ stwu r0,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r1)
/*
* Do early platform-specific initialization,
* and set up the MMU.
* this makes sure it's done.
* -- Cort
*/
-clear_bats:
+SYM_FUNC_START_LOCAL(clear_bats)
li r10,0
mtspr SPRN_DBAT0U,r10
mtspr SPRN_IBAT7L,r10
END_MMU_FTR_SECTION_IFSET(MMU_FTR_USE_HIGH_BATS)
blr
+SYM_FUNC_END(clear_bats)
_GLOBAL(update_bats)
lis r4, 1f@h
mtspr SPRN_SRR1, r6
rfi
-flush_tlbs:
+SYM_FUNC_START_LOCAL(flush_tlbs)
lis r10, 0x40
1: addic. r10, r10, -0x1000
tlbie r10
bgt 1b
sync
blr
+SYM_FUNC_END(flush_tlbs)
-mmu_off:
+SYM_FUNC_START_LOCAL(mmu_off)
addi r4, r3, __after_mmu_off - _start
mfmsr r3
andi. r0,r3,MSR_DR|MSR_IR /* MMU enabled? */
mtspr SPRN_SRR1,r3
sync
rfi
+SYM_FUNC_END(mmu_off)
/* We use one BAT to map up to 256M of RAM at _PAGE_OFFSET */
-initial_bats:
+SYM_FUNC_START_LOCAL(initial_bats)
lis r11,PAGE_OFFSET@h
tophys(r8,r11)
#ifdef CONFIG_SMP
mtspr SPRN_IBAT0U,r11
isync
blr
+SYM_FUNC_END(initial_bats)
#ifdef CONFIG_BOOTX_TEXT
-setup_disp_bat:
+SYM_FUNC_START_LOCAL(setup_disp_bat)
/*
* setup the display bat prepared for us in prom.c
*/
mtspr SPRN_DBAT3L,r8
mtspr SPRN_DBAT3U,r11
blr
+SYM_FUNC_END(setup_disp_bat)
#endif /* CONFIG_BOOTX_TEXT */
#ifdef CONFIG_PPC_EARLY_DEBUG_CPM
-setup_cpm_bat:
+SYM_FUNC_START_LOCAL(setup_cpm_bat)
lis r8, 0xf000
ori r8, r8, 0x002a
mtspr SPRN_DBAT1L, r8
mtspr SPRN_DBAT1U, r11
blr
+SYM_FUNC_END(setup_cpm_bat)
#endif
#ifdef CONFIG_PPC_EARLY_DEBUG_USBGECKO
-setup_usbgecko_bat:
+SYM_FUNC_START_LOCAL(setup_usbgecko_bat)
/* prepare a BAT for early io */
#if defined(CONFIG_GAMECUBE)
lis r8, 0x0c00
mtspr SPRN_DBAT1L, r8
mtspr SPRN_DBAT1U, r11
blr
+SYM_FUNC_END(setup_usbgecko_bat)
#endif
.data
stw r0,GPR0(r1)
lis r10, STACK_FRAME_REGS_MARKER@ha /* exception frame marker */
addi r10, r10, STACK_FRAME_REGS_MARKER@l
- stw r10, 8(r1)
+ stw r10, STACK_INT_FRAME_MARKER(r1)
li r10, \trapno
stw r10,_TRAP(r1)
SAVE_GPRS(3, 8, r1)
mfspr r10,SPRN_XER
addi r2, r2, -THREAD
stw r10,_XER(r1)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
.endm
.macro prepare_transfer_to_handler
ppc_inst_t instr = ppc_inst(0);
int type = 0;
int size = 0;
- unsigned long ea;
+ unsigned long ea = 0;
/* Disable breakpoints during exception handling */
hw_breakpoint_disable();
std r11,_TRAP(r1)
std r12,_CCR(r1)
std r3,ORIG_GPR3(r1)
+ LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER)
+ std r11,STACK_INT_FRAME_MARKER(r1) /* "regs" marker */
/* Calling convention has r3 = regs, r4 = orig r0 */
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
mr r4,r0
- LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER)
- std r11,-16(r3) /* "regshere" marker */
BEGIN_FTR_SECTION
HMT_MEDIUM
* but this is the best we can do.
*/
+ /*
+ * Zero user registers to prevent influencing speculative execution
+ * state of kernel code.
+ */
+ SANITIZE_SYSCALL_GPRS()
bl system_call_exception
.Lsyscall_vectored_\name\()_exit:
- addi r4,r1,STACK_FRAME_OVERHEAD
+ addi r4,r1,STACK_INT_FRAME_REGS
li r5,1 /* scv */
bl syscall_exit_prepare
std r1,PACA_EXIT_SAVE_R1(r13) /* save r1 for restart */
HMT_MEDIUM_LOW
END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
+ SANITIZE_RESTORE_NVGPRS()
cmpdi r3,0
bne .Lsyscall_vectored_\name\()_restore_regs
ld r4,_LINK(r1)
ld r5,_XER(r1)
- REST_NVGPRS(r1)
+ HANDLER_RESTORE_NVGPRS()
REST_GPR(0, r1)
mtcr r2
mtctr r3
ld r1,PACA_EXIT_SAVE_R1(r13)
LOAD_PACA_TOC()
ld r3,RESULT(r1)
- addi r4,r1,STACK_FRAME_OVERHEAD
+ addi r4,r1,STACK_INT_FRAME_REGS
li r11,IRQS_ALL_DISABLED
stb r11,PACAIRQSOFTMASK(r13)
bl syscall_exit_restart
std r11,_TRAP(r1)
std r12,_CCR(r1)
std r3,ORIG_GPR3(r1)
+ LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER)
+ std r11,STACK_INT_FRAME_MARKER(r1) /* "regs" marker */
/* Calling convention has r3 = regs, r4 = orig r0 */
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
mr r4,r0
- LOAD_REG_IMMEDIATE(r11, STACK_FRAME_REGS_MARKER)
- std r11,-16(r3) /* "regshere" marker */
#ifdef CONFIG_PPC_BOOK3S
li r11,1
wrteei 1
#endif
+ /*
+ * Zero user registers to prevent influencing speculative execution
+ * state of kernel code.
+ */
+ SANITIZE_SYSCALL_GPRS()
bl system_call_exception
.Lsyscall_exit:
- addi r4,r1,STACK_FRAME_OVERHEAD
+ addi r4,r1,STACK_INT_FRAME_REGS
li r5,0 /* !scv */
bl syscall_exit_prepare
std r1,PACA_EXIT_SAVE_R1(r13) /* save r1 for restart */
stdcx. r0,0,r1 /* to clear the reservation */
END_FTR_SECTION_IFCLR(CPU_FTR_STCX_CHECKS_ADDRESS)
+ SANITIZE_RESTORE_NVGPRS()
cmpdi r3,0
bne .Lsyscall_restore_regs
/* Zero volatile regs that may contain sensitive kernel data */
.Lsyscall_restore_regs:
ld r3,_CTR(r1)
ld r4,_XER(r1)
- REST_NVGPRS(r1)
+ HANDLER_RESTORE_NVGPRS()
mtctr r3
mtspr SPRN_XER,r4
REST_GPR(0, r1)
ld r1,PACA_EXIT_SAVE_R1(r13)
LOAD_PACA_TOC()
ld r3,RESULT(r1)
- addi r4,r1,STACK_FRAME_OVERHEAD
+ addi r4,r1,STACK_INT_FRAME_REGS
li r11,IRQS_ALL_DISABLED
stb r11,PACAIRQSOFTMASK(r13)
bl syscall_exit_restart
andi. r0,r5,MSR_RI
li r3,0 /* 0 return value, no EMULATE_STACK_STORE */
bne+ .Lfast_kernel_interrupt_return_srr
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl unrecoverable_exception
b . /* should not get here */
#else
beq interrupt_return_\srr\()_kernel
interrupt_return_\srr\()_user: /* make backtraces match the _kernel variant */
_ASM_NOKPROBE_SYMBOL(interrupt_return_\srr\()_user)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl interrupt_exit_user_prepare
+#ifndef CONFIG_INTERRUPT_SANITIZE_REGISTERS
cmpdi r3,0
bne- .Lrestore_nvgprs_\srr
.Lrestore_nvgprs_\srr\()_cont:
+#endif
std r1,PACA_EXIT_SAVE_R1(r13) /* save r1 for restart */
#ifdef CONFIG_PPC_BOOK3S
.Linterrupt_return_\srr\()_user_rst_start:
stb r11,PACAIRQHAPPENED(r13) # clear out possible HARD_DIS
.Lfast_user_interrupt_return_\srr\():
+ SANITIZE_RESTORE_NVGPRS()
#ifdef CONFIG_PPC_BOOK3S
.ifc \srr,srr
lbz r4,PACASRR_VALID(r13)
b . /* prevent speculative execution */
.Linterrupt_return_\srr\()_user_rst_end:
+#ifndef CONFIG_INTERRUPT_SANITIZE_REGISTERS
.Lrestore_nvgprs_\srr\():
REST_NVGPRS(r1)
b .Lrestore_nvgprs_\srr\()_cont
+#endif
#ifdef CONFIG_PPC_BOOK3S
interrupt_return_\srr\()_user_restart:
GET_PACA(r13)
ld r1,PACA_EXIT_SAVE_R1(r13)
LOAD_PACA_TOC()
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
li r11,IRQS_ALL_DISABLED
stb r11,PACAIRQSOFTMASK(r13)
bl interrupt_exit_user_restart
.balign IFETCH_ALIGN_BYTES
interrupt_return_\srr\()_kernel:
_ASM_NOKPROBE_SYMBOL(interrupt_return_\srr\()_kernel)
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
bl interrupt_exit_kernel_prepare
std r1,PACA_EXIT_SAVE_R1(r13) /* save r1 for restart */
stb r11,PACAIRQHAPPENED(r13) // clear the possible HARD_DIS
.Lfast_kernel_interrupt_return_\srr\():
+ SANITIZE_RESTORE_NVGPRS()
cmpdi cr1,r3,0
#ifdef CONFIG_PPC_BOOK3S
.ifc \srr,srr
* Leaving a stale STACK_FRAME_REGS_MARKER on the stack can confuse
* the reliable stack unwinder later on. Clear it.
*/
- std r0,STACK_FRAME_OVERHEAD-16(r1)
+ std r0,STACK_INT_FRAME_MARKER(r1)
REST_GPRS(2, 5, r1)
GET_PACA(r13)
ld r1,PACA_EXIT_SAVE_R1(r13)
LOAD_PACA_TOC()
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
li r11,IRQS_ALL_DISABLED
stb r11,PACAIRQSOFTMASK(r13)
bl interrupt_exit_kernel_restart
PPC_LL " %%r1, 0(%%r1) ;"
: // Outputs
: // Inputs
- [sp] "b" (sp), [offset] "i" (THREAD_SIZE - STACK_FRAME_OVERHEAD),
+ [sp] "b" (sp), [offset] "i" (THREAD_SIZE - STACK_FRAME_MIN_SIZE),
[callee] "i" (__do_softirq)
: // Clobbers
"lr", "xer", "ctr", "memory", "cr0", "cr1", "cr5", "cr6",
: // Outputs
"+r" (r3)
: // Inputs
- [sp] "b" (sp), [offset] "i" (THREAD_SIZE - STACK_FRAME_OVERHEAD),
+ [sp] "b" (sp), [offset] "i" (THREAD_SIZE - STACK_FRAME_MIN_SIZE),
[callee] "i" (__do_irq)
: // Clobbers
"lr", "xer", "ctr", "memory", "cr0", "cr1", "cr5", "cr6",
void sleeping_thread_to_gdb_regs(unsigned long *gdb_regs, struct task_struct *p)
{
struct pt_regs *regs = (struct pt_regs *)(p->thread.ksp +
- STACK_FRAME_OVERHEAD);
+ STACK_INT_FRAME_REGS);
unsigned long *ptr = gdb_regs;
int reg;
#include <linux/kdebug.h>
#include <linux/slab.h>
#include <linux/moduleloader.h>
+#include <linux/set_memory.h>
#include <asm/code-patching.h>
#include <asm/cacheflush.h>
#include <asm/sstep.h>
#include <asm/sections.h>
#include <asm/inst.h>
-#include <asm/set_memory.h>
#include <linux/uaccess.h>
DEFINE_PER_CPU(struct kprobe *, current_kprobe) = NULL;
if (!page)
return NULL;
- if (strict_module_rwx_enabled()) {
- set_memory_ro((unsigned long)page, 1);
- set_memory_x((unsigned long)page, 1);
- }
+ if (strict_module_rwx_enabled())
+ set_memory_rox((unsigned long)page, 1);
+
return page;
}
printk("Cannot register a kprobe on the second word of prefixed instruction\n");
ret = -EINVAL;
}
- preempt_disable();
prev = get_kprobe(p->addr - 1);
- preempt_enable_no_resched();
/*
* When prev is a ftrace-based kprobe, we don't have an insn, and it
if (ret > 0) {
restore_previous_kprobe(kcb);
- preempt_enable_no_resched();
+ preempt_enable();
return 1;
}
}
if (p->pre_handler && p->pre_handler(p, regs)) {
/* handler changed execution path, so skip ss setup */
reset_current_kprobe();
- preempt_enable_no_resched();
+ preempt_enable();
return 1;
}
kcb->kprobe_status = KPROBE_HIT_SSDONE;
reset_current_kprobe();
- preempt_enable_no_resched();
+ preempt_enable();
return 1;
}
}
return 1;
no_kprobe:
- preempt_enable_no_resched();
+ preempt_enable();
return ret;
}
NOKPROBE_SYMBOL(kprobe_handler);
}
reset_current_kprobe();
out:
- preempt_enable_no_resched();
+ preempt_enable();
/*
* if somebody else is singlestepping across a probe point, msr
restore_previous_kprobe(kcb);
else
reset_current_kprobe();
- preempt_enable_no_resched();
+ preempt_enable();
break;
case KPROBE_HIT_ACTIVE:
case KPROBE_HIT_SSDONE:
_GLOBAL(start_secondary_resume)
/* Reset stack */
rlwinm r1, r1, 0, 0, 31 - THREAD_SHIFT
- addi r1,r1,THREAD_SIZE-STACK_FRAME_OVERHEAD
+ addi r1,r1,THREAD_SIZE-STACK_FRAME_MIN_SIZE
li r3,0
stw r3,0(r1) /* Zero the stack frame pointer */
bl start_secondary
* PPC64 updates by Dave Engebretsen (engebret@us.ibm.com)
*/
+#include <linux/linkage.h>
#include <linux/sys.h>
#include <asm/unistd.h>
#include <asm/errno.h>
*
* don't overwrite r3 here, it is live for kexec_wait above.
*/
-real_mode: /* assume normal blr return */
+SYM_FUNC_START_LOCAL(real_mode) /* assume normal blr return */
#ifdef CONFIG_PPC_BOOK3E_64
/* Create an identity mapping. */
b kexec_create_tlb
mtspr SPRN_SRR0,r11
rfid
#endif
+SYM_FUNC_END(real_mode)
/*
* kexec_sequence(newstack, start, image, control, clear_all(),
std r0,16(r1)
/* switch stacks to newstack -- &kexec_stack.stack */
- stdu r1,THREAD_SIZE-STACK_FRAME_OVERHEAD(r3)
+ stdu r1,THREAD_SIZE-STACK_FRAME_MIN_SIZE(r3)
mr r1,r3
li r0,0
std r26,-48(r1)
std r25,-56(r1)
- stdu r1,-STACK_FRAME_OVERHEAD-64(r1)
+ stdu r1,-STACK_FRAME_MIN_SIZE-64(r1)
/* save args into preserved regs */
mr r31,r3 /* newstack (both) */
this, and makes other things simpler. Anton?
--RR. */
+bool module_elf_check_arch(Elf_Ehdr *hdr)
+{
+ unsigned long abi_level = hdr->e_flags & 0x3;
+
+ if (IS_ENABLED(CONFIG_PPC64_ELF_ABI_V2))
+ return abi_level == 2;
+ else
+ return abi_level < 2;
+}
+
#ifdef CONFIG_PPC64_ELF_ABI_V2
static func_desc_t func_desc(unsigned long addr)
__this_cpu_write(current_kprobe, NULL);
}
- preempt_enable_no_resched();
+ preempt_enable();
}
NOKPROBE_SYMBOL(optimized_callback);
TEMPLATE_FOR_IMM_LOAD_INSNS
/* 2. pt_regs pointer in r4 */
- addi r4,r1,STACK_FRAME_OVERHEAD
+ addi r4,r1,STACK_INT_FRAME_REGS
.global optprobe_template_call_handler
optprobe_template_call_handler:
* Parameters for instruction emulation:
* 1. Pass SP in register r3.
*/
- addi r3,r1,STACK_FRAME_OVERHEAD
+ addi r3,r1,STACK_INT_FRAME_REGS
.global optprobe_template_insn
optprobe_template_insn:
* different ABIs, though).
*/
_GLOBAL(ppc_save_regs)
- PPC_STL r0,0*SZL(r3)
+ /* This allows stack frame accessor macros and offsets to be used */
+ subi r3,r3,STACK_INT_FRAME_REGS
+ PPC_STL r0,GPR0(r3)
#ifdef CONFIG_PPC32
- stmw r2, 2*SZL(r3)
+ stmw r2,GPR2(r3)
#else
- PPC_STL r2,2*SZL(r3)
- PPC_STL r3,3*SZL(r3)
- PPC_STL r4,4*SZL(r3)
- PPC_STL r5,5*SZL(r3)
- PPC_STL r6,6*SZL(r3)
- PPC_STL r7,7*SZL(r3)
- PPC_STL r8,8*SZL(r3)
- PPC_STL r9,9*SZL(r3)
- PPC_STL r10,10*SZL(r3)
- PPC_STL r11,11*SZL(r3)
- PPC_STL r12,12*SZL(r3)
- PPC_STL r13,13*SZL(r3)
- PPC_STL r14,14*SZL(r3)
- PPC_STL r15,15*SZL(r3)
- PPC_STL r16,16*SZL(r3)
- PPC_STL r17,17*SZL(r3)
- PPC_STL r18,18*SZL(r3)
- PPC_STL r19,19*SZL(r3)
- PPC_STL r20,20*SZL(r3)
- PPC_STL r21,21*SZL(r3)
- PPC_STL r22,22*SZL(r3)
- PPC_STL r23,23*SZL(r3)
- PPC_STL r24,24*SZL(r3)
- PPC_STL r25,25*SZL(r3)
- PPC_STL r26,26*SZL(r3)
- PPC_STL r27,27*SZL(r3)
- PPC_STL r28,28*SZL(r3)
- PPC_STL r29,29*SZL(r3)
- PPC_STL r30,30*SZL(r3)
- PPC_STL r31,31*SZL(r3)
+ SAVE_GPRS(2, 31, r3)
lbz r0,PACAIRQSOFTMASK(r13)
- PPC_STL r0,SOFTE-STACK_FRAME_OVERHEAD(r3)
+ PPC_STL r0,SOFTE(r3)
#endif
/* go up one stack frame for SP */
PPC_LL r4,0(r1)
- PPC_STL r4,1*SZL(r3)
+ PPC_STL r4,GPR1(r3)
/* get caller's LR */
PPC_LL r0,LRSAVE(r4)
- PPC_STL r0,_LINK-STACK_FRAME_OVERHEAD(r3)
+ PPC_STL r0,_LINK(r3)
mflr r0
- PPC_STL r0,_NIP-STACK_FRAME_OVERHEAD(r3)
+ PPC_STL r0,_NIP(r3)
mfmsr r0
- PPC_STL r0,_MSR-STACK_FRAME_OVERHEAD(r3)
+ PPC_STL r0,_MSR(r3)
mfctr r0
- PPC_STL r0,_CTR-STACK_FRAME_OVERHEAD(r3)
+ PPC_STL r0,_CTR(r3)
mfxer r0
- PPC_STL r0,_XER-STACK_FRAME_OVERHEAD(r3)
+ PPC_STL r0,_XER(r3)
mfcr r0
- PPC_STL r0,_CCR-STACK_FRAME_OVERHEAD(r3)
+ PPC_STL r0,_CCR(r3)
li r0,0
- PPC_STL r0,_TRAP-STACK_FRAME_OVERHEAD(r3)
- PPC_STL r0,ORIG_GPR3-STACK_FRAME_OVERHEAD(r3)
+ PPC_STL r0,_TRAP(r3)
+ PPC_STL r0,ORIG_GPR3(r3)
blr
return 0;
}
-void __set_breakpoint(int nr, struct arch_hw_breakpoint *brk)
+static void set_hw_breakpoint(int nr, struct arch_hw_breakpoint *brk)
{
- memcpy(this_cpu_ptr(¤t_brk[nr]), brk, sizeof(*brk));
-
if (dawr_enabled())
// Power8 or later
set_dawr(nr, brk);
WARN_ON_ONCE(1);
}
+void __set_breakpoint(int nr, struct arch_hw_breakpoint *brk)
+{
+ memcpy(this_cpu_ptr(¤t_brk[nr]), brk, sizeof(*brk));
+ set_hw_breakpoint(nr, brk);
+}
+
/* Check if we have DAWR or DABR hardware */
bool ppc_breakpoint_available(void)
{
}
EXPORT_SYMBOL_GPL(ppc_breakpoint_available);
+/* Disable the breakpoint in hardware without touching current_brk[] */
+void suspend_breakpoints(void)
+{
+ struct arch_hw_breakpoint brk = {0};
+ int i;
+
+ if (!ppc_breakpoint_available())
+ return;
+
+ for (i = 0; i < nr_wp_slots(); i++)
+ set_hw_breakpoint(i, &brk);
+}
+
+/*
+ * Re-enable breakpoints suspended by suspend_breakpoints() in hardware
+ * from current_brk[]
+ */
+void restore_breakpoints(void)
+{
+ int i;
+
+ if (!ppc_breakpoint_available())
+ return;
+
+ for (i = 0; i < nr_wp_slots(); i++)
+ set_hw_breakpoint(i, this_cpu_ptr(¤t_brk[i]));
+}
+
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
static inline bool tm_enabled(struct task_struct *tsk)
unsigned long nip = regs->nip;
unsigned long pc = regs->nip - (NR_INSN_TO_PRINT * 3 / 4 * sizeof(int));
- printk("Instruction dump:");
+ printk("Code: ");
/*
* If we were executing with the MMU off for instructions, adjust pc
for (i = 0; i < NR_INSN_TO_PRINT; i++) {
int instr;
- if (!(i % 8))
- pr_cont("\n");
-
if (!__kernel_text_address(pc) ||
get_kernel_nofault(instr, (const void *)pc)) {
pr_cont("XXXXXXXX ");
klp_init_thread_info(p);
+ /* Create initial stack frame. */
+ sp -= STACK_USER_INT_FRAME_SIZE;
+ *(unsigned long *)(sp + STACK_INT_FRAME_MARKER) = STACK_FRAME_REGS_MARKER;
+
/* Copy registers */
- sp -= sizeof(struct pt_regs);
- childregs = (struct pt_regs *) sp;
+ childregs = (struct pt_regs *)(sp + STACK_INT_FRAME_REGS);
if (unlikely(args->fn)) {
/* kernel thread */
+ ((unsigned long *)sp)[0] = 0;
memset(childregs, 0, sizeof(struct pt_regs));
- childregs->gpr[1] = sp + sizeof(struct pt_regs);
+ childregs->gpr[1] = sp + STACK_USER_INT_FRAME_SIZE;
/* function */
if (args->fn)
childregs->gpr[14] = ppc_function_entry((void *)args->fn);
*childregs = *regs;
if (usp)
childregs->gpr[1] = usp;
+ ((unsigned long *)sp)[0] = childregs->gpr[1];
p->thread.regs = childregs;
/* 64s sets this in ret_from_fork */
if (!IS_ENABLED(CONFIG_PPC_BOOK3S_64))
f = ret_from_fork;
}
childregs->msr &= ~(MSR_FP|MSR_VEC|MSR_VSX);
- sp -= STACK_FRAME_OVERHEAD;
/*
* The way this works is that at some point in the future
* do some house keeping and then return from the fork or clone
* system call, using the stack frame created above.
*/
- ((unsigned long *)sp)[0] = 0;
- sp -= sizeof(struct pt_regs);
- kregs = (struct pt_regs *) sp;
- sp -= STACK_FRAME_OVERHEAD;
+ ((unsigned long *)sp)[STACK_FRAME_LR_SAVE] = (unsigned long)f;
+ sp -= STACK_SWITCH_FRAME_SIZE;
+ ((unsigned long *)sp)[0] = sp + STACK_SWITCH_FRAME_SIZE;
+ kregs = (struct pt_regs *)(sp + STACK_SWITCH_FRAME_REGS);
p->thread.ksp = sp;
+
#ifdef CONFIG_HAVE_HW_BREAKPOINT
for (i = 0; i < nr_wp_slots(); i++)
p->thread.ptrace_bps[i] = NULL;
return 0;
}
-
-int validate_sp(unsigned long sp, struct task_struct *p,
- unsigned long nbytes)
+/*
+ * validate the stack frame of a particular minimum size, used for when we are
+ * looking at a certain object in the stack beyond the minimum.
+ */
+int validate_sp_size(unsigned long sp, struct task_struct *p,
+ unsigned long nbytes)
{
unsigned long stack_page = (unsigned long)task_stack_page(p);
return valid_emergency_stack(sp, p, nbytes);
}
-EXPORT_SYMBOL(validate_sp);
+int validate_sp(unsigned long sp, struct task_struct *p)
+{
+ return validate_sp_size(sp, p, STACK_FRAME_MIN_SIZE);
+}
static unsigned long ___get_wchan(struct task_struct *p)
{
int count = 0;
sp = p->thread.ksp;
- if (!validate_sp(sp, p, STACK_FRAME_OVERHEAD))
+ if (!validate_sp(sp, p))
return 0;
do {
sp = READ_ONCE_NOCHECK(*(unsigned long *)sp);
- if (!validate_sp(sp, p, STACK_FRAME_OVERHEAD) ||
- task_is_running(p))
+ if (!validate_sp(sp, p) || task_is_running(p))
return 0;
if (count > 0) {
ip = READ_ONCE_NOCHECK(((unsigned long *)sp)[STACK_FRAME_LR_SAVE]);
lr = 0;
printk("%sCall Trace:\n", loglvl);
do {
- if (!validate_sp(sp, tsk, STACK_FRAME_OVERHEAD))
+ if (!validate_sp(sp, tsk))
break;
stack = (unsigned long *) sp;
/*
* See if this is an exception frame.
- * We look for the "regshere" marker in the current frame.
+ * We look for the "regs" marker in the current frame.
+ *
+ * STACK_SWITCH_FRAME_SIZE being the smallest frame that
+ * could hold a pt_regs, if that does not fit then it can't
+ * have regs.
*/
- if (validate_sp(sp, tsk, STACK_FRAME_WITH_PT_REGS)
- && stack[STACK_FRAME_MARKER] == STACK_FRAME_REGS_MARKER) {
+ if (validate_sp_size(sp, tsk, STACK_SWITCH_FRAME_SIZE)
+ && stack[STACK_INT_FRAME_MARKER_LONGS] == STACK_FRAME_REGS_MARKER) {
struct pt_regs *regs = (struct pt_regs *)
- (sp + STACK_FRAME_OVERHEAD);
+ (sp + STACK_INT_FRAME_REGS);
lr = regs->link;
printk("%s--- interrupt: %lx at %pS\n",
int __initdata iommu_force_on;
unsigned long tce_alloc_start, tce_alloc_end;
u64 ppc64_rma_size;
+unsigned int boot_cpu_node_count __ro_after_init;
#endif
static phys_addr_t first_memblock_size;
static int __initdata boot_cpu_count;
if (type == NULL || strcmp(type, "cpu") != 0)
return 0;
+ if (IS_ENABLED(CONFIG_PPC64))
+ boot_cpu_node_count++;
+
/* Get physical cpuid */
intserv = of_get_flat_dt_prop(node, "ibm,ppc-interrupt-server#s", &len);
if (!intserv)
* Copyright (C) 2001 IBM.
*/
-#include <linux/stdarg.h>
-#include <linux/kernel.h>
-#include <linux/types.h>
-#include <linux/spinlock.h>
-#include <linux/export.h>
-#include <linux/init.h>
+#define pr_fmt(fmt) "rtas: " fmt
+
#include <linux/capability.h>
#include <linux/delay.h>
-#include <linux/cpu.h>
-#include <linux/sched.h>
-#include <linux/smp.h>
-#include <linux/completion.h>
-#include <linux/cpumask.h>
+#include <linux/export.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
#include <linux/memblock.h>
-#include <linux/slab.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
#include <linux/reboot.h>
+#include <linux/sched.h>
#include <linux/security.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/stdarg.h>
#include <linux/syscalls.h>
-#include <linux/of.h>
-#include <linux/of_fdt.h>
+#include <linux/types.h>
+#include <linux/uaccess.h>
+#include <asm/delay.h>
+#include <asm/firmware.h>
#include <asm/interrupt.h>
-#include <asm/rtas.h>
-#include <asm/hvcall.h>
#include <asm/machdep.h>
-#include <asm/firmware.h>
+#include <asm/mmu.h>
#include <asm/page.h>
-#include <asm/param.h>
-#include <asm/delay.h>
-#include <linux/uaccess.h>
-#include <asm/udbg.h>
-#include <asm/syscalls.h>
-#include <asm/smp.h>
-#include <linux/atomic.h>
+#include <asm/rtas.h>
#include <asm/time.h>
-#include <asm/mmu.h>
-#include <asm/topology.h>
+#include <asm/udbg.h>
/* This is here deliberately so it's only used in this file */
void enter_rtas(unsigned long);
EXPORT_SYMBOL(rtas_service_present);
#ifdef CONFIG_RTAS_ERROR_LOGGING
+
+static u32 rtas_error_log_max __ro_after_init = RTAS_ERROR_LOG_MAX;
+
/*
* Return the firmware-specified size of the error log buffer
* for all rtas calls that require an error buffer argument.
*/
int rtas_get_error_log_max(void)
{
- static int rtas_error_log_max;
- if (rtas_error_log_max)
- return rtas_error_log_max;
-
- rtas_error_log_max = rtas_token ("rtas-error-log-max");
- if ((rtas_error_log_max == RTAS_UNKNOWN_SERVICE) ||
- (rtas_error_log_max > RTAS_ERROR_LOG_MAX)) {
- printk (KERN_WARNING "RTAS: bad log buffer size %d\n",
- rtas_error_log_max);
- rtas_error_log_max = RTAS_ERROR_LOG_MAX;
- }
return rtas_error_log_max;
}
EXPORT_SYMBOL(rtas_get_error_log_max);
+static void __init init_error_log_max(void)
+{
+ static const char propname[] __initconst = "rtas-error-log-max";
+ u32 max;
+
+ if (of_property_read_u32(rtas.dev, propname, &max)) {
+ pr_warn("%s not found, using default of %u\n",
+ propname, RTAS_ERROR_LOG_MAX);
+ max = RTAS_ERROR_LOG_MAX;
+ }
+
+ if (max > RTAS_ERROR_LOG_MAX) {
+ pr_warn("%s = %u, clamping max error log size to %u\n",
+ propname, max, RTAS_ERROR_LOG_MAX);
+ max = RTAS_ERROR_LOG_MAX;
+ }
+
+ rtas_error_log_max = max;
+}
+
static char rtas_err_buf[RTAS_ERROR_LOG_MAX];
static int rtas_last_error_token;
#else /* CONFIG_RTAS_ERROR_LOGGING */
#define __fetch_rtas_last_error(x) NULL
#define get_errorlog_buffer() NULL
+static void __init init_error_log_max(void) {}
#endif
static int ibm_open_errinjct_token;
static int ibm_errinjct_token;
+/**
+ * rtas_call() - Invoke an RTAS firmware function.
+ * @token: Identifies the function being invoked.
+ * @nargs: Number of input parameters. Does not include token.
+ * @nret: Number of output parameters, including the call status.
+ * @outputs: Array of @nret output words.
+ * @....: List of @nargs input parameters.
+ *
+ * Invokes the RTAS function indicated by @token, which the caller
+ * should obtain via rtas_token().
+ *
+ * The @nargs and @nret arguments must match the number of input and
+ * output parameters specified for the RTAS function.
+ *
+ * rtas_call() returns RTAS status codes, not conventional Linux errno
+ * values. Callers must translate any failure to an appropriate errno
+ * in syscall context. Most callers of RTAS functions that can return
+ * -2 or 990x should use rtas_busy_delay() to correctly handle those
+ * statuses before calling again.
+ *
+ * The return value descriptions are adapted from 7.2.8 [RTAS] Return
+ * Codes of the PAPR and CHRP specifications.
+ *
+ * Context: Process context preferably, interrupt context if
+ * necessary. Acquires an internal spinlock and may perform
+ * GFP_ATOMIC slab allocation in error path. Unsafe for NMI
+ * context.
+ * Return:
+ * * 0 - RTAS function call succeeded.
+ * * -1 - RTAS function encountered a hardware or
+ * platform error, or the token is invalid,
+ * or the function is restricted by kernel policy.
+ * * -2 - Specs say "A necessary hardware device was busy,
+ * and the requested function could not be
+ * performed. The operation should be retried at
+ * a later time." This is misleading, at least with
+ * respect to current RTAS implementations. What it
+ * usually means in practice is that the function
+ * could not be completed while meeting RTAS's
+ * deadline for returning control to the OS (250us
+ * for PAPR/PowerVM, typically), but the call may be
+ * immediately reattempted to resume work on it.
+ * * -3 - Parameter error.
+ * * -7 - Unexpected state change.
+ * * 9000...9899 - Vendor-specific success codes.
+ * * 9900...9905 - Advisory extended delay. Caller should try
+ * again after ~10^x ms has elapsed, where x is
+ * the last digit of the status [0-5]. Again going
+ * beyond the PAPR text, 990x on PowerVM indicates
+ * contention for RTAS-internal resources. Other
+ * RTAS call sequences in progress should be
+ * allowed to complete before reattempting the
+ * call.
+ * * -9000 - Multi-level isolation error.
+ * * -9999...-9004 - Vendor-specific error codes.
+ * * Additional negative values - Function-specific error.
+ * * Additional positive values - Function-specific success.
+ */
int rtas_call(int token, int nargs, int nret, int *outputs, ...)
{
va_list list;
rc = -ENODEV;
break;
default:
- printk(KERN_ERR "%s: unexpected RTAS error %d\n",
- __func__, rtas_rc);
+ pr_err("%s: unexpected error %d\n", __func__, rtas_rc);
rc = -ERANGE;
break;
}
{
if (rtas_flash_term_hook)
rtas_flash_term_hook(SYS_RESTART);
- printk("RTAS system-reboot returned %d\n",
- rtas_call(rtas_token("system-reboot"), 0, 1, NULL));
+ pr_emerg("system-reboot returned %d\n",
+ rtas_call(rtas_token("system-reboot"), 0, 1, NULL));
for (;;);
}
if (rtas_flash_term_hook)
rtas_flash_term_hook(SYS_POWER_OFF);
/* allow power on only with power button press */
- printk("RTAS power-off returned %d\n",
- rtas_call(rtas_token("power-off"), 2, 1, NULL, -1, -1));
+ pr_emerg("power-off returned %d\n",
+ rtas_call(rtas_token("power-off"), 2, 1, NULL, -1, -1));
for (;;);
}
if (rtas_flash_term_hook)
rtas_flash_term_hook(SYS_HALT);
/* allow power on only with power button press */
- printk("RTAS power-off returned %d\n",
- rtas_call(rtas_token("power-off"), 2, 1, NULL, -1, -1));
+ pr_emerg("power-off returned %d\n",
+ rtas_call(rtas_token("power-off"), 2, 1, NULL, -1, -1));
for (;;);
}
/* Must be in the RMO region, so we place it here */
static char rtas_os_term_buf[2048];
+static s32 ibm_os_term_token = RTAS_UNKNOWN_SERVICE;
void rtas_os_term(char *str)
{
* this property may terminate the partition which we want to avoid
* since it interferes with panic_timeout.
*/
- if (RTAS_UNKNOWN_SERVICE == rtas_token("ibm,os-term") ||
- RTAS_UNKNOWN_SERVICE == rtas_token("ibm,extended-os-term"))
+ if (ibm_os_term_token == RTAS_UNKNOWN_SERVICE)
return;
snprintf(rtas_os_term_buf, 2048, "OS panic: %s", str);
+ /*
+ * Keep calling as long as RTAS returns a "try again" status,
+ * but don't use rtas_busy_delay(), which potentially
+ * schedules.
+ */
do {
- status = rtas_call(rtas_token("ibm,os-term"), 1, 1, NULL,
+ status = rtas_call(ibm_os_term_token, 1, 1, NULL,
__pa(rtas_os_term_buf));
- } while (rtas_busy_delay(status));
+ } while (rtas_busy_delay_time(status));
if (status != 0)
- printk(KERN_EMERG "ibm,os-term call failed %d\n", status);
+ pr_emerg("ibm,os-term call failed %d\n", status);
}
/**
return NULL;
}
-#ifdef CONFIG_PPC_RTAS_FILTER
-
/*
* The sys_rtas syscall, as originally designed, allows root to pass
* arbitrary physical addresses to RTAS calls. A number of RTAS calls
rtas_filters[i].token = rtas_token(rtas_filters[i].name);
}
-#else
-
-static bool block_rtas_call(int token, int nargs,
- struct rtas_args *args)
-{
- return false;
-}
-
-static void __init rtas_syscall_filter_init(void)
-{
-}
-
-#endif /* CONFIG_PPC_RTAS_FILTER */
-
/* We assume to be passed big endian arguments */
SYSCALL_DEFINE1(rtas, struct rtas_args __user *, uargs)
{
no_entry = of_property_read_u32(rtas.dev, "linux,rtas-entry", &entry);
rtas.entry = no_entry ? rtas.base : entry;
+ init_error_log_max();
+
+ /*
+ * Discover these now to avoid device tree lookups in the
+ * panic path.
+ */
+ if (of_property_read_bool(rtas.dev, "ibm,extended-os-term"))
+ ibm_os_term_token = rtas_token("ibm,os-term");
+
/* If RTAS was found, allocate the RMO buffer for it and look for
* the stop-self token if any
*/
#include <linux/errno.h>
#include <linux/sched.h>
#include <linux/kernel.h>
+#include <linux/of.h>
#include <linux/poll.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
static int __init rtas_event_scan_init(void)
{
+ int err;
+
if (!machine_is(pseries) && !machine_is(chrp))
return 0;
return -ENODEV;
}
- rtas_event_scan_rate = rtas_token("rtas-event-scan-rate");
- if (rtas_event_scan_rate == RTAS_UNKNOWN_SERVICE) {
+ err = of_property_read_u32(rtas.dev, "rtas-event-scan-rate", &rtas_event_scan_rate);
+ if (err) {
printk(KERN_ERR "rtasd: no rtas-event-scan-rate on system\n");
return -ENODEV;
}
#include <asm/xmon.h>
#include <asm/cputhreads.h>
#include <mm/mmu_decl.h>
+#include <asm/archrandom.h>
#include <asm/fadump.h>
#include <asm/udbg.h>
#include <asm/hugetlb.h>
#ifdef CONFIG_PPC64
paca_ptrs[cpu]->__current = idle;
paca_ptrs[cpu]->kstack = (unsigned long)task_stack_page(idle) +
- THREAD_SIZE - STACK_FRAME_OVERHEAD;
+ THREAD_SIZE - STACK_FRAME_MIN_SIZE;
#endif
task_thread_info(idle)->cpu = cpu;
secondary_current = current_set[cpu] = idle;
unsigned long *stack = (unsigned long *) sp;
unsigned long newsp, ip;
- if (!validate_sp(sp, task, STACK_FRAME_OVERHEAD))
+ if (!validate_sp(sp, task))
return;
newsp = stack[0];
/*
* For user tasks, this is the SP value loaded on
* kernel entry, see "PACAKSAVE(r13)" in _switch() and
- * system_call_common()/EXCEPTION_PROLOG_COMMON().
+ * system_call_common().
*
* Likewise for non-swapper kernel threads,
* this also happens to be the top of the stack
* an unreliable stack trace until it's been
* _switch()'ed to for the first time.
*/
- stack_end -= STACK_FRAME_OVERHEAD + sizeof(struct pt_regs);
+ stack_end -= STACK_USER_INT_FRAME_SIZE;
} else {
/*
* idle tasks have a custom stack layout,
* c.f. cpu_idle_thread_init().
*/
- stack_end -= STACK_FRAME_OVERHEAD;
+ stack_end -= STACK_FRAME_MIN_SIZE;
}
if (task == current)
/* Mark stacktraces with exception frames as unreliable. */
if (sp <= stack_end - STACK_INT_FRAME_SIZE &&
- stack[STACK_FRAME_MARKER] == STACK_FRAME_REGS_MARKER) {
+ stack[STACK_INT_FRAME_MARKER_LONGS] == STACK_FRAME_REGS_MARKER) {
return -EINVAL;
}
/* SPDX-License-Identifier: GPL-2.0 */
#include <linux/threads.h>
+#include <linux/linkage.h>
+
#include <asm/processor.h>
#include <asm/page.h>
#include <asm/cputable.h>
/* FIXME:This construct is actually not useful since we don't shut
* down the instruction MMU, we could just flip back MSR-DR on.
*/
-turn_on_mmu:
+SYM_FUNC_START_LOCAL(turn_on_mmu)
mflr r4
mtsrr0 r4
mtsrr1 r3
isync
rfi
_ASM_NOKPROBE_SYMBOL(turn_on_mmu)
+SYM_FUNC_END(turn_on_mmu)
unsigned long tb_ticks_per_usec = 100; /* sane default */
EXPORT_SYMBOL(tb_ticks_per_usec);
unsigned long tb_ticks_per_sec;
-EXPORT_SYMBOL(tb_ticks_per_sec); /* for cputime_t conversions */
+EXPORT_SYMBOL(tb_ticks_per_sec); /* for cputime conversions */
DEFINE_SPINLOCK(rtc_lock);
EXPORT_SYMBOL_GPL(rtc_lock);
#ifdef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE
/*
- * Factor for converting from cputime_t (timebase ticks) to
- * microseconds. This is stored as 0.64 fixed-point binary fraction.
- */
-u64 __cputime_usec_factor;
-EXPORT_SYMBOL(__cputime_usec_factor);
-
-static void calc_cputime_factors(void)
-{
- struct div_result res;
-
- div128_by_32(1000000, 0, tb_ticks_per_sec, &res);
- __cputime_usec_factor = res.result_low;
-}
-
-/*
* Read the SPURR on systems that have it, otherwise the PURR,
* or if that doesn't exist return the timebase value passed in.
*/
acct->hardirq_time = 0;
acct->softirq_time = 0;
}
-
-#else /* ! CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */
-#define calc_cputime_factors()
-#endif
+#endif /* CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */
void __delay(unsigned long loops)
{
tb_ticks_per_jiffy = ppc_tb_freq / HZ;
tb_ticks_per_sec = ppc_tb_freq;
tb_ticks_per_usec = ppc_tb_freq / 1000000;
- calc_cputime_factors();
/*
* Compute scale factor for sched_clock.
std r2, STK_GOT(r1)
stdu r1, -TM_FRAME_SIZE(r1)
- /* We've a struct pt_regs at [r1+STACK_FRAME_OVERHEAD]. */
+ /* We've a struct pt_regs at [r1+STACK_INT_FRAME_REGS]. */
std r3, STK_PARAM(R3)(r1)
SAVE_NVGPRS(r1)
* Make r7 look like an exception frame so that we can use the neat
* GPRx(n) macros. r7 is NOT a pt_regs ptr!
*/
- subi r7, r7, STACK_FRAME_OVERHEAD
+ subi r7, r7, STACK_INT_FRAME_REGS
/* Sync the userland GPRs 2-12, 14-31 to thread->regs: */
SAVE_GPR(0, r7) /* user r0 */
stdu r1, -TM_FRAME_SIZE(r1)
/*
- * We've a struct pt_regs at [r1+STACK_FRAME_OVERHEAD].
+ * We've a struct pt_regs at [r1+STACK_INT_FRAME_REGS].
* This is used for backing up the NVGPRs:
*/
SAVE_NVGPRS(r1)
* Make r7 look like an exception frame so that we can use the neat
* GPRx(n) macros. r7 is now NOT a pt_regs ptr!
*/
- subi r7, r7, STACK_FRAME_OVERHEAD
+ subi r7, r7, STACK_INT_FRAME_REGS
/* We need to setup MSR for FP/VMX/VSX register save instructions. */
mfmsr r6
.endif
/* Load &pt_regs in r6 for call below */
- addi r6, r1, STACK_FRAME_OVERHEAD
+ addi r6, r1, STACK_INT_FRAME_REGS
.endm
.macro ftrace_regs_exit allregs
cmd_vdso64ld_and_check = $(VDSOCC) $(c_flags) $(CC64FLAGS) -o $@ -Wl,-T$(filter %.lds,$^) $(filter %.o,$^) -z noexecstack ; $(cmd_vdso_check)
quiet_cmd_vdso64as = VDSO64A $@
cmd_vdso64as = $(VDSOCC) $(a_flags) $(CC64FLAGS) $(AS64FLAGS) -c -o $@ $<
+
+OBJECT_FILES_NON_STANDARD := y
/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/linkage.h>
#include <asm/processor.h>
#include <asm/ppc_asm.h>
#include <asm/reg.h>
* Internal routine to enable floating point and set FPSCR to 0.
* Don't call it from C; it doesn't use the normal calling convention.
*/
-fpenable:
+SYM_FUNC_START_LOCAL(fpenable)
#ifdef CONFIG_PPC32
stwu r1,-64(r1)
#else
mffs fr31
MTFSF_L(fr1)
blr
+SYM_FUNC_END(fpenable)
fpdisable:
mtlr r12
#include <asm/firmware.h>
#include <asm/kexec_ranges.h>
#include <asm/crashdump-ppc64.h>
+#include <asm/prom.h>
struct umem_info {
u64 *buf; /* data buffer for usable-memory property */
}
/**
+ * get_cpu_node_size - Compute the size of a CPU node in the FDT.
+ * This should be done only once and the value is stored in
+ * a static variable.
+ * Returns the max size of a CPU node in the FDT.
+ */
+static unsigned int cpu_node_size(void)
+{
+ static unsigned int size;
+ struct device_node *dn;
+ struct property *pp;
+
+ /*
+ * Don't compute it twice, we are assuming that the per CPU node size
+ * doesn't change during the system's life.
+ */
+ if (size)
+ return size;
+
+ dn = of_find_node_by_type(NULL, "cpu");
+ if (WARN_ON_ONCE(!dn)) {
+ // Unlikely to happen
+ return 0;
+ }
+
+ /*
+ * We compute the sub node size for a CPU node, assuming it
+ * will be the same for all.
+ */
+ size += strlen(dn->name) + 5;
+ for_each_property_of_node(dn, pp) {
+ size += strlen(pp->name);
+ size += pp->length;
+ }
+
+ of_node_put(dn);
+ return size;
+}
+
+/**
* kexec_extra_fdt_size_ppc64 - Return the estimated additional size needed to
* setup FDT for kexec/kdump kernel.
* @image: kexec image being loaded.
*/
unsigned int kexec_extra_fdt_size_ppc64(struct kimage *image)
{
+ unsigned int cpu_nodes, extra_size;
+ struct device_node *dn;
u64 usm_entries;
if (image->type != KEXEC_TYPE_CRASH)
*/
usm_entries = ((memblock_end_of_DRAM() / drmem_lmb_size()) +
(2 * (resource_size(&crashk_res) / drmem_lmb_size())));
- return (unsigned int)(usm_entries * sizeof(u64));
+
+ extra_size = (unsigned int)(usm_entries * sizeof(u64));
+
+ /*
+ * Get the number of CPU nodes in the current DT. This allows to
+ * reserve places for CPU nodes added since the boot time.
+ */
+ cpu_nodes = 0;
+ for_each_node_by_type(dn, "cpu") {
+ cpu_nodes++;
+ }
+
+ if (cpu_nodes > boot_cpu_node_count)
+ extra_size += (cpu_nodes - boot_cpu_node_count) * cpu_node_size();
+
+ return extra_size;
}
/**
if (rc < 0)
return rc;
- resize_hpt_debug(resize, "resize_hpt_allocate(): HPT @ 0x%lx\n",
+ resize_hpt_debug(resize, "%s(): HPT @ 0x%lx\n", __func__,
resize->hpt.virt);
return 0;
*/
mutex_unlock(&kvm->arch.mmu_setup_lock);
- resize_hpt_debug(resize, "resize_hpt_prepare_work(): order = %d\n",
+ resize_hpt_debug(resize, "%s(): order = %d\n", __func__,
resize->order);
err = resize_hpt_allocate(resize);
ret = kvmppc_virtmode_do_h_enter(kvm, H_EXACT, i, v, r,
tmp);
if (ret != H_SUCCESS) {
- pr_err("kvm_htab_write ret %ld i=%ld v=%lx "
- "r=%lx\n", ret, i, v, r);
+ pr_err("%s ret %ld i=%ld v=%lx r=%lx\n", __func__, ret, i, v, r);
goto out;
}
if (!mmu_ready && is_vrma_hpte(v)) {
struct kvmppc_spapr_tce_table *stt = NULL;
struct kvmppc_spapr_tce_table *siter;
struct mm_struct *mm = kvm->mm;
- unsigned long npages, size = args->size;
+ unsigned long npages;
int ret;
if (!args->size || args->page_shift < 12 || args->page_shift > 34 ||
(args->offset + args->size > (ULLONG_MAX >> args->page_shift)))
return -EINVAL;
- npages = kvmppc_tce_pages(size);
+ npages = kvmppc_tce_pages(args->size);
ret = account_locked_vm(mm, kvmppc_stt_pages(npages), true);
if (ret)
return ret;
stt->liobn = args->liobn;
stt->page_shift = args->page_shift;
stt->offset = args->offset;
- stt->size = size;
+ stt->size = args->size;
stt->kvm = kvm;
mutex_init(&stt->alloc_lock);
INIT_LIST_HEAD_RCU(&stt->iommu_tables);
* Authors: Alexander Graf <agraf@suse.de>
*/
+#include <linux/linkage.h>
#include <asm/ppc_asm.h>
#include <asm/kvm_asm.h>
#include <asm/reg.h>
/*
* void kvmhv_save_host_pmu(void)
*/
-kvmhv_save_host_pmu:
+SYM_FUNC_START_LOCAL(kvmhv_save_host_pmu)
BEGIN_FTR_SECTION
/* Work around P8 PMAE bug */
li r3, -1
stw r8, HSTATE_PMC5(r13)
stw r9, HSTATE_PMC6(r13)
31: blr
+SYM_FUNC_END(kvmhv_save_host_pmu)
* Authors: Alexander Graf <agraf@suse.de>
*/
+#include <linux/linkage.h>
+#include <linux/objtool.h>
#include <asm/ppc_asm.h>
#include <asm/code-patching-asm.h>
#include <asm/kvm_asm.h>
/* Flush the link stack. On Power8 it's up to 32 entries in size. */
.rept 32
+ ANNOTATE_INTRA_FUNCTION_CALL
bl .+4
.endr
/* And on Power9 it's up to 64. */
BEGIN_FTR_SECTION
.rept 32
+ ANNOTATE_INTRA_FUNCTION_CALL
bl .+4
.endr
END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
* This routine calls kvmppc_read_intr, a C function, if an external
* interrupt is pending.
*/
-kvmppc_check_wake_reason:
+SYM_FUNC_START_LOCAL(kvmppc_check_wake_reason)
mfspr r6, SPRN_SRR1
BEGIN_FTR_SECTION
rlwinm r6, r6, 45-31, 0xf /* extract wake reason field (P8) */
addi r1, r1, PPC_MIN_STKFRM
mtlr r0
blr
+SYM_FUNC_END(kvmppc_check_wake_reason)
/*
* Save away FP, VMX and VSX registers.
* N.B. r30 and r31 are volatile across this function,
* thus it is not callable from C.
*/
-kvmppc_save_fp:
+SYM_FUNC_START_LOCAL(kvmppc_save_fp)
mflr r30
mr r31,r3
mfmsr r5
stw r6,VCPU_VRSAVE(r31)
mtlr r30
blr
+SYM_FUNC_END(kvmppc_save_fp)
/*
* Load up FP, VMX and VSX registers
* N.B. r30 and r31 are volatile across this function,
* thus it is not callable from C.
*/
-kvmppc_load_fp:
+SYM_FUNC_START_LOCAL(kvmppc_load_fp)
mflr r30
mr r31,r4
mfmsr r9
mtlr r30
mr r4,r31
blr
+SYM_FUNC_END(kvmppc_load_fp)
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
/*
std r6, SOFTE(r1)
LOAD_PACA_TOC()
LOAD_REG_IMMEDIATE(3, STACK_FRAME_REGS_MARKER)
- std r3, STACK_FRAME_OVERHEAD-16(r1)
+ std r3, STACK_INT_FRAME_MARKER(r1)
/*
* XXX On POWER7 and POWER8, we just spin here since we don't
* r9 has a vcpu pointer (in)
* r0 is used as a scratch register
*/
-kvmppc_msr_interrupt:
+SYM_FUNC_START_LOCAL(kvmppc_msr_interrupt)
rldicl r0, r11, 64 - MSR_TS_S_LG, 62
cmpwi r0, 2 /* Check if we are in transactional state.. */
ld r11, VCPU_INTR_MSR(r9)
li r0, 1
1: rldimi r11, r0, MSR_TS_S_LG, 63 - MSR_TS_T_LG
blr
+SYM_FUNC_END(kvmppc_msr_interrupt)
/*
* void kvmhv_load_guest_pmu(struct kvm_vcpu *vcpu)
*
* Load up guest PMU state. R3 points to the vcpu struct.
*/
-kvmhv_load_guest_pmu:
+SYM_FUNC_START_LOCAL(kvmhv_load_guest_pmu)
mr r4, r3
mflr r0
li r3, 1
isync
mtlr r0
blr
+SYM_FUNC_END(kvmhv_load_guest_pmu)
/*
* void kvmhv_load_host_pmu(void)
*
* Reload host PMU state saved in the PACA by kvmhv_save_host_pmu.
*/
-kvmhv_load_host_pmu:
+SYM_FUNC_START_LOCAL(kvmhv_load_host_pmu)
mflr r0
lbz r4, PACA_PMCINUSE(r13) /* is the host using the PMU? */
cmpwi r4, 0
isync
mtlr r0
23: blr
+SYM_FUNC_END(kvmhv_load_host_pmu)
/*
* void kvmhv_save_guest_pmu(struct kvm_vcpu *vcpu, bool pmu_in_use)
* Save guest PMU state into the vcpu struct.
* r3 = vcpu, r4 = full save flag (PMU in use flag set in VPA)
*/
-kvmhv_save_guest_pmu:
+SYM_FUNC_START_LOCAL(kvmhv_save_guest_pmu)
mr r9, r3
mr r8, r4
BEGIN_FTR_SECTION
mtspr SPRN_MMCRS, r4
END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
22: blr
+SYM_FUNC_END(kvmhv_save_guest_pmu)
/*
* This works around a hardware bug on POWER8E processors, where
pfn_first = res->start >> PAGE_SHIFT;
pfn_last = pfn_first + (resource_size(res) >> PAGE_SHIFT);
- kvmppc_uvmem_bitmap = kcalloc(BITS_TO_LONGS(pfn_last - pfn_first),
- sizeof(unsigned long), GFP_KERNEL);
+ kvmppc_uvmem_bitmap = bitmap_zalloc(pfn_last - pfn_first, GFP_KERNEL);
if (!kvmppc_uvmem_bitmap) {
ret = -ENOMEM;
goto out_unmap;
memunmap_pages(&kvmppc_uvmem_pgmap);
release_mem_region(kvmppc_uvmem_pgmap.range.start,
range_len(&kvmppc_uvmem_pgmap.range));
- kfree(kvmppc_uvmem_bitmap);
+ bitmap_free(kvmppc_uvmem_bitmap);
}
if (irq == XICS_IPI || irq == 0) {
/*
* This barrier orders the setting of xc->cppr vs.
- * subsquent test of xc->mfrr done inside
+ * subsequent test of xc->mfrr done inside
* scan_interrupts and push_pending_to_hw
*/
smp_mb();
/*
* This barrier orders both setting of in_eoi above vs,
* subsequent test of guest_priority, and the setting
- * of xc->cppr vs. subsquent test of xc->mfrr done inside
+ * of xc->cppr vs. subsequent test of xc->mfrr done inside
* scan_interrupts and push_pending_to_hw
*/
smp_mb();
* stale_p (because it has no easy way to address it). Hence we have
* to adjust stale_p before shutting down the interrupt.
*/
-void xive_cleanup_single_escalation(struct kvm_vcpu *vcpu,
- struct kvmppc_xive_vcpu *xc, int irq)
+void xive_cleanup_single_escalation(struct kvm_vcpu *vcpu, int irq)
{
struct irq_data *d = irq_get_irq_data(irq);
struct xive_irq_data *xd = irq_data_get_irq_handler_data(d);
for (i = 0; i < KVMPPC_XIVE_Q_COUNT; i++) {
if (xc->esc_virq[i]) {
if (kvmppc_xive_has_single_escalation(xc->xive))
- xive_cleanup_single_escalation(vcpu, xc,
- xc->esc_virq[i]);
+ xive_cleanup_single_escalation(vcpu, xc->esc_virq[i]);
free_irq(xc->esc_virq[i], vcpu);
irq_dispose_mapping(xc->esc_virq[i]);
kfree(xc->esc_virq_names[i]);
/*
* Now, we select a target if we have one. If we don't we
* leave the interrupt untargetted. It means that an interrupt
- * can become "untargetted" accross migration if it was masked
+ * can become "untargetted" across migration if it was masked
* by set_xive() but there is little we can do about it.
*/
int kvmppc_xive_attach_escalation(struct kvm_vcpu *vcpu, u8 prio,
bool single_escalation);
struct kvmppc_xive *kvmppc_xive_get_device(struct kvm *kvm, u32 type);
-void xive_cleanup_single_escalation(struct kvm_vcpu *vcpu,
- struct kvmppc_xive_vcpu *xc, int irq);
+void xive_cleanup_single_escalation(struct kvm_vcpu *vcpu, int irq);
int kvmppc_xive_compute_vp_id(struct kvmppc_xive *xive, u32 cpu, u32 *vp);
int kvmppc_xive_set_nr_servers(struct kvmppc_xive *xive, u64 addr);
bool kvmppc_xive_check_save_restore(struct kvm_vcpu *vcpu);
/* Free the escalation irq */
if (xc->esc_virq[i]) {
if (kvmppc_xive_has_single_escalation(xc->xive))
- xive_cleanup_single_escalation(vcpu, xc,
- xc->esc_virq[i]);
+ xive_cleanup_single_escalation(vcpu, xc->esc_virq[i]);
free_irq(xc->esc_virq[i], vcpu);
irq_dispose_mapping(xc->esc_virq[i]);
kfree(xc->esc_virq_names[i]);
u32 last_inst = KVM_INST_FETCH_FAILED;
enum emulation_result emulated = EMULATE_DONE;
+ /* Fix irq state (pairs with kvmppc_fix_ee_before_entry()) */
+ kvmppc_fix_ee_after_exit();
+
/* update before a new last_exit_type is rewritten */
kvmppc_update_timing_stats(vcpu);
mtspr SPRN_EPCR, r3
isync
-#ifdef CONFIG_64BIT
- /*
- * We enter with interrupts disabled in hardware, but
- * we need to call RECONCILE_IRQ_STATE to ensure
- * that the software state is kept in sync.
- */
- RECONCILE_IRQ_STATE(r3,r5)
-#endif
-
/* Switch to kernel stack and jump to handler. */
mr r3, r4
mr r5, r14 /* intno */
*/
#include <linux/pgtable.h>
+#include <linux/linkage.h>
+
#include <asm/reg.h>
#include <asm/page.h>
#include <asm/mmu.h>
* R8 = (double*)¶m3 [load_three]
* LR = instruction call function
*/
-fpd_load_three:
+SYM_FUNC_START_LOCAL(fpd_load_three)
lfd 2,0(r8) /* load param3 */
-fpd_load_two:
+SYM_FUNC_START_LOCAL(fpd_load_two)
lfd 1,0(r7) /* load param2 */
-fpd_load_one:
+SYM_FUNC_START_LOCAL(fpd_load_one)
lfd 0,0(r6) /* load param1 */
-fpd_load_none:
+SYM_FUNC_START_LOCAL(fpd_load_none)
lfd 3,0(r3) /* load up fpscr value */
MTFSF_L(3)
lwz r6, 0(r4) /* load cr */
mtcr r6
blr
+SYM_FUNC_END(fpd_load_none)
+SYM_FUNC_END(fpd_load_one)
+SYM_FUNC_END(fpd_load_two)
+SYM_FUNC_END(fpd_load_three)
/*
* End of double instruction processing
* R5 = (double*)&result
* LR = caller of instruction call function
*/
-fpd_return:
+SYM_FUNC_START_LOCAL(fpd_return)
mfcr r6
stfd 0,0(r5) /* save result */
mffs 0
stfd 0,0(r3) /* save new fpscr value */
stw r6,0(r4) /* save new cr value */
blr
+SYM_FUNC_END(fpd_return)
/*
* Double operation with no input operand
obj64-y += copypage_64.o copyuser_64.o mem_64.o hweight_64.o \
memcpy_64.o copy_mc_64.o
-ifndef CONFIG_PPC_QUEUED_SPINLOCKS
+ifdef CONFIG_PPC_QUEUED_SPINLOCKS
+obj-$(CONFIG_SMP) += qspinlock.o
+else
obj64-$(CONFIG_SMP) += locks.o
endif
*/
#include <linux/kprobes.h>
+#include <linux/mmu_context.h>
+#include <linux/random.h>
#include <linux/vmalloc.h>
#include <linux/init.h>
#include <linux/cpuhotplug.h>
#include <linux/uaccess.h>
#include <linux/jump_label.h>
+#include <asm/debug.h>
+#include <asm/pgalloc.h>
+#include <asm/tlb.h>
#include <asm/tlbflush.h>
#include <asm/page.h>
#include <asm/code-patching.h>
return __patch_instruction(addr, instr, addr);
}
-#ifdef CONFIG_STRICT_KERNEL_RWX
-static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);
+struct patch_context {
+ union {
+ struct vm_struct *area;
+ struct mm_struct *mm;
+ };
+ unsigned long addr;
+ pte_t *pte;
+};
+
+static DEFINE_PER_CPU(struct patch_context, cpu_patching_context);
static int map_patch_area(void *addr, unsigned long text_poke_addr);
static void unmap_patch_area(unsigned long addr);
+static bool mm_patch_enabled(void)
+{
+ return IS_ENABLED(CONFIG_SMP) && radix_enabled();
+}
+
+/*
+ * The following applies for Radix MMU. Hash MMU has different requirements,
+ * and so is not supported.
+ *
+ * Changing mm requires context synchronising instructions on both sides of
+ * the context switch, as well as a hwsync between the last instruction for
+ * which the address of an associated storage access was translated using
+ * the current context.
+ *
+ * switch_mm_irqs_off() performs an isync after the context switch. It is
+ * the responsibility of the caller to perform the CSI and hwsync before
+ * starting/stopping the temp mm.
+ */
+static struct mm_struct *start_using_temp_mm(struct mm_struct *temp_mm)
+{
+ struct mm_struct *orig_mm = current->active_mm;
+
+ lockdep_assert_irqs_disabled();
+ switch_mm_irqs_off(orig_mm, temp_mm, current);
+
+ WARN_ON(!mm_is_thread_local(temp_mm));
+
+ suspend_breakpoints();
+ return orig_mm;
+}
+
+static void stop_using_temp_mm(struct mm_struct *temp_mm,
+ struct mm_struct *orig_mm)
+{
+ lockdep_assert_irqs_disabled();
+ switch_mm_irqs_off(temp_mm, orig_mm, current);
+ restore_breakpoints();
+}
+
static int text_area_cpu_up(unsigned int cpu)
{
struct vm_struct *area;
unmap_patch_area(addr);
- this_cpu_write(text_poke_area, area);
+ this_cpu_write(cpu_patching_context.area, area);
+ this_cpu_write(cpu_patching_context.addr, addr);
+ this_cpu_write(cpu_patching_context.pte, virt_to_kpte(addr));
return 0;
}
static int text_area_cpu_down(unsigned int cpu)
{
- free_vm_area(this_cpu_read(text_poke_area));
+ free_vm_area(this_cpu_read(cpu_patching_context.area));
+ this_cpu_write(cpu_patching_context.area, NULL);
+ this_cpu_write(cpu_patching_context.addr, 0);
+ this_cpu_write(cpu_patching_context.pte, NULL);
+ return 0;
+}
+
+static void put_patching_mm(struct mm_struct *mm, unsigned long patching_addr)
+{
+ struct mmu_gather tlb;
+
+ tlb_gather_mmu(&tlb, mm);
+ free_pgd_range(&tlb, patching_addr, patching_addr + PAGE_SIZE, 0, 0);
+ mmput(mm);
+}
+
+static int text_area_cpu_up_mm(unsigned int cpu)
+{
+ struct mm_struct *mm;
+ unsigned long addr;
+ pte_t *pte;
+ spinlock_t *ptl;
+
+ mm = mm_alloc();
+ if (WARN_ON(!mm))
+ goto fail_no_mm;
+
+ /*
+ * Choose a random page-aligned address from the interval
+ * [PAGE_SIZE .. DEFAULT_MAP_WINDOW - PAGE_SIZE].
+ * The lower address bound is PAGE_SIZE to avoid the zero-page.
+ */
+ addr = (1 + (get_random_long() % (DEFAULT_MAP_WINDOW / PAGE_SIZE - 2))) << PAGE_SHIFT;
+
+ /*
+ * PTE allocation uses GFP_KERNEL which means we need to
+ * pre-allocate the PTE here because we cannot do the
+ * allocation during patching when IRQs are disabled.
+ *
+ * Using get_locked_pte() to avoid open coding, the lock
+ * is unnecessary.
+ */
+ pte = get_locked_pte(mm, addr, &ptl);
+ if (!pte)
+ goto fail_no_pte;
+ pte_unmap_unlock(pte, ptl);
+
+ this_cpu_write(cpu_patching_context.mm, mm);
+ this_cpu_write(cpu_patching_context.addr, addr);
+
+ return 0;
+
+fail_no_pte:
+ put_patching_mm(mm, addr);
+fail_no_mm:
+ return -ENOMEM;
+}
+
+static int text_area_cpu_down_mm(unsigned int cpu)
+{
+ put_patching_mm(this_cpu_read(cpu_patching_context.mm),
+ this_cpu_read(cpu_patching_context.addr));
+
+ this_cpu_write(cpu_patching_context.mm, NULL);
+ this_cpu_write(cpu_patching_context.addr, 0);
+
return 0;
}
static __ro_after_init DEFINE_STATIC_KEY_FALSE(poking_init_done);
-/*
- * Although BUG_ON() is rude, in this case it should only happen if ENOMEM, and
- * we judge it as being preferable to a kernel that will crash later when
- * someone tries to use patch_instruction().
- */
void __init poking_init(void)
{
- BUG_ON(!cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
- "powerpc/text_poke:online", text_area_cpu_up,
- text_area_cpu_down));
+ int ret;
+
+ if (!IS_ENABLED(CONFIG_STRICT_KERNEL_RWX))
+ return;
+
+ if (mm_patch_enabled())
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+ "powerpc/text_poke_mm:online",
+ text_area_cpu_up_mm,
+ text_area_cpu_down_mm);
+ else
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+ "powerpc/text_poke:online",
+ text_area_cpu_up,
+ text_area_cpu_down);
+
+ /* cpuhp_setup_state returns >= 0 on success */
+ if (WARN_ON(ret < 0))
+ return;
+
static_branch_enable(&poking_init_done);
}
flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
}
+static int __do_patch_instruction_mm(u32 *addr, ppc_inst_t instr)
+{
+ int err;
+ u32 *patch_addr;
+ unsigned long text_poke_addr;
+ pte_t *pte;
+ unsigned long pfn = get_patch_pfn(addr);
+ struct mm_struct *patching_mm;
+ struct mm_struct *orig_mm;
+ spinlock_t *ptl;
+
+ patching_mm = __this_cpu_read(cpu_patching_context.mm);
+ text_poke_addr = __this_cpu_read(cpu_patching_context.addr);
+ patch_addr = (u32 *)(text_poke_addr + offset_in_page(addr));
+
+ pte = get_locked_pte(patching_mm, text_poke_addr, &ptl);
+ if (!pte)
+ return -ENOMEM;
+
+ __set_pte_at(patching_mm, text_poke_addr, pte, pfn_pte(pfn, PAGE_KERNEL), 0);
+
+ /* order PTE update before use, also serves as the hwsync */
+ asm volatile("ptesync": : :"memory");
+
+ /* order context switch after arbitrary prior code */
+ isync();
+
+ orig_mm = start_using_temp_mm(patching_mm);
+
+ err = __patch_instruction(addr, instr, patch_addr);
+
+ /* hwsync performed by __patch_instruction (sync) if successful */
+ if (err)
+ mb(); /* sync */
+
+ /* context synchronisation performed by __patch_instruction (isync or exception) */
+ stop_using_temp_mm(patching_mm, orig_mm);
+
+ pte_clear(patching_mm, text_poke_addr, pte);
+ /*
+ * ptesync to order PTE update before TLB invalidation done
+ * by radix__local_flush_tlb_page_psize (in _tlbiel_va)
+ */
+ local_flush_tlb_page_psize(patching_mm, text_poke_addr, mmu_virtual_psize);
+
+ pte_unmap_unlock(pte, ptl);
+
+ return err;
+}
+
static int __do_patch_instruction(u32 *addr, ppc_inst_t instr)
{
int err;
pte_t *pte;
unsigned long pfn = get_patch_pfn(addr);
- text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr & PAGE_MASK;
+ text_poke_addr = (unsigned long)__this_cpu_read(cpu_patching_context.addr) & PAGE_MASK;
patch_addr = (u32 *)(text_poke_addr + offset_in_page(addr));
- pte = virt_to_kpte(text_poke_addr);
+ pte = __this_cpu_read(cpu_patching_context.pte);
__set_pte_at(&init_mm, text_poke_addr, pte, pfn_pte(pfn, PAGE_KERNEL), 0);
/* See ptesync comment in radix__set_pte_at() */
if (radix_enabled())
return err;
}
-static int do_patch_instruction(u32 *addr, ppc_inst_t instr)
+int patch_instruction(u32 *addr, ppc_inst_t instr)
{
int err;
unsigned long flags;
* when text_poke_area is not ready, but we still need
* to allow patching. We just do the plain old patching
*/
- if (!static_branch_likely(&poking_init_done))
+ if (!IS_ENABLED(CONFIG_STRICT_KERNEL_RWX) ||
+ !static_branch_likely(&poking_init_done))
return raw_patch_instruction(addr, instr);
local_irq_save(flags);
- err = __do_patch_instruction(addr, instr);
+ if (mm_patch_enabled())
+ err = __do_patch_instruction_mm(addr, instr);
+ else
+ err = __do_patch_instruction(addr, instr);
local_irq_restore(flags);
return err;
}
-#else /* !CONFIG_STRICT_KERNEL_RWX */
-
-static int do_patch_instruction(u32 *addr, ppc_inst_t instr)
-{
- return raw_patch_instruction(addr, instr);
-}
-
-#endif /* CONFIG_STRICT_KERNEL_RWX */
-
-__ro_after_init DEFINE_STATIC_KEY_FALSE(init_mem_is_free);
-
-int patch_instruction(u32 *addr, ppc_inst_t instr)
-{
- /* Make sure we aren't patching a freed init section */
- if (static_branch_likely(&init_mem_is_free) && init_section_contains(addr, 4))
- return 0;
-
- return do_patch_instruction(addr, instr);
-}
NOKPROBE_SYMBOL(patch_instruction);
int patch_branch(u32 *addr, unsigned long target, int flags)
}
}
+#ifdef CONFIG_PPC_BARRIER_NOSPEC
+static bool is_fixup_addr_valid(void *dest, size_t size)
+{
+ return system_state < SYSTEM_FREEING_INITMEM ||
+ !init_section_contains(dest, size);
+}
+
+static int do_patch_fixups(long *start, long *end, unsigned int *instrs, int num)
+{
+ int i;
+
+ for (i = 0; start < end; start++, i++) {
+ int j;
+ unsigned int *dest = (void *)start + *start;
+
+ if (!is_fixup_addr_valid(dest, sizeof(*instrs) * num))
+ continue;
+
+ pr_devel("patching dest %lx\n", (unsigned long)dest);
+
+ for (j = 0; j < num; j++)
+ patch_instruction(dest + j, ppc_inst(instrs[j]));
+ }
+ return i;
+}
+#endif
+
#ifdef CONFIG_PPC_BOOK3S_64
+static int do_patch_entry_fixups(long *start, long *end, unsigned int *instrs,
+ bool do_fallback, void *fallback)
+{
+ int i;
+
+ for (i = 0; start < end; start++, i++) {
+ unsigned int *dest = (void *)start + *start;
+
+ if (!is_fixup_addr_valid(dest, sizeof(*instrs) * 3))
+ continue;
+
+ pr_devel("patching dest %lx\n", (unsigned long)dest);
+
+ // See comment in do_entry_flush_fixups() RE order of patching
+ if (do_fallback) {
+ patch_instruction(dest, ppc_inst(instrs[0]));
+ patch_instruction(dest + 2, ppc_inst(instrs[2]));
+ patch_branch(dest + 1, (unsigned long)fallback, BRANCH_SET_LINK);
+ } else {
+ patch_instruction(dest + 1, ppc_inst(instrs[1]));
+ patch_instruction(dest + 2, ppc_inst(instrs[2]));
+ patch_instruction(dest, ppc_inst(instrs[0]));
+ }
+ }
+ return i;
+}
+
static void do_stf_entry_barrier_fixups(enum stf_barrier_type types)
{
- unsigned int instrs[3], *dest;
+ unsigned int instrs[3];
long *start, *end;
int i;
instrs[i++] = PPC_RAW_ORI(_R31, _R31, 0); /* speculation barrier */
}
- for (i = 0; start < end; start++, i++) {
- dest = (void *)start + *start;
-
- pr_devel("patching dest %lx\n", (unsigned long)dest);
-
- // See comment in do_entry_flush_fixups() RE order of patching
- if (types & STF_BARRIER_FALLBACK) {
- patch_instruction(dest, ppc_inst(instrs[0]));
- patch_instruction(dest + 2, ppc_inst(instrs[2]));
- patch_branch(dest + 1,
- (unsigned long)&stf_barrier_fallback, BRANCH_SET_LINK);
- } else {
- patch_instruction(dest + 1, ppc_inst(instrs[1]));
- patch_instruction(dest + 2, ppc_inst(instrs[2]));
- patch_instruction(dest, ppc_inst(instrs[0]));
- }
- }
+ i = do_patch_entry_fixups(start, end, instrs, types & STF_BARRIER_FALLBACK,
+ &stf_barrier_fallback);
printk(KERN_DEBUG "stf-barrier: patched %d entry locations (%s barrier)\n", i,
(types == STF_BARRIER_NONE) ? "no" :
static void do_stf_exit_barrier_fixups(enum stf_barrier_type types)
{
- unsigned int instrs[6], *dest;
+ unsigned int instrs[6];
long *start, *end;
int i;
instrs[i++] = PPC_RAW_EIEIO() | 0x02000000; /* eieio + bit 6 hint */
}
- for (i = 0; start < end; start++, i++) {
- dest = (void *)start + *start;
+ i = do_patch_fixups(start, end, instrs, ARRAY_SIZE(instrs));
- pr_devel("patching dest %lx\n", (unsigned long)dest);
-
- patch_instruction(dest, ppc_inst(instrs[0]));
- patch_instruction(dest + 1, ppc_inst(instrs[1]));
- patch_instruction(dest + 2, ppc_inst(instrs[2]));
- patch_instruction(dest + 3, ppc_inst(instrs[3]));
- patch_instruction(dest + 4, ppc_inst(instrs[4]));
- patch_instruction(dest + 5, ppc_inst(instrs[5]));
- }
printk(KERN_DEBUG "stf-barrier: patched %d exit locations (%s barrier)\n", i,
(types == STF_BARRIER_NONE) ? "no" :
(types == STF_BARRIER_FALLBACK) ? "fallback" :
void do_uaccess_flush_fixups(enum l1d_flush_type types)
{
- unsigned int instrs[4], *dest;
+ unsigned int instrs[4];
long *start, *end;
int i;
if (types & L1D_FLUSH_MTTRIG)
instrs[i++] = PPC_RAW_MTSPR(SPRN_TRIG2, _R0);
- for (i = 0; start < end; start++, i++) {
- dest = (void *)start + *start;
-
- pr_devel("patching dest %lx\n", (unsigned long)dest);
-
- patch_instruction(dest, ppc_inst(instrs[0]));
-
- patch_instruction(dest + 1, ppc_inst(instrs[1]));
- patch_instruction(dest + 2, ppc_inst(instrs[2]));
- patch_instruction(dest + 3, ppc_inst(instrs[3]));
- }
+ i = do_patch_fixups(start, end, instrs, ARRAY_SIZE(instrs));
printk(KERN_DEBUG "uaccess-flush: patched %d locations (%s flush)\n", i,
(types == L1D_FLUSH_NONE) ? "no" :
static int __do_entry_flush_fixups(void *data)
{
enum l1d_flush_type types = *(enum l1d_flush_type *)data;
- unsigned int instrs[3], *dest;
+ unsigned int instrs[3];
long *start, *end;
int i;
start = PTRRELOC(&__start___entry_flush_fixup);
end = PTRRELOC(&__stop___entry_flush_fixup);
- for (i = 0; start < end; start++, i++) {
- dest = (void *)start + *start;
-
- pr_devel("patching dest %lx\n", (unsigned long)dest);
-
- if (types == L1D_FLUSH_FALLBACK) {
- patch_instruction(dest, ppc_inst(instrs[0]));
- patch_instruction(dest + 2, ppc_inst(instrs[2]));
- patch_branch(dest + 1,
- (unsigned long)&entry_flush_fallback, BRANCH_SET_LINK);
- } else {
- patch_instruction(dest + 1, ppc_inst(instrs[1]));
- patch_instruction(dest + 2, ppc_inst(instrs[2]));
- patch_instruction(dest, ppc_inst(instrs[0]));
- }
- }
+ i = do_patch_entry_fixups(start, end, instrs, types == L1D_FLUSH_FALLBACK,
+ &entry_flush_fallback);
start = PTRRELOC(&__start___scv_entry_flush_fixup);
end = PTRRELOC(&__stop___scv_entry_flush_fixup);
- for (; start < end; start++, i++) {
- dest = (void *)start + *start;
-
- pr_devel("patching dest %lx\n", (unsigned long)dest);
-
- if (types == L1D_FLUSH_FALLBACK) {
- patch_instruction(dest, ppc_inst(instrs[0]));
- patch_instruction(dest + 2, ppc_inst(instrs[2]));
- patch_branch(dest + 1,
- (unsigned long)&scv_entry_flush_fallback, BRANCH_SET_LINK);
- } else {
- patch_instruction(dest + 1, ppc_inst(instrs[1]));
- patch_instruction(dest + 2, ppc_inst(instrs[2]));
- patch_instruction(dest, ppc_inst(instrs[0]));
- }
- }
-
+ i += do_patch_entry_fixups(start, end, instrs, types == L1D_FLUSH_FALLBACK,
+ &scv_entry_flush_fallback);
printk(KERN_DEBUG "entry-flush: patched %d locations (%s flush)\n", i,
(types == L1D_FLUSH_NONE) ? "no" :
static int __do_rfi_flush_fixups(void *data)
{
enum l1d_flush_type types = *(enum l1d_flush_type *)data;
- unsigned int instrs[3], *dest;
+ unsigned int instrs[3];
long *start, *end;
int i;
if (types & L1D_FLUSH_MTTRIG)
instrs[i++] = PPC_RAW_MTSPR(SPRN_TRIG2, _R0);
- for (i = 0; start < end; start++, i++) {
- dest = (void *)start + *start;
-
- pr_devel("patching dest %lx\n", (unsigned long)dest);
-
- patch_instruction(dest, ppc_inst(instrs[0]));
- patch_instruction(dest + 1, ppc_inst(instrs[1]));
- patch_instruction(dest + 2, ppc_inst(instrs[2]));
- }
+ i = do_patch_fixups(start, end, instrs, ARRAY_SIZE(instrs));
printk(KERN_DEBUG "rfi-flush: patched %d locations (%s flush)\n", i,
(types == L1D_FLUSH_NONE) ? "no" :
void do_barrier_nospec_fixups_range(bool enable, void *fixup_start, void *fixup_end)
{
- unsigned int instr, *dest;
+ unsigned int instr;
long *start, *end;
int i;
instr = PPC_RAW_ORI(_R31, _R31, 0); /* speculation barrier */
}
- for (i = 0; start < end; start++, i++) {
- dest = (void *)start + *start;
-
- pr_devel("patching dest %lx\n", (unsigned long)dest);
- patch_instruction(dest, ppc_inst(instr));
- }
+ i = do_patch_fixups(start, end, &instr, 1);
printk(KERN_DEBUG "barrier-nospec: patched %d locations\n", i);
}
#ifdef CONFIG_PPC_E500
void do_barrier_nospec_fixups_range(bool enable, void *fixup_start, void *fixup_end)
{
- unsigned int instr[2], *dest;
+ unsigned int instr[2];
long *start, *end;
int i;
instr[1] = PPC_RAW_SYNC();
}
- for (i = 0; start < end; start++, i++) {
- dest = (void *)start + *start;
-
- pr_devel("patching dest %lx\n", (unsigned long)dest);
- patch_instruction(dest, ppc_inst(instr[0]));
- patch_instruction(dest + 1, ppc_inst(instr[1]));
- }
+ i = do_patch_fixups(start, end, instr, ARRAY_SIZE(instr));
printk(KERN_DEBUG "barrier-nospec: patched %d locations\n", i);
}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-or-later
+#include <linux/bug.h>
+#include <linux/compiler.h>
+#include <linux/export.h>
+#include <linux/percpu.h>
+#include <linux/processor.h>
+#include <linux/smp.h>
+#include <linux/topology.h>
+#include <linux/sched/clock.h>
+#include <asm/qspinlock.h>
+#include <asm/paravirt.h>
+
+#define MAX_NODES 4
+
+struct qnode {
+ struct qnode *next;
+ struct qspinlock *lock;
+ int cpu;
+ int yield_cpu;
+ u8 locked; /* 1 if lock acquired */
+};
+
+struct qnodes {
+ int count;
+ struct qnode nodes[MAX_NODES];
+};
+
+/* Tuning parameters */
+static int steal_spins __read_mostly = (1 << 5);
+static int remote_steal_spins __read_mostly = (1 << 2);
+#if _Q_SPIN_TRY_LOCK_STEAL == 1
+static const bool maybe_stealers = true;
+#else
+static bool maybe_stealers __read_mostly = true;
+#endif
+static int head_spins __read_mostly = (1 << 8);
+
+static bool pv_yield_owner __read_mostly = true;
+static bool pv_yield_allow_steal __read_mostly = false;
+static bool pv_spin_on_preempted_owner __read_mostly = false;
+static bool pv_sleepy_lock __read_mostly = true;
+static bool pv_sleepy_lock_sticky __read_mostly = false;
+static u64 pv_sleepy_lock_interval_ns __read_mostly = 0;
+static int pv_sleepy_lock_factor __read_mostly = 256;
+static bool pv_yield_prev __read_mostly = true;
+static bool pv_yield_propagate_owner __read_mostly = true;
+static bool pv_prod_head __read_mostly = false;
+
+static DEFINE_PER_CPU_ALIGNED(struct qnodes, qnodes);
+static DEFINE_PER_CPU_ALIGNED(u64, sleepy_lock_seen_clock);
+
+#if _Q_SPIN_SPEC_BARRIER == 1
+#define spec_barrier() do { asm volatile("ori 31,31,0" ::: "memory"); } while (0)
+#else
+#define spec_barrier() do { } while (0)
+#endif
+
+static __always_inline bool recently_sleepy(void)
+{
+ /* pv_sleepy_lock is true when this is called */
+ if (pv_sleepy_lock_interval_ns) {
+ u64 seen = this_cpu_read(sleepy_lock_seen_clock);
+
+ if (seen) {
+ u64 delta = sched_clock() - seen;
+ if (delta < pv_sleepy_lock_interval_ns)
+ return true;
+ this_cpu_write(sleepy_lock_seen_clock, 0);
+ }
+ }
+
+ return false;
+}
+
+static __always_inline int get_steal_spins(bool paravirt, bool sleepy)
+{
+ if (paravirt && sleepy)
+ return steal_spins * pv_sleepy_lock_factor;
+ else
+ return steal_spins;
+}
+
+static __always_inline int get_remote_steal_spins(bool paravirt, bool sleepy)
+{
+ if (paravirt && sleepy)
+ return remote_steal_spins * pv_sleepy_lock_factor;
+ else
+ return remote_steal_spins;
+}
+
+static __always_inline int get_head_spins(bool paravirt, bool sleepy)
+{
+ if (paravirt && sleepy)
+ return head_spins * pv_sleepy_lock_factor;
+ else
+ return head_spins;
+}
+
+static inline u32 encode_tail_cpu(int cpu)
+{
+ return (cpu + 1) << _Q_TAIL_CPU_OFFSET;
+}
+
+static inline int decode_tail_cpu(u32 val)
+{
+ return (val >> _Q_TAIL_CPU_OFFSET) - 1;
+}
+
+static inline int get_owner_cpu(u32 val)
+{
+ return (val & _Q_OWNER_CPU_MASK) >> _Q_OWNER_CPU_OFFSET;
+}
+
+/*
+ * Try to acquire the lock if it was not already locked. If the tail matches
+ * mytail then clear it, otherwise leave it unchnaged. Return previous value.
+ *
+ * This is used by the head of the queue to acquire the lock and clean up
+ * its tail if it was the last one queued.
+ */
+static __always_inline u32 trylock_clean_tail(struct qspinlock *lock, u32 tail)
+{
+ u32 newval = queued_spin_encode_locked_val();
+ u32 prev, tmp;
+
+ asm volatile(
+"1: lwarx %0,0,%2,%7 # trylock_clean_tail \n"
+ /* This test is necessary if there could be stealers */
+" andi. %1,%0,%5 \n"
+" bne 3f \n"
+ /* Test whether the lock tail == mytail */
+" and %1,%0,%6 \n"
+" cmpw 0,%1,%3 \n"
+ /* Merge the new locked value */
+" or %1,%1,%4 \n"
+" bne 2f \n"
+ /* If the lock tail matched, then clear it, otherwise leave it. */
+" andc %1,%1,%6 \n"
+"2: stwcx. %1,0,%2 \n"
+" bne- 1b \n"
+"\t" PPC_ACQUIRE_BARRIER " \n"
+"3: \n"
+ : "=&r" (prev), "=&r" (tmp)
+ : "r" (&lock->val), "r"(tail), "r" (newval),
+ "i" (_Q_LOCKED_VAL),
+ "r" (_Q_TAIL_CPU_MASK),
+ "i" (_Q_SPIN_EH_HINT)
+ : "cr0", "memory");
+
+ return prev;
+}
+
+/*
+ * Publish our tail, replacing previous tail. Return previous value.
+ *
+ * This provides a release barrier for publishing node, this pairs with the
+ * acquire barrier in get_tail_qnode() when the next CPU finds this tail
+ * value.
+ */
+static __always_inline u32 publish_tail_cpu(struct qspinlock *lock, u32 tail)
+{
+ u32 prev, tmp;
+
+ asm volatile(
+"\t" PPC_RELEASE_BARRIER " \n"
+"1: lwarx %0,0,%2 # publish_tail_cpu \n"
+" andc %1,%0,%4 \n"
+" or %1,%1,%3 \n"
+" stwcx. %1,0,%2 \n"
+" bne- 1b \n"
+ : "=&r" (prev), "=&r"(tmp)
+ : "r" (&lock->val), "r" (tail), "r"(_Q_TAIL_CPU_MASK)
+ : "cr0", "memory");
+
+ return prev;
+}
+
+static __always_inline u32 set_mustq(struct qspinlock *lock)
+{
+ u32 prev;
+
+ asm volatile(
+"1: lwarx %0,0,%1 # set_mustq \n"
+" or %0,%0,%2 \n"
+" stwcx. %0,0,%1 \n"
+" bne- 1b \n"
+ : "=&r" (prev)
+ : "r" (&lock->val), "r" (_Q_MUST_Q_VAL)
+ : "cr0", "memory");
+
+ return prev;
+}
+
+static __always_inline u32 clear_mustq(struct qspinlock *lock)
+{
+ u32 prev;
+
+ asm volatile(
+"1: lwarx %0,0,%1 # clear_mustq \n"
+" andc %0,%0,%2 \n"
+" stwcx. %0,0,%1 \n"
+" bne- 1b \n"
+ : "=&r" (prev)
+ : "r" (&lock->val), "r" (_Q_MUST_Q_VAL)
+ : "cr0", "memory");
+
+ return prev;
+}
+
+static __always_inline bool try_set_sleepy(struct qspinlock *lock, u32 old)
+{
+ u32 prev;
+ u32 new = old | _Q_SLEEPY_VAL;
+
+ BUG_ON(!(old & _Q_LOCKED_VAL));
+ BUG_ON(old & _Q_SLEEPY_VAL);
+
+ asm volatile(
+"1: lwarx %0,0,%1 # try_set_sleepy \n"
+" cmpw 0,%0,%2 \n"
+" bne- 2f \n"
+" stwcx. %3,0,%1 \n"
+" bne- 1b \n"
+"2: \n"
+ : "=&r" (prev)
+ : "r" (&lock->val), "r"(old), "r" (new)
+ : "cr0", "memory");
+
+ return likely(prev == old);
+}
+
+static __always_inline void seen_sleepy_owner(struct qspinlock *lock, u32 val)
+{
+ if (pv_sleepy_lock) {
+ if (pv_sleepy_lock_interval_ns)
+ this_cpu_write(sleepy_lock_seen_clock, sched_clock());
+ if (!(val & _Q_SLEEPY_VAL))
+ try_set_sleepy(lock, val);
+ }
+}
+
+static __always_inline void seen_sleepy_lock(void)
+{
+ if (pv_sleepy_lock && pv_sleepy_lock_interval_ns)
+ this_cpu_write(sleepy_lock_seen_clock, sched_clock());
+}
+
+static __always_inline void seen_sleepy_node(struct qspinlock *lock, u32 val)
+{
+ if (pv_sleepy_lock) {
+ if (pv_sleepy_lock_interval_ns)
+ this_cpu_write(sleepy_lock_seen_clock, sched_clock());
+ if (val & _Q_LOCKED_VAL) {
+ if (!(val & _Q_SLEEPY_VAL))
+ try_set_sleepy(lock, val);
+ }
+ }
+}
+
+static struct qnode *get_tail_qnode(struct qspinlock *lock, u32 val)
+{
+ int cpu = decode_tail_cpu(val);
+ struct qnodes *qnodesp = per_cpu_ptr(&qnodes, cpu);
+ int idx;
+
+ /*
+ * After publishing the new tail and finding a previous tail in the
+ * previous val (which is the control dependency), this barrier
+ * orders the release barrier in publish_tail_cpu performed by the
+ * last CPU, with subsequently looking at its qnode structures
+ * after the barrier.
+ */
+ smp_acquire__after_ctrl_dep();
+
+ for (idx = 0; idx < MAX_NODES; idx++) {
+ struct qnode *qnode = &qnodesp->nodes[idx];
+ if (qnode->lock == lock)
+ return qnode;
+ }
+
+ BUG();
+}
+
+/* Called inside spin_begin(). Returns whether or not the vCPU was preempted. */
+static __always_inline bool __yield_to_locked_owner(struct qspinlock *lock, u32 val, bool paravirt, bool mustq)
+{
+ int owner;
+ u32 yield_count;
+ bool preempted = false;
+
+ BUG_ON(!(val & _Q_LOCKED_VAL));
+
+ if (!paravirt)
+ goto relax;
+
+ if (!pv_yield_owner)
+ goto relax;
+
+ owner = get_owner_cpu(val);
+ yield_count = yield_count_of(owner);
+
+ if ((yield_count & 1) == 0)
+ goto relax; /* owner vcpu is running */
+
+ spin_end();
+
+ seen_sleepy_owner(lock, val);
+ preempted = true;
+
+ /*
+ * Read the lock word after sampling the yield count. On the other side
+ * there may a wmb because the yield count update is done by the
+ * hypervisor preemption and the value update by the OS, however this
+ * ordering might reduce the chance of out of order accesses and
+ * improve the heuristic.
+ */
+ smp_rmb();
+
+ if (READ_ONCE(lock->val) == val) {
+ if (mustq)
+ clear_mustq(lock);
+ yield_to_preempted(owner, yield_count);
+ if (mustq)
+ set_mustq(lock);
+ spin_begin();
+
+ /* Don't relax if we yielded. Maybe we should? */
+ return preempted;
+ }
+ spin_begin();
+relax:
+ spin_cpu_relax();
+
+ return preempted;
+}
+
+/* Called inside spin_begin(). Returns whether or not the vCPU was preempted. */
+static __always_inline bool yield_to_locked_owner(struct qspinlock *lock, u32 val, bool paravirt)
+{
+ return __yield_to_locked_owner(lock, val, paravirt, false);
+}
+
+/* Called inside spin_begin(). Returns whether or not the vCPU was preempted. */
+static __always_inline bool yield_head_to_locked_owner(struct qspinlock *lock, u32 val, bool paravirt)
+{
+ bool mustq = false;
+
+ if ((val & _Q_MUST_Q_VAL) && pv_yield_allow_steal)
+ mustq = true;
+
+ return __yield_to_locked_owner(lock, val, paravirt, mustq);
+}
+
+static __always_inline void propagate_yield_cpu(struct qnode *node, u32 val, int *set_yield_cpu, bool paravirt)
+{
+ struct qnode *next;
+ int owner;
+
+ if (!paravirt)
+ return;
+ if (!pv_yield_propagate_owner)
+ return;
+
+ owner = get_owner_cpu(val);
+ if (*set_yield_cpu == owner)
+ return;
+
+ next = READ_ONCE(node->next);
+ if (!next)
+ return;
+
+ if (vcpu_is_preempted(owner)) {
+ next->yield_cpu = owner;
+ *set_yield_cpu = owner;
+ } else if (*set_yield_cpu != -1) {
+ next->yield_cpu = owner;
+ *set_yield_cpu = owner;
+ }
+}
+
+/* Called inside spin_begin() */
+static __always_inline bool yield_to_prev(struct qspinlock *lock, struct qnode *node, u32 val, bool paravirt)
+{
+ int prev_cpu = decode_tail_cpu(val);
+ u32 yield_count;
+ int yield_cpu;
+ bool preempted = false;
+
+ if (!paravirt)
+ goto relax;
+
+ if (!pv_yield_propagate_owner)
+ goto yield_prev;
+
+ yield_cpu = READ_ONCE(node->yield_cpu);
+ if (yield_cpu == -1) {
+ /* Propagate back the -1 CPU */
+ if (node->next && node->next->yield_cpu != -1)
+ node->next->yield_cpu = yield_cpu;
+ goto yield_prev;
+ }
+
+ yield_count = yield_count_of(yield_cpu);
+ if ((yield_count & 1) == 0)
+ goto yield_prev; /* owner vcpu is running */
+
+ spin_end();
+
+ preempted = true;
+ seen_sleepy_node(lock, val);
+
+ smp_rmb();
+
+ if (yield_cpu == node->yield_cpu) {
+ if (node->next && node->next->yield_cpu != yield_cpu)
+ node->next->yield_cpu = yield_cpu;
+ yield_to_preempted(yield_cpu, yield_count);
+ spin_begin();
+ return preempted;
+ }
+ spin_begin();
+
+yield_prev:
+ if (!pv_yield_prev)
+ goto relax;
+
+ yield_count = yield_count_of(prev_cpu);
+ if ((yield_count & 1) == 0)
+ goto relax; /* owner vcpu is running */
+
+ spin_end();
+
+ preempted = true;
+ seen_sleepy_node(lock, val);
+
+ smp_rmb(); /* See __yield_to_locked_owner comment */
+
+ if (!node->locked) {
+ yield_to_preempted(prev_cpu, yield_count);
+ spin_begin();
+ return preempted;
+ }
+ spin_begin();
+
+relax:
+ spin_cpu_relax();
+
+ return preempted;
+}
+
+static __always_inline bool steal_break(u32 val, int iters, bool paravirt, bool sleepy)
+{
+ if (iters >= get_steal_spins(paravirt, sleepy))
+ return true;
+
+ if (IS_ENABLED(CONFIG_NUMA) &&
+ (iters >= get_remote_steal_spins(paravirt, sleepy))) {
+ int cpu = get_owner_cpu(val);
+ if (numa_node_id() != cpu_to_node(cpu))
+ return true;
+ }
+ return false;
+}
+
+static __always_inline bool try_to_steal_lock(struct qspinlock *lock, bool paravirt)
+{
+ bool seen_preempted = false;
+ bool sleepy = false;
+ int iters = 0;
+ u32 val;
+
+ if (!steal_spins) {
+ /* XXX: should spin_on_preempted_owner do anything here? */
+ return false;
+ }
+
+ /* Attempt to steal the lock */
+ spin_begin();
+ do {
+ bool preempted = false;
+
+ val = READ_ONCE(lock->val);
+ if (val & _Q_MUST_Q_VAL)
+ break;
+ spec_barrier();
+
+ if (unlikely(!(val & _Q_LOCKED_VAL))) {
+ spin_end();
+ if (__queued_spin_trylock_steal(lock))
+ return true;
+ spin_begin();
+ } else {
+ preempted = yield_to_locked_owner(lock, val, paravirt);
+ }
+
+ if (paravirt && pv_sleepy_lock) {
+ if (!sleepy) {
+ if (val & _Q_SLEEPY_VAL) {
+ seen_sleepy_lock();
+ sleepy = true;
+ } else if (recently_sleepy()) {
+ sleepy = true;
+ }
+ }
+ if (pv_sleepy_lock_sticky && seen_preempted &&
+ !(val & _Q_SLEEPY_VAL)) {
+ if (try_set_sleepy(lock, val))
+ val |= _Q_SLEEPY_VAL;
+ }
+ }
+
+ if (preempted) {
+ seen_preempted = true;
+ sleepy = true;
+ if (!pv_spin_on_preempted_owner)
+ iters++;
+ /*
+ * pv_spin_on_preempted_owner don't increase iters
+ * while the owner is preempted -- we won't interfere
+ * with it by definition. This could introduce some
+ * latency issue if we continually observe preempted
+ * owners, but hopefully that's a rare corner case of
+ * a badly oversubscribed system.
+ */
+ } else {
+ iters++;
+ }
+ } while (!steal_break(val, iters, paravirt, sleepy));
+
+ spin_end();
+
+ return false;
+}
+
+static __always_inline void queued_spin_lock_mcs_queue(struct qspinlock *lock, bool paravirt)
+{
+ struct qnodes *qnodesp;
+ struct qnode *next, *node;
+ u32 val, old, tail;
+ bool seen_preempted = false;
+ bool sleepy = false;
+ bool mustq = false;
+ int idx;
+ int set_yield_cpu = -1;
+ int iters = 0;
+
+ BUILD_BUG_ON(CONFIG_NR_CPUS >= (1U << _Q_TAIL_CPU_BITS));
+
+ qnodesp = this_cpu_ptr(&qnodes);
+ if (unlikely(qnodesp->count >= MAX_NODES)) {
+ spec_barrier();
+ while (!queued_spin_trylock(lock))
+ cpu_relax();
+ return;
+ }
+
+ idx = qnodesp->count++;
+ /*
+ * Ensure that we increment the head node->count before initialising
+ * the actual node. If the compiler is kind enough to reorder these
+ * stores, then an IRQ could overwrite our assignments.
+ */
+ barrier();
+ node = &qnodesp->nodes[idx];
+ node->next = NULL;
+ node->lock = lock;
+ node->cpu = smp_processor_id();
+ node->yield_cpu = -1;
+ node->locked = 0;
+
+ tail = encode_tail_cpu(node->cpu);
+
+ old = publish_tail_cpu(lock, tail);
+
+ /*
+ * If there was a previous node; link it and wait until reaching the
+ * head of the waitqueue.
+ */
+ if (old & _Q_TAIL_CPU_MASK) {
+ struct qnode *prev = get_tail_qnode(lock, old);
+
+ /* Link @node into the waitqueue. */
+ WRITE_ONCE(prev->next, node);
+
+ /* Wait for mcs node lock to be released */
+ spin_begin();
+ while (!node->locked) {
+ spec_barrier();
+
+ if (yield_to_prev(lock, node, old, paravirt))
+ seen_preempted = true;
+ }
+ spec_barrier();
+ spin_end();
+
+ /* Clear out stale propagated yield_cpu */
+ if (paravirt && pv_yield_propagate_owner && node->yield_cpu != -1)
+ node->yield_cpu = -1;
+
+ smp_rmb(); /* acquire barrier for the mcs lock */
+
+ /*
+ * Generic qspinlocks have this prefetch here, but it seems
+ * like it could cause additional line transitions because
+ * the waiter will keep loading from it.
+ */
+ if (_Q_SPIN_PREFETCH_NEXT) {
+ next = READ_ONCE(node->next);
+ if (next)
+ prefetchw(next);
+ }
+ }
+
+ /* We're at the head of the waitqueue, wait for the lock. */
+again:
+ spin_begin();
+ for (;;) {
+ bool preempted;
+
+ val = READ_ONCE(lock->val);
+ if (!(val & _Q_LOCKED_VAL))
+ break;
+ spec_barrier();
+
+ if (paravirt && pv_sleepy_lock && maybe_stealers) {
+ if (!sleepy) {
+ if (val & _Q_SLEEPY_VAL) {
+ seen_sleepy_lock();
+ sleepy = true;
+ } else if (recently_sleepy()) {
+ sleepy = true;
+ }
+ }
+ if (pv_sleepy_lock_sticky && seen_preempted &&
+ !(val & _Q_SLEEPY_VAL)) {
+ if (try_set_sleepy(lock, val))
+ val |= _Q_SLEEPY_VAL;
+ }
+ }
+
+ propagate_yield_cpu(node, val, &set_yield_cpu, paravirt);
+ preempted = yield_head_to_locked_owner(lock, val, paravirt);
+ if (!maybe_stealers)
+ continue;
+
+ if (preempted)
+ seen_preempted = true;
+
+ if (paravirt && preempted) {
+ sleepy = true;
+
+ if (!pv_spin_on_preempted_owner)
+ iters++;
+ } else {
+ iters++;
+ }
+
+ if (!mustq && iters >= get_head_spins(paravirt, sleepy)) {
+ mustq = true;
+ set_mustq(lock);
+ val |= _Q_MUST_Q_VAL;
+ }
+ }
+ spec_barrier();
+ spin_end();
+
+ /* If we're the last queued, must clean up the tail. */
+ old = trylock_clean_tail(lock, tail);
+ if (unlikely(old & _Q_LOCKED_VAL)) {
+ BUG_ON(!maybe_stealers);
+ goto again; /* Can only be true if maybe_stealers. */
+ }
+
+ if ((old & _Q_TAIL_CPU_MASK) == tail)
+ goto release; /* We were the tail, no next. */
+
+ /* There is a next, must wait for node->next != NULL (MCS protocol) */
+ next = READ_ONCE(node->next);
+ if (!next) {
+ spin_begin();
+ while (!(next = READ_ONCE(node->next)))
+ cpu_relax();
+ spin_end();
+ }
+ spec_barrier();
+
+ /*
+ * Unlock the next mcs waiter node. Release barrier is not required
+ * here because the acquirer is only accessing the lock word, and
+ * the acquire barrier we took the lock with orders that update vs
+ * this store to locked. The corresponding barrier is the smp_rmb()
+ * acquire barrier for mcs lock, above.
+ */
+ if (paravirt && pv_prod_head) {
+ int next_cpu = next->cpu;
+ WRITE_ONCE(next->locked, 1);
+ if (_Q_SPIN_MISO)
+ asm volatile("miso" ::: "memory");
+ if (vcpu_is_preempted(next_cpu))
+ prod_cpu(next_cpu);
+ } else {
+ WRITE_ONCE(next->locked, 1);
+ if (_Q_SPIN_MISO)
+ asm volatile("miso" ::: "memory");
+ }
+
+release:
+ qnodesp->count--; /* release the node */
+}
+
+void queued_spin_lock_slowpath(struct qspinlock *lock)
+{
+ /*
+ * This looks funny, but it induces the compiler to inline both
+ * sides of the branch rather than share code as when the condition
+ * is passed as the paravirt argument to the functions.
+ */
+ if (IS_ENABLED(CONFIG_PARAVIRT_SPINLOCKS) && is_shared_processor()) {
+ if (try_to_steal_lock(lock, true)) {
+ spec_barrier();
+ return;
+ }
+ queued_spin_lock_mcs_queue(lock, true);
+ } else {
+ if (try_to_steal_lock(lock, false)) {
+ spec_barrier();
+ return;
+ }
+ queued_spin_lock_mcs_queue(lock, false);
+ }
+}
+EXPORT_SYMBOL(queued_spin_lock_slowpath);
+
+#ifdef CONFIG_PARAVIRT_SPINLOCKS
+void pv_spinlocks_init(void)
+{
+}
+#endif
+
+#include <linux/debugfs.h>
+static int steal_spins_set(void *data, u64 val)
+{
+#if _Q_SPIN_TRY_LOCK_STEAL == 1
+ /* MAYBE_STEAL remains true */
+ steal_spins = val;
+#else
+ static DEFINE_MUTEX(lock);
+
+ /*
+ * The lock slow path has a !maybe_stealers case that can assume
+ * the head of queue will not see concurrent waiters. That waiter
+ * is unsafe in the presence of stealers, so must keep them away
+ * from one another.
+ */
+
+ mutex_lock(&lock);
+ if (val && !steal_spins) {
+ maybe_stealers = true;
+ /* wait for queue head waiter to go away */
+ synchronize_rcu();
+ steal_spins = val;
+ } else if (!val && steal_spins) {
+ steal_spins = val;
+ /* wait for all possible stealers to go away */
+ synchronize_rcu();
+ maybe_stealers = false;
+ } else {
+ steal_spins = val;
+ }
+ mutex_unlock(&lock);
+#endif
+
+ return 0;
+}
+
+static int steal_spins_get(void *data, u64 *val)
+{
+ *val = steal_spins;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_steal_spins, steal_spins_get, steal_spins_set, "%llu\n");
+
+static int remote_steal_spins_set(void *data, u64 val)
+{
+ remote_steal_spins = val;
+
+ return 0;
+}
+
+static int remote_steal_spins_get(void *data, u64 *val)
+{
+ *val = remote_steal_spins;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_remote_steal_spins, remote_steal_spins_get, remote_steal_spins_set, "%llu\n");
+
+static int head_spins_set(void *data, u64 val)
+{
+ head_spins = val;
+
+ return 0;
+}
+
+static int head_spins_get(void *data, u64 *val)
+{
+ *val = head_spins;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_head_spins, head_spins_get, head_spins_set, "%llu\n");
+
+static int pv_yield_owner_set(void *data, u64 val)
+{
+ pv_yield_owner = !!val;
+
+ return 0;
+}
+
+static int pv_yield_owner_get(void *data, u64 *val)
+{
+ *val = pv_yield_owner;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_pv_yield_owner, pv_yield_owner_get, pv_yield_owner_set, "%llu\n");
+
+static int pv_yield_allow_steal_set(void *data, u64 val)
+{
+ pv_yield_allow_steal = !!val;
+
+ return 0;
+}
+
+static int pv_yield_allow_steal_get(void *data, u64 *val)
+{
+ *val = pv_yield_allow_steal;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_pv_yield_allow_steal, pv_yield_allow_steal_get, pv_yield_allow_steal_set, "%llu\n");
+
+static int pv_spin_on_preempted_owner_set(void *data, u64 val)
+{
+ pv_spin_on_preempted_owner = !!val;
+
+ return 0;
+}
+
+static int pv_spin_on_preempted_owner_get(void *data, u64 *val)
+{
+ *val = pv_spin_on_preempted_owner;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_pv_spin_on_preempted_owner, pv_spin_on_preempted_owner_get, pv_spin_on_preempted_owner_set, "%llu\n");
+
+static int pv_sleepy_lock_set(void *data, u64 val)
+{
+ pv_sleepy_lock = !!val;
+
+ return 0;
+}
+
+static int pv_sleepy_lock_get(void *data, u64 *val)
+{
+ *val = pv_sleepy_lock;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_pv_sleepy_lock, pv_sleepy_lock_get, pv_sleepy_lock_set, "%llu\n");
+
+static int pv_sleepy_lock_sticky_set(void *data, u64 val)
+{
+ pv_sleepy_lock_sticky = !!val;
+
+ return 0;
+}
+
+static int pv_sleepy_lock_sticky_get(void *data, u64 *val)
+{
+ *val = pv_sleepy_lock_sticky;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_pv_sleepy_lock_sticky, pv_sleepy_lock_sticky_get, pv_sleepy_lock_sticky_set, "%llu\n");
+
+static int pv_sleepy_lock_interval_ns_set(void *data, u64 val)
+{
+ pv_sleepy_lock_interval_ns = val;
+
+ return 0;
+}
+
+static int pv_sleepy_lock_interval_ns_get(void *data, u64 *val)
+{
+ *val = pv_sleepy_lock_interval_ns;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_pv_sleepy_lock_interval_ns, pv_sleepy_lock_interval_ns_get, pv_sleepy_lock_interval_ns_set, "%llu\n");
+
+static int pv_sleepy_lock_factor_set(void *data, u64 val)
+{
+ pv_sleepy_lock_factor = val;
+
+ return 0;
+}
+
+static int pv_sleepy_lock_factor_get(void *data, u64 *val)
+{
+ *val = pv_sleepy_lock_factor;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_pv_sleepy_lock_factor, pv_sleepy_lock_factor_get, pv_sleepy_lock_factor_set, "%llu\n");
+
+static int pv_yield_prev_set(void *data, u64 val)
+{
+ pv_yield_prev = !!val;
+
+ return 0;
+}
+
+static int pv_yield_prev_get(void *data, u64 *val)
+{
+ *val = pv_yield_prev;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_pv_yield_prev, pv_yield_prev_get, pv_yield_prev_set, "%llu\n");
+
+static int pv_yield_propagate_owner_set(void *data, u64 val)
+{
+ pv_yield_propagate_owner = !!val;
+
+ return 0;
+}
+
+static int pv_yield_propagate_owner_get(void *data, u64 *val)
+{
+ *val = pv_yield_propagate_owner;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_pv_yield_propagate_owner, pv_yield_propagate_owner_get, pv_yield_propagate_owner_set, "%llu\n");
+
+static int pv_prod_head_set(void *data, u64 val)
+{
+ pv_prod_head = !!val;
+
+ return 0;
+}
+
+static int pv_prod_head_get(void *data, u64 *val)
+{
+ *val = pv_prod_head;
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_pv_prod_head, pv_prod_head_get, pv_prod_head_set, "%llu\n");
+
+static __init int spinlock_debugfs_init(void)
+{
+ debugfs_create_file("qspl_steal_spins", 0600, arch_debugfs_dir, NULL, &fops_steal_spins);
+ debugfs_create_file("qspl_remote_steal_spins", 0600, arch_debugfs_dir, NULL, &fops_remote_steal_spins);
+ debugfs_create_file("qspl_head_spins", 0600, arch_debugfs_dir, NULL, &fops_head_spins);
+ if (is_shared_processor()) {
+ debugfs_create_file("qspl_pv_yield_owner", 0600, arch_debugfs_dir, NULL, &fops_pv_yield_owner);
+ debugfs_create_file("qspl_pv_yield_allow_steal", 0600, arch_debugfs_dir, NULL, &fops_pv_yield_allow_steal);
+ debugfs_create_file("qspl_pv_spin_on_preempted_owner", 0600, arch_debugfs_dir, NULL, &fops_pv_spin_on_preempted_owner);
+ debugfs_create_file("qspl_pv_sleepy_lock", 0600, arch_debugfs_dir, NULL, &fops_pv_sleepy_lock);
+ debugfs_create_file("qspl_pv_sleepy_lock_sticky", 0600, arch_debugfs_dir, NULL, &fops_pv_sleepy_lock_sticky);
+ debugfs_create_file("qspl_pv_sleepy_lock_interval_ns", 0600, arch_debugfs_dir, NULL, &fops_pv_sleepy_lock_interval_ns);
+ debugfs_create_file("qspl_pv_sleepy_lock_factor", 0600, arch_debugfs_dir, NULL, &fops_pv_sleepy_lock_factor);
+ debugfs_create_file("qspl_pv_yield_prev", 0600, arch_debugfs_dir, NULL, &fops_pv_yield_prev);
+ debugfs_create_file("qspl_pv_yield_propagate_owner", 0600, arch_debugfs_dir, NULL, &fops_pv_yield_propagate_owner);
+ debugfs_create_file("qspl_pv_prod_head", 0600, arch_debugfs_dir, NULL, &fops_pv_prod_head);
+ }
+
+ return 0;
+}
+device_initcall(spinlock_debugfs_init);
op->type = MKOP(STCX, 0, 4);
break;
-#ifdef __powerpc64__
- case 84: /* ldarx */
- op->type = MKOP(LARX, 0, 8);
- break;
-
- case 214: /* stdcx. */
- op->type = MKOP(STCX, 0, 8);
- break;
-
+#ifdef CONFIG_PPC_HAS_LBARX_LHARX
case 52: /* lbarx */
op->type = MKOP(LARX, 0, 1);
break;
case 726: /* sthcx. */
op->type = MKOP(STCX, 0, 2);
break;
+#endif
+#ifdef __powerpc64__
+ case 84: /* ldarx */
+ op->type = MKOP(LARX, 0, 8);
+ break;
+
+ case 214: /* stdcx. */
+ op->type = MKOP(STCX, 0, 8);
+ break;
case 276: /* lqarx */
if (!((rd & 1) || rd == ra || rd == rb))
err = 0;
val = 0;
switch (size) {
-#ifdef __powerpc64__
+#ifdef CONFIG_PPC_HAS_LBARX_LHARX
case 1:
__get_user_asmx(val, ea, err, "lbarx");
break;
/*
* Stack frame layout (INT_FRAME_SIZE bytes)
- * In-memory pt_regs (SP + STACK_FRAME_OVERHEAD)
+ * In-memory pt_regs (SP + STACK_INT_FRAME_REGS)
* Scratch space (SP + 8)
* Back chain (SP + 0)
*/
#include <asm/machdep.h>
#include <asm/mmu.h>
+#include "internal.h"
+
int __hash_page_4K(unsigned long ea, unsigned long access, unsigned long vsid,
pte_t *ptep, unsigned long trap, unsigned long flags,
int ssize, int subpg_prot)
}
new_pte = (new_pte & ~_PAGE_HPTEFLAGS) | H_PAGE_HASHPTE;
new_pte |= pte_set_hidx(ptep, rpte, 0, slot, PTRS_PER_PTE);
+
+ if (stress_hpt())
+ hpt_do_stress(ea, hpte_group);
}
*ptep = __pte(new_pte & ~H_PAGE_BUSY);
return 0;
#include <asm/machdep.h>
#include <asm/mmu.h>
+#include "internal.h"
+
/*
* Return true, if the entry has a slot value which
* the software considers as invalid.
new_pte |= pte_set_hidx(ptep, rpte, subpg_index, slot, PTRS_PER_PTE);
new_pte |= H_PAGE_HASHPTE;
+ if (stress_hpt())
+ hpt_do_stress(ea, hpte_group);
+
*ptep = __pte(new_pte & ~H_PAGE_BUSY);
return 0;
}
new_pte = (new_pte & ~_PAGE_HPTEFLAGS) | H_PAGE_HASHPTE;
new_pte |= pte_set_hidx(ptep, rpte, 0, slot, PTRS_PER_PTE);
+
+ if (stress_hpt())
+ hpt_do_stress(ea, hpte_group);
}
+
*ptep = __pte(new_pte & ~H_PAGE_BUSY);
+
return 0;
}
return ret;
}
-static bool disable_1tb_segments = false;
+static bool disable_1tb_segments __ro_after_init;
static int __init parse_disable_1tb_segments(char *p)
{
}
early_param("disable_1tb_segments", parse_disable_1tb_segments);
+bool stress_hpt_enabled __initdata;
+
+static int __init parse_stress_hpt(char *p)
+{
+ stress_hpt_enabled = true;
+ return 0;
+}
+early_param("stress_hpt", parse_stress_hpt);
+
+__ro_after_init DEFINE_STATIC_KEY_FALSE(stress_hpt_key);
+
+/*
+ * per-CPU array allocated if we enable stress_hpt.
+ */
+#define STRESS_MAX_GROUPS 16
+struct stress_hpt_struct {
+ unsigned long last_group[STRESS_MAX_GROUPS];
+};
+
+static inline int stress_nr_groups(void)
+{
+ /*
+ * LPAR H_REMOVE flushes TLB, so need some number > 1 of entries
+ * to allow practical forward progress. Bare metal returns 1, which
+ * seems to help uncover more bugs.
+ */
+ if (firmware_has_feature(FW_FEATURE_LPAR))
+ return STRESS_MAX_GROUPS;
+ else
+ return 1;
+}
+
+static struct stress_hpt_struct *stress_hpt_struct;
+
static int __init htab_dt_scan_seg_sizes(unsigned long node,
const char *uname, int depth,
void *data)
pr_info("Partition table %p\n", partition_tb);
}
+void hpt_clear_stress(void);
+static struct timer_list stress_hpt_timer;
+void stress_hpt_timer_fn(struct timer_list *timer)
+{
+ int next_cpu;
+
+ hpt_clear_stress();
+ if (!firmware_has_feature(FW_FEATURE_LPAR))
+ tlbiel_all();
+
+ next_cpu = cpumask_next(raw_smp_processor_id(), cpu_online_mask);
+ if (next_cpu >= nr_cpu_ids)
+ next_cpu = cpumask_first(cpu_online_mask);
+ stress_hpt_timer.expires = jiffies + msecs_to_jiffies(10);
+ add_timer_on(&stress_hpt_timer, next_cpu);
+}
+
static void __init htab_initialize(void)
{
unsigned long table;
if (stress_slb_enabled)
static_branch_enable(&stress_slb_key);
+ if (stress_hpt_enabled) {
+ unsigned long tmp;
+ static_branch_enable(&stress_hpt_key);
+ // Too early to use nr_cpu_ids, so use NR_CPUS
+ tmp = memblock_phys_alloc_range(sizeof(struct stress_hpt_struct) * NR_CPUS,
+ 0, 0, MEMBLOCK_ALLOC_ANYWHERE);
+ memset((void *)tmp, 0xff, sizeof(struct stress_hpt_struct) * NR_CPUS);
+ stress_hpt_struct = __va(tmp);
+
+ timer_setup(&stress_hpt_timer, stress_hpt_timer_fn, 0);
+ stress_hpt_timer.expires = jiffies + msecs_to_jiffies(10);
+ add_timer(&stress_hpt_timer);
+ }
+
/*
* Calculate the required size of the htab. We want the number of
* PTEGs to equal one half the number of real pages.
return slot;
}
+void hpt_clear_stress(void)
+{
+ int cpu = raw_smp_processor_id();
+ int g;
+
+ for (g = 0; g < stress_nr_groups(); g++) {
+ unsigned long last_group;
+ last_group = stress_hpt_struct[cpu].last_group[g];
+
+ if (last_group != -1UL) {
+ int i;
+ for (i = 0; i < HPTES_PER_GROUP; i++) {
+ if (mmu_hash_ops.hpte_remove(last_group) == -1)
+ break;
+ }
+ stress_hpt_struct[cpu].last_group[g] = -1;
+ }
+ }
+}
+
+void hpt_do_stress(unsigned long ea, unsigned long hpte_group)
+{
+ unsigned long last_group;
+ int cpu = raw_smp_processor_id();
+
+ last_group = stress_hpt_struct[cpu].last_group[stress_nr_groups() - 1];
+ if (hpte_group == last_group)
+ return;
+
+ if (last_group != -1UL) {
+ int i;
+ /*
+ * Concurrent CPUs might be inserting into this group, so
+ * give up after a number of iterations, to prevent a live
+ * lock.
+ */
+ for (i = 0; i < HPTES_PER_GROUP; i++) {
+ if (mmu_hash_ops.hpte_remove(last_group) == -1)
+ break;
+ }
+ stress_hpt_struct[cpu].last_group[stress_nr_groups() - 1] = -1;
+ }
+
+ if (ea >= PAGE_OFFSET) {
+ /*
+ * We would really like to prefetch to get the TLB loaded, then
+ * remove the PTE before returning from fault interrupt, to
+ * increase the hash fault rate.
+ *
+ * Unfortunately QEMU TCG does not model the TLB in a way that
+ * makes this possible, and systemsim (mambo) emulator does not
+ * bring in TLBs with prefetches (although loads/stores do
+ * work for non-CI PTEs).
+ *
+ * So remember this PTE and clear it on the next hash fault.
+ */
+ memmove(&stress_hpt_struct[cpu].last_group[1],
+ &stress_hpt_struct[cpu].last_group[0],
+ (stress_nr_groups() - 1) * sizeof(unsigned long));
+ stress_hpt_struct[cpu].last_group[0] = hpte_group;
+ }
+}
+
#if defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE)
static DEFINE_RAW_SPINLOCK(linear_map_hash_lock);
return static_branch_unlikely(&stress_slb_key);
}
+extern bool stress_hpt_enabled;
+
+DECLARE_STATIC_KEY_FALSE(stress_hpt_key);
+
+static inline bool stress_hpt(void)
+{
+ return static_branch_unlikely(&stress_hpt_key);
+}
+
+void hpt_do_stress(unsigned long ea, unsigned long hpte_group);
+
void slb_setup_new_exec(void);
void exit_lazy_flush_tlb(struct mm_struct *mm, bool always_flush);
}
/*
- * Serialize against find_current_mm_pte which does lock-less
+ * Serialize against __find_linux_pte() which does lock-less
* lookup in page tables with local interrupts disabled. For huge pages
* it casts pmd_t to pte_t. Since format of pte_t is different from
* pmd_t we want to prevent transit from pmd pointing to page table
* to pmd pointing to huge page (and back) while interrupts are disabled.
* We clear pmd to possibly replace it with page table pointer in
* different code paths. So make sure we wait for the parallel
- * find_current_mm_pte to finish.
+ * __find_linux_pte() to finish.
*/
void serialize_against_pte_lookup(struct mm_struct *mm)
{
{
ppc_md.progress = ppc_printk_progress;
mark_initmem_nx();
- static_branch_enable(&init_mem_is_free);
free_initmem_default(POISON_FREE_INITMEM);
ftrace_free_init_tramp();
}
#include <asm/cacheflush.h>
#include <asm/kdump.h>
#include <mm/mmu_decl.h>
-#include <generated/utsrelease.h>
struct regions {
unsigned long pa_start;
mmu_get_tsize(mmu_virtual_psize), 0);
}
EXPORT_SYMBOL(local_flush_tlb_page);
+
+void local_flush_tlb_page_psize(struct mm_struct *mm,
+ unsigned long vmaddr, int psize)
+{
+ __local_flush_tlb_page(mm, vmaddr, mmu_get_tsize(psize), 0);
+}
+EXPORT_SYMBOL(local_flush_tlb_page_psize);
+
#endif
/*
{
if (sp & 0xf)
return 0; /* must be 16-byte aligned */
- if (!validate_sp(sp, current, STACK_FRAME_OVERHEAD))
+ if (!validate_sp(sp, current))
return 0;
if (sp >= prev_sp + STACK_FRAME_MIN_SIZE)
return 1;
sp = regs->gpr[1];
perf_callchain_store(entry, perf_instruction_pointer(regs));
- if (!validate_sp(sp, current, STACK_FRAME_OVERHEAD))
+ if (!validate_sp(sp, current))
return;
for (;;) {
next_sp = fp[0];
if (next_sp == sp + STACK_INT_FRAME_SIZE &&
- fp[STACK_FRAME_MARKER] == STACK_FRAME_REGS_MARKER) {
+ validate_sp_size(sp, current, STACK_INT_FRAME_SIZE) &&
+ fp[STACK_INT_FRAME_MARKER_LONGS] == STACK_FRAME_REGS_MARKER) {
/*
* This looks like an interrupt frame for an
* interrupt that occurred in the kernel
*/
- regs = (struct pt_regs *)(sp + STACK_FRAME_OVERHEAD);
+ regs = (struct pt_regs *)(sp + STACK_INT_FRAME_REGS);
next_ip = regs->nip;
lr = regs->link;
level = 0;
)
#include I(REQUEST_END)
+#ifdef ENABLE_EVENTS_COUNTERINFO_V6
/*
* Not available for counter_info_version >= 0x8, use
* run_instruction_cycles_by_partition(0x100) instead.
__count(0x10, 8, cycles)
)
#include I(REQUEST_END)
+#endif
#define REQUEST_NAME system_performance_capabilities
#define REQUEST_NUM 0x40
)
#include I(REQUEST_END)
+#ifdef ENABLE_EVENTS_COUNTERINFO_V6
#define REQUEST_NAME processor_bus_utilization_abc_links
#define REQUEST_NUM 0x50
#define REQUEST_IDX_KIND "hw_chip_id=?"
__count(0x28, 8, instructions_completed)
)
#include I(REQUEST_END)
+#endif
/* Processor_core_power_mode (0x95) skipped, no counters */
/* Affinity_domain_information_by_virtual_processor (0xA0) skipped,
.attrs = format_attrs,
};
-static const struct attribute_group event_group = {
+static struct attribute_group event_group = {
.name = "events",
- .attrs = hv_gpci_event_attrs,
+ /* .attrs is set in init */
};
#define HV_CAPS_ATTR(_name, _format) \
int r;
unsigned long hret;
struct hv_perf_caps caps;
+ struct hv_gpci_request_buffer *arg;
hv_gpci_assert_offsets_correct();
/* sampling not supported */
h_gpci_pmu.capabilities |= PERF_PMU_CAP_NO_INTERRUPT;
+ arg = (void *)get_cpu_var(hv_gpci_reqb);
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ /*
+ * hcall H_GET_PERF_COUNTER_INFO populates the output
+ * counter_info_version value based on the system hypervisor.
+ * Pass the counter request 0x10 corresponds to request type
+ * 'Dispatch_timebase_by_processor', to get the supported
+ * counter_info_version.
+ */
+ arg->params.counter_request = cpu_to_be32(0x10);
+
+ r = plpar_hcall_norets(H_GET_PERF_COUNTER_INFO,
+ virt_to_phys(arg), HGPCI_REQ_BUFFER_SIZE);
+ if (r) {
+ pr_devel("hcall failed, can't get supported counter_info_version: 0x%x\n", r);
+ arg->params.counter_info_version_out = 0x8;
+ }
+
+ /*
+ * Use counter_info_version_out value to assign
+ * required hv-gpci event list.
+ */
+ if (arg->params.counter_info_version_out >= 0x8)
+ event_group.attrs = hv_gpci_event_attrs;
+ else
+ event_group.attrs = hv_gpci_event_attrs_v6;
+
+ put_cpu_var(hv_gpci_reqb);
+
r = perf_pmu_register(&h_gpci_pmu, h_gpci_pmu.name, -1);
if (r)
return r;
#define REQUEST_FILE "../hv-gpci-requests.h"
#define NAME_LOWER hv_gpci
#define NAME_UPPER HV_GPCI
+#define ENABLE_EVENTS_COUNTERINFO_V6
#include "req-gen/perf.h"
#undef REQUEST_FILE
#undef NAME_LOWER
#define REQUEST_(r_name, r_value, r_idx_1, r_fields) \
r_fields
+/* Generate event list for platforms with counter_info_version 0x6 or below */
+static __maybe_unused struct attribute *hv_gpci_event_attrs_v6[] = {
+#include REQUEST_FILE
+ NULL
+};
+
+/*
+ * Based on getPerfCountInfo v1.018 documentation, some of the hv-gpci
+ * events were deprecated for platform firmware that supports
+ * counter_info_version 0x8 or above.
+ * Those deprecated events are still part of platform firmware that
+ * support counter_info_version 0x6 and below. As per the getPerfCountInfo
+ * v1.018 documentation there is no counter_info_version 0x7.
+ * Undefining macro ENABLE_EVENTS_COUNTERINFO_V6, to disable the addition of
+ * deprecated events in "hv_gpci_event_attrs" attribute group, for platforms
+ * that supports counter_info_version 0x8 or above.
+ */
+#undef ENABLE_EVENTS_COUNTERINFO_V6
+
+/* Generate event list for platforms with counter_info_version 0x8 or above*/
static __maybe_unused struct attribute *hv_gpci_event_attrs[] = {
#include REQUEST_FILE
NULL
* Copyright (c) 2008-2009 PIKA Technologies
* Sean MacLennan <smaclennan@pikatech.com>
*/
+#include <linux/err.h>
#include <linux/init.h>
#include <linux/of_platform.h>
#include <linux/kthread.h>
+#include <linux/leds.h>
#include <linux/i2c.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
-#include <linux/of_gpio.h>
+#include <linux/gpio/consumer.h>
#include <linux/slab.h>
#include <linux/export.h>
static LIST_HEAD(dtm_shutdown_list);
static void __iomem *dtm_fpga;
-static unsigned green_led, red_led;
-
struct dtm_shutdown {
struct list_head list;
void *arg;
};
-
int pika_dtm_register_shutdown(void (*func)(void *arg), void *arg)
{
struct dtm_shutdown *shutdown;
return -EINVAL;
}
+#define WARP_GREEN_LED 0
+#define WARP_RED_LED 1
+
+static struct gpio_led warp_gpio_led_pins[] = {
+ [WARP_GREEN_LED] = {
+ .name = "green",
+ .default_state = LEDS_DEFSTATE_KEEP,
+ .gpiod = NULL, /* to be filled by pika_setup_leds() */
+ },
+ [WARP_RED_LED] = {
+ .name = "red",
+ .default_state = LEDS_DEFSTATE_KEEP,
+ .gpiod = NULL, /* to be filled by pika_setup_leds() */
+ },
+};
+
+static struct gpio_led_platform_data warp_gpio_led_data = {
+ .leds = warp_gpio_led_pins,
+ .num_leds = ARRAY_SIZE(warp_gpio_led_pins),
+};
+
+static struct platform_device warp_gpio_leds = {
+ .name = "leds-gpio",
+ .id = -1,
+ .dev = {
+ .platform_data = &warp_gpio_led_data,
+ },
+};
+
static irqreturn_t temp_isr(int irq, void *context)
{
struct dtm_shutdown *shutdown;
local_irq_disable();
- gpio_set_value(green_led, 0);
+ gpiod_set_value(warp_gpio_led_pins[WARP_GREEN_LED].gpiod, 0);
/* Run through the shutdown list. */
list_for_each_entry(shutdown, &dtm_shutdown_list, list)
out_be32(dtm_fpga + 0x14, reset);
}
- gpio_set_value(red_led, value);
+ gpiod_set_value(warp_gpio_led_pins[WARP_RED_LED].gpiod, value);
value ^= 1;
mdelay(500);
}
return IRQ_HANDLED;
}
+/*
+ * Because green and red power LEDs are normally driven by leds-gpio driver,
+ * but in case of critical temperature shutdown we want to drive them
+ * ourselves, we acquire both and then create leds-gpio platform device
+ * ourselves, instead of doing it through device tree. This way we can still
+ * keep access to the gpios and use them when needed.
+ */
static int pika_setup_leds(void)
{
struct device_node *np, *child;
+ struct gpio_desc *gpio;
+ struct gpio_led *led;
+ int led_count = 0;
+ int error;
+ int i;
- np = of_find_compatible_node(NULL, NULL, "gpio-leds");
+ np = of_find_compatible_node(NULL, NULL, "warp-power-leds");
if (!np) {
printk(KERN_ERR __FILE__ ": Unable to find leds\n");
return -ENOENT;
}
- for_each_child_of_node(np, child)
- if (of_node_name_eq(child, "green"))
- green_led = of_get_gpio(child, 0);
- else if (of_node_name_eq(child, "red"))
- red_led = of_get_gpio(child, 0);
+ for_each_child_of_node(np, child) {
+ for (i = 0; i < ARRAY_SIZE(warp_gpio_led_pins); i++) {
+ led = &warp_gpio_led_pins[i];
+
+ if (!of_node_name_eq(child, led->name))
+ continue;
+
+ if (led->gpiod) {
+ printk(KERN_ERR __FILE__ ": %s led has already been defined\n",
+ led->name);
+ continue;
+ }
+
+ gpio = fwnode_gpiod_get_index(of_fwnode_handle(child),
+ NULL, 0, GPIOD_ASIS,
+ led->name);
+ error = PTR_ERR_OR_ZERO(gpio);
+ if (error) {
+ printk(KERN_ERR __FILE__ ": Failed to get %s led gpio: %d\n",
+ led->name, error);
+ of_node_put(child);
+ goto err_cleanup_pins;
+ }
+
+ led->gpiod = gpio;
+ led_count++;
+ }
+ }
of_node_put(np);
+ /* Skip device registration if no leds have been defined */
+ if (led_count) {
+ error = platform_device_register(&warp_gpio_leds);
+ if (error) {
+ printk(KERN_ERR __FILE__ ": Unable to add leds-gpio: %d\n",
+ error);
+ goto err_cleanup_pins;
+ }
+ }
+
return 0;
+
+err_cleanup_pins:
+ for (i = 0; i < ARRAY_SIZE(warp_gpio_led_pins); i++) {
+ led = &warp_gpio_led_pins[i];
+ gpiod_put(led->gpiod);
+ led->gpiod = NULL;
+ }
+ return error;
}
static void pika_setup_critical_temp(struct device_node *np,
msi_bitmap_free_hwirqs(&ppc4xx_hsta_msi.bmp, irq, 1);
pr_debug("%s: Teardown IRQ %u (index %u)\n", __func__,
entry->irq, irq);
+ entry->irq = 0;
}
}
/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/linkage.h>
+
#include <asm/reg.h>
#include <asm/ppc_asm.h>
#include <asm/processor.h>
/* local udelay in sram is needed */
- udelay: /* r11 - tb_ticks_per_usec, r12 - usecs, overwrites r13 */
+SYM_FUNC_START_LOCAL(udelay)
+ /* r11 - tb_ticks_per_usec, r12 - usecs, overwrites r13 */
mullw r12, r12, r11
mftb r13 /* start */
add r12, r13, r12 /* end */
cmp cr0, r13, r12
blt 1b
blr
+SYM_FUNC_END(udelay)
sram_code_end:
SAVE_SR(n+2, addr+2); \
SAVE_SR(n+3, addr+3);
-save_regs:
+SYM_FUNC_START_LOCAL(save_regs)
stw r0, 0(r4)
stw r1, 0x4(r4)
stw r2, 0x8(r4)
SAVE_SPRN(TBRU, 0x5b)
blr
+SYM_FUNC_END(save_regs)
/* restore registers */
LOAD_SR(n+2, addr+2); \
LOAD_SR(n+3, addr+3);
-restore_regs:
+SYM_FUNC_START_LOCAL(restore_regs)
lis r4, registers@h
ori r4, r4, registers@l
blr
_ASM_NOKPROBE_SYMBOL(restore_regs)
+SYM_FUNC_END(restore_regs)
* Flush data cache
* Do this by just reading lots of stuff into the cache.
*/
-flush_data_cache:
+SYM_FUNC_START_LOCAL(flush_data_cache)
lis r3,CONFIG_KERNEL_START@h
ori r3,r3,CONFIG_KERNEL_START@l
li r4,NUM_CACHE_LINES
addi r3,r3,L1_CACHE_BYTES /* Next line, please */
bdnz 1b
blr
+SYM_FUNC_END(flush_data_cache)
/**
* mpc52xx_lpbfifo_kick - Trigger the next block of data to be transferred
+ *
+ * @req: Pointer to request structure
*/
static void mpc52xx_lpbfifo_kick(struct mpc52xx_lpbfifo_request *req)
{
/**
* mpc52xx_lpbfifo_irq - IRQ handler for LPB FIFO
+ * @irq: IRQ number to be handled
+ * @dev_id: device ID cookie
*
* On transmit, the dma completion irq triggers before the fifo completion
* triggers. Handle the dma completion here instead of the LPB FIFO Bestcomm
* or nested spinlock condition. The out path is non-trivial, so
* extra fiddling is done to make sure all paths lead to the same
* outbound code.
+ *
+ * Return: irqreturn code (%IRQ_HANDLED)
*/
static irqreturn_t mpc52xx_lpbfifo_irq(int irq, void *dev_id)
{
/**
* mpc52xx_lpbfifo_bcom_irq - IRQ handler for LPB FIFO Bestcomm task
+ * @irq: IRQ number to be handled
+ * @dev_id: device ID cookie
*
* Only used when receiving data.
+ *
+ * Return: irqreturn code (%IRQ_HANDLED)
*/
static irqreturn_t mpc52xx_lpbfifo_bcom_irq(int irq, void *dev_id)
{
}
/**
- * mpc52xx_lpbfifo_bcom_poll - Poll for DMA completion
+ * mpc52xx_lpbfifo_poll - Poll for DMA completion
*/
void mpc52xx_lpbfifo_poll(void)
{
/**
* mpc52xx_lpbfifo_submit - Submit an LPB FIFO transfer request.
* @req: Pointer to request structure
+ *
+ * Return: %0 on success, -errno code on error
*/
int mpc52xx_lpbfifo_submit(struct mpc52xx_lpbfifo_request *req)
{
err_bcom_rx_irq:
bcom_gen_bd_rx_release(lpbfifo.bcom_rx_task);
err_bcom_rx:
+ free_irq(lpbfifo.irq, &lpbfifo);
err_irq:
iounmap(lpbfifo.regs);
lpbfifo.regs = NULL;
goto next;
unreg:
- platform_device_del(pdev);
+ platform_device_put(pdev);
err:
pr_err("%pOF: registration failed\n", np);
next:
* Copyright 2012 by Servergy, Inc.
*/
+#define pr_fmt(fmt) "gpio-halt: " fmt
+
+#include <linux/err.h>
#include <linux/platform_device.h>
#include <linux/device.h>
+#include <linux/gpio/consumer.h>
#include <linux/module.h>
-#include <linux/of_gpio.h>
#include <linux/of_irq.h>
#include <linux/workqueue.h>
#include <linux/reboot.h>
#include <asm/machdep.h>
-static struct device_node *halt_node;
+static struct gpio_desc *halt_gpio;
+static int halt_irq;
static const struct of_device_id child_match[] = {
{
static void __noreturn gpio_halt_cb(void)
{
- enum of_gpio_flags flags;
- int trigger, gpio;
-
- if (!halt_node)
- panic("No reset GPIO information was provided in DT\n");
-
- gpio = of_get_gpio_flags(halt_node, 0, &flags);
-
- if (!gpio_is_valid(gpio))
- panic("Provided GPIO is invalid\n");
-
- trigger = (flags == OF_GPIO_ACTIVE_LOW);
-
- printk(KERN_INFO "gpio-halt: triggering GPIO.\n");
+ pr_info("triggering GPIO.\n");
/* Probably wont return */
- gpio_set_value(gpio, trigger);
+ gpiod_set_value(halt_gpio, 1);
panic("Halt failed\n");
}
* to handle the shutdown/poweroff. */
static irqreturn_t gpio_halt_irq(int irq, void *__data)
{
- printk(KERN_INFO "gpio-halt: shutdown due to power button IRQ.\n");
+ struct platform_device *pdev = __data;
+
+ dev_info(&pdev->dev, "scheduling shutdown due to power button IRQ\n");
schedule_work(&gpio_halt_wq);
return IRQ_HANDLED;
};
-static int gpio_halt_probe(struct platform_device *pdev)
+static int __gpio_halt_probe(struct platform_device *pdev,
+ struct device_node *halt_node)
{
- enum of_gpio_flags flags;
- struct device_node *node = pdev->dev.of_node;
- struct device_node *child_node;
- int gpio, err, irq;
- int trigger;
-
- if (!node)
- return -ENODEV;
-
- /* If there's no matching child, this isn't really an error */
- child_node = of_find_matching_node(node, child_match);
- if (!child_node)
- return 0;
-
- /* Technically we could just read the first one, but punish
- * DT writers for invalid form. */
- if (of_gpio_count(child_node) != 1) {
- err = -EINVAL;
- goto err_put;
- }
-
- /* Get the gpio number relative to the dynamic base. */
- gpio = of_get_gpio_flags(child_node, 0, &flags);
- if (!gpio_is_valid(gpio)) {
- err = -EINVAL;
- goto err_put;
- }
+ int err;
- err = gpio_request(gpio, "gpio-halt");
+ halt_gpio = fwnode_gpiod_get_index(of_fwnode_handle(halt_node),
+ NULL, 0, GPIOD_OUT_LOW, "gpio-halt");
+ err = PTR_ERR_OR_ZERO(halt_gpio);
if (err) {
- printk(KERN_ERR "gpio-halt: error requesting GPIO %d.\n",
- gpio);
- goto err_put;
+ dev_err(&pdev->dev, "failed to request halt GPIO: %d\n", err);
+ return err;
}
- trigger = (flags == OF_GPIO_ACTIVE_LOW);
-
- gpio_direction_output(gpio, !trigger);
-
/* Now get the IRQ which tells us when the power button is hit */
- irq = irq_of_parse_and_map(child_node, 0);
- err = request_irq(irq, gpio_halt_irq, IRQF_TRIGGER_RISING |
- IRQF_TRIGGER_FALLING, "gpio-halt", child_node);
+ halt_irq = irq_of_parse_and_map(halt_node, 0);
+ err = request_irq(halt_irq, gpio_halt_irq,
+ IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING,
+ "gpio-halt", pdev);
if (err) {
- printk(KERN_ERR "gpio-halt: error requesting IRQ %d for "
- "GPIO %d.\n", irq, gpio);
- gpio_free(gpio);
- goto err_put;
+ dev_err(&pdev->dev, "failed to request IRQ %d: %d\n",
+ halt_irq, err);
+ gpiod_put(halt_gpio);
+ halt_gpio = NULL;
+ return err;
}
/* Register our halt function */
ppc_md.halt = gpio_halt_cb;
pm_power_off = gpio_halt_cb;
- printk(KERN_INFO "gpio-halt: registered GPIO %d (%d trigger, %d"
- " irq).\n", gpio, trigger, irq);
+ dev_info(&pdev->dev, "registered halt GPIO, irq: %d\n", halt_irq);
- halt_node = child_node;
return 0;
-
-err_put:
- of_node_put(child_node);
- return err;
}
-static int gpio_halt_remove(struct platform_device *pdev)
+static int gpio_halt_probe(struct platform_device *pdev)
{
- if (halt_node) {
- int gpio = of_get_gpio(halt_node, 0);
- int irq = irq_of_parse_and_map(halt_node, 0);
+ struct device_node *halt_node;
+ int ret;
+
+ if (!pdev->dev.of_node)
+ return -ENODEV;
+
+ /* If there's no matching child, this isn't really an error */
+ halt_node = of_find_matching_node(pdev->dev.of_node, child_match);
+ if (!halt_node)
+ return -ENODEV;
+
+ ret = __gpio_halt_probe(pdev, halt_node);
+ of_node_put(halt_node);
- free_irq(irq, halt_node);
+ return ret;
+}
- ppc_md.halt = NULL;
- pm_power_off = NULL;
+static int gpio_halt_remove(struct platform_device *pdev)
+{
+ free_irq(halt_irq, pdev);
+ cancel_work_sync(&gpio_halt_wq);
- gpio_free(gpio);
+ ppc_md.halt = NULL;
+ pm_power_off = NULL;
- of_node_put(halt_node);
- halt_node = NULL;
- }
+ gpiod_put(halt_gpio);
+ halt_gpio = NULL;
return 0;
}
depends on PPC_BOOK3S_64 && CPU_LITTLE_ENDIAN
select ARCH_HAS_FAST_MULTIPLIER
select PPC_64S_HASH_MMU
+ select PPC_HAS_LBARX_LHARX
config POWERPC_CPU
bool "Generic 32 bits powerpc"
depends on PPC_BOOK3S_64
select ARCH_HAS_FAST_MULTIPLIER
select PPC_64S_HASH_MMU
+ select PPC_HAS_LBARX_LHARX
config POWER8_CPU
bool "POWER8"
depends on PPC_BOOK3S_64
select ARCH_HAS_FAST_MULTIPLIER
select PPC_64S_HASH_MMU
+ select PPC_HAS_LBARX_LHARX
config POWER9_CPU
bool "POWER9"
depends on PPC_BOOK3S_64
select ARCH_HAS_FAST_MULTIPLIER
+ select PPC_HAS_LBARX_LHARX
config POWER10_CPU
bool "POWER10"
config E6500_CPU
bool "Freescale e6500"
depends on PPC64 && PPC_E500
+ select PPC_HAS_LBARX_LHARX
config 405_CPU
bool "40x family"
endchoice
config PPC64_ELF_ABI_V1
- def_bool PPC64 && CPU_BIG_ENDIAN
+ def_bool PPC64 && (CPU_BIG_ENDIAN && !PPC64_BIG_ENDIAN_ELF_ABI_V2)
config PPC64_ELF_ABI_V2
- def_bool PPC64 && CPU_LITTLE_ENDIAN
+ def_bool PPC64 && !PPC64_ELF_ABI_V1
config PPC64_BOOT_WRAPPER
def_bool n
struct vas_window *txwin;
};
-static char *coproc_devnode(struct device *dev, umode_t *mode)
+static char *coproc_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "crypto/%s", dev_name(dev));
}
msi_for_each_desc(entry, &dev->dev, MSI_DESC_ASSOCIATED) {
irq_set_msi_desc(entry->irq, NULL);
irq_dispose_mapping(entry->irq);
+ entry->irq = 0;
}
}
},
};
-static int gpio_mdio_init(void)
+static int __init gpio_mdio_init(void)
{
struct device_node *np;
}
module_init(gpio_mdio_init);
-static void gpio_mdio_exit(void)
+static void __exit gpio_mdio_exit(void)
{
platform_driver_unregister(&gpio_mdio_driver);
if (gpio_regs)
hwirq = virq_to_hw(entry->irq);
irq_set_msi_desc(entry->irq, NULL);
irq_dispose_mapping(entry->irq);
+ entry->irq = 0;
msi_bitmap_free_hwirqs(&msi_mpic->msi_bitmap, hwirq, ALLOC_CHUNK);
}
}
#undef SHOW_GATWICK_IRQS
-int ppc_override_l2cr = 0;
-int ppc_override_l2cr_value;
-int has_l2cache = 0;
+static int has_l2cache;
int pmac_newworld;
const unsigned int *l2cr =
of_get_property(np, "l2cr-value", NULL);
if (l2cr) {
- ppc_override_l2cr = 1;
- ppc_override_l2cr_value = *l2cr;
_set_L2CR(0);
- _set_L2CR(ppc_override_l2cr_value);
+ _set_L2CR(*l2cr);
+ pr_info("L2CR overridden (0x%x), backside cache is %s\n",
+ *l2cr, ((*l2cr) & 0x80000000) ?
+ "enabled" : "disabled");
}
of_node_put(np);
break;
}
}
-
- if (ppc_override_l2cr)
- printk(KERN_INFO "L2CR overridden (0x%x), "
- "backside cache is %s\n",
- ppc_override_l2cr_value,
- (ppc_override_l2cr_value & 0x80000000)
- ? "enabled" : "disabled");
}
#endif
};
ATTRIBUTE_GROUPS(ps3_system_bus_dev);
-struct bus_type ps3_system_bus_type = {
+static struct bus_type ps3_system_bus_type = {
.name = "ps3_system_bus",
.match = ps3_system_bus_match,
.uevent = ps3_system_bus_uevent,
/**
* pseries_eeh_phb_reset - Reset the specified PHB
* @phb: PCI controller
- * @config_adddr: the associated config address
+ * @config_addr: the associated config address
* @option: reset option
*
* Reset the specified PHB/PE
/**
* pseries_eeh_phb_configure_bridge - Configure PCI bridges in the indicated PE
* @phb: PCI controller
- * @config_adddr: the associated config address
+ * @config_addr: the associated config address
*
* The function will be called to reconfigure the bridges included
* in the specified PE so that the mulfunctional PE would be recovered
}
/* Initialize error log size */
- eeh_error_buf_size = rtas_token("rtas-error-log-max");
- if (eeh_error_buf_size == RTAS_UNKNOWN_SERVICE) {
- pr_info("%s: unknown EEH error log size\n",
- __func__);
- eeh_error_buf_size = 1024;
- } else if (eeh_error_buf_size > RTAS_ERROR_LOG_MAX) {
- pr_info("%s: EEH error log size %d exceeds the maximal %d\n",
- __func__, eeh_error_buf_size, RTAS_ERROR_LOG_MAX);
- eeh_error_buf_size = RTAS_ERROR_LOG_MAX;
- }
+ eeh_error_buf_size = rtas_get_error_log_max();
/* Set EEH probe mode */
eeh_add_flag(EEH_PROBE_MODE_DEVTREE | EEH_ENABLE_IO_FOR_LOG);
xics_teardown_cpu();
unregister_slb_shadow(hwcpu);
+ unregister_vpa(hwcpu);
rtas_stop_self();
/* Should never get here... */
/*
* precall must preserve all registers. use unused STK_PARAM()
- * areas to save snapshots and opcode.
+ * areas to save snapshots and opcode. STK_PARAM() in the caller's
+ * frame will be available even on ELFv2 because these are all
+ * variadic functions.
*/
#define HCALL_INST_PRECALL(FIRST_REG) \
mflr r0; \
std r10,STK_PARAM(R10)(r1); \
std r0,16(r1); \
addi r4,r1,STK_PARAM(FIRST_REG); \
- stdu r1,-STACK_FRAME_OVERHEAD(r1); \
+ stdu r1,-STACK_FRAME_MIN_SIZE(r1); \
bl __trace_hcall_entry; \
- ld r3,STACK_FRAME_OVERHEAD+STK_PARAM(R3)(r1); \
- ld r4,STACK_FRAME_OVERHEAD+STK_PARAM(R4)(r1); \
- ld r5,STACK_FRAME_OVERHEAD+STK_PARAM(R5)(r1); \
- ld r6,STACK_FRAME_OVERHEAD+STK_PARAM(R6)(r1); \
- ld r7,STACK_FRAME_OVERHEAD+STK_PARAM(R7)(r1); \
- ld r8,STACK_FRAME_OVERHEAD+STK_PARAM(R8)(r1); \
- ld r9,STACK_FRAME_OVERHEAD+STK_PARAM(R9)(r1); \
- ld r10,STACK_FRAME_OVERHEAD+STK_PARAM(R10)(r1)
+ ld r3,STACK_FRAME_MIN_SIZE+STK_PARAM(R3)(r1); \
+ ld r4,STACK_FRAME_MIN_SIZE+STK_PARAM(R4)(r1); \
+ ld r5,STACK_FRAME_MIN_SIZE+STK_PARAM(R5)(r1); \
+ ld r6,STACK_FRAME_MIN_SIZE+STK_PARAM(R6)(r1); \
+ ld r7,STACK_FRAME_MIN_SIZE+STK_PARAM(R7)(r1); \
+ ld r8,STACK_FRAME_MIN_SIZE+STK_PARAM(R8)(r1); \
+ ld r9,STACK_FRAME_MIN_SIZE+STK_PARAM(R9)(r1); \
+ ld r10,STACK_FRAME_MIN_SIZE+STK_PARAM(R10)(r1)
/*
* postcall is performed immediately before function return which
* allows liberal use of volatile registers.
*/
#define __HCALL_INST_POSTCALL \
- ld r0,STACK_FRAME_OVERHEAD+STK_PARAM(R3)(r1); \
- std r3,STACK_FRAME_OVERHEAD+STK_PARAM(R3)(r1); \
+ ld r0,STACK_FRAME_MIN_SIZE+STK_PARAM(R3)(r1); \
+ std r3,STACK_FRAME_MIN_SIZE+STK_PARAM(R3)(r1); \
mr r4,r3; \
mr r3,r0; \
bl __trace_hcall_exit; \
- ld r0,STACK_FRAME_OVERHEAD+16(r1); \
- addi r1,r1,STACK_FRAME_OVERHEAD; \
+ ld r0,STACK_FRAME_MIN_SIZE+16(r1); \
+ addi r1,r1,STACK_FRAME_MIN_SIZE; \
ld r3,STK_PARAM(R3)(r1); \
mtlr r0
mr r7,r8
mr r8,r9
mr r9,r10
- ld r10,STACK_FRAME_OVERHEAD+STK_PARAM(R11)(r1)
- ld r11,STACK_FRAME_OVERHEAD+STK_PARAM(R12)(r1)
- ld r12,STACK_FRAME_OVERHEAD+STK_PARAM(R13)(r1)
+ ld r10,STACK_FRAME_MIN_SIZE+STK_PARAM(R11)(r1)
+ ld r11,STACK_FRAME_MIN_SIZE+STK_PARAM(R12)(r1)
+ ld r12,STACK_FRAME_MIN_SIZE+STK_PARAM(R13)(r1)
HVSC
mr r0,r12
- ld r12,STACK_FRAME_OVERHEAD+STK_PARAM(R4)(r1)
+ ld r12,STACK_FRAME_MIN_SIZE+STK_PARAM(R4)(r1)
std r4,0(r12)
std r5,8(r12)
std r6,16(r12)
* Set up the page with TCE data, looping through and setting
* the values.
*/
- limit = min_t(long, npages, 4096/TCE_ENTRY_SIZE);
+ limit = min_t(long, npages, 4096 / TCE_ENTRY_SIZE);
for (l = 0; l < limit; l++) {
tcep[l] = cpu_to_be64(proto_tce | rpn << tceshift);
prod_others();
}
/*
- * Execution may have been suspended for several seconds, so
- * reset the watchdog.
+ * Execution may have been suspended for several seconds, so reset
+ * the watchdogs. touch_nmi_watchdog() also touches the soft lockup
+ * watchdog.
*/
+ rcu_cpu_stall_reset();
touch_nmi_watchdog();
+
return ret;
}
case H_FUNCTION:
err = -ENXIO;
break;
- case H_P1:
+ case H_PARAMETER:
case H_P2:
case H_P3:
case H_P4:
err = -EEXIST;
break;
case H_ABORTED:
- err = -EINTR;
+ err = -EIO;
break;
default:
err = -EINVAL;
if (consumer > PKS_OS_OWNER)
return ERR_PTR(-EINVAL);
- auth = kmalloc(struct_size(auth, password, maxpwsize), GFP_KERNEL);
+ auth = kzalloc(struct_size(auth, password, maxpwsize), GFP_KERNEL);
if (!auth)
return ERR_PTR(-ENOMEM);
auth->version = 1;
auth->consumer = consumer;
- auth->rsvd0 = 0;
- auth->rsvd1 = 0;
- if (consumer == PKS_FW_OWNER || consumer == PKS_BOOTLOADER_OWNER) {
- auth->passwordlength = 0;
+ if (consumer == PKS_FW_OWNER || consumer == PKS_BOOTLOADER_OWNER)
return auth;
- }
memcpy(auth->password, ospassword, ospasswordlength);
if (!rc)
rc = plpks_confirm_object_flushed(label, auth);
- if (rc)
- pr_err("Failed to write variable %s for component %s with error %d\n",
- var.name, var.component, rc);
-
rc = pseries_status_to_err(rc);
kfree(label);
out:
if (!rc)
rc = plpks_confirm_object_flushed(label, auth);
- if (rc)
- pr_err("Failed to remove variable %s for component %s with error %d\n",
- vname.name, component, rc);
-
rc = pseries_status_to_err(rc);
kfree(label);
out:
{
unsigned long retbuf[PLPAR_HCALL_BUFSIZE] = { 0 };
struct plpks_auth *auth;
- struct label *label;
+ struct label *label = NULL;
u8 *output;
int rc;
if (var->namelen > MAX_NAME_SIZE)
return -EINVAL;
- auth = construct_auth(PKS_OS_OWNER);
+ auth = construct_auth(consumer);
if (IS_ERR(auth))
return PTR_ERR(auth);
- label = construct_label(var->component, var->os, var->name,
- var->namelen);
- if (IS_ERR(label)) {
- rc = PTR_ERR(label);
- goto out_free_auth;
+ if (consumer == PKS_OS_OWNER) {
+ label = construct_label(var->component, var->os, var->name,
+ var->namelen);
+ if (IS_ERR(label)) {
+ rc = PTR_ERR(label);
+ goto out_free_auth;
+ }
}
output = kzalloc(maxobjsize, GFP_KERNEL);
goto out_free_label;
}
- rc = plpar_hcall(H_PKS_READ_OBJECT, retbuf, virt_to_phys(auth),
- virt_to_phys(label), label->size, virt_to_phys(output),
- maxobjsize);
+ if (consumer == PKS_OS_OWNER)
+ rc = plpar_hcall(H_PKS_READ_OBJECT, retbuf, virt_to_phys(auth),
+ virt_to_phys(label), label->size, virt_to_phys(output),
+ maxobjsize);
+ else
+ rc = plpar_hcall(H_PKS_READ_OBJECT, retbuf, virt_to_phys(auth),
+ virt_to_phys(var->name), var->namelen, virt_to_phys(output),
+ maxobjsize);
+
if (rc != H_SUCCESS) {
- pr_err("Failed to read variable %s for component %s with error %d\n",
- var->name, var->component, rc);
rc = pseries_status_to_err(rc);
goto out_free_output;
}
#define WORLDREADABLE 0x08000000
#define SIGNEDUPDATE 0x01000000
-#define PLPKS_VAR_LINUX 0x01
+#define PLPKS_VAR_LINUX 0x02
#define PLPKS_VAR_COMMON 0x04
struct plpks_var {
msi_data = irq_get_chip_data(entry->irq);
irq_set_msi_desc(entry->irq, NULL);
irq_dispose_mapping(entry->irq);
+ entry->irq = 0;
msi_bitmap_free_hwirqs(&msi_data->bitmap, hwirq, 1);
}
}
}
/*
+ * If there's no PCI host bridge with ISA then check for
+ * PCI host bridge with alias "pci0" (first PCI host bridge).
+ */
+ np = of_find_node_by_path("pci0");
+ if (np && of_match_node(pci_ids, np) && of_device_is_available(np)) {
+ fsl_pci_primary = np;
+ of_node_put(np);
+ return;
+ }
+ if (np)
+ of_node_put(np);
+
+ /*
* If there's no PCI host bridge with ISA, arbitrarily
* designate one as primary. This can go away once
* various bugs with primary-less systems are fixed.
#define MPIC_MSGR_REGISTERS_PER_BLOCK 4
#define MPIC_MSGR_STRIDE 0x10
-#define MPIC_MSGR_MER_OFFSET 0x100
+#define MPIC_MSGR_MER_OFFSET (0x100 / sizeof(u32))
#define MSGR_INUSE 0
#define MSGR_FREE 1
reg_number = block_number * MPIC_MSGR_REGISTERS_PER_BLOCK + i;
msgr->base = msgr_block_addr + i * MPIC_MSGR_STRIDE;
- msgr->mer = (u32 *)((u8 *)msgr->base + MPIC_MSGR_MER_OFFSET);
+ msgr->mer = msgr->base + MPIC_MSGR_MER_OFFSET;
msgr->in_use = MSGR_FREE;
msgr->num = i;
raw_spin_lock_init(&msgr->lock);
hwirq = virq_to_hw(entry->irq);
irq_set_msi_desc(entry->irq, NULL);
irq_dispose_mapping(entry->irq);
+ entry->irq = 0;
msi_bitmap_free_hwirqs(&msi_mpic->msi_bitmap, hwirq, 1);
}
}
static void __init xive_native_setup_pools(void)
{
/* Allocate a pool big enough */
- pr_debug("XIVE: Allocating VP block for pool size %u\n", nr_cpu_ids);
+ pr_debug("Allocating VP block for pool size %u\n", nr_cpu_ids);
xive_pool_vps = xive_native_alloc_vp_block(nr_cpu_ids);
if (WARN_ON(xive_pool_vps == XIVE_INVALID_VP))
- pr_err("XIVE: Failed to allocate pool VP, KVM might not function\n");
+ pr_err("Failed to allocate pool VP, KVM might not function\n");
- pr_debug("XIVE: Pool VPs allocated at 0x%x for %u max CPUs\n",
+ pr_debug("Pool VPs allocated at 0x%x for %u max CPUs\n",
xive_pool_vps, nr_cpu_ids);
}
data->trig_mmio = ioremap(data->trig_page, 1u << data->esb_shift);
if (!data->trig_mmio) {
+ iounmap(data->eoi_mmio);
pr_err("Failed to map trigger page for irq 0x%x\n", hw_irq);
return -ENOMEM;
}
cmd = inchar();
switch (cmd) {
- static const char badaddr[] = "Only kernel addresses are permitted for breakpoints\n";
- int mode;
- case 'd': /* bd - hardware data breakpoint */
+ case 'd': { /* bd - hardware data breakpoint */
+ static const char badaddr[] = "Only kernel addresses are permitted for breakpoints\n";
+ int mode;
if (xmon_is_ro) {
printf(xmon_ro_msg);
break;
force_enable_xmon();
break;
+ }
case 'i': /* bi - hardware instr breakpoint */
if (xmon_is_ro) {
}
#define LRSAVE_OFFSET (STACK_FRAME_LR_SAVE * sizeof(unsigned long))
-#define MARKER_OFFSET (STACK_FRAME_MARKER * sizeof(unsigned long))
static void xmon_show_stack(unsigned long sp, unsigned long lr,
unsigned long pc)
xmon_print_symbol(ip, " ", "\n");
}
- /* Look for "regshere" marker to see if this is
+ /* Look for "regs" marker to see if this is
an exception frame. */
- if (mread(sp + MARKER_OFFSET, &marker, sizeof(unsigned long))
+ if (mread(sp + STACK_INT_FRAME_MARKER, &marker, sizeof(unsigned long))
&& marker == STACK_FRAME_REGS_MARKER) {
- if (mread(sp + STACK_FRAME_OVERHEAD, ®s, sizeof(regs))
- != sizeof(regs)) {
+ if (mread(sp + STACK_INT_FRAME_REGS, ®s, sizeof(regs)) != sizeof(regs)) {
printf("Couldn't read registers at %lx\n",
- sp + STACK_FRAME_OVERHEAD);
+ sp + STACK_INT_FRAME_REGS);
break;
}
printf("--- Exception: %lx %s at ", regs.trap,
endif
ifeq ($(CONFIG_LD_IS_LLD),y)
-ifeq ($(shell test $(CONFIG_LLD_VERSION) -lt 150000; echo $$?),0)
+ifeq ($(call test-lt, $(CONFIG_LLD_VERSION), 150000),y)
KBUILD_CFLAGS += -mno-relax
KBUILD_AFLAGS += -mno-relax
ifndef CONFIG_AS_IS_LLVM
struct zpci_dev {
struct zpci_bus *zbus;
struct list_head entry; /* list of all zpci_devices, needed for hotplug, etc. */
+ struct list_head iommu_list;
struct kref kref;
+ struct rcu_head rcu;
struct hotplug_slot hotplug_slot;
enum zpci_state state;
/* DMA stuff */
unsigned long *dma_table;
- spinlock_t dma_table_lock;
int tlb_refresh;
spinlock_t iommu_bitmap_lock;
bool zpci_is_device_configured(struct zpci_dev *zdev);
int zpci_hot_reset_device(struct zpci_dev *zdev);
-int zpci_register_ioat(struct zpci_dev *, u8, u64, u64, u64);
+int zpci_register_ioat(struct zpci_dev *, u8, u64, u64, u64, u8 *);
int zpci_unregister_ioat(struct zpci_dev *, u8);
void zpci_remove_reserved_devices(void);
void zpci_update_fh(struct zpci_dev *zdev, u32 fh);
#include <linux/hugetlb.h>
#include <linux/kmemleak.h>
+#include <asm/archrandom.h>
#include <asm/boot_data.h>
#include <asm/ipl.h>
#include <asm/facility.h>
static int kvm_s390_pci_register_kvm(void *opaque, struct kvm *kvm)
{
struct zpci_dev *zdev = opaque;
+ u8 status;
int rc;
if (!zdev)
/* Re-register the IOMMU that was already created */
rc = zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma,
- virt_to_phys(zdev->dma_table));
+ virt_to_phys(zdev->dma_table), &status);
if (rc)
goto clear_gisa;
{
struct zpci_dev *zdev = opaque;
struct kvm *kvm;
+ u8 status;
if (!zdev)
return;
/* Re-register the IOMMU that was already created */
zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma,
- virt_to_phys(zdev->dma_table));
+ virt_to_phys(zdev->dma_table), &status);
out:
spin_lock(&kvm->arch.kzdev_list_lock);
/* Modify PCI: Register I/O address translation parameters */
int zpci_register_ioat(struct zpci_dev *zdev, u8 dmaas,
- u64 base, u64 limit, u64 iota)
+ u64 base, u64 limit, u64 iota, u8 *status)
{
u64 req = ZPCI_CREATE_REQ(zdev->fh, dmaas, ZPCI_MOD_FC_REG_IOAT);
struct zpci_fib fib = {0};
- u8 cc, status;
+ u8 cc;
WARN_ON_ONCE(iota & 0x3fff);
fib.pba = base;
fib.pal = limit;
fib.iota = iota | ZPCI_IOTA_RTTO_FLAG;
fib.gd = zdev->gisa;
- cc = zpci_mod_fc(req, &fib, &status);
+ cc = zpci_mod_fc(req, &fib, status);
if (cc)
- zpci_dbg(3, "reg ioat fid:%x, cc:%d, status:%d\n", zdev->fid, cc, status);
+ zpci_dbg(3, "reg ioat fid:%x, cc:%d, status:%d\n", zdev->fid, cc, *status);
return cc;
}
EXPORT_SYMBOL_GPL(zpci_register_ioat);
*/
int zpci_hot_reset_device(struct zpci_dev *zdev)
{
+ u8 status;
int rc;
zpci_dbg(3, "rst fid:%x, fh:%x\n", zdev->fid, zdev->fh);
if (zdev->dma_table)
rc = zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma,
- virt_to_phys(zdev->dma_table));
+ virt_to_phys(zdev->dma_table), &status);
else
rc = zpci_dma_init_device(zdev);
if (rc) {
break;
}
zpci_dbg(3, "rem fid:%x\n", zdev->fid);
- kfree(zdev);
+ kfree_rcu(zdev, rcu);
}
int zpci_report_error(struct pci_dev *pdev,
kmem_cache_free(dma_page_table_cache, table);
}
-static unsigned long *dma_get_seg_table_origin(unsigned long *entry)
+static unsigned long *dma_get_seg_table_origin(unsigned long *rtep)
{
+ unsigned long old_rte, rte;
unsigned long *sto;
- if (reg_entry_isvalid(*entry))
- sto = get_rt_sto(*entry);
- else {
+ rte = READ_ONCE(*rtep);
+ if (reg_entry_isvalid(rte)) {
+ sto = get_rt_sto(rte);
+ } else {
sto = dma_alloc_cpu_table();
if (!sto)
return NULL;
- set_rt_sto(entry, virt_to_phys(sto));
- validate_rt_entry(entry);
- entry_clr_protected(entry);
+ set_rt_sto(&rte, virt_to_phys(sto));
+ validate_rt_entry(&rte);
+ entry_clr_protected(&rte);
+
+ old_rte = cmpxchg(rtep, ZPCI_TABLE_INVALID, rte);
+ if (old_rte != ZPCI_TABLE_INVALID) {
+ /* Somone else was faster, use theirs */
+ dma_free_cpu_table(sto);
+ sto = get_rt_sto(old_rte);
+ }
}
return sto;
}
-static unsigned long *dma_get_page_table_origin(unsigned long *entry)
+static unsigned long *dma_get_page_table_origin(unsigned long *step)
{
+ unsigned long old_ste, ste;
unsigned long *pto;
- if (reg_entry_isvalid(*entry))
- pto = get_st_pto(*entry);
- else {
+ ste = READ_ONCE(*step);
+ if (reg_entry_isvalid(ste)) {
+ pto = get_st_pto(ste);
+ } else {
pto = dma_alloc_page_table();
if (!pto)
return NULL;
- set_st_pto(entry, virt_to_phys(pto));
- validate_st_entry(entry);
- entry_clr_protected(entry);
+ set_st_pto(&ste, virt_to_phys(pto));
+ validate_st_entry(&ste);
+ entry_clr_protected(&ste);
+
+ old_ste = cmpxchg(step, ZPCI_TABLE_INVALID, ste);
+ if (old_ste != ZPCI_TABLE_INVALID) {
+ /* Somone else was faster, use theirs */
+ dma_free_page_table(pto);
+ pto = get_st_pto(old_ste);
+ }
}
return pto;
}
return &pto[px];
}
-void dma_update_cpu_trans(unsigned long *entry, phys_addr_t page_addr, int flags)
+void dma_update_cpu_trans(unsigned long *ptep, phys_addr_t page_addr, int flags)
{
+ unsigned long pte;
+
+ pte = READ_ONCE(*ptep);
if (flags & ZPCI_PTE_INVALID) {
- invalidate_pt_entry(entry);
+ invalidate_pt_entry(&pte);
} else {
- set_pt_pfaa(entry, page_addr);
- validate_pt_entry(entry);
+ set_pt_pfaa(&pte, page_addr);
+ validate_pt_entry(&pte);
}
if (flags & ZPCI_TABLE_PROTECTED)
- entry_set_protected(entry);
+ entry_set_protected(&pte);
else
- entry_clr_protected(entry);
+ entry_clr_protected(&pte);
+
+ xchg(ptep, pte);
}
static int __dma_update_trans(struct zpci_dev *zdev, phys_addr_t pa,
{
unsigned int nr_pages = PAGE_ALIGN(size) >> PAGE_SHIFT;
phys_addr_t page_addr = (pa & PAGE_MASK);
- unsigned long irq_flags;
unsigned long *entry;
int i, rc = 0;
if (!nr_pages)
return -EINVAL;
- spin_lock_irqsave(&zdev->dma_table_lock, irq_flags);
- if (!zdev->dma_table) {
- rc = -EINVAL;
- goto out_unlock;
- }
+ if (!zdev->dma_table)
+ return -EINVAL;
for (i = 0; i < nr_pages; i++) {
entry = dma_walk_cpu_trans(zdev->dma_table, dma_addr);
dma_update_cpu_trans(entry, page_addr, flags);
}
}
-out_unlock:
- spin_unlock_irqrestore(&zdev->dma_table_lock, irq_flags);
return rc;
}
int zpci_dma_init_device(struct zpci_dev *zdev)
{
+ u8 status;
int rc;
/*
WARN_ON(zdev->s390_domain);
spin_lock_init(&zdev->iommu_bitmap_lock);
- spin_lock_init(&zdev->dma_table_lock);
zdev->dma_table = dma_alloc_cpu_table();
if (!zdev->dma_table) {
}
if (zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma,
- virt_to_phys(zdev->dma_table))) {
+ virt_to_phys(zdev->dma_table), &status)) {
rc = -EIO;
goto free_bitmap;
}
select GENERIC_PCI_IOMAP if PCI
select GENERIC_SCHED_CLOCK
select GENERIC_SMP_IDLE_THREAD
- select GUP_GET_PTE_LOW_HIGH if X2TLB
+ select GUP_GET_PXX_LOW_HIGH if X2TLB
select HAVE_ARCH_AUDITSYSCALL
select HAVE_ARCH_KGDB
select HAVE_ARCH_SECCOMP_FILTER
CONFIG_USB_EMI62=m
CONFIG_USB_EMI26=m
CONFIG_USB_SISUSBVGA=m
-CONFIG_USB_SISUSBVGA_CON=y
CONFIG_EXT2_FS=y
CONFIG_EXT3_FS=y
# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
#define pmd_ERROR(e) \
printk("%s:%d: bad pmd %016llx.\n", __FILE__, __LINE__, pmd_val(e))
-typedef struct { unsigned long long pmd; } pmd_t;
+typedef struct {
+ struct {
+ unsigned long pmd_low;
+ unsigned long pmd_high;
+ };
+ unsigned long long pmd;
+} pmd_t;
#define pmd_val(x) ((x).pmd)
-#define __pmd(x) ((pmd_t) { (x) } )
+#define __pmd(x) ((pmd_t) { .pmd = (x) } )
static inline pmd_t *pud_pgtable(pud_t pud)
{
#define pud_populate(mm, pud, pmd) \
set_pud(pud, __pud(_PAGE_TABLE + __pa(pmd)))
-#ifdef CONFIG_64BIT
-#define set_pud(pudptr, pudval) set_64bit((u64 *) (pudptr), pud_val(pudval))
-#else
#define set_pud(pudptr, pudval) (*(pudptr) = (pudval))
-#endif
static inline int pgd_newpage(pgd_t pgd)
{
static inline void pgd_mkuptodate(pgd_t pgd) { pgd_val(pgd) &= ~_PAGE_NEWPAGE; }
-#ifdef CONFIG_64BIT
-#define set_pmd(pmdptr, pmdval) set_64bit((u64 *) (pmdptr), pmd_val(pmdval))
-#else
#define set_pmd(pmdptr, pmdval) (*(pmdptr) = (pmdval))
-#endif
static inline void pud_clear (pud_t *pud)
{
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
select GENERIC_VDSO_TIME_NS
- select GUP_GET_PTE_LOW_HIGH if X86_PAE
+ select GUP_GET_PXX_LOW_HIGH if X86_PAE
select HARDIRQS_SW_RESEND
select HARDLOCKUP_CHECK_TIMESTAMP if X86_64
select HAVE_ACPI_APEI if ACPI
select HAVE_CONTEXT_TRACKING_USER_OFFSTACK if HAVE_CONTEXT_TRACKING_USER
select HAVE_C_RECORDMCOUNT
select HAVE_OBJTOOL_MCOUNT if HAVE_OBJTOOL
+ select HAVE_OBJTOOL_NOP_MCOUNT if HAVE_OBJTOOL_MCOUNT
select HAVE_BUILDTIME_MCOUNT_SORT
select HAVE_DEBUG_KMEMLEAK
select HAVE_DMA_CONTIGUOUS
KBUILD_LDFLAGS += -m elf_$(UTS_MACHINE)
ifdef CONFIG_LTO_CLANG
-ifeq ($(shell test $(CONFIG_LLD_VERSION) -lt 130000; echo $$?),0)
+ifeq ($(call test-lt, $(CONFIG_LLD_VERSION), 130000),y)
KBUILD_LDFLAGS += -plugin-opt=-stack-alignment=$(if $(CONFIG_X86_32),4,8)
endif
endif
* you need to test for the feature in boot_cpu_data.
*/
-/*
- * CMPXCHG8B only writes to the target if we had the previous
- * value in registers, otherwise it acts as a read and gives us the
- * "new previous" value. That is why there is a loop. Preloading
- * EDX:EAX is a performance optimization: in the common case it means
- * we need only one locked operation.
- *
- * A SIMD/3DNOW!/MMX/FPU 64-bit store here would require at the very
- * least an FPU save and/or %cr0.ts manipulation.
- *
- * cmpxchg8b must be used with the lock prefix here to allow the
- * instruction to be executed atomically. We need to have the reader
- * side to see the coherent 64bit value.
- */
-static inline void set_64bit(volatile u64 *ptr, u64 value)
-{
- u32 low = value;
- u32 high = value >> 32;
- u64 prev = *ptr;
-
- asm volatile("\n1:\t"
- LOCK_PREFIX "cmpxchg8b %0\n\t"
- "jnz 1b"
- : "=m" (*ptr), "+A" (prev)
- : "b" (low), "c" (high)
- : "memory");
-}
-
#ifdef CONFIG_X86_CMPXCHG64
#define arch_cmpxchg64(ptr, o, n) \
((__typeof__(*(ptr)))__cmpxchg64((ptr), (unsigned long long)(o), \
#ifndef _ASM_X86_CMPXCHG_64_H
#define _ASM_X86_CMPXCHG_64_H
-static inline void set_64bit(volatile u64 *ptr, u64 val)
-{
- *ptr = val;
-}
-
#define arch_cmpxchg64(ptr, o, n) \
({ \
BUILD_BUG_ON(sizeof(*(ptr)) != 8); \
};
#define CPU_ENTRY_AREA_SIZE (sizeof(struct cpu_entry_area))
-#define CPU_ENTRY_AREA_ARRAY_SIZE (CPU_ENTRY_AREA_SIZE * NR_CPUS)
-
-/* Total size includes the readonly IDT mapping page as well: */
-#define CPU_ENTRY_AREA_TOTAL_SIZE (CPU_ENTRY_AREA_ARRAY_SIZE + PAGE_SIZE)
DECLARE_PER_CPU(struct cpu_entry_area *, cpu_entry_area);
DECLARE_PER_CPU(struct cea_exception_stacks *, cea_exception_stacks);
#ifdef CONFIG_KASAN
void __init kasan_early_init(void);
void __init kasan_init(void);
+void __init kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid);
#else
static inline void kasan_early_init(void) { }
static inline void kasan_init(void) { }
+static inline void kasan_populate_shadow_for_vaddr(void *va, size_t size,
+ int nid) { }
#endif
#endif
#define PAGE_SIZE (_AC(1,UL) << PAGE_SHIFT)
#define PAGE_MASK (~(PAGE_SIZE-1))
-#define PMD_PAGE_SIZE (_AC(1, UL) << PMD_SHIFT)
-#define PMD_PAGE_MASK (~(PMD_PAGE_SIZE-1))
-
-#define PUD_PAGE_SIZE (_AC(1, UL) << PUD_SHIFT)
-#define PUD_PAGE_MASK (~(PUD_PAGE_SIZE-1))
-
#define __VIRTUAL_MASK ((1UL << __VIRTUAL_MASK_SHIFT) - 1)
-/* Cast *PAGE_MASK to a signed type so that it is sign-extended if
+/* Cast P*D_MASK to a signed type so that it is sign-extended if
virtual addresses are 32-bits but physical addresses are larger
(ie, 32-bit PAE). */
#define PHYSICAL_PAGE_MASK (((signed long)PAGE_MASK) & __PHYSICAL_MASK)
-#define PHYSICAL_PMD_PAGE_MASK (((signed long)PMD_PAGE_MASK) & __PHYSICAL_MASK)
-#define PHYSICAL_PUD_PAGE_MASK (((signed long)PUD_PAGE_MASK) & __PHYSICAL_MASK)
+#define PHYSICAL_PMD_PAGE_MASK (((signed long)PMD_MASK) & __PHYSICAL_MASK)
+#define PHYSICAL_PUD_PAGE_MASK (((signed long)PUD_MASK) & __PHYSICAL_MASK)
#define HPAGE_SHIFT PMD_SHIFT
#define HPAGE_SIZE (_AC(1,UL) << HPAGE_SHIFT)
#ifndef _ASM_X86_PGTABLE_3LEVEL_H
#define _ASM_X86_PGTABLE_3LEVEL_H
-#include <asm/atomic64_32.h>
-
/*
* Intel Physical Address Extension (PAE) Mode - three-level page
* tables on PPro+ CPUs.
pr_err("%s:%d: bad pgd %p(%016Lx)\n", \
__FILE__, __LINE__, &(e), pgd_val(e))
-/* Rules for using set_pte: the pte being assigned *must* be
+#define pxx_xchg64(_pxx, _ptr, _val) ({ \
+ _pxx##val_t *_p = (_pxx##val_t *)_ptr; \
+ _pxx##val_t _o = *_p; \
+ do { } while (!try_cmpxchg64(_p, &_o, (_val))); \
+ native_make_##_pxx(_o); \
+})
+
+/*
+ * Rules for using set_pte: the pte being assigned *must* be
* either not present or in a state where the hardware will
* not attempt to update the pte. In places where this is
* not possible, use pte_get_and_clear to obtain the old pte
*/
static inline void native_set_pte(pte_t *ptep, pte_t pte)
{
- ptep->pte_high = pte.pte_high;
+ WRITE_ONCE(ptep->pte_high, pte.pte_high);
smp_wmb();
- ptep->pte_low = pte.pte_low;
-}
-
-#define pmd_read_atomic pmd_read_atomic
-/*
- * pte_offset_map_lock() on 32-bit PAE kernels was reading the pmd_t with
- * a "*pmdp" dereference done by GCC. Problem is, in certain places
- * where pte_offset_map_lock() is called, concurrent page faults are
- * allowed, if the mmap_lock is hold for reading. An example is mincore
- * vs page faults vs MADV_DONTNEED. On the page fault side
- * pmd_populate() rightfully does a set_64bit(), but if we're reading the
- * pmd_t with a "*pmdp" on the mincore side, a SMP race can happen
- * because GCC will not read the 64-bit value of the pmd atomically.
- *
- * To fix this all places running pte_offset_map_lock() while holding the
- * mmap_lock in read mode, shall read the pmdp pointer using this
- * function to know if the pmd is null or not, and in turn to know if
- * they can run pte_offset_map_lock() or pmd_trans_huge() or other pmd
- * operations.
- *
- * Without THP if the mmap_lock is held for reading, the pmd can only
- * transition from null to not null while pmd_read_atomic() runs. So
- * we can always return atomic pmd values with this function.
- *
- * With THP if the mmap_lock is held for reading, the pmd can become
- * trans_huge or none or point to a pte (and in turn become "stable")
- * at any time under pmd_read_atomic(). We could read it truly
- * atomically here with an atomic64_read() for the THP enabled case (and
- * it would be a whole lot simpler), but to avoid using cmpxchg8b we
- * only return an atomic pmdval if the low part of the pmdval is later
- * found to be stable (i.e. pointing to a pte). We are also returning a
- * 'none' (zero) pmdval if the low part of the pmd is zero.
- *
- * In some cases the high and low part of the pmdval returned may not be
- * consistent if THP is enabled (the low part may point to previously
- * mapped hugepage, while the high part may point to a more recently
- * mapped hugepage), but pmd_none_or_trans_huge_or_clear_bad() only
- * needs the low part of the pmd to be read atomically to decide if the
- * pmd is unstable or not, with the only exception when the low part
- * of the pmd is zero, in which case we return a 'none' pmd.
- */
-static inline pmd_t pmd_read_atomic(pmd_t *pmdp)
-{
- pmdval_t ret;
- u32 *tmp = (u32 *)pmdp;
-
- ret = (pmdval_t) (*tmp);
- if (ret) {
- /*
- * If the low part is null, we must not read the high part
- * or we can end up with a partial pmd.
- */
- smp_rmb();
- ret |= ((pmdval_t)*(tmp + 1)) << 32;
- }
-
- return (pmd_t) { ret };
+ WRITE_ONCE(ptep->pte_low, pte.pte_low);
}
static inline void native_set_pte_atomic(pte_t *ptep, pte_t pte)
{
- set_64bit((unsigned long long *)(ptep), native_pte_val(pte));
+ pxx_xchg64(pte, ptep, native_pte_val(pte));
}
static inline void native_set_pmd(pmd_t *pmdp, pmd_t pmd)
{
- set_64bit((unsigned long long *)(pmdp), native_pmd_val(pmd));
+ pxx_xchg64(pmd, pmdp, native_pmd_val(pmd));
}
static inline void native_set_pud(pud_t *pudp, pud_t pud)
#ifdef CONFIG_PAGE_TABLE_ISOLATION
pud.p4d.pgd = pti_set_user_pgtbl(&pudp->p4d.pgd, pud.p4d.pgd);
#endif
- set_64bit((unsigned long long *)(pudp), native_pud_val(pud));
+ pxx_xchg64(pud, pudp, native_pud_val(pud));
}
/*
static inline void native_pte_clear(struct mm_struct *mm, unsigned long addr,
pte_t *ptep)
{
- ptep->pte_low = 0;
+ WRITE_ONCE(ptep->pte_low, 0);
smp_wmb();
- ptep->pte_high = 0;
+ WRITE_ONCE(ptep->pte_high, 0);
}
-static inline void native_pmd_clear(pmd_t *pmd)
+static inline void native_pmd_clear(pmd_t *pmdp)
{
- u32 *tmp = (u32 *)pmd;
- *tmp = 0;
+ WRITE_ONCE(pmdp->pmd_low, 0);
smp_wmb();
- *(tmp + 1) = 0;
+ WRITE_ONCE(pmdp->pmd_high, 0);
}
static inline void native_pud_clear(pud_t *pudp)
*/
}
+
#ifdef CONFIG_SMP
static inline pte_t native_ptep_get_and_clear(pte_t *ptep)
{
- pte_t res;
-
- res.pte = (pteval_t)arch_atomic64_xchg((atomic64_t *)ptep, 0);
-
- return res;
+ return pxx_xchg64(pte, ptep, 0ULL);
}
-#else
-#define native_ptep_get_and_clear(xp) native_local_ptep_get_and_clear(xp)
-#endif
-union split_pmd {
- struct {
- u32 pmd_low;
- u32 pmd_high;
- };
- pmd_t pmd;
-};
-
-#ifdef CONFIG_SMP
static inline pmd_t native_pmdp_get_and_clear(pmd_t *pmdp)
{
- union split_pmd res, *orig = (union split_pmd *)pmdp;
-
- /* xchg acts as a barrier before setting of the high bits */
- res.pmd_low = xchg(&orig->pmd_low, 0);
- res.pmd_high = orig->pmd_high;
- orig->pmd_high = 0;
+ return pxx_xchg64(pmd, pmdp, 0ULL);
+}
- return res.pmd;
+static inline pud_t native_pudp_get_and_clear(pud_t *pudp)
+{
+ return pxx_xchg64(pud, pudp, 0ULL);
}
#else
+#define native_ptep_get_and_clear(xp) native_local_ptep_get_and_clear(xp)
#define native_pmdp_get_and_clear(xp) native_local_pmdp_get_and_clear(xp)
+#define native_pudp_get_and_clear(xp) native_local_pudp_get_and_clear(xp)
#endif
#ifndef pmdp_establish
* anybody.
*/
if (!(pmd_val(pmd) & _PAGE_PRESENT)) {
- union split_pmd old, new, *ptr;
-
- ptr = (union split_pmd *)pmdp;
-
- new.pmd = pmd;
-
/* xchg acts as a barrier before setting of the high bits */
- old.pmd_low = xchg(&ptr->pmd_low, new.pmd_low);
- old.pmd_high = ptr->pmd_high;
- ptr->pmd_high = new.pmd_high;
- return old.pmd;
- }
-
- do {
- old = *pmdp;
- } while (cmpxchg64(&pmdp->pmd, old.pmd, pmd.pmd) != old.pmd);
-
- return old;
-}
-#endif
-
-#ifdef CONFIG_SMP
-union split_pud {
- struct {
- u32 pud_low;
- u32 pud_high;
- };
- pud_t pud;
-};
-
-static inline pud_t native_pudp_get_and_clear(pud_t *pudp)
-{
- union split_pud res, *orig = (union split_pud *)pudp;
+ old.pmd_low = xchg(&pmdp->pmd_low, pmd.pmd_low);
+ old.pmd_high = READ_ONCE(pmdp->pmd_high);
+ WRITE_ONCE(pmdp->pmd_high, pmd.pmd_high);
-#ifdef CONFIG_PAGE_TABLE_ISOLATION
- pti_set_user_pgtbl(&pudp->p4d.pgd, __pgd(0));
-#endif
-
- /* xchg acts as a barrier before setting of the high bits */
- res.pud_low = xchg(&orig->pud_low, 0);
- res.pud_high = orig->pud_high;
- orig->pud_high = 0;
+ return old;
+ }
- return res.pud;
+ return pxx_xchg64(pmd, pmdp, pmd.pmd);
}
-#else
-#define native_pudp_get_and_clear(xp) native_local_pudp_get_and_clear(xp)
#endif
/* Encode and de-code a swap entry */
};
pteval_t pte;
} pte_t;
+
+typedef union {
+ struct {
+ unsigned long pmd_low, pmd_high;
+ };
+ pmdval_t pmd;
+} pmd_t;
#endif /* !__ASSEMBLY__ */
#define SHARED_KERNEL_PMD (!static_cpu_has(X86_FEATURE_PTI))
typedef unsigned long pgprotval_t;
typedef struct { pteval_t pte; } pte_t;
+typedef struct { pmdval_t pmd; } pmd_t;
#ifdef CONFIG_X86_5LEVEL
extern unsigned int __pgtable_l5_enabled;
#define CPU_ENTRY_AREA_RO_IDT_VADDR ((void *)CPU_ENTRY_AREA_RO_IDT)
-#define CPU_ENTRY_AREA_MAP_SIZE (CPU_ENTRY_AREA_PER_CPU + CPU_ENTRY_AREA_ARRAY_SIZE - CPU_ENTRY_AREA_BASE)
+#ifdef CONFIG_X86_32
+#define CPU_ENTRY_AREA_MAP_SIZE (CPU_ENTRY_AREA_PER_CPU + \
+ (CPU_ENTRY_AREA_SIZE * NR_CPUS) - \
+ CPU_ENTRY_AREA_BASE)
+#else
+#define CPU_ENTRY_AREA_MAP_SIZE P4D_SIZE
+#endif
#endif /* _ASM_X86_PGTABLE_AREAS_H */
#endif
#if CONFIG_PGTABLE_LEVELS > 2
-typedef struct { pmdval_t pmd; } pmd_t;
-
static inline pmd_t native_make_pmd(pmdval_t val)
{
- return (pmd_t) { val };
+ return (pmd_t) { .pmd = val };
}
static inline pmdval_t native_pmd_val(pmd_t pmd)
*/
#ifdef CONFIG_X86_64
/* Mask off the address space ID and SME encryption bits. */
-#define CR3_ADDR_MASK __sme_clr(0x7FFFFFFFFFFFF000ull)
+#define CR3_ADDR_MASK __sme_clr(PHYSICAL_PAGE_MASK)
#define CR3_PCID_MASK 0xFFFull
#define CR3_NOFLUSH BIT_ULL(63)
#include <asm/page.h>
#include <asm-generic/set_memory.h>
+#define set_memory_rox set_memory_rox
+int set_memory_rox(unsigned long addr, int numpages);
+
/*
* The set_memory_* API can be used to change various attributes of a virtual
* address range. The attributes include:
{
struct text_poke_loc *tp;
- if (unlikely(system_state == SYSTEM_BOOTING)) {
- text_poke_early(addr, opcode, len);
- return;
- }
-
text_poke_flush(addr);
tp = &tp_vec[tp_vec_nr++];
{
struct text_poke_loc tp;
- if (unlikely(system_state == SYSTEM_BOOTING)) {
- text_poke_early(addr, opcode, len);
- return;
- }
-
text_poke_loc_init(&tp, addr, opcode, len, emulate);
text_poke_bp_batch(&tp, 1);
}
}
a = aper + iommu_size;
- iommu_size -= round_up(a, PMD_PAGE_SIZE) - a;
+ iommu_size -= round_up(a, PMD_SIZE) - a;
if (iommu_size < 64*1024*1024) {
pr_warn("PCI-DMA: Warning: Small IOMMU %luMB."
.mmap = pseudo_lock_dev_mmap,
};
-static char *pseudo_lock_devnode(struct device *dev, umode_t *mode)
+static char *pseudo_lock_devnode(const struct device *dev, umode_t *mode)
{
- struct rdtgroup *rdtgrp;
+ const struct rdtgroup *rdtgrp;
rdtgrp = dev_get_drvdata(dev);
if (mode)
return 0;
}
-static char *cpuid_devnode(struct device *dev, umode_t *mode)
+static char *cpuid_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "cpu/%u/cpuid", MINOR(dev->devt));
}
#include <linux/module.h>
#include <linux/memory.h>
#include <linux/vmalloc.h>
+#include <linux/set_memory.h>
#include <trace/syscall.h>
-#include <asm/set_memory.h>
#include <asm/kprobes.h>
#include <asm/ftrace.h>
#include <asm/nops.h>
ret = ftrace_verify_code(rec->ip, old);
if (ret) {
+ ftrace_expected = old;
ftrace_bug(ret, rec);
+ ftrace_expected = NULL;
return;
}
}
/* ALLOC_TRAMP flags lets us know we created it */
ops->flags |= FTRACE_OPS_FL_ALLOC_TRAMP;
- if (likely(system_state != SYSTEM_BOOTING))
- set_memory_ro((unsigned long)trampoline, npages);
- set_memory_x((unsigned long)trampoline, npages);
+ set_memory_rox((unsigned long)trampoline, npages);
return (unsigned long)trampoline;
fail:
tramp_free(trampoline);
load_delta = physaddr - (unsigned long)(_text - __START_KERNEL_map);
/* Is the address not 2M aligned? */
- if (load_delta & ~PMD_PAGE_MASK)
+ if (load_delta & ~PMD_MASK)
for (;;);
/* Include the SME encryption mask in the fixup value */
/* CPU entry erea is always used for CPU entry */
if (within_area(addr, end, CPU_ENTRY_AREA_BASE,
- CPU_ENTRY_AREA_TOTAL_SIZE))
+ CPU_ENTRY_AREA_MAP_SIZE))
return true;
/*
#include <linux/objtool.h>
#include <linux/vmalloc.h>
#include <linux/pgtable.h>
+#include <linux/set_memory.h>
#include <asm/text-patching.h>
#include <asm/cacheflush.h>
#include <asm/alternative.h>
#include <asm/insn.h>
#include <asm/debugreg.h>
-#include <asm/set_memory.h>
#include <asm/ibt.h>
#include "common.h"
return NULL;
/*
- * First make the page read-only, and only then make it executable to
- * prevent it from being W+X in between.
- */
- set_memory_ro((unsigned long)page, 1);
-
- /*
* TODO: Once additional kernel code protection mechanisms are set, ensure
* that the page was not maliciously altered and it is still zeroed.
*/
- set_memory_x((unsigned long)page, 1);
+ set_memory_rox((unsigned long)page, 1);
return page;
}
return 0;
}
-static char *msr_devnode(struct device *dev, umode_t *mode)
+static char *msr_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "cpu/%u/msr", MINOR(dev->devt));
}
#include <asm/cpu_entry_area.h>
#include <asm/fixmap.h>
#include <asm/desc.h>
+#include <asm/kasan.h>
static DEFINE_PER_CPU_PAGE_ALIGNED(struct entry_stack_page, entry_stack_storage);
#ifdef CONFIG_X86_64
static DEFINE_PER_CPU_PAGE_ALIGNED(struct exception_stacks, exception_stacks);
DEFINE_PER_CPU(struct cea_exception_stacks*, cea_exception_stacks);
-#endif
-#ifdef CONFIG_X86_32
+static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, _cea_offset);
+
+static __always_inline unsigned int cea_offset(unsigned int cpu)
+{
+ return per_cpu(_cea_offset, cpu);
+}
+
+static __init void init_cea_offsets(void)
+{
+ unsigned int max_cea;
+ unsigned int i, j;
+
+ max_cea = (CPU_ENTRY_AREA_MAP_SIZE - PAGE_SIZE) / CPU_ENTRY_AREA_SIZE;
+
+ /* O(sodding terrible) */
+ for_each_possible_cpu(i) {
+ unsigned int cea;
+
+again:
+ cea = get_random_u32_below(max_cea);
+
+ for_each_possible_cpu(j) {
+ if (cea_offset(j) == cea)
+ goto again;
+
+ if (i == j)
+ break;
+ }
+
+ per_cpu(_cea_offset, i) = cea;
+ }
+}
+#else /* !X86_64 */
DECLARE_PER_CPU_PAGE_ALIGNED(struct doublefault_stack, doublefault_stack);
+
+static __always_inline unsigned int cea_offset(unsigned int cpu)
+{
+ return cpu;
+}
+static inline void init_cea_offsets(void) { }
#endif
/* Is called from entry code, so must be noinstr */
noinstr struct cpu_entry_area *get_cpu_entry_area(int cpu)
{
- unsigned long va = CPU_ENTRY_AREA_PER_CPU + cpu * CPU_ENTRY_AREA_SIZE;
+ unsigned long va = CPU_ENTRY_AREA_PER_CPU + cea_offset(cpu) * CPU_ENTRY_AREA_SIZE;
BUILD_BUG_ON(sizeof(struct cpu_entry_area) % PAGE_SIZE != 0);
return (struct cpu_entry_area *) va;
pgprot_t tss_prot = PAGE_KERNEL;
#endif
+ kasan_populate_shadow_for_vaddr(cea, CPU_ENTRY_AREA_SIZE,
+ early_cpu_to_node(cpu));
+
cea_set_pte(&cea->gdt, get_cpu_gdt_paddr(cpu), gdt_prot);
cea_map_percpu_pages(&cea->entry_stack_page,
/* The +1 is for the readonly IDT: */
BUILD_BUG_ON((CPU_ENTRY_AREA_PAGES+1)*PAGE_SIZE != CPU_ENTRY_AREA_MAP_SIZE);
- BUILD_BUG_ON(CPU_ENTRY_AREA_TOTAL_SIZE != CPU_ENTRY_AREA_MAP_SIZE);
BUG_ON(CPU_ENTRY_AREA_BASE & ~PMD_MASK);
start = CPU_ENTRY_AREA_BASE;
{
unsigned int cpu;
+ init_cea_offsets();
+
setup_cpu_entry_area_ptes();
for_each_possible_cpu(cpu)
spinlock_t *ptl;
pte_t *ptep;
- poking_mm = copy_init_mm();
+ poking_mm = mm_alloc();
BUG_ON(!poking_mm);
/*
kasan_map_early_shadow(init_top_pgt);
}
+static unsigned long kasan_mem_to_shadow_align_down(unsigned long va)
+{
+ unsigned long shadow = (unsigned long)kasan_mem_to_shadow((void *)va);
+
+ return round_down(shadow, PAGE_SIZE);
+}
+
+static unsigned long kasan_mem_to_shadow_align_up(unsigned long va)
+{
+ unsigned long shadow = (unsigned long)kasan_mem_to_shadow((void *)va);
+
+ return round_up(shadow, PAGE_SIZE);
+}
+
+void __init kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid)
+{
+ unsigned long shadow_start, shadow_end;
+
+ shadow_start = kasan_mem_to_shadow_align_down((unsigned long)va);
+ shadow_end = kasan_mem_to_shadow_align_up((unsigned long)va + size);
+ kasan_populate_shadow(shadow_start, shadow_end, nid);
+}
+
void __init kasan_init(void)
{
+ unsigned long shadow_cea_begin, shadow_cea_per_cpu_begin, shadow_cea_end;
int i;
- void *shadow_cpu_entry_begin, *shadow_cpu_entry_end;
memcpy(early_top_pgt, init_top_pgt, sizeof(early_top_pgt));
map_range(&pfn_mapped[i]);
}
- shadow_cpu_entry_begin = (void *)CPU_ENTRY_AREA_BASE;
- shadow_cpu_entry_begin = kasan_mem_to_shadow(shadow_cpu_entry_begin);
- shadow_cpu_entry_begin = (void *)round_down(
- (unsigned long)shadow_cpu_entry_begin, PAGE_SIZE);
-
- shadow_cpu_entry_end = (void *)(CPU_ENTRY_AREA_BASE +
- CPU_ENTRY_AREA_MAP_SIZE);
- shadow_cpu_entry_end = kasan_mem_to_shadow(shadow_cpu_entry_end);
- shadow_cpu_entry_end = (void *)round_up(
- (unsigned long)shadow_cpu_entry_end, PAGE_SIZE);
+ shadow_cea_begin = kasan_mem_to_shadow_align_down(CPU_ENTRY_AREA_BASE);
+ shadow_cea_per_cpu_begin = kasan_mem_to_shadow_align_up(CPU_ENTRY_AREA_PER_CPU);
+ shadow_cea_end = kasan_mem_to_shadow_align_up(CPU_ENTRY_AREA_BASE +
+ CPU_ENTRY_AREA_MAP_SIZE);
kasan_populate_early_shadow(
kasan_mem_to_shadow((void *)PAGE_OFFSET + MAXMEM),
kasan_populate_early_shadow(
kasan_mem_to_shadow((void *)VMALLOC_END + 1),
- shadow_cpu_entry_begin);
+ (void *)shadow_cea_begin);
- kasan_populate_shadow((unsigned long)shadow_cpu_entry_begin,
- (unsigned long)shadow_cpu_entry_end, 0);
+ /*
+ * Populate the shadow for the shared portion of the CPU entry area.
+ * Shadows for the per-CPU areas are mapped on-demand, as each CPU's
+ * area is randomly placed somewhere in the 512GiB range and mapping
+ * the entire 512GiB range is prohibitively expensive.
+ */
+ kasan_populate_shadow(shadow_cea_begin,
+ shadow_cea_per_cpu_begin, 0);
- kasan_populate_early_shadow(shadow_cpu_entry_end,
+ kasan_populate_early_shadow((void *)shadow_cea_end,
kasan_mem_to_shadow((void *)__START_KERNEL_map));
kasan_populate_shadow((unsigned long)kasan_mem_to_shadow(_stext),
int active;
};
-static DEFINE_SPINLOCK(kmmio_lock);
+/*
+ * The kmmio_lock is taken in int3 context, which is treated as NMI context.
+ * This causes lockdep to complain about it bein in both NMI and normal
+ * context. Hide it from lockdep, as it should not have any other locks
+ * taken under it, and this is only enabled for debugging mmio anyway.
+ */
+static arch_spinlock_t kmmio_lock = __ARCH_SPIN_LOCK_UNLOCKED;
/* Protected by kmmio_lock */
unsigned int kmmio_count;
page_base &= page_level_mask(l);
/*
- * Preemption is now disabled to prevent process switch during
- * single stepping. We can only handle one active kmmio trace
+ * Hold the RCU read lock over single stepping to avoid looking
+ * up the probe and kmmio_fault_page again. The rcu_read_lock_sched()
+ * also disables preemption and prevents process switch during
+ * the single stepping. We can only handle one active kmmio trace
* per cpu, so ensure that we finish it before something else
- * gets to run. We also hold the RCU read lock over single
- * stepping to avoid looking up the probe and kmmio_fault_page
- * again.
+ * gets to run.
*/
- preempt_disable();
- rcu_read_lock();
+ rcu_read_lock_sched_notrace();
faultpage = get_kmmio_fault_page(page_base);
if (!faultpage) {
return 1; /* fault handled */
no_kmmio:
- rcu_read_unlock();
- preempt_enable_no_resched();
+ rcu_read_unlock_sched_notrace();
return ret;
}
ctx->probe->post_handler(ctx->probe, condition, regs);
/* Prevent racing against release_kmmio_fault_page(). */
- spin_lock(&kmmio_lock);
+ arch_spin_lock(&kmmio_lock);
if (ctx->fpage->count)
arm_kmmio_fault_page(ctx->fpage);
- spin_unlock(&kmmio_lock);
+ arch_spin_unlock(&kmmio_lock);
regs->flags &= ~X86_EFLAGS_TF;
regs->flags |= ctx->saved_flags;
/* These were acquired in kmmio_handler(). */
ctx->active--;
BUG_ON(ctx->active);
- rcu_read_unlock();
- preempt_enable_no_resched();
+ rcu_read_unlock_sched_notrace();
/*
* if somebody else is singlestepping across a probe point, flags
unsigned int l;
pte_t *pte;
- spin_lock_irqsave(&kmmio_lock, flags);
+ local_irq_save(flags);
+ arch_spin_lock(&kmmio_lock);
if (get_kmmio_probe(addr)) {
ret = -EEXIST;
goto out;
size += page_level_size(l);
}
out:
- spin_unlock_irqrestore(&kmmio_lock, flags);
+ arch_spin_unlock(&kmmio_lock);
+ local_irq_restore(flags);
+
/*
* XXX: What should I do here?
* Here was a call to global_flush_tlb(), but it does not exist
struct kmmio_fault_page **prevp = &dr->release_list;
unsigned long flags;
- spin_lock_irqsave(&kmmio_lock, flags);
+ local_irq_save(flags);
+ arch_spin_lock(&kmmio_lock);
while (f) {
if (!f->count) {
list_del_rcu(&f->list);
}
f = *prevp;
}
- spin_unlock_irqrestore(&kmmio_lock, flags);
+ arch_spin_unlock(&kmmio_lock);
+ local_irq_restore(flags);
/* This is the real RCU destroy call. */
call_rcu(&dr->rcu, rcu_free_kmmio_fault_pages);
if (!pte)
return;
- spin_lock_irqsave(&kmmio_lock, flags);
+ local_irq_save(flags);
+ arch_spin_lock(&kmmio_lock);
while (size < size_lim) {
release_kmmio_fault_page(addr + size, &release_list);
size += page_level_size(l);
}
list_del_rcu(&p->list);
kmmio_count--;
- spin_unlock_irqrestore(&kmmio_lock, flags);
+ arch_spin_unlock(&kmmio_lock);
+ local_irq_restore(flags);
if (!release_list)
return;
* RCX - virtual address of the encryption workarea, including:
* - stack page (PAGE_SIZE)
* - encryption routine page (PAGE_SIZE)
- * - intermediate copy buffer (PMD_PAGE_SIZE)
+ * - intermediate copy buffer (PMD_SIZE)
* R8 - physical address of the pagetables to use for encryption
*/
wbinvd /* Invalidate any cache entries */
/* Copy/encrypt up to 2MB at a time */
- movq $PMD_PAGE_SIZE, %r12
+ movq $PMD_SIZE, %r12
1:
cmpq %r12, %r9
jnb 2f
* section is 2MB aligned to allow for simple pagetable setup using only
* PMD entries (see vmlinux.lds.S).
*/
-static char sme_workarea[2 * PMD_PAGE_SIZE] __section(".init.scratch");
+static char sme_workarea[2 * PMD_SIZE] __section(".init.scratch");
static char sme_cmdline_arg[] __initdata = "mem_encrypt";
static char sme_cmdline_on[] __initdata = "on";
while (ppd->vaddr < ppd->vaddr_end) {
sme_populate_pgd_large(ppd);
- ppd->vaddr += PMD_PAGE_SIZE;
- ppd->paddr += PMD_PAGE_SIZE;
+ ppd->vaddr += PMD_SIZE;
+ ppd->paddr += PMD_SIZE;
}
}
vaddr_end = ppd->vaddr_end;
/* If start is not 2MB aligned, create PTE entries */
- ppd->vaddr_end = ALIGN(ppd->vaddr, PMD_PAGE_SIZE);
+ ppd->vaddr_end = ALIGN(ppd->vaddr, PMD_SIZE);
__sme_map_range_pte(ppd);
/* Create PMD entries */
- ppd->vaddr_end = vaddr_end & PMD_PAGE_MASK;
+ ppd->vaddr_end = vaddr_end & PMD_MASK;
__sme_map_range_pmd(ppd);
/* If end is not 2MB aligned, create PTE entries */
/* Physical addresses gives us the identity mapped virtual addresses */
kernel_start = __pa_symbol(_text);
- kernel_end = ALIGN(__pa_symbol(_end), PMD_PAGE_SIZE);
+ kernel_end = ALIGN(__pa_symbol(_end), PMD_SIZE);
kernel_len = kernel_end - kernel_start;
initrd_start = 0;
* executable encryption area size:
* stack page (PAGE_SIZE)
* encryption routine page (PAGE_SIZE)
- * intermediate copy buffer (PMD_PAGE_SIZE)
+ * intermediate copy buffer (PMD_SIZE)
* pagetable structures for the encryption of the kernel
* pagetable structures for workarea (in case not currently mapped)
*/
execute_start = workarea_start;
- execute_end = execute_start + (PAGE_SIZE * 2) + PMD_PAGE_SIZE;
+ execute_end = execute_start + (PAGE_SIZE * 2) + PMD_SIZE;
execute_len = execute_end - execute_start;
/*
* before it is mapped.
*/
workarea_len = execute_len + pgtable_area_len;
- workarea_end = ALIGN(workarea_start + workarea_len, PMD_PAGE_SIZE);
+ workarea_end = ALIGN(workarea_start + workarea_len, PMD_SIZE);
/*
* Set the address to the start of where newly created pagetable
#ifdef CONFIG_X86_64
+/*
+ * The kernel image is mapped into two places in the virtual address space
+ * (addresses without KASLR, of course):
+ *
+ * 1. The kernel direct map (0xffff880000000000)
+ * 2. The "high kernel map" (0xffffffff81000000)
+ *
+ * We actually execute out of #2. If we get the address of a kernel symbol, it
+ * points to #2, but almost all physical-to-virtual translations point to #1.
+ *
+ * This is so that we can have both a directmap of all physical memory *and*
+ * take full advantage of the the limited (s32) immediate addressing range (2G)
+ * of x86_64.
+ *
+ * See Documentation/x86/x86_64/mm.rst for more detail.
+ */
+
static inline unsigned long highmap_start_pfn(void)
{
return __pa_symbol(_text) >> PAGE_SHIFT;
{
unsigned long end;
- /* Kernel text is rw at boot up */
- if (system_state == SYSTEM_BOOTING)
- return new;
-
/*
* 32-bit has some unfixable W+X issues, like EFI code
* and writeable data being in the same page. Disable
switch (level) {
case PG_LEVEL_1G:
phys_addr = (phys_addr_t)pud_pfn(*(pud_t *)pte) << PAGE_SHIFT;
- offset = virt_addr & ~PUD_PAGE_MASK;
+ offset = virt_addr & ~PUD_MASK;
break;
case PG_LEVEL_2M:
phys_addr = (phys_addr_t)pmd_pfn(*(pmd_t *)pte) << PAGE_SHIFT;
- offset = virt_addr & ~PMD_PAGE_MASK;
+ offset = virt_addr & ~PMD_MASK;
break;
default:
phys_addr = (phys_addr_t)pte_pfn(*pte) << PAGE_SHIFT;
case PG_LEVEL_1G:
ref_prot = pud_pgprot(*(pud_t *)kpte);
ref_pfn = pud_pfn(*(pud_t *)kpte);
- pfninc = PMD_PAGE_SIZE >> PAGE_SHIFT;
+ pfninc = PMD_SIZE >> PAGE_SHIFT;
lpaddr = address & PUD_MASK;
lpinc = PMD_SIZE;
/*
return err;
}
-static int __change_page_attr_set_clr(struct cpa_data *cpa, int checkalias);
+static int __change_page_attr_set_clr(struct cpa_data *cpa, int primary);
+/*
+ * Check the directmap and "high kernel map" 'aliases'.
+ */
static int cpa_process_alias(struct cpa_data *cpa)
{
struct cpa_data alias_cpa;
alias_cpa.flags &= ~(CPA_PAGES_ARRAY | CPA_ARRAY);
alias_cpa.curpage = 0;
+ /* Directmap always has NX set, do not modify. */
+ if (__supported_pte_mask & _PAGE_NX) {
+ alias_cpa.mask_clr.pgprot &= ~_PAGE_NX;
+ alias_cpa.mask_set.pgprot &= ~_PAGE_NX;
+ }
+
cpa->force_flush_all = 1;
ret = __change_page_attr_set_clr(&alias_cpa, 0);
alias_cpa.flags &= ~(CPA_PAGES_ARRAY | CPA_ARRAY);
alias_cpa.curpage = 0;
+ /*
+ * [_text, _brk_end) also covers data, do not modify NX except
+ * in cases where the highmap is the primary target.
+ */
+ if (__supported_pte_mask & _PAGE_NX) {
+ alias_cpa.mask_clr.pgprot &= ~_PAGE_NX;
+ alias_cpa.mask_set.pgprot &= ~_PAGE_NX;
+ }
+
cpa->force_flush_all = 1;
/*
* The high mapping range is imprecise, so ignore the
return 0;
}
-static int __change_page_attr_set_clr(struct cpa_data *cpa, int checkalias)
+static int __change_page_attr_set_clr(struct cpa_data *cpa, int primary)
{
unsigned long numpages = cpa->numpages;
unsigned long rempages = numpages;
int ret = 0;
+ /*
+ * No changes, easy!
+ */
+ if (!(pgprot_val(cpa->mask_set) | pgprot_val(cpa->mask_clr)) &&
+ !cpa->force_split)
+ return ret;
+
while (rempages) {
/*
* Store the remaining nr of pages for the large page
if (!debug_pagealloc_enabled())
spin_lock(&cpa_lock);
- ret = __change_page_attr(cpa, checkalias);
+ ret = __change_page_attr(cpa, primary);
if (!debug_pagealloc_enabled())
spin_unlock(&cpa_lock);
if (ret)
goto out;
- if (checkalias) {
+ if (primary && !(cpa->flags & CPA_NO_CHECK_ALIAS)) {
ret = cpa_process_alias(cpa);
if (ret)
goto out;
struct page **pages)
{
struct cpa_data cpa;
- int ret, cache, checkalias;
+ int ret, cache;
memset(&cpa, 0, sizeof(cpa));
cpa.numpages = numpages;
cpa.mask_set = mask_set;
cpa.mask_clr = mask_clr;
- cpa.flags = 0;
+ cpa.flags = in_flag;
cpa.curpage = 0;
cpa.force_split = force_split;
- if (in_flag & (CPA_ARRAY | CPA_PAGES_ARRAY))
- cpa.flags |= in_flag;
-
- /* No alias checking for _NX bit modifications */
- checkalias = (pgprot_val(mask_set) | pgprot_val(mask_clr)) != _PAGE_NX;
- /* Has caller explicitly disabled alias checking? */
- if (in_flag & CPA_NO_CHECK_ALIAS)
- checkalias = 0;
-
- ret = __change_page_attr_set_clr(&cpa, checkalias);
+ ret = __change_page_attr_set_clr(&cpa, 1);
/*
* Check whether we really changed something:
return change_page_attr_clear(&addr, numpages, __pgprot(_PAGE_RW), 0);
}
+int set_memory_rox(unsigned long addr, int numpages)
+{
+ pgprot_t clr = __pgprot(_PAGE_RW);
+
+ if (__supported_pte_mask & _PAGE_NX)
+ clr.pgprot |= _PAGE_NX;
+
+ return change_page_attr_clear(&addr, numpages, clr, 0);
+}
+
int set_memory_rw(unsigned long addr, int numpages)
{
return change_page_attr_set(&addr, numpages, __pgprot(_PAGE_RW), 0);
int set_memory_np_noalias(unsigned long addr, int numpages)
{
- int cpa_flags = CPA_NO_CHECK_ALIAS;
-
return change_page_attr_set_clr(&addr, numpages, __pgprot(0),
__pgprot(_PAGE_PRESENT), 0,
- cpa_flags, NULL);
+ CPA_NO_CHECK_ALIAS, NULL);
}
int set_memory_4k(unsigned long addr, int numpages)
.numpages = numpages,
.mask_set = __pgprot(_PAGE_PRESENT | _PAGE_RW),
.mask_clr = __pgprot(0),
- .flags = 0};
+ .flags = CPA_NO_CHECK_ALIAS };
/*
* No alias checking needed for setting present flag. otherwise,
* mappings (this adds to complexity if we want to do this from
* atomic context especially). Let's keep it simple!
*/
- return __change_page_attr_set_clr(&cpa, 0);
+ return __change_page_attr_set_clr(&cpa, 1);
}
static int __set_pages_np(struct page *page, int numpages)
.numpages = numpages,
.mask_set = __pgprot(0),
.mask_clr = __pgprot(_PAGE_PRESENT | _PAGE_RW),
- .flags = 0};
+ .flags = CPA_NO_CHECK_ALIAS };
/*
* No alias checking needed for setting not present flag. otherwise,
* mappings (this adds to complexity if we want to do this from
* atomic context especially). Let's keep it simple!
*/
- return __change_page_attr_set_clr(&cpa, 0);
+ return __change_page_attr_set_clr(&cpa, 1);
}
int set_direct_map_invalid_noflush(struct page *page)
.numpages = numpages,
.mask_set = __pgprot(0),
.mask_clr = __pgprot(~page_flags & (_PAGE_NX|_PAGE_RW)),
- .flags = 0,
+ .flags = CPA_NO_CHECK_ALIAS,
};
WARN_ONCE(num_online_cpus() > 1, "Don't call after initializing SMP");
cpa.mask_set = __pgprot(_PAGE_PRESENT | page_flags);
- retval = __change_page_attr_set_clr(&cpa, 0);
+ retval = __change_page_attr_set_clr(&cpa, 1);
__flush_tlb_all();
out:
.numpages = numpages,
.mask_set = __pgprot(0),
.mask_clr = __pgprot(_PAGE_PRESENT | _PAGE_RW),
- .flags = 0,
+ .flags = CPA_NO_CHECK_ALIAS,
};
WARN_ONCE(num_online_cpus() > 1, "Don't call after initializing SMP");
- retval = __change_page_attr_set_clr(&cpa, 0);
+ retval = __change_page_attr_set_clr(&cpa, 1);
__flush_tlb_all();
return retval;
* of the image.
*/
unsigned long start = PFN_ALIGN(_text);
- unsigned long end = ALIGN((unsigned long)_end, PMD_PAGE_SIZE);
+ unsigned long end = ALIGN((unsigned long)_end, PMD_SIZE);
/*
* This clears _PAGE_GLOBAL from the entire kernel image.
}
EXPORT_SYMBOL_GPL(bsg_register_queue);
-static char *bsg_devnode(struct device *dev, umode_t *mode)
+static char *bsg_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "bsg/%s", dev_name(dev));
}
iput(disk->part0->bd_inode); /* frees the disk */
}
-static int block_uevent(struct device *dev, struct kobj_uevent_env *env)
+static int block_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
- struct gendisk *disk = dev_to_disk(dev);
+ const struct gendisk *disk = dev_to_disk(dev);
return add_uevent_var(env, "DISKSEQ=%llu", disk->diskseq);
}
.name = "accel_minor"
};
-static char *accel_devnode(struct device *dev, umode_t *mode)
+static char *accel_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "accel/%s", dev_name(dev));
}
makemapdata-objs := makemapdata.o
quiet_cmd_mkmap = MKMAP $@
- cmd_mkmap = TOPDIR=$(srctree) $(obj)/makemapdata > $@
+ cmd_mkmap = TOPDIR=$(srctree) \
+ SPKDIR=$(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD),$(srctree)/drivers/accessibility/speakup) \
+ $(obj)/makemapdata > $@
$(obj)/mapdata.h: $(obj)/makemapdata
$(call cmd,mkmap)
__ATTR(say_word_ctl, 0644, spk_var_show, spk_var_store);
static struct kobj_attribute spell_delay_attribute =
__ATTR(spell_delay, 0644, spk_var_show, spk_var_store);
+static struct kobj_attribute cur_phonetic_attribute =
+ __ATTR(cur_phonetic, 0644, spk_var_show, spk_var_store);
/*
* These attributes are i18n related.
&say_control_attribute.attr,
&say_word_ctl_attribute.attr,
&spell_delay_attribute.attr,
+ &cur_phonetic_attribute.attr,
NULL,
};
int spk_say_ctrl, spk_bell_pos;
short spk_punc_mask;
int spk_punc_level, spk_reading_punc;
+int spk_cur_phonetic;
char spk_str_caps_start[MAXVARLEN + 1] = "\0";
char spk_str_caps_stop[MAXVARLEN + 1] = "\0";
char spk_str_pause[MAXVARLEN + 1] = "\0";
return 0;
}
-static struct var_t spk_vars[] = {
+enum spk_vars_id {
+ BELL_POS_ID = 0, SPELL_DELAY_ID, ATTRIB_BLEEP_ID,
+ BLEEPS_ID, BLEEP_TIME_ID, PUNC_LEVEL_ID,
+ READING_PUNC_ID, CURSOR_TIME_ID, SAY_CONTROL_ID,
+ SAY_WORD_CTL_ID, NO_INTERRUPT_ID, KEY_ECHO_ID,
+ CUR_PHONETIC_ID, V_LAST_VAR_ID, NB_ID
+};
+
+static struct var_t spk_vars[NB_ID] = {
/* bell must be first to set high limit */
- {BELL_POS, .u.n = {NULL, 0, 0, 0, 0, 0, NULL} },
- {SPELL_DELAY, .u.n = {NULL, 0, 0, 4, 0, 0, NULL} },
- {ATTRIB_BLEEP, .u.n = {NULL, 1, 0, 3, 0, 0, NULL} },
- {BLEEPS, .u.n = {NULL, 3, 0, 3, 0, 0, NULL} },
- {BLEEP_TIME, .u.n = {NULL, 30, 1, 200, 0, 0, NULL} },
- {PUNC_LEVEL, .u.n = {NULL, 1, 0, 4, 0, 0, NULL} },
- {READING_PUNC, .u.n = {NULL, 1, 0, 4, 0, 0, NULL} },
- {CURSOR_TIME, .u.n = {NULL, 120, 50, 600, 0, 0, NULL} },
- {SAY_CONTROL, TOGGLE_0},
- {SAY_WORD_CTL, TOGGLE_0},
- {NO_INTERRUPT, TOGGLE_0},
- {KEY_ECHO, .u.n = {NULL, 1, 0, 2, 0, 0, NULL} },
+ [BELL_POS_ID] = { BELL_POS, .u.n = {NULL, 0, 0, 0, 0, 0, NULL} },
+ [SPELL_DELAY_ID] = { SPELL_DELAY, .u.n = {NULL, 0, 0, 4, 0, 0, NULL} },
+ [ATTRIB_BLEEP_ID] = { ATTRIB_BLEEP, .u.n = {NULL, 1, 0, 3, 0, 0, NULL} },
+ [BLEEPS_ID] = { BLEEPS, .u.n = {NULL, 3, 0, 3, 0, 0, NULL} },
+ [BLEEP_TIME_ID] = { BLEEP_TIME, .u.n = {NULL, 30, 1, 200, 0, 0, NULL} },
+ [PUNC_LEVEL_ID] = { PUNC_LEVEL, .u.n = {NULL, 1, 0, 4, 0, 0, NULL} },
+ [READING_PUNC_ID] = { READING_PUNC, .u.n = {NULL, 1, 0, 4, 0, 0, NULL} },
+ [CURSOR_TIME_ID] = { CURSOR_TIME, .u.n = {NULL, 120, 50, 600, 0, 0, NULL} },
+ [SAY_CONTROL_ID] { SAY_CONTROL, TOGGLE_0},
+ [SAY_WORD_CTL_ID] = {SAY_WORD_CTL, TOGGLE_0},
+ [NO_INTERRUPT_ID] = { NO_INTERRUPT, TOGGLE_0},
+ [KEY_ECHO_ID] = { KEY_ECHO, .u.n = {NULL, 1, 0, 2, 0, 0, NULL} },
+ [CUR_PHONETIC_ID] = { CUR_PHONETIC, .u.n = {NULL, 0, 0, 1, 0, 0, NULL} },
V_LAST_VAR
};
speakup_win_say(vc);
else if (is_cursor == 1 || is_cursor == 4)
say_line_from_to(vc, 0, vc->vc_cols, 0);
- else
- say_char(vc);
+ else {
+ if (spk_cur_phonetic == 1)
+ say_phonetic_char(vc);
+ else
+ say_char(vc);
+ }
spk_keydown = 0;
is_cursor = 0;
out:
return err;
}
+module_param_named(bell_pos, spk_vars[BELL_POS_ID].u.n.default_val, int, 0444);
+module_param_named(spell_delay, spk_vars[SPELL_DELAY_ID].u.n.default_val, int, 0444);
+module_param_named(attrib_bleep, spk_vars[ATTRIB_BLEEP_ID].u.n.default_val, int, 0444);
+module_param_named(bleeps, spk_vars[BLEEPS_ID].u.n.default_val, int, 0444);
+module_param_named(bleep_time, spk_vars[BLEEP_TIME_ID].u.n.default_val, int, 0444);
+module_param_named(punc_level, spk_vars[PUNC_LEVEL_ID].u.n.default_val, int, 0444);
+module_param_named(reading_punc, spk_vars[READING_PUNC_ID].u.n.default_val, int, 0444);
+module_param_named(cursor_time, spk_vars[CURSOR_TIME_ID].u.n.default_val, int, 0444);
+module_param_named(say_control, spk_vars[SAY_CONTROL_ID].u.n.default_val, int, 0444);
+module_param_named(say_word_ctl, spk_vars[SAY_WORD_CTL_ID].u.n.default_val, int, 0444);
+module_param_named(no_interrupt, spk_vars[NO_INTERRUPT_ID].u.n.default_val, int, 0444);
+module_param_named(key_echo, spk_vars[KEY_ECHO_ID].u.n.default_val, int, 0444);
+module_param_named(cur_phonetic, spk_vars[CUR_PHONETIC_ID].u.n.default_val, int, 0444);
+
+MODULE_PARM_DESC(bell_pos, "This works much like a typewriter bell. If for example 72 is echoed to bell_pos, it will beep the PC speaker when typing on a line past character 72.");
+MODULE_PARM_DESC(spell_delay, "This controls how fast a word is spelled when speakup's spell word review command is pressed.");
+MODULE_PARM_DESC(attrib_bleep, "Beeps the PC speaker when there is an attribute change such as background color when using speakup review commands. One = on, zero = off.");
+MODULE_PARM_DESC(bleeps, "This controls whether one hears beeps through the PC speaker when using speakup review commands.");
+MODULE_PARM_DESC(bleep_time, "This controls the duration of the PC speaker beeps speakup produces.");
+MODULE_PARM_DESC(punc_level, "Controls the level of punctuation spoken as the screen is displayed, not reviewed.");
+MODULE_PARM_DESC(reading_punc, "It controls the level of punctuation when reviewing the screen with speakup's screen review commands.");
+MODULE_PARM_DESC(cursor_time, "This controls cursor delay when using arrow keys.");
+MODULE_PARM_DESC(say_control, "This controls if speakup speaks shift, alt and control when those keys are pressed or not.");
+MODULE_PARM_DESC(say_word_ctl, "Sets thw say_word_ctl on load.");
+MODULE_PARM_DESC(no_interrupt, "Controls if typing interrupts output from speakup.");
+MODULE_PARM_DESC(key_echo, "Controls if speakup speaks keys when they are typed. One = on zero = off or don't echo keys.");
+MODULE_PARM_DESC(cur_phonetic, "Controls if speakup speaks letters phonetically during navigation. One = on zero = off or don't speak phonetically.");
+
module_init(speakup_init);
module_exit(speakup_exit);
{
int value, i;
struct st_key *this;
- const char *dir_name;
+ const char *dir_name, *spk_dir_name;
char *cp;
dir_name = getenv("TOPDIR");
if (!dir_name)
dir_name = ".";
+ spk_dir_name = getenv("SPKDIR");
+ if (!spk_dir_name)
+ spk_dir_name = "drivers/accessibility/speakup";
bzero(key_table, sizeof(key_table));
add_key("shift", 1, is_shift);
add_key("altgr", 2, is_shift);
add_key(def_name, value, is_input);
}
- open_input(dir_name, "drivers/accessibility/speakup/spk_priv_keyinfo.h");
+ open_input(spk_dir_name, "spk_priv_keyinfo.h");
while (get_define()) {
if (strlen(def_val) > 5) {
//if (def_val[0] == '(')
extern int spk_reading_punc, spk_attrib_bleep, spk_bleeps;
extern int spk_bleep_time, spk_bell_pos;
extern int spk_spell_delay, spk_key_echo;
+extern int spk_cur_phonetic;
extern short spk_punc_mask;
extern short spk_pitch_shift, synth_flags;
extern bool spk_quiet_boot;
static int port_forced;
static unsigned int synth_portlist[] = { 0x2a8, 0 };
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"\033P8" } },
- { CAPS_STOP, .u.s = {"\033P5" } },
- { RATE, .u.n = {"\033R%c", 9, 0, 17, 0, 0, "0123456789abcdefgh" } },
- { PITCH, .u.n = {"\033P%d", 5, 0, 9, 0, 0, NULL } },
- { VOL, .u.n = {"\033A%d", 5, 0, 9, 0, 0, NULL } },
- { TONE, .u.n = {"\033V%d", 5, 0, 9, 0, 0, NULL } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID,
+ VOL_ID, TONE_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"\033P8" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"\033P5" } },
+ [RATE_ID] = { RATE, .u.n = {"\033R%c", 9, 0, 17, 0, 0, "0123456789abcdefgh" } },
+ [PITCH_ID] = { PITCH, .u.n = {"\033P%d", 5, 0, 9, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"\033A%d", 5, 0, 9, 0, 0, NULL } },
+ [TONE_ID] = { TONE, .u.n = {"\033V%d", 5, 0, 9, 0, 0, NULL } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_hw_named(port, port_forced, int, ioport, 0444);
module_param_named(start, synth_acntpc.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(tone, vars[TONE_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
+
MODULE_PARM_DESC(port, "Set the port for the synthesizer (override probing).");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(tone, "Set the tone variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
module_spk_synth(synth_acntpc);
static int synth_probe(struct spk_synth *synth);
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"\033P8" } },
- { CAPS_STOP, .u.s = {"\033P5" } },
- { RATE, .u.n = {"\033R%c", 9, 0, 17, 0, 0, "0123456789abcdefgh" } },
- { PITCH, .u.n = {"\033P%d", 5, 0, 9, 0, 0, NULL } },
- { VOL, .u.n = {"\033A%d", 9, 0, 9, 0, 0, NULL } },
- { TONE, .u.n = {"\033V%d", 5, 0, 9, 0, 0, NULL } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID,
+ VOL_ID, TONE_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"\033P8" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"\033P5" } },
+ [RATE_ID] = { RATE, .u.n = {"\033R%c", 9, 0, 17, 0, 0, "0123456789abcdefgh" } },
+ [PITCH_ID] = { PITCH, .u.n = {"\033P%d", 5, 0, 9, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"\033A%d", 9, 0, 9, 0, 0, NULL } },
+ [TONE_ID] = { TONE, .u.n = {"\033V%d", 5, 0, 9, 0, 0, NULL } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_named(ser, synth_acntsa.ser, int, 0444);
module_param_named(dev, synth_acntsa.dev_name, charp, 0444);
module_param_named(start, synth_acntsa.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(tone, vars[TONE_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
MODULE_PARM_DESC(ser, "Set the serial port for the synthesizer (0-based).");
MODULE_PARM_DESC(dev, "Set the device e.g. ttyUSB0, for the synthesizer.");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(tone, "Set the tone variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
module_spk_synth(synth_acntsa);
static void do_catch_up(struct spk_synth *synth);
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"cap, " } },
- { CAPS_STOP, .u.s = {"" } },
- { RATE, .u.n = {"@W%d", 6, 1, 9, 0, 0, NULL } },
- { PITCH, .u.n = {"@F%x", 10, 0, 15, 0, 0, NULL } },
- { VOL, .u.n = {"@A%x", 10, 0, 15, 0, 0, NULL } },
- { VOICE, .u.n = {"@V%d", 1, 1, 6, 0, 0, NULL } },
- { LANG, .u.n = {"@=%d,", 1, 1, 4, 0, 0, NULL } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID,
+ VOL_ID, VOICE_ID, LANG_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+
+
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"cap, " } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"" } },
+ [RATE_ID] = { RATE, .u.n = {"@W%d", 6, 1, 9, 0, 0, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"@F%x", 10, 0, 15, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"@A%x", 10, 0, 15, 0, 0, NULL } },
+ [VOICE_ID] = { VOICE, .u.n = {"@V%d", 1, 1, 6, 0, 0, NULL } },
+ [LANG_ID] = { LANG, .u.n = {"@=%d,", 1, 1, 4, 0, 0, NULL } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_named(ser, synth_apollo.ser, int, 0444);
module_param_named(dev, synth_apollo.dev_name, charp, 0444);
module_param_named(start, synth_apollo.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(voice, vars[VOICE_ID].u.n.default_val, int, 0444);
+module_param_named(lang, vars[LANG_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
MODULE_PARM_DESC(ser, "Set the serial port for the synthesizer (0-based).");
MODULE_PARM_DESC(dev, "Set the device e.g. ttyUSB0, for the synthesizer.");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(voice, "Set the voice variable on load.");
+MODULE_PARM_DESC(lang, "Set the lang variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
+
module_spk_synth(synth_apollo);
static int synth_probe(struct spk_synth *synth);
static void synth_flush(struct spk_synth *synth);
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"\x05[f99]" } },
- { CAPS_STOP, .u.s = {"\x05[f80]" } },
- { RATE, .u.n = {"\x05[r%d]", 10, 0, 20, 100, -10, NULL } },
- { PITCH, .u.n = {"\x05[f%d]", 80, 39, 4500, 0, 0, NULL } },
- { VOL, .u.n = {"\x05[g%d]", 21, 0, 40, 0, 0, NULL } },
- { TONE, .u.n = {"\x05[s%d]", 9, 0, 63, 0, 0, NULL } },
- { PUNCT, .u.n = {"\x05[A%c]", 0, 0, 3, 0, 0, "nmsa" } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID,
+ VOL_ID, TONE_ID, PUNCT_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"\x05[f99]" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"\x05[f80]" } },
+ [RATE_ID] = { RATE, .u.n = {"\x05[r%d]", 10, 0, 20, 100, -10, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"\x05[f%d]", 80, 39, 4500, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"\x05[g%d]", 21, 0, 40, 0, 0, NULL } },
+ [TONE_ID] = { TONE, .u.n = {"\x05[s%d]", 9, 0, 63, 0, 0, NULL } },
+ [PUNCT_ID] = { PUNCT, .u.n = {"\x05[A%c]", 0, 0, 3, 0, 0, "nmsa" } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_named(ser, synth_audptr.ser, int, 0444);
module_param_named(dev, synth_audptr.dev_name, charp, 0444);
module_param_named(start, synth_audptr.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(tone, vars[TONE_ID].u.n.default_val, int, 0444);
+module_param_named(punct, vars[PUNCT_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
+
MODULE_PARM_DESC(ser, "Set the serial port for the synthesizer (0-based).");
MODULE_PARM_DESC(dev, "Set the device e.g. ttyUSB0, for the synthesizer.");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(tone, "Set the tone variable on load.");
+MODULE_PARM_DESC(punct, "Set the punct variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
module_spk_synth(synth_audptr);
#define SYNTH_CLEAR 0x18
#define PROCSPEECH '\r'
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"\x05\x31\x32P" } },
- { CAPS_STOP, .u.s = {"\x05\x38P" } },
- { RATE, .u.n = {"\x05%dE", 8, 1, 16, 0, 0, NULL } },
- { PITCH, .u.n = {"\x05%dP", 8, 0, 16, 0, 0, NULL } },
- { VOL, .u.n = {"\x05%dV", 8, 0, 16, 0, 0, NULL } },
- { TONE, .u.n = {"\x05%dT", 8, 0, 16, 0, 0, NULL } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID,
+ VOL_ID, TONE_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"\x05\x31\x32P" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"\x05\x38P" } },
+ [RATE_ID] = { RATE, .u.n = {"\x05%dE", 8, 1, 16, 0, 0, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"\x05%dP", 8, 0, 16, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"\x05%dV", 8, 0, 16, 0, 0, NULL } },
+ [TONE_ID] = { TONE, .u.n = {"\x05%dT", 8, 0, 16, 0, 0, NULL } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_named(ser, synth_bns.ser, int, 0444);
module_param_named(dev, synth_bns.dev_name, charp, 0444);
module_param_named(start, synth_bns.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(tone, vars[TONE_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
MODULE_PARM_DESC(ser, "Set the serial port for the synthesizer (0-based).");
MODULE_PARM_DESC(dev, "Set the device e.g. ttyUSB0, for the synthesizer.");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(tone, "Set the tone variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
module_spk_synth(synth_bns);
static int in_escape;
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"[:dv ap 222]" } },
- { CAPS_STOP, .u.s = {"[:dv ap 100]" } },
- { RATE, .u.n = {"[:ra %d]", 7, 0, 9, 150, 25, NULL } },
- { PITCH, .u.n = {"[:dv ap %d]", 100, 0, 100, 0, 0, NULL } },
- { INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } },
- { VOL, .u.n = {"[:dv gv %d]", 13, 0, 16, 0, 5, NULL } },
- { PUNCT, .u.n = {"[:pu %c]", 0, 0, 2, 0, 0, "nsa" } },
- { VOICE, .u.n = {"[:n%c]", 0, 0, 9, 0, 0, "phfdburwkv" } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID, INFLECTION_ID,
+ VOL_ID, PUNCT_ID, VOICE_ID,
+ DIRECT_ID, V_LAST_ID,
+ NB_ID,
+};
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"[:dv ap 222]" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"[:dv ap 100]" } },
+ [RATE_ID] = { RATE, .u.n = {"[:ra %d]", 7, 0, 9, 150, 25, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"[:dv ap %d]", 100, 0, 100, 0, 0, NULL } },
+ [INFLECTION_ID] = { INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"[:dv gv %d]", 13, 0, 16, 0, 5, NULL } },
+ [PUNCT_ID] = { PUNCT, .u.n = {"[:pu %c]", 0, 0, 2, 0, 0, "nsa" } },
+ [VOICE_ID] = { VOICE, .u.n = {"[:n%c]", 0, 0, 9, 0, 0, "phfdburwkv" } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_named(ser, synth_decext.ser, int, 0444);
module_param_named(dev, synth_decext.dev_name, charp, 0444);
module_param_named(start, synth_decext.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(inflection, vars[INFLECTION_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(punct, vars[PUNCT_ID].u.n.default_val, int, 0444);
+module_param_named(voice, vars[VOICE_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
MODULE_PARM_DESC(ser, "Set the serial port for the synthesizer (0-based).");
MODULE_PARM_DESC(dev, "Set the device e.g. ttyUSB0, for the synthesizer.");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(inflection, "Set the inflection variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(punct, "Set the punct variable on load.");
+MODULE_PARM_DESC(voice, "Set the voice variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
module_spk_synth(synth_decext);
static int in_escape, is_flushing;
static int dt_stat, dma_state;
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"[:dv ap 200]" } },
- { CAPS_STOP, .u.s = {"[:dv ap 100]" } },
- { RATE, .u.n = {"[:ra %d]", 9, 0, 18, 150, 25, NULL } },
- { PITCH, .u.n = {"[:dv ap %d]", 80, 0, 100, 20, 0, NULL } },
- { INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } },
- { VOL, .u.n = {"[:vo se %d]", 5, 0, 9, 5, 10, NULL } },
- { PUNCT, .u.n = {"[:pu %c]", 0, 0, 2, 0, 0, "nsa" } },
- { VOICE, .u.n = {"[:n%c]", 0, 0, 9, 0, 0, "phfdburwkv" } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID, INFLECTION_ID,
+ VOL_ID, PUNCT_ID, VOICE_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID,
+};
+
+
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"[:dv ap 200]" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"[:dv ap 100]" } },
+ [RATE_ID] = { RATE, .u.n = {"[:ra %d]", 9, 0, 18, 150, 25, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"[:dv ap %d]", 80, 0, 100, 20, 0, NULL } },
+ [INFLECTION_ID] = { INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"[:vo se %d]", 5, 0, 9, 5, 10, NULL } },
+ [PUNCT_ID] = { PUNCT, .u.n = {"[:pu %c]", 0, 0, 2, 0, 0, "nsa" } },
+ [VOICE_ID] = { VOICE, .u.n = {"[:n%c]", 0, 0, 9, 0, 0, "phfdburwkv" } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
}
module_param_named(start, synth_dec_pc.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(inflection, vars[INFLECTION_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(punct, vars[PUNCT_ID].u.n.default_val, int, 0444);
+module_param_named(voice, vars[VOICE_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
+
+
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(inflection, "Set the inflection variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(punct, "Set the punct variable on load.");
+MODULE_PARM_DESC(voice, "Set the voice variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
module_spk_synth(synth_dec_pc);
static DEFINE_SPINLOCK(flush_lock);
static DECLARE_WAIT_QUEUE_HEAD(flush);
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"[:dv ap 160] " } },
- { CAPS_STOP, .u.s = {"[:dv ap 100 ] " } },
- { RATE, .u.n = {"[:ra %d] ", 180, 75, 650, 0, 0, NULL } },
- { PITCH, .u.n = {"[:dv ap %d] ", 122, 50, 350, 0, 0, NULL } },
- { INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } },
- { VOL, .u.n = {"[:dv g5 %d] ", 86, 60, 86, 0, 0, NULL } },
- { PUNCT, .u.n = {"[:pu %c] ", 0, 0, 2, 0, 0, "nsa" } },
- { VOICE, .u.n = {"[:n%c] ", 0, 0, 9, 0, 0, "phfdburwkv" } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID, INFLECTION_ID,
+ VOL_ID, PUNCT_ID, VOICE_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID,
+};
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"[:dv ap 160] " } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"[:dv ap 100 ] " } },
+ [RATE_ID] = { RATE, .u.n = {"[:ra %d] ", 180, 75, 650, 0, 0, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"[:dv ap %d] ", 122, 50, 350, 0, 0, NULL } },
+ [INFLECTION_ID] = { INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"[:dv g5 %d] ", 86, 60, 86, 0, 0, NULL } },
+ [PUNCT_ID] = { PUNCT, .u.n = {"[:pu %c] ", 0, 0, 2, 0, 0, "nsa" } },
+ [VOICE_ID] = { VOICE, .u.n = {"[:n%c] ", 0, 0, 9, 0, 0, "phfdburwkv" } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_named(ser, synth_dectlk.ser, int, 0444);
module_param_named(dev, synth_dectlk.dev_name, charp, 0444);
module_param_named(start, synth_dectlk.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(inflection, vars[INFLECTION_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(punct, vars[PUNCT_ID].u.n.default_val, int, 0444);
+module_param_named(voice, vars[VOICE_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
+
MODULE_PARM_DESC(ser, "Set the serial port for the synthesizer (0-based).");
MODULE_PARM_DESC(dev, "Set the device e.g. ttyUSB0, for the synthesizer.");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(inflection, "Set the inflection variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(punct, "Set the punct variable on load.");
+MODULE_PARM_DESC(voice, "Set the voice variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
module_spk_synth(synth_dectlk);
static u_char synth_status;
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"\x01+35p" } },
- { CAPS_STOP, .u.s = {"\x01-35p" } },
- { RATE, .u.n = {"\x01%ds", 8, 0, 9, 0, 0, NULL } },
- { PITCH, .u.n = {"\x01%dp", 50, 0, 99, 0, 0, NULL } },
- { VOL, .u.n = {"\x01%dv", 5, 0, 9, 0, 0, NULL } },
- { TONE, .u.n = {"\x01%dx", 1, 0, 2, 0, 0, NULL } },
- { PUNCT, .u.n = {"\x01%db", 7, 0, 15, 0, 0, NULL } },
- { VOICE, .u.n = {"\x01%do", 0, 0, 7, 0, 0, NULL } },
- { FREQUENCY, .u.n = {"\x01%df", 5, 0, 9, 0, 0, NULL } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID,
+ VOL_ID, TONE_ID, PUNCT_ID,
+ VOICE_ID, FREQUENCY_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID,
+};
+
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"\x01+35p" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"\x01-35p" } },
+ [RATE_ID] = { RATE, .u.n = {"\x01%ds", 8, 0, 9, 0, 0, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"\x01%dp", 50, 0, 99, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"\x01%dv", 5, 0, 9, 0, 0, NULL } },
+ [TONE_ID] = { TONE, .u.n = {"\x01%dx", 1, 0, 2, 0, 0, NULL } },
+ [PUNCT_ID] = { PUNCT, .u.n = {"\x01%db", 7, 0, 15, 0, 0, NULL } },
+ [VOICE_ID] = { VOICE, .u.n = {"\x01%do", 0, 0, 7, 0, 0, NULL } },
+ [FREQUENCY_ID] = { FREQUENCY, .u.n = {"\x01%df", 5, 0, 9, 0, 0, NULL } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_hw_named(port, port_forced, int, ioport, 0444);
module_param_named(start, synth_dtlk.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(tone, vars[TONE_ID].u.n.default_val, int, 0444);
+module_param_named(punct, vars[PUNCT_ID].u.n.default_val, int, 0444);
+module_param_named(voice, vars[VOICE_ID].u.n.default_val, int, 0444);
+module_param_named(frequency, vars[FREQUENCY_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
MODULE_PARM_DESC(port, "Set the port for the synthesizer (override probing).");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(tone, "Set the tone variable on load.");
+MODULE_PARM_DESC(punct, "Set the punct variable on load.");
+MODULE_PARM_DESC(voice, "Set the voice variable on load.");
+MODULE_PARM_DESC(frequency, "Set the frequency variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
module_spk_synth(synth_dtlk);
#define DRV_VERSION "2.11"
#define SYNTH_CLEAR '!'
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"CAPS_START\n" } },
- { CAPS_STOP, .u.s = {"CAPS_STOP\n" } },
- { PAUSE, .u.s = {"PAUSE\n"} },
- { RATE, .u.n = {"RATE %d\n", 8, 1, 16, 0, 0, NULL } },
- { PITCH, .u.n = {"PITCH %d\n", 8, 0, 16, 0, 0, NULL } },
- { INFLECTION, .u.n = {"INFLECTION %d\n", 8, 0, 16, 0, 0, NULL } },
- { VOL, .u.n = {"VOL %d\n", 8, 0, 16, 0, 0, NULL } },
- { TONE, .u.n = {"TONE %d\n", 8, 0, 16, 0, 0, NULL } },
- { PUNCT, .u.n = {"PUNCT %d\n", 0, 0, 3, 0, 0, NULL } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ PAUSE_ID,
+ RATE_ID, PITCH_ID, INFLECTION_ID,
+ VOL_ID, TONE_ID, PUNCT_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+
+
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"CAPS_START\n" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"CAPS_STOP\n" } },
+ [PAUSE_ID] = { PAUSE, .u.s = {"PAUSE\n"} },
+ [RATE_ID] = { RATE, .u.n = {"RATE %d\n", 8, 1, 16, 0, 0, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"PITCH %d\n", 8, 0, 16, 0, 0, NULL } },
+ [INFLECTION_ID] = { INFLECTION, .u.n = {"INFLECTION %d\n", 8, 0, 16, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"VOL %d\n", 8, 0, 16, 0, 0, NULL } },
+ [TONE_ID] = { TONE, .u.n = {"TONE %d\n", 8, 0, 16, 0, 0, NULL } },
+ [PUNCT_ID] = { PUNCT, .u.n = {"PUNCT %d\n", 0, 0, 3, 0, 0, NULL } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_named(ser, synth_dummy.ser, int, 0444);
module_param_named(dev, synth_dummy.dev_name, charp, 0444);
module_param_named(start, synth_dummy.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(inflection, vars[INFLECTION_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(tone, vars[TONE_ID].u.n.default_val, int, 0444);
+module_param_named(punct, vars[PUNCT_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
+
+
MODULE_PARM_DESC(ser, "Set the serial port for the synthesizer (0-based).");
MODULE_PARM_DESC(dev, "Set the device e.g. ttyUSB0, for the synthesizer.");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(inflection, "Set the inflection variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(tone, "Set the tone variable on load.");
+MODULE_PARM_DESC(punct, "Set the punct variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
module_spk_synth(synth_dummy);
static int port_forced;
static unsigned int synth_portlist[] = { 0x2a8, 0 };
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"[f130]" } },
- { CAPS_STOP, .u.s = {"[f90]" } },
- { RATE, .u.n = {"\04%c ", 8, 0, 10, 81, -8, NULL } },
- { PITCH, .u.n = {"[f%d]", 5, 0, 9, 40, 10, NULL } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"[f130]" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"[f90]" } },
+ [RATE_ID] = { RATE, .u.n = {"\04%c ", 8, 0, 10, 81, -8, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"[f%d]", 5, 0, 9, 40, 10, NULL } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_hw_named(port, port_forced, int, ioport, 0444);
module_param_named(start, synth_keypc.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
MODULE_PARM_DESC(port, "Set the port for the synthesizer (override probing).");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
+
module_spk_synth(synth_keypc);
static int synth_probe(struct spk_synth *synth);
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"\x01+35p" } },
- { CAPS_STOP, .u.s = {"\x01-35p" } },
- { RATE, .u.n = {"\x01%ds", 8, 0, 9, 0, 0, NULL } },
- { PITCH, .u.n = {"\x01%dp", 50, 0, 99, 0, 0, NULL } },
- { VOL, .u.n = {"\x01%dv", 5, 0, 9, 0, 0, NULL } },
- { TONE, .u.n = {"\x01%dx", 1, 0, 2, 0, 0, NULL } },
- { PUNCT, .u.n = {"\x01%db", 7, 0, 15, 0, 0, NULL } },
- { VOICE, .u.n = {"\x01%do", 0, 0, 7, 0, 0, NULL } },
- { FREQUENCY, .u.n = {"\x01%df", 5, 0, 9, 0, 0, NULL } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID,
+ VOL_ID, TONE_ID, PUNCT_ID,
+ VOICE_ID, FREQUENCY_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"\x01+35p" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"\x01-35p" } },
+ [RATE_ID] = { RATE, .u.n = {"\x01%ds", 8, 0, 9, 0, 0, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"\x01%dp", 50, 0, 99, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"\x01%dv", 5, 0, 9, 0, 0, NULL } },
+ [TONE_ID] = { TONE, .u.n = {"\x01%dx", 1, 0, 2, 0, 0, NULL } },
+ [PUNCT_ID] = { PUNCT, .u.n = {"\x01%db", 7, 0, 15, 0, 0, NULL } },
+ [VOICE_ID] = { VOICE, .u.n = {"\x01%do", 0, 0, 7, 0, 0, NULL } },
+ [FREQUENCY_ID] = { FREQUENCY, .u.n = {"\x01%df", 5, 0, 9, 0, 0, NULL } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_named(ser, synth_ltlk.ser, int, 0444);
module_param_named(dev, synth_ltlk.dev_name, charp, 0444);
module_param_named(start, synth_ltlk.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(tone, vars[TONE_ID].u.n.default_val, int, 0444);
+module_param_named(punct, vars[PUNCT_ID].u.n.default_val, int, 0444);
+module_param_named(voice, vars[VOICE_ID].u.n.default_val, int, 0444);
+module_param_named(frequency, vars[FREQUENCY_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
+
+
MODULE_PARM_DESC(ser, "Set the serial port for the synthesizer (0-based).");
MODULE_PARM_DESC(dev, "Set the device e.g. ttyUSB0, for the synthesizer.");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(tone, "Set the tone variable on load.");
+MODULE_PARM_DESC(punct, "Set the punct variable on load.");
+MODULE_PARM_DESC(voice, "Set the voice variable on load.");
+MODULE_PARM_DESC(frequency, "Set the frequency variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
module_spk_synth(synth_ltlk);
static int init_pos;
static int misc_registered;
-static struct var_t vars[] = {
- /* DIRECT is put first so that module_param_named can access it easily */
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
-
- { CAPS_START, .u.s = {"\x01+3p" } },
- { CAPS_STOP, .u.s = {"\x01-3p" } },
- { PAUSE, .u.n = {"\x01P" } },
- { RATE, .u.n = {"\x01%ds", 2, 0, 9, 0, 0, NULL } },
- { PITCH, .u.n = {"\x01%dp", 5, 0, 9, 0, 0, NULL } },
- { INFLECTION, .u.n = {"\x01%dr", 5, 0, 9, 0, 0, NULL } },
- { VOL, .u.n = {"\x01%dv", 5, 0, 9, 0, 0, NULL } },
- { TONE, .u.n = {"\x01%dx", 1, 0, 2, 0, 0, NULL } },
- { PUNCT, .u.n = {"\x01%db", 0, 0, 3, 0, 0, NULL } },
- { VOICE, .u.n = {"\x01%do", 0, 0, 7, 0, 0, NULL } },
- { FREQUENCY, .u.n = {"\x01%df", 5, 0, 9, 0, 0, NULL } },
+
+enum default_vars_id {
+ DIRECT_ID = 0, CAPS_START_ID, CAPS_STOP_ID,
+ PAUSE_ID, RATE_ID, PITCH_ID, INFLECTION_ID,
+ VOL_ID, TONE_ID, PUNCT_ID, VOICE_ID,
+ FREQUENCY_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+
+static struct var_t vars[NB_ID] = {
+
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"\x01+3p" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"\x01-3p" } },
+ [PAUSE_ID] = { PAUSE, .u.n = {"\x01P" } },
+ [RATE_ID] = { RATE, .u.n = {"\x01%ds", 2, 0, 9, 0, 0, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"\x01%dp", 5, 0, 9, 0, 0, NULL } },
+ [INFLECTION_ID] = { INFLECTION, .u.n = {"\x01%dr", 5, 0, 9, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"\x01%dv", 5, 0, 9, 0, 0, NULL } },
+ [TONE_ID] = { TONE, .u.n = {"\x01%dx", 1, 0, 2, 0, 0, NULL } },
+ [PUNCT_ID] = { PUNCT, .u.n = {"\x01%db", 0, 0, 3, 0, 0, NULL } },
+ [VOICE_ID] = { VOICE, .u.n = {"\x01%do", 0, 0, 7, 0, 0, NULL } },
+ [FREQUENCY_ID] = { FREQUENCY, .u.n = {"\x01%df", 5, 0, 9, 0, 0, NULL } },
V_LAST_VAR
};
}
module_param_named(start, synth_soft.startup, short, 0444);
-module_param_named(direct, vars[0].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(inflection, vars[INFLECTION_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(tone, vars[TONE_ID].u.n.default_val, int, 0444);
+module_param_named(punct, vars[PUNCT_ID].u.n.default_val, int, 0444);
+module_param_named(voice, vars[VOICE_ID].u.n.default_val, int, 0444);
+module_param_named(frequency, vars[FREQUENCY_ID].u.n.default_val, int, 0444);
+
+
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+MODULE_PARM_DESC(rate, "Sets the rate of the synthesizer.");
+MODULE_PARM_DESC(pitch, "Sets the pitch of the synthesizer.");
+MODULE_PARM_DESC(inflection, "Sets the inflection of the synthesizer.");
+MODULE_PARM_DESC(vol, "Sets the volume of the speech synthesizer.");
+MODULE_PARM_DESC(tone, "Sets the tone of the speech synthesizer.");
+MODULE_PARM_DESC(punct, "Sets the amount of punctuation spoken by the synthesizer.");
+MODULE_PARM_DESC(voice, "Sets the voice used by the synthesizer.");
+MODULE_PARM_DESC(frequency, "Sets the frequency of speech synthesizer.");
module_spk_synth(synth_soft);
static void synth_flush(struct spk_synth *synth);
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"\x05P+" } },
- { CAPS_STOP, .u.s = {"\x05P-" } },
- { RATE, .u.n = {"\x05R%d", 7, 0, 9, 0, 0, NULL } },
- { PITCH, .u.n = {"\x05P%d", 3, 0, 9, 0, 0, NULL } },
- { VOL, .u.n = {"\x05V%d", 9, 0, 9, 0, 0, NULL } },
- { TONE, .u.n = {"\x05T%c", 8, 0, 25, 65, 0, NULL } },
- { PUNCT, .u.n = {"\x05M%c", 0, 0, 3, 0, 0, "nsma" } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID,
+ VOL_ID, TONE_ID, PUNCT_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"\x05P+" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"\x05P-" } },
+ [RATE_ID] = { RATE, .u.n = {"\x05R%d", 7, 0, 9, 0, 0, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"\x05P%d", 3, 0, 9, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"\x05V%d", 9, 0, 9, 0, 0, NULL } },
+ [TONE_ID] = { TONE, .u.n = {"\x05T%c", 8, 0, 25, 65, 0, NULL } },
+ [PUNCT_ID] = { PUNCT, .u.n = {"\x05M%c", 0, 0, 3, 0, 0, "nsma" } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_named(ser, synth_spkout.ser, int, 0444);
module_param_named(dev, synth_spkout.dev_name, charp, 0444);
module_param_named(start, synth_spkout.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(tone, vars[TONE_ID].u.n.default_val, int, 0444);
+module_param_named(punct, vars[PUNCT_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
+
MODULE_PARM_DESC(ser, "Set the serial port for the synthesizer (0-based).");
MODULE_PARM_DESC(dev, "Set the device e.g. ttyUSB0, for the synthesizer.");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(tone, "Set the tone variable on load.");
+MODULE_PARM_DESC(punct, "Set the punct variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
+
module_spk_synth(synth_spkout);
#define SYNTH_CLEAR 0x18
#define PROCSPEECH '\r' /* process speech char */
-static struct var_t vars[] = {
- { CAPS_START, .u.s = {"\x05P8" } },
- { CAPS_STOP, .u.s = {"\x05P5" } },
- { RATE, .u.n = {"\x05R%d", 5, 0, 9, 0, 0, NULL } },
- { PITCH, .u.n = {"\x05P%d", 5, 0, 9, 0, 0, NULL } },
- { VOL, .u.n = {"\x05V%d", 5, 0, 9, 0, 0, NULL } },
- { TONE, .u.n = {"\x05T%c", 12, 0, 25, 61, 0, NULL } },
- { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
+
+
+
+enum default_vars_id {
+ CAPS_START_ID = 0, CAPS_STOP_ID,
+ RATE_ID, PITCH_ID,
+ VOL_ID, TONE_ID,
+ DIRECT_ID, V_LAST_VAR_ID,
+ NB_ID
+};
+
+
+
+
+
+static struct var_t vars[NB_ID] = {
+ [CAPS_START_ID] = { CAPS_START, .u.s = {"\x05P8" } },
+ [CAPS_STOP_ID] = { CAPS_STOP, .u.s = {"\x05P5" } },
+ [RATE_ID] = { RATE, .u.n = {"\x05R%d", 5, 0, 9, 0, 0, NULL } },
+ [PITCH_ID] = { PITCH, .u.n = {"\x05P%d", 5, 0, 9, 0, 0, NULL } },
+ [VOL_ID] = { VOL, .u.n = {"\x05V%d", 5, 0, 9, 0, 0, NULL } },
+ [TONE_ID] = { TONE, .u.n = {"\x05T%c", 12, 0, 25, 61, 0, NULL } },
+ [DIRECT_ID] = { DIRECT, .u.n = {NULL, 0, 0, 1, 0, 0, NULL } },
V_LAST_VAR
};
module_param_named(ser, synth_txprt.ser, int, 0444);
module_param_named(dev, synth_txprt.dev_name, charp, 0444);
module_param_named(start, synth_txprt.startup, short, 0444);
+module_param_named(rate, vars[RATE_ID].u.n.default_val, int, 0444);
+module_param_named(pitch, vars[PITCH_ID].u.n.default_val, int, 0444);
+module_param_named(vol, vars[VOL_ID].u.n.default_val, int, 0444);
+module_param_named(tone, vars[TONE_ID].u.n.default_val, int, 0444);
+module_param_named(direct, vars[DIRECT_ID].u.n.default_val, int, 0444);
+
+
MODULE_PARM_DESC(ser, "Set the serial port for the synthesizer (0-based).");
MODULE_PARM_DESC(dev, "Set the device e.g. ttyUSB0, for the synthesizer.");
MODULE_PARM_DESC(start, "Start the synthesizer once it is loaded.");
+MODULE_PARM_DESC(rate, "Set the rate variable on load.");
+MODULE_PARM_DESC(pitch, "Set the pitch variable on load.");
+MODULE_PARM_DESC(vol, "Set the vol variable on load.");
+MODULE_PARM_DESC(tone, "Set the tone variable on load.");
+MODULE_PARM_DESC(direct, "Set the direct variable on load.");
+
+
module_spk_synth(synth_txprt);
RATE, PITCH, VOL, TONE, PUNCT, VOICE, FREQUENCY, LANG,
DIRECT, PAUSE,
CAPS_START, CAPS_STOP, CHARTAB, INFLECTION, FLUSH,
- MAXVARS
+ CUR_PHONETIC, MAXVARS
};
typedef int (*special_func)(struct vc_data *vc, u_char type, u_char ch,
{ "chartab", CHARTAB, VAR_PROC, NULL, NULL },
{ "direct", DIRECT, VAR_NUM, NULL, NULL },
{ "pause", PAUSE, VAR_STRING, spk_str_pause, NULL },
+ { "cur_phonetic", CUR_PHONETIC, VAR_NUM, &spk_cur_phonetic, NULL },
};
static struct st_var_header *var_ptrs[MAXVARS] = { NULL, NULL, NULL };
{
return drv->bus->match ? drv->bus->match(dev, drv) : 1;
}
-extern bool driver_allows_async_probing(struct device_driver *drv);
extern int driver_add_groups(struct device_driver *drv,
const struct attribute_group **groups);
.release = bus_release,
};
-static int bus_uevent_filter(struct kobject *kobj)
+static int bus_uevent_filter(const struct kobject *kobj)
{
const struct kobj_type *ktype = get_ktype(kobj);
static int cache_setup_of_node(unsigned int cpu)
{
- struct device_node *np;
+ struct device_node *np, *prev;
struct cacheinfo *this_leaf;
unsigned int index = 0;
return -ENOENT;
}
+ prev = np;
+
while (index < cache_leaves(cpu)) {
this_leaf = per_cpu_cacheinfo_idx(cpu, index);
- if (this_leaf->level != 1)
+ if (this_leaf->level != 1) {
np = of_find_next_cache_node(np);
- else
- np = of_node_get(np);/* cpu node itself */
- if (!np)
- break;
+ of_node_put(prev);
+ prev = np;
+ if (!np)
+ break;
+ }
cache_of_set_props(this_leaf, np);
this_leaf->fw_token = np;
index++;
}
+ of_node_put(np);
+
if (index != cache_leaves(cpu)) /* not all OF nodes populated */
return -ENOENT;
cpumask_clear_cpu(cpu, &sib_leaf->shared_cpu_map);
cpumask_clear_cpu(sibling, &this_leaf->shared_cpu_map);
}
- if (of_have_populated_dt())
- of_node_put(this_leaf->fw_token);
}
}
kfree(cp);
}
-static const struct kobj_ns_type_operations *class_child_ns_type(struct kobject *kobj)
+static const struct kobj_ns_type_operations *class_child_ns_type(const struct kobject *kobj)
{
struct subsys_private *cp = to_subsys_private(kobj);
struct class *class = cp->class;
}
error = class_add_groups(class_get(cls), cls->class_groups);
class_put(cls);
+ if (error) {
+ kobject_del(&cp->subsys.kobj);
+ kfree_const(cp->subsys.kobj.name);
+ kfree(cp);
+ }
return error;
}
EXPORT_SYMBOL_GPL(__class_register);
#include <linux/err.h>
#include <linux/fwnode.h>
#include <linux/init.h>
+#include <linux/kstrtox.h>
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/string.h>
static bool fw_devlink_strict;
static int __init fw_devlink_strict_setup(char *arg)
{
- return strtobool(arg, &fw_devlink_strict);
+ return kstrtobool(arg, &fw_devlink_strict);
}
early_param("fw_devlink.strict", fw_devlink_strict_setup);
{
struct dev_ext_attribute *ea = to_ext_attr(attr);
- if (strtobool(buf, ea->var) < 0)
+ if (kstrtobool(buf, ea->var) < 0)
return -EINVAL;
return size;
kfree(p);
}
-static const void *device_namespace(struct kobject *kobj)
+static const void *device_namespace(const struct kobject *kobj)
{
- struct device *dev = kobj_to_dev(kobj);
+ const struct device *dev = kobj_to_dev(kobj);
const void *ns = NULL;
if (dev->class && dev->class->ns_type)
return ns;
}
-static void device_get_ownership(struct kobject *kobj, kuid_t *uid, kgid_t *gid)
+static void device_get_ownership(const struct kobject *kobj, kuid_t *uid, kgid_t *gid)
{
- struct device *dev = kobj_to_dev(kobj);
+ const struct device *dev = kobj_to_dev(kobj);
if (dev->class && dev->class->get_ownership)
dev->class->get_ownership(dev, uid, gid);
};
-static int dev_uevent_filter(struct kobject *kobj)
+static int dev_uevent_filter(const struct kobject *kobj)
{
const struct kobj_type *ktype = get_ktype(kobj);
if (ktype == &device_ktype) {
- struct device *dev = kobj_to_dev(kobj);
+ const struct device *dev = kobj_to_dev(kobj);
if (dev->bus)
return 1;
if (dev->class)
return 0;
}
-static const char *dev_uevent_name(struct kobject *kobj)
+static const char *dev_uevent_name(const struct kobject *kobj)
{
- struct device *dev = kobj_to_dev(kobj);
+ const struct device *dev = kobj_to_dev(kobj);
if (dev->bus)
return dev->bus->name;
bool val;
int ret;
- ret = strtobool(buf, &val);
+ ret = kstrtobool(buf, &val);
if (ret < 0)
return ret;
const struct attribute_group **groups;
};
-static int devm_attr_group_match(struct device *dev, void *res, void *data)
-{
- return ((union device_attr_group_devres *)res)->group == data;
-}
-
static void devm_attr_group_remove(struct device *dev, void *res)
{
union device_attr_group_devres *devres = res;
EXPORT_SYMBOL_GPL(devm_device_add_group);
/**
- * devm_device_remove_group: remove a managed group from a device
- * @dev: device to remove the group from
- * @grp: group to remove
- *
- * This function removes a group of attributes from a device. The attributes
- * previously have to have been created for this group, otherwise it will fail.
- */
-void devm_device_remove_group(struct device *dev,
- const struct attribute_group *grp)
-{
- WARN_ON(devres_release(dev, devm_attr_group_remove,
- devm_attr_group_match,
- /* cast away const */ (void *)grp));
-}
-EXPORT_SYMBOL_GPL(devm_device_remove_group);
-
-/**
* devm_device_add_groups - create a bunch of managed attribute groups
* @dev: The device to create the group for
* @groups: The attribute groups to create, NULL terminated
}
EXPORT_SYMBOL_GPL(devm_device_add_groups);
-/**
- * devm_device_remove_groups - remove a list of managed groups
- *
- * @dev: The device for the groups to be removed from
- * @groups: NULL terminated list of groups to be removed
- *
- * If groups is not NULL, remove the specified groups from the device.
- */
-void devm_device_remove_groups(struct device *dev,
- const struct attribute_group **groups)
-{
- WARN_ON(devres_release(dev, devm_attr_groups_remove,
- devm_attr_group_match,
- /* cast away const */ (void *)groups));
-}
-EXPORT_SYMBOL_GPL(devm_device_remove_groups);
-
static int device_add_attrs(struct device *dev)
{
struct class *class = dev->class;
}
static const
-struct kobj_ns_type_operations *class_dir_child_ns_type(struct kobject *kobj)
+struct kobj_ns_type_operations *class_dir_child_ns_type(const struct kobject *kobj)
{
- struct class_dir *dir = to_class_dir(kobj);
+ const struct class_dir *dir = to_class_dir(kobj);
return dir->class->ns_type;
}
}
__setup("driver_async_probe=", save_async_options);
-bool driver_allows_async_probing(struct device_driver *drv)
+static bool driver_allows_async_probing(struct device_driver *drv)
{
switch (drv->probe_type) {
case PROBE_PREFER_ASYNCHRONOUS:
return 0;
} else if (ret < 0) {
dev_dbg(dev, "Bus failed to match device: %d\n", ret);
- return ret;
+ /*
+ * Driver could not match with device, but may match with
+ * another device on the bus.
+ */
+ return 0;
} /* ret > 0 means positive match */
if (driver_allows_async_probing(drv)) {
size, tot_size)))
return false;
+ /* Actually allocate the full kmalloc bucket size. */
+ *tot_size = kmalloc_size_roundup(*tot_size);
+
return true;
}
#include <linux/list.h>
#include <linux/completion.h>
-#include <generated/utsrelease.h>
-
/**
* enum fw_opt - options to control firmware loading behaviour
*
};
ATTRIBUTE_GROUPS(firmware_class);
-static int do_firmware_uevent(struct fw_sysfs *fw_sysfs, struct kobj_uevent_env *env)
+static int do_firmware_uevent(const struct fw_sysfs *fw_sysfs, struct kobj_uevent_env *env)
{
if (add_uevent_var(env, "FIRMWARE=%s", fw_sysfs->fw_priv->fw_name))
return -ENOMEM;
return 0;
}
-static int firmware_uevent(struct device *dev, struct kobj_uevent_env *env)
+static int firmware_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
- struct fw_sysfs *fw_sysfs = to_fw_sysfs(dev);
+ const struct fw_sysfs *fw_sysfs = to_fw_sysfs(dev);
int err = 0;
mutex_lock(&fw_lock);
struct firmware *fw;
void *fw_upload_priv;
};
-
-static inline struct fw_sysfs *to_fw_sysfs(struct device *dev)
-{
- return container_of(dev, struct fw_sysfs, dev);
-}
+#define to_fw_sysfs(__dev) container_of_const(__dev, struct fw_sysfs, dev)
void __fw_load_abort(struct fw_priv *fw_priv);
struct resource *r;
int ret;
- if (IS_ENABLED(CONFIG_OF_IRQ) && dev->dev.of_node) {
- ret = of_irq_get_byname(dev->dev.of_node, name);
+ if (!dev->dev.of_node || IS_ENABLED(CONFIG_OF_IRQ)) {
+ ret = fwnode_irq_get_byname(dev_fwnode(&dev->dev), name);
if (ret > 0 || ret == -EPROBE_DEFER)
return ret;
}
#include <linux/property.h>
#include <linux/phy.h>
-struct fwnode_handle *dev_fwnode(const struct device *dev)
+struct fwnode_handle *__dev_fwnode(struct device *dev)
{
return IS_ENABLED(CONFIG_OF) && dev->of_node ?
of_fwnode_handle(dev->of_node) : dev->fwnode;
}
-EXPORT_SYMBOL_GPL(dev_fwnode);
+EXPORT_SYMBOL_GPL(__dev_fwnode);
+
+const struct fwnode_handle *__dev_fwnode_const(const struct device *dev)
+{
+ return IS_ENABLED(CONFIG_OF) && dev->of_node ?
+ of_fwnode_handle(dev->of_node) : dev->fwnode;
+}
+EXPORT_SYMBOL_GPL(__dev_fwnode_const);
/**
* device_property_present - check if a property of a device is present
ret = fwnode_property_read_string_array(fwnode, propname, values, nval);
if (ret < 0)
- goto out;
+ goto out_free;
ret = match_string(values, nval, string);
if (ret < 0)
ret = -ENODATA;
-out:
+
+out_free:
kfree(values);
return ret;
}
* node's parents.
*
* Returns a node pointer with refcount incremented, use
- * fwnode_handle_node() on it when done.
+ * fwnode_handle_put() on it when done.
*/
struct fwnode_handle *fwnode_get_next_parent(struct fwnode_handle *fwnode)
{
* @dev: Device to find the next child node for.
* @child: Handle to one of the device's child nodes or a null handle.
*/
-struct fwnode_handle *device_get_next_child_node(struct device *dev,
+struct fwnode_handle *device_get_next_child_node(const struct device *dev,
struct fwnode_handle *child)
{
const struct fwnode_handle *fwnode = dev_fwnode(dev);
* @dev: Device to find the named child node for.
* @childname: String to match child node name against.
*/
-struct fwnode_handle *device_get_named_child_node(struct device *dev,
+struct fwnode_handle *device_get_named_child_node(const struct device *dev,
const char *childname)
{
return fwnode_get_named_child_node(dev_fwnode(dev), childname);
* device_get_child_node_count - return the number of child nodes for device
* @dev: Device to cound the child nodes for
*/
-unsigned int device_get_child_node_count(struct device *dev)
+unsigned int device_get_child_node_count(const struct device *dev)
{
struct fwnode_handle *child;
unsigned int count = 0;
}
EXPORT_SYMBOL_GPL(device_get_child_node_count);
-bool device_dma_supported(struct device *dev)
+bool device_dma_supported(const struct device *dev)
{
return fwnode_call_bool_op(dev_fwnode(dev), device_dma_supported);
}
EXPORT_SYMBOL_GPL(device_dma_supported);
-enum dev_dma_attr device_get_dma_attr(struct device *dev)
+enum dev_dma_attr device_get_dma_attr(const struct device *dev)
{
if (!fwnode_has_op(dev_fwnode(dev), device_get_dma_attr))
return DEV_DMA_NOT_SUPPORTED;
}
EXPORT_SYMBOL_GPL(device_get_match_data);
-static unsigned int fwnode_graph_devcon_matches(struct fwnode_handle *fwnode,
+static unsigned int fwnode_graph_devcon_matches(const struct fwnode_handle *fwnode,
const char *con_id, void *data,
devcon_match_fn_t match,
void **matches,
return count;
}
-static unsigned int fwnode_devcon_matches(struct fwnode_handle *fwnode,
+static unsigned int fwnode_devcon_matches(const struct fwnode_handle *fwnode,
const char *con_id, void *data,
devcon_match_fn_t match,
void **matches,
* device node. @match will be used to convert the connection description to
* data the caller is expecting to be returned.
*/
-void *fwnode_connection_find_match(struct fwnode_handle *fwnode,
+void *fwnode_connection_find_match(const struct fwnode_handle *fwnode,
const char *con_id, void *data,
devcon_match_fn_t match)
{
*
* Return: Number of matches resolved, or negative errno.
*/
-int fwnode_connection_find_matches(struct fwnode_handle *fwnode,
+int fwnode_connection_find_matches(const struct fwnode_handle *fwnode,
const char *con_id, void *data,
devcon_match_fn_t match,
void **matches, unsigned int matches_len)
The module name will be test_async_driver_probe.ko
If unsure say N.
+
config DRIVER_PE_KUNIT_TEST
bool "KUnit Tests for property entry API" if !KUNIT_ALL_TESTS
depends on KUNIT=y
.llseek = noop_llseek,
};
-static char *aoe_devnode(struct device *dev, umode_t *mode)
+static char *aoe_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "etherd/%s", dev_name(dev));
}
sock.socket->sk->sk_allocation = GFP_NOIO;
msock.socket->sk->sk_allocation = GFP_NOIO;
+ sock.socket->sk->sk_use_task_frag = false;
+ msock.socket->sk->sk_use_task_frag = false;
+
sock.socket->sk->sk_priority = TC_PRIO_INTERACTIVE_BULK;
msock.socket->sk->sk_priority = TC_PRIO_INTERACTIVE;
noreclaim_flag = memalloc_noreclaim_save();
do {
sock->sk->sk_allocation = GFP_NOIO | __GFP_MEMALLOC;
+ sock->sk->sk_use_task_frag = false;
msg.msg_name = NULL;
msg.msg_namelen = 0;
msg.msg_control = NULL;
/* Hardware reset so force device to enter RDDM */
dev_dbg(dev,
"Did not enter RDDM, do a host req reset\n");
- mhi_write_reg(mhi_cntrl, mhi_cntrl->regs,
- MHI_SOC_RESET_REQ_OFFSET,
- MHI_SOC_RESET_REQ);
+ mhi_soc_reset(mhi_cntrl);
udelay(delayus);
}
#define HEALTH_CHECK_PERIOD (HZ * 2)
+/* PCI VID definitions */
+#define PCI_VENDOR_ID_THALES 0x1269
+#define PCI_VENDOR_ID_QUECTEL 0x1eac
+
/**
* struct mhi_pci_dev_info - MHI PCI device specific information
* @config: MHI controller configuration
MHI_CHANNEL_CONFIG_DL(13, "MBIM", 32, 0),
MHI_CHANNEL_CONFIG_UL(32, "DUN", 32, 0),
MHI_CHANNEL_CONFIG_DL(33, "DUN", 32, 0),
+ MHI_CHANNEL_CONFIG_UL(92, "DUN2", 32, 1),
+ MHI_CHANNEL_CONFIG_DL(93, "DUN2", 32, 1),
MHI_CHANNEL_CONFIG_HW_UL(100, "IP_HW0_MBIM", 128, 2),
MHI_CHANNEL_CONFIG_HW_DL(101, "IP_HW0_MBIM", 128, 3),
};
static const struct pci_device_id mhi_pci_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_QCOM, 0x0304),
.driver_data = (kernel_ulong_t) &mhi_qcom_sdx24_info },
+ { PCI_DEVICE_SUB(PCI_VENDOR_ID_QCOM, 0x0306, PCI_VENDOR_ID_QCOM, 0x010c),
+ .driver_data = (kernel_ulong_t) &mhi_foxconn_sdx55_info },
/* EM919x (sdx55), use the same vid:pid as qcom-sdx55m */
{ PCI_DEVICE_SUB(PCI_VENDOR_ID_QCOM, 0x0306, 0x18d7, 0x0200),
.driver_data = (kernel_ulong_t) &mhi_sierra_em919x_info },
.driver_data = (kernel_ulong_t) &mhi_telit_fn990_info },
{ PCI_DEVICE(PCI_VENDOR_ID_QCOM, 0x0308),
.driver_data = (kernel_ulong_t) &mhi_qcom_sdx65_info },
- { PCI_DEVICE(0x1eac, 0x1001), /* EM120R-GL (sdx24) */
+ { PCI_DEVICE(PCI_VENDOR_ID_QUECTEL, 0x1001), /* EM120R-GL (sdx24) */
.driver_data = (kernel_ulong_t) &mhi_quectel_em1xx_info },
- { PCI_DEVICE(0x1eac, 0x1002), /* EM160R-GL (sdx24) */
+ { PCI_DEVICE(PCI_VENDOR_ID_QUECTEL, 0x1002), /* EM160R-GL (sdx24) */
.driver_data = (kernel_ulong_t) &mhi_quectel_em1xx_info },
- { PCI_DEVICE(0x1eac, 0x2001), /* EM120R-GL for FCCL (sdx24) */
+ { PCI_DEVICE(PCI_VENDOR_ID_QUECTEL, 0x2001), /* EM120R-GL for FCCL (sdx24) */
.driver_data = (kernel_ulong_t) &mhi_quectel_em1xx_info },
/* T99W175 (sdx55), Both for eSIM and Non-eSIM */
{ PCI_DEVICE(PCI_VENDOR_ID_FOXCONN, 0xe0ab),
{ PCI_DEVICE(PCI_VENDOR_ID_FOXCONN, 0xe0d9),
.driver_data = (kernel_ulong_t) &mhi_foxconn_sdx65_info },
/* MV31-W (Cinterion) */
- { PCI_DEVICE(0x1269, 0x00b3),
+ { PCI_DEVICE(PCI_VENDOR_ID_THALES, 0x00b3),
.driver_data = (kernel_ulong_t) &mhi_mv31_info },
/* MV31-W (Cinterion), based on new baseline */
- { PCI_DEVICE(0x1269, 0x00b4),
+ { PCI_DEVICE(PCI_VENDOR_ID_THALES, 0x00b4),
.driver_data = (kernel_ulong_t) &mhi_mv31_info },
/* MV32-WA (Cinterion) */
- { PCI_DEVICE(0x1269, 0x00ba),
+ { PCI_DEVICE(PCI_VENDOR_ID_THALES, 0x00ba),
.driver_data = (kernel_ulong_t) &mhi_mv32_info },
/* MV32-WB (Cinterion) */
- { PCI_DEVICE(0x1269, 0x00bb),
+ { PCI_DEVICE(PCI_VENDOR_ID_THALES, 0x00bb),
.driver_data = (kernel_ulong_t) &mhi_mv32_info },
+ /* T99W175 (sdx55), HP variant */
+ { PCI_DEVICE(0x03f0, 0x0a6c),
+ .driver_data = (kernel_ulong_t) &mhi_foxconn_sdx55_info },
{ }
};
MODULE_DEVICE_TABLE(pci, mhi_pci_id_table);
read_lock_irq(&mhi_chan->lock);
/* Only ring DB if ring is not empty */
- if (tre_ring->base && tre_ring->wp != tre_ring->rp)
+ if (tre_ring->base && tre_ring->wp != tre_ring->rp &&
+ mhi_chan->ch_state == MHI_CH_STATE_ENABLED)
mhi_ring_chan_db(mhi_cntrl, mhi_chan);
read_unlock_irq(&mhi_chan->lock);
}
#include <linux/platform_device.h>
#include <linux/random.h>
#include <linux/hw_random.h>
+#include <asm/archrandom.h>
static int powernv_rng_read(struct hwrng *rng, void *data, size_t max, bool wait)
{
#include <linux/sched/signal.h>
#include <asm/debug.h>
#include <asm/cpacf.h>
+#include <asm/archrandom.h>
MODULE_LICENSE("GPL v2");
MODULE_AUTHOR("IBM Corporation");
.llseek = noop_llseek,
};
-static char *mem_devnode(struct device *dev, umode_t *mode)
+static char *mem_devnode(const struct device *dev, umode_t *mode)
{
if (mode && devlist[MINOR(dev->devt)].mode)
*mode = devlist[MINOR(dev->devt)].mode;
* Assigned numbers, used for dynamic minors
*/
#define DYNAMIC_MINORS 128 /* like dynamic majors */
-static DECLARE_BITMAP(misc_minors, DYNAMIC_MINORS);
+static DEFINE_IDA(misc_minors_ida);
+
+static int misc_minor_alloc(void)
+{
+ int ret;
+
+ ret = ida_alloc_max(&misc_minors_ida, DYNAMIC_MINORS - 1, GFP_KERNEL);
+ if (ret >= 0) {
+ ret = DYNAMIC_MINORS - ret - 1;
+ } else {
+ ret = ida_alloc_range(&misc_minors_ida, MISC_DYNAMIC_MINOR + 1,
+ MINORMASK, GFP_KERNEL);
+ }
+ return ret;
+}
+
+static void misc_minor_free(int minor)
+{
+ if (minor < DYNAMIC_MINORS)
+ ida_free(&misc_minors_ida, DYNAMIC_MINORS - minor - 1);
+ else if (minor > MISC_DYNAMIC_MINOR)
+ ida_free(&misc_minors_ida, minor);
+}
#ifdef CONFIG_PROC_FS
static void *misc_seq_start(struct seq_file *seq, loff_t *pos)
mutex_lock(&misc_mtx);
if (is_dynamic) {
- int i = find_first_zero_bit(misc_minors, DYNAMIC_MINORS);
+ int i = misc_minor_alloc();
- if (i >= DYNAMIC_MINORS) {
+ if (i < 0) {
err = -EBUSY;
goto out;
}
- misc->minor = DYNAMIC_MINORS - i - 1;
- set_bit(i, misc_minors);
+ misc->minor = i;
} else {
struct miscdevice *c;
misc, misc->groups, "%s", misc->name);
if (IS_ERR(misc->this_device)) {
if (is_dynamic) {
- int i = DYNAMIC_MINORS - misc->minor - 1;
-
- if (i < DYNAMIC_MINORS && i >= 0)
- clear_bit(i, misc_minors);
+ misc_minor_free(misc->minor);
misc->minor = MISC_DYNAMIC_MINOR;
}
err = PTR_ERR(misc->this_device);
void misc_deregister(struct miscdevice *misc)
{
- int i = DYNAMIC_MINORS - misc->minor - 1;
-
if (WARN_ON(list_empty(&misc->list)))
return;
mutex_lock(&misc_mtx);
list_del(&misc->list);
device_destroy(misc_class, MKDEV(MISC_MAJOR, misc->minor));
- if (i < DYNAMIC_MINORS && i >= 0)
- clear_bit(i, misc_minors);
+ misc_minor_free(misc->minor);
mutex_unlock(&misc_mtx);
}
EXPORT_SYMBOL(misc_deregister);
-static char *misc_devnode(struct device *dev, umode_t *mode)
+static char *misc_devnode(const struct device *dev, umode_t *mode)
{
- struct miscdevice *c = dev_get_drvdata(dev);
+ const struct miscdevice *c = dev_get_drvdata(dev);
if (mode && c->mode)
*mode = c->mode;
#include <linux/sched/isolation.h>
#include <crypto/chacha.h>
#include <crypto/blake2s.h>
+#include <asm/archrandom.h>
#include <asm/processor.h>
#include <asm/irq.h>
#include <asm/irq_regs.h>
#include <linux/fs.h>
#include <linux/splice.h>
#include <linux/pagemap.h>
+#include <linux/idr.h>
#include <linux/init.h>
#include <linux/list.h>
#include <linux/poll.h>
/* List of all the devices we're handling */
struct list_head portdevs;
- /*
- * This is used to keep track of the number of hvc consoles
- * spawned by this driver. This number is given as the first
- * argument to hvc_alloc(). To correctly map an initial
- * console spawned via hvc_instantiate to the console being
- * hooked up via hvc_alloc, we need to pass the same vtermno.
- *
- * We also just assume the first console being initialised was
- * the first one that got used as the initial console.
- */
- unsigned int next_vtermno;
-
/* All the console devices handled by this driver */
struct list_head consoles;
};
-static struct ports_driver_data pdrvdata = { .next_vtermno = 1};
+
+static struct ports_driver_data pdrvdata;
static DEFINE_SPINLOCK(pdrvdata_lock);
static DECLARE_COMPLETION(early_console_added);
u32 vtermno;
};
+static DEFINE_IDA(vtermno_ida);
+
struct port_buffer {
char *buf;
* pointers. The final argument is the output buffer size: we
* can do any size, so we put PAGE_SIZE here.
*/
- port->cons.vtermno = pdrvdata.next_vtermno;
+ ret = ida_alloc_min(&vtermno_ida, 1, GFP_KERNEL);
+ if (ret < 0)
+ return ret;
+ port->cons.vtermno = ret;
port->cons.hvc = hvc_alloc(port->cons.vtermno, 0, &hv_ops, PAGE_SIZE);
if (IS_ERR(port->cons.hvc)) {
ret = PTR_ERR(port->cons.hvc);
dev_err(port->dev,
"error %d allocating hvc for port\n", ret);
port->cons.hvc = NULL;
+ ida_free(&vtermno_ida, port->cons.vtermno);
return ret;
}
spin_lock_irq(&pdrvdata_lock);
- pdrvdata.next_vtermno++;
list_add_tail(&port->cons.list, &pdrvdata.consoles);
spin_unlock_irq(&pdrvdata_lock);
port->guest_connected = true;
list_del(&port->cons.list);
spin_unlock_irq(&pdrvdata_lock);
hvc_remove(port->cons.hvc);
+ ida_free(&vtermno_ida, port->cons.vtermno);
}
remove_port_data(port);
break;
}
- mutex_unlock(&unit_mutex);
-
- if (!unit)
+ if (!unit) {
+ mutex_unlock(&unit_mutex);
return -ENODEV;
+ }
*private_data = unit->private_data;
*index = minor - unit->lowest_minor;
+ mutex_unlock(&unit_mutex);
return 0;
}
EXPORT_SYMBOL(xillybus_find_inode);
struct mutex process_in_mutex; /* synchronize wakeup_all() */
};
+/*
+ * kref_mutex is used in xillyusb_open() to prevent the xillyusb_dev
+ * struct from being freed during the gap between being found by
+ * xillybus_find_inode() and having its reference count incremented.
+ */
+
+static DEFINE_MUTEX(kref_mutex);
+
/* FPGA to host opcodes */
enum {
OPCODE_DATA = 0,
int rc;
int index;
+ mutex_lock(&kref_mutex);
+
rc = xillybus_find_inode(inode, (void **)&xdev, &index);
- if (rc)
+ if (rc) {
+ mutex_unlock(&kref_mutex);
return rc;
+ }
+
+ kref_get(&xdev->kref);
+ mutex_unlock(&kref_mutex);
chan = &xdev->channels[index];
filp->private_data = chan;
((filp->f_mode & FMODE_WRITE) && chan->open_for_write))
goto unmutex_fail;
- kref_get(&xdev->kref);
-
if (filp->f_mode & FMODE_READ)
chan->open_for_read = 1;
return rc;
unmutex_fail:
+ kref_put(&xdev->kref, cleanup_dev);
mutex_unlock(&chan->lock);
return rc;
}
xdev->dev = NULL;
+ mutex_lock(&kref_mutex);
kref_put(&xdev->kref, cleanup_dev);
+ mutex_unlock(&kref_mutex);
}
static struct usb_driver xillyusb_driver = {
/* ensure CMP & ARR registers are properly written */
ret = regmap_read_poll_timeout(priv->regmap, STM32_LPTIM_ISR, val,
- (val & STM32_LPTIM_CMPOK_ARROK),
+ (val & STM32_LPTIM_CMPOK_ARROK) == STM32_LPTIM_CMPOK_ARROK,
100, 1000);
if (ret)
return ret;
local_irq_enable();
snooze_exit_time = get_tb() + get_snooze_timeout(dev, drv, index);
+ dev->poll_time_limit = false;
ppc64_runlatch_off();
HMT_very_low();
while (!need_resched()) {
* cleared to order subsequent test of need_resched().
*/
clear_thread_flag(TIF_POLLING_NRFLAG);
+ dev->poll_time_limit = true;
smp_mb();
break;
}
.desc = "snooze",
.exit_latency = 0,
.target_residency = 0,
- .enter = snooze_loop },
+ .enter = snooze_loop,
+ .flags = CPUIDLE_FLAG_POLLING },
};
static int powernv_cpuidle_cpu_online(unsigned int cpu)
pseries_idle_prolog();
local_irq_enable();
snooze_exit_time = get_tb() + snooze_timeout;
+ dev->poll_time_limit = false;
while (!need_resched()) {
HMT_low();
* loop anyway. Require a barrier after polling is
* cleared to order subsequent test of need_resched().
*/
+ dev->poll_time_limit = true;
clear_thread_flag(TIF_POLLING_NRFLAG);
smp_mb();
break;
.desc = "snooze",
.exit_latency = 0,
.target_residency = 0,
- .enter = &snooze_loop },
+ .enter = &snooze_loop,
+ .flags = CPUIDLE_FLAG_POLLING },
{ /* CEDE */
.name = "CEDE",
.desc = "CEDE",
.desc = "snooze",
.exit_latency = 0,
.target_residency = 0,
- .enter = &snooze_loop },
+ .enter = &snooze_loop,
+ .flags = CPUIDLE_FLAG_POLLING },
{ /* Shared Cede */
.name = "Shared Cede",
.desc = "Shared Cede",
obj-$(CONFIG_CRYPTO_DEV_VMX_ENCRYPT) += vmx-crypto.o
vmx-crypto-objs := vmx.o aesp8-ppc.o ghashp8-ppc.o aes.o aes_cbc.o aes_ctr.o aes_xts.o ghash.o
+ifeq ($(CONFIG_CPU_LITTLE_ENDIAN),y)
+override flavour := linux-ppc64le
+else
+ifdef CONFIG_PPC64_ELF_ABI_V2
+override flavour := linux-ppc64-elfv2
+else
+override flavour := linux-ppc64
+endif
+endif
+
quiet_cmd_perl = PERL $@
- cmd_perl = $(PERL) $< $(if $(CONFIG_CPU_LITTLE_ENDIAN), linux-ppc64le, linux-ppc64) > $@
+ cmd_perl = $(PERL) $< $(flavour) > $@
targets += aesp8-ppc.S ghashp8-ppc.S
$(obj)/aesp8-ppc.S $(obj)/ghashp8-ppc.S: $(obj)/%.S: $(src)/%.pl FORCE
$(call if_changed,perl)
+
+OBJECT_FILES_NON_STANDARD_aesp8-ppc.o := y
my %GLOBALS;
my $dotinlocallabels=($flavour=~/linux/)?1:0;
+my $elfv2abi=(($flavour =~ /linux-ppc64le/) or ($flavour =~ /linux-ppc64-elfv2/))?1:0;
+my $dotfunctions=($elfv2abi=~1)?0:1;
################################################################
# directives which need special treatment on different platforms
};
my $text = sub {
my $ret = ($flavour =~ /aix/) ? ".csect\t.text[PR],7" : ".text";
- $ret = ".abiversion 2\n".$ret if ($flavour =~ /linux.*64le/);
+ $ret = ".abiversion 2\n".$ret if ($elfv2abi);
$ret;
};
my $machine = sub {
if ($flavour =~ /linux/)
{ shift;
my $name = shift; $name =~ s|^[\.\_]||;
- my $ret = ".size $name,.-".($flavour=~/64$/?".":"").$name;
- $ret .= "\n.size .$name,.-.$name" if ($flavour=~/64$/);
+ my $ret = ".size $name,.-".($dotfunctions?".":"").$name;
+ $ret .= "\n.size .$name,.-.$name" if ($dotfunctions);
$ret;
}
else
# Some ABIs specify vrsave, special-purpose register #256, as reserved
# for system use.
-my $no_vrsave = ($flavour =~ /linux-ppc64le/);
+my $no_vrsave = ($elfv2abi);
my $mtspr = sub {
my ($f,$idx,$ra) = @_;
if ($idx == 256 && $no_vrsave) {
#endif /* CONFIG_DIO_CONSTANTS */
+static void dio_dev_release(struct device *dev)
+{
+ struct dio_dev *ddev = container_of(dev, typeof(struct dio_dev), dev);
+ kfree(ddev);
+}
+
int __init dio_find(int deviceid)
{
/* Called to find a DIO device before the full bus scan has run.
dev->bus = &dio_bus;
dev->dev.parent = &dio_bus.dev;
dev->dev.bus = &dio_bus_type;
+ dev->dev.release = dio_dev_release;
dev->scode = scode;
dev->resource.start = pa;
dev->resource.end = pa + DIO_SIZE(scode, va);
if (error) {
pr_err("DIO: Error registering device %s\n",
dev->name);
+ put_device(&dev->dev);
continue;
}
error = dio_create_sysfs_dev_files(dev);
/* Statistics files do not need to send uevents. */
-static int dmabuf_sysfs_uevent_filter(struct kobject *kobj)
+static int dmabuf_sysfs_uevent_filter(const struct kobject *kobj)
{
return 0;
}
return err_ret;
}
-static char *dma_heap_devnode(struct device *dev, umode_t *mode)
+static char *dma_heap_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "dma_heap/%s", dev_name(dev));
}
tristate "Atmel AHB DMA support"
depends on ARCH_AT91
select DMA_ENGINE
+ select DMA_VIRTUAL_CHANNELS
help
Support the Atmel AHB DMA controller.
If unsure, say N.
-config INTEL_IOP_ADMA
- tristate "Intel IOP32x ADMA support"
- depends on ARCH_IOP32X || COMPILE_TEST
- select DMA_ENGINE
- select ASYNC_TX_ENABLE_CHANNEL_SWITCH
- help
- Enable support for the Intel(R) IOP Series RAID engines.
-
config K3_DMA
tristate "Hisilicon K3 DMA support"
depends on ARCH_HI3xxx || ARCH_HISI || COMPILE_TEST
obj-$(CONFIG_INTEL_IDMA64) += idma64.o
obj-$(CONFIG_INTEL_IOATDMA) += ioat/
obj-y += idxd/
-obj-$(CONFIG_INTEL_IOP_ADMA) += iop-adma.o
obj-$(CONFIG_K3_DMA) += k3dma.o
obj-$(CONFIG_LPC18XX_DMAMUX) += lpc18xx-dmamux.o
obj-$(CONFIG_MILBEAUT_HDMAC) += milbeaut-hdmac.o
#define NCHANNELS_MAX 64
#define IRQ_NOUTPUTS 4
+/*
+ * For allocation purposes we split the cache
+ * memory into blocks of fixed size (given in bytes).
+ */
+#define SRAM_BLOCK 2048
+
#define RING_WRITE_SLOT GENMASK(1, 0)
#define RING_READ_SLOT GENMASK(5, 4)
#define RING_FULL BIT(9)
#define REG_TX_STOP 0x0004
#define REG_RX_START 0x0008
#define REG_RX_STOP 0x000c
+#define REG_IMPRINT 0x0090
+#define REG_TX_SRAM_SIZE 0x0094
+#define REG_RX_SRAM_SIZE 0x0098
#define REG_CHAN_CTL(ch) (0x8000 + (ch) * 0x200)
#define REG_CHAN_CTL_RST_RINGS BIT(0)
#define BUS_WIDTH_FRAME_2_WORDS 0x10
#define BUS_WIDTH_FRAME_4_WORDS 0x20
-#define CHAN_BUFSIZE 0x8000
+#define REG_CHAN_SRAM_CARVEOUT(ch) (0x8050 + (ch) * 0x200)
+#define CHAN_SRAM_CARVEOUT_SIZE GENMASK(31, 16)
+#define CHAN_SRAM_CARVEOUT_BASE GENMASK(15, 0)
#define REG_CHAN_FIFOCTL(ch) (0x8054 + (ch) * 0x200)
#define CHAN_FIFOCTL_LIMIT GENMASK(31, 16)
struct dma_chan chan;
struct tasklet_struct tasklet;
+ u32 carveout;
+
spinlock_t lock;
struct admac_tx *current_tx;
int nperiod_acks;
struct list_head to_free;
};
+struct admac_sram {
+ u32 size;
+ /*
+ * SRAM_CARVEOUT has 16-bit fields, so the SRAM cannot be larger than
+ * 64K and a 32-bit bitfield over 2K blocks covers it.
+ */
+ u32 allocated;
+};
+
struct admac_data {
struct dma_device dma;
struct device *dev;
__iomem void *base;
struct reset_control *rstc;
+ struct mutex cache_alloc_lock;
+ struct admac_sram txcache, rxcache;
+
int irq;
int irq_index;
int nchannels;
struct list_head node;
};
+static int admac_alloc_sram_carveout(struct admac_data *ad,
+ enum dma_transfer_direction dir,
+ u32 *out)
+{
+ struct admac_sram *sram;
+ int i, ret = 0, nblocks;
+
+ if (dir == DMA_MEM_TO_DEV)
+ sram = &ad->txcache;
+ else
+ sram = &ad->rxcache;
+
+ mutex_lock(&ad->cache_alloc_lock);
+
+ nblocks = sram->size / SRAM_BLOCK;
+ for (i = 0; i < nblocks; i++)
+ if (!(sram->allocated & BIT(i)))
+ break;
+
+ if (i < nblocks) {
+ *out = FIELD_PREP(CHAN_SRAM_CARVEOUT_BASE, i * SRAM_BLOCK) |
+ FIELD_PREP(CHAN_SRAM_CARVEOUT_SIZE, SRAM_BLOCK);
+ sram->allocated |= BIT(i);
+ } else {
+ ret = -EBUSY;
+ }
+
+ mutex_unlock(&ad->cache_alloc_lock);
+
+ return ret;
+}
+
+static void admac_free_sram_carveout(struct admac_data *ad,
+ enum dma_transfer_direction dir,
+ u32 carveout)
+{
+ struct admac_sram *sram;
+ u32 base = FIELD_GET(CHAN_SRAM_CARVEOUT_BASE, carveout);
+ int i;
+
+ if (dir == DMA_MEM_TO_DEV)
+ sram = &ad->txcache;
+ else
+ sram = &ad->rxcache;
+
+ if (WARN_ON(base >= sram->size))
+ return;
+
+ mutex_lock(&ad->cache_alloc_lock);
+ i = base / SRAM_BLOCK;
+ sram->allocated &= ~BIT(i);
+ mutex_unlock(&ad->cache_alloc_lock);
+}
+
static void admac_modify(struct admac_data *ad, int reg, u32 mask, u32 val)
{
void __iomem *addr = ad->base + reg;
static int admac_alloc_chan_resources(struct dma_chan *chan)
{
struct admac_chan *adchan = to_admac_chan(chan);
+ struct admac_data *ad = adchan->host;
+ int ret;
dma_cookie_init(&adchan->chan);
+ ret = admac_alloc_sram_carveout(ad, admac_chan_direction(adchan->no),
+ &adchan->carveout);
+ if (ret < 0)
+ return ret;
+
+ writel_relaxed(adchan->carveout,
+ ad->base + REG_CHAN_SRAM_CARVEOUT(adchan->no));
return 0;
}
static void admac_free_chan_resources(struct dma_chan *chan)
{
+ struct admac_chan *adchan = to_admac_chan(chan);
+
admac_terminate_all(chan);
admac_synchronize(chan);
+ admac_free_sram_carveout(adchan->host, admac_chan_direction(adchan->no),
+ adchan->carveout);
}
static struct dma_chan *admac_dma_of_xlate(struct of_phandle_args *dma_spec,
platform_set_drvdata(pdev, ad);
ad->dev = &pdev->dev;
ad->nchannels = nchannels;
+ mutex_init(&ad->cache_alloc_lock);
/*
* The controller has 4 IRQ outputs. Try them all until
goto free_irq;
}
+ ad->txcache.size = readl_relaxed(ad->base + REG_TX_SRAM_SIZE);
+ ad->rxcache.size = readl_relaxed(ad->base + REG_RX_SRAM_SIZE);
+
+ dev_info(&pdev->dev, "Audio DMA Controller\n");
+ dev_info(&pdev->dev, "imprint %x TX cache %u RX cache %u\n",
+ readl_relaxed(ad->base + REG_IMPRINT), ad->txcache.size, ad->rxcache.size);
+
return 0;
free_irq:
* Driver for the Atmel AHB DMA Controller (aka HDMA or DMAC on AT91 systems)
*
* Copyright (C) 2008 Atmel Corporation
+ * Copyright (C) 2022 Microchip Technology, Inc. and its subsidiaries
*
* This supports the Atmel AHB DMA Controller found in several Atmel SoCs.
* The only Atmel DMA Controller that is not covered by this driver is the one
*/
#include <dt-bindings/dma/at91.h>
+#include <linux/bitfield.h>
#include <linux/clk.h>
#include <linux/dmaengine.h>
-#include <linux/dma-mapping.h>
#include <linux/dmapool.h>
+#include <linux/dma-mapping.h>
#include <linux/interrupt.h>
#include <linux/module.h>
-#include <linux/platform_device.h>
-#include <linux/slab.h>
#include <linux/of.h>
+#include <linux/overflow.h>
#include <linux/of_device.h>
#include <linux/of_dma.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
-#include "at_hdmac_regs.h"
#include "dmaengine.h"
+#include "virt-dma.h"
/*
* Glossary
* atc_ / atchan : ATmel DMA Channel entity related
*/
-#define ATC_DEFAULT_CFG (ATC_FIFOCFG_HALFFIFO)
-#define ATC_DEFAULT_CTRLB (ATC_SIF(AT_DMA_MEM_IF) \
- |ATC_DIF(AT_DMA_MEM_IF))
+#define AT_DMA_MAX_NR_CHANNELS 8
+
+/* Global Configuration Register */
+#define AT_DMA_GCFG 0x00
+#define AT_DMA_IF_BIGEND(i) BIT((i)) /* AHB-Lite Interface i in Big-endian mode */
+#define AT_DMA_ARB_CFG BIT(4) /* Arbiter mode. */
+
+/* Controller Enable Register */
+#define AT_DMA_EN 0x04
+#define AT_DMA_ENABLE BIT(0)
+
+/* Software Single Request Register */
+#define AT_DMA_SREQ 0x08
+#define AT_DMA_SSREQ(x) BIT((x) << 1) /* Request a source single transfer on channel x */
+#define AT_DMA_DSREQ(x) BIT(1 + ((x) << 1)) /* Request a destination single transfer on channel x */
+
+/* Software Chunk Transfer Request Register */
+#define AT_DMA_CREQ 0x0c
+#define AT_DMA_SCREQ(x) BIT((x) << 1) /* Request a source chunk transfer on channel x */
+#define AT_DMA_DCREQ(x) BIT(1 + ((x) << 1)) /* Request a destination chunk transfer on channel x */
+
+/* Software Last Transfer Flag Register */
+#define AT_DMA_LAST 0x10
+#define AT_DMA_SLAST(x) BIT((x) << 1) /* This src rq is last tx of buffer on channel x */
+#define AT_DMA_DLAST(x) BIT(1 + ((x) << 1)) /* This dst rq is last tx of buffer on channel x */
+
+/* Request Synchronization Register */
+#define AT_DMA_SYNC 0x14
+#define AT_DMA_SYR(h) BIT((h)) /* Synchronize handshake line h */
+
+/* Error, Chained Buffer transfer completed and Buffer transfer completed Interrupt registers */
+#define AT_DMA_EBCIER 0x18 /* Enable register */
+#define AT_DMA_EBCIDR 0x1c /* Disable register */
+#define AT_DMA_EBCIMR 0x20 /* Mask Register */
+#define AT_DMA_EBCISR 0x24 /* Status Register */
+#define AT_DMA_CBTC_OFFSET 8
+#define AT_DMA_ERR_OFFSET 16
+#define AT_DMA_BTC(x) BIT((x))
+#define AT_DMA_CBTC(x) BIT(AT_DMA_CBTC_OFFSET + (x))
+#define AT_DMA_ERR(x) BIT(AT_DMA_ERR_OFFSET + (x))
+
+/* Channel Handler Enable Register */
+#define AT_DMA_CHER 0x28
+#define AT_DMA_ENA(x) BIT((x))
+#define AT_DMA_SUSP(x) BIT(8 + (x))
+#define AT_DMA_KEEP(x) BIT(24 + (x))
+
+/* Channel Handler Disable Register */
+#define AT_DMA_CHDR 0x2c
+#define AT_DMA_DIS(x) BIT(x)
+#define AT_DMA_RES(x) BIT(8 + (x))
+
+/* Channel Handler Status Register */
+#define AT_DMA_CHSR 0x30
+#define AT_DMA_EMPT(x) BIT(16 + (x))
+#define AT_DMA_STAL(x) BIT(24 + (x))
+
+/* Channel registers base address */
+#define AT_DMA_CH_REGS_BASE 0x3c
+#define ch_regs(x) (AT_DMA_CH_REGS_BASE + (x) * 0x28) /* Channel x base addr */
+
+/* Hardware register offset for each channel */
+#define ATC_SADDR_OFFSET 0x00 /* Source Address Register */
+#define ATC_DADDR_OFFSET 0x04 /* Destination Address Register */
+#define ATC_DSCR_OFFSET 0x08 /* Descriptor Address Register */
+#define ATC_CTRLA_OFFSET 0x0c /* Control A Register */
+#define ATC_CTRLB_OFFSET 0x10 /* Control B Register */
+#define ATC_CFG_OFFSET 0x14 /* Configuration Register */
+#define ATC_SPIP_OFFSET 0x18 /* Src PIP Configuration Register */
+#define ATC_DPIP_OFFSET 0x1c /* Dst PIP Configuration Register */
+
+
+/* Bitfield definitions */
+
+/* Bitfields in DSCR */
+#define ATC_DSCR_IF GENMASK(1, 0) /* Dsc feched via AHB-Lite Interface */
+
+/* Bitfields in CTRLA */
+#define ATC_BTSIZE_MAX GENMASK(15, 0) /* Maximum Buffer Transfer Size */
+#define ATC_BTSIZE GENMASK(15, 0) /* Buffer Transfer Size */
+#define ATC_SCSIZE GENMASK(18, 16) /* Source Chunk Transfer Size */
+#define ATC_DCSIZE GENMASK(22, 20) /* Destination Chunk Transfer Size */
+#define ATC_SRC_WIDTH GENMASK(25, 24) /* Source Single Transfer Size */
+#define ATC_DST_WIDTH GENMASK(29, 28) /* Destination Single Transfer Size */
+#define ATC_DONE BIT(31) /* Tx Done (only written back in descriptor) */
+
+/* Bitfields in CTRLB */
+#define ATC_SIF GENMASK(1, 0) /* Src tx done via AHB-Lite Interface i */
+#define ATC_DIF GENMASK(5, 4) /* Dst tx done via AHB-Lite Interface i */
+#define AT_DMA_MEM_IF 0x0 /* interface 0 as memory interface */
+#define AT_DMA_PER_IF 0x1 /* interface 1 as peripheral interface */
+#define ATC_SRC_PIP BIT(8) /* Source Picture-in-Picture enabled */
+#define ATC_DST_PIP BIT(12) /* Destination Picture-in-Picture enabled */
+#define ATC_SRC_DSCR_DIS BIT(16) /* Src Descriptor fetch disable */
+#define ATC_DST_DSCR_DIS BIT(20) /* Dst Descriptor fetch disable */
+#define ATC_FC GENMASK(22, 21) /* Choose Flow Controller */
+#define ATC_FC_MEM2MEM 0x0 /* Mem-to-Mem (DMA) */
+#define ATC_FC_MEM2PER 0x1 /* Mem-to-Periph (DMA) */
+#define ATC_FC_PER2MEM 0x2 /* Periph-to-Mem (DMA) */
+#define ATC_FC_PER2PER 0x3 /* Periph-to-Periph (DMA) */
+#define ATC_FC_PER2MEM_PER 0x4 /* Periph-to-Mem (Peripheral) */
+#define ATC_FC_MEM2PER_PER 0x5 /* Mem-to-Periph (Peripheral) */
+#define ATC_FC_PER2PER_SRCPER 0x6 /* Periph-to-Periph (Src Peripheral) */
+#define ATC_FC_PER2PER_DSTPER 0x7 /* Periph-to-Periph (Dst Peripheral) */
+#define ATC_SRC_ADDR_MODE GENMASK(25, 24)
+#define ATC_SRC_ADDR_MODE_INCR 0x0 /* Incrementing Mode */
+#define ATC_SRC_ADDR_MODE_DECR 0x1 /* Decrementing Mode */
+#define ATC_SRC_ADDR_MODE_FIXED 0x2 /* Fixed Mode */
+#define ATC_DST_ADDR_MODE GENMASK(29, 28)
+#define ATC_DST_ADDR_MODE_INCR 0x0 /* Incrementing Mode */
+#define ATC_DST_ADDR_MODE_DECR 0x1 /* Decrementing Mode */
+#define ATC_DST_ADDR_MODE_FIXED 0x2 /* Fixed Mode */
+#define ATC_IEN BIT(30) /* BTC interrupt enable (active low) */
+#define ATC_AUTO BIT(31) /* Auto multiple buffer tx enable */
+
+/* Bitfields in CFG */
+#define ATC_PER_MSB(h) ((0x30U & (h)) >> 4) /* Extract most significant bits of a handshaking identifier */
+
+#define ATC_SRC_PER GENMASK(3, 0) /* Channel src rq associated with periph handshaking ifc h */
+#define ATC_DST_PER GENMASK(7, 4) /* Channel dst rq associated with periph handshaking ifc h */
+#define ATC_SRC_REP BIT(8) /* Source Replay Mod */
+#define ATC_SRC_H2SEL BIT(9) /* Source Handshaking Mod */
+#define ATC_SRC_PER_MSB GENMASK(11, 10) /* Channel src rq (most significant bits) */
+#define ATC_DST_REP BIT(12) /* Destination Replay Mod */
+#define ATC_DST_H2SEL BIT(13) /* Destination Handshaking Mod */
+#define ATC_DST_PER_MSB GENMASK(15, 14) /* Channel dst rq (most significant bits) */
+#define ATC_SOD BIT(16) /* Stop On Done */
+#define ATC_LOCK_IF BIT(20) /* Interface Lock */
+#define ATC_LOCK_B BIT(21) /* AHB Bus Lock */
+#define ATC_LOCK_IF_L BIT(22) /* Master Interface Arbiter Lock */
+#define ATC_AHB_PROT GENMASK(26, 24) /* AHB Protection */
+#define ATC_FIFOCFG GENMASK(29, 28) /* FIFO Request Configuration */
+#define ATC_FIFOCFG_LARGESTBURST 0x0
+#define ATC_FIFOCFG_HALFFIFO 0x1
+#define ATC_FIFOCFG_ENOUGHSPACE 0x2
+
+/* Bitfields in SPIP */
+#define ATC_SPIP_HOLE GENMASK(15, 0)
+#define ATC_SPIP_BOUNDARY GENMASK(25, 16)
+
+/* Bitfields in DPIP */
+#define ATC_DPIP_HOLE GENMASK(15, 0)
+#define ATC_DPIP_BOUNDARY GENMASK(25, 16)
+
+#define ATC_SRC_PER_ID(id) (FIELD_PREP(ATC_SRC_PER_MSB, (id)) | \
+ FIELD_PREP(ATC_SRC_PER, (id)))
+#define ATC_DST_PER_ID(id) (FIELD_PREP(ATC_DST_PER_MSB, (id)) | \
+ FIELD_PREP(ATC_DST_PER, (id)))
+
+
+
+/*-- descriptors -----------------------------------------------------*/
+
+/* LLI == Linked List Item; aka DMA buffer descriptor */
+struct at_lli {
+ /* values that are not changed by hardware */
+ u32 saddr;
+ u32 daddr;
+ /* value that may get written back: */
+ u32 ctrla;
+ /* more values that are not changed by hardware */
+ u32 ctrlb;
+ u32 dscr; /* chain to next lli */
+};
+
+/**
+ * struct atdma_sg - atdma scatter gather entry
+ * @len: length of the current Linked List Item.
+ * @lli: linked list item that is passed to the DMA controller
+ * @lli_phys: physical address of the LLI.
+ */
+struct atdma_sg {
+ unsigned int len;
+ struct at_lli *lli;
+ dma_addr_t lli_phys;
+};
+
+/**
+ * struct at_desc - software descriptor
+ * @vd: pointer to the virtual dma descriptor.
+ * @atchan: pointer to the atmel dma channel.
+ * @total_len: total transaction byte count
+ * @sg_len: number of sg entries.
+ * @sg: array of sgs.
+ */
+struct at_desc {
+ struct virt_dma_desc vd;
+ struct at_dma_chan *atchan;
+ size_t total_len;
+ unsigned int sglen;
+ /* Interleaved data */
+ size_t boundary;
+ size_t dst_hole;
+ size_t src_hole;
+
+ /* Memset temporary buffer */
+ bool memset_buffer;
+ dma_addr_t memset_paddr;
+ int *memset_vaddr;
+ struct atdma_sg sg[];
+};
+
+/*-- Channels --------------------------------------------------------*/
+
+/**
+ * atc_status - information bits stored in channel status flag
+ *
+ * Manipulated with atomic operations.
+ */
+enum atc_status {
+ ATC_IS_PAUSED = 1,
+ ATC_IS_CYCLIC = 24,
+};
+
+/**
+ * struct at_dma_chan - internal representation of an Atmel HDMAC channel
+ * @vc: virtual dma channel entry.
+ * @atdma: pointer to the driver data.
+ * @ch_regs: memory mapped register base
+ * @mask: channel index in a mask
+ * @per_if: peripheral interface
+ * @mem_if: memory interface
+ * @status: transmit status information from irq/prep* functions
+ * to tasklet (use atomic operations)
+ * @save_cfg: configuration register that is saved on suspend/resume cycle
+ * @save_dscr: for cyclic operations, preserve next descriptor address in
+ * the cyclic list on suspend/resume cycle
+ * @dma_sconfig: configuration for slave transfers, passed via
+ * .device_config
+ * @desc: pointer to the atmel dma descriptor.
+ */
+struct at_dma_chan {
+ struct virt_dma_chan vc;
+ struct at_dma *atdma;
+ void __iomem *ch_regs;
+ u8 mask;
+ u8 per_if;
+ u8 mem_if;
+ unsigned long status;
+ u32 save_cfg;
+ u32 save_dscr;
+ struct dma_slave_config dma_sconfig;
+ bool cyclic;
+ struct at_desc *desc;
+};
+
+#define channel_readl(atchan, name) \
+ __raw_readl((atchan)->ch_regs + ATC_##name##_OFFSET)
+
+#define channel_writel(atchan, name, val) \
+ __raw_writel((val), (atchan)->ch_regs + ATC_##name##_OFFSET)
+
+/*
+ * Fix sconfig's burst size according to at_hdmac. We need to convert them as:
+ * 1 -> 0, 4 -> 1, 8 -> 2, 16 -> 3, 32 -> 4, 64 -> 5, 128 -> 6, 256 -> 7.
+ *
+ * This can be done by finding most significant bit set.
+ */
+static inline void convert_burst(u32 *maxburst)
+{
+ if (*maxburst > 1)
+ *maxburst = fls(*maxburst) - 2;
+ else
+ *maxburst = 0;
+}
+
+/*
+ * Fix sconfig's bus width according to at_hdmac.
+ * 1 byte -> 0, 2 bytes -> 1, 4 bytes -> 2.
+ */
+static inline u8 convert_buswidth(enum dma_slave_buswidth addr_width)
+{
+ switch (addr_width) {
+ case DMA_SLAVE_BUSWIDTH_2_BYTES:
+ return 1;
+ case DMA_SLAVE_BUSWIDTH_4_BYTES:
+ return 2;
+ default:
+ /* For 1 byte width or fallback */
+ return 0;
+ }
+}
+
+/*-- Controller ------------------------------------------------------*/
+
+/**
+ * struct at_dma - internal representation of an Atmel HDMA Controller
+ * @dma_device: dmaengine dma_device object members
+ * @atdma_devtype: identifier of DMA controller compatibility
+ * @ch_regs: memory mapped register base
+ * @clk: dma controller clock
+ * @save_imr: interrupt mask register that is saved on suspend/resume cycle
+ * @all_chan_mask: all channels availlable in a mask
+ * @lli_pool: hw lli table
+ * @chan: channels table to store at_dma_chan structures
+ */
+struct at_dma {
+ struct dma_device dma_device;
+ void __iomem *regs;
+ struct clk *clk;
+ u32 save_imr;
+
+ u8 all_chan_mask;
+
+ struct dma_pool *lli_pool;
+ struct dma_pool *memset_pool;
+ /* AT THE END channels table */
+ struct at_dma_chan chan[];
+};
+
+#define dma_readl(atdma, name) \
+ __raw_readl((atdma)->regs + AT_DMA_##name)
+#define dma_writel(atdma, name, val) \
+ __raw_writel((val), (atdma)->regs + AT_DMA_##name)
+
+static inline struct at_desc *to_atdma_desc(struct dma_async_tx_descriptor *t)
+{
+ return container_of(t, struct at_desc, vd.tx);
+}
+
+static inline struct at_dma_chan *to_at_dma_chan(struct dma_chan *chan)
+{
+ return container_of(chan, struct at_dma_chan, vc.chan);
+}
+
+static inline struct at_dma *to_at_dma(struct dma_device *ddev)
+{
+ return container_of(ddev, struct at_dma, dma_device);
+}
+
+
+/*-- Helper functions ------------------------------------------------*/
+
+static struct device *chan2dev(struct dma_chan *chan)
+{
+ return &chan->dev->device;
+}
+
+#if defined(VERBOSE_DEBUG)
+static void vdbg_dump_regs(struct at_dma_chan *atchan)
+{
+ struct at_dma *atdma = to_at_dma(atchan->vc.chan.device);
+
+ dev_err(chan2dev(&atchan->vc.chan),
+ " channel %d : imr = 0x%x, chsr = 0x%x\n",
+ atchan->vc.chan.chan_id,
+ dma_readl(atdma, EBCIMR),
+ dma_readl(atdma, CHSR));
+
+ dev_err(chan2dev(&atchan->vc.chan),
+ " channel: s0x%x d0x%x ctrl0x%x:0x%x cfg0x%x l0x%x\n",
+ channel_readl(atchan, SADDR),
+ channel_readl(atchan, DADDR),
+ channel_readl(atchan, CTRLA),
+ channel_readl(atchan, CTRLB),
+ channel_readl(atchan, CFG),
+ channel_readl(atchan, DSCR));
+}
+#else
+static void vdbg_dump_regs(struct at_dma_chan *atchan) {}
+#endif
+
+static void atc_dump_lli(struct at_dma_chan *atchan, struct at_lli *lli)
+{
+ dev_crit(chan2dev(&atchan->vc.chan),
+ "desc: s%pad d%pad ctrl0x%x:0x%x l%pad\n",
+ &lli->saddr, &lli->daddr,
+ lli->ctrla, lli->ctrlb, &lli->dscr);
+}
+
+
+static void atc_setup_irq(struct at_dma *atdma, int chan_id, int on)
+{
+ u32 ebci;
+
+ /* enable interrupts on buffer transfer completion & error */
+ ebci = AT_DMA_BTC(chan_id)
+ | AT_DMA_ERR(chan_id);
+ if (on)
+ dma_writel(atdma, EBCIER, ebci);
+ else
+ dma_writel(atdma, EBCIDR, ebci);
+}
+
+static void atc_enable_chan_irq(struct at_dma *atdma, int chan_id)
+{
+ atc_setup_irq(atdma, chan_id, 1);
+}
+
+static void atc_disable_chan_irq(struct at_dma *atdma, int chan_id)
+{
+ atc_setup_irq(atdma, chan_id, 0);
+}
+
+
+/**
+ * atc_chan_is_enabled - test if given channel is enabled
+ * @atchan: channel we want to test status
+ */
+static inline int atc_chan_is_enabled(struct at_dma_chan *atchan)
+{
+ struct at_dma *atdma = to_at_dma(atchan->vc.chan.device);
+
+ return !!(dma_readl(atdma, CHSR) & atchan->mask);
+}
+
+/**
+ * atc_chan_is_paused - test channel pause/resume status
+ * @atchan: channel we want to test status
+ */
+static inline int atc_chan_is_paused(struct at_dma_chan *atchan)
+{
+ return test_bit(ATC_IS_PAUSED, &atchan->status);
+}
+
+/**
+ * atc_chan_is_cyclic - test if given channel has cyclic property set
+ * @atchan: channel we want to test status
+ */
+static inline int atc_chan_is_cyclic(struct at_dma_chan *atchan)
+{
+ return test_bit(ATC_IS_CYCLIC, &atchan->status);
+}
+
+/**
+ * set_lli_eol - set end-of-link to descriptor so it will end transfer
+ * @desc: descriptor, signle or at the end of a chain, to end chain on
+ * @i: index of the atmel scatter gather entry that is at the end of the chain.
+ */
+static void set_lli_eol(struct at_desc *desc, unsigned int i)
+{
+ u32 ctrlb = desc->sg[i].lli->ctrlb;
+
+ ctrlb &= ~ATC_IEN;
+ ctrlb |= ATC_SRC_DSCR_DIS | ATC_DST_DSCR_DIS;
+
+ desc->sg[i].lli->ctrlb = ctrlb;
+ desc->sg[i].lli->dscr = 0;
+}
+
+#define ATC_DEFAULT_CFG FIELD_PREP(ATC_FIFOCFG, ATC_FIFOCFG_HALFFIFO)
+#define ATC_DEFAULT_CTRLB (FIELD_PREP(ATC_SIF, AT_DMA_MEM_IF) | \
+ FIELD_PREP(ATC_DIF, AT_DMA_MEM_IF))
#define ATC_DMA_BUSWIDTHS\
(BIT(DMA_SLAVE_BUSWIDTH_UNDEFINED) |\
BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |\
u32 cfg;
};
-/* prototypes */
-static dma_cookie_t atc_tx_submit(struct dma_async_tx_descriptor *tx);
-static void atc_issue_pending(struct dma_chan *chan);
-
-
-/*----------------------------------------------------------------------*/
-
static inline unsigned int atc_get_xfer_width(dma_addr_t src, dma_addr_t dst,
size_t len)
{
return width;
}
-static struct at_desc *atc_first_active(struct at_dma_chan *atchan)
-{
- return list_first_entry(&atchan->active_list,
- struct at_desc, desc_node);
-}
-
-static struct at_desc *atc_first_queued(struct at_dma_chan *atchan)
-{
- return list_first_entry(&atchan->queue,
- struct at_desc, desc_node);
-}
-
-/**
- * atc_alloc_descriptor - allocate and return an initialized descriptor
- * @chan: the channel to allocate descriptors for
- * @gfp_flags: GFP allocation flags
- *
- * Note: The ack-bit is positioned in the descriptor flag at creation time
- * to make initial allocation more convenient. This bit will be cleared
- * and control will be given to client at usage time (during
- * preparation functions).
- */
-static struct at_desc *atc_alloc_descriptor(struct dma_chan *chan,
- gfp_t gfp_flags)
-{
- struct at_desc *desc = NULL;
- struct at_dma *atdma = to_at_dma(chan->device);
- dma_addr_t phys;
-
- desc = dma_pool_zalloc(atdma->dma_desc_pool, gfp_flags, &phys);
- if (desc) {
- INIT_LIST_HEAD(&desc->tx_list);
- dma_async_tx_descriptor_init(&desc->txd, chan);
- /* txd.flags will be overwritten in prep functions */
- desc->txd.flags = DMA_CTRL_ACK;
- desc->txd.tx_submit = atc_tx_submit;
- desc->txd.phys = phys;
- }
-
- return desc;
-}
-
-/**
- * atc_desc_get - get an unused descriptor from free_list
- * @atchan: channel we want a new descriptor for
- */
-static struct at_desc *atc_desc_get(struct at_dma_chan *atchan)
+static void atdma_lli_chain(struct at_desc *desc, unsigned int i)
{
- struct at_desc *desc, *_desc;
- struct at_desc *ret = NULL;
- unsigned long flags;
- unsigned int i = 0;
-
- spin_lock_irqsave(&atchan->lock, flags);
- list_for_each_entry_safe(desc, _desc, &atchan->free_list, desc_node) {
- i++;
- if (async_tx_test_ack(&desc->txd)) {
- list_del(&desc->desc_node);
- ret = desc;
- break;
- }
- dev_dbg(chan2dev(&atchan->chan_common),
- "desc %p not ACKed\n", desc);
- }
- spin_unlock_irqrestore(&atchan->lock, flags);
- dev_vdbg(chan2dev(&atchan->chan_common),
- "scanned %u descriptors on freelist\n", i);
-
- /* no more descriptor available in initial pool: create one more */
- if (!ret)
- ret = atc_alloc_descriptor(&atchan->chan_common, GFP_NOWAIT);
-
- return ret;
-}
+ struct atdma_sg *atdma_sg = &desc->sg[i];
-/**
- * atc_desc_put - move a descriptor, including any children, to the free list
- * @atchan: channel we work on
- * @desc: descriptor, at the head of a chain, to move to free list
- */
-static void atc_desc_put(struct at_dma_chan *atchan, struct at_desc *desc)
-{
- if (desc) {
- struct at_desc *child;
- unsigned long flags;
-
- spin_lock_irqsave(&atchan->lock, flags);
- list_for_each_entry(child, &desc->tx_list, desc_node)
- dev_vdbg(chan2dev(&atchan->chan_common),
- "moving child desc %p to freelist\n",
- child);
- list_splice_init(&desc->tx_list, &atchan->free_list);
- dev_vdbg(chan2dev(&atchan->chan_common),
- "moving desc %p to freelist\n", desc);
- list_add(&desc->desc_node, &atchan->free_list);
- spin_unlock_irqrestore(&atchan->lock, flags);
- }
-}
-
-/**
- * atc_desc_chain - build chain adding a descriptor
- * @first: address of first descriptor of the chain
- * @prev: address of previous descriptor of the chain
- * @desc: descriptor to queue
- *
- * Called from prep_* functions
- */
-static void atc_desc_chain(struct at_desc **first, struct at_desc **prev,
- struct at_desc *desc)
-{
- if (!(*first)) {
- *first = desc;
- } else {
- /* inform the HW lli about chaining */
- (*prev)->lli.dscr = desc->txd.phys;
- /* insert the link descriptor to the LD ring */
- list_add_tail(&desc->desc_node,
- &(*first)->tx_list);
- }
- *prev = desc;
+ if (i)
+ desc->sg[i - 1].lli->dscr = atdma_sg->lli_phys;
}
/**
* atc_dostart - starts the DMA engine for real
* @atchan: the channel we want to start
- * @first: first descriptor in the list we want to begin with
- *
- * Called with atchan->lock held and bh disabled
*/
-static void atc_dostart(struct at_dma_chan *atchan, struct at_desc *first)
+static void atc_dostart(struct at_dma_chan *atchan)
{
- struct at_dma *atdma = to_at_dma(atchan->chan_common.device);
+ struct virt_dma_desc *vd = vchan_next_desc(&atchan->vc);
+ struct at_desc *desc;
- /* ASSERT: channel is idle */
- if (atc_chan_is_enabled(atchan)) {
- dev_err(chan2dev(&atchan->chan_common),
- "BUG: Attempted to start non-idle channel\n");
- dev_err(chan2dev(&atchan->chan_common),
- " channel: s0x%x d0x%x ctrl0x%x:0x%x l0x%x\n",
- channel_readl(atchan, SADDR),
- channel_readl(atchan, DADDR),
- channel_readl(atchan, CTRLA),
- channel_readl(atchan, CTRLB),
- channel_readl(atchan, DSCR));
-
- /* The tasklet will hopefully advance the queue... */
+ if (!vd) {
+ atchan->desc = NULL;
return;
}
vdbg_dump_regs(atchan);
+ list_del(&vd->node);
+ atchan->desc = desc = to_atdma_desc(&vd->tx);
+
channel_writel(atchan, SADDR, 0);
channel_writel(atchan, DADDR, 0);
channel_writel(atchan, CTRLA, 0);
channel_writel(atchan, CTRLB, 0);
- channel_writel(atchan, DSCR, first->txd.phys);
- channel_writel(atchan, SPIP, ATC_SPIP_HOLE(first->src_hole) |
- ATC_SPIP_BOUNDARY(first->boundary));
- channel_writel(atchan, DPIP, ATC_DPIP_HOLE(first->dst_hole) |
- ATC_DPIP_BOUNDARY(first->boundary));
+ channel_writel(atchan, DSCR, desc->sg[0].lli_phys);
+ channel_writel(atchan, SPIP,
+ FIELD_PREP(ATC_SPIP_HOLE, desc->src_hole) |
+ FIELD_PREP(ATC_SPIP_BOUNDARY, desc->boundary));
+ channel_writel(atchan, DPIP,
+ FIELD_PREP(ATC_DPIP_HOLE, desc->dst_hole) |
+ FIELD_PREP(ATC_DPIP_BOUNDARY, desc->boundary));
+
/* Don't allow CPU to reorder channel enable. */
wmb();
- dma_writel(atdma, CHER, atchan->mask);
+ dma_writel(atchan->atdma, CHER, atchan->mask);
vdbg_dump_regs(atchan);
}
-/*
- * atc_get_desc_by_cookie - get the descriptor of a cookie
- * @atchan: the DMA channel
- * @cookie: the cookie to get the descriptor for
- */
-static struct at_desc *atc_get_desc_by_cookie(struct at_dma_chan *atchan,
- dma_cookie_t cookie)
+static void atdma_desc_free(struct virt_dma_desc *vd)
{
- struct at_desc *desc, *_desc;
+ struct at_dma *atdma = to_at_dma(vd->tx.chan->device);
+ struct at_desc *desc = to_atdma_desc(&vd->tx);
+ unsigned int i;
- list_for_each_entry_safe(desc, _desc, &atchan->queue, desc_node) {
- if (desc->txd.cookie == cookie)
- return desc;
+ for (i = 0; i < desc->sglen; i++) {
+ if (desc->sg[i].lli)
+ dma_pool_free(atdma->lli_pool, desc->sg[i].lli,
+ desc->sg[i].lli_phys);
}
- list_for_each_entry_safe(desc, _desc, &atchan->active_list, desc_node) {
- if (desc->txd.cookie == cookie)
- return desc;
+ /* If the transfer was a memset, free our temporary buffer */
+ if (desc->memset_buffer) {
+ dma_pool_free(atdma->memset_pool, desc->memset_vaddr,
+ desc->memset_paddr);
+ desc->memset_buffer = false;
}
- return NULL;
+ kfree(desc);
}
/**
* @current_len: the number of bytes left before reading CTRLA
* @ctrla: the value of CTRLA
*/
-static inline int atc_calc_bytes_left(int current_len, u32 ctrla)
+static inline u32 atc_calc_bytes_left(u32 current_len, u32 ctrla)
{
- u32 btsize = (ctrla & ATC_BTSIZE_MAX);
- u32 src_width = ATC_REG_TO_SRC_WIDTH(ctrla);
+ u32 btsize = FIELD_GET(ATC_BTSIZE, ctrla);
+ u32 src_width = FIELD_GET(ATC_SRC_WIDTH, ctrla);
/*
* According to the datasheet, when reading the Control A Register
}
/**
- * atc_get_bytes_left - get the number of bytes residue for a cookie
- * @chan: DMA channel
- * @cookie: transaction identifier to check status of
+ * atc_get_llis_residue - Get residue for a hardware linked list transfer
+ *
+ * Calculate the residue by removing the length of the Linked List Item (LLI)
+ * already transferred from the total length. To get the current LLI we can use
+ * the value of the channel's DSCR register and compare it against the DSCR
+ * value of each LLI.
+ *
+ * The CTRLA register provides us with the amount of data already read from the
+ * source for the LLI. So we can compute a more accurate residue by also
+ * removing the number of bytes corresponding to this amount of data.
+ *
+ * However, the DSCR and CTRLA registers cannot be read both atomically. Hence a
+ * race condition may occur: the first read register may refer to one LLI
+ * whereas the second read may refer to a later LLI in the list because of the
+ * DMA transfer progression inbetween the two reads.
+ *
+ * One solution could have been to pause the DMA transfer, read the DSCR and
+ * CTRLA then resume the DMA transfer. Nonetheless, this approach presents some
+ * drawbacks:
+ * - If the DMA transfer is paused, RX overruns or TX underruns are more likey
+ * to occur depending on the system latency. Taking the USART driver as an
+ * example, it uses a cyclic DMA transfer to read data from the Receive
+ * Holding Register (RHR) to avoid RX overruns since the RHR is not protected
+ * by any FIFO on most Atmel SoCs. So pausing the DMA transfer to compute the
+ * residue would break the USART driver design.
+ * - The atc_pause() function masks interrupts but we'd rather avoid to do so
+ * for system latency purpose.
+ *
+ * Then we'd rather use another solution: the DSCR is read a first time, the
+ * CTRLA is read in turn, next the DSCR is read a second time. If the two
+ * consecutive read values of the DSCR are the same then we assume both refers
+ * to the very same LLI as well as the CTRLA value read inbetween does. For
+ * cyclic tranfers, the assumption is that a full loop is "not so fast". If the
+ * two DSCR values are different, we read again the CTRLA then the DSCR till two
+ * consecutive read values from DSCR are equal or till the maximum trials is
+ * reach. This algorithm is very unlikely not to find a stable value for DSCR.
+ * @atchan: pointer to an atmel hdmac channel.
+ * @desc: pointer to the descriptor for which the residue is calculated.
+ * @residue: residue to be set to dma_tx_state.
+ * Returns 0 on success, -errno otherwise.
*/
-static int atc_get_bytes_left(struct dma_chan *chan, dma_cookie_t cookie)
+static int atc_get_llis_residue(struct at_dma_chan *atchan,
+ struct at_desc *desc, u32 *residue)
{
- struct at_dma_chan *atchan = to_at_dma_chan(chan);
- struct at_desc *desc_first = atc_first_active(atchan);
- struct at_desc *desc;
- int ret;
- u32 ctrla, dscr;
+ u32 len, ctrla, dscr;
unsigned int i;
- /*
- * If the cookie doesn't match to the currently running transfer then
- * we can return the total length of the associated DMA transfer,
- * because it is still queued.
- */
- desc = atc_get_desc_by_cookie(atchan, cookie);
- if (desc == NULL)
- return -EINVAL;
- else if (desc != desc_first)
- return desc->total_len;
+ len = desc->total_len;
+ dscr = channel_readl(atchan, DSCR);
+ rmb(); /* ensure DSCR is read before CTRLA */
+ ctrla = channel_readl(atchan, CTRLA);
+ for (i = 0; i < ATC_MAX_DSCR_TRIALS; ++i) {
+ u32 new_dscr;
- /* cookie matches to the currently running transfer */
- ret = desc_first->total_len;
-
- if (desc_first->lli.dscr) {
- /* hardware linked list transfer */
+ rmb(); /* ensure DSCR is read after CTRLA */
+ new_dscr = channel_readl(atchan, DSCR);
/*
- * Calculate the residue by removing the length of the child
- * descriptors already transferred from the total length.
- * To get the current child descriptor we can use the value of
- * the channel's DSCR register and compare it against the value
- * of the hardware linked list structure of each child
- * descriptor.
- *
- * The CTRLA register provides us with the amount of data
- * already read from the source for the current child
- * descriptor. So we can compute a more accurate residue by also
- * removing the number of bytes corresponding to this amount of
- * data.
- *
- * However, the DSCR and CTRLA registers cannot be read both
- * atomically. Hence a race condition may occur: the first read
- * register may refer to one child descriptor whereas the second
- * read may refer to a later child descriptor in the list
- * because of the DMA transfer progression inbetween the two
- * reads.
- *
- * One solution could have been to pause the DMA transfer, read
- * the DSCR and CTRLA then resume the DMA transfer. Nonetheless,
- * this approach presents some drawbacks:
- * - If the DMA transfer is paused, RX overruns or TX underruns
- * are more likey to occur depending on the system latency.
- * Taking the USART driver as an example, it uses a cyclic DMA
- * transfer to read data from the Receive Holding Register
- * (RHR) to avoid RX overruns since the RHR is not protected
- * by any FIFO on most Atmel SoCs. So pausing the DMA transfer
- * to compute the residue would break the USART driver design.
- * - The atc_pause() function masks interrupts but we'd rather
- * avoid to do so for system latency purpose.
- *
- * Then we'd rather use another solution: the DSCR is read a
- * first time, the CTRLA is read in turn, next the DSCR is read
- * a second time. If the two consecutive read values of the DSCR
- * are the same then we assume both refers to the very same
- * child descriptor as well as the CTRLA value read inbetween
- * does. For cyclic tranfers, the assumption is that a full loop
- * is "not so fast".
- * If the two DSCR values are different, we read again the CTRLA
- * then the DSCR till two consecutive read values from DSCR are
- * equal or till the maxium trials is reach.
- * This algorithm is very unlikely not to find a stable value for
- * DSCR.
+ * If the DSCR register value has not changed inside the DMA
+ * controller since the previous read, we assume that both the
+ * dscr and ctrla values refers to the very same descriptor.
*/
-
- dscr = channel_readl(atchan, DSCR);
- rmb(); /* ensure DSCR is read before CTRLA */
- ctrla = channel_readl(atchan, CTRLA);
- for (i = 0; i < ATC_MAX_DSCR_TRIALS; ++i) {
- u32 new_dscr;
-
- rmb(); /* ensure DSCR is read after CTRLA */
- new_dscr = channel_readl(atchan, DSCR);
-
- /*
- * If the DSCR register value has not changed inside the
- * DMA controller since the previous read, we assume
- * that both the dscr and ctrla values refers to the
- * very same descriptor.
- */
- if (likely(new_dscr == dscr))
- break;
-
- /*
- * DSCR has changed inside the DMA controller, so the
- * previouly read value of CTRLA may refer to an already
- * processed descriptor hence could be outdated.
- * We need to update ctrla to match the current
- * descriptor.
- */
- dscr = new_dscr;
- rmb(); /* ensure DSCR is read before CTRLA */
- ctrla = channel_readl(atchan, CTRLA);
- }
- if (unlikely(i == ATC_MAX_DSCR_TRIALS))
- return -ETIMEDOUT;
-
- /* for the first descriptor we can be more accurate */
- if (desc_first->lli.dscr == dscr)
- return atc_calc_bytes_left(ret, ctrla);
-
- ret -= desc_first->len;
- list_for_each_entry(desc, &desc_first->tx_list, desc_node) {
- if (desc->lli.dscr == dscr)
- break;
-
- ret -= desc->len;
- }
+ if (likely(new_dscr == dscr))
+ break;
/*
- * For the current descriptor in the chain we can calculate
- * the remaining bytes using the channel's register.
+ * DSCR has changed inside the DMA controller, so the previouly
+ * read value of CTRLA may refer to an already processed
+ * descriptor hence could be outdated. We need to update ctrla
+ * to match the current descriptor.
*/
- ret = atc_calc_bytes_left(ret, ctrla);
- } else {
- /* single transfer */
+ dscr = new_dscr;
+ rmb(); /* ensure DSCR is read before CTRLA */
ctrla = channel_readl(atchan, CTRLA);
- ret = atc_calc_bytes_left(ret, ctrla);
}
+ if (unlikely(i == ATC_MAX_DSCR_TRIALS))
+ return -ETIMEDOUT;
- return ret;
-}
-
-/**
- * atc_chain_complete - finish work for one transaction chain
- * @atchan: channel we work on
- * @desc: descriptor at the head of the chain we want do complete
- */
-static void
-atc_chain_complete(struct at_dma_chan *atchan, struct at_desc *desc)
-{
- struct dma_async_tx_descriptor *txd = &desc->txd;
- struct at_dma *atdma = to_at_dma(atchan->chan_common.device);
- unsigned long flags;
-
- dev_vdbg(chan2dev(&atchan->chan_common),
- "descriptor %u complete\n", txd->cookie);
-
- spin_lock_irqsave(&atchan->lock, flags);
-
- /* mark the descriptor as complete for non cyclic cases only */
- if (!atc_chan_is_cyclic(atchan))
- dma_cookie_complete(txd);
-
- spin_unlock_irqrestore(&atchan->lock, flags);
-
- dma_descriptor_unmap(txd);
- /* for cyclic transfers,
- * no need to replay callback function while stopping */
- if (!atc_chan_is_cyclic(atchan))
- dmaengine_desc_get_callback_invoke(txd, NULL);
+ /* For the first descriptor we can be more accurate. */
+ if (desc->sg[0].lli->dscr == dscr) {
+ *residue = atc_calc_bytes_left(len, ctrla);
+ return 0;
+ }
+ len -= desc->sg[0].len;
- dma_run_dependencies(txd);
+ for (i = 1; i < desc->sglen; i++) {
+ if (desc->sg[i].lli && desc->sg[i].lli->dscr == dscr)
+ break;
+ len -= desc->sg[i].len;
+ }
- spin_lock_irqsave(&atchan->lock, flags);
- /* move children to free_list */
- list_splice_init(&desc->tx_list, &atchan->free_list);
- /* add myself to free_list */
- list_add(&desc->desc_node, &atchan->free_list);
- spin_unlock_irqrestore(&atchan->lock, flags);
+ /*
+ * For the current LLI in the chain we can calculate the remaining bytes
+ * using the channel's CTRLA register.
+ */
+ *residue = atc_calc_bytes_left(len, ctrla);
+ return 0;
- /* If the transfer was a memset, free our temporary buffer */
- if (desc->memset_buffer) {
- dma_pool_free(atdma->memset_pool, desc->memset_vaddr,
- desc->memset_paddr);
- desc->memset_buffer = false;
- }
}
/**
- * atc_advance_work - at the end of a transaction, move forward
- * @atchan: channel where the transaction ended
+ * atc_get_residue - get the number of bytes residue for a cookie.
+ * The residue is passed by address and updated on success.
+ * @chan: DMA channel
+ * @cookie: transaction identifier to check status of
+ * @residue: residue to be updated.
+ * Return 0 on success, -errono otherwise.
*/
-static void atc_advance_work(struct at_dma_chan *atchan)
+static int atc_get_residue(struct dma_chan *chan, dma_cookie_t cookie,
+ u32 *residue)
{
- struct at_desc *desc;
- unsigned long flags;
+ struct at_dma_chan *atchan = to_at_dma_chan(chan);
+ struct virt_dma_desc *vd;
+ struct at_desc *desc = NULL;
+ u32 len, ctrla;
- dev_vdbg(chan2dev(&atchan->chan_common), "advance_work\n");
+ vd = vchan_find_desc(&atchan->vc, cookie);
+ if (vd)
+ desc = to_atdma_desc(&vd->tx);
+ else if (atchan->desc && atchan->desc->vd.tx.cookie == cookie)
+ desc = atchan->desc;
- spin_lock_irqsave(&atchan->lock, flags);
- if (atc_chan_is_enabled(atchan) || list_empty(&atchan->active_list))
- return spin_unlock_irqrestore(&atchan->lock, flags);
+ if (!desc)
+ return -EINVAL;
- desc = atc_first_active(atchan);
- /* Remove the transfer node from the active list. */
- list_del_init(&desc->desc_node);
- spin_unlock_irqrestore(&atchan->lock, flags);
- atc_chain_complete(atchan, desc);
+ if (desc->sg[0].lli->dscr)
+ /* hardware linked list transfer */
+ return atc_get_llis_residue(atchan, desc, residue);
- /* advance work */
- spin_lock_irqsave(&atchan->lock, flags);
- if (!list_empty(&atchan->active_list)) {
- desc = atc_first_queued(atchan);
- list_move_tail(&desc->desc_node, &atchan->active_list);
- atc_dostart(atchan, desc);
- }
- spin_unlock_irqrestore(&atchan->lock, flags);
+ /* single transfer */
+ len = desc->total_len;
+ ctrla = channel_readl(atchan, CTRLA);
+ *residue = atc_calc_bytes_left(len, ctrla);
+ return 0;
}
-
/**
* atc_handle_error - handle errors reported by DMA controller
- * @atchan: channel where error occurs
+ * @atchan: channel where error occurs.
+ * @i: channel index
*/
-static void atc_handle_error(struct at_dma_chan *atchan)
+static void atc_handle_error(struct at_dma_chan *atchan, unsigned int i)
{
- struct at_desc *bad_desc;
- struct at_desc *desc;
- struct at_desc *child;
- unsigned long flags;
+ struct at_desc *desc = atchan->desc;
- spin_lock_irqsave(&atchan->lock, flags);
- /*
- * The descriptor currently at the head of the active list is
- * broked. Since we don't have any way to report errors, we'll
- * just have to scream loudly and try to carry on.
- */
- bad_desc = atc_first_active(atchan);
- list_del_init(&bad_desc->desc_node);
-
- /* Try to restart the controller */
- if (!list_empty(&atchan->active_list)) {
- desc = atc_first_queued(atchan);
- list_move_tail(&desc->desc_node, &atchan->active_list);
- atc_dostart(atchan, desc);
- }
+ /* Disable channel on AHB error */
+ dma_writel(atchan->atdma, CHDR, AT_DMA_RES(i) | atchan->mask);
/*
* KERN_CRITICAL may seem harsh, but since this only happens
* controller flagged an error instead of scribbling over
* random memory locations.
*/
- dev_crit(chan2dev(&atchan->chan_common),
- "Bad descriptor submitted for DMA!\n");
- dev_crit(chan2dev(&atchan->chan_common),
- " cookie: %d\n", bad_desc->txd.cookie);
- atc_dump_lli(atchan, &bad_desc->lli);
- list_for_each_entry(child, &bad_desc->tx_list, desc_node)
- atc_dump_lli(atchan, &child->lli);
-
- spin_unlock_irqrestore(&atchan->lock, flags);
-
- /* Pretend the descriptor completed successfully */
- atc_chain_complete(atchan, bad_desc);
+ dev_crit(chan2dev(&atchan->vc.chan), "Bad descriptor submitted for DMA!\n");
+ dev_crit(chan2dev(&atchan->vc.chan), "cookie: %d\n",
+ desc->vd.tx.cookie);
+ for (i = 0; i < desc->sglen; i++)
+ atc_dump_lli(atchan, desc->sg[i].lli);
}
-/**
- * atc_handle_cyclic - at the end of a period, run callback function
- * @atchan: channel used for cyclic operations
- */
-static void atc_handle_cyclic(struct at_dma_chan *atchan)
+static void atdma_handle_chan_done(struct at_dma_chan *atchan, u32 pending,
+ unsigned int i)
{
- struct at_desc *first = atc_first_active(atchan);
- struct dma_async_tx_descriptor *txd = &first->txd;
-
- dev_vdbg(chan2dev(&atchan->chan_common),
- "new cyclic period llp 0x%08x\n",
- channel_readl(atchan, DSCR));
-
- dmaengine_desc_get_callback_invoke(txd, NULL);
-}
-
-/*-- IRQ & Tasklet ---------------------------------------------------*/
-
-static void atc_tasklet(struct tasklet_struct *t)
-{
- struct at_dma_chan *atchan = from_tasklet(atchan, t, tasklet);
+ struct at_desc *desc;
- if (test_and_clear_bit(ATC_IS_ERROR, &atchan->status))
- return atc_handle_error(atchan);
+ spin_lock(&atchan->vc.lock);
+ desc = atchan->desc;
- if (atc_chan_is_cyclic(atchan))
- return atc_handle_cyclic(atchan);
+ if (desc) {
+ if (pending & AT_DMA_ERR(i)) {
+ atc_handle_error(atchan, i);
+ /* Pretend the descriptor completed successfully */
+ }
- atc_advance_work(atchan);
+ if (atc_chan_is_cyclic(atchan)) {
+ vchan_cyclic_callback(&desc->vd);
+ } else {
+ vchan_cookie_complete(&desc->vd);
+ atchan->desc = NULL;
+ if (!(atc_chan_is_enabled(atchan)))
+ atc_dostart(atchan);
+ }
+ }
+ spin_unlock(&atchan->vc.lock);
}
static irqreturn_t at_dma_interrupt(int irq, void *dev_id)
{
- struct at_dma *atdma = (struct at_dma *)dev_id;
+ struct at_dma *atdma = dev_id;
struct at_dma_chan *atchan;
int i;
u32 status, pending, imr;
if (!pending)
break;
- dev_vdbg(atdma->dma_common.dev,
+ dev_vdbg(atdma->dma_device.dev,
"interrupt: status = 0x%08x, 0x%08x, 0x%08x\n",
status, imr, pending);
- for (i = 0; i < atdma->dma_common.chancnt; i++) {
+ for (i = 0; i < atdma->dma_device.chancnt; i++) {
atchan = &atdma->chan[i];
- if (pending & (AT_DMA_BTC(i) | AT_DMA_ERR(i))) {
- if (pending & AT_DMA_ERR(i)) {
- /* Disable channel on AHB error */
- dma_writel(atdma, CHDR,
- AT_DMA_RES(i) | atchan->mask);
- /* Give information to tasklet */
- set_bit(ATC_IS_ERROR, &atchan->status);
- }
- tasklet_schedule(&atchan->tasklet);
- ret = IRQ_HANDLED;
- }
+ if (!(pending & (AT_DMA_BTC(i) | AT_DMA_ERR(i))))
+ continue;
+ atdma_handle_chan_done(atchan, pending, i);
+ ret = IRQ_HANDLED;
}
} while (pending);
return ret;
}
-
/*-- DMA Engine API --------------------------------------------------*/
-
-/**
- * atc_tx_submit - set the prepared descriptor(s) to be executed by the engine
- * @tx: descriptor at the head of the transaction chain
- *
- * Queue chain if DMA engine is working already
- *
- * Cookie increment and adding to active_list or queue must be atomic
- */
-static dma_cookie_t atc_tx_submit(struct dma_async_tx_descriptor *tx)
-{
- struct at_desc *desc = txd_to_at_desc(tx);
- struct at_dma_chan *atchan = to_at_dma_chan(tx->chan);
- dma_cookie_t cookie;
- unsigned long flags;
-
- spin_lock_irqsave(&atchan->lock, flags);
- cookie = dma_cookie_assign(tx);
-
- list_add_tail(&desc->desc_node, &atchan->queue);
- spin_unlock_irqrestore(&atchan->lock, flags);
-
- dev_vdbg(chan2dev(tx->chan), "tx_submit: queued %u\n",
- desc->txd.cookie);
- return cookie;
-}
-
/**
* atc_prep_dma_interleaved - prepare memory to memory interleaved operation
* @chan: the channel to prepare operation on
struct dma_interleaved_template *xt,
unsigned long flags)
{
+ struct at_dma *atdma = to_at_dma(chan->device);
struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct data_chunk *first;
- struct at_desc *desc = NULL;
+ struct atdma_sg *atdma_sg;
+ struct at_desc *desc;
+ struct at_lli *lli;
size_t xfer_count;
unsigned int dwidth;
u32 ctrla;
len += chunk->size;
}
- dwidth = atc_get_xfer_width(xt->src_start,
- xt->dst_start, len);
+ dwidth = atc_get_xfer_width(xt->src_start, xt->dst_start, len);
xfer_count = len >> dwidth;
if (xfer_count > ATC_BTSIZE_MAX) {
return NULL;
}
- ctrla = ATC_SRC_WIDTH(dwidth) |
- ATC_DST_WIDTH(dwidth);
-
- ctrlb = ATC_DEFAULT_CTRLB | ATC_IEN
- | ATC_SRC_ADDR_MODE_INCR
- | ATC_DST_ADDR_MODE_INCR
- | ATC_SRC_PIP
- | ATC_DST_PIP
- | ATC_FC_MEM2MEM;
-
- /* create the transfer */
- desc = atc_desc_get(atchan);
- if (!desc) {
- dev_err(chan2dev(chan),
- "%s: couldn't allocate our descriptor\n", __func__);
+ ctrla = FIELD_PREP(ATC_SRC_WIDTH, dwidth) |
+ FIELD_PREP(ATC_DST_WIDTH, dwidth);
+
+ ctrlb = ATC_DEFAULT_CTRLB | ATC_IEN |
+ FIELD_PREP(ATC_SRC_ADDR_MODE, ATC_SRC_ADDR_MODE_INCR) |
+ FIELD_PREP(ATC_DST_ADDR_MODE, ATC_DST_ADDR_MODE_INCR) |
+ ATC_SRC_PIP | ATC_DST_PIP |
+ FIELD_PREP(ATC_FC, ATC_FC_MEM2MEM);
+
+ desc = kzalloc(struct_size(desc, sg, 1), GFP_ATOMIC);
+ if (!desc)
+ return NULL;
+ desc->sglen = 1;
+
+ atdma_sg = desc->sg;
+ atdma_sg->lli = dma_pool_alloc(atdma->lli_pool, GFP_NOWAIT,
+ &atdma_sg->lli_phys);
+ if (!atdma_sg->lli) {
+ kfree(desc);
return NULL;
}
+ lli = atdma_sg->lli;
- desc->lli.saddr = xt->src_start;
- desc->lli.daddr = xt->dst_start;
- desc->lli.ctrla = ctrla | xfer_count;
- desc->lli.ctrlb = ctrlb;
+ lli->saddr = xt->src_start;
+ lli->daddr = xt->dst_start;
+ lli->ctrla = ctrla | xfer_count;
+ lli->ctrlb = ctrlb;
desc->boundary = first->size >> dwidth;
desc->dst_hole = (dmaengine_get_dst_icg(xt, first) >> dwidth) + 1;
desc->src_hole = (dmaengine_get_src_icg(xt, first) >> dwidth) + 1;
- desc->txd.cookie = -EBUSY;
- desc->total_len = desc->len = len;
-
- /* set end-of-link to the last link descriptor of list*/
- set_desc_eol(desc);
-
- desc->txd.flags = flags; /* client is in control of this ack */
+ atdma_sg->len = len;
+ desc->total_len = len;
- return &desc->txd;
+ set_lli_eol(desc, 0);
+ return vchan_tx_prep(&atchan->vc, &desc->vd, flags);
}
/**
atc_prep_dma_memcpy(struct dma_chan *chan, dma_addr_t dest, dma_addr_t src,
size_t len, unsigned long flags)
{
+ struct at_dma *atdma = to_at_dma(chan->device);
struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct at_desc *desc = NULL;
- struct at_desc *first = NULL;
- struct at_desc *prev = NULL;
size_t xfer_count;
size_t offset;
+ size_t sg_len;
unsigned int src_width;
unsigned int dst_width;
+ unsigned int i;
u32 ctrla;
u32 ctrlb;
- dev_vdbg(chan2dev(chan), "prep_dma_memcpy: d%pad s%pad l0x%zx f0x%lx\n",
- &dest, &src, len, flags);
+ dev_dbg(chan2dev(chan), "prep_dma_memcpy: d%pad s%pad l0x%zx f0x%lx\n",
+ &dest, &src, len, flags);
if (unlikely(!len)) {
- dev_dbg(chan2dev(chan), "prep_dma_memcpy: length is zero!\n");
+ dev_err(chan2dev(chan), "prep_dma_memcpy: length is zero!\n");
return NULL;
}
- ctrlb = ATC_DEFAULT_CTRLB | ATC_IEN
- | ATC_SRC_ADDR_MODE_INCR
- | ATC_DST_ADDR_MODE_INCR
- | ATC_FC_MEM2MEM;
+ sg_len = DIV_ROUND_UP(len, ATC_BTSIZE_MAX);
+ desc = kzalloc(struct_size(desc, sg, sg_len), GFP_ATOMIC);
+ if (!desc)
+ return NULL;
+ desc->sglen = sg_len;
+
+ ctrlb = ATC_DEFAULT_CTRLB | ATC_IEN |
+ FIELD_PREP(ATC_SRC_ADDR_MODE, ATC_SRC_ADDR_MODE_INCR) |
+ FIELD_PREP(ATC_DST_ADDR_MODE, ATC_DST_ADDR_MODE_INCR) |
+ FIELD_PREP(ATC_FC, ATC_FC_MEM2MEM);
/*
* We can be a lot more clever here, but this should take care
*/
src_width = dst_width = atc_get_xfer_width(src, dest, len);
- ctrla = ATC_SRC_WIDTH(src_width) |
- ATC_DST_WIDTH(dst_width);
+ ctrla = FIELD_PREP(ATC_SRC_WIDTH, src_width) |
+ FIELD_PREP(ATC_DST_WIDTH, dst_width);
- for (offset = 0; offset < len; offset += xfer_count << src_width) {
- xfer_count = min_t(size_t, (len - offset) >> src_width,
- ATC_BTSIZE_MAX);
+ for (offset = 0, i = 0; offset < len;
+ offset += xfer_count << src_width, i++) {
+ struct atdma_sg *atdma_sg = &desc->sg[i];
+ struct at_lli *lli;
- desc = atc_desc_get(atchan);
- if (!desc)
+ atdma_sg->lli = dma_pool_alloc(atdma->lli_pool, GFP_NOWAIT,
+ &atdma_sg->lli_phys);
+ if (!atdma_sg->lli)
goto err_desc_get;
+ lli = atdma_sg->lli;
+
+ xfer_count = min_t(size_t, (len - offset) >> src_width,
+ ATC_BTSIZE_MAX);
- desc->lli.saddr = src + offset;
- desc->lli.daddr = dest + offset;
- desc->lli.ctrla = ctrla | xfer_count;
- desc->lli.ctrlb = ctrlb;
+ lli->saddr = src + offset;
+ lli->daddr = dest + offset;
+ lli->ctrla = ctrla | xfer_count;
+ lli->ctrlb = ctrlb;
- desc->txd.cookie = 0;
- desc->len = xfer_count << src_width;
+ desc->sg[i].len = xfer_count << src_width;
- atc_desc_chain(&first, &prev, desc);
+ atdma_lli_chain(desc, i);
}
- /* First descriptor of the chain embedds additional information */
- first->txd.cookie = -EBUSY;
- first->total_len = len;
+ desc->total_len = len;
/* set end-of-link to the last link descriptor of list*/
- set_desc_eol(desc);
-
- first->txd.flags = flags; /* client is in control of this ack */
+ set_lli_eol(desc, i - 1);
- return &first->txd;
+ return vchan_tx_prep(&atchan->vc, &desc->vd, flags);
err_desc_get:
- atc_desc_put(atchan, first);
+ atdma_desc_free(&desc->vd);
return NULL;
}
-static struct at_desc *atc_create_memset_desc(struct dma_chan *chan,
- dma_addr_t psrc,
- dma_addr_t pdst,
- size_t len)
+static int atdma_create_memset_lli(struct dma_chan *chan,
+ struct atdma_sg *atdma_sg,
+ dma_addr_t psrc, dma_addr_t pdst, size_t len)
{
- struct at_dma_chan *atchan = to_at_dma_chan(chan);
- struct at_desc *desc;
+ struct at_dma *atdma = to_at_dma(chan->device);
+ struct at_lli *lli;
size_t xfer_count;
-
- u32 ctrla = ATC_SRC_WIDTH(2) | ATC_DST_WIDTH(2);
+ u32 ctrla = FIELD_PREP(ATC_SRC_WIDTH, 2) | FIELD_PREP(ATC_DST_WIDTH, 2);
u32 ctrlb = ATC_DEFAULT_CTRLB | ATC_IEN |
- ATC_SRC_ADDR_MODE_FIXED |
- ATC_DST_ADDR_MODE_INCR |
- ATC_FC_MEM2MEM;
+ FIELD_PREP(ATC_SRC_ADDR_MODE, ATC_SRC_ADDR_MODE_FIXED) |
+ FIELD_PREP(ATC_DST_ADDR_MODE, ATC_DST_ADDR_MODE_INCR) |
+ FIELD_PREP(ATC_FC, ATC_FC_MEM2MEM);
xfer_count = len >> 2;
if (xfer_count > ATC_BTSIZE_MAX) {
- dev_err(chan2dev(chan), "%s: buffer is too big\n",
- __func__);
- return NULL;
+ dev_err(chan2dev(chan), "%s: buffer is too big\n", __func__);
+ return -EINVAL;
}
- desc = atc_desc_get(atchan);
- if (!desc) {
- dev_err(chan2dev(chan), "%s: can't get a descriptor\n",
- __func__);
- return NULL;
- }
+ atdma_sg->lli = dma_pool_alloc(atdma->lli_pool, GFP_NOWAIT,
+ &atdma_sg->lli_phys);
+ if (!atdma_sg->lli)
+ return -ENOMEM;
+ lli = atdma_sg->lli;
- desc->lli.saddr = psrc;
- desc->lli.daddr = pdst;
- desc->lli.ctrla = ctrla | xfer_count;
- desc->lli.ctrlb = ctrlb;
+ lli->saddr = psrc;
+ lli->daddr = pdst;
+ lli->ctrla = ctrla | xfer_count;
+ lli->ctrlb = ctrlb;
- desc->txd.cookie = 0;
- desc->len = len;
+ atdma_sg->len = len;
- return desc;
+ return 0;
}
/**
atc_prep_dma_memset(struct dma_chan *chan, dma_addr_t dest, int value,
size_t len, unsigned long flags)
{
+ struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct at_dma *atdma = to_at_dma(chan->device);
struct at_desc *desc;
void __iomem *vaddr;
dma_addr_t paddr;
char fill_pattern;
+ int ret;
dev_vdbg(chan2dev(chan), "%s: d%pad v0x%x l0x%zx f0x%lx\n", __func__,
&dest, value, len, flags);
(fill_pattern << 8) |
fill_pattern;
- desc = atc_create_memset_desc(chan, paddr, dest, len);
- if (!desc) {
- dev_err(chan2dev(chan), "%s: couldn't get a descriptor\n",
- __func__);
+ desc = kzalloc(struct_size(desc, sg, 1), GFP_ATOMIC);
+ if (!desc)
goto err_free_buffer;
- }
+ desc->sglen = 1;
+
+ ret = atdma_create_memset_lli(chan, desc->sg, paddr, dest, len);
+ if (ret)
+ goto err_free_desc;
desc->memset_paddr = paddr;
desc->memset_vaddr = vaddr;
desc->memset_buffer = true;
- desc->txd.cookie = -EBUSY;
desc->total_len = len;
/* set end-of-link on the descriptor */
- set_desc_eol(desc);
-
- desc->txd.flags = flags;
+ set_lli_eol(desc, 0);
- return &desc->txd;
+ return vchan_tx_prep(&atchan->vc, &desc->vd, flags);
+err_free_desc:
+ kfree(desc);
err_free_buffer:
dma_pool_free(atdma->memset_pool, vaddr, paddr);
return NULL;
{
struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct at_dma *atdma = to_at_dma(chan->device);
- struct at_desc *desc = NULL, *first = NULL, *prev = NULL;
+ struct at_desc *desc;
struct scatterlist *sg;
void __iomem *vaddr;
dma_addr_t paddr;
size_t total_len = 0;
int i;
+ int ret;
dev_vdbg(chan2dev(chan), "%s: v0x%x l0x%zx f0x%lx\n", __func__,
value, sg_len, flags);
}
*(u32*)vaddr = value;
+ desc = kzalloc(struct_size(desc, sg, sg_len), GFP_ATOMIC);
+ if (!desc)
+ goto err_free_dma_buf;
+ desc->sglen = sg_len;
+
for_each_sg(sgl, sg, sg_len, i) {
dma_addr_t dest = sg_dma_address(sg);
size_t len = sg_dma_len(sg);
if (!is_dma_fill_aligned(chan->device, dest, 0, len)) {
dev_err(chan2dev(chan), "%s: buffer is not aligned\n",
__func__);
- goto err_put_desc;
+ goto err_free_desc;
}
- desc = atc_create_memset_desc(chan, paddr, dest, len);
- if (!desc)
- goto err_put_desc;
-
- atc_desc_chain(&first, &prev, desc);
+ ret = atdma_create_memset_lli(chan, &desc->sg[i], paddr, dest,
+ len);
+ if (ret)
+ goto err_free_desc;
+ atdma_lli_chain(desc, i);
total_len += len;
}
- /*
- * Only set the buffer pointers on the last descriptor to
- * avoid free'ing while we have our transfer still going
- */
desc->memset_paddr = paddr;
desc->memset_vaddr = vaddr;
desc->memset_buffer = true;
- first->txd.cookie = -EBUSY;
- first->total_len = total_len;
+ desc->total_len = total_len;
/* set end-of-link on the descriptor */
- set_desc_eol(desc);
-
- first->txd.flags = flags;
+ set_lli_eol(desc, i - 1);
- return &first->txd;
+ return vchan_tx_prep(&atchan->vc, &desc->vd, flags);
-err_put_desc:
- atc_desc_put(atchan, first);
+err_free_desc:
+ atdma_desc_free(&desc->vd);
+err_free_dma_buf:
+ dma_pool_free(atdma->memset_pool, vaddr, paddr);
return NULL;
}
unsigned int sg_len, enum dma_transfer_direction direction,
unsigned long flags, void *context)
{
+ struct at_dma *atdma = to_at_dma(chan->device);
struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct at_dma_slave *atslave = chan->private;
struct dma_slave_config *sconfig = &atchan->dma_sconfig;
- struct at_desc *first = NULL;
- struct at_desc *prev = NULL;
+ struct at_desc *desc;
u32 ctrla;
u32 ctrlb;
dma_addr_t reg;
return NULL;
}
- ctrla = ATC_SCSIZE(sconfig->src_maxburst)
- | ATC_DCSIZE(sconfig->dst_maxburst);
+ desc = kzalloc(struct_size(desc, sg, sg_len), GFP_ATOMIC);
+ if (!desc)
+ return NULL;
+ desc->sglen = sg_len;
+
+ ctrla = FIELD_PREP(ATC_SCSIZE, sconfig->src_maxburst) |
+ FIELD_PREP(ATC_DCSIZE, sconfig->dst_maxburst);
ctrlb = ATC_IEN;
switch (direction) {
case DMA_MEM_TO_DEV:
reg_width = convert_buswidth(sconfig->dst_addr_width);
- ctrla |= ATC_DST_WIDTH(reg_width);
- ctrlb |= ATC_DST_ADDR_MODE_FIXED
- | ATC_SRC_ADDR_MODE_INCR
- | ATC_FC_MEM2PER
- | ATC_SIF(atchan->mem_if) | ATC_DIF(atchan->per_if);
+ ctrla |= FIELD_PREP(ATC_DST_WIDTH, reg_width);
+ ctrlb |= FIELD_PREP(ATC_DST_ADDR_MODE,
+ ATC_DST_ADDR_MODE_FIXED) |
+ FIELD_PREP(ATC_SRC_ADDR_MODE, ATC_SRC_ADDR_MODE_INCR) |
+ FIELD_PREP(ATC_FC, ATC_FC_MEM2PER) |
+ FIELD_PREP(ATC_SIF, atchan->mem_if) |
+ FIELD_PREP(ATC_DIF, atchan->per_if);
reg = sconfig->dst_addr;
for_each_sg(sgl, sg, sg_len, i) {
- struct at_desc *desc;
+ struct atdma_sg *atdma_sg = &desc->sg[i];
+ struct at_lli *lli;
u32 len;
u32 mem;
- desc = atc_desc_get(atchan);
- if (!desc)
+ atdma_sg->lli = dma_pool_alloc(atdma->lli_pool,
+ GFP_NOWAIT,
+ &atdma_sg->lli_phys);
+ if (!atdma_sg->lli)
goto err_desc_get;
+ lli = atdma_sg->lli;
mem = sg_dma_address(sg);
len = sg_dma_len(sg);
if (unlikely(mem & 3 || len & 3))
mem_width = 0;
- desc->lli.saddr = mem;
- desc->lli.daddr = reg;
- desc->lli.ctrla = ctrla
- | ATC_SRC_WIDTH(mem_width)
- | len >> mem_width;
- desc->lli.ctrlb = ctrlb;
- desc->len = len;
+ lli->saddr = mem;
+ lli->daddr = reg;
+ lli->ctrla = ctrla |
+ FIELD_PREP(ATC_SRC_WIDTH, mem_width) |
+ len >> mem_width;
+ lli->ctrlb = ctrlb;
- atc_desc_chain(&first, &prev, desc);
+ atdma_sg->len = len;
total_len += len;
+
+ desc->sg[i].len = len;
+ atdma_lli_chain(desc, i);
}
break;
case DMA_DEV_TO_MEM:
reg_width = convert_buswidth(sconfig->src_addr_width);
- ctrla |= ATC_SRC_WIDTH(reg_width);
- ctrlb |= ATC_DST_ADDR_MODE_INCR
- | ATC_SRC_ADDR_MODE_FIXED
- | ATC_FC_PER2MEM
- | ATC_SIF(atchan->per_if) | ATC_DIF(atchan->mem_if);
+ ctrla |= FIELD_PREP(ATC_SRC_WIDTH, reg_width);
+ ctrlb |= FIELD_PREP(ATC_DST_ADDR_MODE, ATC_DST_ADDR_MODE_INCR) |
+ FIELD_PREP(ATC_SRC_ADDR_MODE,
+ ATC_SRC_ADDR_MODE_FIXED) |
+ FIELD_PREP(ATC_FC, ATC_FC_PER2MEM) |
+ FIELD_PREP(ATC_SIF, atchan->per_if) |
+ FIELD_PREP(ATC_DIF, atchan->mem_if);
reg = sconfig->src_addr;
for_each_sg(sgl, sg, sg_len, i) {
- struct at_desc *desc;
+ struct atdma_sg *atdma_sg = &desc->sg[i];
+ struct at_lli *lli;
u32 len;
u32 mem;
- desc = atc_desc_get(atchan);
- if (!desc)
+ atdma_sg->lli = dma_pool_alloc(atdma->lli_pool,
+ GFP_NOWAIT,
+ &atdma_sg->lli_phys);
+ if (!atdma_sg->lli)
goto err_desc_get;
+ lli = atdma_sg->lli;
mem = sg_dma_address(sg);
len = sg_dma_len(sg);
if (unlikely(mem & 3 || len & 3))
mem_width = 0;
- desc->lli.saddr = reg;
- desc->lli.daddr = mem;
- desc->lli.ctrla = ctrla
- | ATC_DST_WIDTH(mem_width)
- | len >> reg_width;
- desc->lli.ctrlb = ctrlb;
- desc->len = len;
+ lli->saddr = reg;
+ lli->daddr = mem;
+ lli->ctrla = ctrla |
+ FIELD_PREP(ATC_DST_WIDTH, mem_width) |
+ len >> reg_width;
+ lli->ctrlb = ctrlb;
- atc_desc_chain(&first, &prev, desc);
+ desc->sg[i].len = len;
total_len += len;
+
+ atdma_lli_chain(desc, i);
}
break;
default:
}
/* set end-of-link to the last link descriptor of list*/
- set_desc_eol(prev);
+ set_lli_eol(desc, i - 1);
- /* First descriptor of the chain embedds additional information */
- first->txd.cookie = -EBUSY;
- first->total_len = total_len;
+ desc->total_len = total_len;
- /* first link descriptor of list is responsible of flags */
- first->txd.flags = flags; /* client is in control of this ack */
-
- return &first->txd;
+ return vchan_tx_prep(&atchan->vc, &desc->vd, flags);
err_desc_get:
dev_err(chan2dev(chan), "not enough descriptors available\n");
err:
- atc_desc_put(atchan, first);
+ atdma_desc_free(&desc->vd);
return NULL;
}
*/
static int
atc_dma_cyclic_fill_desc(struct dma_chan *chan, struct at_desc *desc,
- unsigned int period_index, dma_addr_t buf_addr,
+ unsigned int i, dma_addr_t buf_addr,
unsigned int reg_width, size_t period_len,
enum dma_transfer_direction direction)
{
+ struct at_dma *atdma = to_at_dma(chan->device);
struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct dma_slave_config *sconfig = &atchan->dma_sconfig;
- u32 ctrla;
+ struct atdma_sg *atdma_sg = &desc->sg[i];
+ struct at_lli *lli;
- /* prepare common CRTLA value */
- ctrla = ATC_SCSIZE(sconfig->src_maxburst)
- | ATC_DCSIZE(sconfig->dst_maxburst)
- | ATC_DST_WIDTH(reg_width)
- | ATC_SRC_WIDTH(reg_width)
- | period_len >> reg_width;
+ atdma_sg->lli = dma_pool_alloc(atdma->lli_pool, GFP_ATOMIC,
+ &atdma_sg->lli_phys);
+ if (!atdma_sg->lli)
+ return -ENOMEM;
+ lli = atdma_sg->lli;
switch (direction) {
case DMA_MEM_TO_DEV:
- desc->lli.saddr = buf_addr + (period_len * period_index);
- desc->lli.daddr = sconfig->dst_addr;
- desc->lli.ctrla = ctrla;
- desc->lli.ctrlb = ATC_DST_ADDR_MODE_FIXED
- | ATC_SRC_ADDR_MODE_INCR
- | ATC_FC_MEM2PER
- | ATC_SIF(atchan->mem_if)
- | ATC_DIF(atchan->per_if);
- desc->len = period_len;
+ lli->saddr = buf_addr + (period_len * i);
+ lli->daddr = sconfig->dst_addr;
+ lli->ctrlb = FIELD_PREP(ATC_DST_ADDR_MODE,
+ ATC_DST_ADDR_MODE_FIXED) |
+ FIELD_PREP(ATC_SRC_ADDR_MODE,
+ ATC_SRC_ADDR_MODE_INCR) |
+ FIELD_PREP(ATC_FC, ATC_FC_MEM2PER) |
+ FIELD_PREP(ATC_SIF, atchan->mem_if) |
+ FIELD_PREP(ATC_DIF, atchan->per_if);
+
break;
case DMA_DEV_TO_MEM:
- desc->lli.saddr = sconfig->src_addr;
- desc->lli.daddr = buf_addr + (period_len * period_index);
- desc->lli.ctrla = ctrla;
- desc->lli.ctrlb = ATC_DST_ADDR_MODE_INCR
- | ATC_SRC_ADDR_MODE_FIXED
- | ATC_FC_PER2MEM
- | ATC_SIF(atchan->per_if)
- | ATC_DIF(atchan->mem_if);
- desc->len = period_len;
+ lli->saddr = sconfig->src_addr;
+ lli->daddr = buf_addr + (period_len * i);
+ lli->ctrlb = FIELD_PREP(ATC_DST_ADDR_MODE,
+ ATC_DST_ADDR_MODE_INCR) |
+ FIELD_PREP(ATC_SRC_ADDR_MODE,
+ ATC_SRC_ADDR_MODE_FIXED) |
+ FIELD_PREP(ATC_FC, ATC_FC_PER2MEM) |
+ FIELD_PREP(ATC_SIF, atchan->per_if) |
+ FIELD_PREP(ATC_DIF, atchan->mem_if);
break;
default:
return -EINVAL;
}
+ lli->ctrla = FIELD_PREP(ATC_SCSIZE, sconfig->src_maxburst) |
+ FIELD_PREP(ATC_DCSIZE, sconfig->dst_maxburst) |
+ FIELD_PREP(ATC_DST_WIDTH, reg_width) |
+ FIELD_PREP(ATC_SRC_WIDTH, reg_width) |
+ period_len >> reg_width;
+ desc->sg[i].len = period_len;
+
return 0;
}
struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct at_dma_slave *atslave = chan->private;
struct dma_slave_config *sconfig = &atchan->dma_sconfig;
- struct at_desc *first = NULL;
- struct at_desc *prev = NULL;
+ struct at_desc *desc;
unsigned long was_cyclic;
unsigned int reg_width;
unsigned int periods = buf_len / period_len;
if (atc_dma_cyclic_check_values(reg_width, buf_addr, period_len))
goto err_out;
+ desc = kzalloc(struct_size(desc, sg, periods), GFP_ATOMIC);
+ if (!desc)
+ goto err_out;
+ desc->sglen = periods;
+
/* build cyclic linked list */
for (i = 0; i < periods; i++) {
- struct at_desc *desc;
-
- desc = atc_desc_get(atchan);
- if (!desc)
- goto err_desc_get;
-
if (atc_dma_cyclic_fill_desc(chan, desc, i, buf_addr,
reg_width, period_len, direction))
- goto err_desc_get;
-
- atc_desc_chain(&first, &prev, desc);
+ goto err_fill_desc;
+ atdma_lli_chain(desc, i);
}
-
+ desc->total_len = buf_len;
/* lets make a cyclic list */
- prev->lli.dscr = first->txd.phys;
+ desc->sg[i - 1].lli->dscr = desc->sg[0].lli_phys;
- /* First descriptor of the chain embedds additional information */
- first->txd.cookie = -EBUSY;
- first->total_len = buf_len;
+ return vchan_tx_prep(&atchan->vc, &desc->vd, flags);
- return &first->txd;
-
-err_desc_get:
- dev_err(chan2dev(chan), "not enough descriptors available\n");
- atc_desc_put(atchan, first);
+err_fill_desc:
+ atdma_desc_free(&desc->vd);
err_out:
clear_bit(ATC_IS_CYCLIC, &atchan->status);
return NULL;
{
struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct at_dma *atdma = to_at_dma(chan->device);
- int chan_id = atchan->chan_common.chan_id;
+ int chan_id = atchan->vc.chan.chan_id;
unsigned long flags;
dev_vdbg(chan2dev(chan), "%s\n", __func__);
- spin_lock_irqsave(&atchan->lock, flags);
+ spin_lock_irqsave(&atchan->vc.lock, flags);
dma_writel(atdma, CHER, AT_DMA_SUSP(chan_id));
set_bit(ATC_IS_PAUSED, &atchan->status);
- spin_unlock_irqrestore(&atchan->lock, flags);
+ spin_unlock_irqrestore(&atchan->vc.lock, flags);
return 0;
}
{
struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct at_dma *atdma = to_at_dma(chan->device);
- int chan_id = atchan->chan_common.chan_id;
+ int chan_id = atchan->vc.chan.chan_id;
unsigned long flags;
dev_vdbg(chan2dev(chan), "%s\n", __func__);
if (!atc_chan_is_paused(atchan))
return 0;
- spin_lock_irqsave(&atchan->lock, flags);
+ spin_lock_irqsave(&atchan->vc.lock, flags);
dma_writel(atdma, CHDR, AT_DMA_RES(chan_id));
clear_bit(ATC_IS_PAUSED, &atchan->status);
- spin_unlock_irqrestore(&atchan->lock, flags);
+ spin_unlock_irqrestore(&atchan->vc.lock, flags);
return 0;
}
{
struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct at_dma *atdma = to_at_dma(chan->device);
- int chan_id = atchan->chan_common.chan_id;
+ int chan_id = atchan->vc.chan.chan_id;
unsigned long flags;
+ LIST_HEAD(list);
+
dev_vdbg(chan2dev(chan), "%s\n", __func__);
/*
* channel. We still have to poll the channel enable bit due
* to AHB/HSB limitations.
*/
- spin_lock_irqsave(&atchan->lock, flags);
+ spin_lock_irqsave(&atchan->vc.lock, flags);
/* disabling channel: must also remove suspend state */
dma_writel(atdma, CHDR, AT_DMA_RES(chan_id) | atchan->mask);
while (dma_readl(atdma, CHSR) & atchan->mask)
cpu_relax();
- /* active_list entries will end up before queued entries */
- list_splice_tail_init(&atchan->queue, &atchan->free_list);
- list_splice_tail_init(&atchan->active_list, &atchan->free_list);
+ if (atchan->desc) {
+ vchan_terminate_vdesc(&atchan->desc->vd);
+ atchan->desc = NULL;
+ }
+
+ vchan_get_all_descriptors(&atchan->vc, &list);
clear_bit(ATC_IS_PAUSED, &atchan->status);
/* if channel dedicated to cyclic operations, free it */
clear_bit(ATC_IS_CYCLIC, &atchan->status);
- spin_unlock_irqrestore(&atchan->lock, flags);
+ spin_unlock_irqrestore(&atchan->vc.lock, flags);
+
+ vchan_dma_desc_free_list(&atchan->vc, &list);
return 0;
}
{
struct at_dma_chan *atchan = to_at_dma_chan(chan);
unsigned long flags;
- enum dma_status ret;
- int bytes = 0;
-
- ret = dma_cookie_status(chan, cookie, txstate);
- if (ret == DMA_COMPLETE)
- return ret;
- /*
- * There's no point calculating the residue if there's
- * no txstate to store the value.
- */
- if (!txstate)
- return DMA_ERROR;
+ enum dma_status dma_status;
+ u32 residue;
+ int ret;
- spin_lock_irqsave(&atchan->lock, flags);
+ dma_status = dma_cookie_status(chan, cookie, txstate);
+ if (dma_status == DMA_COMPLETE || !txstate)
+ return dma_status;
+ spin_lock_irqsave(&atchan->vc.lock, flags);
/* Get number of bytes left in the active transactions */
- bytes = atc_get_bytes_left(chan, cookie);
-
- spin_unlock_irqrestore(&atchan->lock, flags);
+ ret = atc_get_residue(chan, cookie, &residue);
+ spin_unlock_irqrestore(&atchan->vc.lock, flags);
- if (unlikely(bytes < 0)) {
+ if (unlikely(ret < 0)) {
dev_vdbg(chan2dev(chan), "get residual bytes error\n");
return DMA_ERROR;
} else {
- dma_set_residue(txstate, bytes);
+ dma_set_residue(txstate, residue);
}
- dev_vdbg(chan2dev(chan), "tx_status %d: cookie = %d residue = %d\n",
- ret, cookie, bytes);
+ dev_vdbg(chan2dev(chan), "tx_status %d: cookie = %d residue = %u\n",
+ dma_status, cookie, residue);
- return ret;
+ return dma_status;
}
-/**
- * atc_issue_pending - takes the first transaction descriptor in the pending
- * queue and starts the transfer.
- * @chan: target DMA channel
- */
static void atc_issue_pending(struct dma_chan *chan)
{
struct at_dma_chan *atchan = to_at_dma_chan(chan);
- struct at_desc *desc;
unsigned long flags;
- dev_vdbg(chan2dev(chan), "issue_pending\n");
-
- spin_lock_irqsave(&atchan->lock, flags);
- if (atc_chan_is_enabled(atchan) || list_empty(&atchan->queue))
- return spin_unlock_irqrestore(&atchan->lock, flags);
-
- desc = atc_first_queued(atchan);
- list_move_tail(&desc->desc_node, &atchan->active_list);
- atc_dostart(atchan, desc);
- spin_unlock_irqrestore(&atchan->lock, flags);
+ spin_lock_irqsave(&atchan->vc.lock, flags);
+ if (vchan_issue_pending(&atchan->vc) && !atchan->desc) {
+ if (!(atc_chan_is_enabled(atchan)))
+ atc_dostart(atchan);
+ }
+ spin_unlock_irqrestore(&atchan->vc.lock, flags);
}
/**
{
struct at_dma_chan *atchan = to_at_dma_chan(chan);
struct at_dma *atdma = to_at_dma(chan->device);
- struct at_desc *desc;
struct at_dma_slave *atslave;
- int i;
u32 cfg;
dev_vdbg(chan2dev(chan), "alloc_chan_resources\n");
return -EIO;
}
- if (!list_empty(&atchan->free_list)) {
- dev_dbg(chan2dev(chan), "can't allocate channel resources (channel not freed from a previous use)\n");
- return -EIO;
- }
-
cfg = ATC_DEFAULT_CFG;
atslave = chan->private;
* We need controller-specific data to set up slave
* transfers.
*/
- BUG_ON(!atslave->dma_dev || atslave->dma_dev != atdma->dma_common.dev);
+ BUG_ON(!atslave->dma_dev || atslave->dma_dev != atdma->dma_device.dev);
/* if cfg configuration specified take it instead of default */
if (atslave->cfg)
cfg = atslave->cfg;
}
- /* Allocate initial pool of descriptors */
- for (i = 0; i < init_nr_desc_per_channel; i++) {
- desc = atc_alloc_descriptor(chan, GFP_KERNEL);
- if (!desc) {
- dev_err(atdma->dma_common.dev,
- "Only %d initial descriptors\n", i);
- break;
- }
- list_add_tail(&desc->desc_node, &atchan->free_list);
- }
-
- dma_cookie_init(chan);
-
/* channel parameters */
channel_writel(atchan, CFG, cfg);
- dev_dbg(chan2dev(chan),
- "alloc_chan_resources: allocated %d descriptors\n", i);
-
- return i;
+ return 0;
}
/**
static void atc_free_chan_resources(struct dma_chan *chan)
{
struct at_dma_chan *atchan = to_at_dma_chan(chan);
- struct at_dma *atdma = to_at_dma(chan->device);
- struct at_desc *desc, *_desc;
- LIST_HEAD(list);
- /* ASSERT: channel is idle */
- BUG_ON(!list_empty(&atchan->active_list));
- BUG_ON(!list_empty(&atchan->queue));
BUG_ON(atc_chan_is_enabled(atchan));
- list_for_each_entry_safe(desc, _desc, &atchan->free_list, desc_node) {
- dev_vdbg(chan2dev(chan), " freeing descriptor %p\n", desc);
- list_del(&desc->desc_node);
- /* free link descriptor */
- dma_pool_free(atdma->dma_desc_pool, desc, desc->txd.phys);
- }
- list_splice_init(&atchan->free_list, &list);
+ vchan_free_chan_resources(to_virt_chan(chan));
atchan->status = 0;
/*
return NULL;
}
- atslave->cfg = ATC_DST_H2SEL_HW | ATC_SRC_H2SEL_HW;
+ atslave->cfg = ATC_DST_H2SEL | ATC_SRC_H2SEL;
/*
* We can fill both SRC_PER and DST_PER, one of these fields will be
* ignored depending on DMA transfer direction.
*/
per_id = dma_spec->args[1] & AT91_DMA_CFG_PER_ID_MASK;
- atslave->cfg |= ATC_DST_PER_MSB(per_id) | ATC_DST_PER(per_id)
- | ATC_SRC_PER_MSB(per_id) | ATC_SRC_PER(per_id);
+ atslave->cfg |= ATC_DST_PER_ID(per_id) | ATC_SRC_PER_ID(per_id);
/*
* We have to translate the value we get from the device tree since
* the half FIFO configuration value had to be 0 to keep backward
*/
switch (dma_spec->args[1] & AT91_DMA_CFG_FIFOCFG_MASK) {
case AT91_DMA_CFG_FIFOCFG_ALAP:
- atslave->cfg |= ATC_FIFOCFG_LARGESTBURST;
+ atslave->cfg |= FIELD_PREP(ATC_FIFOCFG,
+ ATC_FIFOCFG_LARGESTBURST);
break;
case AT91_DMA_CFG_FIFOCFG_ASAP:
- atslave->cfg |= ATC_FIFOCFG_ENOUGHSPACE;
+ atslave->cfg |= FIELD_PREP(ATC_FIFOCFG,
+ ATC_FIFOCFG_ENOUGHSPACE);
break;
case AT91_DMA_CFG_FIFOCFG_HALF:
default:
- atslave->cfg |= ATC_FIFOCFG_HALFFIFO;
+ atslave->cfg |= FIELD_PREP(ATC_FIFOCFG, ATC_FIFOCFG_HALFFIFO);
}
atslave->dma_dev = &dmac_pdev->dev;
static int __init at_dma_probe(struct platform_device *pdev)
{
- struct resource *io;
struct at_dma *atdma;
- size_t size;
int irq;
int err;
int i;
if (!plat_dat)
return -ENODEV;
- io = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!io)
- return -EINVAL;
+ atdma = devm_kzalloc(&pdev->dev,
+ struct_size(atdma, chan, plat_dat->nr_channels),
+ GFP_KERNEL);
+ if (!atdma)
+ return -ENOMEM;
+
+ atdma->regs = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(atdma->regs))
+ return PTR_ERR(atdma->regs);
irq = platform_get_irq(pdev, 0);
if (irq < 0)
return irq;
- size = sizeof(struct at_dma);
- size += plat_dat->nr_channels * sizeof(struct at_dma_chan);
- atdma = kzalloc(size, GFP_KERNEL);
- if (!atdma)
- return -ENOMEM;
-
/* discover transaction capabilities */
- atdma->dma_common.cap_mask = plat_dat->cap_mask;
+ atdma->dma_device.cap_mask = plat_dat->cap_mask;
atdma->all_chan_mask = (1 << plat_dat->nr_channels) - 1;
- size = resource_size(io);
- if (!request_mem_region(io->start, size, pdev->dev.driver->name)) {
- err = -EBUSY;
- goto err_kfree;
- }
-
- atdma->regs = ioremap(io->start, size);
- if (!atdma->regs) {
- err = -ENOMEM;
- goto err_release_r;
- }
+ atdma->clk = devm_clk_get(&pdev->dev, "dma_clk");
+ if (IS_ERR(atdma->clk))
+ return PTR_ERR(atdma->clk);
- atdma->clk = clk_get(&pdev->dev, "dma_clk");
- if (IS_ERR(atdma->clk)) {
- err = PTR_ERR(atdma->clk);
- goto err_clk;
- }
err = clk_prepare_enable(atdma->clk);
if (err)
- goto err_clk_prepare;
+ return err;
/* force dma off, just in case */
at_dma_off(atdma);
platform_set_drvdata(pdev, atdma);
/* create a pool of consistent memory blocks for hardware descriptors */
- atdma->dma_desc_pool = dma_pool_create("at_hdmac_desc_pool",
- &pdev->dev, sizeof(struct at_desc),
- 4 /* word alignment */, 0);
- if (!atdma->dma_desc_pool) {
- dev_err(&pdev->dev, "No memory for descriptors dma pool\n");
+ atdma->lli_pool = dma_pool_create("at_hdmac_lli_pool",
+ &pdev->dev, sizeof(struct at_lli),
+ 4 /* word alignment */, 0);
+ if (!atdma->lli_pool) {
+ dev_err(&pdev->dev, "Unable to allocate DMA LLI descriptor pool\n");
err = -ENOMEM;
goto err_desc_pool_create;
}
cpu_relax();
/* initialize channels related values */
- INIT_LIST_HEAD(&atdma->dma_common.channels);
+ INIT_LIST_HEAD(&atdma->dma_device.channels);
for (i = 0; i < plat_dat->nr_channels; i++) {
struct at_dma_chan *atchan = &atdma->chan[i];
atchan->mem_if = AT_DMA_MEM_IF;
atchan->per_if = AT_DMA_PER_IF;
- atchan->chan_common.device = &atdma->dma_common;
- dma_cookie_init(&atchan->chan_common);
- list_add_tail(&atchan->chan_common.device_node,
- &atdma->dma_common.channels);
atchan->ch_regs = atdma->regs + ch_regs(i);
- spin_lock_init(&atchan->lock);
atchan->mask = 1 << i;
- INIT_LIST_HEAD(&atchan->active_list);
- INIT_LIST_HEAD(&atchan->queue);
- INIT_LIST_HEAD(&atchan->free_list);
-
- tasklet_setup(&atchan->tasklet, atc_tasklet);
+ atchan->atdma = atdma;
+ atchan->vc.desc_free = atdma_desc_free;
+ vchan_init(&atchan->vc, &atdma->dma_device);
atc_enable_chan_irq(atdma, i);
}
/* set base routines */
- atdma->dma_common.device_alloc_chan_resources = atc_alloc_chan_resources;
- atdma->dma_common.device_free_chan_resources = atc_free_chan_resources;
- atdma->dma_common.device_tx_status = atc_tx_status;
- atdma->dma_common.device_issue_pending = atc_issue_pending;
- atdma->dma_common.dev = &pdev->dev;
+ atdma->dma_device.device_alloc_chan_resources = atc_alloc_chan_resources;
+ atdma->dma_device.device_free_chan_resources = atc_free_chan_resources;
+ atdma->dma_device.device_tx_status = atc_tx_status;
+ atdma->dma_device.device_issue_pending = atc_issue_pending;
+ atdma->dma_device.dev = &pdev->dev;
/* set prep routines based on capability */
- if (dma_has_cap(DMA_INTERLEAVE, atdma->dma_common.cap_mask))
- atdma->dma_common.device_prep_interleaved_dma = atc_prep_dma_interleaved;
+ if (dma_has_cap(DMA_INTERLEAVE, atdma->dma_device.cap_mask))
+ atdma->dma_device.device_prep_interleaved_dma = atc_prep_dma_interleaved;
- if (dma_has_cap(DMA_MEMCPY, atdma->dma_common.cap_mask))
- atdma->dma_common.device_prep_dma_memcpy = atc_prep_dma_memcpy;
+ if (dma_has_cap(DMA_MEMCPY, atdma->dma_device.cap_mask))
+ atdma->dma_device.device_prep_dma_memcpy = atc_prep_dma_memcpy;
- if (dma_has_cap(DMA_MEMSET, atdma->dma_common.cap_mask)) {
- atdma->dma_common.device_prep_dma_memset = atc_prep_dma_memset;
- atdma->dma_common.device_prep_dma_memset_sg = atc_prep_dma_memset_sg;
- atdma->dma_common.fill_align = DMAENGINE_ALIGN_4_BYTES;
+ if (dma_has_cap(DMA_MEMSET, atdma->dma_device.cap_mask)) {
+ atdma->dma_device.device_prep_dma_memset = atc_prep_dma_memset;
+ atdma->dma_device.device_prep_dma_memset_sg = atc_prep_dma_memset_sg;
+ atdma->dma_device.fill_align = DMAENGINE_ALIGN_4_BYTES;
}
- if (dma_has_cap(DMA_SLAVE, atdma->dma_common.cap_mask)) {
- atdma->dma_common.device_prep_slave_sg = atc_prep_slave_sg;
+ if (dma_has_cap(DMA_SLAVE, atdma->dma_device.cap_mask)) {
+ atdma->dma_device.device_prep_slave_sg = atc_prep_slave_sg;
/* controller can do slave DMA: can trigger cyclic transfers */
- dma_cap_set(DMA_CYCLIC, atdma->dma_common.cap_mask);
- atdma->dma_common.device_prep_dma_cyclic = atc_prep_dma_cyclic;
- atdma->dma_common.device_config = atc_config;
- atdma->dma_common.device_pause = atc_pause;
- atdma->dma_common.device_resume = atc_resume;
- atdma->dma_common.device_terminate_all = atc_terminate_all;
- atdma->dma_common.src_addr_widths = ATC_DMA_BUSWIDTHS;
- atdma->dma_common.dst_addr_widths = ATC_DMA_BUSWIDTHS;
- atdma->dma_common.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
- atdma->dma_common.residue_granularity = DMA_RESIDUE_GRANULARITY_BURST;
+ dma_cap_set(DMA_CYCLIC, atdma->dma_device.cap_mask);
+ atdma->dma_device.device_prep_dma_cyclic = atc_prep_dma_cyclic;
+ atdma->dma_device.device_config = atc_config;
+ atdma->dma_device.device_pause = atc_pause;
+ atdma->dma_device.device_resume = atc_resume;
+ atdma->dma_device.device_terminate_all = atc_terminate_all;
+ atdma->dma_device.src_addr_widths = ATC_DMA_BUSWIDTHS;
+ atdma->dma_device.dst_addr_widths = ATC_DMA_BUSWIDTHS;
+ atdma->dma_device.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
+ atdma->dma_device.residue_granularity = DMA_RESIDUE_GRANULARITY_BURST;
}
dma_writel(atdma, EN, AT_DMA_ENABLE);
dev_info(&pdev->dev, "Atmel AHB DMA Controller ( %s%s%s), %d channels\n",
- dma_has_cap(DMA_MEMCPY, atdma->dma_common.cap_mask) ? "cpy " : "",
- dma_has_cap(DMA_MEMSET, atdma->dma_common.cap_mask) ? "set " : "",
- dma_has_cap(DMA_SLAVE, atdma->dma_common.cap_mask) ? "slave " : "",
+ dma_has_cap(DMA_MEMCPY, atdma->dma_device.cap_mask) ? "cpy " : "",
+ dma_has_cap(DMA_MEMSET, atdma->dma_device.cap_mask) ? "set " : "",
+ dma_has_cap(DMA_SLAVE, atdma->dma_device.cap_mask) ? "slave " : "",
plat_dat->nr_channels);
- err = dma_async_device_register(&atdma->dma_common);
+ err = dma_async_device_register(&atdma->dma_device);
if (err) {
dev_err(&pdev->dev, "Unable to register: %d.\n", err);
goto err_dma_async_device_register;
return 0;
err_of_dma_controller_register:
- dma_async_device_unregister(&atdma->dma_common);
+ dma_async_device_unregister(&atdma->dma_device);
err_dma_async_device_register:
dma_pool_destroy(atdma->memset_pool);
err_memset_pool_create:
- dma_pool_destroy(atdma->dma_desc_pool);
+ dma_pool_destroy(atdma->lli_pool);
err_desc_pool_create:
free_irq(platform_get_irq(pdev, 0), atdma);
err_irq:
clk_disable_unprepare(atdma->clk);
-err_clk_prepare:
- clk_put(atdma->clk);
-err_clk:
- iounmap(atdma->regs);
- atdma->regs = NULL;
-err_release_r:
- release_mem_region(io->start, size);
-err_kfree:
- kfree(atdma);
return err;
}
{
struct at_dma *atdma = platform_get_drvdata(pdev);
struct dma_chan *chan, *_chan;
- struct resource *io;
at_dma_off(atdma);
if (pdev->dev.of_node)
of_dma_controller_free(pdev->dev.of_node);
- dma_async_device_unregister(&atdma->dma_common);
+ dma_async_device_unregister(&atdma->dma_device);
dma_pool_destroy(atdma->memset_pool);
- dma_pool_destroy(atdma->dma_desc_pool);
+ dma_pool_destroy(atdma->lli_pool);
free_irq(platform_get_irq(pdev, 0), atdma);
- list_for_each_entry_safe(chan, _chan, &atdma->dma_common.channels,
+ list_for_each_entry_safe(chan, _chan, &atdma->dma_device.channels,
device_node) {
- struct at_dma_chan *atchan = to_at_dma_chan(chan);
-
/* Disable interrupts */
atc_disable_chan_irq(atdma, chan->chan_id);
-
- tasklet_kill(&atchan->tasklet);
list_del(&chan->device_node);
}
clk_disable_unprepare(atdma->clk);
- clk_put(atdma->clk);
-
- iounmap(atdma->regs);
- atdma->regs = NULL;
-
- io = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- release_mem_region(io->start, resource_size(io));
-
- kfree(atdma);
return 0;
}
struct at_dma *atdma = dev_get_drvdata(dev);
struct dma_chan *chan, *_chan;
- list_for_each_entry_safe(chan, _chan, &atdma->dma_common.channels,
+ list_for_each_entry_safe(chan, _chan, &atdma->dma_device.channels,
device_node) {
struct at_dma_chan *atchan = to_at_dma_chan(chan);
/* wait for transaction completion (except in cyclic case) */
static void atc_suspend_cyclic(struct at_dma_chan *atchan)
{
- struct dma_chan *chan = &atchan->chan_common;
+ struct dma_chan *chan = &atchan->vc.chan;
/* Channel should be paused by user
* do it anyway even if it is not done already */
struct dma_chan *chan, *_chan;
/* preserve data */
- list_for_each_entry_safe(chan, _chan, &atdma->dma_common.channels,
+ list_for_each_entry_safe(chan, _chan, &atdma->dma_device.channels,
device_node) {
struct at_dma_chan *atchan = to_at_dma_chan(chan);
static void atc_resume_cyclic(struct at_dma_chan *atchan)
{
- struct at_dma *atdma = to_at_dma(atchan->chan_common.device);
+ struct at_dma *atdma = to_at_dma(atchan->vc.chan.device);
/* restore channel status for cyclic descriptors list:
* next descriptor in the cyclic list at the time of suspend */
/* restore saved data */
dma_writel(atdma, EBCIER, atdma->save_imr);
- list_for_each_entry_safe(chan, _chan, &atdma->dma_common.channels,
+ list_for_each_entry_safe(chan, _chan, &atdma->dma_device.channels,
device_node) {
struct at_dma_chan *atchan = to_at_dma_chan(chan);
return 0;
}
-static const struct dev_pm_ops at_dma_dev_pm_ops = {
+static const struct dev_pm_ops __maybe_unused at_dma_dev_pm_ops = {
.prepare = at_dma_prepare,
.suspend_noirq = at_dma_suspend_noirq,
.resume_noirq = at_dma_resume_noirq,
.id_table = atdma_devtypes,
.driver = {
.name = "at_hdmac",
- .pm = &at_dma_dev_pm_ops,
+ .pm = pm_ptr(&at_dma_dev_pm_ops),
.of_match_table = of_match_ptr(atmel_dma_dt_ids),
},
};
MODULE_DESCRIPTION("Atmel AHB DMA Controller driver");
MODULE_AUTHOR("Nicolas Ferre <nicolas.ferre@atmel.com>");
+MODULE_AUTHOR("Tudor Ambarus <tudor.ambarus@microchip.com>");
MODULE_LICENSE("GPL");
MODULE_ALIAS("platform:at_hdmac");
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/*
- * Header file for the Atmel AHB DMA Controller driver
- *
- * Copyright (C) 2008 Atmel Corporation
- */
-#ifndef AT_HDMAC_REGS_H
-#define AT_HDMAC_REGS_H
-
-#define AT_DMA_MAX_NR_CHANNELS 8
-
-
-#define AT_DMA_GCFG 0x00 /* Global Configuration Register */
-#define AT_DMA_IF_BIGEND(i) (0x1 << (i)) /* AHB-Lite Interface i in Big-endian mode */
-#define AT_DMA_ARB_CFG (0x1 << 4) /* Arbiter mode. */
-#define AT_DMA_ARB_CFG_FIXED (0x0 << 4)
-#define AT_DMA_ARB_CFG_ROUND_ROBIN (0x1 << 4)
-
-#define AT_DMA_EN 0x04 /* Controller Enable Register */
-#define AT_DMA_ENABLE (0x1 << 0)
-
-#define AT_DMA_SREQ 0x08 /* Software Single Request Register */
-#define AT_DMA_SSREQ(x) (0x1 << ((x) << 1)) /* Request a source single transfer on channel x */
-#define AT_DMA_DSREQ(x) (0x1 << (1 + ((x) << 1))) /* Request a destination single transfer on channel x */
-
-#define AT_DMA_CREQ 0x0C /* Software Chunk Transfer Request Register */
-#define AT_DMA_SCREQ(x) (0x1 << ((x) << 1)) /* Request a source chunk transfer on channel x */
-#define AT_DMA_DCREQ(x) (0x1 << (1 + ((x) << 1))) /* Request a destination chunk transfer on channel x */
-
-#define AT_DMA_LAST 0x10 /* Software Last Transfer Flag Register */
-#define AT_DMA_SLAST(x) (0x1 << ((x) << 1)) /* This src rq is last tx of buffer on channel x */
-#define AT_DMA_DLAST(x) (0x1 << (1 + ((x) << 1))) /* This dst rq is last tx of buffer on channel x */
-
-#define AT_DMA_SYNC 0x14 /* Request Synchronization Register */
-#define AT_DMA_SYR(h) (0x1 << (h)) /* Synchronize handshake line h */
-
-/* Error, Chained Buffer transfer completed and Buffer transfer completed Interrupt registers */
-#define AT_DMA_EBCIER 0x18 /* Enable register */
-#define AT_DMA_EBCIDR 0x1C /* Disable register */
-#define AT_DMA_EBCIMR 0x20 /* Mask Register */
-#define AT_DMA_EBCISR 0x24 /* Status Register */
-#define AT_DMA_CBTC_OFFSET 8
-#define AT_DMA_ERR_OFFSET 16
-#define AT_DMA_BTC(x) (0x1 << (x))
-#define AT_DMA_CBTC(x) (0x1 << (AT_DMA_CBTC_OFFSET + (x)))
-#define AT_DMA_ERR(x) (0x1 << (AT_DMA_ERR_OFFSET + (x)))
-
-#define AT_DMA_CHER 0x28 /* Channel Handler Enable Register */
-#define AT_DMA_ENA(x) (0x1 << (x))
-#define AT_DMA_SUSP(x) (0x1 << ( 8 + (x)))
-#define AT_DMA_KEEP(x) (0x1 << (24 + (x)))
-
-#define AT_DMA_CHDR 0x2C /* Channel Handler Disable Register */
-#define AT_DMA_DIS(x) (0x1 << (x))
-#define AT_DMA_RES(x) (0x1 << ( 8 + (x)))
-
-#define AT_DMA_CHSR 0x30 /* Channel Handler Status Register */
-#define AT_DMA_EMPT(x) (0x1 << (16 + (x)))
-#define AT_DMA_STAL(x) (0x1 << (24 + (x)))
-
-
-#define AT_DMA_CH_REGS_BASE 0x3C /* Channel registers base address */
-#define ch_regs(x) (AT_DMA_CH_REGS_BASE + (x) * 0x28) /* Channel x base addr */
-
-/* Hardware register offset for each channel */
-#define ATC_SADDR_OFFSET 0x00 /* Source Address Register */
-#define ATC_DADDR_OFFSET 0x04 /* Destination Address Register */
-#define ATC_DSCR_OFFSET 0x08 /* Descriptor Address Register */
-#define ATC_CTRLA_OFFSET 0x0C /* Control A Register */
-#define ATC_CTRLB_OFFSET 0x10 /* Control B Register */
-#define ATC_CFG_OFFSET 0x14 /* Configuration Register */
-#define ATC_SPIP_OFFSET 0x18 /* Src PIP Configuration Register */
-#define ATC_DPIP_OFFSET 0x1C /* Dst PIP Configuration Register */
-
-
-/* Bitfield definitions */
-
-/* Bitfields in DSCR */
-#define ATC_DSCR_IF(i) (0x3 & (i)) /* Dsc feched via AHB-Lite Interface i */
-
-/* Bitfields in CTRLA */
-#define ATC_BTSIZE_MAX 0xFFFFUL /* Maximum Buffer Transfer Size */
-#define ATC_BTSIZE(x) (ATC_BTSIZE_MAX & (x)) /* Buffer Transfer Size */
-#define ATC_SCSIZE_MASK (0x7 << 16) /* Source Chunk Transfer Size */
-#define ATC_SCSIZE(x) (ATC_SCSIZE_MASK & ((x) << 16))
-#define ATC_SCSIZE_1 (0x0 << 16)
-#define ATC_SCSIZE_4 (0x1 << 16)
-#define ATC_SCSIZE_8 (0x2 << 16)
-#define ATC_SCSIZE_16 (0x3 << 16)
-#define ATC_SCSIZE_32 (0x4 << 16)
-#define ATC_SCSIZE_64 (0x5 << 16)
-#define ATC_SCSIZE_128 (0x6 << 16)
-#define ATC_SCSIZE_256 (0x7 << 16)
-#define ATC_DCSIZE_MASK (0x7 << 20) /* Destination Chunk Transfer Size */
-#define ATC_DCSIZE(x) (ATC_DCSIZE_MASK & ((x) << 20))
-#define ATC_DCSIZE_1 (0x0 << 20)
-#define ATC_DCSIZE_4 (0x1 << 20)
-#define ATC_DCSIZE_8 (0x2 << 20)
-#define ATC_DCSIZE_16 (0x3 << 20)
-#define ATC_DCSIZE_32 (0x4 << 20)
-#define ATC_DCSIZE_64 (0x5 << 20)
-#define ATC_DCSIZE_128 (0x6 << 20)
-#define ATC_DCSIZE_256 (0x7 << 20)
-#define ATC_SRC_WIDTH_MASK (0x3 << 24) /* Source Single Transfer Size */
-#define ATC_SRC_WIDTH(x) ((x) << 24)
-#define ATC_SRC_WIDTH_BYTE (0x0 << 24)
-#define ATC_SRC_WIDTH_HALFWORD (0x1 << 24)
-#define ATC_SRC_WIDTH_WORD (0x2 << 24)
-#define ATC_REG_TO_SRC_WIDTH(r) (((r) >> 24) & 0x3)
-#define ATC_DST_WIDTH_MASK (0x3 << 28) /* Destination Single Transfer Size */
-#define ATC_DST_WIDTH(x) ((x) << 28)
-#define ATC_DST_WIDTH_BYTE (0x0 << 28)
-#define ATC_DST_WIDTH_HALFWORD (0x1 << 28)
-#define ATC_DST_WIDTH_WORD (0x2 << 28)
-#define ATC_DONE (0x1 << 31) /* Tx Done (only written back in descriptor) */
-
-/* Bitfields in CTRLB */
-#define ATC_SIF(i) (0x3 & (i)) /* Src tx done via AHB-Lite Interface i */
-#define ATC_DIF(i) ((0x3 & (i)) << 4) /* Dst tx done via AHB-Lite Interface i */
- /* Specify AHB interfaces */
-#define AT_DMA_MEM_IF 0 /* interface 0 as memory interface */
-#define AT_DMA_PER_IF 1 /* interface 1 as peripheral interface */
-
-#define ATC_SRC_PIP (0x1 << 8) /* Source Picture-in-Picture enabled */
-#define ATC_DST_PIP (0x1 << 12) /* Destination Picture-in-Picture enabled */
-#define ATC_SRC_DSCR_DIS (0x1 << 16) /* Src Descriptor fetch disable */
-#define ATC_DST_DSCR_DIS (0x1 << 20) /* Dst Descriptor fetch disable */
-#define ATC_FC_MASK (0x7 << 21) /* Choose Flow Controller */
-#define ATC_FC_MEM2MEM (0x0 << 21) /* Mem-to-Mem (DMA) */
-#define ATC_FC_MEM2PER (0x1 << 21) /* Mem-to-Periph (DMA) */
-#define ATC_FC_PER2MEM (0x2 << 21) /* Periph-to-Mem (DMA) */
-#define ATC_FC_PER2PER (0x3 << 21) /* Periph-to-Periph (DMA) */
-#define ATC_FC_PER2MEM_PER (0x4 << 21) /* Periph-to-Mem (Peripheral) */
-#define ATC_FC_MEM2PER_PER (0x5 << 21) /* Mem-to-Periph (Peripheral) */
-#define ATC_FC_PER2PER_SRCPER (0x6 << 21) /* Periph-to-Periph (Src Peripheral) */
-#define ATC_FC_PER2PER_DSTPER (0x7 << 21) /* Periph-to-Periph (Dst Peripheral) */
-#define ATC_SRC_ADDR_MODE_MASK (0x3 << 24)
-#define ATC_SRC_ADDR_MODE_INCR (0x0 << 24) /* Incrementing Mode */
-#define ATC_SRC_ADDR_MODE_DECR (0x1 << 24) /* Decrementing Mode */
-#define ATC_SRC_ADDR_MODE_FIXED (0x2 << 24) /* Fixed Mode */
-#define ATC_DST_ADDR_MODE_MASK (0x3 << 28)
-#define ATC_DST_ADDR_MODE_INCR (0x0 << 28) /* Incrementing Mode */
-#define ATC_DST_ADDR_MODE_DECR (0x1 << 28) /* Decrementing Mode */
-#define ATC_DST_ADDR_MODE_FIXED (0x2 << 28) /* Fixed Mode */
-#define ATC_IEN (0x1 << 30) /* BTC interrupt enable (active low) */
-#define ATC_AUTO (0x1 << 31) /* Auto multiple buffer tx enable */
-
-/* Bitfields in CFG */
-#define ATC_PER_MSB(h) ((0x30U & (h)) >> 4) /* Extract most significant bits of a handshaking identifier */
-
-#define ATC_SRC_PER(h) (0xFU & (h)) /* Channel src rq associated with periph handshaking ifc h */
-#define ATC_DST_PER(h) ((0xFU & (h)) << 4) /* Channel dst rq associated with periph handshaking ifc h */
-#define ATC_SRC_REP (0x1 << 8) /* Source Replay Mod */
-#define ATC_SRC_H2SEL (0x1 << 9) /* Source Handshaking Mod */
-#define ATC_SRC_H2SEL_SW (0x0 << 9)
-#define ATC_SRC_H2SEL_HW (0x1 << 9)
-#define ATC_SRC_PER_MSB(h) (ATC_PER_MSB(h) << 10) /* Channel src rq (most significant bits) */
-#define ATC_DST_REP (0x1 << 12) /* Destination Replay Mod */
-#define ATC_DST_H2SEL (0x1 << 13) /* Destination Handshaking Mod */
-#define ATC_DST_H2SEL_SW (0x0 << 13)
-#define ATC_DST_H2SEL_HW (0x1 << 13)
-#define ATC_DST_PER_MSB(h) (ATC_PER_MSB(h) << 14) /* Channel dst rq (most significant bits) */
-#define ATC_SOD (0x1 << 16) /* Stop On Done */
-#define ATC_LOCK_IF (0x1 << 20) /* Interface Lock */
-#define ATC_LOCK_B (0x1 << 21) /* AHB Bus Lock */
-#define ATC_LOCK_IF_L (0x1 << 22) /* Master Interface Arbiter Lock */
-#define ATC_LOCK_IF_L_CHUNK (0x0 << 22)
-#define ATC_LOCK_IF_L_BUFFER (0x1 << 22)
-#define ATC_AHB_PROT_MASK (0x7 << 24) /* AHB Protection */
-#define ATC_FIFOCFG_MASK (0x3 << 28) /* FIFO Request Configuration */
-#define ATC_FIFOCFG_LARGESTBURST (0x0 << 28)
-#define ATC_FIFOCFG_HALFFIFO (0x1 << 28)
-#define ATC_FIFOCFG_ENOUGHSPACE (0x2 << 28)
-
-/* Bitfields in SPIP */
-#define ATC_SPIP_HOLE(x) (0xFFFFU & (x))
-#define ATC_SPIP_BOUNDARY(x) ((0x3FF & (x)) << 16)
-
-/* Bitfields in DPIP */
-#define ATC_DPIP_HOLE(x) (0xFFFFU & (x))
-#define ATC_DPIP_BOUNDARY(x) ((0x3FF & (x)) << 16)
-
-
-/*-- descriptors -----------------------------------------------------*/
-
-/* LLI == Linked List Item; aka DMA buffer descriptor */
-struct at_lli {
- /* values that are not changed by hardware */
- u32 saddr;
- u32 daddr;
- /* value that may get written back: */
- u32 ctrla;
- /* more values that are not changed by hardware */
- u32 ctrlb;
- u32 dscr; /* chain to next lli */
-};
-
-/**
- * struct at_desc - software descriptor
- * @at_lli: hardware lli structure
- * @txd: support for the async_tx api
- * @desc_node: node on the channed descriptors list
- * @len: descriptor byte count
- * @total_len: total transaction byte count
- */
-struct at_desc {
- /* FIRST values the hardware uses */
- struct at_lli lli;
-
- /* THEN values for driver housekeeping */
- struct list_head tx_list;
- struct dma_async_tx_descriptor txd;
- struct list_head desc_node;
- size_t len;
- size_t total_len;
-
- /* Interleaved data */
- size_t boundary;
- size_t dst_hole;
- size_t src_hole;
-
- /* Memset temporary buffer */
- bool memset_buffer;
- dma_addr_t memset_paddr;
- int *memset_vaddr;
-};
-
-static inline struct at_desc *
-txd_to_at_desc(struct dma_async_tx_descriptor *txd)
-{
- return container_of(txd, struct at_desc, txd);
-}
-
-
-/*-- Channels --------------------------------------------------------*/
-
-/**
- * atc_status - information bits stored in channel status flag
- *
- * Manipulated with atomic operations.
- */
-enum atc_status {
- ATC_IS_ERROR = 0,
- ATC_IS_PAUSED = 1,
- ATC_IS_CYCLIC = 24,
-};
-
-/**
- * struct at_dma_chan - internal representation of an Atmel HDMAC channel
- * @chan_common: common dmaengine channel object members
- * @device: parent device
- * @ch_regs: memory mapped register base
- * @mask: channel index in a mask
- * @per_if: peripheral interface
- * @mem_if: memory interface
- * @status: transmit status information from irq/prep* functions
- * to tasklet (use atomic operations)
- * @tasklet: bottom half to finish transaction work
- * @save_cfg: configuration register that is saved on suspend/resume cycle
- * @save_dscr: for cyclic operations, preserve next descriptor address in
- * the cyclic list on suspend/resume cycle
- * @dma_sconfig: configuration for slave transfers, passed via
- * .device_config
- * @lock: serializes enqueue/dequeue operations to descriptors lists
- * @active_list: list of descriptors dmaengine is being running on
- * @queue: list of descriptors ready to be submitted to engine
- * @free_list: list of descriptors usable by the channel
- */
-struct at_dma_chan {
- struct dma_chan chan_common;
- struct at_dma *device;
- void __iomem *ch_regs;
- u8 mask;
- u8 per_if;
- u8 mem_if;
- unsigned long status;
- struct tasklet_struct tasklet;
- u32 save_cfg;
- u32 save_dscr;
- struct dma_slave_config dma_sconfig;
-
- spinlock_t lock;
-
- /* these other elements are all protected by lock */
- struct list_head active_list;
- struct list_head queue;
- struct list_head free_list;
-};
-
-#define channel_readl(atchan, name) \
- __raw_readl((atchan)->ch_regs + ATC_##name##_OFFSET)
-
-#define channel_writel(atchan, name, val) \
- __raw_writel((val), (atchan)->ch_regs + ATC_##name##_OFFSET)
-
-static inline struct at_dma_chan *to_at_dma_chan(struct dma_chan *dchan)
-{
- return container_of(dchan, struct at_dma_chan, chan_common);
-}
-
-/*
- * Fix sconfig's burst size according to at_hdmac. We need to convert them as:
- * 1 -> 0, 4 -> 1, 8 -> 2, 16 -> 3, 32 -> 4, 64 -> 5, 128 -> 6, 256 -> 7.
- *
- * This can be done by finding most significant bit set.
- */
-static inline void convert_burst(u32 *maxburst)
-{
- if (*maxburst > 1)
- *maxburst = fls(*maxburst) - 2;
- else
- *maxburst = 0;
-}
-
-/*
- * Fix sconfig's bus width according to at_hdmac.
- * 1 byte -> 0, 2 bytes -> 1, 4 bytes -> 2.
- */
-static inline u8 convert_buswidth(enum dma_slave_buswidth addr_width)
-{
- switch (addr_width) {
- case DMA_SLAVE_BUSWIDTH_2_BYTES:
- return 1;
- case DMA_SLAVE_BUSWIDTH_4_BYTES:
- return 2;
- default:
- /* For 1 byte width or fallback */
- return 0;
- }
-}
-
-/*-- Controller ------------------------------------------------------*/
-
-/**
- * struct at_dma - internal representation of an Atmel HDMA Controller
- * @chan_common: common dmaengine dma_device object members
- * @atdma_devtype: identifier of DMA controller compatibility
- * @ch_regs: memory mapped register base
- * @clk: dma controller clock
- * @save_imr: interrupt mask register that is saved on suspend/resume cycle
- * @all_chan_mask: all channels availlable in a mask
- * @dma_desc_pool: base of DMA descriptor region (DMA address)
- * @chan: channels table to store at_dma_chan structures
- */
-struct at_dma {
- struct dma_device dma_common;
- void __iomem *regs;
- struct clk *clk;
- u32 save_imr;
-
- u8 all_chan_mask;
-
- struct dma_pool *dma_desc_pool;
- struct dma_pool *memset_pool;
- /* AT THE END channels table */
- struct at_dma_chan chan[];
-};
-
-#define dma_readl(atdma, name) \
- __raw_readl((atdma)->regs + AT_DMA_##name)
-#define dma_writel(atdma, name, val) \
- __raw_writel((val), (atdma)->regs + AT_DMA_##name)
-
-static inline struct at_dma *to_at_dma(struct dma_device *ddev)
-{
- return container_of(ddev, struct at_dma, dma_common);
-}
-
-
-/*-- Helper functions ------------------------------------------------*/
-
-static struct device *chan2dev(struct dma_chan *chan)
-{
- return &chan->dev->device;
-}
-
-#if defined(VERBOSE_DEBUG)
-static void vdbg_dump_regs(struct at_dma_chan *atchan)
-{
- struct at_dma *atdma = to_at_dma(atchan->chan_common.device);
-
- dev_err(chan2dev(&atchan->chan_common),
- " channel %d : imr = 0x%x, chsr = 0x%x\n",
- atchan->chan_common.chan_id,
- dma_readl(atdma, EBCIMR),
- dma_readl(atdma, CHSR));
-
- dev_err(chan2dev(&atchan->chan_common),
- " channel: s0x%x d0x%x ctrl0x%x:0x%x cfg0x%x l0x%x\n",
- channel_readl(atchan, SADDR),
- channel_readl(atchan, DADDR),
- channel_readl(atchan, CTRLA),
- channel_readl(atchan, CTRLB),
- channel_readl(atchan, CFG),
- channel_readl(atchan, DSCR));
-}
-#else
-static void vdbg_dump_regs(struct at_dma_chan *atchan) {}
-#endif
-
-static void atc_dump_lli(struct at_dma_chan *atchan, struct at_lli *lli)
-{
- dev_crit(chan2dev(&atchan->chan_common),
- "desc: s%pad d%pad ctrl0x%x:0x%x l%pad\n",
- &lli->saddr, &lli->daddr,
- lli->ctrla, lli->ctrlb, &lli->dscr);
-}
-
-
-static void atc_setup_irq(struct at_dma *atdma, int chan_id, int on)
-{
- u32 ebci;
-
- /* enable interrupts on buffer transfer completion & error */
- ebci = AT_DMA_BTC(chan_id)
- | AT_DMA_ERR(chan_id);
- if (on)
- dma_writel(atdma, EBCIER, ebci);
- else
- dma_writel(atdma, EBCIDR, ebci);
-}
-
-static void atc_enable_chan_irq(struct at_dma *atdma, int chan_id)
-{
- atc_setup_irq(atdma, chan_id, 1);
-}
-
-static void atc_disable_chan_irq(struct at_dma *atdma, int chan_id)
-{
- atc_setup_irq(atdma, chan_id, 0);
-}
-
-
-/**
- * atc_chan_is_enabled - test if given channel is enabled
- * @atchan: channel we want to test status
- */
-static inline int atc_chan_is_enabled(struct at_dma_chan *atchan)
-{
- struct at_dma *atdma = to_at_dma(atchan->chan_common.device);
-
- return !!(dma_readl(atdma, CHSR) & atchan->mask);
-}
-
-/**
- * atc_chan_is_paused - test channel pause/resume status
- * @atchan: channel we want to test status
- */
-static inline int atc_chan_is_paused(struct at_dma_chan *atchan)
-{
- return test_bit(ATC_IS_PAUSED, &atchan->status);
-}
-
-/**
- * atc_chan_is_cyclic - test if given channel has cyclic property set
- * @atchan: channel we want to test status
- */
-static inline int atc_chan_is_cyclic(struct at_dma_chan *atchan)
-{
- return test_bit(ATC_IS_CYCLIC, &atchan->status);
-}
-
-/**
- * set_desc_eol - set end-of-link to descriptor so it will end transfer
- * @desc: descriptor, signle or at the end of a chain, to end chain on
- */
-static void set_desc_eol(struct at_desc *desc)
-{
- u32 ctrlb = desc->lli.ctrlb;
-
- ctrlb &= ~ATC_IEN;
- ctrlb |= ATC_SRC_DSCR_DIS | ATC_DST_DSCR_DIS;
-
- desc->lli.ctrlb = ctrlb;
- desc->lli.dscr = 0;
-}
-
-#endif /* AT_HDMAC_REGS_H */
JZ_SOC_DATA_BREAK_LINKS,
};
+static const struct jz4780_dma_soc_data jz4755_dma_soc_data = {
+ .nb_channels = 4,
+ .transfer_ord_max = 5,
+ .flags = JZ_SOC_DATA_PER_CHAN_PM | JZ_SOC_DATA_NO_DCKES_DCKEC |
+ JZ_SOC_DATA_BREAK_LINKS,
+};
+
static const struct jz4780_dma_soc_data jz4760_dma_soc_data = {
.nb_channels = 5,
.transfer_ord_max = 6,
static const struct of_device_id jz4780_dma_dt_match[] = {
{ .compatible = "ingenic,jz4740-dma", .data = &jz4740_dma_soc_data },
{ .compatible = "ingenic,jz4725b-dma", .data = &jz4725b_dma_soc_data },
+ { .compatible = "ingenic,jz4755-dma", .data = &jz4755_dma_soc_data },
{ .compatible = "ingenic,jz4760-dma", .data = &jz4760_dma_soc_data },
{ .compatible = "ingenic,jz4760-mdma", .data = &jz4760_mdma_soc_data },
{ .compatible = "ingenic,jz4760-bdma", .data = &jz4760_bdma_soc_data },
return 0;
}
-static int idma64_remove(struct idma64_chip *chip)
+static void idma64_remove(struct idma64_chip *chip)
{
struct idma64 *idma64 = chip->idma64;
unsigned short i;
tasklet_kill(&idma64c->vchan.task);
}
-
- return 0;
}
/* ---------------------------------------------------------------------- */
{
struct idma64_chip *chip = platform_get_drvdata(pdev);
- return idma64_remove(chip);
+ idma64_remove(chip);
+
+ return 0;
}
static int __maybe_unused idma64_pm_suspend(struct device *dev)
#include <linux/io-64-nonatomic-lo-hi.h>
#include <linux/dmaengine.h>
#include <linux/irq.h>
-#include <linux/msi.h>
#include <uapi/linux/idxd.h>
#include "../dmaengine.h"
#include "idxd.h"
!idxd->hw.group_cap.progress_limit;
}
+static bool idxd_group_attr_read_buffers_invisible(struct attribute *attr,
+ struct idxd_device *idxd)
+{
+ /*
+ * Intel IAA does not support Read Buffer allocation control,
+ * make these attributes invisible.
+ */
+ return (attr == &dev_attr_group_use_token_limit.attr ||
+ attr == &dev_attr_group_use_read_buffer_limit.attr ||
+ attr == &dev_attr_group_tokens_allowed.attr ||
+ attr == &dev_attr_group_read_buffers_allowed.attr ||
+ attr == &dev_attr_group_tokens_reserved.attr ||
+ attr == &dev_attr_group_read_buffers_reserved.attr) &&
+ idxd->data->type == IDXD_TYPE_IAX;
+}
+
static umode_t idxd_group_attr_visible(struct kobject *kobj,
struct attribute *attr, int n)
{
if (idxd_group_attr_progress_limit_invisible(attr, idxd))
return 0;
+ if (idxd_group_attr_read_buffers_invisible(attr, idxd))
+ return 0;
+
return attr->mode;
}
!idxd->hw.wq_cap.op_config;
}
+static bool idxd_wq_attr_max_batch_size_invisible(struct attribute *attr,
+ struct idxd_device *idxd)
+{
+ /* Intel IAA does not support batch processing, make it invisible */
+ return attr == &dev_attr_wq_max_batch_size.attr &&
+ idxd->data->type == IDXD_TYPE_IAX;
+}
+
static umode_t idxd_wq_attr_visible(struct kobject *kobj,
struct attribute *attr, int n)
{
if (idxd_wq_attr_op_config_invisible(attr, idxd))
return 0;
+ if (idxd_wq_attr_max_batch_size_invisible(attr, idxd))
+ return 0;
+
return attr->mode;
}
}
static DEVICE_ATTR_RW(cmd_status);
+static bool idxd_device_attr_max_batch_size_invisible(struct attribute *attr,
+ struct idxd_device *idxd)
+{
+ /* Intel IAA does not support batch processing, make it invisible */
+ return attr == &dev_attr_max_batch_size.attr &&
+ idxd->data->type == IDXD_TYPE_IAX;
+}
+
+static bool idxd_device_attr_read_buffers_invisible(struct attribute *attr,
+ struct idxd_device *idxd)
+{
+ /*
+ * Intel IAA does not support Read Buffer allocation control,
+ * make these attributes invisible.
+ */
+ return (attr == &dev_attr_max_tokens.attr ||
+ attr == &dev_attr_max_read_buffers.attr ||
+ attr == &dev_attr_token_limit.attr ||
+ attr == &dev_attr_read_buffer_limit.attr) &&
+ idxd->data->type == IDXD_TYPE_IAX;
+}
+
+static umode_t idxd_device_attr_visible(struct kobject *kobj,
+ struct attribute *attr, int n)
+{
+ struct device *dev = container_of(kobj, struct device, kobj);
+ struct idxd_device *idxd = confdev_to_idxd(dev);
+
+ if (idxd_device_attr_max_batch_size_invisible(attr, idxd))
+ return 0;
+
+ if (idxd_device_attr_read_buffers_invisible(attr, idxd))
+ return 0;
+
+ return attr->mode;
+}
+
static struct attribute *idxd_device_attributes[] = {
&dev_attr_version.attr,
&dev_attr_max_groups.attr,
static const struct attribute_group idxd_device_attribute_group = {
.attrs = idxd_device_attributes,
+ .is_visible = idxd_device_attr_visible,
};
static const struct attribute_group *idxd_attribute_groups[] = {
static int idle_timeout = 2000;
module_param(idle_timeout, int, 0644);
MODULE_PARM_DESC(idle_timeout,
- "set ioat idel timeout [msec] (default 2000 [msec])");
+ "set ioat idle timeout [msec] (default 2000 [msec])");
#define IDLE_TIMEOUT msecs_to_jiffies(idle_timeout)
#define COMPLETION_TIMEOUT msecs_to_jiffies(completion_timeout)
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * offload engine driver for the Intel Xscale series of i/o processors
- * Copyright © 2006, Intel Corporation.
- */
-
-/*
- * This driver supports the asynchrounous DMA copy and RAID engines available
- * on the Intel Xscale(R) family of I/O Processors (IOP 32x, 33x, 134x)
- */
-
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/delay.h>
-#include <linux/dma-mapping.h>
-#include <linux/spinlock.h>
-#include <linux/interrupt.h>
-#include <linux/platform_device.h>
-#include <linux/prefetch.h>
-#include <linux/memory.h>
-#include <linux/ioport.h>
-#include <linux/raid/pq.h>
-#include <linux/slab.h>
-
-#include "iop-adma.h"
-#include "dmaengine.h"
-
-#define to_iop_adma_chan(chan) container_of(chan, struct iop_adma_chan, common)
-#define to_iop_adma_device(dev) \
- container_of(dev, struct iop_adma_device, common)
-#define tx_to_iop_adma_slot(tx) \
- container_of(tx, struct iop_adma_desc_slot, async_tx)
-
-/**
- * iop_adma_free_slots - flags descriptor slots for reuse
- * @slot: Slot to free
- * Caller must hold &iop_chan->lock while calling this function
- */
-static void iop_adma_free_slots(struct iop_adma_desc_slot *slot)
-{
- int stride = slot->slots_per_op;
-
- while (stride--) {
- slot->slots_per_op = 0;
- slot = list_entry(slot->slot_node.next,
- struct iop_adma_desc_slot,
- slot_node);
- }
-}
-
-static dma_cookie_t
-iop_adma_run_tx_complete_actions(struct iop_adma_desc_slot *desc,
- struct iop_adma_chan *iop_chan, dma_cookie_t cookie)
-{
- struct dma_async_tx_descriptor *tx = &desc->async_tx;
-
- BUG_ON(tx->cookie < 0);
- if (tx->cookie > 0) {
- cookie = tx->cookie;
- tx->cookie = 0;
-
- /* call the callback (must not sleep or submit new
- * operations to this channel)
- */
- dmaengine_desc_get_callback_invoke(tx, NULL);
-
- dma_descriptor_unmap(tx);
- if (desc->group_head)
- desc->group_head = NULL;
- }
-
- /* run dependent operations */
- dma_run_dependencies(tx);
-
- return cookie;
-}
-
-static int
-iop_adma_clean_slot(struct iop_adma_desc_slot *desc,
- struct iop_adma_chan *iop_chan)
-{
- /* the client is allowed to attach dependent operations
- * until 'ack' is set
- */
- if (!async_tx_test_ack(&desc->async_tx))
- return 0;
-
- /* leave the last descriptor in the chain
- * so we can append to it
- */
- if (desc->chain_node.next == &iop_chan->chain)
- return 1;
-
- dev_dbg(iop_chan->device->common.dev,
- "\tfree slot: %d slots_per_op: %d\n",
- desc->idx, desc->slots_per_op);
-
- list_del(&desc->chain_node);
- iop_adma_free_slots(desc);
-
- return 0;
-}
-
-static void __iop_adma_slot_cleanup(struct iop_adma_chan *iop_chan)
-{
- struct iop_adma_desc_slot *iter, *_iter, *grp_start = NULL;
- dma_cookie_t cookie = 0;
- u32 current_desc = iop_chan_get_current_descriptor(iop_chan);
- int busy = iop_chan_is_busy(iop_chan);
- int seen_current = 0, slot_cnt = 0, slots_per_op = 0;
-
- dev_dbg(iop_chan->device->common.dev, "%s\n", __func__);
- /* free completed slots from the chain starting with
- * the oldest descriptor
- */
- list_for_each_entry_safe(iter, _iter, &iop_chan->chain,
- chain_node) {
- pr_debug("\tcookie: %d slot: %d busy: %d "
- "this_desc: %pad next_desc: %#llx ack: %d\n",
- iter->async_tx.cookie, iter->idx, busy,
- &iter->async_tx.phys, (u64)iop_desc_get_next_desc(iter),
- async_tx_test_ack(&iter->async_tx));
- prefetch(_iter);
- prefetch(&_iter->async_tx);
-
- /* do not advance past the current descriptor loaded into the
- * hardware channel, subsequent descriptors are either in
- * process or have not been submitted
- */
- if (seen_current)
- break;
-
- /* stop the search if we reach the current descriptor and the
- * channel is busy, or if it appears that the current descriptor
- * needs to be re-read (i.e. has been appended to)
- */
- if (iter->async_tx.phys == current_desc) {
- BUG_ON(seen_current++);
- if (busy || iop_desc_get_next_desc(iter))
- break;
- }
-
- /* detect the start of a group transaction */
- if (!slot_cnt && !slots_per_op) {
- slot_cnt = iter->slot_cnt;
- slots_per_op = iter->slots_per_op;
- if (slot_cnt <= slots_per_op) {
- slot_cnt = 0;
- slots_per_op = 0;
- }
- }
-
- if (slot_cnt) {
- pr_debug("\tgroup++\n");
- if (!grp_start)
- grp_start = iter;
- slot_cnt -= slots_per_op;
- }
-
- /* all the members of a group are complete */
- if (slots_per_op != 0 && slot_cnt == 0) {
- struct iop_adma_desc_slot *grp_iter, *_grp_iter;
- int end_of_chain = 0;
- pr_debug("\tgroup end\n");
-
- /* collect the total results */
- if (grp_start->xor_check_result) {
- u32 zero_sum_result = 0;
- slot_cnt = grp_start->slot_cnt;
- grp_iter = grp_start;
-
- list_for_each_entry_from(grp_iter,
- &iop_chan->chain, chain_node) {
- zero_sum_result |=
- iop_desc_get_zero_result(grp_iter);
- pr_debug("\titer%d result: %d\n",
- grp_iter->idx, zero_sum_result);
- slot_cnt -= slots_per_op;
- if (slot_cnt == 0)
- break;
- }
- pr_debug("\tgrp_start->xor_check_result: %p\n",
- grp_start->xor_check_result);
- *grp_start->xor_check_result = zero_sum_result;
- }
-
- /* clean up the group */
- slot_cnt = grp_start->slot_cnt;
- grp_iter = grp_start;
- list_for_each_entry_safe_from(grp_iter, _grp_iter,
- &iop_chan->chain, chain_node) {
- cookie = iop_adma_run_tx_complete_actions(
- grp_iter, iop_chan, cookie);
-
- slot_cnt -= slots_per_op;
- end_of_chain = iop_adma_clean_slot(grp_iter,
- iop_chan);
-
- if (slot_cnt == 0 || end_of_chain)
- break;
- }
-
- /* the group should be complete at this point */
- BUG_ON(slot_cnt);
-
- slots_per_op = 0;
- grp_start = NULL;
- if (end_of_chain)
- break;
- else
- continue;
- } else if (slots_per_op) /* wait for group completion */
- continue;
-
- /* write back zero sum results (single descriptor case) */
- if (iter->xor_check_result && iter->async_tx.cookie)
- *iter->xor_check_result =
- iop_desc_get_zero_result(iter);
-
- cookie = iop_adma_run_tx_complete_actions(
- iter, iop_chan, cookie);
-
- if (iop_adma_clean_slot(iter, iop_chan))
- break;
- }
-
- if (cookie > 0) {
- iop_chan->common.completed_cookie = cookie;
- pr_debug("\tcompleted cookie %d\n", cookie);
- }
-}
-
-static void
-iop_adma_slot_cleanup(struct iop_adma_chan *iop_chan)
-{
- spin_lock_bh(&iop_chan->lock);
- __iop_adma_slot_cleanup(iop_chan);
- spin_unlock_bh(&iop_chan->lock);
-}
-
-static void iop_adma_tasklet(struct tasklet_struct *t)
-{
- struct iop_adma_chan *iop_chan = from_tasklet(iop_chan, t,
- irq_tasklet);
-
- /* lockdep will flag depedency submissions as potentially
- * recursive locking, this is not the case as a dependency
- * submission will never recurse a channels submit routine.
- * There are checks in async_tx.c to prevent this.
- */
- spin_lock_nested(&iop_chan->lock, SINGLE_DEPTH_NESTING);
- __iop_adma_slot_cleanup(iop_chan);
- spin_unlock(&iop_chan->lock);
-}
-
-static struct iop_adma_desc_slot *
-iop_adma_alloc_slots(struct iop_adma_chan *iop_chan, int num_slots,
- int slots_per_op)
-{
- struct iop_adma_desc_slot *iter, *_iter, *alloc_start = NULL;
- LIST_HEAD(chain);
- int slots_found, retry = 0;
-
- /* start search from the last allocated descrtiptor
- * if a contiguous allocation can not be found start searching
- * from the beginning of the list
- */
-retry:
- slots_found = 0;
- if (retry == 0)
- iter = iop_chan->last_used;
- else
- iter = list_entry(&iop_chan->all_slots,
- struct iop_adma_desc_slot,
- slot_node);
-
- list_for_each_entry_safe_continue(
- iter, _iter, &iop_chan->all_slots, slot_node) {
- prefetch(_iter);
- prefetch(&_iter->async_tx);
- if (iter->slots_per_op) {
- /* give up after finding the first busy slot
- * on the second pass through the list
- */
- if (retry)
- break;
-
- slots_found = 0;
- continue;
- }
-
- /* start the allocation if the slot is correctly aligned */
- if (!slots_found++) {
- if (iop_desc_is_aligned(iter, slots_per_op))
- alloc_start = iter;
- else {
- slots_found = 0;
- continue;
- }
- }
-
- if (slots_found == num_slots) {
- struct iop_adma_desc_slot *alloc_tail = NULL;
- struct iop_adma_desc_slot *last_used = NULL;
- iter = alloc_start;
- while (num_slots) {
- int i;
- dev_dbg(iop_chan->device->common.dev,
- "allocated slot: %d "
- "(desc %p phys: %#llx) slots_per_op %d\n",
- iter->idx, iter->hw_desc,
- (u64)iter->async_tx.phys, slots_per_op);
-
- /* pre-ack all but the last descriptor */
- if (num_slots != slots_per_op)
- async_tx_ack(&iter->async_tx);
-
- list_add_tail(&iter->chain_node, &chain);
- alloc_tail = iter;
- iter->async_tx.cookie = 0;
- iter->slot_cnt = num_slots;
- iter->xor_check_result = NULL;
- for (i = 0; i < slots_per_op; i++) {
- iter->slots_per_op = slots_per_op - i;
- last_used = iter;
- iter = list_entry(iter->slot_node.next,
- struct iop_adma_desc_slot,
- slot_node);
- }
- num_slots -= slots_per_op;
- }
- alloc_tail->group_head = alloc_start;
- alloc_tail->async_tx.cookie = -EBUSY;
- list_splice(&chain, &alloc_tail->tx_list);
- iop_chan->last_used = last_used;
- iop_desc_clear_next_desc(alloc_start);
- iop_desc_clear_next_desc(alloc_tail);
- return alloc_tail;
- }
- }
- if (!retry++)
- goto retry;
-
- /* perform direct reclaim if the allocation fails */
- __iop_adma_slot_cleanup(iop_chan);
-
- return NULL;
-}
-
-static void iop_adma_check_threshold(struct iop_adma_chan *iop_chan)
-{
- dev_dbg(iop_chan->device->common.dev, "pending: %d\n",
- iop_chan->pending);
-
- if (iop_chan->pending >= IOP_ADMA_THRESHOLD) {
- iop_chan->pending = 0;
- iop_chan_append(iop_chan);
- }
-}
-
-static dma_cookie_t
-iop_adma_tx_submit(struct dma_async_tx_descriptor *tx)
-{
- struct iop_adma_desc_slot *sw_desc = tx_to_iop_adma_slot(tx);
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(tx->chan);
- struct iop_adma_desc_slot *grp_start, *old_chain_tail;
- int slot_cnt;
- dma_cookie_t cookie;
- dma_addr_t next_dma;
-
- grp_start = sw_desc->group_head;
- slot_cnt = grp_start->slot_cnt;
-
- spin_lock_bh(&iop_chan->lock);
- cookie = dma_cookie_assign(tx);
-
- old_chain_tail = list_entry(iop_chan->chain.prev,
- struct iop_adma_desc_slot, chain_node);
- list_splice_init(&sw_desc->tx_list,
- &old_chain_tail->chain_node);
-
- /* fix up the hardware chain */
- next_dma = grp_start->async_tx.phys;
- iop_desc_set_next_desc(old_chain_tail, next_dma);
- BUG_ON(iop_desc_get_next_desc(old_chain_tail) != next_dma); /* flush */
-
- /* check for pre-chained descriptors */
- iop_paranoia(iop_desc_get_next_desc(sw_desc));
-
- /* increment the pending count by the number of slots
- * memcpy operations have a 1:1 (slot:operation) relation
- * other operations are heavier and will pop the threshold
- * more often.
- */
- iop_chan->pending += slot_cnt;
- iop_adma_check_threshold(iop_chan);
- spin_unlock_bh(&iop_chan->lock);
-
- dev_dbg(iop_chan->device->common.dev, "%s cookie: %d slot: %d\n",
- __func__, sw_desc->async_tx.cookie, sw_desc->idx);
-
- return cookie;
-}
-
-static void iop_chan_start_null_memcpy(struct iop_adma_chan *iop_chan);
-static void iop_chan_start_null_xor(struct iop_adma_chan *iop_chan);
-
-/**
- * iop_adma_alloc_chan_resources - returns the number of allocated descriptors
- * @chan: allocate descriptor resources for this channel
- *
- * Note: We keep the slots for 1 operation on iop_chan->chain at all times. To
- * avoid deadlock, via async_xor, num_descs_in_pool must at a minimum be
- * greater than 2x the number slots needed to satisfy a device->max_xor
- * request.
- * */
-static int iop_adma_alloc_chan_resources(struct dma_chan *chan)
-{
- char *hw_desc;
- dma_addr_t dma_desc;
- int idx;
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(chan);
- struct iop_adma_desc_slot *slot = NULL;
- int init = iop_chan->slots_allocated ? 0 : 1;
- struct iop_adma_platform_data *plat_data =
- dev_get_platdata(&iop_chan->device->pdev->dev);
- int num_descs_in_pool = plat_data->pool_size/IOP_ADMA_SLOT_SIZE;
-
- /* Allocate descriptor slots */
- do {
- idx = iop_chan->slots_allocated;
- if (idx == num_descs_in_pool)
- break;
-
- slot = kzalloc(sizeof(*slot), GFP_KERNEL);
- if (!slot) {
- printk(KERN_INFO "IOP ADMA Channel only initialized"
- " %d descriptor slots", idx);
- break;
- }
- hw_desc = (char *) iop_chan->device->dma_desc_pool_virt;
- slot->hw_desc = (void *) &hw_desc[idx * IOP_ADMA_SLOT_SIZE];
-
- dma_async_tx_descriptor_init(&slot->async_tx, chan);
- slot->async_tx.tx_submit = iop_adma_tx_submit;
- INIT_LIST_HEAD(&slot->tx_list);
- INIT_LIST_HEAD(&slot->chain_node);
- INIT_LIST_HEAD(&slot->slot_node);
- dma_desc = iop_chan->device->dma_desc_pool;
- slot->async_tx.phys = dma_desc + idx * IOP_ADMA_SLOT_SIZE;
- slot->idx = idx;
-
- spin_lock_bh(&iop_chan->lock);
- iop_chan->slots_allocated++;
- list_add_tail(&slot->slot_node, &iop_chan->all_slots);
- spin_unlock_bh(&iop_chan->lock);
- } while (iop_chan->slots_allocated < num_descs_in_pool);
-
- if (idx && !iop_chan->last_used)
- iop_chan->last_used = list_entry(iop_chan->all_slots.next,
- struct iop_adma_desc_slot,
- slot_node);
-
- dev_dbg(iop_chan->device->common.dev,
- "allocated %d descriptor slots last_used: %p\n",
- iop_chan->slots_allocated, iop_chan->last_used);
-
- /* initialize the channel and the chain with a null operation */
- if (init) {
- if (dma_has_cap(DMA_MEMCPY,
- iop_chan->device->common.cap_mask))
- iop_chan_start_null_memcpy(iop_chan);
- else if (dma_has_cap(DMA_XOR,
- iop_chan->device->common.cap_mask))
- iop_chan_start_null_xor(iop_chan);
- else
- BUG();
- }
-
- return (idx > 0) ? idx : -ENOMEM;
-}
-
-static struct dma_async_tx_descriptor *
-iop_adma_prep_dma_interrupt(struct dma_chan *chan, unsigned long flags)
-{
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(chan);
- struct iop_adma_desc_slot *sw_desc, *grp_start;
- int slot_cnt, slots_per_op;
-
- dev_dbg(iop_chan->device->common.dev, "%s\n", __func__);
-
- spin_lock_bh(&iop_chan->lock);
- slot_cnt = iop_chan_interrupt_slot_count(&slots_per_op, iop_chan);
- sw_desc = iop_adma_alloc_slots(iop_chan, slot_cnt, slots_per_op);
- if (sw_desc) {
- grp_start = sw_desc->group_head;
- iop_desc_init_interrupt(grp_start, iop_chan);
- sw_desc->async_tx.flags = flags;
- }
- spin_unlock_bh(&iop_chan->lock);
-
- return sw_desc ? &sw_desc->async_tx : NULL;
-}
-
-static struct dma_async_tx_descriptor *
-iop_adma_prep_dma_memcpy(struct dma_chan *chan, dma_addr_t dma_dest,
- dma_addr_t dma_src, size_t len, unsigned long flags)
-{
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(chan);
- struct iop_adma_desc_slot *sw_desc, *grp_start;
- int slot_cnt, slots_per_op;
-
- if (unlikely(!len))
- return NULL;
- BUG_ON(len > IOP_ADMA_MAX_BYTE_COUNT);
-
- dev_dbg(iop_chan->device->common.dev, "%s len: %zu\n",
- __func__, len);
-
- spin_lock_bh(&iop_chan->lock);
- slot_cnt = iop_chan_memcpy_slot_count(len, &slots_per_op);
- sw_desc = iop_adma_alloc_slots(iop_chan, slot_cnt, slots_per_op);
- if (sw_desc) {
- grp_start = sw_desc->group_head;
- iop_desc_init_memcpy(grp_start, flags);
- iop_desc_set_byte_count(grp_start, iop_chan, len);
- iop_desc_set_dest_addr(grp_start, iop_chan, dma_dest);
- iop_desc_set_memcpy_src_addr(grp_start, dma_src);
- sw_desc->async_tx.flags = flags;
- }
- spin_unlock_bh(&iop_chan->lock);
-
- return sw_desc ? &sw_desc->async_tx : NULL;
-}
-
-static struct dma_async_tx_descriptor *
-iop_adma_prep_dma_xor(struct dma_chan *chan, dma_addr_t dma_dest,
- dma_addr_t *dma_src, unsigned int src_cnt, size_t len,
- unsigned long flags)
-{
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(chan);
- struct iop_adma_desc_slot *sw_desc, *grp_start;
- int slot_cnt, slots_per_op;
-
- if (unlikely(!len))
- return NULL;
- BUG_ON(len > IOP_ADMA_XOR_MAX_BYTE_COUNT);
-
- dev_dbg(iop_chan->device->common.dev,
- "%s src_cnt: %d len: %zu flags: %lx\n",
- __func__, src_cnt, len, flags);
-
- spin_lock_bh(&iop_chan->lock);
- slot_cnt = iop_chan_xor_slot_count(len, src_cnt, &slots_per_op);
- sw_desc = iop_adma_alloc_slots(iop_chan, slot_cnt, slots_per_op);
- if (sw_desc) {
- grp_start = sw_desc->group_head;
- iop_desc_init_xor(grp_start, src_cnt, flags);
- iop_desc_set_byte_count(grp_start, iop_chan, len);
- iop_desc_set_dest_addr(grp_start, iop_chan, dma_dest);
- sw_desc->async_tx.flags = flags;
- while (src_cnt--)
- iop_desc_set_xor_src_addr(grp_start, src_cnt,
- dma_src[src_cnt]);
- }
- spin_unlock_bh(&iop_chan->lock);
-
- return sw_desc ? &sw_desc->async_tx : NULL;
-}
-
-static struct dma_async_tx_descriptor *
-iop_adma_prep_dma_xor_val(struct dma_chan *chan, dma_addr_t *dma_src,
- unsigned int src_cnt, size_t len, u32 *result,
- unsigned long flags)
-{
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(chan);
- struct iop_adma_desc_slot *sw_desc, *grp_start;
- int slot_cnt, slots_per_op;
-
- if (unlikely(!len))
- return NULL;
-
- dev_dbg(iop_chan->device->common.dev, "%s src_cnt: %d len: %zu\n",
- __func__, src_cnt, len);
-
- spin_lock_bh(&iop_chan->lock);
- slot_cnt = iop_chan_zero_sum_slot_count(len, src_cnt, &slots_per_op);
- sw_desc = iop_adma_alloc_slots(iop_chan, slot_cnt, slots_per_op);
- if (sw_desc) {
- grp_start = sw_desc->group_head;
- iop_desc_init_zero_sum(grp_start, src_cnt, flags);
- iop_desc_set_zero_sum_byte_count(grp_start, len);
- grp_start->xor_check_result = result;
- pr_debug("\t%s: grp_start->xor_check_result: %p\n",
- __func__, grp_start->xor_check_result);
- sw_desc->async_tx.flags = flags;
- while (src_cnt--)
- iop_desc_set_zero_sum_src_addr(grp_start, src_cnt,
- dma_src[src_cnt]);
- }
- spin_unlock_bh(&iop_chan->lock);
-
- return sw_desc ? &sw_desc->async_tx : NULL;
-}
-
-static struct dma_async_tx_descriptor *
-iop_adma_prep_dma_pq(struct dma_chan *chan, dma_addr_t *dst, dma_addr_t *src,
- unsigned int src_cnt, const unsigned char *scf, size_t len,
- unsigned long flags)
-{
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(chan);
- struct iop_adma_desc_slot *sw_desc, *g;
- int slot_cnt, slots_per_op;
- int continue_srcs;
-
- if (unlikely(!len))
- return NULL;
- BUG_ON(len > IOP_ADMA_XOR_MAX_BYTE_COUNT);
-
- dev_dbg(iop_chan->device->common.dev,
- "%s src_cnt: %d len: %zu flags: %lx\n",
- __func__, src_cnt, len, flags);
-
- if (dmaf_p_disabled_continue(flags))
- continue_srcs = 1+src_cnt;
- else if (dmaf_continue(flags))
- continue_srcs = 3+src_cnt;
- else
- continue_srcs = 0+src_cnt;
-
- spin_lock_bh(&iop_chan->lock);
- slot_cnt = iop_chan_pq_slot_count(len, continue_srcs, &slots_per_op);
- sw_desc = iop_adma_alloc_slots(iop_chan, slot_cnt, slots_per_op);
- if (sw_desc) {
- int i;
-
- g = sw_desc->group_head;
- iop_desc_set_byte_count(g, iop_chan, len);
-
- /* even if P is disabled its destination address (bits
- * [3:0]) must match Q. It is ok if P points to an
- * invalid address, it won't be written.
- */
- if (flags & DMA_PREP_PQ_DISABLE_P)
- dst[0] = dst[1] & 0x7;
-
- iop_desc_set_pq_addr(g, dst);
- sw_desc->async_tx.flags = flags;
- for (i = 0; i < src_cnt; i++)
- iop_desc_set_pq_src_addr(g, i, src[i], scf[i]);
-
- /* if we are continuing a previous operation factor in
- * the old p and q values, see the comment for dma_maxpq
- * in include/linux/dmaengine.h
- */
- if (dmaf_p_disabled_continue(flags))
- iop_desc_set_pq_src_addr(g, i++, dst[1], 1);
- else if (dmaf_continue(flags)) {
- iop_desc_set_pq_src_addr(g, i++, dst[0], 0);
- iop_desc_set_pq_src_addr(g, i++, dst[1], 1);
- iop_desc_set_pq_src_addr(g, i++, dst[1], 0);
- }
- iop_desc_init_pq(g, i, flags);
- }
- spin_unlock_bh(&iop_chan->lock);
-
- return sw_desc ? &sw_desc->async_tx : NULL;
-}
-
-static struct dma_async_tx_descriptor *
-iop_adma_prep_dma_pq_val(struct dma_chan *chan, dma_addr_t *pq, dma_addr_t *src,
- unsigned int src_cnt, const unsigned char *scf,
- size_t len, enum sum_check_flags *pqres,
- unsigned long flags)
-{
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(chan);
- struct iop_adma_desc_slot *sw_desc, *g;
- int slot_cnt, slots_per_op;
-
- if (unlikely(!len))
- return NULL;
- BUG_ON(len > IOP_ADMA_XOR_MAX_BYTE_COUNT);
-
- dev_dbg(iop_chan->device->common.dev, "%s src_cnt: %d len: %zu\n",
- __func__, src_cnt, len);
-
- spin_lock_bh(&iop_chan->lock);
- slot_cnt = iop_chan_pq_zero_sum_slot_count(len, src_cnt + 2, &slots_per_op);
- sw_desc = iop_adma_alloc_slots(iop_chan, slot_cnt, slots_per_op);
- if (sw_desc) {
- /* for validate operations p and q are tagged onto the
- * end of the source list
- */
- int pq_idx = src_cnt;
-
- g = sw_desc->group_head;
- iop_desc_init_pq_zero_sum(g, src_cnt+2, flags);
- iop_desc_set_pq_zero_sum_byte_count(g, len);
- g->pq_check_result = pqres;
- pr_debug("\t%s: g->pq_check_result: %p\n",
- __func__, g->pq_check_result);
- sw_desc->async_tx.flags = flags;
- while (src_cnt--)
- iop_desc_set_pq_zero_sum_src_addr(g, src_cnt,
- src[src_cnt],
- scf[src_cnt]);
- iop_desc_set_pq_zero_sum_addr(g, pq_idx, src);
- }
- spin_unlock_bh(&iop_chan->lock);
-
- return sw_desc ? &sw_desc->async_tx : NULL;
-}
-
-static void iop_adma_free_chan_resources(struct dma_chan *chan)
-{
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(chan);
- struct iop_adma_desc_slot *iter, *_iter;
- int in_use_descs = 0;
-
- iop_adma_slot_cleanup(iop_chan);
-
- spin_lock_bh(&iop_chan->lock);
- list_for_each_entry_safe(iter, _iter, &iop_chan->chain,
- chain_node) {
- in_use_descs++;
- list_del(&iter->chain_node);
- }
- list_for_each_entry_safe_reverse(
- iter, _iter, &iop_chan->all_slots, slot_node) {
- list_del(&iter->slot_node);
- kfree(iter);
- iop_chan->slots_allocated--;
- }
- iop_chan->last_used = NULL;
-
- dev_dbg(iop_chan->device->common.dev, "%s slots_allocated %d\n",
- __func__, iop_chan->slots_allocated);
- spin_unlock_bh(&iop_chan->lock);
-
- /* one is ok since we left it on there on purpose */
- if (in_use_descs > 1)
- printk(KERN_ERR "IOP: Freeing %d in use descriptors!\n",
- in_use_descs - 1);
-}
-
-/**
- * iop_adma_status - poll the status of an ADMA transaction
- * @chan: ADMA channel handle
- * @cookie: ADMA transaction identifier
- * @txstate: a holder for the current state of the channel or NULL
- */
-static enum dma_status iop_adma_status(struct dma_chan *chan,
- dma_cookie_t cookie,
- struct dma_tx_state *txstate)
-{
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(chan);
- int ret;
-
- ret = dma_cookie_status(chan, cookie, txstate);
- if (ret == DMA_COMPLETE)
- return ret;
-
- iop_adma_slot_cleanup(iop_chan);
-
- return dma_cookie_status(chan, cookie, txstate);
-}
-
-static irqreturn_t iop_adma_eot_handler(int irq, void *data)
-{
- struct iop_adma_chan *chan = data;
-
- dev_dbg(chan->device->common.dev, "%s\n", __func__);
-
- tasklet_schedule(&chan->irq_tasklet);
-
- iop_adma_device_clear_eot_status(chan);
-
- return IRQ_HANDLED;
-}
-
-static irqreturn_t iop_adma_eoc_handler(int irq, void *data)
-{
- struct iop_adma_chan *chan = data;
-
- dev_dbg(chan->device->common.dev, "%s\n", __func__);
-
- tasklet_schedule(&chan->irq_tasklet);
-
- iop_adma_device_clear_eoc_status(chan);
-
- return IRQ_HANDLED;
-}
-
-static irqreturn_t iop_adma_err_handler(int irq, void *data)
-{
- struct iop_adma_chan *chan = data;
- unsigned long status = iop_chan_get_status(chan);
-
- dev_err(chan->device->common.dev,
- "error ( %s%s%s%s%s%s%s)\n",
- iop_is_err_int_parity(status, chan) ? "int_parity " : "",
- iop_is_err_mcu_abort(status, chan) ? "mcu_abort " : "",
- iop_is_err_int_tabort(status, chan) ? "int_tabort " : "",
- iop_is_err_int_mabort(status, chan) ? "int_mabort " : "",
- iop_is_err_pci_tabort(status, chan) ? "pci_tabort " : "",
- iop_is_err_pci_mabort(status, chan) ? "pci_mabort " : "",
- iop_is_err_split_tx(status, chan) ? "split_tx " : "");
-
- iop_adma_device_clear_err_status(chan);
-
- BUG();
-
- return IRQ_HANDLED;
-}
-
-static void iop_adma_issue_pending(struct dma_chan *chan)
-{
- struct iop_adma_chan *iop_chan = to_iop_adma_chan(chan);
-
- if (iop_chan->pending) {
- iop_chan->pending = 0;
- iop_chan_append(iop_chan);
- }
-}
-
-/*
- * Perform a transaction to verify the HW works.
- */
-#define IOP_ADMA_TEST_SIZE 2000
-
-static int iop_adma_memcpy_self_test(struct iop_adma_device *device)
-{
- int i;
- void *src, *dest;
- dma_addr_t src_dma, dest_dma;
- struct dma_chan *dma_chan;
- dma_cookie_t cookie;
- struct dma_async_tx_descriptor *tx;
- int err = 0;
- struct iop_adma_chan *iop_chan;
-
- dev_dbg(device->common.dev, "%s\n", __func__);
-
- src = kmalloc(IOP_ADMA_TEST_SIZE, GFP_KERNEL);
- if (!src)
- return -ENOMEM;
- dest = kzalloc(IOP_ADMA_TEST_SIZE, GFP_KERNEL);
- if (!dest) {
- kfree(src);
- return -ENOMEM;
- }
-
- /* Fill in src buffer */
- for (i = 0; i < IOP_ADMA_TEST_SIZE; i++)
- ((u8 *) src)[i] = (u8)i;
-
- /* Start copy, using first DMA channel */
- dma_chan = container_of(device->common.channels.next,
- struct dma_chan,
- device_node);
- if (iop_adma_alloc_chan_resources(dma_chan) < 1) {
- err = -ENODEV;
- goto out;
- }
-
- dest_dma = dma_map_single(dma_chan->device->dev, dest,
- IOP_ADMA_TEST_SIZE, DMA_FROM_DEVICE);
- src_dma = dma_map_single(dma_chan->device->dev, src,
- IOP_ADMA_TEST_SIZE, DMA_TO_DEVICE);
- tx = iop_adma_prep_dma_memcpy(dma_chan, dest_dma, src_dma,
- IOP_ADMA_TEST_SIZE,
- DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
-
- cookie = iop_adma_tx_submit(tx);
- iop_adma_issue_pending(dma_chan);
- msleep(1);
-
- if (iop_adma_status(dma_chan, cookie, NULL) !=
- DMA_COMPLETE) {
- dev_err(dma_chan->device->dev,
- "Self-test copy timed out, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
- iop_chan = to_iop_adma_chan(dma_chan);
- dma_sync_single_for_cpu(&iop_chan->device->pdev->dev, dest_dma,
- IOP_ADMA_TEST_SIZE, DMA_FROM_DEVICE);
- if (memcmp(src, dest, IOP_ADMA_TEST_SIZE)) {
- dev_err(dma_chan->device->dev,
- "Self-test copy failed compare, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
-free_resources:
- iop_adma_free_chan_resources(dma_chan);
-out:
- kfree(src);
- kfree(dest);
- return err;
-}
-
-#define IOP_ADMA_NUM_SRC_TEST 4 /* must be <= 15 */
-static int
-iop_adma_xor_val_self_test(struct iop_adma_device *device)
-{
- int i, src_idx;
- struct page *dest;
- struct page *xor_srcs[IOP_ADMA_NUM_SRC_TEST];
- struct page *zero_sum_srcs[IOP_ADMA_NUM_SRC_TEST + 1];
- dma_addr_t dma_srcs[IOP_ADMA_NUM_SRC_TEST + 1];
- dma_addr_t dest_dma;
- struct dma_async_tx_descriptor *tx;
- struct dma_chan *dma_chan;
- dma_cookie_t cookie;
- u8 cmp_byte = 0;
- u32 cmp_word;
- u32 zero_sum_result;
- int err = 0;
- struct iop_adma_chan *iop_chan;
-
- dev_dbg(device->common.dev, "%s\n", __func__);
-
- for (src_idx = 0; src_idx < IOP_ADMA_NUM_SRC_TEST; src_idx++) {
- xor_srcs[src_idx] = alloc_page(GFP_KERNEL);
- if (!xor_srcs[src_idx]) {
- while (src_idx--)
- __free_page(xor_srcs[src_idx]);
- return -ENOMEM;
- }
- }
-
- dest = alloc_page(GFP_KERNEL);
- if (!dest) {
- while (src_idx--)
- __free_page(xor_srcs[src_idx]);
- return -ENOMEM;
- }
-
- /* Fill in src buffers */
- for (src_idx = 0; src_idx < IOP_ADMA_NUM_SRC_TEST; src_idx++) {
- u8 *ptr = page_address(xor_srcs[src_idx]);
- for (i = 0; i < PAGE_SIZE; i++)
- ptr[i] = (1 << src_idx);
- }
-
- for (src_idx = 0; src_idx < IOP_ADMA_NUM_SRC_TEST; src_idx++)
- cmp_byte ^= (u8) (1 << src_idx);
-
- cmp_word = (cmp_byte << 24) | (cmp_byte << 16) |
- (cmp_byte << 8) | cmp_byte;
-
- memset(page_address(dest), 0, PAGE_SIZE);
-
- dma_chan = container_of(device->common.channels.next,
- struct dma_chan,
- device_node);
- if (iop_adma_alloc_chan_resources(dma_chan) < 1) {
- err = -ENODEV;
- goto out;
- }
-
- /* test xor */
- dest_dma = dma_map_page(dma_chan->device->dev, dest, 0,
- PAGE_SIZE, DMA_FROM_DEVICE);
- for (i = 0; i < IOP_ADMA_NUM_SRC_TEST; i++)
- dma_srcs[i] = dma_map_page(dma_chan->device->dev, xor_srcs[i],
- 0, PAGE_SIZE, DMA_TO_DEVICE);
- tx = iop_adma_prep_dma_xor(dma_chan, dest_dma, dma_srcs,
- IOP_ADMA_NUM_SRC_TEST, PAGE_SIZE,
- DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
-
- cookie = iop_adma_tx_submit(tx);
- iop_adma_issue_pending(dma_chan);
- msleep(8);
-
- if (iop_adma_status(dma_chan, cookie, NULL) !=
- DMA_COMPLETE) {
- dev_err(dma_chan->device->dev,
- "Self-test xor timed out, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
- iop_chan = to_iop_adma_chan(dma_chan);
- dma_sync_single_for_cpu(&iop_chan->device->pdev->dev, dest_dma,
- PAGE_SIZE, DMA_FROM_DEVICE);
- for (i = 0; i < (PAGE_SIZE / sizeof(u32)); i++) {
- u32 *ptr = page_address(dest);
- if (ptr[i] != cmp_word) {
- dev_err(dma_chan->device->dev,
- "Self-test xor failed compare, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
- }
- dma_sync_single_for_device(&iop_chan->device->pdev->dev, dest_dma,
- PAGE_SIZE, DMA_TO_DEVICE);
-
- /* skip zero sum if the capability is not present */
- if (!dma_has_cap(DMA_XOR_VAL, dma_chan->device->cap_mask))
- goto free_resources;
-
- /* zero sum the sources with the destintation page */
- for (i = 0; i < IOP_ADMA_NUM_SRC_TEST; i++)
- zero_sum_srcs[i] = xor_srcs[i];
- zero_sum_srcs[i] = dest;
-
- zero_sum_result = 1;
-
- for (i = 0; i < IOP_ADMA_NUM_SRC_TEST + 1; i++)
- dma_srcs[i] = dma_map_page(dma_chan->device->dev,
- zero_sum_srcs[i], 0, PAGE_SIZE,
- DMA_TO_DEVICE);
- tx = iop_adma_prep_dma_xor_val(dma_chan, dma_srcs,
- IOP_ADMA_NUM_SRC_TEST + 1, PAGE_SIZE,
- &zero_sum_result,
- DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
-
- cookie = iop_adma_tx_submit(tx);
- iop_adma_issue_pending(dma_chan);
- msleep(8);
-
- if (iop_adma_status(dma_chan, cookie, NULL) != DMA_COMPLETE) {
- dev_err(dma_chan->device->dev,
- "Self-test zero sum timed out, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
- if (zero_sum_result != 0) {
- dev_err(dma_chan->device->dev,
- "Self-test zero sum failed compare, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
- /* test for non-zero parity sum */
- zero_sum_result = 0;
- for (i = 0; i < IOP_ADMA_NUM_SRC_TEST + 1; i++)
- dma_srcs[i] = dma_map_page(dma_chan->device->dev,
- zero_sum_srcs[i], 0, PAGE_SIZE,
- DMA_TO_DEVICE);
- tx = iop_adma_prep_dma_xor_val(dma_chan, dma_srcs,
- IOP_ADMA_NUM_SRC_TEST + 1, PAGE_SIZE,
- &zero_sum_result,
- DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
-
- cookie = iop_adma_tx_submit(tx);
- iop_adma_issue_pending(dma_chan);
- msleep(8);
-
- if (iop_adma_status(dma_chan, cookie, NULL) != DMA_COMPLETE) {
- dev_err(dma_chan->device->dev,
- "Self-test non-zero sum timed out, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
- if (zero_sum_result != 1) {
- dev_err(dma_chan->device->dev,
- "Self-test non-zero sum failed compare, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
-free_resources:
- iop_adma_free_chan_resources(dma_chan);
-out:
- src_idx = IOP_ADMA_NUM_SRC_TEST;
- while (src_idx--)
- __free_page(xor_srcs[src_idx]);
- __free_page(dest);
- return err;
-}
-
-#ifdef CONFIG_RAID6_PQ
-static int
-iop_adma_pq_zero_sum_self_test(struct iop_adma_device *device)
-{
- /* combined sources, software pq results, and extra hw pq results */
- struct page *pq[IOP_ADMA_NUM_SRC_TEST+2+2];
- /* ptr to the extra hw pq buffers defined above */
- struct page **pq_hw = &pq[IOP_ADMA_NUM_SRC_TEST+2];
- /* address conversion buffers (dma_map / page_address) */
- void *pq_sw[IOP_ADMA_NUM_SRC_TEST+2];
- dma_addr_t pq_src[IOP_ADMA_NUM_SRC_TEST+2];
- dma_addr_t *pq_dest = &pq_src[IOP_ADMA_NUM_SRC_TEST];
-
- int i;
- struct dma_async_tx_descriptor *tx;
- struct dma_chan *dma_chan;
- dma_cookie_t cookie;
- u32 zero_sum_result;
- int err = 0;
- struct device *dev;
-
- dev_dbg(device->common.dev, "%s\n", __func__);
-
- for (i = 0; i < ARRAY_SIZE(pq); i++) {
- pq[i] = alloc_page(GFP_KERNEL);
- if (!pq[i]) {
- while (i--)
- __free_page(pq[i]);
- return -ENOMEM;
- }
- }
-
- /* Fill in src buffers */
- for (i = 0; i < IOP_ADMA_NUM_SRC_TEST; i++) {
- pq_sw[i] = page_address(pq[i]);
- memset(pq_sw[i], 0x11111111 * (1<<i), PAGE_SIZE);
- }
- pq_sw[i] = page_address(pq[i]);
- pq_sw[i+1] = page_address(pq[i+1]);
-
- dma_chan = container_of(device->common.channels.next,
- struct dma_chan,
- device_node);
- if (iop_adma_alloc_chan_resources(dma_chan) < 1) {
- err = -ENODEV;
- goto out;
- }
-
- dev = dma_chan->device->dev;
-
- /* initialize the dests */
- memset(page_address(pq_hw[0]), 0 , PAGE_SIZE);
- memset(page_address(pq_hw[1]), 0 , PAGE_SIZE);
-
- /* test pq */
- pq_dest[0] = dma_map_page(dev, pq_hw[0], 0, PAGE_SIZE, DMA_FROM_DEVICE);
- pq_dest[1] = dma_map_page(dev, pq_hw[1], 0, PAGE_SIZE, DMA_FROM_DEVICE);
- for (i = 0; i < IOP_ADMA_NUM_SRC_TEST; i++)
- pq_src[i] = dma_map_page(dev, pq[i], 0, PAGE_SIZE,
- DMA_TO_DEVICE);
-
- tx = iop_adma_prep_dma_pq(dma_chan, pq_dest, pq_src,
- IOP_ADMA_NUM_SRC_TEST, (u8 *)raid6_gfexp,
- PAGE_SIZE,
- DMA_PREP_INTERRUPT |
- DMA_CTRL_ACK);
-
- cookie = iop_adma_tx_submit(tx);
- iop_adma_issue_pending(dma_chan);
- msleep(8);
-
- if (iop_adma_status(dma_chan, cookie, NULL) !=
- DMA_COMPLETE) {
- dev_err(dev, "Self-test pq timed out, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
- raid6_call.gen_syndrome(IOP_ADMA_NUM_SRC_TEST+2, PAGE_SIZE, pq_sw);
-
- if (memcmp(pq_sw[IOP_ADMA_NUM_SRC_TEST],
- page_address(pq_hw[0]), PAGE_SIZE) != 0) {
- dev_err(dev, "Self-test p failed compare, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
- if (memcmp(pq_sw[IOP_ADMA_NUM_SRC_TEST+1],
- page_address(pq_hw[1]), PAGE_SIZE) != 0) {
- dev_err(dev, "Self-test q failed compare, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
- /* test correct zero sum using the software generated pq values */
- for (i = 0; i < IOP_ADMA_NUM_SRC_TEST + 2; i++)
- pq_src[i] = dma_map_page(dev, pq[i], 0, PAGE_SIZE,
- DMA_TO_DEVICE);
-
- zero_sum_result = ~0;
- tx = iop_adma_prep_dma_pq_val(dma_chan, &pq_src[IOP_ADMA_NUM_SRC_TEST],
- pq_src, IOP_ADMA_NUM_SRC_TEST,
- raid6_gfexp, PAGE_SIZE, &zero_sum_result,
- DMA_PREP_INTERRUPT|DMA_CTRL_ACK);
-
- cookie = iop_adma_tx_submit(tx);
- iop_adma_issue_pending(dma_chan);
- msleep(8);
-
- if (iop_adma_status(dma_chan, cookie, NULL) !=
- DMA_COMPLETE) {
- dev_err(dev, "Self-test pq-zero-sum timed out, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
- if (zero_sum_result != 0) {
- dev_err(dev, "Self-test pq-zero-sum failed to validate: %x\n",
- zero_sum_result);
- err = -ENODEV;
- goto free_resources;
- }
-
- /* test incorrect zero sum */
- i = IOP_ADMA_NUM_SRC_TEST;
- memset(pq_sw[i] + 100, 0, 100);
- memset(pq_sw[i+1] + 200, 0, 200);
- for (i = 0; i < IOP_ADMA_NUM_SRC_TEST + 2; i++)
- pq_src[i] = dma_map_page(dev, pq[i], 0, PAGE_SIZE,
- DMA_TO_DEVICE);
-
- zero_sum_result = 0;
- tx = iop_adma_prep_dma_pq_val(dma_chan, &pq_src[IOP_ADMA_NUM_SRC_TEST],
- pq_src, IOP_ADMA_NUM_SRC_TEST,
- raid6_gfexp, PAGE_SIZE, &zero_sum_result,
- DMA_PREP_INTERRUPT|DMA_CTRL_ACK);
-
- cookie = iop_adma_tx_submit(tx);
- iop_adma_issue_pending(dma_chan);
- msleep(8);
-
- if (iop_adma_status(dma_chan, cookie, NULL) !=
- DMA_COMPLETE) {
- dev_err(dev, "Self-test !pq-zero-sum timed out, disabling\n");
- err = -ENODEV;
- goto free_resources;
- }
-
- if (zero_sum_result != (SUM_CHECK_P_RESULT | SUM_CHECK_Q_RESULT)) {
- dev_err(dev, "Self-test !pq-zero-sum failed to validate: %x\n",
- zero_sum_result);
- err = -ENODEV;
- goto free_resources;
- }
-
-free_resources:
- iop_adma_free_chan_resources(dma_chan);
-out:
- i = ARRAY_SIZE(pq);
- while (i--)
- __free_page(pq[i]);
- return err;
-}
-#endif
-
-static int iop_adma_remove(struct platform_device *dev)
-{
- struct iop_adma_device *device = platform_get_drvdata(dev);
- struct dma_chan *chan, *_chan;
- struct iop_adma_chan *iop_chan;
- struct iop_adma_platform_data *plat_data = dev_get_platdata(&dev->dev);
-
- dma_async_device_unregister(&device->common);
-
- dma_free_coherent(&dev->dev, plat_data->pool_size,
- device->dma_desc_pool_virt, device->dma_desc_pool);
-
- list_for_each_entry_safe(chan, _chan, &device->common.channels,
- device_node) {
- iop_chan = to_iop_adma_chan(chan);
- list_del(&chan->device_node);
- kfree(iop_chan);
- }
- kfree(device);
-
- return 0;
-}
-
-static int iop_adma_probe(struct platform_device *pdev)
-{
- struct resource *res;
- int ret = 0, i;
- struct iop_adma_device *adev;
- struct iop_adma_chan *iop_chan;
- struct dma_device *dma_dev;
- struct iop_adma_platform_data *plat_data = dev_get_platdata(&pdev->dev);
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!res)
- return -ENODEV;
-
- if (!devm_request_mem_region(&pdev->dev, res->start,
- resource_size(res), pdev->name))
- return -EBUSY;
-
- adev = kzalloc(sizeof(*adev), GFP_KERNEL);
- if (!adev)
- return -ENOMEM;
- dma_dev = &adev->common;
-
- /* allocate coherent memory for hardware descriptors
- * note: writecombine gives slightly better performance, but
- * requires that we explicitly flush the writes
- */
- adev->dma_desc_pool_virt = dma_alloc_wc(&pdev->dev,
- plat_data->pool_size,
- &adev->dma_desc_pool,
- GFP_KERNEL);
- if (!adev->dma_desc_pool_virt) {
- ret = -ENOMEM;
- goto err_free_adev;
- }
-
- dev_dbg(&pdev->dev, "%s: allocated descriptor pool virt %p phys %pad\n",
- __func__, adev->dma_desc_pool_virt, &adev->dma_desc_pool);
-
- adev->id = plat_data->hw_id;
-
- /* discover transaction capabilites from the platform data */
- dma_dev->cap_mask = plat_data->cap_mask;
-
- adev->pdev = pdev;
- platform_set_drvdata(pdev, adev);
-
- INIT_LIST_HEAD(&dma_dev->channels);
-
- /* set base routines */
- dma_dev->device_alloc_chan_resources = iop_adma_alloc_chan_resources;
- dma_dev->device_free_chan_resources = iop_adma_free_chan_resources;
- dma_dev->device_tx_status = iop_adma_status;
- dma_dev->device_issue_pending = iop_adma_issue_pending;
- dma_dev->dev = &pdev->dev;
-
- /* set prep routines based on capability */
- if (dma_has_cap(DMA_MEMCPY, dma_dev->cap_mask))
- dma_dev->device_prep_dma_memcpy = iop_adma_prep_dma_memcpy;
- if (dma_has_cap(DMA_XOR, dma_dev->cap_mask)) {
- dma_dev->max_xor = iop_adma_get_max_xor();
- dma_dev->device_prep_dma_xor = iop_adma_prep_dma_xor;
- }
- if (dma_has_cap(DMA_XOR_VAL, dma_dev->cap_mask))
- dma_dev->device_prep_dma_xor_val =
- iop_adma_prep_dma_xor_val;
- if (dma_has_cap(DMA_PQ, dma_dev->cap_mask)) {
- dma_set_maxpq(dma_dev, iop_adma_get_max_pq(), 0);
- dma_dev->device_prep_dma_pq = iop_adma_prep_dma_pq;
- }
- if (dma_has_cap(DMA_PQ_VAL, dma_dev->cap_mask))
- dma_dev->device_prep_dma_pq_val =
- iop_adma_prep_dma_pq_val;
- if (dma_has_cap(DMA_INTERRUPT, dma_dev->cap_mask))
- dma_dev->device_prep_dma_interrupt =
- iop_adma_prep_dma_interrupt;
-
- iop_chan = kzalloc(sizeof(*iop_chan), GFP_KERNEL);
- if (!iop_chan) {
- ret = -ENOMEM;
- goto err_free_dma;
- }
- iop_chan->device = adev;
-
- iop_chan->mmr_base = devm_ioremap(&pdev->dev, res->start,
- resource_size(res));
- if (!iop_chan->mmr_base) {
- ret = -ENOMEM;
- goto err_free_iop_chan;
- }
- tasklet_setup(&iop_chan->irq_tasklet, iop_adma_tasklet);
-
- /* clear errors before enabling interrupts */
- iop_adma_device_clear_err_status(iop_chan);
-
- for (i = 0; i < 3; i++) {
- static const irq_handler_t handler[] = {
- iop_adma_eot_handler,
- iop_adma_eoc_handler,
- iop_adma_err_handler
- };
- int irq = platform_get_irq(pdev, i);
- if (irq < 0) {
- ret = -ENXIO;
- goto err_free_iop_chan;
- } else {
- ret = devm_request_irq(&pdev->dev, irq,
- handler[i], 0, pdev->name, iop_chan);
- if (ret)
- goto err_free_iop_chan;
- }
- }
-
- spin_lock_init(&iop_chan->lock);
- INIT_LIST_HEAD(&iop_chan->chain);
- INIT_LIST_HEAD(&iop_chan->all_slots);
- iop_chan->common.device = dma_dev;
- dma_cookie_init(&iop_chan->common);
- list_add_tail(&iop_chan->common.device_node, &dma_dev->channels);
-
- if (dma_has_cap(DMA_MEMCPY, dma_dev->cap_mask)) {
- ret = iop_adma_memcpy_self_test(adev);
- dev_dbg(&pdev->dev, "memcpy self test returned %d\n", ret);
- if (ret)
- goto err_free_iop_chan;
- }
-
- if (dma_has_cap(DMA_XOR, dma_dev->cap_mask)) {
- ret = iop_adma_xor_val_self_test(adev);
- dev_dbg(&pdev->dev, "xor self test returned %d\n", ret);
- if (ret)
- goto err_free_iop_chan;
- }
-
- if (dma_has_cap(DMA_PQ, dma_dev->cap_mask) &&
- dma_has_cap(DMA_PQ_VAL, dma_dev->cap_mask)) {
- #ifdef CONFIG_RAID6_PQ
- ret = iop_adma_pq_zero_sum_self_test(adev);
- dev_dbg(&pdev->dev, "pq self test returned %d\n", ret);
- #else
- /* can not test raid6, so do not publish capability */
- dma_cap_clear(DMA_PQ, dma_dev->cap_mask);
- dma_cap_clear(DMA_PQ_VAL, dma_dev->cap_mask);
- ret = 0;
- #endif
- if (ret)
- goto err_free_iop_chan;
- }
-
- dev_info(&pdev->dev, "Intel(R) IOP: ( %s%s%s%s%s%s)\n",
- dma_has_cap(DMA_PQ, dma_dev->cap_mask) ? "pq " : "",
- dma_has_cap(DMA_PQ_VAL, dma_dev->cap_mask) ? "pq_val " : "",
- dma_has_cap(DMA_XOR, dma_dev->cap_mask) ? "xor " : "",
- dma_has_cap(DMA_XOR_VAL, dma_dev->cap_mask) ? "xor_val " : "",
- dma_has_cap(DMA_MEMCPY, dma_dev->cap_mask) ? "cpy " : "",
- dma_has_cap(DMA_INTERRUPT, dma_dev->cap_mask) ? "intr " : "");
-
- dma_async_device_register(dma_dev);
- goto out;
-
- err_free_iop_chan:
- kfree(iop_chan);
- err_free_dma:
- dma_free_coherent(&adev->pdev->dev, plat_data->pool_size,
- adev->dma_desc_pool_virt, adev->dma_desc_pool);
- err_free_adev:
- kfree(adev);
- out:
- return ret;
-}
-
-static void iop_chan_start_null_memcpy(struct iop_adma_chan *iop_chan)
-{
- struct iop_adma_desc_slot *sw_desc, *grp_start;
- dma_cookie_t cookie;
- int slot_cnt, slots_per_op;
-
- dev_dbg(iop_chan->device->common.dev, "%s\n", __func__);
-
- spin_lock_bh(&iop_chan->lock);
- slot_cnt = iop_chan_memcpy_slot_count(0, &slots_per_op);
- sw_desc = iop_adma_alloc_slots(iop_chan, slot_cnt, slots_per_op);
- if (sw_desc) {
- grp_start = sw_desc->group_head;
-
- list_splice_init(&sw_desc->tx_list, &iop_chan->chain);
- async_tx_ack(&sw_desc->async_tx);
- iop_desc_init_memcpy(grp_start, 0);
- iop_desc_set_byte_count(grp_start, iop_chan, 0);
- iop_desc_set_dest_addr(grp_start, iop_chan, 0);
- iop_desc_set_memcpy_src_addr(grp_start, 0);
-
- cookie = dma_cookie_assign(&sw_desc->async_tx);
-
- /* initialize the completed cookie to be less than
- * the most recently used cookie
- */
- iop_chan->common.completed_cookie = cookie - 1;
-
- /* channel should not be busy */
- BUG_ON(iop_chan_is_busy(iop_chan));
-
- /* clear any prior error-status bits */
- iop_adma_device_clear_err_status(iop_chan);
-
- /* disable operation */
- iop_chan_disable(iop_chan);
-
- /* set the descriptor address */
- iop_chan_set_next_descriptor(iop_chan, sw_desc->async_tx.phys);
-
- /* 1/ don't add pre-chained descriptors
- * 2/ dummy read to flush next_desc write
- */
- BUG_ON(iop_desc_get_next_desc(sw_desc));
-
- /* run the descriptor */
- iop_chan_enable(iop_chan);
- } else
- dev_err(iop_chan->device->common.dev,
- "failed to allocate null descriptor\n");
- spin_unlock_bh(&iop_chan->lock);
-}
-
-static void iop_chan_start_null_xor(struct iop_adma_chan *iop_chan)
-{
- struct iop_adma_desc_slot *sw_desc, *grp_start;
- dma_cookie_t cookie;
- int slot_cnt, slots_per_op;
-
- dev_dbg(iop_chan->device->common.dev, "%s\n", __func__);
-
- spin_lock_bh(&iop_chan->lock);
- slot_cnt = iop_chan_xor_slot_count(0, 2, &slots_per_op);
- sw_desc = iop_adma_alloc_slots(iop_chan, slot_cnt, slots_per_op);
- if (sw_desc) {
- grp_start = sw_desc->group_head;
- list_splice_init(&sw_desc->tx_list, &iop_chan->chain);
- async_tx_ack(&sw_desc->async_tx);
- iop_desc_init_null_xor(grp_start, 2, 0);
- iop_desc_set_byte_count(grp_start, iop_chan, 0);
- iop_desc_set_dest_addr(grp_start, iop_chan, 0);
- iop_desc_set_xor_src_addr(grp_start, 0, 0);
- iop_desc_set_xor_src_addr(grp_start, 1, 0);
-
- cookie = dma_cookie_assign(&sw_desc->async_tx);
-
- /* initialize the completed cookie to be less than
- * the most recently used cookie
- */
- iop_chan->common.completed_cookie = cookie - 1;
-
- /* channel should not be busy */
- BUG_ON(iop_chan_is_busy(iop_chan));
-
- /* clear any prior error-status bits */
- iop_adma_device_clear_err_status(iop_chan);
-
- /* disable operation */
- iop_chan_disable(iop_chan);
-
- /* set the descriptor address */
- iop_chan_set_next_descriptor(iop_chan, sw_desc->async_tx.phys);
-
- /* 1/ don't add pre-chained descriptors
- * 2/ dummy read to flush next_desc write
- */
- BUG_ON(iop_desc_get_next_desc(sw_desc));
-
- /* run the descriptor */
- iop_chan_enable(iop_chan);
- } else
- dev_err(iop_chan->device->common.dev,
- "failed to allocate null descriptor\n");
- spin_unlock_bh(&iop_chan->lock);
-}
-
-static struct platform_driver iop_adma_driver = {
- .probe = iop_adma_probe,
- .remove = iop_adma_remove,
- .driver = {
- .name = "iop-adma",
- },
-};
-
-module_platform_driver(iop_adma_driver);
-
-MODULE_AUTHOR("Intel Corporation");
-MODULE_DESCRIPTION("IOP ADMA Engine Driver");
-MODULE_LICENSE("GPL");
-MODULE_ALIAS("platform:iop-adma");
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * Copyright © 2006, Intel Corporation.
- */
-#ifndef _ADMA_H
-#define _ADMA_H
-#include <linux/types.h>
-#include <linux/io.h>
-#include <linux/platform_data/dma-iop32x.h>
-
-/* Memory copy units */
-#define DMA_CCR(chan) (chan->mmr_base + 0x0)
-#define DMA_CSR(chan) (chan->mmr_base + 0x4)
-#define DMA_DAR(chan) (chan->mmr_base + 0xc)
-#define DMA_NDAR(chan) (chan->mmr_base + 0x10)
-#define DMA_PADR(chan) (chan->mmr_base + 0x14)
-#define DMA_PUADR(chan) (chan->mmr_base + 0x18)
-#define DMA_LADR(chan) (chan->mmr_base + 0x1c)
-#define DMA_BCR(chan) (chan->mmr_base + 0x20)
-#define DMA_DCR(chan) (chan->mmr_base + 0x24)
-
-/* Application accelerator unit */
-#define AAU_ACR(chan) (chan->mmr_base + 0x0)
-#define AAU_ASR(chan) (chan->mmr_base + 0x4)
-#define AAU_ADAR(chan) (chan->mmr_base + 0x8)
-#define AAU_ANDAR(chan) (chan->mmr_base + 0xc)
-#define AAU_SAR(src, chan) (chan->mmr_base + (0x10 + ((src) << 2)))
-#define AAU_DAR(chan) (chan->mmr_base + 0x20)
-#define AAU_ABCR(chan) (chan->mmr_base + 0x24)
-#define AAU_ADCR(chan) (chan->mmr_base + 0x28)
-#define AAU_SAR_EDCR(src_edc) (chan->mmr_base + (0x02c + ((src_edc-4) << 2)))
-#define AAU_EDCR0_IDX 8
-#define AAU_EDCR1_IDX 17
-#define AAU_EDCR2_IDX 26
-
-struct iop3xx_aau_desc_ctrl {
- unsigned int int_en:1;
- unsigned int blk1_cmd_ctrl:3;
- unsigned int blk2_cmd_ctrl:3;
- unsigned int blk3_cmd_ctrl:3;
- unsigned int blk4_cmd_ctrl:3;
- unsigned int blk5_cmd_ctrl:3;
- unsigned int blk6_cmd_ctrl:3;
- unsigned int blk7_cmd_ctrl:3;
- unsigned int blk8_cmd_ctrl:3;
- unsigned int blk_ctrl:2;
- unsigned int dual_xor_en:1;
- unsigned int tx_complete:1;
- unsigned int zero_result_err:1;
- unsigned int zero_result_en:1;
- unsigned int dest_write_en:1;
-};
-
-struct iop3xx_aau_e_desc_ctrl {
- unsigned int reserved:1;
- unsigned int blk1_cmd_ctrl:3;
- unsigned int blk2_cmd_ctrl:3;
- unsigned int blk3_cmd_ctrl:3;
- unsigned int blk4_cmd_ctrl:3;
- unsigned int blk5_cmd_ctrl:3;
- unsigned int blk6_cmd_ctrl:3;
- unsigned int blk7_cmd_ctrl:3;
- unsigned int blk8_cmd_ctrl:3;
- unsigned int reserved2:7;
-};
-
-struct iop3xx_dma_desc_ctrl {
- unsigned int pci_transaction:4;
- unsigned int int_en:1;
- unsigned int dac_cycle_en:1;
- unsigned int mem_to_mem_en:1;
- unsigned int crc_data_tx_en:1;
- unsigned int crc_gen_en:1;
- unsigned int crc_seed_dis:1;
- unsigned int reserved:21;
- unsigned int crc_tx_complete:1;
-};
-
-struct iop3xx_desc_dma {
- u32 next_desc;
- union {
- u32 pci_src_addr;
- u32 pci_dest_addr;
- u32 src_addr;
- };
- union {
- u32 upper_pci_src_addr;
- u32 upper_pci_dest_addr;
- };
- union {
- u32 local_pci_src_addr;
- u32 local_pci_dest_addr;
- u32 dest_addr;
- };
- u32 byte_count;
- union {
- u32 desc_ctrl;
- struct iop3xx_dma_desc_ctrl desc_ctrl_field;
- };
- u32 crc_addr;
-};
-
-struct iop3xx_desc_aau {
- u32 next_desc;
- u32 src[4];
- u32 dest_addr;
- u32 byte_count;
- union {
- u32 desc_ctrl;
- struct iop3xx_aau_desc_ctrl desc_ctrl_field;
- };
- union {
- u32 src_addr;
- u32 e_desc_ctrl;
- struct iop3xx_aau_e_desc_ctrl e_desc_ctrl_field;
- } src_edc[31];
-};
-
-struct iop3xx_aau_gfmr {
- unsigned int gfmr1:8;
- unsigned int gfmr2:8;
- unsigned int gfmr3:8;
- unsigned int gfmr4:8;
-};
-
-struct iop3xx_desc_pq_xor {
- u32 next_desc;
- u32 src[3];
- union {
- u32 data_mult1;
- struct iop3xx_aau_gfmr data_mult1_field;
- };
- u32 dest_addr;
- u32 byte_count;
- union {
- u32 desc_ctrl;
- struct iop3xx_aau_desc_ctrl desc_ctrl_field;
- };
- union {
- u32 src_addr;
- u32 e_desc_ctrl;
- struct iop3xx_aau_e_desc_ctrl e_desc_ctrl_field;
- u32 data_multiplier;
- struct iop3xx_aau_gfmr data_mult_field;
- u32 reserved;
- } src_edc_gfmr[19];
-};
-
-struct iop3xx_desc_dual_xor {
- u32 next_desc;
- u32 src0_addr;
- u32 src1_addr;
- u32 h_src_addr;
- u32 d_src_addr;
- u32 h_dest_addr;
- u32 byte_count;
- union {
- u32 desc_ctrl;
- struct iop3xx_aau_desc_ctrl desc_ctrl_field;
- };
- u32 d_dest_addr;
-};
-
-union iop3xx_desc {
- struct iop3xx_desc_aau *aau;
- struct iop3xx_desc_dma *dma;
- struct iop3xx_desc_pq_xor *pq_xor;
- struct iop3xx_desc_dual_xor *dual_xor;
- void *ptr;
-};
-
-/* No support for p+q operations */
-static inline int
-iop_chan_pq_slot_count(size_t len, int src_cnt, int *slots_per_op)
-{
- BUG();
- return 0;
-}
-
-static inline void
-iop_desc_init_pq(struct iop_adma_desc_slot *desc, int src_cnt,
- unsigned long flags)
-{
- BUG();
-}
-
-static inline void
-iop_desc_set_pq_addr(struct iop_adma_desc_slot *desc, dma_addr_t *addr)
-{
- BUG();
-}
-
-static inline void
-iop_desc_set_pq_src_addr(struct iop_adma_desc_slot *desc, int src_idx,
- dma_addr_t addr, unsigned char coef)
-{
- BUG();
-}
-
-static inline int
-iop_chan_pq_zero_sum_slot_count(size_t len, int src_cnt, int *slots_per_op)
-{
- BUG();
- return 0;
-}
-
-static inline void
-iop_desc_init_pq_zero_sum(struct iop_adma_desc_slot *desc, int src_cnt,
- unsigned long flags)
-{
- BUG();
-}
-
-static inline void
-iop_desc_set_pq_zero_sum_byte_count(struct iop_adma_desc_slot *desc, u32 len)
-{
- BUG();
-}
-
-#define iop_desc_set_pq_zero_sum_src_addr iop_desc_set_pq_src_addr
-
-static inline void
-iop_desc_set_pq_zero_sum_addr(struct iop_adma_desc_slot *desc, int pq_idx,
- dma_addr_t *src)
-{
- BUG();
-}
-
-static inline int iop_adma_get_max_xor(void)
-{
- return 32;
-}
-
-static inline int iop_adma_get_max_pq(void)
-{
- BUG();
- return 0;
-}
-
-static inline u32 iop_chan_get_current_descriptor(struct iop_adma_chan *chan)
-{
- int id = chan->device->id;
-
- switch (id) {
- case DMA0_ID:
- case DMA1_ID:
- return __raw_readl(DMA_DAR(chan));
- case AAU_ID:
- return __raw_readl(AAU_ADAR(chan));
- default:
- BUG();
- }
- return 0;
-}
-
-static inline void iop_chan_set_next_descriptor(struct iop_adma_chan *chan,
- u32 next_desc_addr)
-{
- int id = chan->device->id;
-
- switch (id) {
- case DMA0_ID:
- case DMA1_ID:
- __raw_writel(next_desc_addr, DMA_NDAR(chan));
- break;
- case AAU_ID:
- __raw_writel(next_desc_addr, AAU_ANDAR(chan));
- break;
- }
-
-}
-
-#define IOP_ADMA_STATUS_BUSY (1 << 10)
-#define IOP_ADMA_ZERO_SUM_MAX_BYTE_COUNT (1024)
-#define IOP_ADMA_XOR_MAX_BYTE_COUNT (16 * 1024 * 1024)
-#define IOP_ADMA_MAX_BYTE_COUNT (16 * 1024 * 1024)
-
-static inline int iop_chan_is_busy(struct iop_adma_chan *chan)
-{
- u32 status = __raw_readl(DMA_CSR(chan));
- return (status & IOP_ADMA_STATUS_BUSY) ? 1 : 0;
-}
-
-static inline int iop_desc_is_aligned(struct iop_adma_desc_slot *desc,
- int num_slots)
-{
- /* num_slots will only ever be 1, 2, 4, or 8 */
- return (desc->idx & (num_slots - 1)) ? 0 : 1;
-}
-
-/* to do: support large (i.e. > hw max) buffer sizes */
-static inline int iop_chan_memcpy_slot_count(size_t len, int *slots_per_op)
-{
- *slots_per_op = 1;
- return 1;
-}
-
-/* to do: support large (i.e. > hw max) buffer sizes */
-static inline int iop_chan_memset_slot_count(size_t len, int *slots_per_op)
-{
- *slots_per_op = 1;
- return 1;
-}
-
-static inline int iop3xx_aau_xor_slot_count(size_t len, int src_cnt,
- int *slots_per_op)
-{
- static const char slot_count_table[] = {
- 1, 1, 1, 1, /* 01 - 04 */
- 2, 2, 2, 2, /* 05 - 08 */
- 4, 4, 4, 4, /* 09 - 12 */
- 4, 4, 4, 4, /* 13 - 16 */
- 8, 8, 8, 8, /* 17 - 20 */
- 8, 8, 8, 8, /* 21 - 24 */
- 8, 8, 8, 8, /* 25 - 28 */
- 8, 8, 8, 8, /* 29 - 32 */
- };
- *slots_per_op = slot_count_table[src_cnt - 1];
- return *slots_per_op;
-}
-
-static inline int
-iop_chan_interrupt_slot_count(int *slots_per_op, struct iop_adma_chan *chan)
-{
- switch (chan->device->id) {
- case DMA0_ID:
- case DMA1_ID:
- return iop_chan_memcpy_slot_count(0, slots_per_op);
- case AAU_ID:
- return iop3xx_aau_xor_slot_count(0, 2, slots_per_op);
- default:
- BUG();
- }
- return 0;
-}
-
-static inline int iop_chan_xor_slot_count(size_t len, int src_cnt,
- int *slots_per_op)
-{
- int slot_cnt = iop3xx_aau_xor_slot_count(len, src_cnt, slots_per_op);
-
- if (len <= IOP_ADMA_XOR_MAX_BYTE_COUNT)
- return slot_cnt;
-
- len -= IOP_ADMA_XOR_MAX_BYTE_COUNT;
- while (len > IOP_ADMA_XOR_MAX_BYTE_COUNT) {
- len -= IOP_ADMA_XOR_MAX_BYTE_COUNT;
- slot_cnt += *slots_per_op;
- }
-
- slot_cnt += *slots_per_op;
-
- return slot_cnt;
-}
-
-/* zero sum on iop3xx is limited to 1k at a time so it requires multiple
- * descriptors
- */
-static inline int iop_chan_zero_sum_slot_count(size_t len, int src_cnt,
- int *slots_per_op)
-{
- int slot_cnt = iop3xx_aau_xor_slot_count(len, src_cnt, slots_per_op);
-
- if (len <= IOP_ADMA_ZERO_SUM_MAX_BYTE_COUNT)
- return slot_cnt;
-
- len -= IOP_ADMA_ZERO_SUM_MAX_BYTE_COUNT;
- while (len > IOP_ADMA_ZERO_SUM_MAX_BYTE_COUNT) {
- len -= IOP_ADMA_ZERO_SUM_MAX_BYTE_COUNT;
- slot_cnt += *slots_per_op;
- }
-
- slot_cnt += *slots_per_op;
-
- return slot_cnt;
-}
-
-static inline u32 iop_desc_get_byte_count(struct iop_adma_desc_slot *desc,
- struct iop_adma_chan *chan)
-{
- union iop3xx_desc hw_desc = { .ptr = desc->hw_desc, };
-
- switch (chan->device->id) {
- case DMA0_ID:
- case DMA1_ID:
- return hw_desc.dma->byte_count;
- case AAU_ID:
- return hw_desc.aau->byte_count;
- default:
- BUG();
- }
- return 0;
-}
-
-/* translate the src_idx to a descriptor word index */
-static inline int __desc_idx(int src_idx)
-{
- static const int desc_idx_table[] = { 0, 0, 0, 0,
- 0, 1, 2, 3,
- 5, 6, 7, 8,
- 9, 10, 11, 12,
- 14, 15, 16, 17,
- 18, 19, 20, 21,
- 23, 24, 25, 26,
- 27, 28, 29, 30,
- };
-
- return desc_idx_table[src_idx];
-}
-
-static inline u32 iop_desc_get_src_addr(struct iop_adma_desc_slot *desc,
- struct iop_adma_chan *chan,
- int src_idx)
-{
- union iop3xx_desc hw_desc = { .ptr = desc->hw_desc, };
-
- switch (chan->device->id) {
- case DMA0_ID:
- case DMA1_ID:
- return hw_desc.dma->src_addr;
- case AAU_ID:
- break;
- default:
- BUG();
- }
-
- if (src_idx < 4)
- return hw_desc.aau->src[src_idx];
- else
- return hw_desc.aau->src_edc[__desc_idx(src_idx)].src_addr;
-}
-
-static inline void iop3xx_aau_desc_set_src_addr(struct iop3xx_desc_aau *hw_desc,
- int src_idx, dma_addr_t addr)
-{
- if (src_idx < 4)
- hw_desc->src[src_idx] = addr;
- else
- hw_desc->src_edc[__desc_idx(src_idx)].src_addr = addr;
-}
-
-static inline void
-iop_desc_init_memcpy(struct iop_adma_desc_slot *desc, unsigned long flags)
-{
- struct iop3xx_desc_dma *hw_desc = desc->hw_desc;
- union {
- u32 value;
- struct iop3xx_dma_desc_ctrl field;
- } u_desc_ctrl;
-
- u_desc_ctrl.value = 0;
- u_desc_ctrl.field.mem_to_mem_en = 1;
- u_desc_ctrl.field.pci_transaction = 0xe; /* memory read block */
- u_desc_ctrl.field.int_en = flags & DMA_PREP_INTERRUPT;
- hw_desc->desc_ctrl = u_desc_ctrl.value;
- hw_desc->upper_pci_src_addr = 0;
- hw_desc->crc_addr = 0;
-}
-
-static inline void
-iop_desc_init_memset(struct iop_adma_desc_slot *desc, unsigned long flags)
-{
- struct iop3xx_desc_aau *hw_desc = desc->hw_desc;
- union {
- u32 value;
- struct iop3xx_aau_desc_ctrl field;
- } u_desc_ctrl;
-
- u_desc_ctrl.value = 0;
- u_desc_ctrl.field.blk1_cmd_ctrl = 0x2; /* memory block fill */
- u_desc_ctrl.field.dest_write_en = 1;
- u_desc_ctrl.field.int_en = flags & DMA_PREP_INTERRUPT;
- hw_desc->desc_ctrl = u_desc_ctrl.value;
-}
-
-static inline u32
-iop3xx_desc_init_xor(struct iop3xx_desc_aau *hw_desc, int src_cnt,
- unsigned long flags)
-{
- int i, shift;
- u32 edcr;
- union {
- u32 value;
- struct iop3xx_aau_desc_ctrl field;
- } u_desc_ctrl;
-
- u_desc_ctrl.value = 0;
- switch (src_cnt) {
- case 25 ... 32:
- u_desc_ctrl.field.blk_ctrl = 0x3; /* use EDCR[2:0] */
- edcr = 0;
- shift = 1;
- for (i = 24; i < src_cnt; i++) {
- edcr |= (1 << shift);
- shift += 3;
- }
- hw_desc->src_edc[AAU_EDCR2_IDX].e_desc_ctrl = edcr;
- src_cnt = 24;
- fallthrough;
- case 17 ... 24:
- if (!u_desc_ctrl.field.blk_ctrl) {
- hw_desc->src_edc[AAU_EDCR2_IDX].e_desc_ctrl = 0;
- u_desc_ctrl.field.blk_ctrl = 0x3; /* use EDCR[2:0] */
- }
- edcr = 0;
- shift = 1;
- for (i = 16; i < src_cnt; i++) {
- edcr |= (1 << shift);
- shift += 3;
- }
- hw_desc->src_edc[AAU_EDCR1_IDX].e_desc_ctrl = edcr;
- src_cnt = 16;
- fallthrough;
- case 9 ... 16:
- if (!u_desc_ctrl.field.blk_ctrl)
- u_desc_ctrl.field.blk_ctrl = 0x2; /* use EDCR0 */
- edcr = 0;
- shift = 1;
- for (i = 8; i < src_cnt; i++) {
- edcr |= (1 << shift);
- shift += 3;
- }
- hw_desc->src_edc[AAU_EDCR0_IDX].e_desc_ctrl = edcr;
- src_cnt = 8;
- fallthrough;
- case 2 ... 8:
- shift = 1;
- for (i = 0; i < src_cnt; i++) {
- u_desc_ctrl.value |= (1 << shift);
- shift += 3;
- }
-
- if (!u_desc_ctrl.field.blk_ctrl && src_cnt > 4)
- u_desc_ctrl.field.blk_ctrl = 0x1; /* use mini-desc */
- }
-
- u_desc_ctrl.field.dest_write_en = 1;
- u_desc_ctrl.field.blk1_cmd_ctrl = 0x7; /* direct fill */
- u_desc_ctrl.field.int_en = flags & DMA_PREP_INTERRUPT;
- hw_desc->desc_ctrl = u_desc_ctrl.value;
-
- return u_desc_ctrl.value;
-}
-
-static inline void
-iop_desc_init_xor(struct iop_adma_desc_slot *desc, int src_cnt,
- unsigned long flags)
-{
- iop3xx_desc_init_xor(desc->hw_desc, src_cnt, flags);
-}
-
-/* return the number of operations */
-static inline int
-iop_desc_init_zero_sum(struct iop_adma_desc_slot *desc, int src_cnt,
- unsigned long flags)
-{
- int slot_cnt = desc->slot_cnt, slots_per_op = desc->slots_per_op;
- struct iop3xx_desc_aau *hw_desc, *prev_hw_desc, *iter;
- union {
- u32 value;
- struct iop3xx_aau_desc_ctrl field;
- } u_desc_ctrl;
- int i, j;
-
- hw_desc = desc->hw_desc;
-
- for (i = 0, j = 0; (slot_cnt -= slots_per_op) >= 0;
- i += slots_per_op, j++) {
- iter = iop_hw_desc_slot_idx(hw_desc, i);
- u_desc_ctrl.value = iop3xx_desc_init_xor(iter, src_cnt, flags);
- u_desc_ctrl.field.dest_write_en = 0;
- u_desc_ctrl.field.zero_result_en = 1;
- u_desc_ctrl.field.int_en = flags & DMA_PREP_INTERRUPT;
- iter->desc_ctrl = u_desc_ctrl.value;
-
- /* for the subsequent descriptors preserve the store queue
- * and chain them together
- */
- if (i) {
- prev_hw_desc =
- iop_hw_desc_slot_idx(hw_desc, i - slots_per_op);
- prev_hw_desc->next_desc =
- (u32) (desc->async_tx.phys + (i << 5));
- }
- }
-
- return j;
-}
-
-static inline void
-iop_desc_init_null_xor(struct iop_adma_desc_slot *desc, int src_cnt,
- unsigned long flags)
-{
- struct iop3xx_desc_aau *hw_desc = desc->hw_desc;
- union {
- u32 value;
- struct iop3xx_aau_desc_ctrl field;
- } u_desc_ctrl;
-
- u_desc_ctrl.value = 0;
- switch (src_cnt) {
- case 25 ... 32:
- u_desc_ctrl.field.blk_ctrl = 0x3; /* use EDCR[2:0] */
- hw_desc->src_edc[AAU_EDCR2_IDX].e_desc_ctrl = 0;
- fallthrough;
- case 17 ... 24:
- if (!u_desc_ctrl.field.blk_ctrl) {
- hw_desc->src_edc[AAU_EDCR2_IDX].e_desc_ctrl = 0;
- u_desc_ctrl.field.blk_ctrl = 0x3; /* use EDCR[2:0] */
- }
- hw_desc->src_edc[AAU_EDCR1_IDX].e_desc_ctrl = 0;
- fallthrough;
- case 9 ... 16:
- if (!u_desc_ctrl.field.blk_ctrl)
- u_desc_ctrl.field.blk_ctrl = 0x2; /* use EDCR0 */
- hw_desc->src_edc[AAU_EDCR0_IDX].e_desc_ctrl = 0;
- fallthrough;
- case 1 ... 8:
- if (!u_desc_ctrl.field.blk_ctrl && src_cnt > 4)
- u_desc_ctrl.field.blk_ctrl = 0x1; /* use mini-desc */
- }
-
- u_desc_ctrl.field.dest_write_en = 0;
- u_desc_ctrl.field.int_en = flags & DMA_PREP_INTERRUPT;
- hw_desc->desc_ctrl = u_desc_ctrl.value;
-}
-
-static inline void iop_desc_set_byte_count(struct iop_adma_desc_slot *desc,
- struct iop_adma_chan *chan,
- u32 byte_count)
-{
- union iop3xx_desc hw_desc = { .ptr = desc->hw_desc, };
-
- switch (chan->device->id) {
- case DMA0_ID:
- case DMA1_ID:
- hw_desc.dma->byte_count = byte_count;
- break;
- case AAU_ID:
- hw_desc.aau->byte_count = byte_count;
- break;
- default:
- BUG();
- }
-}
-
-static inline void
-iop_desc_init_interrupt(struct iop_adma_desc_slot *desc,
- struct iop_adma_chan *chan)
-{
- union iop3xx_desc hw_desc = { .ptr = desc->hw_desc, };
-
- switch (chan->device->id) {
- case DMA0_ID:
- case DMA1_ID:
- iop_desc_init_memcpy(desc, 1);
- hw_desc.dma->byte_count = 0;
- hw_desc.dma->dest_addr = 0;
- hw_desc.dma->src_addr = 0;
- break;
- case AAU_ID:
- iop_desc_init_null_xor(desc, 2, 1);
- hw_desc.aau->byte_count = 0;
- hw_desc.aau->dest_addr = 0;
- hw_desc.aau->src[0] = 0;
- hw_desc.aau->src[1] = 0;
- break;
- default:
- BUG();
- }
-}
-
-static inline void
-iop_desc_set_zero_sum_byte_count(struct iop_adma_desc_slot *desc, u32 len)
-{
- int slots_per_op = desc->slots_per_op;
- struct iop3xx_desc_aau *hw_desc = desc->hw_desc, *iter;
- int i = 0;
-
- if (len <= IOP_ADMA_ZERO_SUM_MAX_BYTE_COUNT) {
- hw_desc->byte_count = len;
- } else {
- do {
- iter = iop_hw_desc_slot_idx(hw_desc, i);
- iter->byte_count = IOP_ADMA_ZERO_SUM_MAX_BYTE_COUNT;
- len -= IOP_ADMA_ZERO_SUM_MAX_BYTE_COUNT;
- i += slots_per_op;
- } while (len > IOP_ADMA_ZERO_SUM_MAX_BYTE_COUNT);
-
- iter = iop_hw_desc_slot_idx(hw_desc, i);
- iter->byte_count = len;
- }
-}
-
-static inline void iop_desc_set_dest_addr(struct iop_adma_desc_slot *desc,
- struct iop_adma_chan *chan,
- dma_addr_t addr)
-{
- union iop3xx_desc hw_desc = { .ptr = desc->hw_desc, };
-
- switch (chan->device->id) {
- case DMA0_ID:
- case DMA1_ID:
- hw_desc.dma->dest_addr = addr;
- break;
- case AAU_ID:
- hw_desc.aau->dest_addr = addr;
- break;
- default:
- BUG();
- }
-}
-
-static inline void iop_desc_set_memcpy_src_addr(struct iop_adma_desc_slot *desc,
- dma_addr_t addr)
-{
- struct iop3xx_desc_dma *hw_desc = desc->hw_desc;
- hw_desc->src_addr = addr;
-}
-
-static inline void
-iop_desc_set_zero_sum_src_addr(struct iop_adma_desc_slot *desc, int src_idx,
- dma_addr_t addr)
-{
-
- struct iop3xx_desc_aau *hw_desc = desc->hw_desc, *iter;
- int slot_cnt = desc->slot_cnt, slots_per_op = desc->slots_per_op;
- int i;
-
- for (i = 0; (slot_cnt -= slots_per_op) >= 0;
- i += slots_per_op, addr += IOP_ADMA_ZERO_SUM_MAX_BYTE_COUNT) {
- iter = iop_hw_desc_slot_idx(hw_desc, i);
- iop3xx_aau_desc_set_src_addr(iter, src_idx, addr);
- }
-}
-
-static inline void iop_desc_set_xor_src_addr(struct iop_adma_desc_slot *desc,
- int src_idx, dma_addr_t addr)
-{
-
- struct iop3xx_desc_aau *hw_desc = desc->hw_desc, *iter;
- int slot_cnt = desc->slot_cnt, slots_per_op = desc->slots_per_op;
- int i;
-
- for (i = 0; (slot_cnt -= slots_per_op) >= 0;
- i += slots_per_op, addr += IOP_ADMA_XOR_MAX_BYTE_COUNT) {
- iter = iop_hw_desc_slot_idx(hw_desc, i);
- iop3xx_aau_desc_set_src_addr(iter, src_idx, addr);
- }
-}
-
-static inline void iop_desc_set_next_desc(struct iop_adma_desc_slot *desc,
- u32 next_desc_addr)
-{
- /* hw_desc->next_desc is the same location for all channels */
- union iop3xx_desc hw_desc = { .ptr = desc->hw_desc, };
-
- iop_paranoia(hw_desc.dma->next_desc);
- hw_desc.dma->next_desc = next_desc_addr;
-}
-
-static inline u32 iop_desc_get_next_desc(struct iop_adma_desc_slot *desc)
-{
- /* hw_desc->next_desc is the same location for all channels */
- union iop3xx_desc hw_desc = { .ptr = desc->hw_desc, };
- return hw_desc.dma->next_desc;
-}
-
-static inline void iop_desc_clear_next_desc(struct iop_adma_desc_slot *desc)
-{
- /* hw_desc->next_desc is the same location for all channels */
- union iop3xx_desc hw_desc = { .ptr = desc->hw_desc, };
- hw_desc.dma->next_desc = 0;
-}
-
-static inline void iop_desc_set_block_fill_val(struct iop_adma_desc_slot *desc,
- u32 val)
-{
- struct iop3xx_desc_aau *hw_desc = desc->hw_desc;
- hw_desc->src[0] = val;
-}
-
-static inline enum sum_check_flags
-iop_desc_get_zero_result(struct iop_adma_desc_slot *desc)
-{
- struct iop3xx_desc_aau *hw_desc = desc->hw_desc;
- struct iop3xx_aau_desc_ctrl desc_ctrl = hw_desc->desc_ctrl_field;
-
- iop_paranoia(!(desc_ctrl.tx_complete && desc_ctrl.zero_result_en));
- return desc_ctrl.zero_result_err << SUM_CHECK_P;
-}
-
-static inline void iop_chan_append(struct iop_adma_chan *chan)
-{
- u32 dma_chan_ctrl;
-
- dma_chan_ctrl = __raw_readl(DMA_CCR(chan));
- dma_chan_ctrl |= 0x2;
- __raw_writel(dma_chan_ctrl, DMA_CCR(chan));
-}
-
-static inline u32 iop_chan_get_status(struct iop_adma_chan *chan)
-{
- return __raw_readl(DMA_CSR(chan));
-}
-
-static inline void iop_chan_disable(struct iop_adma_chan *chan)
-{
- u32 dma_chan_ctrl = __raw_readl(DMA_CCR(chan));
- dma_chan_ctrl &= ~1;
- __raw_writel(dma_chan_ctrl, DMA_CCR(chan));
-}
-
-static inline void iop_chan_enable(struct iop_adma_chan *chan)
-{
- u32 dma_chan_ctrl = __raw_readl(DMA_CCR(chan));
-
- dma_chan_ctrl |= 1;
- __raw_writel(dma_chan_ctrl, DMA_CCR(chan));
-}
-
-static inline void iop_adma_device_clear_eot_status(struct iop_adma_chan *chan)
-{
- u32 status = __raw_readl(DMA_CSR(chan));
- status &= (1 << 9);
- __raw_writel(status, DMA_CSR(chan));
-}
-
-static inline void iop_adma_device_clear_eoc_status(struct iop_adma_chan *chan)
-{
- u32 status = __raw_readl(DMA_CSR(chan));
- status &= (1 << 8);
- __raw_writel(status, DMA_CSR(chan));
-}
-
-static inline void iop_adma_device_clear_err_status(struct iop_adma_chan *chan)
-{
- u32 status = __raw_readl(DMA_CSR(chan));
-
- switch (chan->device->id) {
- case DMA0_ID:
- case DMA1_ID:
- status &= (1 << 5) | (1 << 3) | (1 << 2) | (1 << 1);
- break;
- case AAU_ID:
- status &= (1 << 5);
- break;
- default:
- BUG();
- }
-
- __raw_writel(status, DMA_CSR(chan));
-}
-
-static inline int
-iop_is_err_int_parity(unsigned long status, struct iop_adma_chan *chan)
-{
- return 0;
-}
-
-static inline int
-iop_is_err_mcu_abort(unsigned long status, struct iop_adma_chan *chan)
-{
- return 0;
-}
-
-static inline int
-iop_is_err_int_tabort(unsigned long status, struct iop_adma_chan *chan)
-{
- return 0;
-}
-
-static inline int
-iop_is_err_int_mabort(unsigned long status, struct iop_adma_chan *chan)
-{
- return test_bit(5, &status);
-}
-
-static inline int
-iop_is_err_pci_tabort(unsigned long status, struct iop_adma_chan *chan)
-{
- switch (chan->device->id) {
- case DMA0_ID:
- case DMA1_ID:
- return test_bit(2, &status);
- default:
- return 0;
- }
-}
-
-static inline int
-iop_is_err_pci_mabort(unsigned long status, struct iop_adma_chan *chan)
-{
- switch (chan->device->id) {
- case DMA0_ID:
- case DMA1_ID:
- return test_bit(3, &status);
- default:
- return 0;
- }
-}
-
-static inline int
-iop_is_err_split_tx(unsigned long status, struct iop_adma_chan *chan)
-{
- switch (chan->device->id) {
- case DMA0_ID:
- case DMA1_ID:
- return test_bit(1, &status);
- default:
- return 0;
- }
-}
-#endif /* _ADMA_H */
}
static const struct of_device_id gpi_of_match[] = {
- { .compatible = "qcom,sc7280-gpi-dma", .data = (void *)0x10000 },
{ .compatible = "qcom,sdm845-gpi-dma", .data = (void *)0x0 },
{ .compatible = "qcom,sm6350-gpi-dma", .data = (void *)0x10000 },
+ /*
+ * Do not grow the list for compatible devices. Instead use
+ * qcom,sdm845-gpi-dma (for ee_offset = 0x0) or qcom,sm6350-gpi-dma
+ * (for ee_offset = 0x10000).
+ */
+ { .compatible = "qcom,sc7280-gpi-dma", .data = (void *)0x10000 },
{ .compatible = "qcom,sm8150-gpi-dma", .data = (void *)0x0 },
{ .compatible = "qcom,sm8250-gpi-dma", .data = (void *)0x0 },
{ .compatible = "qcom,sm8350-gpi-dma", .data = (void *)0x10000 },
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * Renesas SuperH DMA Engine support
- *
- * Copyright (C) 2013 Renesas Electronics, Inc.
- */
-
-#ifndef SHDMA_ARM_H
-#define SHDMA_ARM_H
-
-#include "shdma.h"
-
-/* Transmit sizes and respective CHCR register values */
-enum {
- XMIT_SZ_8BIT = 0,
- XMIT_SZ_16BIT = 1,
- XMIT_SZ_32BIT = 2,
- XMIT_SZ_64BIT = 7,
- XMIT_SZ_128BIT = 3,
- XMIT_SZ_256BIT = 4,
- XMIT_SZ_512BIT = 5,
-};
-
-/* log2(size / 8) - used to calculate number of transfers */
-#define SH_DMAE_TS_SHIFT { \
- [XMIT_SZ_8BIT] = 0, \
- [XMIT_SZ_16BIT] = 1, \
- [XMIT_SZ_32BIT] = 2, \
- [XMIT_SZ_64BIT] = 3, \
- [XMIT_SZ_128BIT] = 4, \
- [XMIT_SZ_256BIT] = 5, \
- [XMIT_SZ_512BIT] = 6, \
-}
-
-#define TS_LOW_BIT 0x3 /* --xx */
-#define TS_HI_BIT 0xc /* xx-- */
-
-#define TS_LOW_SHIFT (3)
-#define TS_HI_SHIFT (20 - 2) /* 2 bits for shifted low TS */
-
-#define TS_INDEX2VAL(i) \
- ((((i) & TS_LOW_BIT) << TS_LOW_SHIFT) |\
- (((i) & TS_HI_BIT) << TS_HI_SHIFT))
-
-#define CHCR_TX(xmit_sz) (DM_FIX | SM_INC | RS_ERS | TS_INDEX2VAL((xmit_sz)))
-#define CHCR_RX(xmit_sz) (DM_INC | SM_FIX | RS_ERS | TS_INDEX2VAL((xmit_sz)))
-
-#endif
#define TEGRA_GPCDMA_BURST_COMPLETION_TIMEOUT 5000 /* 5 msec */
/* Channel base address offset from GPCDMA base address */
-#define TEGRA_GPCDMA_CHANNEL_BASE_ADD_OFFSET 0x20000
+#define TEGRA_GPCDMA_CHANNEL_BASE_ADDR_OFFSET 0x10000
+
+/* Default channel mask reserving channel0 */
+#define TEGRA_GPCDMA_DEFAULT_CHANNEL_MASK 0xfffffffe
struct tegra_dma;
struct tegra_dma_channel;
const struct tegra_dma_chip_data *chip_data;
unsigned long sid_m2d_reserved;
unsigned long sid_d2m_reserved;
+ u32 chan_mask;
void __iomem *base_addr;
struct device *dev;
struct dma_device dma_dev;
}
static const struct tegra_dma_chip_data tegra186_dma_chip_data = {
- .nr_channels = 31,
+ .nr_channels = 32,
.channel_reg_size = SZ_64K,
.max_dma_count = SZ_1G,
.hw_support_pause = false,
};
static const struct tegra_dma_chip_data tegra194_dma_chip_data = {
- .nr_channels = 31,
+ .nr_channels = 32,
.channel_reg_size = SZ_64K,
.max_dma_count = SZ_1G,
.hw_support_pause = true,
};
static const struct tegra_dma_chip_data tegra234_dma_chip_data = {
- .nr_channels = 31,
+ .nr_channels = 32,
.channel_reg_size = SZ_64K,
.max_dma_count = SZ_1G,
.hw_support_pause = true,
}
stream_id = iommu_spec->ids[0] & 0xffff;
+ ret = device_property_read_u32(&pdev->dev, "dma-channel-mask",
+ &tdma->chan_mask);
+ if (ret) {
+ dev_warn(&pdev->dev,
+ "Missing dma-channel-mask property, using default channel mask %#x\n",
+ TEGRA_GPCDMA_DEFAULT_CHANNEL_MASK);
+ tdma->chan_mask = TEGRA_GPCDMA_DEFAULT_CHANNEL_MASK;
+ }
+
INIT_LIST_HEAD(&tdma->dma_dev.channels);
for (i = 0; i < cdata->nr_channels; i++) {
struct tegra_dma_channel *tdc = &tdma->channels[i];
+ /* Check for channel mask */
+ if (!(tdma->chan_mask & BIT(i)))
+ continue;
+
tdc->irq = platform_get_irq(pdev, i);
if (tdc->irq < 0)
return tdc->irq;
- tdc->chan_base_offset = TEGRA_GPCDMA_CHANNEL_BASE_ADD_OFFSET +
+ tdc->chan_base_offset = TEGRA_GPCDMA_CHANNEL_BASE_ADDR_OFFSET +
i * cdata->channel_reg_size;
snprintf(tdc->name, sizeof(tdc->name), "gpcdma.%d", i);
tdc->tdma = tdma;
return ret;
}
- dev_info(&pdev->dev, "GPC DMA driver register %d channels\n",
- cdata->nr_channels);
+ dev_info(&pdev->dev, "GPC DMA driver register %lu channels\n",
+ hweight_long(tdma->chan_mask));
return 0;
}
for (i = 0; i < tdma->chip_data->nr_channels; i++) {
struct tegra_dma_channel *tdc = &tdma->channels[i];
+ if (!(tdma->chan_mask & BIT(i)))
+ continue;
+
if (tdc->dma_desc) {
dev_err(tdma->dev, "channel %u busy\n", i);
return -EBUSY;
for (i = 0; i < tdma->chip_data->nr_channels; i++) {
struct tegra_dma_channel *tdc = &tdma->channels[i];
+ if (!(tdma->chan_mask & BIT(i)))
+ continue;
+
tegra_dma_program_sid(tdc, tdc->stream_id);
}
DMA engine is found on OMAP and DRA7xx parts.
config TI_K3_UDMA
- bool "Texas Instruments UDMA support"
+ tristate "Texas Instruments UDMA support"
depends on ARCH_K3
depends on TI_SCI_PROTOCOL
depends on TI_SCI_INTA_IRQCHIP
DMA engine is used in AM65x and j721e.
config TI_K3_UDMA_GLUE_LAYER
- bool "Texas Instruments UDMA Glue layer for non DMAengine users"
+ tristate "Texas Instruments UDMA Glue layer for non DMAengine users"
depends on ARCH_K3
depends on TI_K3_UDMA
help
If unsure, say N.
config TI_K3_PSIL
- bool
+ tristate
+ default TI_K3_UDMA
config TI_DMA_CROSSBAR
bool
obj-$(CONFIG_DMA_OMAP) += omap-dma.o
obj-$(CONFIG_TI_K3_UDMA) += k3-udma.o
obj-$(CONFIG_TI_K3_UDMA_GLUE_LAYER) += k3-udma-glue.o
-obj-$(CONFIG_TI_K3_PSIL) += k3-psil.o \
- k3-psil-am654.o \
- k3-psil-j721e.o \
- k3-psil-j7200.o \
- k3-psil-am64.o \
- k3-psil-j721s2.o \
- k3-psil-am62.o
+k3-psil-lib-objs := k3-psil.o \
+ k3-psil-am654.o \
+ k3-psil-j721e.o \
+ k3-psil-j7200.o \
+ k3-psil-am64.o \
+ k3-psil-j721s2.o \
+ k3-psil-am62.o
+obj-$(CONFIG_TI_K3_PSIL) += k3-psil-lib.o
obj-$(CONFIG_TI_DMA_CROSSBAR) += dma-crossbar.o
*/
#include <linux/kernel.h>
+#include <linux/module.h>
#include <linux/device.h>
#include <linux/init.h>
#include <linux/mutex.h>
return 0;
}
EXPORT_SYMBOL_GPL(psil_set_new_ep_config);
+MODULE_LICENSE("GPL v2");
*
*/
+#include <linux/module.h>
#include <linux/atomic.h>
#include <linux/delay.h>
#include <linux/dma-mapping.h>
{
return class_register(&k3_udma_glue_devclass);
}
-arch_initcall(k3_udma_glue_class_init);
+
+module_init(k3_udma_glue_class_init);
+MODULE_LICENSE("GPL v2");
*/
#include <linux/kernel.h>
+#include <linux/module.h>
#include <linux/delay.h>
#include <linux/dmaengine.h>
#include <linux/dma-mapping.h>
.compatible = "ti,j721e-navss-mcu-udmap",
.data = &j721e_mcu_data,
},
- { /* Sentinel */ },
-};
-
-static const struct of_device_id bcdma_of_match[] = {
{
.compatible = "ti,am64-dmss-bcdma",
.data = &am64_bcdma_data,
},
- { /* Sentinel */ },
-};
-
-static const struct of_device_id pktdma_of_match[] = {
{
.compatible = "ti,am64-dmss-pktdma",
.data = &am64_pktdma_data,
return -ENOMEM;
match = of_match_node(udma_of_match, dev->of_node);
- if (!match)
- match = of_match_node(bcdma_of_match, dev->of_node);
if (!match) {
- match = of_match_node(pktdma_of_match, dev->of_node);
- if (!match) {
- dev_err(dev, "No compatible match found\n");
- return -ENODEV;
- }
+ dev_err(dev, "No compatible match found\n");
+ return -ENODEV;
}
ud->match_data = match->data;
},
.probe = udma_probe,
};
-builtin_platform_driver(udma_driver);
-static struct platform_driver bcdma_driver = {
- .driver = {
- .name = "ti-bcdma",
- .of_match_table = bcdma_of_match,
- .suppress_bind_attrs = true,
- },
- .probe = udma_probe,
-};
-builtin_platform_driver(bcdma_driver);
-
-static struct platform_driver pktdma_driver = {
- .driver = {
- .name = "ti-pktdma",
- .of_match_table = pktdma_of_match,
- .suppress_bind_attrs = true,
- },
- .probe = udma_probe,
-};
-builtin_platform_driver(pktdma_driver);
+module_platform_driver(udma_driver);
+MODULE_LICENSE("GPL v2");
/* Private interfaces to UDMA */
#include "k3-udma-private.c"
* xilinx_dma_device_config - Configure the DMA channel
* @dchan: DMA channel
* @config: channel configuration
+ *
+ * Return: 0 always.
*/
static int xilinx_dma_device_config(struct dma_chan *dchan,
struct dma_slave_config *config)
* @xdev: Driver specific device structure
* @node: Device node
*
- * Return: 0 always.
+ * Return: '0' on success and failure value on error.
*/
static int xilinx_dma_child_probe(struct xilinx_dma_device *xdev,
struct device_node *node)
return IRQ_HANDLED;
}
-static int fsa9480_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int fsa9480_probe(struct i2c_client *client)
{
struct fsa9480_usbsw *info;
int ret;
.pm = &fsa9480_pm_ops,
.of_match_table = fsa9480_of_match,
},
- .probe = fsa9480_probe,
+ .probe_new = fsa9480_probe,
.id_table = fsa9480_id,
};
static const struct regmap_irq_chip max77843_muic_irq_chip = {
.name = "max77843-muic",
.status_base = MAX77843_MUIC_REG_INT1,
- .mask_base = MAX77843_MUIC_REG_INTMASK1,
- .mask_invert = true,
+ .unmask_base = MAX77843_MUIC_REG_INTMASK1,
.num_regs = 3,
.irqs = max77843_muic_irq,
.num_irqs = ARRAY_SIZE(max77843_muic_irq),
}
}
-static int rt8973a_muic_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int rt8973a_muic_i2c_probe(struct i2c_client *i2c)
{
struct device_node *np = i2c->dev.of_node;
struct rt8973a_muic_info *info;
.pm = &rt8973a_muic_pm_ops,
.of_match_table = rt8973a_dt_match,
},
- .probe = rt8973a_muic_i2c_probe,
+ .probe_new = rt8973a_muic_i2c_probe,
.remove = rt8973a_muic_i2c_remove,
.id_table = rt8973a_i2c_id,
};
typec_set_pwr_opmode(port, TYPEC_PWR_MODE_USB);
}
-static irqreturn_t tusb320_irq_handler(int irq, void *dev_id)
+static irqreturn_t tusb320_state_update_handler(struct tusb320_priv *priv,
+ bool force_update)
{
- struct tusb320_priv *priv = dev_id;
unsigned int reg;
if (regmap_read(priv->regmap, TUSB320_REG9, ®)) {
return IRQ_NONE;
}
- if (!(reg & TUSB320_REG9_INTERRUPT_STATUS))
+ if (!force_update && !(reg & TUSB320_REG9_INTERRUPT_STATUS))
return IRQ_NONE;
tusb320_extcon_irq_handler(priv, reg);
return IRQ_HANDLED;
}
+static irqreturn_t tusb320_irq_handler(int irq, void *dev_id)
+{
+ struct tusb320_priv *priv = dev_id;
+
+ return tusb320_state_update_handler(priv, false);
+}
+
static const struct regmap_config tusb320_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
return 0;
}
-static int tusb320_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tusb320_probe(struct i2c_client *client)
{
struct tusb320_priv *priv;
const void *match_data;
return ret;
/* update initial state */
- tusb320_irq_handler(client->irq, priv);
+ tusb320_state_update_handler(priv, true);
/* Reset chip to its default state */
ret = tusb320_reset(priv);
* State and polarity might change after a reset, so update
* them again and make sure the interrupt status bit is cleared.
*/
- tusb320_irq_handler(client->irq, priv);
+ tusb320_state_update_handler(priv, true);
ret = devm_request_threaded_irq(priv->dev, client->irq, NULL,
tusb320_irq_handler,
MODULE_DEVICE_TABLE(of, tusb320_extcon_dt_match);
static struct i2c_driver tusb320_extcon_driver = {
- .probe = tusb320_probe,
+ .probe_new = tusb320_probe,
.driver = {
.name = "extcon-tusb320",
.of_match_table = tusb320_extcon_dt_match,
NULL
};
-static int dmi_dev_uevent(struct device *dev, struct kobj_uevent_env *env)
+static int dmi_dev_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
ssize_t len;
platform_device_register_simple("efi_secret", 0, NULL, 0);
#endif
- execute_with_initialized_rng(&refresh_nv_rng_seed_nb);
+ if (efi_rt_services_supported(EFI_RT_SUPPORTED_SET_VARIABLE))
+ execute_with_initialized_rng(&refresh_nv_rng_seed_nb);
+
return 0;
err_remove_group:
driver provides an interface for reading and writing NVRAM
variables.
+config GOOGLE_CBMEM
+ tristate "CBMEM entries in sysfs"
+ depends on GOOGLE_COREBOOT_TABLE
+ help
+ CBMEM is a downwards-growing memory region created by the
+ Coreboot BIOS containing tagged data structures from the
+ BIOS. These data structures expose things like the verified
+ boot firmware variables, flash layout, firmware event log,
+ and more.
+
+ This option enables the cbmem module, which causes the
+ kernel to search for Coreboot CBMEM entries, and expose the
+ memory for each entry in sysfs under
+ /sys/bus/coreboot/devices/cbmem-<id>.
+
config GOOGLE_COREBOOT_TABLE
tristate "Coreboot Table Access"
depends on HAS_IOMEM && (ACPI || OF)
obj-$(CONFIG_GOOGLE_MEMCONSOLE_COREBOOT) += memconsole-coreboot.o
obj-$(CONFIG_GOOGLE_MEMCONSOLE_X86_LEGACY) += memconsole-x86-legacy.o
+# Must come after coreboot_table.o, as this driver depends on that bus type.
+obj-$(CONFIG_GOOGLE_CBMEM) += cbmem.o
+
vpd-sysfs-y := vpd.o vpd_decode.o
obj-$(CONFIG_GOOGLE_VPD) += vpd-sysfs.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * cbmem.c
+ *
+ * Driver for exporting cbmem entries in sysfs.
+ *
+ * Copyright 2022 Google LLC
+ */
+
+#include <linux/device.h>
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/kobject.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/sysfs.h>
+
+#include "coreboot_table.h"
+
+struct cbmem_entry {
+ char *mem_file_buf;
+ u32 size;
+};
+
+static struct cbmem_entry *to_cbmem_entry(struct kobject *kobj)
+{
+ return dev_get_drvdata(kobj_to_dev(kobj));
+}
+
+static ssize_t mem_read(struct file *filp, struct kobject *kobj,
+ struct bin_attribute *bin_attr, char *buf, loff_t pos,
+ size_t count)
+{
+ struct cbmem_entry *entry = to_cbmem_entry(kobj);
+
+ return memory_read_from_buffer(buf, count, &pos, entry->mem_file_buf,
+ entry->size);
+}
+
+static ssize_t mem_write(struct file *filp, struct kobject *kobj,
+ struct bin_attribute *bin_attr, char *buf, loff_t pos,
+ size_t count)
+{
+ struct cbmem_entry *entry = to_cbmem_entry(kobj);
+
+ if (pos < 0 || pos >= entry->size)
+ return -EINVAL;
+ if (count > entry->size - pos)
+ count = entry->size - pos;
+
+ memcpy(entry->mem_file_buf + pos, buf, count);
+ return count;
+}
+static BIN_ATTR_ADMIN_RW(mem, 0);
+
+static ssize_t address_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct coreboot_device *cbdev = dev_to_coreboot_device(dev);
+
+ return sysfs_emit(buf, "0x%llx\n", cbdev->cbmem_entry.address);
+}
+static DEVICE_ATTR_RO(address);
+
+static ssize_t size_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct coreboot_device *cbdev = dev_to_coreboot_device(dev);
+
+ return sysfs_emit(buf, "0x%x\n", cbdev->cbmem_entry.entry_size);
+}
+static DEVICE_ATTR_RO(size);
+
+static struct attribute *attrs[] = {
+ &dev_attr_address.attr,
+ &dev_attr_size.attr,
+ NULL,
+};
+
+static struct bin_attribute *bin_attrs[] = {
+ &bin_attr_mem,
+ NULL,
+};
+
+static const struct attribute_group cbmem_entry_group = {
+ .attrs = attrs,
+ .bin_attrs = bin_attrs,
+};
+
+static const struct attribute_group *dev_groups[] = {
+ &cbmem_entry_group,
+ NULL,
+};
+
+static int cbmem_entry_probe(struct coreboot_device *dev)
+{
+ struct cbmem_entry *entry;
+
+ entry = devm_kzalloc(&dev->dev, sizeof(*entry), GFP_KERNEL);
+ if (!entry)
+ return -ENOMEM;
+
+ dev_set_drvdata(&dev->dev, entry);
+ entry->mem_file_buf = devm_memremap(&dev->dev, dev->cbmem_entry.address,
+ dev->cbmem_entry.entry_size,
+ MEMREMAP_WB);
+ if (IS_ERR(entry->mem_file_buf))
+ return PTR_ERR(entry->mem_file_buf);
+
+ entry->size = dev->cbmem_entry.entry_size;
+
+ return 0;
+}
+
+static struct coreboot_driver cbmem_entry_driver = {
+ .probe = cbmem_entry_probe,
+ .drv = {
+ .name = "cbmem",
+ .owner = THIS_MODULE,
+ .dev_groups = dev_groups,
+ },
+ .tag = LB_TAG_CBMEM_ENTRY,
+};
+module_coreboot_driver(cbmem_entry_driver);
+
+MODULE_AUTHOR("Jack Rosenthal <jrosenth@chromium.org>");
+MODULE_LICENSE("GPL");
if (!device)
return -ENOMEM;
- dev_set_name(&device->dev, "coreboot%d", i);
device->dev.parent = dev;
device->dev.bus = &coreboot_bus_type;
device->dev.release = coreboot_device_release;
memcpy(&device->entry, ptr_entry, entry->size);
+ switch (device->entry.tag) {
+ case LB_TAG_CBMEM_ENTRY:
+ dev_set_name(&device->dev, "cbmem-%08x",
+ device->cbmem_entry.id);
+ break;
+ default:
+ dev_set_name(&device->dev, "coreboot%d", i);
+ break;
+ }
+
ret = device_register(&device->dev);
if (ret) {
put_device(&device->dev);
u64 cbmem_addr;
};
+#define LB_TAG_CBMEM_ENTRY 0x31
+
+/* Corresponds to LB_TAG_CBMEM_ENTRY */
+struct lb_cbmem_entry {
+ u32 tag;
+ u32 size;
+
+ u64 address;
+ u32 entry_size;
+ u32 id;
+};
+
/* Describes framebuffer setup by coreboot */
struct lb_framebuffer {
u32 tag;
union {
struct coreboot_table_entry entry;
struct lb_cbmem_ref cbmem_ref;
+ struct lb_cbmem_entry cbmem_entry;
struct lb_framebuffer framebuffer;
};
};
+static inline struct coreboot_device *dev_to_coreboot_device(struct device *dev)
+{
+ return container_of(dev, struct coreboot_device, dev);
+}
+
/* A driver for handling devices described in coreboot tables. */
struct coreboot_driver {
int (*probe)(struct coreboot_device *);
fw->cl.tx_block = true;
fw->chan = mbox_request_channel(&fw->cl, 0);
- if (IS_ERR(fw->chan))
- return dev_err_probe(dev, PTR_ERR(fw->chan),
- "Failed to get mbox channel\n");
+ if (IS_ERR(fw->chan)) {
+ int ret = PTR_ERR(fw->chan);
+ kfree(fw);
+ return dev_err_probe(dev, ret, "Failed to get mbox channel\n");
+ }
init_completion(&fw->c);
kref_init(&fw->consumers);
EXPORT_SYMBOL_GPL(zynqmp_pm_release_node);
/**
+ * zynqmp_pm_get_rpu_mode() - Get RPU mode
+ * @node_id: Node ID of the device
+ * @rpu_mode: return by reference value
+ * either split or lockstep
+ *
+ * Return: return 0 on success or error+reason.
+ * if success, then rpu_mode will be set
+ * to current rpu mode.
+ */
+int zynqmp_pm_get_rpu_mode(u32 node_id, enum rpu_oper_mode *rpu_mode)
+{
+ u32 ret_payload[PAYLOAD_ARG_CNT];
+ int ret;
+
+ ret = zynqmp_pm_invoke_fn(PM_IOCTL, node_id,
+ IOCTL_GET_RPU_OPER_MODE, 0, 0, ret_payload);
+
+ /* only set rpu_mode if no error */
+ if (ret == XST_PM_SUCCESS)
+ *rpu_mode = ret_payload[0];
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_get_rpu_mode);
+
+/**
+ * zynqmp_pm_set_rpu_mode() - Set RPU mode
+ * @node_id: Node ID of the device
+ * @rpu_mode: Argument 1 to requested IOCTL call. either split or lockstep
+ *
+ * This function is used to set RPU mode to split or
+ * lockstep
+ *
+ * Return: Returns status, either success or error+reason
+ */
+int zynqmp_pm_set_rpu_mode(u32 node_id, enum rpu_oper_mode rpu_mode)
+{
+ return zynqmp_pm_invoke_fn(PM_IOCTL, node_id,
+ IOCTL_SET_RPU_OPER_MODE, (u32)rpu_mode,
+ 0, NULL);
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_set_rpu_mode);
+
+/**
+ * zynqmp_pm_set_tcm_config - configure TCM
+ * @node_id: Firmware specific TCM subsystem ID
+ * @tcm_mode: Argument 1 to requested IOCTL call
+ * either PM_RPU_TCM_COMB or PM_RPU_TCM_SPLIT
+ *
+ * This function is used to set RPU mode to split or combined
+ *
+ * Return: status: 0 for success, else failure
+ */
+int zynqmp_pm_set_tcm_config(u32 node_id, enum rpu_tcm_comb tcm_mode)
+{
+ return zynqmp_pm_invoke_fn(PM_IOCTL, node_id,
+ IOCTL_TCM_COMB_CONFIG, (u32)tcm_mode, 0,
+ NULL);
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_set_tcm_config);
+
+/**
+ * zynqmp_pm_force_pwrdwn - PM call to request for another PU or subsystem to
+ * be powered down forcefully
+ * @node: Node ID of the targeted PU or subsystem
+ * @ack: Flag to specify whether acknowledge is requested
+ *
+ * Return: status, either success or error+reason
+ */
+int zynqmp_pm_force_pwrdwn(const u32 node,
+ const enum zynqmp_pm_request_ack ack)
+{
+ return zynqmp_pm_invoke_fn(PM_FORCE_POWERDOWN, node, ack, 0, 0, NULL);
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_force_pwrdwn);
+
+/**
+ * zynqmp_pm_request_wake - PM call to wake up selected master or subsystem
+ * @node: Node ID of the master or subsystem
+ * @set_addr: Specifies whether the address argument is relevant
+ * @address: Address from which to resume when woken up
+ * @ack: Flag to specify whether acknowledge requested
+ *
+ * Return: status, either success or error+reason
+ */
+int zynqmp_pm_request_wake(const u32 node,
+ const bool set_addr,
+ const u64 address,
+ const enum zynqmp_pm_request_ack ack)
+{
+ /* set_addr flag is encoded into 1st bit of address */
+ return zynqmp_pm_invoke_fn(PM_REQUEST_WAKEUP, node, address | set_addr,
+ address >> 32, ack, NULL);
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_request_wake);
+
+/**
* zynqmp_pm_set_requirement() - PM call to set requirement for PM slaves
* @node: Node ID of the slave
* @capabilities: Requested capabilities of the slave
programming over slave SPI interface with .dat formatted
bitstream image.
+config FPGA_MGR_LATTICE_SYSCONFIG
+ tristate
+
+config FPGA_MGR_LATTICE_SYSCONFIG_SPI
+ tristate "Lattice sysCONFIG SPI FPGA manager"
+ depends on SPI
+ select FPGA_MGR_LATTICE_SYSCONFIG
+ help
+ FPGA manager driver support for Lattice FPGAs programming over slave
+ SPI sysCONFIG interface.
+
endif # FPGA
obj-$(CONFIG_FPGA_MGR_ZYNQMP_FPGA) += zynqmp-fpga.o
obj-$(CONFIG_FPGA_MGR_VERSAL_FPGA) += versal-fpga.o
obj-$(CONFIG_FPGA_MGR_MICROCHIP_SPI) += microchip-spi.o
+obj-$(CONFIG_FPGA_MGR_LATTICE_SYSCONFIG) += lattice-sysconfig.o
+obj-$(CONFIG_FPGA_MGR_LATTICE_SYSCONFIG_SPI) += lattice-sysconfig-spi.o
obj-$(CONFIG_ALTERA_PR_IP_CORE) += altera-pr-ip-core.o
obj-$(CONFIG_ALTERA_PR_IP_CORE_PLAT) += altera-pr-ip-core-plat.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Lattice FPGA programming over slave SPI sysCONFIG interface.
+ */
+
+#include <linux/spi/spi.h>
+
+#include "lattice-sysconfig.h"
+
+static const u32 ecp5_spi_max_speed_hz = 60000000;
+
+static int sysconfig_spi_cmd_transfer(struct sysconfig_priv *priv,
+ const void *tx_buf, size_t tx_len,
+ void *rx_buf, size_t rx_len)
+{
+ struct spi_device *spi = to_spi_device(priv->dev);
+
+ return spi_write_then_read(spi, tx_buf, tx_len, rx_buf, rx_len);
+}
+
+static int sysconfig_spi_bitstream_burst_init(struct sysconfig_priv *priv)
+{
+ const u8 lsc_bitstream_burst[] = SYSCONFIG_LSC_BITSTREAM_BURST;
+ struct spi_device *spi = to_spi_device(priv->dev);
+ struct spi_transfer xfer = {};
+ struct spi_message msg;
+ size_t buf_len;
+ void *buf;
+ int ret;
+
+ buf_len = sizeof(lsc_bitstream_burst);
+
+ buf = kmemdup(lsc_bitstream_burst, buf_len, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ xfer.len = buf_len;
+ xfer.tx_buf = buf;
+ xfer.cs_change = 1;
+
+ spi_message_init_with_transfers(&msg, &xfer, 1);
+
+ /*
+ * Lock SPI bus for exclusive usage until FPGA programming is done.
+ * SPI bus will be released in sysconfig_spi_bitstream_burst_complete().
+ */
+ spi_bus_lock(spi->controller);
+
+ ret = spi_sync_locked(spi, &msg);
+ if (ret)
+ spi_bus_unlock(spi->controller);
+
+ kfree(buf);
+
+ return ret;
+}
+
+static int sysconfig_spi_bitstream_burst_write(struct sysconfig_priv *priv,
+ const char *buf, size_t len)
+{
+ struct spi_device *spi = to_spi_device(priv->dev);
+ struct spi_transfer xfer = {
+ .tx_buf = buf,
+ .len = len,
+ .cs_change = 1,
+ };
+ struct spi_message msg;
+
+ spi_message_init_with_transfers(&msg, &xfer, 1);
+
+ return spi_sync_locked(spi, &msg);
+}
+
+static int sysconfig_spi_bitstream_burst_complete(struct sysconfig_priv *priv)
+{
+ struct spi_device *spi = to_spi_device(priv->dev);
+
+ /* Bitstream burst write is done, release SPI bus */
+ spi_bus_unlock(spi->controller);
+
+ /* Toggle CS to finish bitstream write */
+ return spi_write(spi, NULL, 0);
+}
+
+static int sysconfig_spi_probe(struct spi_device *spi)
+{
+ const struct spi_device_id *dev_id;
+ struct device *dev = &spi->dev;
+ struct sysconfig_priv *priv;
+ const u32 *spi_max_speed;
+
+ priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ spi_max_speed = device_get_match_data(dev);
+ if (!spi_max_speed) {
+ dev_id = spi_get_device_id(spi);
+ if (!dev_id)
+ return -ENODEV;
+
+ spi_max_speed = (const u32 *)dev_id->driver_data;
+ }
+
+ if (!spi_max_speed)
+ return -EINVAL;
+
+ if (spi->max_speed_hz > *spi_max_speed) {
+ dev_err(dev, "SPI speed %u is too high, maximum speed is %u\n",
+ spi->max_speed_hz, *spi_max_speed);
+ return -EINVAL;
+ }
+
+ priv->dev = dev;
+ priv->command_transfer = sysconfig_spi_cmd_transfer;
+ priv->bitstream_burst_write_init = sysconfig_spi_bitstream_burst_init;
+ priv->bitstream_burst_write = sysconfig_spi_bitstream_burst_write;
+ priv->bitstream_burst_write_complete = sysconfig_spi_bitstream_burst_complete;
+
+ return sysconfig_probe(priv);
+}
+
+static const struct spi_device_id sysconfig_spi_ids[] = {
+ {
+ .name = "sysconfig-ecp5",
+ .driver_data = (kernel_ulong_t)&ecp5_spi_max_speed_hz,
+ }, {},
+};
+MODULE_DEVICE_TABLE(spi, sysconfig_spi_ids);
+
+#if IS_ENABLED(CONFIG_OF)
+static const struct of_device_id sysconfig_of_ids[] = {
+ {
+ .compatible = "lattice,sysconfig-ecp5",
+ .data = &ecp5_spi_max_speed_hz,
+ }, {},
+};
+MODULE_DEVICE_TABLE(of, sysconfig_of_ids);
+#endif /* IS_ENABLED(CONFIG_OF) */
+
+static struct spi_driver lattice_sysconfig_driver = {
+ .probe = sysconfig_spi_probe,
+ .id_table = sysconfig_spi_ids,
+ .driver = {
+ .name = "lattice_sysconfig_spi_fpga_mgr",
+ .of_match_table = of_match_ptr(sysconfig_of_ids),
+ },
+};
+module_spi_driver(lattice_sysconfig_driver);
+
+MODULE_DESCRIPTION("Lattice sysCONFIG Slave SPI FPGA Manager");
+MODULE_LICENSE("GPL");
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Lattice FPGA sysCONFIG interface functions independent of port type.
+ */
+
+#include <linux/delay.h>
+#include <linux/fpga/fpga-mgr.h>
+#include <linux/gpio/consumer.h>
+#include <linux/iopoll.h>
+
+#include "lattice-sysconfig.h"
+
+static int sysconfig_cmd_write(struct sysconfig_priv *priv, const void *buf,
+ size_t buf_len)
+{
+ return priv->command_transfer(priv, buf, buf_len, NULL, 0);
+}
+
+static int sysconfig_cmd_read(struct sysconfig_priv *priv, const void *tx_buf,
+ size_t tx_len, void *rx_buf, size_t rx_len)
+{
+ return priv->command_transfer(priv, tx_buf, tx_len, rx_buf, rx_len);
+}
+
+static int sysconfig_read_busy(struct sysconfig_priv *priv)
+{
+ const u8 lsc_check_busy[] = SYSCONFIG_LSC_CHECK_BUSY;
+ u8 busy;
+ int ret;
+
+ ret = sysconfig_cmd_read(priv, lsc_check_busy, sizeof(lsc_check_busy),
+ &busy, sizeof(busy));
+
+ return ret ? : busy;
+}
+
+static int sysconfig_poll_busy(struct sysconfig_priv *priv)
+{
+ int ret, busy;
+
+ ret = read_poll_timeout(sysconfig_read_busy, busy, busy <= 0,
+ SYSCONFIG_POLL_INTERVAL_US,
+ SYSCONFIG_POLL_BUSY_TIMEOUT_US, false, priv);
+
+ return ret ? : busy;
+}
+
+static int sysconfig_read_status(struct sysconfig_priv *priv, u32 *status)
+{
+ const u8 lsc_read_status[] = SYSCONFIG_LSC_READ_STATUS;
+ __be32 device_status;
+ int ret;
+
+ ret = sysconfig_cmd_read(priv, lsc_read_status, sizeof(lsc_read_status),
+ &device_status, sizeof(device_status));
+ if (ret)
+ return ret;
+
+ *status = be32_to_cpu(device_status);
+
+ return 0;
+}
+
+static int sysconfig_poll_status(struct sysconfig_priv *priv, u32 *status)
+{
+ int ret = sysconfig_poll_busy(priv);
+
+ if (ret)
+ return ret;
+
+ return sysconfig_read_status(priv, status);
+}
+
+static int sysconfig_poll_gpio(struct gpio_desc *gpio, bool is_active)
+{
+ int ret, val;
+
+ ret = read_poll_timeout(gpiod_get_value, val,
+ val < 0 || !!val == is_active,
+ SYSCONFIG_POLL_INTERVAL_US,
+ SYSCONFIG_POLL_GPIO_TIMEOUT_US, false, gpio);
+
+ if (val < 0)
+ return val;
+
+ return ret;
+}
+
+static int sysconfig_gpio_refresh(struct sysconfig_priv *priv)
+{
+ struct gpio_desc *program = priv->program;
+ struct gpio_desc *init = priv->init;
+ struct gpio_desc *done = priv->done;
+ int ret;
+
+ /* Enter init mode */
+ gpiod_set_value(program, 1);
+
+ ret = sysconfig_poll_gpio(init, true);
+ if (!ret)
+ ret = sysconfig_poll_gpio(done, false);
+
+ if (ret)
+ return ret;
+
+ /* Enter program mode */
+ gpiod_set_value(program, 0);
+
+ return sysconfig_poll_gpio(init, false);
+}
+
+static int sysconfig_lsc_refresh(struct sysconfig_priv *priv)
+{
+ static const u8 lsc_refresh[] = SYSCONFIG_LSC_REFRESH;
+ int ret;
+
+ ret = sysconfig_cmd_write(priv, lsc_refresh, sizeof(lsc_refresh));
+ if (ret)
+ return ret;
+
+ usleep_range(4000, 8000);
+
+ return 0;
+}
+
+static int sysconfig_refresh(struct sysconfig_priv *priv)
+{
+ struct gpio_desc *program = priv->program;
+ struct gpio_desc *init = priv->init;
+ struct gpio_desc *done = priv->done;
+
+ if (program && init && done)
+ return sysconfig_gpio_refresh(priv);
+
+ return sysconfig_lsc_refresh(priv);
+}
+
+static int sysconfig_isc_enable(struct sysconfig_priv *priv)
+{
+ u8 isc_enable[] = SYSCONFIG_ISC_ENABLE;
+ u32 status;
+ int ret;
+
+ ret = sysconfig_cmd_write(priv, isc_enable, sizeof(isc_enable));
+ if (ret)
+ return ret;
+
+ ret = sysconfig_poll_status(priv, &status);
+ if (ret)
+ return ret;
+
+ if (status & SYSCONFIG_STATUS_FAIL)
+ return -EFAULT;
+
+ return 0;
+}
+
+static int sysconfig_isc_erase(struct sysconfig_priv *priv)
+{
+ u8 isc_erase[] = SYSCONFIG_ISC_ERASE;
+ u32 status;
+ int ret;
+
+ ret = sysconfig_cmd_write(priv, isc_erase, sizeof(isc_erase));
+ if (ret)
+ return ret;
+
+ ret = sysconfig_poll_status(priv, &status);
+ if (ret)
+ return ret;
+
+ if (status & SYSCONFIG_STATUS_FAIL)
+ return -EFAULT;
+
+ return 0;
+}
+
+static int sysconfig_isc_init(struct sysconfig_priv *priv)
+{
+ int ret = sysconfig_isc_enable(priv);
+
+ if (ret)
+ return ret;
+
+ return sysconfig_isc_erase(priv);
+}
+
+static int sysconfig_lsc_init_addr(struct sysconfig_priv *priv)
+{
+ const u8 lsc_init_addr[] = SYSCONFIG_LSC_INIT_ADDR;
+
+ return sysconfig_cmd_write(priv, lsc_init_addr, sizeof(lsc_init_addr));
+}
+
+static int sysconfig_burst_write_init(struct sysconfig_priv *priv)
+{
+ return priv->bitstream_burst_write_init(priv);
+}
+
+static int sysconfig_burst_write_complete(struct sysconfig_priv *priv)
+{
+ return priv->bitstream_burst_write_complete(priv);
+}
+
+static int sysconfig_bitstream_burst_write(struct sysconfig_priv *priv,
+ const char *buf, size_t count)
+{
+ int ret = priv->bitstream_burst_write(priv, buf, count);
+
+ if (ret)
+ sysconfig_burst_write_complete(priv);
+
+ return ret;
+}
+
+static int sysconfig_isc_disable(struct sysconfig_priv *priv)
+{
+ const u8 isc_disable[] = SYSCONFIG_ISC_DISABLE;
+
+ return sysconfig_cmd_write(priv, isc_disable, sizeof(isc_disable));
+}
+
+static void sysconfig_cleanup(struct sysconfig_priv *priv)
+{
+ sysconfig_isc_erase(priv);
+ sysconfig_refresh(priv);
+}
+
+static int sysconfig_isc_finish(struct sysconfig_priv *priv)
+{
+ struct gpio_desc *done_gpio = priv->done;
+ u32 status;
+ int ret;
+
+ if (done_gpio) {
+ ret = sysconfig_isc_disable(priv);
+ if (ret)
+ return ret;
+
+ return sysconfig_poll_gpio(done_gpio, true);
+ }
+
+ ret = sysconfig_poll_status(priv, &status);
+ if (ret)
+ return ret;
+
+ if ((status & SYSCONFIG_STATUS_DONE) &&
+ !(status & SYSCONFIG_STATUS_BUSY) &&
+ !(status & SYSCONFIG_STATUS_ERR))
+ return sysconfig_isc_disable(priv);
+
+ return -EFAULT;
+}
+
+static enum fpga_mgr_states sysconfig_ops_state(struct fpga_manager *mgr)
+{
+ struct sysconfig_priv *priv = mgr->priv;
+ struct gpio_desc *done = priv->done;
+ u32 status;
+ int ret;
+
+ if (done && (gpiod_get_value(done) > 0))
+ return FPGA_MGR_STATE_OPERATING;
+
+ ret = sysconfig_read_status(priv, &status);
+ if (!ret && (status & SYSCONFIG_STATUS_DONE))
+ return FPGA_MGR_STATE_OPERATING;
+
+ return FPGA_MGR_STATE_UNKNOWN;
+}
+
+static int sysconfig_ops_write_init(struct fpga_manager *mgr,
+ struct fpga_image_info *info,
+ const char *buf, size_t count)
+{
+ struct sysconfig_priv *priv = mgr->priv;
+ struct device *dev = &mgr->dev;
+ int ret;
+
+ if (info->flags & FPGA_MGR_PARTIAL_RECONFIG) {
+ dev_err(dev, "Partial reconfiguration is not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ /* Enter program mode */
+ ret = sysconfig_refresh(priv);
+ if (ret) {
+ dev_err(dev, "Failed to go to program mode\n");
+ return ret;
+ }
+
+ /* Enter ISC mode */
+ ret = sysconfig_isc_init(priv);
+ if (ret) {
+ dev_err(dev, "Failed to go to ISC mode\n");
+ return ret;
+ }
+
+ /* Initialize the Address Shift Register */
+ ret = sysconfig_lsc_init_addr(priv);
+ if (ret) {
+ dev_err(dev,
+ "Failed to initialize the Address Shift Register\n");
+ return ret;
+ }
+
+ /* Prepare for bitstream burst write */
+ ret = sysconfig_burst_write_init(priv);
+ if (ret)
+ dev_err(dev, "Failed to prepare for bitstream burst write\n");
+
+ return ret;
+}
+
+static int sysconfig_ops_write(struct fpga_manager *mgr, const char *buf,
+ size_t count)
+{
+ return sysconfig_bitstream_burst_write(mgr->priv, buf, count);
+}
+
+static int sysconfig_ops_write_complete(struct fpga_manager *mgr,
+ struct fpga_image_info *info)
+{
+ struct sysconfig_priv *priv = mgr->priv;
+ struct device *dev = &mgr->dev;
+ int ret;
+
+ ret = sysconfig_burst_write_complete(priv);
+ if (!ret)
+ ret = sysconfig_poll_busy(priv);
+
+ if (ret) {
+ dev_err(dev, "Error while waiting bitstream write to finish\n");
+ goto fail;
+ }
+
+ ret = sysconfig_isc_finish(priv);
+
+fail:
+ if (ret)
+ sysconfig_cleanup(priv);
+
+ return ret;
+}
+
+static const struct fpga_manager_ops sysconfig_fpga_mgr_ops = {
+ .state = sysconfig_ops_state,
+ .write_init = sysconfig_ops_write_init,
+ .write = sysconfig_ops_write,
+ .write_complete = sysconfig_ops_write_complete,
+};
+
+int sysconfig_probe(struct sysconfig_priv *priv)
+{
+ struct gpio_desc *program, *init, *done;
+ struct device *dev = priv->dev;
+ struct fpga_manager *mgr;
+
+ if (!dev)
+ return -ENODEV;
+
+ if (!priv->command_transfer ||
+ !priv->bitstream_burst_write_init ||
+ !priv->bitstream_burst_write ||
+ !priv->bitstream_burst_write_complete) {
+ dev_err(dev, "Essential callback is missing\n");
+ return -EINVAL;
+ }
+
+ program = devm_gpiod_get_optional(dev, "program", GPIOD_OUT_LOW);
+ if (IS_ERR(program))
+ return dev_err_probe(dev, PTR_ERR(program),
+ "Failed to get PROGRAM GPIO\n");
+
+ init = devm_gpiod_get_optional(dev, "init", GPIOD_IN);
+ if (IS_ERR(init))
+ return dev_err_probe(dev, PTR_ERR(init),
+ "Failed to get INIT GPIO\n");
+
+ done = devm_gpiod_get_optional(dev, "done", GPIOD_IN);
+ if (IS_ERR(done))
+ return dev_err_probe(dev, PTR_ERR(done),
+ "Failed to get DONE GPIO\n");
+
+ priv->program = program;
+ priv->init = init;
+ priv->done = done;
+
+ mgr = devm_fpga_mgr_register(dev, "Lattice sysCONFIG FPGA Manager",
+ &sysconfig_fpga_mgr_ops, priv);
+
+ return PTR_ERR_OR_ZERO(mgr);
+}
+EXPORT_SYMBOL(sysconfig_probe);
+
+MODULE_DESCRIPTION("Lattice sysCONFIG FPGA Manager Core");
+MODULE_LICENSE("GPL");
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __LATTICE_SYSCONFIG_H
+#define __LATTICE_SYSCONFIG_H
+
+#define SYSCONFIG_ISC_ENABLE {0xC6, 0x00, 0x00, 0x00}
+#define SYSCONFIG_ISC_DISABLE {0x26, 0x00, 0x00, 0x00}
+#define SYSCONFIG_ISC_ERASE {0x0E, 0x01, 0x00, 0x00}
+#define SYSCONFIG_LSC_READ_STATUS {0x3C, 0x00, 0x00, 0x00}
+#define SYSCONFIG_LSC_CHECK_BUSY {0xF0, 0x00, 0x00, 0x00}
+#define SYSCONFIG_LSC_REFRESH {0x79, 0x00, 0x00, 0x00}
+#define SYSCONFIG_LSC_INIT_ADDR {0x46, 0x00, 0x00, 0x00}
+#define SYSCONFIG_LSC_BITSTREAM_BURST {0x7a, 0x00, 0x00, 0x00}
+
+#define SYSCONFIG_STATUS_DONE BIT(8)
+#define SYSCONFIG_STATUS_BUSY BIT(12)
+#define SYSCONFIG_STATUS_FAIL BIT(13)
+#define SYSCONFIG_STATUS_ERR GENMASK(25, 23)
+
+#define SYSCONFIG_POLL_INTERVAL_US 30
+#define SYSCONFIG_POLL_BUSY_TIMEOUT_US 1000000
+#define SYSCONFIG_POLL_GPIO_TIMEOUT_US 100000
+
+struct sysconfig_priv {
+ struct gpio_desc *program;
+ struct gpio_desc *init;
+ struct gpio_desc *done;
+ struct device *dev;
+ int (*command_transfer)(struct sysconfig_priv *priv, const void *tx_buf,
+ size_t tx_len, void *rx_buf, size_t rx_len);
+ int (*bitstream_burst_write_init)(struct sysconfig_priv *priv);
+ int (*bitstream_burst_write)(struct sysconfig_priv *priv,
+ const char *tx_buf, size_t tx_len);
+ int (*bitstream_burst_write_complete)(struct sysconfig_priv *priv);
+};
+
+int sysconfig_probe(struct sysconfig_priv *priv);
+
+#endif /* __LATTICE_SYSCONFIG_H */
return priv->irq;
priv->clk = devm_clk_get(dev, "ref_clk");
- if (IS_ERR(priv->clk)) {
- if (PTR_ERR(priv->clk) != -EPROBE_DEFER)
- dev_err(dev, "input clock not found\n");
- return PTR_ERR(priv->clk);
- }
+ if (IS_ERR(priv->clk))
+ return dev_err_probe(dev, PTR_ERR(priv->clk),
+ "input clock not found\n");
err = clk_prepare_enable(priv->clk);
if (err) {
[GNSS_TYPE_MTK] = "MTK",
};
-static const char *gnss_type_name(struct gnss_device *gdev)
+static const char *gnss_type_name(const struct gnss_device *gdev)
{
const char *name = NULL;
};
ATTRIBUTE_GROUPS(gnss);
-static int gnss_uevent(struct device *dev, struct kobj_uevent_env *env)
+static int gnss_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
- struct gnss_device *gdev = to_gnss_device(dev);
+ const struct gnss_device *gdev = to_gnss_device(dev);
int ret;
ret = add_uevent_var(env, "GNSS_TYPE=%s", gnss_type_name(gdev));
spin_unlock_irqrestore(&mvpwm->lock, flags);
}
-static void mvebu_pwm_get_state(struct pwm_chip *chip,
- struct pwm_device *pwm,
- struct pwm_state *state) {
+static int mvebu_pwm_get_state(struct pwm_chip *chip,
+ struct pwm_device *pwm,
+ struct pwm_state *state)
+{
struct mvebu_pwm *mvpwm = to_mvebu_pwm(chip);
struct mvebu_gpio_chip *mvchip = mvpwm->mvchip;
state->enabled = false;
spin_unlock_irqrestore(&mvpwm->lock, flags);
+
+ return 0;
}
static int mvebu_pwm_apply(struct pwm_chip *chip, struct pwm_device *pwm,
return ret;
}
-static void ti_sn_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int ti_sn_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct ti_sn65dsi86 *pdata = pwm_chip_to_ti_sn_bridge(chip);
unsigned int pwm_en_inv;
ret = regmap_read(pdata->regmap, SN_PWM_EN_INV_REG, &pwm_en_inv);
if (ret)
- return;
+ return ret;
ret = ti_sn65dsi86_read_u16(pdata, SN_BACKLIGHT_SCALE_REG, &scale);
if (ret)
- return;
+ return ret;
ret = ti_sn65dsi86_read_u16(pdata, SN_BACKLIGHT_REG, &backlight);
if (ret)
- return;
+ return ret;
ret = regmap_read(pdata->regmap, SN_PWM_PRE_DIV_REG, &pre_div);
if (ret)
- return;
+ return ret;
state->enabled = FIELD_GET(SN_PWM_EN_MASK, pwm_en_inv);
if (FIELD_GET(SN_PWM_INV_MASK, pwm_en_inv))
if (state->duty_cycle > state->period)
state->duty_cycle = state->period;
+
+ return 0;
}
static const struct pwm_ops ti_sn_pwm_ops = {
static void drm_sysfs_acpi_unregister(void) { }
#endif
-static char *drm_devnode(struct device *dev, umode_t *mode)
+static char *drm_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "dri/%s", dev_name(dev));
}
*/
#include <linux/debugfs.h>
+#include <linux/kstrtox.h>
#include <linux/workqueue.h>
#include <linux/greybus.h>
int retval;
bool user_request;
- retval = strtobool(buf, &user_request);
+ retval = kstrtobool(buf, &user_request);
if (retval)
return retval;
select CRC32
select POWER_SUPPLY
help
- Provides support for Sony PS5 controllers including support for
+ Provides support for Sony PS4/PS5 controllers including support for
its special functionalities e.g. touchpad, lights and motion
sensors.
in_data->sensor_virt_addr[i] = dma_alloc_coherent(dev, sizeof(int) * 8,
&cl_data->sensor_dma_addr[i],
GFP_KERNEL);
+ if (!in_data->sensor_virt_addr[i]) {
+ rc = -ENOMEM;
+ goto cleanup;
+ }
cl_data->sensor_sts[i] = SENSOR_DISABLED;
cl_data->sensor_requested_cnt[i] = 0;
cl_data->cur_hid_dev = i;
#define USB_DEVICE_ID_HP_X2_10_COVER 0x0755
#define I2C_DEVICE_ID_HP_ENVY_X360_15 0x2d05
#define I2C_DEVICE_ID_HP_ENVY_X360_15T_DR100 0x29CF
+#define I2C_DEVICE_ID_HP_ENVY_X360_EU0009NV 0x2CF9
#define I2C_DEVICE_ID_HP_SPECTRE_X360_15 0x2817
#define USB_DEVICE_ID_ASUS_UX550VE_TOUCHSCREEN 0x2544
#define USB_DEVICE_ID_ASUS_UX550_TOUCHSCREEN 0x2706
#define USB_DEVICE_ID_ORTEK_IHOME_IMAC_A210S 0x8003
#define USB_VENDOR_ID_PLANTRONICS 0x047f
+#define USB_DEVICE_ID_PLANTRONICS_BLACKWIRE_3210_SERIES 0xc055
#define USB_DEVICE_ID_PLANTRONICS_BLACKWIRE_3220_SERIES 0xc056
+#define USB_DEVICE_ID_PLANTRONICS_BLACKWIRE_3215_SERIES 0xc057
+#define USB_DEVICE_ID_PLANTRONICS_BLACKWIRE_3225_SERIES 0xc058
#define USB_VENDOR_ID_PANASONIC 0x04da
#define USB_DEVICE_ID_PANABOARD_UBT780 0x1044
HID_BATTERY_QUIRK_IGNORE },
{ HID_I2C_DEVICE(USB_VENDOR_ID_ELAN, I2C_DEVICE_ID_HP_ENVY_X360_15T_DR100),
HID_BATTERY_QUIRK_IGNORE },
+ { HID_I2C_DEVICE(USB_VENDOR_ID_ELAN, I2C_DEVICE_ID_HP_ENVY_X360_EU0009NV),
+ HID_BATTERY_QUIRK_IGNORE },
{ HID_I2C_DEVICE(USB_VENDOR_ID_ELAN, I2C_DEVICE_ID_HP_SPECTRE_X360_15),
HID_BATTERY_QUIRK_IGNORE },
{ HID_I2C_DEVICE(USB_VENDOR_ID_ELAN, I2C_DEVICE_ID_SURFACE_GO_TOUCHSCREEN),
struct hid_device *hid = hidpp->hid_dev;
struct hid_input *hidinput;
struct input_dev *dev;
- const struct usb_device_descriptor *udesc = &(hid_to_usb_dev(hid)->descriptor);
- const u16 bcdDevice = le16_to_cpu(udesc->bcdDevice);
+ struct usb_device_descriptor *udesc;
+ u16 bcdDevice;
struct ff_device *ff;
int error, j, num_slots = data->num_effects;
u8 version;
+ if (!hid_is_usb(hid)) {
+ hid_err(hid, "device is not USB\n");
+ return -ENODEV;
+ }
+
if (list_empty(&hid->inputs)) {
hid_err(hid, "no inputs found\n");
return -ENODEV;
}
/* Get firmware release */
+ udesc = &(hid_to_usb_dev(hid)->descriptor);
+ bcdDevice = le16_to_cpu(udesc->bcdDevice);
version = bcdDevice & 255;
/* Set supported force feedback capabilities */
return ret;
}
- ret = hid_hw_start(hdev, HID_CONNECT_HIDRAW);
+ /*
+ * This driver uses the .raw_event callback and therefore does not need any
+ * HID_CONNECT_xxx flags.
+ */
+ ret = hid_hw_start(hdev, 0);
if (ret) {
hid_err(hdev, "can't start hardware\n");
return ret;
}
+ hid_info(hdev, "USB HID v%x.%02x Device [%s] on %s\n", hdev->version >> 8,
+ hdev->version & 0xff, hdev->name, hdev->phys);
+
ret = hid_hw_open(hdev);
if (ret) {
hid_err(hdev, "can't open device\n");
mcp->adapter.retries = 1;
mcp->adapter.dev.parent = &hdev->dev;
snprintf(mcp->adapter.name, sizeof(mcp->adapter.name),
- "MCP2221 usb-i2c bridge on hidraw%d",
- ((struct hidraw *)hdev->hidraw)->minor);
+ "MCP2221 usb-i2c bridge");
ret = devm_i2c_add_adapter(&hdev->dev, &mcp->adapter);
if (ret) {
HID_DEVICE(BUS_I2C, HID_GROUP_MULTITOUCH_WIN_8,
USB_VENDOR_ID_ELAN, 0x313a) },
+ { .driver_data = MT_CLS_WIN_8_FORCE_MULTI_INPUT,
+ HID_DEVICE(BUS_I2C, HID_GROUP_MULTITOUCH_WIN_8,
+ USB_VENDOR_ID_ELAN, 0x3148) },
+
/* Elitegroup panel */
{ .driver_data = MT_CLS_SERIAL,
MT_USB_DEVICE(USB_VENDOR_ID_ELITEGROUP,
static const struct hid_device_id plantronics_devices[] = {
{ HID_USB_DEVICE(USB_VENDOR_ID_PLANTRONICS,
+ USB_DEVICE_ID_PLANTRONICS_BLACKWIRE_3210_SERIES),
+ .driver_data = PLT_QUIRK_DOUBLE_VOLUME_KEYS },
+ { HID_USB_DEVICE(USB_VENDOR_ID_PLANTRONICS,
USB_DEVICE_ID_PLANTRONICS_BLACKWIRE_3220_SERIES),
.driver_data = PLT_QUIRK_DOUBLE_VOLUME_KEYS },
+ { HID_USB_DEVICE(USB_VENDOR_ID_PLANTRONICS,
+ USB_DEVICE_ID_PLANTRONICS_BLACKWIRE_3215_SERIES),
+ .driver_data = PLT_QUIRK_DOUBLE_VOLUME_KEYS },
+ { HID_USB_DEVICE(USB_VENDOR_ID_PLANTRONICS,
+ USB_DEVICE_ID_PLANTRONICS_BLACKWIRE_3225_SERIES),
+ .driver_data = PLT_QUIRK_DOUBLE_VOLUME_KEYS },
{ HID_USB_DEVICE(USB_VENDOR_ID_PLANTRONICS, HID_ANY_ID) },
{ }
};
if (ret != sizeof(ds4->base.mac_address))
return -EINVAL;
- ret = 0;
+ return 0;
}
err_free:
hid_hw_output_report(hdev, (u8 *)report, MOTION_REPORT_0x02_SIZE);
}
+#ifdef CONFIG_SONY_FF
static inline void sony_send_output_report(struct sony_sc *sc)
{
if (sc->send_output_report)
sc->send_output_report(sc);
}
+#endif
static void sony_state_worker(struct work_struct *work)
{
.llseek = noop_llseek,
};
-static char *hiddev_devnode(struct device *dev, umode_t *mode)
+static char *hiddev_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "usb/%s", dev_name(dev));
}
{
struct wacom *wacom = hid_get_drvdata(hdev);
+ if (wacom->wacom_wac.features.type == BOOTLOADER)
+ return 0;
+
if (size > WACOM_PKGLEN_MAX)
return 1;
return error;
}
+ if (features->type == BOOTLOADER) {
+ hid_warn(hdev, "Using device in hidraw-only mode");
+ return hid_hw_start(hdev, HID_CONNECT_HIDRAW);
+ }
+
error = wacom_parse_and_register(wacom, false);
if (error)
return error;
static const struct wacom_features wacom_features_HID_ANY_ID =
{ "Wacom HID", .type = HID_GENERIC, .oVid = HID_ANY_ID, .oPid = HID_ANY_ID };
+static const struct wacom_features wacom_features_0x94 =
+ { "Wacom Bootloader", .type = BOOTLOADER };
+
#define USB_DEVICE_WACOM(prod) \
HID_DEVICE(BUS_USB, HID_GROUP_WACOM, USB_VENDOR_ID_WACOM, prod),\
.driver_data = (kernel_ulong_t)&wacom_features_##prod
{ USB_DEVICE_WACOM(0x84) },
{ USB_DEVICE_WACOM(0x90) },
{ USB_DEVICE_WACOM(0x93) },
+ { USB_DEVICE_WACOM(0x94) },
{ USB_DEVICE_WACOM(0x97) },
{ USB_DEVICE_WACOM(0x9A) },
{ USB_DEVICE_WACOM(0x9F) },
MTTPC,
MTTPC_B,
HID_GENERIC,
+ BOOTLOADER,
MAX_TYPE
};
ssip_xmit(cl);
}
-static int ssip_pn_xmit(struct sk_buff *skb, struct net_device *dev)
+static netdev_tx_t ssip_pn_xmit(struct sk_buff *skb, struct net_device *dev)
{
struct hsi_client *cl = to_hsi_client(dev->dev.parent);
struct ssi_protocol *ssi = hsi_client_drvdata(cl);
dev->stats.tx_packets++;
dev->stats.tx_bytes += skb->len;
- return 0;
+ return NETDEV_TX_OK;
drop2:
hsi_free_msg(msg);
drop:
inc_dropped:
dev->stats.tx_dropped++;
- return 0;
+ return NETDEV_TX_OK;
}
/* CMT reset event handler */
platform_set_drvdata(pd, ssi);
err = ssi_add_controller(ssi, pd);
- if (err < 0)
+ if (err < 0) {
+ hsi_put_controller(ssi);
goto out1;
+ }
pm_runtime_enable(&pd->dev);
device_for_each_child(&pd->dev, NULL, ssi_remove_ports);
out2:
ssi_remove_controller(ssi);
+ pm_runtime_disable(&pd->dev);
out1:
platform_set_drvdata(pd, NULL);
- pm_runtime_disable(&pd->dev);
return err;
}
if (ret)
return ret;
- return platform_driver_register(&ssi_port_pdriver);
+ ret = platform_driver_register(&ssi_port_pdriver);
+ if (ret) {
+ platform_driver_unregister(&ssi_pdriver);
+ return ret;
+ }
+
+ return 0;
}
module_init(ssi_init);
* if we found a matching csdev then update the ECT
* association pointer for the device with this CTI.
*/
- coresight_set_assoc_ectdev_mutex(csdev->ect_dev,
+ coresight_set_assoc_ectdev_mutex(csdev,
ect_item->csdev);
break;
}
struct etm4_init_arg {
unsigned int pid;
- struct etmv4_drvdata *drvdata;
+ struct device *dev;
struct csdev_access *csa;
};
+static DEFINE_PER_CPU(struct etm4_init_arg *, delayed_probe);
+static int etm4_probe_cpu(unsigned int cpu);
+
/*
* Check if TRCSSPCICRn(i) is implemented for a given instance.
*
struct csdev_access *csa;
int i;
- drvdata = init_arg->drvdata;
+ drvdata = dev_get_drvdata(init_arg->dev);
csa = init_arg->csa;
/*
/*
* If filters::ssstatus == 1, trace acquisition was
* started but the process was yanked away before the
- * the stop address was hit. As such the start/stop
+ * stop address was hit. As such the start/stop
* logic needs to be re-started so that tracing can
* resume where it left.
*
static int etm4_online_cpu(unsigned int cpu)
{
if (!etmdrvdata[cpu])
- return 0;
+ return etm4_probe_cpu(cpu);
if (etmdrvdata[cpu]->boot_enable && !etmdrvdata[cpu]->sticky_enable)
coresight_enable(etmdrvdata[cpu]->csdev);
}
}
-static int etm4_probe(struct device *dev, void __iomem *base, u32 etm_pid)
+static int etm4_add_coresight_dev(struct etm4_init_arg *init_arg)
{
int ret;
struct coresight_platform_data *pdata = NULL;
- struct etmv4_drvdata *drvdata;
+ struct device *dev = init_arg->dev;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev);
struct coresight_desc desc = { 0 };
- struct etm4_init_arg init_arg = { 0 };
u8 major, minor;
char *type_name;
- drvdata = devm_kzalloc(dev, sizeof(*drvdata), GFP_KERNEL);
if (!drvdata)
- return -ENOMEM;
-
- dev_set_drvdata(dev, drvdata);
-
- if (pm_save_enable == PARAM_PM_SAVE_FIRMWARE)
- pm_save_enable = coresight_loses_context_with_cpu(dev) ?
- PARAM_PM_SAVE_SELF_HOSTED : PARAM_PM_SAVE_NEVER;
-
- if (pm_save_enable != PARAM_PM_SAVE_NEVER) {
- drvdata->save_state = devm_kmalloc(dev,
- sizeof(struct etmv4_save_state), GFP_KERNEL);
- if (!drvdata->save_state)
- return -ENOMEM;
- }
-
- drvdata->base = base;
-
- spin_lock_init(&drvdata->spinlock);
-
- drvdata->cpu = coresight_get_cpu(dev);
- if (drvdata->cpu < 0)
- return drvdata->cpu;
-
- init_arg.drvdata = drvdata;
- init_arg.csa = &desc.access;
- init_arg.pid = etm_pid;
+ return -EINVAL;
- if (smp_call_function_single(drvdata->cpu,
- etm4_init_arch_data, &init_arg, 1))
- dev_err(dev, "ETM arch init failed\n");
+ desc.access = *init_arg->csa;
if (!drvdata->arch)
return -EINVAL;
return 0;
}
+static int etm4_probe(struct device *dev, void __iomem *base, u32 etm_pid)
+{
+ struct etmv4_drvdata *drvdata;
+ struct csdev_access access = { 0 };
+ struct etm4_init_arg init_arg = { 0 };
+ struct etm4_init_arg *delayed;
+
+ drvdata = devm_kzalloc(dev, sizeof(*drvdata), GFP_KERNEL);
+ if (!drvdata)
+ return -ENOMEM;
+
+ dev_set_drvdata(dev, drvdata);
+
+ if (pm_save_enable == PARAM_PM_SAVE_FIRMWARE)
+ pm_save_enable = coresight_loses_context_with_cpu(dev) ?
+ PARAM_PM_SAVE_SELF_HOSTED : PARAM_PM_SAVE_NEVER;
+
+ if (pm_save_enable != PARAM_PM_SAVE_NEVER) {
+ drvdata->save_state = devm_kmalloc(dev,
+ sizeof(struct etmv4_save_state), GFP_KERNEL);
+ if (!drvdata->save_state)
+ return -ENOMEM;
+ }
+
+ drvdata->base = base;
+
+ spin_lock_init(&drvdata->spinlock);
+
+ drvdata->cpu = coresight_get_cpu(dev);
+ if (drvdata->cpu < 0)
+ return drvdata->cpu;
+
+ init_arg.dev = dev;
+ init_arg.csa = &access;
+ init_arg.pid = etm_pid;
+
+ /*
+ * Serialize against CPUHP callbacks to avoid race condition
+ * between the smp call and saving the delayed probe.
+ */
+ cpus_read_lock();
+ if (smp_call_function_single(drvdata->cpu,
+ etm4_init_arch_data, &init_arg, 1)) {
+ /* The CPU was offline, try again once it comes online. */
+ delayed = devm_kmalloc(dev, sizeof(*delayed), GFP_KERNEL);
+ if (!delayed) {
+ cpus_read_unlock();
+ return -ENOMEM;
+ }
+
+ *delayed = init_arg;
+
+ per_cpu(delayed_probe, drvdata->cpu) = delayed;
+
+ cpus_read_unlock();
+ return 0;
+ }
+ cpus_read_unlock();
+
+ return etm4_add_coresight_dev(&init_arg);
+}
+
static int etm4_probe_amba(struct amba_device *adev, const struct amba_id *id)
{
void __iomem *base;
return ret;
}
+static int etm4_probe_cpu(unsigned int cpu)
+{
+ int ret;
+ struct etm4_init_arg init_arg;
+ struct csdev_access access = { 0 };
+ struct etm4_init_arg *iap = *this_cpu_ptr(&delayed_probe);
+
+ if (!iap)
+ return 0;
+
+ init_arg = *iap;
+ devm_kfree(init_arg.dev, iap);
+ *this_cpu_ptr(&delayed_probe) = NULL;
+
+ ret = pm_runtime_resume_and_get(init_arg.dev);
+ if (ret < 0) {
+ dev_err(init_arg.dev, "Failed to get PM runtime!\n");
+ return 0;
+ }
+
+ init_arg.csa = &access;
+ etm4_init_arch_data(&init_arg);
+
+ etm4_add_coresight_dev(&init_arg);
+
+ pm_runtime_put(init_arg.dev);
+ return 0;
+}
+
static struct amba_cs_uci_id uci_id_etm4[] = {
{
/* ETMv4 UCI data */
int cpu = *(int *)info;
etmdrvdata[cpu] = NULL;
+ per_cpu(delayed_probe, cpu) = NULL;
}
static int __exit etm4_remove_dev(struct etmv4_drvdata *drvdata)
{
- etm_perf_symlink(drvdata->csdev, false);
+ bool had_delayed_probe;
/*
* Taking hotplug lock here to avoid racing between etm4_remove_dev()
* and CPU hotplug call backs.
*/
cpus_read_lock();
+
+ had_delayed_probe = per_cpu(delayed_probe, drvdata->cpu);
+
/*
* The readers for etmdrvdata[] are CPU hotplug call backs
* and PM notification call backs. Change etmdrvdata[i] on
* inside one call back function.
*/
if (smp_call_function_single(drvdata->cpu, clear_etmdrvdata, &drvdata->cpu, 1))
- etmdrvdata[drvdata->cpu] = NULL;
+ clear_etmdrvdata(&drvdata->cpu);
cpus_read_unlock();
- cscfg_unregister_csdev(drvdata->csdev);
- coresight_unregister(drvdata->csdev);
+ if (!had_delayed_probe) {
+ etm_perf_symlink(drvdata->csdev, false);
+ cscfg_unregister_csdev(drvdata->csdev);
+ coresight_unregister(drvdata->csdev);
+ }
return 0;
}
static void arm_trbe_remove_cpuhp(struct trbe_drvdata *drvdata)
{
+ cpuhp_state_remove_instance(drvdata->trbe_online, &drvdata->hotplug_node);
cpuhp_remove_multi_state(drvdata->trbe_online);
}
- ABI Documentation
- Audit driviers/iio/staging/Documentation
-- Replace iio_dev->mlock by either a local lock or use
-iio_claim_direct.(Requires analysis of the purpose of the lock.)
-
- Converting drivers from device tree centric to more generic
property handlers.
To compile this driver as a module, choose M here. The module
will be called st_accel_spi.
+config IIO_KX022A
+ tristate
+
+config IIO_KX022A_SPI
+ tristate "Kionix KX022A tri-axis digital accelerometer SPI interface"
+ depends on SPI
+ select IIO_KX022A
+ select REGMAP_SPI
+ help
+ Enable support for the Kionix KX022A digital tri-axis
+ accelerometer connected to I2C interface.
+
+config IIO_KX022A_I2C
+ tristate "Kionix KX022A tri-axis digital accelerometer I2C interface"
+ depends on I2C
+ select IIO_KX022A
+ select REGMAP_I2C
+ help
+ Enable support for the Kionix KX022A digital tri-axis
+ accelerometer connected to I2C interface.
+
config KXSD9
tristate "Kionix KXSD9 Accelerometer Driver"
select IIO_BUFFER
obj-$(CONFIG_FXLS8962AF_I2C) += fxls8962af-i2c.o
obj-$(CONFIG_FXLS8962AF_SPI) += fxls8962af-spi.o
obj-$(CONFIG_HID_SENSOR_ACCEL_3D) += hid-sensor-accel-3d.o
+obj-$(CONFIG_IIO_KX022A) += kionix-kx022a.o
+obj-$(CONFIG_IIO_KX022A_I2C) += kionix-kx022a-i2c.o
+obj-$(CONFIG_IIO_KX022A_SPI) += kionix-kx022a-spi.o
obj-$(CONFIG_KXCJK1013) += kxcjk-1013.o
obj-$(CONFIG_KXSD9) += kxsd9.o
obj-$(CONFIG_KXSD9_SPI) += kxsd9-spi.o
if (ret)
return ret;
- ret = adis_initial_startup(st);
+ ret = __adis_initial_startup(st);
if (ret)
return ret;
if (ret)
return ret;
- ret = adis_initial_startup(st);
+ ret = __adis_initial_startup(st);
if (ret)
return ret;
#include <linux/regmap.h>
+enum adxl355_device_type {
+ ADXL355,
+ ADXL359,
+};
+
+struct adxl355_fractional_type {
+ int integer;
+ int decimal;
+};
+
struct device;
+struct adxl355_chip_info {
+ const char *name;
+ u8 part_id;
+ struct adxl355_fractional_type accel_scale;
+ struct adxl355_fractional_type temp_offset;
+};
+
extern const struct regmap_access_table adxl355_readable_regs_tbl;
extern const struct regmap_access_table adxl355_writeable_regs_tbl;
+extern const struct adxl355_chip_info adxl35x_chip_info[];
int adxl355_core_probe(struct device *dev, struct regmap *regmap,
- const char *name);
+ const struct adxl355_chip_info *chip_info);
#endif /* _ADXL355_H_ */
#define ADXL355_DEVID_AD_VAL 0xAD
#define ADXL355_DEVID_MST_VAL 0x1D
#define ADXL355_PARTID_VAL 0xED
+#define ADXL359_PARTID_VAL 0xE9
#define ADXL355_RESET_CODE 0x52
static const struct regmap_range adxl355_read_reg_range[] = {
};
EXPORT_SYMBOL_NS_GPL(adxl355_writeable_regs_tbl, IIO_ADXL355);
+const struct adxl355_chip_info adxl35x_chip_info[] = {
+ [ADXL355] = {
+ .name = "adxl355",
+ .part_id = ADXL355_PARTID_VAL,
+ /*
+ * At +/- 2g with 20-bit resolution, scale is given in datasheet
+ * as 3.9ug/LSB = 0.0000039 * 9.80665 = 0.00003824593 m/s^2.
+ */
+ .accel_scale = {
+ .integer = 0,
+ .decimal = 38245,
+ },
+ /*
+ * The datasheet defines an intercept of 1885 LSB at 25 degC
+ * and a slope of -9.05 LSB/C. The following formula can be used
+ * to find the temperature:
+ * Temp = ((RAW - 1885)/(-9.05)) + 25 but this doesn't follow
+ * the format of the IIO which is Temp = (RAW + OFFSET) * SCALE.
+ * Hence using some rearranging we get the scale as -110.497238
+ * and offset as -2111.25.
+ */
+ .temp_offset = {
+ .integer = -2111,
+ .decimal = 250000,
+ },
+ },
+ [ADXL359] = {
+ .name = "adxl359",
+ .part_id = ADXL359_PARTID_VAL,
+ /*
+ * At +/- 10g with 20-bit resolution, scale is given in datasheet
+ * as 19.5ug/LSB = 0.0000195 * 9.80665 = 0.0.00019122967 m/s^2.
+ */
+ .accel_scale = {
+ .integer = 0,
+ .decimal = 191229,
+ },
+ /*
+ * The datasheet defines an intercept of 1852 LSB at 25 degC
+ * and a slope of -9.05 LSB/C. The following formula can be used
+ * to find the temperature:
+ * Temp = ((RAW - 1852)/(-9.05)) + 25 but this doesn't follow
+ * the format of the IIO which is Temp = (RAW + OFFSET) * SCALE.
+ * Hence using some rearranging we get the scale as -110.497238
+ * and offset as -2079.25.
+ */
+ .temp_offset = {
+ .integer = -2079,
+ .decimal = 250000,
+ },
+ },
+};
+EXPORT_SYMBOL_NS_GPL(adxl35x_chip_info, IIO_ADXL355);
+
enum adxl355_op_mode {
ADXL355_MEASUREMENT,
ADXL355_STANDBY,
};
struct adxl355_data {
+ const struct adxl355_chip_info *chip_info;
struct regmap *regmap;
struct device *dev;
struct mutex lock; /* lock to protect op_mode */
if (ret)
return ret;
- if (regval != ADXL355_PARTID_VAL) {
- dev_err(data->dev, "Invalid DEV ID 0x%02x\n", regval);
- return -ENODEV;
- }
+ if (regval != ADXL355_PARTID_VAL)
+ dev_warn(data->dev, "Invalid DEV ID 0x%02x\n", regval);
/*
* Perform a software reset to make sure the device is in a consistent
case IIO_CHAN_INFO_SCALE:
switch (chan->type) {
- /*
- * The datasheet defines an intercept of 1885 LSB at 25 degC
- * and a slope of -9.05 LSB/C. The following formula can be used
- * to find the temperature:
- * Temp = ((RAW - 1885)/(-9.05)) + 25 but this doesn't follow
- * the format of the IIO which is Temp = (RAW + OFFSET) * SCALE.
- * Hence using some rearranging we get the scale as -110.497238
- * and offset as -2111.25.
- */
case IIO_TEMP:
+ /*
+ * Temperature scale is -110.497238.
+ * See the detailed explanation in adxl35x_chip_info
+ * definition above.
+ */
*val = -110;
*val2 = 497238;
return IIO_VAL_INT_PLUS_MICRO;
- /*
- * At +/- 2g with 20-bit resolution, scale is given in datasheet
- * as 3.9ug/LSB = 0.0000039 * 9.80665 = 0.00003824593 m/s^2.
- */
case IIO_ACCEL:
- *val = 0;
- *val2 = 38245;
+ *val = data->chip_info->accel_scale.integer;
+ *val2 = data->chip_info->accel_scale.decimal;
return IIO_VAL_INT_PLUS_NANO;
default:
return -EINVAL;
}
case IIO_CHAN_INFO_OFFSET:
- *val = -2111;
- *val2 = 250000;
+ *val = data->chip_info->temp_offset.integer;
+ *val2 = data->chip_info->temp_offset.decimal;
return IIO_VAL_INT_PLUS_MICRO;
case IIO_CHAN_INFO_CALIBBIAS:
*val = sign_extend32(data->calibbias[chan->address], 15);
}
int adxl355_core_probe(struct device *dev, struct regmap *regmap,
- const char *name)
+ const struct adxl355_chip_info *chip_info)
{
struct adxl355_data *data;
struct iio_dev *indio_dev;
data->regmap = regmap;
data->dev = dev;
data->op_mode = ADXL355_STANDBY;
+ data->chip_info = chip_info;
mutex_init(&data->lock);
- indio_dev->name = name;
+ indio_dev->name = chip_info->name;
indio_dev->info = &adxl355_info;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->channels = adxl355_channels;
static int adxl355_i2c_probe(struct i2c_client *client)
{
struct regmap *regmap;
+ const struct adxl355_chip_info *chip_data;
+ const struct i2c_device_id *adxl355;
+
+ chip_data = device_get_match_data(&client->dev);
+ if (!chip_data) {
+ adxl355 = to_i2c_driver(client->dev.driver)->id_table;
+ if (!adxl355)
+ return -EINVAL;
+
+ chip_data = (void *)i2c_match_id(adxl355, client)->driver_data;
+
+ if (!chip_data)
+ return -EINVAL;
+ }
regmap = devm_regmap_init_i2c(client, &adxl355_i2c_regmap_config);
if (IS_ERR(regmap)) {
return PTR_ERR(regmap);
}
- return adxl355_core_probe(&client->dev, regmap, client->name);
+ return adxl355_core_probe(&client->dev, regmap, chip_data);
}
static const struct i2c_device_id adxl355_i2c_id[] = {
- { "adxl355", 0 },
+ { "adxl355", (kernel_ulong_t)&adxl35x_chip_info[ADXL355] },
+ { "adxl359", (kernel_ulong_t)&adxl35x_chip_info[ADXL359] },
{ }
};
MODULE_DEVICE_TABLE(i2c, adxl355_i2c_id);
static const struct of_device_id adxl355_of_match[] = {
- { .compatible = "adi,adxl355" },
+ { .compatible = "adi,adxl355", .data = &adxl35x_chip_info[ADXL355] },
+ { .compatible = "adi,adxl359", .data = &adxl35x_chip_info[ADXL359] },
{ }
};
MODULE_DEVICE_TABLE(of, adxl355_of_match);
#include <linux/mod_devicetable.h>
#include <linux/regmap.h>
#include <linux/spi/spi.h>
+#include <linux/property.h>
#include "adxl355.h"
static int adxl355_spi_probe(struct spi_device *spi)
{
- const struct spi_device_id *id = spi_get_device_id(spi);
+ const struct adxl355_chip_info *chip_data;
struct regmap *regmap;
+ chip_data = device_get_match_data(&spi->dev);
+ if (!chip_data) {
+ chip_data = (void *)spi_get_device_id(spi)->driver_data;
+
+ if (!chip_data)
+ return -EINVAL;
+ }
+
regmap = devm_regmap_init_spi(spi, &adxl355_spi_regmap_config);
if (IS_ERR(regmap)) {
dev_err(&spi->dev, "Error initializing spi regmap: %ld\n",
return PTR_ERR(regmap);
}
- return adxl355_core_probe(&spi->dev, regmap, id->name);
+ return adxl355_core_probe(&spi->dev, regmap, chip_data);
}
static const struct spi_device_id adxl355_spi_id[] = {
- { "adxl355", 0 },
+ { "adxl355", (kernel_ulong_t)&adxl35x_chip_info[ADXL355] },
+ { "adxl359", (kernel_ulong_t)&adxl35x_chip_info[ADXL359] },
{ }
};
MODULE_DEVICE_TABLE(spi, adxl355_spi_id);
static const struct of_device_id adxl355_of_match[] = {
- { .compatible = "adi,adxl355" },
+ { .compatible = "adi,adxl355", .data = &adxl35x_chip_info[ADXL355] },
+ { .compatible = "adi,adxl359", .data = &adxl35x_chip_info[ADXL359] },
{ }
};
MODULE_DEVICE_TABLE(of, adxl355_of_match);
struct device *dev;
struct regmap *regmap;
- struct regulator_bulk_data regulators[2];
-
/*
* Synchronize access to members of driver state, and ensure atomicity
* of consecutive regmap operations.
return sysfs_emit(buf, "%d\n", fifo_watermark);
}
-static ssize_t hwfifo_watermark_min_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- return sysfs_emit(buf, "%s\n", "1");
-}
-
-static ssize_t hwfifo_watermark_max_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- return sysfs_emit(buf, "%s\n", __stringify(ADXL367_FIFO_MAX_WATERMARK));
-}
-
-static IIO_DEVICE_ATTR_RO(hwfifo_watermark_min, 0);
-static IIO_DEVICE_ATTR_RO(hwfifo_watermark_max, 0);
+IIO_STATIC_CONST_DEVICE_ATTR(hwfifo_watermark_min, "1");
+IIO_STATIC_CONST_DEVICE_ATTR(hwfifo_watermark_max,
+ __stringify(ADXL367_FIFO_MAX_WATERMARK));
static IIO_DEVICE_ATTR(hwfifo_watermark, 0444,
adxl367_get_fifo_watermark, NULL, 0);
static IIO_DEVICE_ATTR(hwfifo_enabled, 0444,
adxl367_get_fifo_enabled, NULL, 0);
-static const struct attribute *adxl367_fifo_attributes[] = {
- &iio_dev_attr_hwfifo_watermark_min.dev_attr.attr,
- &iio_dev_attr_hwfifo_watermark_max.dev_attr.attr,
- &iio_dev_attr_hwfifo_watermark.dev_attr.attr,
- &iio_dev_attr_hwfifo_enabled.dev_attr.attr,
+static const struct iio_dev_attr *adxl367_fifo_attributes[] = {
+ &iio_dev_attr_hwfifo_watermark_min,
+ &iio_dev_attr_hwfifo_watermark_max,
+ &iio_dev_attr_hwfifo_watermark,
+ &iio_dev_attr_hwfifo_enabled,
NULL,
};
return adxl367_set_measure_en(st, true);
}
-static void adxl367_disable_regulators(void *data)
-{
- struct adxl367_state *st = data;
-
- regulator_bulk_disable(ARRAY_SIZE(st->regulators), st->regulators);
-}
-
int adxl367_probe(struct device *dev, const struct adxl367_ops *ops,
void *context, struct regmap *regmap, int irq)
{
+ static const char * const regulator_names[] = { "vdd", "vddio" };
struct iio_dev *indio_dev;
struct adxl367_state *st;
int ret;
indio_dev->info = &adxl367_info;
indio_dev->modes = INDIO_DIRECT_MODE;
- st->regulators[0].supply = "vdd";
- st->regulators[1].supply = "vddio";
-
- ret = devm_regulator_bulk_get(st->dev, ARRAY_SIZE(st->regulators),
- st->regulators);
+ ret = devm_regulator_bulk_get_enable(st->dev,
+ ARRAY_SIZE(regulator_names),
+ regulator_names);
if (ret)
return dev_err_probe(st->dev, ret,
"Failed to get regulators\n");
- ret = regulator_bulk_enable(ARRAY_SIZE(st->regulators), st->regulators);
- if (ret)
- return dev_err_probe(st->dev, ret,
- "Failed to enable regulators\n");
-
- ret = devm_add_action_or_reset(st->dev, adxl367_disable_regulators, st);
- if (ret)
- return dev_err_probe(st->dev, ret,
- "Failed to add regulators disable action\n");
-
ret = regmap_write(st->regmap, ADXL367_REG_RESET, ADXL367_RESET_CODE);
if (ret)
return ret;
.read_fifo = adxl367_i2c_read_fifo,
};
-static int adxl367_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int adxl367_i2c_probe(struct i2c_client *client)
{
struct adxl367_i2c_state *st;
struct regmap *regmap;
.name = "adxl367_i2c",
.of_match_table = adxl367_of_match,
},
- .probe = adxl367_i2c_probe,
+ .probe_new = adxl367_i2c_probe,
.id_table = adxl367_i2c_id,
};
return sprintf(buf, "%d\n", st->watermark);
}
-static ssize_t hwfifo_watermark_min_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- return sysfs_emit(buf, "%s\n", "1");
-}
-
-static ssize_t hwfifo_watermark_max_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- return sysfs_emit(buf, "%s\n", __stringify(ADXL372_FIFO_SIZE));
-}
-
-static IIO_DEVICE_ATTR_RO(hwfifo_watermark_min, 0);
-static IIO_DEVICE_ATTR_RO(hwfifo_watermark_max, 0);
+IIO_STATIC_CONST_DEVICE_ATTR(hwfifo_watermark_min, "1");
+IIO_STATIC_CONST_DEVICE_ATTR(hwfifo_watermark_max,
+ __stringify(ADXL372_FIFO_SIZE));
static IIO_DEVICE_ATTR(hwfifo_watermark, 0444,
adxl372_get_fifo_watermark, NULL, 0);
static IIO_DEVICE_ATTR(hwfifo_enabled, 0444,
adxl372_get_fifo_enabled, NULL, 0);
-static const struct attribute *adxl372_fifo_attributes[] = {
- &iio_dev_attr_hwfifo_watermark_min.dev_attr.attr,
- &iio_dev_attr_hwfifo_watermark_max.dev_attr.attr,
- &iio_dev_attr_hwfifo_watermark.dev_attr.attr,
- &iio_dev_attr_hwfifo_enabled.dev_attr.attr,
+static const struct iio_dev_attr *adxl372_fifo_attributes[] = {
+ &iio_dev_attr_hwfifo_watermark_min,
+ &iio_dev_attr_hwfifo_watermark_max,
+ &iio_dev_attr_hwfifo_watermark,
+ &iio_dev_attr_hwfifo_enabled,
NULL,
};
.readable_noinc_reg = adxl372_readable_noinc_reg,
};
-static int adxl372_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int adxl372_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct regmap *regmap;
unsigned int regval;
int ret;
.name = "adxl372_i2c",
.of_match_table = adxl372_of_match,
},
- .probe = adxl372_i2c_probe,
+ .probe_new = adxl372_i2c_probe,
.id_table = adxl372_i2c_id,
};
.reenable = bma180_trig_reen,
};
-static int bma180_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bma180_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct bma180_data *data;
struct iio_dev *indio_dev;
.pm = pm_sleep_ptr(&bma180_pm_ops),
.of_match_table = bma180_of_match,
},
- .probe = bma180_probe,
+ .probe_new = bma180_probe,
.remove = bma180_remove,
.id_table = bma180_ids,
};
ret = devm_regulator_bulk_get(data->dev,
ARRAY_SIZE(data->regulators),
data->regulators);
- if (ret) {
- if (ret != -EPROBE_DEFER)
- dev_err(data->dev,
- "Failed to get regulators: %d\n",
- ret);
+ if (ret)
+ return dev_err_probe(data->dev, ret, "Failed to get regulators: %d\n",
+ ret);
- return ret;
- }
ret = regulator_bulk_enable(ARRAY_SIZE(data->regulators),
data->regulators);
if (ret) {
#include "bma400.h"
-static int bma400_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bma400_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct regmap *regmap;
regmap = devm_regmap_init_i2c(client, &bma400_regmap_config);
.name = "bma400",
.of_match_table = bma400_of_i2c_match,
},
- .probe = bma400_i2c_probe,
+ .probe_new = bma400_i2c_probe,
.id_table = bma400_i2c_ids,
};
{ }
};
-static ssize_t hwfifo_watermark_min_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- return sysfs_emit(buf, "%s\n", "1");
-}
-
-static ssize_t hwfifo_watermark_max_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- return sysfs_emit(buf, "%s\n", __stringify(BMC150_ACCEL_FIFO_LENGTH));
-}
-
-static IIO_DEVICE_ATTR_RO(hwfifo_watermark_min, 0);
-static IIO_DEVICE_ATTR_RO(hwfifo_watermark_max, 0);
+IIO_STATIC_CONST_DEVICE_ATTR(hwfifo_watermark_min, "1");
+IIO_STATIC_CONST_DEVICE_ATTR(hwfifo_watermark_max,
+ __stringify(BMC150_ACCEL_FIFO_LENGTH));
static IIO_DEVICE_ATTR(hwfifo_enabled, S_IRUGO,
bmc150_accel_get_fifo_state, NULL, 0);
static IIO_DEVICE_ATTR(hwfifo_watermark, S_IRUGO,
bmc150_accel_get_fifo_watermark, NULL, 0);
-static const struct attribute *bmc150_accel_fifo_attributes[] = {
- &iio_dev_attr_hwfifo_watermark_min.dev_attr.attr,
- &iio_dev_attr_hwfifo_watermark_max.dev_attr.attr,
- &iio_dev_attr_hwfifo_watermark.dev_attr.attr,
- &iio_dev_attr_hwfifo_enabled.dev_attr.attr,
+static const struct iio_dev_attr *bmc150_accel_fifo_attributes[] = {
+ &iio_dev_attr_hwfifo_watermark_min,
+ &iio_dev_attr_hwfifo_watermark_max,
+ &iio_dev_attr_hwfifo_watermark,
+ &iio_dev_attr_hwfifo_enabled,
NULL,
};
enum bmc150_type type, const char *name,
bool block_supported)
{
- const struct attribute **fifo_attrs;
+ const struct iio_dev_attr **fifo_attrs;
struct bmc150_accel_data *data;
struct iio_dev *indio_dev;
int ret;
static void bmc150_acpi_dual_accel_remove(struct i2c_client *client) {}
#endif
-static int bmc150_accel_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bmc150_accel_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct regmap *regmap;
const char *name = NULL;
enum bmc150_type type = BOSCH_UNKNOWN;
.acpi_match_table = ACPI_PTR(bmc150_accel_acpi_match),
.pm = &bmc150_accel_pm_ops,
},
- .probe = bmc150_accel_probe,
+ .probe_new = bmc150_accel_probe,
.remove = bmc150_accel_remove,
.id_table = bmc150_accel_id,
};
da280_enable(client, false);
}
-static int da280_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int da280_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
int ret;
struct iio_dev *indio_dev;
struct da280_data *data;
.acpi_match_table = ACPI_PTR(da280_acpi_match),
.pm = pm_sleep_ptr(&da280_pm_ops),
},
- .probe = da280_probe,
+ .probe_new = da280_probe,
.id_table = da280_i2c_id,
};
da311_enable(client, false);
}
-static int da311_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int da311_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
.name = "da311",
.pm = pm_sleep_ptr(&da311_pm_ops),
},
- .probe = da311_probe,
+ .probe_new = da311_probe,
.id_table = da311_i2c_id,
};
.read_raw = dmard06_read_raw,
};
-static int dmard06_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int dmard06_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
MODULE_DEVICE_TABLE(of, dmard06_of_match);
static struct i2c_driver dmard06_driver = {
- .probe = dmard06_probe,
+ .probe_new = dmard06_probe,
.id_table = dmard06_id,
.driver = {
.name = DMARD06_DRV_NAME,
.read_raw = dmard09_read_raw,
};
-static int dmard09_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int dmard09_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
.driver = {
.name = DMARD09_DRV_NAME
},
- .probe = dmard09_probe,
+ .probe_new = dmard09_probe,
.id_table = dmard09_id,
};
dmard10_shutdown(client);
}
-static int dmard10_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int dmard10_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
.name = "dmard10",
.pm = pm_sleep_ptr(&dmard10_pm_ops),
},
- .probe = dmard10_probe,
+ .probe_new = dmard10_probe,
.id_table = dmard10_i2c_id,
};
struct fxls8962af_data {
struct regmap *regmap;
const struct fxls8962af_chip_info *chip_info;
- struct regulator *vdd_reg;
struct {
__le16 channels[3];
s64 ts __aligned(8);
return IRQ_NONE;
}
-static void fxls8962af_regulator_disable(void *data_ptr)
-{
- struct fxls8962af_data *data = data_ptr;
-
- regulator_disable(data->vdd_reg);
-}
-
static void fxls8962af_pm_disable(void *dev_ptr)
{
struct device *dev = dev_ptr;
if (ret)
return ret;
- data->vdd_reg = devm_regulator_get(dev, "vdd");
- if (IS_ERR(data->vdd_reg))
- return dev_err_probe(dev, PTR_ERR(data->vdd_reg),
- "Failed to get vdd regulator\n");
-
- ret = regulator_enable(data->vdd_reg);
- if (ret) {
- dev_err(dev, "Failed to enable vdd regulator: %d\n", ret);
- return ret;
- }
-
- ret = devm_add_action_or_reset(dev, fxls8962af_regulator_disable, data);
+ ret = devm_regulator_get_enable(dev, "vdd");
if (ret)
- return ret;
+ return dev_err_probe(dev, ret,
+ "Failed to get vdd regulator\n");
ret = regmap_read(data->regmap, FXLS8962AF_WHO_AM_I, ®);
if (ret)
}
EXPORT_SYMBOL_NS_GPL(fxls8962af_core_probe, IIO_FXLS8962AF);
-static int __maybe_unused fxls8962af_runtime_suspend(struct device *dev)
+static int fxls8962af_runtime_suspend(struct device *dev)
{
struct fxls8962af_data *data = iio_priv(dev_get_drvdata(dev));
int ret;
return 0;
}
-static int __maybe_unused fxls8962af_runtime_resume(struct device *dev)
+static int fxls8962af_runtime_resume(struct device *dev)
{
struct fxls8962af_data *data = iio_priv(dev_get_drvdata(dev));
return fxls8962af_active(data);
}
-static int __maybe_unused fxls8962af_suspend(struct device *dev)
+static int fxls8962af_suspend(struct device *dev)
{
struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct fxls8962af_data *data = iio_priv(indio_dev);
return 0;
}
-static int __maybe_unused fxls8962af_resume(struct device *dev)
+static int fxls8962af_resume(struct device *dev)
{
struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct fxls8962af_data *data = iio_priv(indio_dev);
return 0;
}
-const struct dev_pm_ops fxls8962af_pm_ops = {
- SET_SYSTEM_SLEEP_PM_OPS(fxls8962af_suspend, fxls8962af_resume)
- SET_RUNTIME_PM_OPS(fxls8962af_runtime_suspend,
- fxls8962af_runtime_resume, NULL)
+EXPORT_NS_GPL_DEV_PM_OPS(fxls8962af_pm_ops, IIO_FXLS8962AF) = {
+ SYSTEM_SLEEP_PM_OPS(fxls8962af_suspend, fxls8962af_resume)
+ RUNTIME_PM_OPS(fxls8962af_runtime_suspend, fxls8962af_runtime_resume, NULL)
};
-EXPORT_SYMBOL_NS_GPL(fxls8962af_pm_ops, IIO_FXLS8962AF);
MODULE_AUTHOR("Sean Nyekjaer <sean@geanix.com>");
MODULE_DESCRIPTION("NXP FXLS8962AF/FXLS8964AF accelerometer driver");
.driver = {
.name = "fxls8962af_i2c",
.of_match_table = fxls8962af_of_match,
- .pm = &fxls8962af_pm_ops,
+ .pm = pm_ptr(&fxls8962af_pm_ops),
},
.probe_new = fxls8962af_probe,
.id_table = fxls8962af_id,
static struct spi_driver fxls8962af_driver = {
.driver = {
.name = "fxls8962af_spi",
- .pm = &fxls8962af_pm_ops,
+ .pm = pm_ptr(&fxls8962af_pm_ops),
.of_match_table = fxls8962af_spi_of_match,
},
.probe = fxls8962af_probe,
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2022 ROHM Semiconductors
+ *
+ * ROHM/KIONIX KX022A accelerometer driver
+ */
+
+#include <linux/i2c.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/regmap.h>
+
+#include "kionix-kx022a.h"
+
+static int kx022a_i2c_probe(struct i2c_client *i2c)
+{
+ struct device *dev = &i2c->dev;
+ struct regmap *regmap;
+
+ if (!i2c->irq) {
+ dev_err(dev, "No IRQ configured\n");
+ return -EINVAL;
+ }
+
+ regmap = devm_regmap_init_i2c(i2c, &kx022a_regmap);
+ if (IS_ERR(regmap))
+ return dev_err_probe(dev, PTR_ERR(regmap),
+ "Failed to initialize Regmap\n");
+
+ return kx022a_probe_internal(dev);
+}
+
+static const struct of_device_id kx022a_of_match[] = {
+ { .compatible = "kionix,kx022a", },
+ { }
+};
+MODULE_DEVICE_TABLE(of, kx022a_of_match);
+
+static struct i2c_driver kx022a_i2c_driver = {
+ .driver = {
+ .name = "kx022a-i2c",
+ .of_match_table = kx022a_of_match,
+ },
+ .probe_new = kx022a_i2c_probe,
+};
+module_i2c_driver(kx022a_i2c_driver);
+
+MODULE_DESCRIPTION("ROHM/Kionix KX022A accelerometer driver");
+MODULE_AUTHOR("Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>");
+MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(IIO_KX022A);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2022 ROHM Semiconductors
+ *
+ * ROHM/KIONIX KX022A accelerometer driver
+ */
+
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/regmap.h>
+#include <linux/spi/spi.h>
+
+#include "kionix-kx022a.h"
+
+static int kx022a_spi_probe(struct spi_device *spi)
+{
+ struct device *dev = &spi->dev;
+ struct regmap *regmap;
+
+ if (!spi->irq) {
+ dev_err(dev, "No IRQ configured\n");
+ return -EINVAL;
+ }
+
+ regmap = devm_regmap_init_spi(spi, &kx022a_regmap);
+ if (IS_ERR(regmap))
+ return dev_err_probe(dev, PTR_ERR(regmap),
+ "Failed to initialize Regmap\n");
+
+ return kx022a_probe_internal(dev);
+}
+
+static const struct spi_device_id kx022a_id[] = {
+ { "kx022a" },
+ { }
+};
+MODULE_DEVICE_TABLE(spi, kx022a_id);
+
+static const struct of_device_id kx022a_of_match[] = {
+ { .compatible = "kionix,kx022a", },
+ { }
+};
+MODULE_DEVICE_TABLE(of, kx022a_of_match);
+
+static struct spi_driver kx022a_spi_driver = {
+ .driver = {
+ .name = "kx022a-spi",
+ .of_match_table = kx022a_of_match,
+ },
+ .probe = kx022a_spi_probe,
+ .id_table = kx022a_id,
+};
+module_spi_driver(kx022a_spi_driver);
+
+MODULE_DESCRIPTION("ROHM/Kionix kx022A accelerometer driver");
+MODULE_AUTHOR("Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>");
+MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(IIO_KX022A);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2022 ROHM Semiconductors
+ *
+ * ROHM/KIONIX KX022A accelerometer driver
+ */
+
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/mutex.h>
+#include <linux/property.h>
+#include <linux/regmap.h>
+#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
+#include <linux/string_helpers.h>
+#include <linux/units.h>
+
+#include <linux/iio/iio.h>
+#include <linux/iio/sysfs.h>
+#include <linux/iio/trigger.h>
+#include <linux/iio/trigger_consumer.h>
+#include <linux/iio/triggered_buffer.h>
+
+#include "kionix-kx022a.h"
+
+/*
+ * The KX022A has FIFO which can store 43 samples of HiRes data from 2
+ * channels. This equals to 43 (samples) * 3 (channels) * 2 (bytes/sample) to
+ * 258 bytes of sample data. The quirk to know is that the amount of bytes in
+ * the FIFO is advertised via 8 bit register (max value 255). The thing to note
+ * is that full 258 bytes of data is indicated using the max value 255.
+ */
+#define KX022A_FIFO_LENGTH 43
+#define KX022A_FIFO_FULL_VALUE 255
+#define KX022A_SOFT_RESET_WAIT_TIME_US (5 * USEC_PER_MSEC)
+#define KX022A_SOFT_RESET_TOTAL_WAIT_TIME_US (500 * USEC_PER_MSEC)
+
+/* 3 axis, 2 bytes of data for each of the axis */
+#define KX022A_FIFO_SAMPLES_SIZE_BYTES 6
+#define KX022A_FIFO_MAX_BYTES \
+ (KX022A_FIFO_LENGTH * KX022A_FIFO_SAMPLES_SIZE_BYTES)
+
+enum {
+ KX022A_STATE_SAMPLE,
+ KX022A_STATE_FIFO,
+};
+
+/* Regmap configs */
+static const struct regmap_range kx022a_volatile_ranges[] = {
+ {
+ .range_min = KX022A_REG_XHP_L,
+ .range_max = KX022A_REG_COTR,
+ }, {
+ .range_min = KX022A_REG_TSCP,
+ .range_max = KX022A_REG_INT_REL,
+ }, {
+ /* The reset bit will be cleared by sensor */
+ .range_min = KX022A_REG_CNTL2,
+ .range_max = KX022A_REG_CNTL2,
+ }, {
+ .range_min = KX022A_REG_BUF_STATUS_1,
+ .range_max = KX022A_REG_BUF_READ,
+ },
+};
+
+static const struct regmap_access_table kx022a_volatile_regs = {
+ .yes_ranges = &kx022a_volatile_ranges[0],
+ .n_yes_ranges = ARRAY_SIZE(kx022a_volatile_ranges),
+};
+
+static const struct regmap_range kx022a_precious_ranges[] = {
+ {
+ .range_min = KX022A_REG_INT_REL,
+ .range_max = KX022A_REG_INT_REL,
+ },
+};
+
+static const struct regmap_access_table kx022a_precious_regs = {
+ .yes_ranges = &kx022a_precious_ranges[0],
+ .n_yes_ranges = ARRAY_SIZE(kx022a_precious_ranges),
+};
+
+/*
+ * The HW does not set WHO_AM_I reg as read-only but we don't want to write it
+ * so we still include it in the read-only ranges.
+ */
+static const struct regmap_range kx022a_read_only_ranges[] = {
+ {
+ .range_min = KX022A_REG_XHP_L,
+ .range_max = KX022A_REG_INT_REL,
+ }, {
+ .range_min = KX022A_REG_BUF_STATUS_1,
+ .range_max = KX022A_REG_BUF_STATUS_2,
+ }, {
+ .range_min = KX022A_REG_BUF_READ,
+ .range_max = KX022A_REG_BUF_READ,
+ },
+};
+
+static const struct regmap_access_table kx022a_ro_regs = {
+ .no_ranges = &kx022a_read_only_ranges[0],
+ .n_no_ranges = ARRAY_SIZE(kx022a_read_only_ranges),
+};
+
+static const struct regmap_range kx022a_write_only_ranges[] = {
+ {
+ .range_min = KX022A_REG_BTS_WUF_TH,
+ .range_max = KX022A_REG_BTS_WUF_TH,
+ }, {
+ .range_min = KX022A_REG_MAN_WAKE,
+ .range_max = KX022A_REG_MAN_WAKE,
+ }, {
+ .range_min = KX022A_REG_SELF_TEST,
+ .range_max = KX022A_REG_SELF_TEST,
+ }, {
+ .range_min = KX022A_REG_BUF_CLEAR,
+ .range_max = KX022A_REG_BUF_CLEAR,
+ },
+};
+
+static const struct regmap_access_table kx022a_wo_regs = {
+ .no_ranges = &kx022a_write_only_ranges[0],
+ .n_no_ranges = ARRAY_SIZE(kx022a_write_only_ranges),
+};
+
+static const struct regmap_range kx022a_noinc_read_ranges[] = {
+ {
+ .range_min = KX022A_REG_BUF_READ,
+ .range_max = KX022A_REG_BUF_READ,
+ },
+};
+
+static const struct regmap_access_table kx022a_nir_regs = {
+ .yes_ranges = &kx022a_noinc_read_ranges[0],
+ .n_yes_ranges = ARRAY_SIZE(kx022a_noinc_read_ranges),
+};
+
+const struct regmap_config kx022a_regmap = {
+ .reg_bits = 8,
+ .val_bits = 8,
+ .volatile_table = &kx022a_volatile_regs,
+ .rd_table = &kx022a_wo_regs,
+ .wr_table = &kx022a_ro_regs,
+ .rd_noinc_table = &kx022a_nir_regs,
+ .precious_table = &kx022a_precious_regs,
+ .max_register = KX022A_MAX_REGISTER,
+ .cache_type = REGCACHE_RBTREE,
+};
+EXPORT_SYMBOL_NS_GPL(kx022a_regmap, IIO_KX022A);
+
+struct kx022a_data {
+ struct regmap *regmap;
+ struct iio_trigger *trig;
+ struct device *dev;
+ struct iio_mount_matrix orientation;
+ int64_t timestamp, old_timestamp;
+
+ int irq;
+ int inc_reg;
+ int ien_reg;
+
+ unsigned int g_range;
+ unsigned int state;
+ unsigned int odr_ns;
+
+ bool trigger_enabled;
+ /*
+ * Prevent toggling the sensor stby/active state (PC1 bit) in the
+ * middle of a configuration, or when the fifo is enabled. Also,
+ * protect the data stored/retrieved from this structure from
+ * concurrent accesses.
+ */
+ struct mutex mutex;
+ u8 watermark;
+
+ /* 3 x 16bit accel data + timestamp */
+ __le16 buffer[8] __aligned(IIO_DMA_MINALIGN);
+ struct {
+ __le16 channels[3];
+ s64 ts __aligned(8);
+ } scan;
+};
+
+static const struct iio_mount_matrix *
+kx022a_get_mount_matrix(const struct iio_dev *idev,
+ const struct iio_chan_spec *chan)
+{
+ struct kx022a_data *data = iio_priv(idev);
+
+ return &data->orientation;
+}
+
+enum {
+ AXIS_X,
+ AXIS_Y,
+ AXIS_Z,
+ AXIS_MAX
+};
+
+static const unsigned long kx022a_scan_masks[] = {
+ BIT(AXIS_X) | BIT(AXIS_Y) | BIT(AXIS_Z), 0
+};
+
+static const struct iio_chan_spec_ext_info kx022a_ext_info[] = {
+ IIO_MOUNT_MATRIX(IIO_SHARED_BY_TYPE, kx022a_get_mount_matrix),
+ { }
+};
+
+#define KX022A_ACCEL_CHAN(axis, index) \
+{ \
+ .type = IIO_ACCEL, \
+ .modified = 1, \
+ .channel2 = IIO_MOD_##axis, \
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW), \
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE) | \
+ BIT(IIO_CHAN_INFO_SAMP_FREQ), \
+ .info_mask_shared_by_type_available = \
+ BIT(IIO_CHAN_INFO_SCALE) | \
+ BIT(IIO_CHAN_INFO_SAMP_FREQ), \
+ .ext_info = kx022a_ext_info, \
+ .address = KX022A_REG_##axis##OUT_L, \
+ .scan_index = index, \
+ .scan_type = { \
+ .sign = 's', \
+ .realbits = 16, \
+ .storagebits = 16, \
+ .endianness = IIO_LE, \
+ }, \
+}
+
+static const struct iio_chan_spec kx022a_channels[] = {
+ KX022A_ACCEL_CHAN(X, 0),
+ KX022A_ACCEL_CHAN(Y, 1),
+ KX022A_ACCEL_CHAN(Z, 2),
+ IIO_CHAN_SOFT_TIMESTAMP(3),
+};
+
+/*
+ * The sensor HW can support ODR up to 1600 Hz, which is beyond what most of the
+ * Linux CPUs can handle without dropping samples. Also, the low power mode is
+ * not available for higher sample rates. Thus, the driver only supports 200 Hz
+ * and slower ODRs. The slowest is 0.78 Hz.
+ */
+static const int kx022a_accel_samp_freq_table[][2] = {
+ { 0, 780000 },
+ { 1, 563000 },
+ { 3, 125000 },
+ { 6, 250000 },
+ { 12, 500000 },
+ { 25, 0 },
+ { 50, 0 },
+ { 100, 0 },
+ { 200, 0 },
+};
+
+static const unsigned int kx022a_odrs[] = {
+ 1282051282,
+ 639795266,
+ 320 * MEGA,
+ 160 * MEGA,
+ 80 * MEGA,
+ 40 * MEGA,
+ 20 * MEGA,
+ 10 * MEGA,
+ 5 * MEGA,
+};
+
+/*
+ * range is typically +-2G/4G/8G/16G, distributed over the amount of bits.
+ * The scale table can be calculated using
+ * (range / 2^bits) * g = (range / 2^bits) * 9.80665 m/s^2
+ * => KX022A uses 16 bit (HiRes mode - assume the low 8 bits are zeroed
+ * in low-power mode(?) )
+ * => +/-2G => 4 / 2^16 * 9,80665 * 10^6 (to scale to micro)
+ * => +/-2G - 598.550415
+ * +/-4G - 1197.10083
+ * +/-8G - 2394.20166
+ * +/-16G - 4788.40332
+ */
+static const int kx022a_scale_table[][2] = {
+ { 598, 550415 },
+ { 1197, 100830 },
+ { 2394, 201660 },
+ { 4788, 403320 },
+};
+
+static int kx022a_read_avail(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ const int **vals, int *type, int *length,
+ long mask)
+{
+ switch (mask) {
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ *vals = (const int *)kx022a_accel_samp_freq_table;
+ *length = ARRAY_SIZE(kx022a_accel_samp_freq_table) *
+ ARRAY_SIZE(kx022a_accel_samp_freq_table[0]);
+ *type = IIO_VAL_INT_PLUS_MICRO;
+ return IIO_AVAIL_LIST;
+ case IIO_CHAN_INFO_SCALE:
+ *vals = (const int *)kx022a_scale_table;
+ *length = ARRAY_SIZE(kx022a_scale_table) *
+ ARRAY_SIZE(kx022a_scale_table[0]);
+ *type = IIO_VAL_INT_PLUS_MICRO;
+ return IIO_AVAIL_LIST;
+ default:
+ return -EINVAL;
+ }
+}
+
+#define KX022A_DEFAULT_PERIOD_NS (20 * NSEC_PER_MSEC)
+
+static void kx022a_reg2freq(unsigned int val, int *val1, int *val2)
+{
+ *val1 = kx022a_accel_samp_freq_table[val & KX022A_MASK_ODR][0];
+ *val2 = kx022a_accel_samp_freq_table[val & KX022A_MASK_ODR][1];
+}
+
+static void kx022a_reg2scale(unsigned int val, unsigned int *val1,
+ unsigned int *val2)
+{
+ val &= KX022A_MASK_GSEL;
+ val >>= KX022A_GSEL_SHIFT;
+
+ *val1 = kx022a_scale_table[val][0];
+ *val2 = kx022a_scale_table[val][1];
+}
+
+static int kx022a_turn_on_off_unlocked(struct kx022a_data *data, bool on)
+{
+ int ret;
+
+ if (on)
+ ret = regmap_set_bits(data->regmap, KX022A_REG_CNTL,
+ KX022A_MASK_PC1);
+ else
+ ret = regmap_clear_bits(data->regmap, KX022A_REG_CNTL,
+ KX022A_MASK_PC1);
+ if (ret)
+ dev_err(data->dev, "Turn %s fail %d\n", str_on_off(on), ret);
+
+ return ret;
+
+}
+
+static int kx022a_turn_off_lock(struct kx022a_data *data)
+{
+ int ret;
+
+ mutex_lock(&data->mutex);
+ ret = kx022a_turn_on_off_unlocked(data, false);
+ if (ret)
+ mutex_unlock(&data->mutex);
+
+ return ret;
+}
+
+static int kx022a_turn_on_unlock(struct kx022a_data *data)
+{
+ int ret;
+
+ ret = kx022a_turn_on_off_unlocked(data, true);
+ mutex_unlock(&data->mutex);
+
+ return ret;
+}
+
+static int kx022a_write_raw(struct iio_dev *idev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ struct kx022a_data *data = iio_priv(idev);
+ int ret, n;
+
+ /*
+ * We should not allow changing scale or frequency when FIFO is running
+ * as it will mess the timestamp/scale for samples existing in the
+ * buffer. If this turns out to be an issue we can later change logic
+ * to internally flush the fifo before reconfiguring so the samples in
+ * fifo keep matching the freq/scale settings. (Such setup could cause
+ * issues if users trust the watermark to be reached within known
+ * time-limit).
+ */
+ ret = iio_device_claim_direct_mode(idev);
+ if (ret)
+ return ret;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ n = ARRAY_SIZE(kx022a_accel_samp_freq_table);
+
+ while (n--)
+ if (val == kx022a_accel_samp_freq_table[n][0] &&
+ val2 == kx022a_accel_samp_freq_table[n][1])
+ break;
+ if (n < 0) {
+ ret = -EINVAL;
+ goto unlock_out;
+ }
+ ret = kx022a_turn_off_lock(data);
+ if (ret)
+ break;
+
+ ret = regmap_update_bits(data->regmap,
+ KX022A_REG_ODCNTL,
+ KX022A_MASK_ODR, n);
+ data->odr_ns = kx022a_odrs[n];
+ kx022a_turn_on_unlock(data);
+ break;
+ case IIO_CHAN_INFO_SCALE:
+ n = ARRAY_SIZE(kx022a_scale_table);
+
+ while (n-- > 0)
+ if (val == kx022a_scale_table[n][0] &&
+ val2 == kx022a_scale_table[n][1])
+ break;
+ if (n < 0) {
+ ret = -EINVAL;
+ goto unlock_out;
+ }
+
+ ret = kx022a_turn_off_lock(data);
+ if (ret)
+ break;
+
+ ret = regmap_update_bits(data->regmap, KX022A_REG_CNTL,
+ KX022A_MASK_GSEL,
+ n << KX022A_GSEL_SHIFT);
+ kx022a_turn_on_unlock(data);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+unlock_out:
+ iio_device_release_direct_mode(idev);
+
+ return ret;
+}
+
+static int kx022a_fifo_set_wmi(struct kx022a_data *data)
+{
+ u8 threshold;
+
+ threshold = data->watermark;
+
+ return regmap_update_bits(data->regmap, KX022A_REG_BUF_CNTL1,
+ KX022A_MASK_WM_TH, threshold);
+}
+
+static int kx022a_get_axis(struct kx022a_data *data,
+ struct iio_chan_spec const *chan,
+ int *val)
+{
+ int ret;
+
+ ret = regmap_bulk_read(data->regmap, chan->address, &data->buffer[0],
+ sizeof(__le16));
+ if (ret)
+ return ret;
+
+ *val = le16_to_cpu(data->buffer[0]);
+
+ return IIO_VAL_INT;
+}
+
+static int kx022a_read_raw(struct iio_dev *idev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ struct kx022a_data *data = iio_priv(idev);
+ unsigned int regval;
+ int ret;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ ret = iio_device_claim_direct_mode(idev);
+ if (ret)
+ return ret;
+
+ mutex_lock(&data->mutex);
+ ret = kx022a_get_axis(data, chan, val);
+ mutex_unlock(&data->mutex);
+
+ iio_device_release_direct_mode(idev);
+
+ return ret;
+
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ ret = regmap_read(data->regmap, KX022A_REG_ODCNTL, ®val);
+ if (ret)
+ return ret;
+
+ if ((regval & KX022A_MASK_ODR) >
+ ARRAY_SIZE(kx022a_accel_samp_freq_table)) {
+ dev_err(data->dev, "Invalid ODR\n");
+ return -EINVAL;
+ }
+
+ kx022a_reg2freq(regval, val, val2);
+
+ return IIO_VAL_INT_PLUS_MICRO;
+
+ case IIO_CHAN_INFO_SCALE:
+ ret = regmap_read(data->regmap, KX022A_REG_CNTL, ®val);
+ if (ret < 0)
+ return ret;
+
+ kx022a_reg2scale(regval, val, val2);
+
+ return IIO_VAL_INT_PLUS_MICRO;
+ }
+
+ return -EINVAL;
+};
+
+static int kx022a_validate_trigger(struct iio_dev *idev,
+ struct iio_trigger *trig)
+{
+ struct kx022a_data *data = iio_priv(idev);
+
+ if (data->trig != trig)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int kx022a_set_watermark(struct iio_dev *idev, unsigned int val)
+{
+ struct kx022a_data *data = iio_priv(idev);
+
+ if (val > KX022A_FIFO_LENGTH)
+ val = KX022A_FIFO_LENGTH;
+
+ mutex_lock(&data->mutex);
+ data->watermark = val;
+ mutex_unlock(&data->mutex);
+
+ return 0;
+}
+
+static ssize_t hwfifo_enabled_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct iio_dev *idev = dev_to_iio_dev(dev);
+ struct kx022a_data *data = iio_priv(idev);
+ bool state;
+
+ mutex_lock(&data->mutex);
+ state = data->state;
+ mutex_unlock(&data->mutex);
+
+ return sysfs_emit(buf, "%d\n", state);
+}
+
+static ssize_t hwfifo_watermark_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct iio_dev *idev = dev_to_iio_dev(dev);
+ struct kx022a_data *data = iio_priv(idev);
+ int wm;
+
+ mutex_lock(&data->mutex);
+ wm = data->watermark;
+ mutex_unlock(&data->mutex);
+
+ return sysfs_emit(buf, "%d\n", wm);
+}
+
+static IIO_DEVICE_ATTR_RO(hwfifo_enabled, 0);
+static IIO_DEVICE_ATTR_RO(hwfifo_watermark, 0);
+
+static const struct iio_dev_attr *kx022a_fifo_attributes[] = {
+ &iio_dev_attr_hwfifo_watermark,
+ &iio_dev_attr_hwfifo_enabled,
+ NULL
+};
+
+static int kx022a_drop_fifo_contents(struct kx022a_data *data)
+{
+ /*
+ * We must clear the old time-stamp to avoid computing the timestamps
+ * based on samples acquired when buffer was last enabled.
+ *
+ * We don't need to protect the timestamp as long as we are only
+ * called from fifo-disable where we can guarantee the sensor is not
+ * triggering interrupts and where the mutex is locked to prevent the
+ * user-space access.
+ */
+ data->timestamp = 0;
+
+ return regmap_write(data->regmap, KX022A_REG_BUF_CLEAR, 0x0);
+}
+
+static int __kx022a_fifo_flush(struct iio_dev *idev, unsigned int samples,
+ bool irq)
+{
+ struct kx022a_data *data = iio_priv(idev);
+ struct device *dev = regmap_get_device(data->regmap);
+ __le16 buffer[KX022A_FIFO_LENGTH * 3];
+ uint64_t sample_period;
+ int count, fifo_bytes;
+ bool renable = false;
+ int64_t tstamp;
+ int ret, i;
+
+ ret = regmap_read(data->regmap, KX022A_REG_BUF_STATUS_1, &fifo_bytes);
+ if (ret) {
+ dev_err(dev, "Error reading buffer status\n");
+ return ret;
+ }
+
+ /* Let's not overflow if we for some reason get bogus value from i2c */
+ if (fifo_bytes == KX022A_FIFO_FULL_VALUE)
+ fifo_bytes = KX022A_FIFO_MAX_BYTES;
+
+ if (fifo_bytes % KX022A_FIFO_SAMPLES_SIZE_BYTES)
+ dev_warn(data->dev, "Bad FIFO alignment. Data may be corrupt\n");
+
+ count = fifo_bytes / KX022A_FIFO_SAMPLES_SIZE_BYTES;
+ if (!count)
+ return 0;
+
+ /*
+ * If we are being called from IRQ handler we know the stored timestamp
+ * is fairly accurate for the last stored sample. Otherwise, if we are
+ * called as a result of a read operation from userspace and hence
+ * before the watermark interrupt was triggered, take a timestamp
+ * now. We can fall anywhere in between two samples so the error in this
+ * case is at most one sample period.
+ */
+ if (!irq) {
+ /*
+ * We need to have the IRQ disabled or we risk of messing-up
+ * the timestamps. If we are ran from IRQ, then the
+ * IRQF_ONESHOT has us covered - but if we are ran by the
+ * user-space read we need to disable the IRQ to be on a safe
+ * side. We do this usng synchronous disable so that if the
+ * IRQ thread is being ran on other CPU we wait for it to be
+ * finished.
+ */
+ disable_irq(data->irq);
+ renable = true;
+
+ data->old_timestamp = data->timestamp;
+ data->timestamp = iio_get_time_ns(idev);
+ }
+
+ /*
+ * Approximate timestamps for each of the sample based on the sampling
+ * frequency, timestamp for last sample and number of samples.
+ *
+ * We'd better not use the current bandwidth settings to compute the
+ * sample period. The real sample rate varies with the device and
+ * small variation adds when we store a large number of samples.
+ *
+ * To avoid this issue we compute the actual sample period ourselves
+ * based on the timestamp delta between the last two flush operations.
+ */
+ if (data->old_timestamp) {
+ sample_period = data->timestamp - data->old_timestamp;
+ do_div(sample_period, count);
+ } else {
+ sample_period = data->odr_ns;
+ }
+ tstamp = data->timestamp - (count - 1) * sample_period;
+
+ if (samples && count > samples) {
+ /*
+ * Here we leave some old samples to the buffer. We need to
+ * adjust the timestamp to match the first sample in the buffer
+ * or we will miscalculate the sample_period at next round.
+ */
+ data->timestamp -= (count - samples) * sample_period;
+ count = samples;
+ }
+
+ fifo_bytes = count * KX022A_FIFO_SAMPLES_SIZE_BYTES;
+ ret = regmap_noinc_read(data->regmap, KX022A_REG_BUF_READ,
+ &buffer[0], fifo_bytes);
+ if (ret)
+ goto renable_out;
+
+ for (i = 0; i < count; i++) {
+ __le16 *sam = &buffer[i * 3];
+ __le16 *chs;
+ int bit;
+
+ chs = &data->scan.channels[0];
+ for_each_set_bit(bit, idev->active_scan_mask, AXIS_MAX)
+ chs[bit] = sam[bit];
+
+ iio_push_to_buffers_with_timestamp(idev, &data->scan, tstamp);
+
+ tstamp += sample_period;
+ }
+
+ ret = count;
+
+renable_out:
+ if (renable)
+ enable_irq(data->irq);
+
+ return ret;
+}
+
+static int kx022a_fifo_flush(struct iio_dev *idev, unsigned int samples)
+{
+ struct kx022a_data *data = iio_priv(idev);
+ int ret;
+
+ mutex_lock(&data->mutex);
+ ret = __kx022a_fifo_flush(idev, samples, false);
+ mutex_unlock(&data->mutex);
+
+ return ret;
+}
+
+static const struct iio_info kx022a_info = {
+ .read_raw = &kx022a_read_raw,
+ .write_raw = &kx022a_write_raw,
+ .read_avail = &kx022a_read_avail,
+
+ .validate_trigger = kx022a_validate_trigger,
+ .hwfifo_set_watermark = kx022a_set_watermark,
+ .hwfifo_flush_to_buffer = kx022a_fifo_flush,
+};
+
+static int kx022a_set_drdy_irq(struct kx022a_data *data, bool en)
+{
+ if (en)
+ return regmap_set_bits(data->regmap, KX022A_REG_CNTL,
+ KX022A_MASK_DRDY);
+
+ return regmap_clear_bits(data->regmap, KX022A_REG_CNTL,
+ KX022A_MASK_DRDY);
+}
+
+static int kx022a_prepare_irq_pin(struct kx022a_data *data)
+{
+ /* Enable IRQ1 pin. Set polarity to active low */
+ int mask = KX022A_MASK_IEN | KX022A_MASK_IPOL |
+ KX022A_MASK_ITYP;
+ int val = KX022A_MASK_IEN | KX022A_IPOL_LOW |
+ KX022A_ITYP_LEVEL;
+ int ret;
+
+ ret = regmap_update_bits(data->regmap, data->inc_reg, mask, val);
+ if (ret)
+ return ret;
+
+ /* We enable WMI to IRQ pin only at buffer_enable */
+ mask = KX022A_MASK_INS2_DRDY;
+
+ return regmap_set_bits(data->regmap, data->ien_reg, mask);
+}
+
+static int kx022a_fifo_disable(struct kx022a_data *data)
+{
+ int ret = 0;
+
+ ret = kx022a_turn_off_lock(data);
+ if (ret)
+ return ret;
+
+ ret = regmap_clear_bits(data->regmap, data->ien_reg, KX022A_MASK_WMI);
+ if (ret)
+ goto unlock_out;
+
+ ret = regmap_clear_bits(data->regmap, KX022A_REG_BUF_CNTL2,
+ KX022A_MASK_BUF_EN);
+ if (ret)
+ goto unlock_out;
+
+ data->state &= ~KX022A_STATE_FIFO;
+
+ kx022a_drop_fifo_contents(data);
+
+ return kx022a_turn_on_unlock(data);
+
+unlock_out:
+ mutex_unlock(&data->mutex);
+
+ return ret;
+}
+
+static int kx022a_buffer_predisable(struct iio_dev *idev)
+{
+ struct kx022a_data *data = iio_priv(idev);
+
+ if (iio_device_get_current_mode(idev) == INDIO_BUFFER_TRIGGERED)
+ return 0;
+
+ return kx022a_fifo_disable(data);
+}
+
+static int kx022a_fifo_enable(struct kx022a_data *data)
+{
+ int ret;
+
+ ret = kx022a_turn_off_lock(data);
+ if (ret)
+ return ret;
+
+ /* Update watermark to HW */
+ ret = kx022a_fifo_set_wmi(data);
+ if (ret)
+ goto unlock_out;
+
+ /* Enable buffer */
+ ret = regmap_set_bits(data->regmap, KX022A_REG_BUF_CNTL2,
+ KX022A_MASK_BUF_EN);
+ if (ret)
+ goto unlock_out;
+
+ data->state |= KX022A_STATE_FIFO;
+ ret = regmap_set_bits(data->regmap, data->ien_reg,
+ KX022A_MASK_WMI);
+ if (ret)
+ goto unlock_out;
+
+ return kx022a_turn_on_unlock(data);
+
+unlock_out:
+ mutex_unlock(&data->mutex);
+
+ return ret;
+}
+
+static int kx022a_buffer_postenable(struct iio_dev *idev)
+{
+ struct kx022a_data *data = iio_priv(idev);
+
+ /*
+ * If we use data-ready trigger, then the IRQ masks should be handled by
+ * trigger enable and the hardware buffer is not used but we just update
+ * results to the IIO fifo when data-ready triggers.
+ */
+ if (iio_device_get_current_mode(idev) == INDIO_BUFFER_TRIGGERED)
+ return 0;
+
+ return kx022a_fifo_enable(data);
+}
+
+static const struct iio_buffer_setup_ops kx022a_buffer_ops = {
+ .postenable = kx022a_buffer_postenable,
+ .predisable = kx022a_buffer_predisable,
+};
+
+static irqreturn_t kx022a_trigger_handler(int irq, void *p)
+{
+ struct iio_poll_func *pf = p;
+ struct iio_dev *idev = pf->indio_dev;
+ struct kx022a_data *data = iio_priv(idev);
+ int ret;
+
+ ret = regmap_bulk_read(data->regmap, KX022A_REG_XOUT_L, data->buffer,
+ KX022A_FIFO_SAMPLES_SIZE_BYTES);
+ if (ret < 0)
+ goto err_read;
+
+ iio_push_to_buffers_with_timestamp(idev, data->buffer, pf->timestamp);
+err_read:
+ iio_trigger_notify_done(idev->trig);
+
+ return IRQ_HANDLED;
+}
+
+/* Get timestamps and wake the thread if we need to read data */
+static irqreturn_t kx022a_irq_handler(int irq, void *private)
+{
+ struct iio_dev *idev = private;
+ struct kx022a_data *data = iio_priv(idev);
+
+ data->old_timestamp = data->timestamp;
+ data->timestamp = iio_get_time_ns(idev);
+
+ if (data->state & KX022A_STATE_FIFO || data->trigger_enabled)
+ return IRQ_WAKE_THREAD;
+
+ return IRQ_NONE;
+}
+
+/*
+ * WMI and data-ready IRQs are acked when results are read. If we add
+ * TILT/WAKE or other IRQs - then we may need to implement the acking
+ * (which is racy).
+ */
+static irqreturn_t kx022a_irq_thread_handler(int irq, void *private)
+{
+ struct iio_dev *idev = private;
+ struct kx022a_data *data = iio_priv(idev);
+ irqreturn_t ret = IRQ_NONE;
+
+ mutex_lock(&data->mutex);
+
+ if (data->trigger_enabled) {
+ iio_trigger_poll_chained(data->trig);
+ ret = IRQ_HANDLED;
+ }
+
+ if (data->state & KX022A_STATE_FIFO) {
+ int ok;
+
+ ok = __kx022a_fifo_flush(idev, KX022A_FIFO_LENGTH, true);
+ if (ok > 0)
+ ret = IRQ_HANDLED;
+ }
+
+ mutex_unlock(&data->mutex);
+
+ return ret;
+}
+
+static int kx022a_trigger_set_state(struct iio_trigger *trig,
+ bool state)
+{
+ struct kx022a_data *data = iio_trigger_get_drvdata(trig);
+ int ret = 0;
+
+ mutex_lock(&data->mutex);
+
+ if (data->trigger_enabled == state)
+ goto unlock_out;
+
+ if (data->state & KX022A_STATE_FIFO) {
+ dev_warn(data->dev, "Can't set trigger when FIFO enabled\n");
+ ret = -EBUSY;
+ goto unlock_out;
+ }
+
+ ret = kx022a_turn_on_off_unlocked(data, false);
+ if (ret)
+ goto unlock_out;
+
+ data->trigger_enabled = state;
+ ret = kx022a_set_drdy_irq(data, state);
+ if (ret)
+ goto unlock_out;
+
+ ret = kx022a_turn_on_off_unlocked(data, true);
+
+unlock_out:
+ mutex_unlock(&data->mutex);
+
+ return ret;
+}
+
+static const struct iio_trigger_ops kx022a_trigger_ops = {
+ .set_trigger_state = kx022a_trigger_set_state,
+};
+
+static int kx022a_chip_init(struct kx022a_data *data)
+{
+ int ret, val;
+
+ /* Reset the senor */
+ ret = regmap_write(data->regmap, KX022A_REG_CNTL2, KX022A_MASK_SRST);
+ if (ret)
+ return ret;
+
+ /*
+ * I've seen I2C read failures if we poll too fast after the sensor
+ * reset. Slight delay gives I2C block the time to recover.
+ */
+ msleep(1);
+
+ ret = regmap_read_poll_timeout(data->regmap, KX022A_REG_CNTL2, val,
+ !(val & KX022A_MASK_SRST),
+ KX022A_SOFT_RESET_WAIT_TIME_US,
+ KX022A_SOFT_RESET_TOTAL_WAIT_TIME_US);
+ if (ret) {
+ dev_err(data->dev, "Sensor reset %s\n",
+ val & KX022A_MASK_SRST ? "timeout" : "fail#");
+ return ret;
+ }
+
+ ret = regmap_reinit_cache(data->regmap, &kx022a_regmap);
+ if (ret) {
+ dev_err(data->dev, "Failed to reinit reg cache\n");
+ return ret;
+ }
+
+ /* set data res 16bit */
+ ret = regmap_set_bits(data->regmap, KX022A_REG_BUF_CNTL2,
+ KX022A_MASK_BRES16);
+ if (ret) {
+ dev_err(data->dev, "Failed to set data resolution\n");
+ return ret;
+ }
+
+ return kx022a_prepare_irq_pin(data);
+}
+
+int kx022a_probe_internal(struct device *dev)
+{
+ static const char * const regulator_names[] = {"io-vdd", "vdd"};
+ struct iio_trigger *indio_trig;
+ struct fwnode_handle *fwnode;
+ struct kx022a_data *data;
+ struct regmap *regmap;
+ unsigned int chip_id;
+ struct iio_dev *idev;
+ int ret, irq;
+ char *name;
+
+ regmap = dev_get_regmap(dev, NULL);
+ if (!regmap) {
+ dev_err(dev, "no regmap\n");
+ return -EINVAL;
+ }
+
+ fwnode = dev_fwnode(dev);
+ if (!fwnode)
+ return -ENODEV;
+
+ idev = devm_iio_device_alloc(dev, sizeof(*data));
+ if (!idev)
+ return -ENOMEM;
+
+ data = iio_priv(idev);
+
+ /*
+ * VDD is the analog and digital domain voltage supply and
+ * IO_VDD is the digital I/O voltage supply.
+ */
+ ret = devm_regulator_bulk_get_enable(dev, ARRAY_SIZE(regulator_names),
+ regulator_names);
+ if (ret && ret != -ENODEV)
+ return dev_err_probe(dev, ret, "failed to enable regulator\n");
+
+ ret = regmap_read(regmap, KX022A_REG_WHO, &chip_id);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to access sensor\n");
+
+ if (chip_id != KX022A_ID) {
+ dev_err(dev, "unsupported device 0x%x\n", chip_id);
+ return -EINVAL;
+ }
+
+ irq = fwnode_irq_get_byname(fwnode, "INT1");
+ if (irq > 0) {
+ data->inc_reg = KX022A_REG_INC1;
+ data->ien_reg = KX022A_REG_INC4;
+ } else {
+ irq = fwnode_irq_get_byname(fwnode, "INT2");
+ if (irq <= 0)
+ return dev_err_probe(dev, irq, "No suitable IRQ\n");
+
+ data->inc_reg = KX022A_REG_INC5;
+ data->ien_reg = KX022A_REG_INC6;
+ }
+
+ data->regmap = regmap;
+ data->dev = dev;
+ data->irq = irq;
+ data->odr_ns = KX022A_DEFAULT_PERIOD_NS;
+ mutex_init(&data->mutex);
+
+ idev->channels = kx022a_channels;
+ idev->num_channels = ARRAY_SIZE(kx022a_channels);
+ idev->name = "kx022-accel";
+ idev->info = &kx022a_info;
+ idev->modes = INDIO_DIRECT_MODE | INDIO_BUFFER_SOFTWARE;
+ idev->available_scan_masks = kx022a_scan_masks;
+
+ /* Read the mounting matrix, if present */
+ ret = iio_read_mount_matrix(dev, &data->orientation);
+ if (ret)
+ return ret;
+
+ /* The sensor must be turned off for configuration */
+ ret = kx022a_turn_off_lock(data);
+ if (ret)
+ return ret;
+
+ ret = kx022a_chip_init(data);
+ if (ret) {
+ mutex_unlock(&data->mutex);
+ return ret;
+ }
+
+ ret = kx022a_turn_on_unlock(data);
+ if (ret)
+ return ret;
+
+ ret = devm_iio_triggered_buffer_setup_ext(dev, idev,
+ &iio_pollfunc_store_time,
+ kx022a_trigger_handler,
+ IIO_BUFFER_DIRECTION_IN,
+ &kx022a_buffer_ops,
+ kx022a_fifo_attributes);
+
+ if (ret)
+ return dev_err_probe(data->dev, ret,
+ "iio_triggered_buffer_setup_ext FAIL\n");
+ indio_trig = devm_iio_trigger_alloc(dev, "%sdata-rdy-dev%d", idev->name,
+ iio_device_id(idev));
+ if (!indio_trig)
+ return -ENOMEM;
+
+ data->trig = indio_trig;
+
+ indio_trig->ops = &kx022a_trigger_ops;
+ iio_trigger_set_drvdata(indio_trig, data);
+
+ /*
+ * No need to check for NULL. request_threaded_irq() defaults to
+ * dev_name() should the alloc fail.
+ */
+ name = devm_kasprintf(data->dev, GFP_KERNEL, "%s-kx022a",
+ dev_name(data->dev));
+
+ ret = devm_request_threaded_irq(data->dev, irq, kx022a_irq_handler,
+ &kx022a_irq_thread_handler,
+ IRQF_ONESHOT, name, idev);
+ if (ret)
+ return dev_err_probe(data->dev, ret, "Could not request IRQ\n");
+
+
+ ret = devm_iio_trigger_register(dev, indio_trig);
+ if (ret)
+ return dev_err_probe(data->dev, ret,
+ "Trigger registration failed\n");
+
+ ret = devm_iio_device_register(data->dev, idev);
+ if (ret < 0)
+ return dev_err_probe(dev, ret,
+ "Unable to register iio device\n");
+
+ return ret;
+}
+EXPORT_SYMBOL_NS_GPL(kx022a_probe_internal, IIO_KX022A);
+
+MODULE_DESCRIPTION("ROHM/Kionix KX022A accelerometer driver");
+MODULE_AUTHOR("Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>");
+MODULE_LICENSE("GPL");
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2022 ROHM Semiconductors
+ *
+ * ROHM/KIONIX KX022A accelerometer driver
+ */
+
+#ifndef _KX022A_H_
+#define _KX022A_H_
+
+#include <linux/bits.h>
+#include <linux/regmap.h>
+
+#define KX022A_REG_WHO 0x0f
+#define KX022A_ID 0xc8
+
+#define KX022A_REG_CNTL2 0x19
+#define KX022A_MASK_SRST BIT(7)
+#define KX022A_REG_CNTL 0x18
+#define KX022A_MASK_PC1 BIT(7)
+#define KX022A_MASK_RES BIT(6)
+#define KX022A_MASK_DRDY BIT(5)
+#define KX022A_MASK_GSEL GENMASK(4, 3)
+#define KX022A_GSEL_SHIFT 3
+#define KX022A_GSEL_2 0x0
+#define KX022A_GSEL_4 BIT(3)
+#define KX022A_GSEL_8 BIT(4)
+#define KX022A_GSEL_16 GENMASK(4, 3)
+
+#define KX022A_REG_INS2 0x13
+#define KX022A_MASK_INS2_DRDY BIT(4)
+#define KX122_MASK_INS2_WMI BIT(5)
+
+#define KX022A_REG_XHP_L 0x0
+#define KX022A_REG_XOUT_L 0x06
+#define KX022A_REG_YOUT_L 0x08
+#define KX022A_REG_ZOUT_L 0x0a
+#define KX022A_REG_COTR 0x0c
+#define KX022A_REG_TSCP 0x10
+#define KX022A_REG_INT_REL 0x17
+
+#define KX022A_REG_ODCNTL 0x1b
+
+#define KX022A_REG_BTS_WUF_TH 0x31
+#define KX022A_REG_MAN_WAKE 0x2c
+
+#define KX022A_REG_BUF_CNTL1 0x3a
+#define KX022A_MASK_WM_TH GENMASK(6, 0)
+#define KX022A_REG_BUF_CNTL2 0x3b
+#define KX022A_MASK_BUF_EN BIT(7)
+#define KX022A_MASK_BRES16 BIT(6)
+#define KX022A_REG_BUF_STATUS_1 0x3c
+#define KX022A_REG_BUF_STATUS_2 0x3d
+#define KX022A_REG_BUF_CLEAR 0x3e
+#define KX022A_REG_BUF_READ 0x3f
+#define KX022A_MASK_ODR GENMASK(3, 0)
+#define KX022A_ODR_SHIFT 3
+#define KX022A_FIFO_MAX_WMI_TH 41
+
+#define KX022A_REG_INC1 0x1c
+#define KX022A_REG_INC5 0x20
+#define KX022A_REG_INC6 0x21
+#define KX022A_MASK_IEN BIT(5)
+#define KX022A_MASK_IPOL BIT(4)
+#define KX022A_IPOL_LOW 0
+#define KX022A_IPOL_HIGH KX022A_MASK_IPOL1
+#define KX022A_MASK_ITYP BIT(3)
+#define KX022A_ITYP_PULSE KX022A_MASK_ITYP
+#define KX022A_ITYP_LEVEL 0
+
+#define KX022A_REG_INC4 0x1f
+#define KX022A_MASK_WMI BIT(5)
+
+#define KX022A_REG_SELF_TEST 0x60
+#define KX022A_MAX_REGISTER 0x60
+
+struct device;
+
+int kx022a_probe_internal(struct device *dev);
+extern const struct regmap_config kx022a_regmap;
+
+#endif
};
struct kxcjk1013_data {
- struct regulator_bulk_data regulators[2];
struct i2c_client *client;
struct iio_trigger *dready_trig;
struct iio_trigger *motion_trig;
return dev_name(dev);
}
-static void kxcjk1013_disable_regulators(void *d)
-{
- struct kxcjk1013_data *data = d;
-
- regulator_bulk_disable(ARRAY_SIZE(data->regulators), data->regulators);
-}
-
-static int kxcjk1013_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int kxcjk1013_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
+ static const char * const regulator_names[] = { "vdd", "vddio" };
struct kxcjk1013_data *data;
struct iio_dev *indio_dev;
struct kxcjk_1013_platform_data *pdata;
return ret;
}
- data->regulators[0].supply = "vdd";
- data->regulators[1].supply = "vddio";
- ret = devm_regulator_bulk_get(&client->dev, ARRAY_SIZE(data->regulators),
- data->regulators);
+ ret = devm_regulator_bulk_get_enable(&client->dev,
+ ARRAY_SIZE(regulator_names),
+ regulator_names);
if (ret)
return dev_err_probe(&client->dev, ret, "Failed to get regulators\n");
- ret = regulator_bulk_enable(ARRAY_SIZE(data->regulators),
- data->regulators);
- if (ret)
- return ret;
-
- ret = devm_add_action_or_reset(&client->dev, kxcjk1013_disable_regulators, data);
- if (ret)
- return ret;
-
/*
* A typical delay of 10ms is required for powering up
* according to the data sheets of supported chips.
.of_match_table = kxcjk1013_of_match,
.pm = &kxcjk1013_pm_ops,
},
- .probe = kxcjk1013_probe,
+ .probe_new = kxcjk1013_probe,
.remove = kxcjk1013_remove,
.id_table = kxcjk1013_id,
};
#include "kxsd9.h"
-static int kxsd9_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int kxsd9_i2c_probe(struct i2c_client *i2c)
{
static const struct regmap_config config = {
.reg_bits = 8,
.of_match_table = kxsd9_of_match,
.pm = pm_ptr(&kxsd9_dev_pm_ops),
},
- .probe = kxsd9_i2c_probe,
+ .probe_new = kxsd9_i2c_probe,
.remove = kxsd9_i2c_remove,
.id_table = kxsd9_i2c_id,
};
.read_raw = mc3230_read_raw,
};
-static int mc3230_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mc3230_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
.name = "mc3230",
.pm = pm_sleep_ptr(&mc3230_pm_ops),
},
- .probe = mc3230_probe,
+ .probe_new = mc3230_probe,
.remove = mc3230_remove,
.id_table = mc3230_i2c_id,
};
#include "mma7455.h"
-static int mma7455_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int mma7455_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct regmap *regmap;
const char *name = NULL;
MODULE_DEVICE_TABLE(of, mma7455_of_match);
static struct i2c_driver mma7455_i2c_driver = {
- .probe = mma7455_i2c_probe,
+ .probe_new = mma7455_i2c_probe,
.remove = mma7455_i2c_remove,
.id_table = mma7455_i2c_ids,
.driver = {
.attrs = &mma7660_attribute_group,
};
-static int mma7660_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mma7660_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
.of_match_table = mma7660_of_match,
.acpi_match_table = mma7660_acpi_id,
},
- .probe = mma7660_probe,
+ .probe_new = mma7660_probe,
.remove = mma7660_remove,
.id_table = mma7660_i2c_id,
};
};
MODULE_DEVICE_TABLE(of, mma8452_dt_ids);
-static int mma8452_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mma8452_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct mma8452_data *data;
struct iio_dev *indio_dev;
int ret;
.of_match_table = mma8452_dt_ids,
.pm = &mma8452_pm_ops,
},
- .probe = mma8452_probe,
+ .probe_new = mma8452_probe,
.remove = mma8452_remove,
.id_table = mma8452_id,
};
return dev_name(dev);
}
-static int mma9551_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mma9551_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct mma9551_data *data;
struct iio_dev *indio_dev;
const char *name = NULL;
.acpi_match_table = ACPI_PTR(mma9551_acpi_match),
.pm = pm_ptr(&mma9551_pm_ops),
},
- .probe = mma9551_probe,
+ .probe_new = mma9551_probe,
.remove = mma9551_remove,
.id_table = mma9551_id,
};
return dev_name(dev);
}
-static int mma9553_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mma9553_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct mma9553_data *data;
struct iio_dev *indio_dev;
const char *name = NULL;
.acpi_match_table = ACPI_PTR(mma9553_acpi_match),
.pm = pm_ptr(&mma9553_pm_ops),
},
- .probe = mma9553_probe,
+ .probe_new = mma9553_probe,
.remove = mma9553_remove,
.id_table = mma9553_id,
};
* @chip_name: Chip name in the format "msa311-%02x" % partid
* @new_data_trig: Optional NEW_DATA interrupt driven trigger used
* to notify external consumers a new sample is ready
- * @vdd: Optional external voltage regulator for the device power supply
*/
struct msa311_priv {
struct regmap *regs;
char *chip_name;
struct iio_trigger *new_data_trig;
- struct regulator *vdd;
};
enum msa311_si {
msa311_set_pwr_mode(msa311, MSA311_PWR_MODE_SUSPEND);
}
-static void msa311_vdd_disable(void *vdd)
-{
- regulator_disable(vdd);
-}
-
static int msa311_probe(struct i2c_client *i2c)
{
struct device *dev = &i2c->dev;
mutex_init(&msa311->lock);
- msa311->vdd = devm_regulator_get(dev, "vdd");
- if (IS_ERR(msa311->vdd))
- return dev_err_probe(dev, PTR_ERR(msa311->vdd),
- "can't get vdd supply\n");
-
- err = regulator_enable(msa311->vdd);
+ err = devm_regulator_get_enable(dev, "vdd");
if (err)
- return dev_err_probe(dev, err, "can't enable vdd supply\n");
-
- err = devm_add_action_or_reset(dev, msa311_vdd_disable, msa311->vdd);
- if (err)
- return dev_err_probe(dev, err,
- "can't add vdd disable action\n");
+ return dev_err_probe(dev, err, "can't get vdd supply\n");
err = msa311_check_partid(msa311);
if (err)
return 0;
}
-static int mxc4005_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mxc4005_probe(struct i2c_client *client)
{
struct mxc4005_data *data;
struct iio_dev *indio_dev;
.name = MXC4005_DRV_NAME,
.acpi_match_table = ACPI_PTR(mxc4005_acpi_match),
},
- .probe = mxc4005_probe,
+ .probe_new = mxc4005_probe,
.id_table = mxc4005_id,
};
.readable_reg = mxc6255_is_readable_reg,
};
-static int mxc6255_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mxc6255_probe(struct i2c_client *client)
{
struct mxc6255_data *data;
struct iio_dev *indio_dev;
.name = MXC6255_DRV_NAME,
.acpi_match_table = ACPI_PTR(mxc6255_acpi_match),
},
- .probe = mxc6255_probe,
+ .probe_new = mxc6255_probe,
.id_table = mxc6255_id,
};
};
MODULE_DEVICE_TABLE(of, sca3300_dt_ids);
+static const struct spi_device_id sca3300_ids[] = {
+ { "sca3300" },
+ { "scl3300" },
+ {}
+};
+MODULE_DEVICE_TABLE(spi, sca3300_ids);
+
static struct spi_driver sca3300_driver = {
- .driver = {
+ .driver = {
.name = SCA3300_ALIAS,
.of_match_table = sca3300_dt_ids,
},
- .probe = sca3300_probe,
+ .probe = sca3300_probe,
+ .id_table = sca3300_ids,
};
module_spi_driver(sca3300_driver);
.postdisable = stk8312_buffer_postdisable,
};
-static int stk8312_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int stk8312_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
.name = STK8312_DRIVER_NAME,
.pm = pm_sleep_ptr(&stk8312_pm_ops),
},
- .probe = stk8312_probe,
+ .probe_new = stk8312_probe,
.remove = stk8312_remove,
.id_table = stk8312_i2c_id,
};
.postdisable = stk8ba50_buffer_postdisable,
};
-static int stk8ba50_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int stk8ba50_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
.pm = pm_sleep_ptr(&stk8ba50_pm_ops),
.acpi_match_table = ACPI_PTR(stk8ba50_acpi_id),
},
- .probe = stk8ba50_probe,
+ .probe_new = stk8ba50_probe,
.remove = stk8ba50_remove,
.id_table = stk8ba50_i2c_id,
};
select IIO_BUFFER
select IIO_TRIGGERED_BUFFER
+config AD4130
+ tristate "Analog Device AD4130 ADC Driver"
+ depends on SPI
+ depends on GPIOLIB
+ select IIO_BUFFER
+ select IIO_KFIFO_BUF
+ select REGMAP_SPI
+ depends on COMMON_CLK
+ help
+ Say yes here to build support for Analog Devices AD4130-8 SPI analog
+ to digital converters (ADC).
+
+ To compile this driver as a module, choose M here: the module will be
+ called ad4130.
+
config AD7091R5
tristate "Analog Devices AD7091R5 ADC Driver"
depends on I2C
To compile this driver as a module, choose M here: the module will be
called max11205.
+config MAX11410
+ tristate "Analog Devices MAX11410 ADC driver"
+ depends on SPI
+ select REGMAP_SPI
+ select IIO_BUFFER
+ select IIO_TRIGGER
+ select IIO_TRIGGERED_BUFFER
+ help
+ Say yes here to build support for Analog Devices MAX11410 ADCs.
+
+ To compile this driver as a module, choose M here: the module will be
+ called max11410.
+
config MAX1241
tristate "Maxim max1241 ADC driver"
depends on SPI_MASTER
is used in smartphones and tablets and supports a 11 channel
general purpose ADC.
+config MEDIATEK_MT6370_ADC
+ tristate "MediaTek MT6370 ADC driver"
+ depends on MFD_MT6370
+ help
+ Say yes here to enable MediaTek MT6370 ADC support.
+
+ This ADC driver provides 9 channels for system monitoring (charger
+ current, voltage, and temperature).
+
+ This driver can also be built as a module. If so, the module
+ will be called "mt6370-adc".
+
config MEDIATEK_MT6577_AUXADC
tristate "MediaTek AUXADC driver"
depends on ARCH_MEDIATEK || COMPILE_TEST
# When adding new entries keep the list in alphabetical order
obj-$(CONFIG_AB8500_GPADC) += ab8500-gpadc.o
obj-$(CONFIG_AD_SIGMA_DELTA) += ad_sigma_delta.o
+obj-$(CONFIG_AD4130) += ad4130.o
obj-$(CONFIG_AD7091R5) += ad7091r5.o ad7091r-base.o
obj-$(CONFIG_AD7124) += ad7124.o
obj-$(CONFIG_AD7192) += ad7192.o
obj-$(CONFIG_MAX11100) += max11100.o
obj-$(CONFIG_MAX1118) += max1118.o
obj-$(CONFIG_MAX11205) += max11205.o
+obj-$(CONFIG_MAX11410) += max11410.o
obj-$(CONFIG_MAX1241) += max1241.o
obj-$(CONFIG_MAX1363) += max1363.o
obj-$(CONFIG_MAX9611) += max9611.o
obj-$(CONFIG_MCP3422) += mcp3422.o
obj-$(CONFIG_MCP3911) += mcp3911.o
obj-$(CONFIG_MEDIATEK_MT6360_ADC) += mt6360-adc.o
+obj-$(CONFIG_MEDIATEK_MT6370_ADC) += mt6370-adc.o
obj-$(CONFIG_MEDIATEK_MT6577_AUXADC) += mt6577_auxadc.o
obj-$(CONFIG_MEN_Z188_ADC) += men_z188_adc.o
obj-$(CONFIG_MESON_SARADC) += meson_saradc.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2022 Analog Devices, Inc.
+ * Author: Cosmin Tanislav <cosmin.tanislav@analog.com>
+ */
+
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/gpio/driver.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/property.h>
+#include <linux/regmap.h>
+#include <linux/regulator/consumer.h>
+#include <linux/spi/spi.h>
+#include <linux/units.h>
+
+#include <asm/div64.h>
+#include <asm/unaligned.h>
+
+#include <linux/iio/buffer.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/kfifo_buf.h>
+#include <linux/iio/sysfs.h>
+
+#define AD4130_NAME "ad4130"
+
+#define AD4130_COMMS_READ_MASK BIT(6)
+
+#define AD4130_STATUS_REG 0x00
+
+#define AD4130_ADC_CONTROL_REG 0x01
+#define AD4130_ADC_CONTROL_BIPOLAR_MASK BIT(14)
+#define AD4130_ADC_CONTROL_INT_REF_VAL_MASK BIT(13)
+#define AD4130_INT_REF_2_5V 2500000
+#define AD4130_INT_REF_1_25V 1250000
+#define AD4130_ADC_CONTROL_CSB_EN_MASK BIT(9)
+#define AD4130_ADC_CONTROL_INT_REF_EN_MASK BIT(8)
+#define AD4130_ADC_CONTROL_MODE_MASK GENMASK(5, 2)
+#define AD4130_ADC_CONTROL_MCLK_SEL_MASK GENMASK(1, 0)
+#define AD4130_MCLK_FREQ_76_8KHZ 76800
+#define AD4130_MCLK_FREQ_153_6KHZ 153600
+
+#define AD4130_DATA_REG 0x02
+
+#define AD4130_IO_CONTROL_REG 0x03
+#define AD4130_IO_CONTROL_INT_PIN_SEL_MASK GENMASK(9, 8)
+#define AD4130_IO_CONTROL_GPIO_DATA_MASK GENMASK(7, 4)
+#define AD4130_IO_CONTROL_GPIO_CTRL_MASK GENMASK(3, 0)
+
+#define AD4130_VBIAS_REG 0x04
+
+#define AD4130_ID_REG 0x05
+
+#define AD4130_ERROR_REG 0x06
+
+#define AD4130_ERROR_EN_REG 0x07
+
+#define AD4130_MCLK_COUNT_REG 0x08
+
+#define AD4130_CHANNEL_X_REG(x) (0x09 + (x))
+#define AD4130_CHANNEL_EN_MASK BIT(23)
+#define AD4130_CHANNEL_SETUP_MASK GENMASK(22, 20)
+#define AD4130_CHANNEL_AINP_MASK GENMASK(17, 13)
+#define AD4130_CHANNEL_AINM_MASK GENMASK(12, 8)
+#define AD4130_CHANNEL_IOUT1_MASK GENMASK(7, 4)
+#define AD4130_CHANNEL_IOUT2_MASK GENMASK(3, 0)
+
+#define AD4130_CONFIG_X_REG(x) (0x19 + (x))
+#define AD4130_CONFIG_IOUT1_VAL_MASK GENMASK(15, 13)
+#define AD4130_CONFIG_IOUT2_VAL_MASK GENMASK(12, 10)
+#define AD4130_CONFIG_BURNOUT_MASK GENMASK(9, 8)
+#define AD4130_CONFIG_REF_BUFP_MASK BIT(7)
+#define AD4130_CONFIG_REF_BUFM_MASK BIT(6)
+#define AD4130_CONFIG_REF_SEL_MASK GENMASK(5, 4)
+#define AD4130_CONFIG_PGA_MASK GENMASK(3, 1)
+
+#define AD4130_FILTER_X_REG(x) (0x21 + (x))
+#define AD4130_FILTER_MODE_MASK GENMASK(15, 12)
+#define AD4130_FILTER_SELECT_MASK GENMASK(10, 0)
+#define AD4130_FILTER_SELECT_MIN 1
+
+#define AD4130_OFFSET_X_REG(x) (0x29 + (x))
+
+#define AD4130_GAIN_X_REG(x) (0x31 + (x))
+
+#define AD4130_MISC_REG 0x39
+
+#define AD4130_FIFO_CONTROL_REG 0x3a
+#define AD4130_FIFO_CONTROL_HEADER_MASK BIT(18)
+#define AD4130_FIFO_CONTROL_MODE_MASK GENMASK(17, 16)
+#define AD4130_FIFO_CONTROL_WM_INT_EN_MASK BIT(9)
+#define AD4130_FIFO_CONTROL_WM_MASK GENMASK(7, 0)
+#define AD4130_WATERMARK_256 0
+
+#define AD4130_FIFO_STATUS_REG 0x3b
+
+#define AD4130_FIFO_THRESHOLD_REG 0x3c
+
+#define AD4130_FIFO_DATA_REG 0x3d
+#define AD4130_FIFO_SIZE 256
+#define AD4130_FIFO_MAX_SAMPLE_SIZE 3
+
+#define AD4130_MAX_ANALOG_PINS 16
+#define AD4130_MAX_CHANNELS 16
+#define AD4130_MAX_DIFF_INPUTS 30
+#define AD4130_MAX_GPIOS 4
+#define AD4130_MAX_ODR 2400
+#define AD4130_MAX_PGA 8
+#define AD4130_MAX_SETUPS 8
+
+#define AD4130_AIN2_P1 0x2
+#define AD4130_AIN3_P2 0x3
+
+#define AD4130_RESET_BUF_SIZE 8
+#define AD4130_RESET_SLEEP_US (160 * MICRO / AD4130_MCLK_FREQ_76_8KHZ)
+
+#define AD4130_INVALID_SLOT -1
+
+static const unsigned int ad4130_reg_size[] = {
+ [AD4130_STATUS_REG] = 1,
+ [AD4130_ADC_CONTROL_REG] = 2,
+ [AD4130_DATA_REG] = 3,
+ [AD4130_IO_CONTROL_REG] = 2,
+ [AD4130_VBIAS_REG] = 2,
+ [AD4130_ID_REG] = 1,
+ [AD4130_ERROR_REG] = 2,
+ [AD4130_ERROR_EN_REG] = 2,
+ [AD4130_MCLK_COUNT_REG] = 1,
+ [AD4130_CHANNEL_X_REG(0) ... AD4130_CHANNEL_X_REG(AD4130_MAX_CHANNELS - 1)] = 3,
+ [AD4130_CONFIG_X_REG(0) ... AD4130_CONFIG_X_REG(AD4130_MAX_SETUPS - 1)] = 2,
+ [AD4130_FILTER_X_REG(0) ... AD4130_FILTER_X_REG(AD4130_MAX_SETUPS - 1)] = 3,
+ [AD4130_OFFSET_X_REG(0) ... AD4130_OFFSET_X_REG(AD4130_MAX_SETUPS - 1)] = 3,
+ [AD4130_GAIN_X_REG(0) ... AD4130_GAIN_X_REG(AD4130_MAX_SETUPS - 1)] = 3,
+ [AD4130_MISC_REG] = 2,
+ [AD4130_FIFO_CONTROL_REG] = 3,
+ [AD4130_FIFO_STATUS_REG] = 1,
+ [AD4130_FIFO_THRESHOLD_REG] = 3,
+ [AD4130_FIFO_DATA_REG] = 3,
+};
+
+enum ad4130_int_ref_val {
+ AD4130_INT_REF_VAL_2_5V,
+ AD4130_INT_REF_VAL_1_25V,
+};
+
+enum ad4130_mclk_sel {
+ AD4130_MCLK_76_8KHZ,
+ AD4130_MCLK_76_8KHZ_OUT,
+ AD4130_MCLK_76_8KHZ_EXT,
+ AD4130_MCLK_153_6KHZ_EXT,
+};
+
+enum ad4130_int_pin_sel {
+ AD4130_INT_PIN_INT,
+ AD4130_INT_PIN_CLK,
+ AD4130_INT_PIN_P2,
+ AD4130_INT_PIN_DOUT,
+};
+
+enum ad4130_iout {
+ AD4130_IOUT_OFF,
+ AD4130_IOUT_10000NA,
+ AD4130_IOUT_20000NA,
+ AD4130_IOUT_50000NA,
+ AD4130_IOUT_100000NA,
+ AD4130_IOUT_150000NA,
+ AD4130_IOUT_200000NA,
+ AD4130_IOUT_100NA,
+ AD4130_IOUT_MAX
+};
+
+enum ad4130_burnout {
+ AD4130_BURNOUT_OFF,
+ AD4130_BURNOUT_500NA,
+ AD4130_BURNOUT_2000NA,
+ AD4130_BURNOUT_4000NA,
+ AD4130_BURNOUT_MAX
+};
+
+enum ad4130_ref_sel {
+ AD4130_REF_REFIN1,
+ AD4130_REF_REFIN2,
+ AD4130_REF_REFOUT_AVSS,
+ AD4130_REF_AVDD_AVSS,
+ AD4130_REF_SEL_MAX
+};
+
+enum ad4130_fifo_mode {
+ AD4130_FIFO_MODE_DISABLED = 0b00,
+ AD4130_FIFO_MODE_WM = 0b01,
+};
+
+enum ad4130_mode {
+ AD4130_MODE_CONTINUOUS = 0b0000,
+ AD4130_MODE_IDLE = 0b0100,
+};
+
+enum ad4130_filter_mode {
+ AD4130_FILTER_SINC4,
+ AD4130_FILTER_SINC4_SINC1,
+ AD4130_FILTER_SINC3,
+ AD4130_FILTER_SINC3_REJ60,
+ AD4130_FILTER_SINC3_SINC1,
+ AD4130_FILTER_SINC3_PF1,
+ AD4130_FILTER_SINC3_PF2,
+ AD4130_FILTER_SINC3_PF3,
+ AD4130_FILTER_SINC3_PF4,
+};
+
+enum ad4130_pin_function {
+ AD4130_PIN_FN_NONE,
+ AD4130_PIN_FN_SPECIAL = BIT(0),
+ AD4130_PIN_FN_DIFF = BIT(1),
+ AD4130_PIN_FN_EXCITATION = BIT(2),
+ AD4130_PIN_FN_VBIAS = BIT(3),
+};
+
+struct ad4130_setup_info {
+ unsigned int iout0_val;
+ unsigned int iout1_val;
+ unsigned int burnout;
+ unsigned int pga;
+ unsigned int fs;
+ u32 ref_sel;
+ enum ad4130_filter_mode filter_mode;
+ bool ref_bufp;
+ bool ref_bufm;
+};
+
+struct ad4130_slot_info {
+ struct ad4130_setup_info setup;
+ unsigned int enabled_channels;
+ unsigned int channels;
+};
+
+struct ad4130_chan_info {
+ struct ad4130_setup_info setup;
+ u32 iout0;
+ u32 iout1;
+ int slot;
+ bool enabled;
+ bool initialized;
+};
+
+struct ad4130_filter_config {
+ enum ad4130_filter_mode filter_mode;
+ unsigned int odr_div;
+ unsigned int fs_max;
+ enum iio_available_type samp_freq_avail_type;
+ int samp_freq_avail_len;
+ int samp_freq_avail[3][2];
+};
+
+struct ad4130_state {
+ struct regmap *regmap;
+ struct spi_device *spi;
+ struct clk *mclk;
+ struct regulator_bulk_data regulators[4];
+ u32 irq_trigger;
+ u32 inv_irq_trigger;
+
+ /*
+ * Synchronize access to members the of driver state, and ensure
+ * atomicity of consecutive regmap operations.
+ */
+ struct mutex lock;
+ struct completion completion;
+
+ struct iio_chan_spec chans[AD4130_MAX_CHANNELS];
+ struct ad4130_chan_info chans_info[AD4130_MAX_CHANNELS];
+ struct ad4130_slot_info slots_info[AD4130_MAX_SETUPS];
+ enum ad4130_pin_function pins_fn[AD4130_MAX_ANALOG_PINS];
+ u32 vbias_pins[AD4130_MAX_ANALOG_PINS];
+ u32 num_vbias_pins;
+ int scale_tbls[AD4130_REF_SEL_MAX][AD4130_MAX_PGA][2];
+ struct gpio_chip gc;
+ struct clk_hw int_clk_hw;
+
+ u32 int_pin_sel;
+ u32 int_ref_uv;
+ u32 mclk_sel;
+ bool int_ref_en;
+ bool bipolar;
+
+ unsigned int num_enabled_channels;
+ unsigned int effective_watermark;
+ unsigned int watermark;
+
+ struct spi_message fifo_msg;
+ struct spi_transfer fifo_xfer[2];
+
+ /*
+ * DMA (thus cache coherency maintenance) requires any transfer
+ * buffers to live in their own cache lines. As the use of these
+ * buffers is synchronous, all of the buffers used for DMA in this
+ * driver may share a cache line.
+ */
+ u8 reset_buf[AD4130_RESET_BUF_SIZE] __aligned(IIO_DMA_MINALIGN);
+ u8 reg_write_tx_buf[4];
+ u8 reg_read_tx_buf[1];
+ u8 reg_read_rx_buf[3];
+ u8 fifo_tx_buf[2];
+ u8 fifo_rx_buf[AD4130_FIFO_SIZE *
+ AD4130_FIFO_MAX_SAMPLE_SIZE];
+};
+
+static const char * const ad4130_int_pin_names[] = {
+ [AD4130_INT_PIN_INT] = "int",
+ [AD4130_INT_PIN_CLK] = "clk",
+ [AD4130_INT_PIN_P2] = "p2",
+ [AD4130_INT_PIN_DOUT] = "dout",
+};
+
+static const unsigned int ad4130_iout_current_na_tbl[AD4130_IOUT_MAX] = {
+ [AD4130_IOUT_OFF] = 0,
+ [AD4130_IOUT_100NA] = 100,
+ [AD4130_IOUT_10000NA] = 10000,
+ [AD4130_IOUT_20000NA] = 20000,
+ [AD4130_IOUT_50000NA] = 50000,
+ [AD4130_IOUT_100000NA] = 100000,
+ [AD4130_IOUT_150000NA] = 150000,
+ [AD4130_IOUT_200000NA] = 200000,
+};
+
+static const unsigned int ad4130_burnout_current_na_tbl[AD4130_BURNOUT_MAX] = {
+ [AD4130_BURNOUT_OFF] = 0,
+ [AD4130_BURNOUT_500NA] = 500,
+ [AD4130_BURNOUT_2000NA] = 2000,
+ [AD4130_BURNOUT_4000NA] = 4000,
+};
+
+#define AD4130_VARIABLE_ODR_CONFIG(_filter_mode, _odr_div, _fs_max) \
+{ \
+ .filter_mode = (_filter_mode), \
+ .odr_div = (_odr_div), \
+ .fs_max = (_fs_max), \
+ .samp_freq_avail_type = IIO_AVAIL_RANGE, \
+ .samp_freq_avail = { \
+ { AD4130_MAX_ODR, (_odr_div) * (_fs_max) }, \
+ { AD4130_MAX_ODR, (_odr_div) * (_fs_max) }, \
+ { AD4130_MAX_ODR, (_odr_div) }, \
+ }, \
+}
+
+#define AD4130_FIXED_ODR_CONFIG(_filter_mode, _odr_div) \
+{ \
+ .filter_mode = (_filter_mode), \
+ .odr_div = (_odr_div), \
+ .fs_max = AD4130_FILTER_SELECT_MIN, \
+ .samp_freq_avail_type = IIO_AVAIL_LIST, \
+ .samp_freq_avail_len = 1, \
+ .samp_freq_avail = { \
+ { AD4130_MAX_ODR, (_odr_div) }, \
+ }, \
+}
+
+static const struct ad4130_filter_config ad4130_filter_configs[] = {
+ AD4130_VARIABLE_ODR_CONFIG(AD4130_FILTER_SINC4, 1, 10),
+ AD4130_VARIABLE_ODR_CONFIG(AD4130_FILTER_SINC4_SINC1, 11, 10),
+ AD4130_VARIABLE_ODR_CONFIG(AD4130_FILTER_SINC3, 1, 2047),
+ AD4130_VARIABLE_ODR_CONFIG(AD4130_FILTER_SINC3_REJ60, 1, 2047),
+ AD4130_VARIABLE_ODR_CONFIG(AD4130_FILTER_SINC3_SINC1, 10, 2047),
+ AD4130_FIXED_ODR_CONFIG(AD4130_FILTER_SINC3_PF1, 92),
+ AD4130_FIXED_ODR_CONFIG(AD4130_FILTER_SINC3_PF2, 100),
+ AD4130_FIXED_ODR_CONFIG(AD4130_FILTER_SINC3_PF3, 124),
+ AD4130_FIXED_ODR_CONFIG(AD4130_FILTER_SINC3_PF4, 148),
+};
+
+static const char * const ad4130_filter_modes_str[] = {
+ [AD4130_FILTER_SINC4] = "sinc4",
+ [AD4130_FILTER_SINC4_SINC1] = "sinc4+sinc1",
+ [AD4130_FILTER_SINC3] = "sinc3",
+ [AD4130_FILTER_SINC3_REJ60] = "sinc3+rej60",
+ [AD4130_FILTER_SINC3_SINC1] = "sinc3+sinc1",
+ [AD4130_FILTER_SINC3_PF1] = "sinc3+pf1",
+ [AD4130_FILTER_SINC3_PF2] = "sinc3+pf2",
+ [AD4130_FILTER_SINC3_PF3] = "sinc3+pf3",
+ [AD4130_FILTER_SINC3_PF4] = "sinc3+pf4",
+};
+
+static int ad4130_get_reg_size(struct ad4130_state *st, unsigned int reg,
+ unsigned int *size)
+{
+ if (reg >= ARRAY_SIZE(ad4130_reg_size))
+ return -EINVAL;
+
+ *size = ad4130_reg_size[reg];
+
+ return 0;
+}
+
+static unsigned int ad4130_data_reg_size(struct ad4130_state *st)
+{
+ unsigned int data_reg_size;
+ int ret;
+
+ ret = ad4130_get_reg_size(st, AD4130_DATA_REG, &data_reg_size);
+ if (ret)
+ return 0;
+
+ return data_reg_size;
+}
+
+static unsigned int ad4130_resolution(struct ad4130_state *st)
+{
+ return ad4130_data_reg_size(st) * BITS_PER_BYTE;
+}
+
+static int ad4130_reg_write(void *context, unsigned int reg, unsigned int val)
+{
+ struct ad4130_state *st = context;
+ unsigned int size;
+ int ret;
+
+ ret = ad4130_get_reg_size(st, reg, &size);
+ if (ret)
+ return ret;
+
+ st->reg_write_tx_buf[0] = reg;
+
+ switch (size) {
+ case 3:
+ put_unaligned_be24(val, &st->reg_write_tx_buf[1]);
+ break;
+ case 2:
+ put_unaligned_be16(val, &st->reg_write_tx_buf[1]);
+ break;
+ case 1:
+ st->reg_write_tx_buf[1] = val;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return spi_write(st->spi, st->reg_write_tx_buf, size + 1);
+}
+
+static int ad4130_reg_read(void *context, unsigned int reg, unsigned int *val)
+{
+ struct ad4130_state *st = context;
+ struct spi_transfer t[] = {
+ {
+ .tx_buf = st->reg_read_tx_buf,
+ .len = sizeof(st->reg_read_tx_buf),
+ },
+ {
+ .rx_buf = st->reg_read_rx_buf,
+ },
+ };
+ unsigned int size;
+ int ret;
+
+ ret = ad4130_get_reg_size(st, reg, &size);
+ if (ret)
+ return ret;
+
+ st->reg_read_tx_buf[0] = AD4130_COMMS_READ_MASK | reg;
+ t[1].len = size;
+
+ ret = spi_sync_transfer(st->spi, t, ARRAY_SIZE(t));
+ if (ret)
+ return ret;
+
+ switch (size) {
+ case 3:
+ *val = get_unaligned_be24(st->reg_read_rx_buf);
+ break;
+ case 2:
+ *val = get_unaligned_be16(st->reg_read_rx_buf);
+ break;
+ case 1:
+ *val = st->reg_read_rx_buf[0];
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct regmap_config ad4130_regmap_config = {
+ .reg_read = ad4130_reg_read,
+ .reg_write = ad4130_reg_write,
+};
+
+static int ad4130_gpio_init_valid_mask(struct gpio_chip *gc,
+ unsigned long *valid_mask,
+ unsigned int ngpios)
+{
+ struct ad4130_state *st = gpiochip_get_data(gc);
+ unsigned int i;
+
+ /*
+ * Output-only GPIO functionality is available on pins AIN2 through
+ * AIN5. If these pins are used for anything else, do not expose them.
+ */
+ for (i = 0; i < ngpios; i++) {
+ unsigned int pin = i + AD4130_AIN2_P1;
+ bool valid = st->pins_fn[pin] == AD4130_PIN_FN_NONE;
+
+ __assign_bit(i, valid_mask, valid);
+ }
+
+ return 0;
+}
+
+static int ad4130_gpio_get_direction(struct gpio_chip *gc, unsigned int offset)
+{
+ return GPIO_LINE_DIRECTION_OUT;
+}
+
+static void ad4130_gpio_set(struct gpio_chip *gc, unsigned int offset,
+ int value)
+{
+ struct ad4130_state *st = gpiochip_get_data(gc);
+ unsigned int mask = FIELD_PREP(AD4130_IO_CONTROL_GPIO_DATA_MASK,
+ BIT(offset));
+
+ regmap_update_bits(st->regmap, AD4130_IO_CONTROL_REG, mask,
+ value ? mask : 0);
+}
+
+static int ad4130_set_mode(struct ad4130_state *st, enum ad4130_mode mode)
+{
+ return regmap_update_bits(st->regmap, AD4130_ADC_CONTROL_REG,
+ AD4130_ADC_CONTROL_MODE_MASK,
+ FIELD_PREP(AD4130_ADC_CONTROL_MODE_MASK, mode));
+}
+
+static int ad4130_set_watermark_interrupt_en(struct ad4130_state *st, bool en)
+{
+ return regmap_update_bits(st->regmap, AD4130_FIFO_CONTROL_REG,
+ AD4130_FIFO_CONTROL_WM_INT_EN_MASK,
+ FIELD_PREP(AD4130_FIFO_CONTROL_WM_INT_EN_MASK, en));
+}
+
+static unsigned int ad4130_watermark_reg_val(unsigned int val)
+{
+ if (val == AD4130_FIFO_SIZE)
+ val = AD4130_WATERMARK_256;
+
+ return val;
+}
+
+static int ad4130_set_fifo_mode(struct ad4130_state *st,
+ enum ad4130_fifo_mode mode)
+{
+ return regmap_update_bits(st->regmap, AD4130_FIFO_CONTROL_REG,
+ AD4130_FIFO_CONTROL_MODE_MASK,
+ FIELD_PREP(AD4130_FIFO_CONTROL_MODE_MASK, mode));
+}
+
+static void ad4130_push_fifo_data(struct iio_dev *indio_dev)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ unsigned int data_reg_size = ad4130_data_reg_size(st);
+ unsigned int transfer_len = st->effective_watermark * data_reg_size;
+ unsigned int set_size = st->num_enabled_channels * data_reg_size;
+ unsigned int i;
+ int ret;
+
+ st->fifo_tx_buf[1] = ad4130_watermark_reg_val(st->effective_watermark);
+ st->fifo_xfer[1].len = transfer_len;
+
+ ret = spi_sync(st->spi, &st->fifo_msg);
+ if (ret)
+ return;
+
+ for (i = 0; i < transfer_len; i += set_size)
+ iio_push_to_buffers(indio_dev, &st->fifo_rx_buf[i]);
+}
+
+static irqreturn_t ad4130_irq_handler(int irq, void *private)
+{
+ struct iio_dev *indio_dev = private;
+ struct ad4130_state *st = iio_priv(indio_dev);
+
+ if (iio_buffer_enabled(indio_dev))
+ ad4130_push_fifo_data(indio_dev);
+ else
+ complete(&st->completion);
+
+ return IRQ_HANDLED;
+}
+
+static int ad4130_find_slot(struct ad4130_state *st,
+ struct ad4130_setup_info *target_setup_info,
+ unsigned int *slot, bool *overwrite)
+{
+ unsigned int i;
+
+ *slot = AD4130_INVALID_SLOT;
+ *overwrite = false;
+
+ for (i = 0; i < AD4130_MAX_SETUPS; i++) {
+ struct ad4130_slot_info *slot_info = &st->slots_info[i];
+
+ /* Immediately accept a matching setup info. */
+ if (!memcmp(target_setup_info, &slot_info->setup,
+ sizeof(*target_setup_info))) {
+ *slot = i;
+ return 0;
+ }
+
+ /* Ignore all setups which are used by enabled channels. */
+ if (slot_info->enabled_channels)
+ continue;
+
+ /* Find the least used slot. */
+ if (*slot == AD4130_INVALID_SLOT ||
+ slot_info->channels < st->slots_info[*slot].channels)
+ *slot = i;
+ }
+
+ if (*slot == AD4130_INVALID_SLOT)
+ return -EINVAL;
+
+ *overwrite = true;
+
+ return 0;
+}
+
+static void ad4130_unlink_channel(struct ad4130_state *st, unsigned int channel)
+{
+ struct ad4130_chan_info *chan_info = &st->chans_info[channel];
+ struct ad4130_slot_info *slot_info = &st->slots_info[chan_info->slot];
+
+ chan_info->slot = AD4130_INVALID_SLOT;
+ slot_info->channels--;
+}
+
+static int ad4130_unlink_slot(struct ad4130_state *st, unsigned int slot)
+{
+ unsigned int i;
+
+ for (i = 0; i < AD4130_MAX_CHANNELS; i++) {
+ struct ad4130_chan_info *chan_info = &st->chans_info[i];
+
+ if (!chan_info->initialized || chan_info->slot != slot)
+ continue;
+
+ ad4130_unlink_channel(st, i);
+ }
+
+ return 0;
+}
+
+static int ad4130_link_channel_slot(struct ad4130_state *st,
+ unsigned int channel, unsigned int slot)
+{
+ struct ad4130_slot_info *slot_info = &st->slots_info[slot];
+ struct ad4130_chan_info *chan_info = &st->chans_info[channel];
+ int ret;
+
+ ret = regmap_update_bits(st->regmap, AD4130_CHANNEL_X_REG(channel),
+ AD4130_CHANNEL_SETUP_MASK,
+ FIELD_PREP(AD4130_CHANNEL_SETUP_MASK, slot));
+ if (ret)
+ return ret;
+
+ chan_info->slot = slot;
+ slot_info->channels++;
+
+ return 0;
+}
+
+static int ad4130_write_slot_setup(struct ad4130_state *st,
+ unsigned int slot,
+ struct ad4130_setup_info *setup_info)
+{
+ unsigned int val;
+ int ret;
+
+ val = FIELD_PREP(AD4130_CONFIG_IOUT1_VAL_MASK, setup_info->iout0_val) |
+ FIELD_PREP(AD4130_CONFIG_IOUT1_VAL_MASK, setup_info->iout1_val) |
+ FIELD_PREP(AD4130_CONFIG_BURNOUT_MASK, setup_info->burnout) |
+ FIELD_PREP(AD4130_CONFIG_REF_BUFP_MASK, setup_info->ref_bufp) |
+ FIELD_PREP(AD4130_CONFIG_REF_BUFM_MASK, setup_info->ref_bufm) |
+ FIELD_PREP(AD4130_CONFIG_REF_SEL_MASK, setup_info->ref_sel) |
+ FIELD_PREP(AD4130_CONFIG_PGA_MASK, setup_info->pga);
+
+ ret = regmap_write(st->regmap, AD4130_CONFIG_X_REG(slot), val);
+ if (ret)
+ return ret;
+
+ val = FIELD_PREP(AD4130_FILTER_MODE_MASK, setup_info->filter_mode) |
+ FIELD_PREP(AD4130_FILTER_SELECT_MASK, setup_info->fs);
+
+ ret = regmap_write(st->regmap, AD4130_FILTER_X_REG(slot), val);
+ if (ret)
+ return ret;
+
+ memcpy(&st->slots_info[slot].setup, setup_info, sizeof(*setup_info));
+
+ return 0;
+}
+
+static int ad4130_write_channel_setup(struct ad4130_state *st,
+ unsigned int channel, bool on_enable)
+{
+ struct ad4130_chan_info *chan_info = &st->chans_info[channel];
+ struct ad4130_setup_info *setup_info = &chan_info->setup;
+ bool overwrite;
+ int slot;
+ int ret;
+
+ /*
+ * The following cases need to be handled.
+ *
+ * 1. Enabled and linked channel with setup changes:
+ * - Find a slot. If not possible, return error.
+ * - Unlink channel from current slot.
+ * - If the slot has channels linked to it, unlink all channels, and
+ * write the new setup to it.
+ * - Link channel to new slot.
+ *
+ * 2. Soon to be enabled and unlinked channel:
+ * - Find a slot. If not possible, return error.
+ * - If the slot has channels linked to it, unlink all channels, and
+ * write the new setup to it.
+ * - Link channel to the slot.
+ *
+ * 3. Disabled and linked channel with setup changes:
+ * - Unlink channel from current slot.
+ *
+ * 4. Soon to be enabled and linked channel:
+ * 5. Disabled and unlinked channel with setup changes:
+ * - Do nothing.
+ */
+
+ /* Case 4 */
+ if (on_enable && chan_info->slot != AD4130_INVALID_SLOT)
+ return 0;
+
+ if (!on_enable && !chan_info->enabled) {
+ if (chan_info->slot != AD4130_INVALID_SLOT)
+ /* Case 3 */
+ ad4130_unlink_channel(st, channel);
+
+ /* Cases 3 & 5 */
+ return 0;
+ }
+
+ /* Cases 1 & 2 */
+ ret = ad4130_find_slot(st, setup_info, &slot, &overwrite);
+ if (ret)
+ return ret;
+
+ if (chan_info->slot != AD4130_INVALID_SLOT)
+ /* Case 1 */
+ ad4130_unlink_channel(st, channel);
+
+ if (overwrite) {
+ ret = ad4130_unlink_slot(st, slot);
+ if (ret)
+ return ret;
+
+ ret = ad4130_write_slot_setup(st, slot, setup_info);
+ if (ret)
+ return ret;
+ }
+
+ return ad4130_link_channel_slot(st, channel, slot);
+}
+
+static int ad4130_set_channel_enable(struct ad4130_state *st,
+ unsigned int channel, bool status)
+{
+ struct ad4130_chan_info *chan_info = &st->chans_info[channel];
+ struct ad4130_slot_info *slot_info;
+ int ret;
+
+ if (chan_info->enabled == status)
+ return 0;
+
+ if (status) {
+ ret = ad4130_write_channel_setup(st, channel, true);
+ if (ret)
+ return ret;
+ }
+
+ slot_info = &st->slots_info[chan_info->slot];
+
+ ret = regmap_update_bits(st->regmap, AD4130_CHANNEL_X_REG(channel),
+ AD4130_CHANNEL_EN_MASK,
+ FIELD_PREP(AD4130_CHANNEL_EN_MASK, status));
+ if (ret)
+ return ret;
+
+ slot_info->enabled_channels += status ? 1 : -1;
+ chan_info->enabled = status;
+
+ return 0;
+}
+
+/*
+ * Table 58. FILTER_MODE_n bits and Filter Types of the datasheet describes
+ * the relation between filter mode, ODR and FS.
+ *
+ * Notice that the max ODR of each filter mode is not necessarily the
+ * absolute max ODR supported by the chip.
+ *
+ * The ODR divider is not explicitly specified, but it can be deduced based
+ * on the ODR range of each filter mode.
+ *
+ * For example, for Sinc4+Sinc1, max ODR is 218.18. That means that the
+ * absolute max ODR is divided by 11 to achieve the max ODR of this filter
+ * mode.
+ *
+ * The formulas for converting between ODR and FS for a specific filter
+ * mode can be deduced from the same table.
+ *
+ * Notice that FS = 1 actually means max ODR, and that ODR decreases by
+ * (maximum ODR / maximum FS) for each increment of FS.
+ *
+ * odr = MAX_ODR / odr_div * (1 - (fs - 1) / fs_max) <=>
+ * odr = MAX_ODR * (1 - (fs - 1) / fs_max) / odr_div <=>
+ * odr = MAX_ODR * (1 - (fs - 1) / fs_max) / odr_div <=>
+ * odr = MAX_ODR * (fs_max - fs + 1) / (fs_max * odr_div)
+ * (used in ad4130_fs_to_freq)
+ *
+ * For the opposite formula, FS can be extracted from the last one.
+ *
+ * MAX_ODR * (fs_max - fs + 1) = fs_max * odr_div * odr <=>
+ * fs_max - fs + 1 = fs_max * odr_div * odr / MAX_ODR <=>
+ * fs = 1 + fs_max - fs_max * odr_div * odr / MAX_ODR
+ * (used in ad4130_fs_to_freq)
+ */
+
+static void ad4130_freq_to_fs(enum ad4130_filter_mode filter_mode,
+ int val, int val2, unsigned int *fs)
+{
+ const struct ad4130_filter_config *filter_config =
+ &ad4130_filter_configs[filter_mode];
+ u64 dividend, divisor;
+ int temp;
+
+ dividend = filter_config->fs_max * filter_config->odr_div *
+ ((u64)val * NANO + val2);
+ divisor = (u64)AD4130_MAX_ODR * NANO;
+
+ temp = AD4130_FILTER_SELECT_MIN + filter_config->fs_max -
+ DIV64_U64_ROUND_CLOSEST(dividend, divisor);
+
+ if (temp < AD4130_FILTER_SELECT_MIN)
+ temp = AD4130_FILTER_SELECT_MIN;
+ else if (temp > filter_config->fs_max)
+ temp = filter_config->fs_max;
+
+ *fs = temp;
+}
+
+static void ad4130_fs_to_freq(enum ad4130_filter_mode filter_mode,
+ unsigned int fs, int *val, int *val2)
+{
+ const struct ad4130_filter_config *filter_config =
+ &ad4130_filter_configs[filter_mode];
+ unsigned int dividend, divisor;
+ u64 temp;
+
+ dividend = (filter_config->fs_max - fs + AD4130_FILTER_SELECT_MIN) *
+ AD4130_MAX_ODR;
+ divisor = filter_config->fs_max * filter_config->odr_div;
+
+ temp = div_u64((u64)dividend * NANO, divisor);
+ *val = div_u64_rem(temp, NANO, val2);
+}
+
+static int ad4130_set_filter_mode(struct iio_dev *indio_dev,
+ const struct iio_chan_spec *chan,
+ unsigned int val)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ unsigned int channel = chan->scan_index;
+ struct ad4130_chan_info *chan_info = &st->chans_info[channel];
+ struct ad4130_setup_info *setup_info = &chan_info->setup;
+ enum ad4130_filter_mode old_filter_mode;
+ int freq_val, freq_val2;
+ unsigned int old_fs;
+ int ret = 0;
+
+ mutex_lock(&st->lock);
+ if (setup_info->filter_mode == val)
+ goto out;
+
+ old_fs = setup_info->fs;
+ old_filter_mode = setup_info->filter_mode;
+
+ /*
+ * When switching between filter modes, try to match the ODR as
+ * close as possible. To do this, convert the current FS into ODR
+ * using the old filter mode, then convert it back into FS using
+ * the new filter mode.
+ */
+ ad4130_fs_to_freq(setup_info->filter_mode, setup_info->fs,
+ &freq_val, &freq_val2);
+
+ ad4130_freq_to_fs(val, freq_val, freq_val2, &setup_info->fs);
+
+ setup_info->filter_mode = val;
+
+ ret = ad4130_write_channel_setup(st, channel, false);
+ if (ret) {
+ setup_info->fs = old_fs;
+ setup_info->filter_mode = old_filter_mode;
+ }
+
+ out:
+ mutex_unlock(&st->lock);
+
+ return ret;
+}
+
+static int ad4130_get_filter_mode(struct iio_dev *indio_dev,
+ const struct iio_chan_spec *chan)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ unsigned int channel = chan->scan_index;
+ struct ad4130_setup_info *setup_info = &st->chans_info[channel].setup;
+ enum ad4130_filter_mode filter_mode;
+
+ mutex_lock(&st->lock);
+ filter_mode = setup_info->filter_mode;
+ mutex_unlock(&st->lock);
+
+ return filter_mode;
+}
+
+static const struct iio_enum ad4130_filter_mode_enum = {
+ .items = ad4130_filter_modes_str,
+ .num_items = ARRAY_SIZE(ad4130_filter_modes_str),
+ .set = ad4130_set_filter_mode,
+ .get = ad4130_get_filter_mode,
+};
+
+static const struct iio_chan_spec_ext_info ad4130_filter_mode_ext_info[] = {
+ IIO_ENUM("filter_mode", IIO_SEPARATE, &ad4130_filter_mode_enum),
+ IIO_ENUM_AVAILABLE("filter_mode", IIO_SHARED_BY_TYPE,
+ &ad4130_filter_mode_enum),
+ { }
+};
+
+static const struct iio_chan_spec ad4130_channel_template = {
+ .type = IIO_VOLTAGE,
+ .indexed = 1,
+ .differential = 1,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_SCALE) |
+ BIT(IIO_CHAN_INFO_OFFSET) |
+ BIT(IIO_CHAN_INFO_SAMP_FREQ),
+ .info_mask_separate_available = BIT(IIO_CHAN_INFO_SCALE) |
+ BIT(IIO_CHAN_INFO_SAMP_FREQ),
+ .ext_info = ad4130_filter_mode_ext_info,
+ .scan_type = {
+ .sign = 'u',
+ .endianness = IIO_BE,
+ },
+};
+
+static int ad4130_set_channel_pga(struct ad4130_state *st, unsigned int channel,
+ int val, int val2)
+{
+ struct ad4130_chan_info *chan_info = &st->chans_info[channel];
+ struct ad4130_setup_info *setup_info = &chan_info->setup;
+ unsigned int pga, old_pga;
+ int ret = 0;
+
+ for (pga = 0; pga < AD4130_MAX_PGA; pga++)
+ if (val == st->scale_tbls[setup_info->ref_sel][pga][0] &&
+ val2 == st->scale_tbls[setup_info->ref_sel][pga][1])
+ break;
+
+ if (pga == AD4130_MAX_PGA)
+ return -EINVAL;
+
+ mutex_lock(&st->lock);
+ if (pga == setup_info->pga)
+ goto out;
+
+ old_pga = setup_info->pga;
+ setup_info->pga = pga;
+
+ ret = ad4130_write_channel_setup(st, channel, false);
+ if (ret)
+ setup_info->pga = old_pga;
+
+out:
+ mutex_unlock(&st->lock);
+
+ return ret;
+}
+
+static int ad4130_set_channel_freq(struct ad4130_state *st,
+ unsigned int channel, int val, int val2)
+{
+ struct ad4130_chan_info *chan_info = &st->chans_info[channel];
+ struct ad4130_setup_info *setup_info = &chan_info->setup;
+ unsigned int fs, old_fs;
+ int ret = 0;
+
+ mutex_lock(&st->lock);
+ old_fs = setup_info->fs;
+
+ ad4130_freq_to_fs(setup_info->filter_mode, val, val2, &fs);
+
+ if (fs == setup_info->fs)
+ goto out;
+
+ setup_info->fs = fs;
+
+ ret = ad4130_write_channel_setup(st, channel, false);
+ if (ret)
+ setup_info->fs = old_fs;
+
+out:
+ mutex_unlock(&st->lock);
+
+ return ret;
+}
+
+static int _ad4130_read_sample(struct iio_dev *indio_dev, unsigned int channel,
+ int *val)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ int ret;
+
+ ret = ad4130_set_channel_enable(st, channel, true);
+ if (ret)
+ return ret;
+
+ reinit_completion(&st->completion);
+
+ ret = ad4130_set_mode(st, AD4130_MODE_CONTINUOUS);
+ if (ret)
+ return ret;
+
+ ret = wait_for_completion_timeout(&st->completion,
+ msecs_to_jiffies(1000));
+ if (!ret)
+ return -ETIMEDOUT;
+
+ ret = ad4130_set_mode(st, AD4130_MODE_IDLE);
+ if (ret)
+ return ret;
+
+ ret = regmap_read(st->regmap, AD4130_DATA_REG, val);
+ if (ret)
+ return ret;
+
+ ret = ad4130_set_channel_enable(st, channel, false);
+ if (ret)
+ return ret;
+
+ return IIO_VAL_INT;
+}
+
+static int ad4130_read_sample(struct iio_dev *indio_dev, unsigned int channel,
+ int *val)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ int ret;
+
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
+
+ mutex_lock(&st->lock);
+ ret = _ad4130_read_sample(indio_dev, channel, val);
+ mutex_unlock(&st->lock);
+
+ iio_device_release_direct_mode(indio_dev);
+
+ return ret;
+}
+
+static int ad4130_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long info)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ unsigned int channel = chan->scan_index;
+ struct ad4130_setup_info *setup_info = &st->chans_info[channel].setup;
+
+ switch (info) {
+ case IIO_CHAN_INFO_RAW:
+ return ad4130_read_sample(indio_dev, channel, val);
+ case IIO_CHAN_INFO_SCALE:
+ mutex_lock(&st->lock);
+ *val = st->scale_tbls[setup_info->ref_sel][setup_info->pga][0];
+ *val2 = st->scale_tbls[setup_info->ref_sel][setup_info->pga][1];
+ mutex_unlock(&st->lock);
+
+ return IIO_VAL_INT_PLUS_NANO;
+ case IIO_CHAN_INFO_OFFSET:
+ *val = st->bipolar ? -BIT(chan->scan_type.realbits - 1) : 0;
+
+ return IIO_VAL_INT;
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ mutex_lock(&st->lock);
+ ad4130_fs_to_freq(setup_info->filter_mode, setup_info->fs,
+ val, val2);
+ mutex_unlock(&st->lock);
+
+ return IIO_VAL_INT_PLUS_NANO;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int ad4130_read_avail(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ const int **vals, int *type, int *length,
+ long info)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ unsigned int channel = chan->scan_index;
+ struct ad4130_setup_info *setup_info = &st->chans_info[channel].setup;
+ const struct ad4130_filter_config *filter_config;
+
+ switch (info) {
+ case IIO_CHAN_INFO_SCALE:
+ *vals = (int *)st->scale_tbls[setup_info->ref_sel];
+ *length = ARRAY_SIZE(st->scale_tbls[setup_info->ref_sel]) * 2;
+
+ *type = IIO_VAL_INT_PLUS_NANO;
+
+ return IIO_AVAIL_LIST;
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ mutex_lock(&st->lock);
+ filter_config = &ad4130_filter_configs[setup_info->filter_mode];
+ mutex_unlock(&st->lock);
+
+ *vals = (int *)filter_config->samp_freq_avail;
+ *length = filter_config->samp_freq_avail_len * 2;
+ *type = IIO_VAL_FRACTIONAL;
+
+ return filter_config->samp_freq_avail_type;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int ad4130_write_raw_get_fmt(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ long info)
+{
+ switch (info) {
+ case IIO_CHAN_INFO_SCALE:
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ return IIO_VAL_INT_PLUS_NANO;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int ad4130_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long info)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ unsigned int channel = chan->scan_index;
+
+ switch (info) {
+ case IIO_CHAN_INFO_SCALE:
+ return ad4130_set_channel_pga(st, channel, val, val2);
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ return ad4130_set_channel_freq(st, channel, val, val2);
+ default:
+ return -EINVAL;
+ }
+}
+
+static int ad4130_reg_access(struct iio_dev *indio_dev, unsigned int reg,
+ unsigned int writeval, unsigned int *readval)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+
+ if (readval)
+ return regmap_read(st->regmap, reg, readval);
+
+ return regmap_write(st->regmap, reg, writeval);
+}
+
+static int ad4130_update_scan_mode(struct iio_dev *indio_dev,
+ const unsigned long *scan_mask)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ unsigned int channel;
+ unsigned int val = 0;
+ int ret;
+
+ mutex_lock(&st->lock);
+
+ for_each_set_bit(channel, scan_mask, indio_dev->num_channels) {
+ ret = ad4130_set_channel_enable(st, channel, true);
+ if (ret)
+ goto out;
+
+ val++;
+ }
+
+ st->num_enabled_channels = val;
+
+out:
+ mutex_unlock(&st->lock);
+
+ return 0;
+}
+
+static int ad4130_set_fifo_watermark(struct iio_dev *indio_dev, unsigned int val)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ unsigned int eff;
+ int ret;
+
+ if (val > AD4130_FIFO_SIZE)
+ return -EINVAL;
+
+ eff = val * st->num_enabled_channels;
+ if (eff > AD4130_FIFO_SIZE)
+ /*
+ * Always set watermark to a multiple of the number of
+ * enabled channels to avoid making the FIFO unaligned.
+ */
+ eff = rounddown(AD4130_FIFO_SIZE, st->num_enabled_channels);
+
+ mutex_lock(&st->lock);
+
+ ret = regmap_update_bits(st->regmap, AD4130_FIFO_CONTROL_REG,
+ AD4130_FIFO_CONTROL_WM_MASK,
+ FIELD_PREP(AD4130_FIFO_CONTROL_WM_MASK,
+ ad4130_watermark_reg_val(eff)));
+ if (ret)
+ goto out;
+
+ st->effective_watermark = eff;
+ st->watermark = val;
+
+out:
+ mutex_unlock(&st->lock);
+
+ return ret;
+}
+
+static const struct iio_info ad4130_info = {
+ .read_raw = ad4130_read_raw,
+ .read_avail = ad4130_read_avail,
+ .write_raw_get_fmt = ad4130_write_raw_get_fmt,
+ .write_raw = ad4130_write_raw,
+ .update_scan_mode = ad4130_update_scan_mode,
+ .hwfifo_set_watermark = ad4130_set_fifo_watermark,
+ .debugfs_reg_access = ad4130_reg_access,
+};
+
+static int ad4130_buffer_postenable(struct iio_dev *indio_dev)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ int ret;
+
+ mutex_lock(&st->lock);
+
+ ret = ad4130_set_watermark_interrupt_en(st, true);
+ if (ret)
+ goto out;
+
+ ret = irq_set_irq_type(st->spi->irq, st->inv_irq_trigger);
+ if (ret)
+ goto out;
+
+ ret = ad4130_set_fifo_mode(st, AD4130_FIFO_MODE_WM);
+ if (ret)
+ goto out;
+
+ ret = ad4130_set_mode(st, AD4130_MODE_CONTINUOUS);
+
+out:
+ mutex_unlock(&st->lock);
+
+ return ret;
+}
+
+static int ad4130_buffer_predisable(struct iio_dev *indio_dev)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ unsigned int i;
+ int ret;
+
+ mutex_lock(&st->lock);
+
+ ret = ad4130_set_mode(st, AD4130_MODE_IDLE);
+ if (ret)
+ goto out;
+
+ ret = irq_set_irq_type(st->spi->irq, st->irq_trigger);
+ if (ret)
+ goto out;
+
+ ret = ad4130_set_fifo_mode(st, AD4130_FIFO_MODE_DISABLED);
+ if (ret)
+ goto out;
+
+ ret = ad4130_set_watermark_interrupt_en(st, false);
+ if (ret)
+ goto out;
+
+ /*
+ * update_scan_mode() is not called in the disable path, disable all
+ * channels here.
+ */
+ for (i = 0; i < indio_dev->num_channels; i++) {
+ ret = ad4130_set_channel_enable(st, i, false);
+ if (ret)
+ goto out;
+ }
+
+out:
+ mutex_unlock(&st->lock);
+
+ return ret;
+}
+
+static const struct iio_buffer_setup_ops ad4130_buffer_ops = {
+ .postenable = ad4130_buffer_postenable,
+ .predisable = ad4130_buffer_predisable,
+};
+
+static ssize_t hwfifo_watermark_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ad4130_state *st = iio_priv(dev_to_iio_dev(dev));
+ unsigned int val;
+
+ mutex_lock(&st->lock);
+ val = st->watermark;
+ mutex_unlock(&st->lock);
+
+ return sysfs_emit(buf, "%d\n", val);
+}
+
+static ssize_t hwfifo_enabled_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ad4130_state *st = iio_priv(dev_to_iio_dev(dev));
+ unsigned int val;
+ int ret;
+
+ ret = regmap_read(st->regmap, AD4130_FIFO_CONTROL_REG, &val);
+ if (ret)
+ return ret;
+
+ val = FIELD_GET(AD4130_FIFO_CONTROL_MODE_MASK, val);
+
+ return sysfs_emit(buf, "%d\n", val != AD4130_FIFO_MODE_DISABLED);
+}
+
+static ssize_t hwfifo_watermark_min_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ return sysfs_emit(buf, "%s\n", "1");
+}
+
+static ssize_t hwfifo_watermark_max_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ return sysfs_emit(buf, "%s\n", __stringify(AD4130_FIFO_SIZE));
+}
+
+static IIO_DEVICE_ATTR_RO(hwfifo_watermark_min, 0);
+static IIO_DEVICE_ATTR_RO(hwfifo_watermark_max, 0);
+static IIO_DEVICE_ATTR_RO(hwfifo_watermark, 0);
+static IIO_DEVICE_ATTR_RO(hwfifo_enabled, 0);
+
+static const struct iio_dev_attr *ad4130_fifo_attributes[] = {
+ &iio_dev_attr_hwfifo_watermark_min,
+ &iio_dev_attr_hwfifo_watermark_max,
+ &iio_dev_attr_hwfifo_watermark,
+ &iio_dev_attr_hwfifo_enabled,
+ NULL
+};
+
+static int _ad4130_find_table_index(const unsigned int *tbl, size_t len,
+ unsigned int val)
+{
+ unsigned int i;
+
+ for (i = 0; i < len; i++)
+ if (tbl[i] == val)
+ return i;
+
+ return -EINVAL;
+}
+
+#define ad4130_find_table_index(table, val) \
+ _ad4130_find_table_index(table, ARRAY_SIZE(table), val)
+
+static int ad4130_get_ref_voltage(struct ad4130_state *st,
+ enum ad4130_ref_sel ref_sel)
+{
+ switch (ref_sel) {
+ case AD4130_REF_REFIN1:
+ return regulator_get_voltage(st->regulators[2].consumer);
+ case AD4130_REF_REFIN2:
+ return regulator_get_voltage(st->regulators[3].consumer);
+ case AD4130_REF_AVDD_AVSS:
+ return regulator_get_voltage(st->regulators[0].consumer);
+ case AD4130_REF_REFOUT_AVSS:
+ return st->int_ref_uv;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int ad4130_parse_fw_setup(struct ad4130_state *st,
+ struct fwnode_handle *child,
+ struct ad4130_setup_info *setup_info)
+{
+ struct device *dev = &st->spi->dev;
+ u32 tmp;
+ int ret;
+
+ tmp = 0;
+ fwnode_property_read_u32(child, "adi,excitation-current-0-nanoamp", &tmp);
+ ret = ad4130_find_table_index(ad4130_iout_current_na_tbl, tmp);
+ if (ret < 0)
+ return dev_err_probe(dev, ret,
+ "Invalid excitation current %unA\n", tmp);
+ setup_info->iout0_val = ret;
+
+ tmp = 0;
+ fwnode_property_read_u32(child, "adi,excitation-current-1-nanoamp", &tmp);
+ ret = ad4130_find_table_index(ad4130_iout_current_na_tbl, tmp);
+ if (ret < 0)
+ return dev_err_probe(dev, ret,
+ "Invalid excitation current %unA\n", tmp);
+ setup_info->iout1_val = ret;
+
+ tmp = 0;
+ fwnode_property_read_u32(child, "adi,burnout-current-nanoamp", &tmp);
+ ret = ad4130_find_table_index(ad4130_burnout_current_na_tbl, tmp);
+ if (ret < 0)
+ return dev_err_probe(dev, ret,
+ "Invalid burnout current %unA\n", tmp);
+ setup_info->burnout = ret;
+
+ setup_info->ref_bufp = fwnode_property_read_bool(child, "adi,buffered-positive");
+ setup_info->ref_bufm = fwnode_property_read_bool(child, "adi,buffered-negative");
+
+ setup_info->ref_sel = AD4130_REF_REFIN1;
+ fwnode_property_read_u32(child, "adi,reference-select",
+ &setup_info->ref_sel);
+ if (setup_info->ref_sel >= AD4130_REF_SEL_MAX)
+ return dev_err_probe(dev, -EINVAL,
+ "Invalid reference selected %u\n",
+ setup_info->ref_sel);
+
+ if (setup_info->ref_sel == AD4130_REF_REFOUT_AVSS)
+ st->int_ref_en = true;
+
+ ret = ad4130_get_ref_voltage(st, setup_info->ref_sel);
+ if (ret < 0)
+ return dev_err_probe(dev, ret, "Cannot use reference %u\n",
+ setup_info->ref_sel);
+
+ return 0;
+}
+
+static int ad4130_validate_diff_channel(struct ad4130_state *st, u32 pin)
+{
+ struct device *dev = &st->spi->dev;
+
+ if (pin >= AD4130_MAX_DIFF_INPUTS)
+ return dev_err_probe(dev, -EINVAL,
+ "Invalid differential channel %u\n", pin);
+
+ if (pin >= AD4130_MAX_ANALOG_PINS)
+ return 0;
+
+ if (st->pins_fn[pin] == AD4130_PIN_FN_SPECIAL)
+ return dev_err_probe(dev, -EINVAL,
+ "Pin %u already used with fn %u\n", pin,
+ st->pins_fn[pin]);
+
+ st->pins_fn[pin] |= AD4130_PIN_FN_DIFF;
+
+ return 0;
+}
+
+static int ad4130_validate_diff_channels(struct ad4130_state *st,
+ u32 *pins, unsigned int len)
+{
+ unsigned int i;
+ int ret;
+
+ for (i = 0; i < len; i++) {
+ ret = ad4130_validate_diff_channel(st, pins[i]);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int ad4130_validate_excitation_pin(struct ad4130_state *st, u32 pin)
+{
+ struct device *dev = &st->spi->dev;
+
+ if (pin >= AD4130_MAX_ANALOG_PINS)
+ return dev_err_probe(dev, -EINVAL,
+ "Invalid excitation pin %u\n", pin);
+
+ if (st->pins_fn[pin] == AD4130_PIN_FN_SPECIAL)
+ return dev_err_probe(dev, -EINVAL,
+ "Pin %u already used with fn %u\n", pin,
+ st->pins_fn[pin]);
+
+ st->pins_fn[pin] |= AD4130_PIN_FN_EXCITATION;
+
+ return 0;
+}
+
+static int ad4130_validate_vbias_pin(struct ad4130_state *st, u32 pin)
+{
+ struct device *dev = &st->spi->dev;
+
+ if (pin >= AD4130_MAX_ANALOG_PINS)
+ return dev_err_probe(dev, -EINVAL, "Invalid vbias pin %u\n",
+ pin);
+
+ if (st->pins_fn[pin] == AD4130_PIN_FN_SPECIAL)
+ return dev_err_probe(dev, -EINVAL,
+ "Pin %u already used with fn %u\n", pin,
+ st->pins_fn[pin]);
+
+ st->pins_fn[pin] |= AD4130_PIN_FN_VBIAS;
+
+ return 0;
+}
+
+static int ad4130_validate_vbias_pins(struct ad4130_state *st,
+ u32 *pins, unsigned int len)
+{
+ unsigned int i;
+ int ret;
+
+ for (i = 0; i < st->num_vbias_pins; i++) {
+ ret = ad4130_validate_vbias_pin(st, pins[i]);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int ad4130_parse_fw_channel(struct iio_dev *indio_dev,
+ struct fwnode_handle *child)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ unsigned int resolution = ad4130_resolution(st);
+ unsigned int index = indio_dev->num_channels++;
+ struct device *dev = &st->spi->dev;
+ struct ad4130_chan_info *chan_info;
+ struct iio_chan_spec *chan;
+ u32 pins[2];
+ int ret;
+
+ if (index >= AD4130_MAX_CHANNELS)
+ return dev_err_probe(dev, -EINVAL, "Too many channels\n");
+
+ chan = &st->chans[index];
+ chan_info = &st->chans_info[index];
+
+ *chan = ad4130_channel_template;
+ chan->scan_type.realbits = resolution;
+ chan->scan_type.storagebits = resolution;
+ chan->scan_index = index;
+
+ chan_info->slot = AD4130_INVALID_SLOT;
+ chan_info->setup.fs = AD4130_FILTER_SELECT_MIN;
+ chan_info->initialized = true;
+
+ ret = fwnode_property_read_u32_array(child, "diff-channels", pins,
+ ARRAY_SIZE(pins));
+ if (ret)
+ return ret;
+
+ ret = ad4130_validate_diff_channels(st, pins, ARRAY_SIZE(pins));
+ if (ret)
+ return ret;
+
+ chan->channel = pins[0];
+ chan->channel2 = pins[1];
+
+ ret = ad4130_parse_fw_setup(st, child, &chan_info->setup);
+ if (ret)
+ return ret;
+
+ fwnode_property_read_u32(child, "adi,excitation-pin-0",
+ &chan_info->iout0);
+ if (chan_info->setup.iout0_val != AD4130_IOUT_OFF) {
+ ret = ad4130_validate_excitation_pin(st, chan_info->iout0);
+ if (ret)
+ return ret;
+ }
+
+ fwnode_property_read_u32(child, "adi,excitation-pin-1",
+ &chan_info->iout1);
+ if (chan_info->setup.iout1_val != AD4130_IOUT_OFF) {
+ ret = ad4130_validate_excitation_pin(st, chan_info->iout1);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int ad4130_parse_fw_children(struct iio_dev *indio_dev)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ struct device *dev = &st->spi->dev;
+ struct fwnode_handle *child;
+ int ret;
+
+ indio_dev->channels = st->chans;
+
+ device_for_each_child_node(dev, child) {
+ ret = ad4130_parse_fw_channel(indio_dev, child);
+ if (ret) {
+ fwnode_handle_put(child);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static int ad4310_parse_fw(struct iio_dev *indio_dev)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ struct device *dev = &st->spi->dev;
+ u32 ext_clk_freq = AD4130_MCLK_FREQ_76_8KHZ;
+ unsigned int i;
+ int avdd_uv;
+ int irq;
+ int ret;
+
+ st->mclk = devm_clk_get_optional(dev, "mclk");
+ if (IS_ERR(st->mclk))
+ return dev_err_probe(dev, PTR_ERR(st->mclk),
+ "Failed to get mclk\n");
+
+ st->int_pin_sel = AD4130_INT_PIN_INT;
+
+ for (i = 0; i < ARRAY_SIZE(ad4130_int_pin_names); i++) {
+ irq = fwnode_irq_get_byname(dev_fwnode(dev),
+ ad4130_int_pin_names[i]);
+ if (irq > 0) {
+ st->int_pin_sel = i;
+ break;
+ }
+ }
+
+ if (st->int_pin_sel == AD4130_INT_PIN_DOUT)
+ return dev_err_probe(dev, -EINVAL,
+ "Cannot use DOUT as interrupt pin\n");
+
+ if (st->int_pin_sel == AD4130_INT_PIN_P2)
+ st->pins_fn[AD4130_AIN3_P2] = AD4130_PIN_FN_SPECIAL;
+
+ device_property_read_u32(dev, "adi,ext-clk-freq-hz", &ext_clk_freq);
+ if (ext_clk_freq != AD4130_MCLK_FREQ_153_6KHZ &&
+ ext_clk_freq != AD4130_MCLK_FREQ_76_8KHZ)
+ return dev_err_probe(dev, -EINVAL,
+ "Invalid external clock frequency %u\n",
+ ext_clk_freq);
+
+ if (st->mclk && ext_clk_freq == AD4130_MCLK_FREQ_153_6KHZ)
+ st->mclk_sel = AD4130_MCLK_153_6KHZ_EXT;
+ else if (st->mclk)
+ st->mclk_sel = AD4130_MCLK_76_8KHZ_EXT;
+ else
+ st->mclk_sel = AD4130_MCLK_76_8KHZ;
+
+ if (st->int_pin_sel == AD4130_INT_PIN_CLK &&
+ st->mclk_sel != AD4130_MCLK_76_8KHZ)
+ return dev_err_probe(dev, -EINVAL,
+ "Invalid clock %u for interrupt pin %u\n",
+ st->mclk_sel, st->int_pin_sel);
+
+ st->int_ref_uv = AD4130_INT_REF_2_5V;
+
+ /*
+ * When the AVDD supply is set to below 2.5V the internal reference of
+ * 1.25V should be selected.
+ * See datasheet page 37, section ADC REFERENCE.
+ */
+ avdd_uv = regulator_get_voltage(st->regulators[0].consumer);
+ if (avdd_uv > 0 && avdd_uv < AD4130_INT_REF_2_5V)
+ st->int_ref_uv = AD4130_INT_REF_1_25V;
+
+ st->bipolar = device_property_read_bool(dev, "adi,bipolar");
+
+ ret = device_property_count_u32(dev, "adi,vbias-pins");
+ if (ret > 0) {
+ if (ret > AD4130_MAX_ANALOG_PINS)
+ return dev_err_probe(dev, -EINVAL,
+ "Too many vbias pins %u\n", ret);
+
+ st->num_vbias_pins = ret;
+
+ ret = device_property_read_u32_array(dev, "adi,vbias-pins",
+ st->vbias_pins,
+ st->num_vbias_pins);
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "Failed to read vbias pins\n");
+
+ ret = ad4130_validate_vbias_pins(st, st->vbias_pins,
+ st->num_vbias_pins);
+ if (ret)
+ return ret;
+ }
+
+ ret = ad4130_parse_fw_children(indio_dev);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static void ad4130_fill_scale_tbls(struct ad4130_state *st)
+{
+ unsigned int pow = ad4130_resolution(st) - st->bipolar;
+ unsigned int i, j;
+
+ for (i = 0; i < AD4130_REF_SEL_MAX; i++) {
+ int ret;
+ u64 nv;
+
+ ret = ad4130_get_ref_voltage(st, i);
+ if (ret < 0)
+ continue;
+
+ nv = (u64)ret * NANO;
+
+ for (j = 0; j < AD4130_MAX_PGA; j++)
+ st->scale_tbls[i][j][1] = div_u64(nv >> (pow + j), MILLI);
+ }
+}
+
+static void ad4130_clk_disable_unprepare(void *clk)
+{
+ clk_disable_unprepare(clk);
+}
+
+static int ad4130_set_mclk_sel(struct ad4130_state *st,
+ enum ad4130_mclk_sel mclk_sel)
+{
+ return regmap_update_bits(st->regmap, AD4130_ADC_CONTROL_REG,
+ AD4130_ADC_CONTROL_MCLK_SEL_MASK,
+ FIELD_PREP(AD4130_ADC_CONTROL_MCLK_SEL_MASK,
+ mclk_sel));
+}
+
+static unsigned long ad4130_int_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ return AD4130_MCLK_FREQ_76_8KHZ;
+}
+
+static int ad4130_int_clk_is_enabled(struct clk_hw *hw)
+{
+ struct ad4130_state *st = container_of(hw, struct ad4130_state, int_clk_hw);
+
+ return st->mclk_sel == AD4130_MCLK_76_8KHZ_OUT;
+}
+
+static int ad4130_int_clk_prepare(struct clk_hw *hw)
+{
+ struct ad4130_state *st = container_of(hw, struct ad4130_state, int_clk_hw);
+ int ret;
+
+ ret = ad4130_set_mclk_sel(st, AD4130_MCLK_76_8KHZ_OUT);
+ if (ret)
+ return ret;
+
+ st->mclk_sel = AD4130_MCLK_76_8KHZ_OUT;
+
+ return 0;
+}
+
+static void ad4130_int_clk_unprepare(struct clk_hw *hw)
+{
+ struct ad4130_state *st = container_of(hw, struct ad4130_state, int_clk_hw);
+ int ret;
+
+ ret = ad4130_set_mclk_sel(st, AD4130_MCLK_76_8KHZ);
+ if (ret)
+ return;
+
+ st->mclk_sel = AD4130_MCLK_76_8KHZ;
+}
+
+static const struct clk_ops ad4130_int_clk_ops = {
+ .recalc_rate = ad4130_int_clk_recalc_rate,
+ .is_enabled = ad4130_int_clk_is_enabled,
+ .prepare = ad4130_int_clk_prepare,
+ .unprepare = ad4130_int_clk_unprepare,
+};
+
+static int ad4130_setup_int_clk(struct ad4130_state *st)
+{
+ struct device *dev = &st->spi->dev;
+ struct device_node *of_node = dev_of_node(dev);
+ struct clk_init_data init;
+ const char *clk_name;
+ struct clk *clk;
+
+ if (st->int_pin_sel == AD4130_INT_PIN_CLK ||
+ st->mclk_sel != AD4130_MCLK_76_8KHZ)
+ return 0;
+
+ if (!of_node)
+ return 0;
+
+ clk_name = of_node->name;
+ of_property_read_string(of_node, "clock-output-names", &clk_name);
+
+ init.name = clk_name;
+ init.ops = &ad4130_int_clk_ops;
+
+ st->int_clk_hw.init = &init;
+ clk = devm_clk_register(dev, &st->int_clk_hw);
+ if (IS_ERR(clk))
+ return PTR_ERR(clk);
+
+ return of_clk_add_provider(of_node, of_clk_src_simple_get, clk);
+}
+
+static int ad4130_setup(struct iio_dev *indio_dev)
+{
+ struct ad4130_state *st = iio_priv(indio_dev);
+ struct device *dev = &st->spi->dev;
+ unsigned int int_ref_val;
+ unsigned long rate = AD4130_MCLK_FREQ_76_8KHZ;
+ unsigned int val;
+ unsigned int i;
+ int ret;
+
+ if (st->mclk_sel == AD4130_MCLK_153_6KHZ_EXT)
+ rate = AD4130_MCLK_FREQ_153_6KHZ;
+
+ ret = clk_set_rate(st->mclk, rate);
+ if (ret)
+ return ret;
+
+ ret = clk_prepare_enable(st->mclk);
+ if (ret)
+ return ret;
+
+ ret = devm_add_action_or_reset(dev, ad4130_clk_disable_unprepare,
+ st->mclk);
+ if (ret)
+ return ret;
+
+ if (st->int_ref_uv == AD4130_INT_REF_2_5V)
+ int_ref_val = AD4130_INT_REF_VAL_2_5V;
+ else
+ int_ref_val = AD4130_INT_REF_VAL_1_25V;
+
+ /* Switch to SPI 4-wire mode. */
+ val = FIELD_PREP(AD4130_ADC_CONTROL_CSB_EN_MASK, 1);
+ val |= FIELD_PREP(AD4130_ADC_CONTROL_BIPOLAR_MASK, st->bipolar);
+ val |= FIELD_PREP(AD4130_ADC_CONTROL_INT_REF_EN_MASK, st->int_ref_en);
+ val |= FIELD_PREP(AD4130_ADC_CONTROL_MODE_MASK, AD4130_MODE_IDLE);
+ val |= FIELD_PREP(AD4130_ADC_CONTROL_MCLK_SEL_MASK, st->mclk_sel);
+ val |= FIELD_PREP(AD4130_ADC_CONTROL_INT_REF_VAL_MASK, int_ref_val);
+
+ ret = regmap_write(st->regmap, AD4130_ADC_CONTROL_REG, val);
+ if (ret)
+ return ret;
+
+ /*
+ * Configure all GPIOs for output. If configured, the interrupt function
+ * of P2 takes priority over the GPIO out function.
+ */
+ val = AD4130_IO_CONTROL_GPIO_CTRL_MASK;
+ val |= FIELD_PREP(AD4130_IO_CONTROL_INT_PIN_SEL_MASK, st->int_pin_sel);
+
+ ret = regmap_write(st->regmap, AD4130_IO_CONTROL_REG, val);
+ if (ret)
+ return ret;
+
+ val = 0;
+ for (i = 0; i < st->num_vbias_pins; i++)
+ val |= BIT(st->vbias_pins[i]);
+
+ ret = regmap_write(st->regmap, AD4130_VBIAS_REG, val);
+ if (ret)
+ return ret;
+
+ ret = regmap_update_bits(st->regmap, AD4130_FIFO_CONTROL_REG,
+ AD4130_FIFO_CONTROL_HEADER_MASK, 0);
+ if (ret)
+ return ret;
+
+ /* FIFO watermark interrupt starts out as enabled, disable it. */
+ ret = ad4130_set_watermark_interrupt_en(st, false);
+ if (ret)
+ return ret;
+
+ /* Setup channels. */
+ for (i = 0; i < indio_dev->num_channels; i++) {
+ struct ad4130_chan_info *chan_info = &st->chans_info[i];
+ struct iio_chan_spec *chan = &st->chans[i];
+ unsigned int val;
+
+ val = FIELD_PREP(AD4130_CHANNEL_AINP_MASK, chan->channel) |
+ FIELD_PREP(AD4130_CHANNEL_AINM_MASK, chan->channel2) |
+ FIELD_PREP(AD4130_CHANNEL_IOUT1_MASK, chan_info->iout0) |
+ FIELD_PREP(AD4130_CHANNEL_IOUT2_MASK, chan_info->iout1);
+
+ ret = regmap_write(st->regmap, AD4130_CHANNEL_X_REG(i), val);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int ad4130_soft_reset(struct ad4130_state *st)
+{
+ int ret;
+
+ ret = spi_write(st->spi, st->reset_buf, sizeof(st->reset_buf));
+ if (ret)
+ return ret;
+
+ fsleep(AD4130_RESET_SLEEP_US);
+
+ return 0;
+}
+
+static void ad4130_disable_regulators(void *data)
+{
+ struct ad4130_state *st = data;
+
+ regulator_bulk_disable(ARRAY_SIZE(st->regulators), st->regulators);
+}
+
+static int ad4130_probe(struct spi_device *spi)
+{
+ struct device *dev = &spi->dev;
+ struct iio_dev *indio_dev;
+ struct ad4130_state *st;
+ int ret;
+
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*st));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ st = iio_priv(indio_dev);
+
+ memset(st->reset_buf, 0xff, sizeof(st->reset_buf));
+ init_completion(&st->completion);
+ mutex_init(&st->lock);
+ st->spi = spi;
+
+ /*
+ * Xfer: [ XFR1 ] [ XFR2 ]
+ * Master: 0x7D N ......................
+ * Slave: ...... DATA1 DATA2 ... DATAN
+ */
+ st->fifo_tx_buf[0] = AD4130_COMMS_READ_MASK | AD4130_FIFO_DATA_REG;
+ st->fifo_xfer[0].tx_buf = st->fifo_tx_buf;
+ st->fifo_xfer[0].len = sizeof(st->fifo_tx_buf);
+ st->fifo_xfer[1].rx_buf = st->fifo_rx_buf;
+ spi_message_init_with_transfers(&st->fifo_msg, st->fifo_xfer,
+ ARRAY_SIZE(st->fifo_xfer));
+
+ indio_dev->name = AD4130_NAME;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+ indio_dev->info = &ad4130_info;
+
+ st->regmap = devm_regmap_init(dev, NULL, st, &ad4130_regmap_config);
+ if (IS_ERR(st->regmap))
+ return PTR_ERR(st->regmap);
+
+ st->regulators[0].supply = "avdd";
+ st->regulators[1].supply = "iovdd";
+ st->regulators[2].supply = "refin1";
+ st->regulators[3].supply = "refin2";
+
+ ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(st->regulators),
+ st->regulators);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to get regulators\n");
+
+ ret = regulator_bulk_enable(ARRAY_SIZE(st->regulators), st->regulators);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to enable regulators\n");
+
+ ret = devm_add_action_or_reset(dev, ad4130_disable_regulators, st);
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "Failed to add regulators disable action\n");
+
+ ret = ad4130_soft_reset(st);
+ if (ret)
+ return ret;
+
+ ret = ad4310_parse_fw(indio_dev);
+ if (ret)
+ return ret;
+
+ ret = ad4130_setup(indio_dev);
+ if (ret)
+ return ret;
+
+ ret = ad4130_setup_int_clk(st);
+ if (ret)
+ return ret;
+
+ ad4130_fill_scale_tbls(st);
+
+ st->gc.owner = THIS_MODULE;
+ st->gc.label = AD4130_NAME;
+ st->gc.base = -1;
+ st->gc.ngpio = AD4130_MAX_GPIOS;
+ st->gc.parent = dev;
+ st->gc.can_sleep = true;
+ st->gc.init_valid_mask = ad4130_gpio_init_valid_mask;
+ st->gc.get_direction = ad4130_gpio_get_direction;
+ st->gc.set = ad4130_gpio_set;
+
+ ret = devm_gpiochip_add_data(dev, &st->gc, st);
+ if (ret)
+ return ret;
+
+ ret = devm_iio_kfifo_buffer_setup_ext(dev, indio_dev,
+ &ad4130_buffer_ops,
+ ad4130_fifo_attributes);
+ if (ret)
+ return ret;
+
+ ret = devm_request_threaded_irq(dev, spi->irq, NULL,
+ ad4130_irq_handler, IRQF_ONESHOT,
+ indio_dev->name, indio_dev);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to request irq\n");
+
+ /*
+ * When the chip enters FIFO mode, IRQ polarity is inverted.
+ * When the chip exits FIFO mode, IRQ polarity returns to normal.
+ * See datasheet pages: 65, FIFO Watermark Interrupt section,
+ * and 71, Bit Descriptions for STATUS Register, RDYB.
+ * Cache the normal and inverted IRQ triggers to set them when
+ * entering and exiting FIFO mode.
+ */
+ st->irq_trigger = irq_get_trigger_type(spi->irq);
+ if (st->irq_trigger & IRQF_TRIGGER_RISING)
+ st->inv_irq_trigger = IRQF_TRIGGER_FALLING;
+ else if (st->irq_trigger & IRQF_TRIGGER_FALLING)
+ st->inv_irq_trigger = IRQF_TRIGGER_RISING;
+ else
+ return dev_err_probe(dev, -EINVAL, "Invalid irq flags: %u\n",
+ st->irq_trigger);
+
+ return devm_iio_device_register(dev, indio_dev);
+}
+
+static const struct of_device_id ad4130_of_match[] = {
+ {
+ .compatible = "adi,ad4130",
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, ad4130_of_match);
+
+static struct spi_driver ad4130_driver = {
+ .driver = {
+ .name = AD4130_NAME,
+ .of_match_table = ad4130_of_match,
+ },
+ .probe = ad4130_probe,
+};
+module_spi_driver(ad4130_driver);
+
+MODULE_AUTHOR("Cosmin Tanislav <cosmin.tanislav@analog.com>");
+MODULE_DESCRIPTION("Analog Devices AD4130 SPI driver");
+MODULE_LICENSE("GPL");
.vref_mV = 2500,
};
-static int ad7091r5_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int ad7091r5_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
const struct ad7091r_chip_info *chip_info;
struct regmap *map = devm_regmap_init_i2c(i2c, &ad7091r_regmap_config);
.name = "ad7091r5",
.of_match_table = ad7091r5_dt_ids,
},
- .probe = ad7091r5_i2c_probe,
+ .probe_new = ad7091r5_i2c_probe,
.id_table = ad7091r5_i2c_ids,
};
module_i2c_driver(ad7091r5_driver);
info = of_device_get_match_data(&spi->dev);
if (!info)
+ info = (void *)spi_get_device_id(spi)->driver_data;
+ if (!info)
return -ENODEV;
indio_dev = devm_iio_device_alloc(&spi->dev, sizeof(*st));
};
MODULE_DEVICE_TABLE(of, ad7124_of_match);
+static const struct spi_device_id ad71124_ids[] = {
+ { "ad7124-4", (kernel_ulong_t)&ad7124_chip_info_tbl[ID_AD7124_4] },
+ { "ad7124-8", (kernel_ulong_t)&ad7124_chip_info_tbl[ID_AD7124_8] },
+ {}
+};
+MODULE_DEVICE_TABLE(spi, ad71124_ids);
+
static struct spi_driver ad71124_driver = {
.driver = {
.name = "ad7124",
.of_match_table = ad7124_of_match,
},
.probe = ad7124_probe,
+ .id_table = ad71124_ids,
};
module_spi_driver(ad71124_driver);
struct ad7192_state {
const struct ad7192_chip_info *chip_info;
struct regulator *avdd;
- struct regulator *dvdd;
struct clk *mclk;
u16 int_vref_mv;
u32 fclk;
if (ret)
return ret;
- st->dvdd = devm_regulator_get(&spi->dev, "dvdd");
- if (IS_ERR(st->dvdd))
- return PTR_ERR(st->dvdd);
-
- ret = regulator_enable(st->dvdd);
- if (ret) {
- dev_err(&spi->dev, "Failed to enable specified DVdd supply\n");
- return ret;
- }
-
- ret = devm_add_action_or_reset(&spi->dev, ad7192_reg_disable, st->dvdd);
+ ret = devm_regulator_get_enable(&spi->dev, "dvdd");
if (ret)
- return ret;
+ return dev_err_probe(&spi->dev, ret, "Failed to enable specified DVdd supply\n");
ret = regulator_get_voltage(st->avdd);
if (ret < 0) {
st->int_vref_mv = ret / 1000;
st->chip_info = of_device_get_match_data(&spi->dev);
+ if (!st->chip_info)
+ st->chip_info = (void *)spi_get_device_id(spi)->driver_data;
indio_dev->name = st->chip_info->name;
indio_dev->modes = INDIO_DIRECT_MODE;
};
MODULE_DEVICE_TABLE(of, ad7192_of_match);
+static const struct spi_device_id ad7192_ids[] = {
+ { "ad7190", (kernel_ulong_t)&ad7192_chip_info_tbl[ID_AD7190] },
+ { "ad7192", (kernel_ulong_t)&ad7192_chip_info_tbl[ID_AD7192] },
+ { "ad7193", (kernel_ulong_t)&ad7192_chip_info_tbl[ID_AD7193] },
+ { "ad7195", (kernel_ulong_t)&ad7192_chip_info_tbl[ID_AD7195] },
+ {}
+};
+MODULE_DEVICE_TABLE(spi, ad7192_ids);
+
static struct spi_driver ad7192_driver = {
.driver = {
.name = "ad7192",
.of_match_table = ad7192_of_match,
},
.probe = ad7192_probe,
+ .id_table = ad7192_ids,
};
module_spi_driver(ad7192_driver);
regulator_disable(reg);
}
-static int ad7291_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ad7291_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct ad7291_chip_info *chip;
struct iio_dev *indio_dev;
int ret;
.name = KBUILD_MODNAME,
.of_match_table = ad7291_of_match,
},
- .probe = ad7291_probe,
+ .probe_new = ad7291_probe,
.id_table = ad7291_id,
};
module_i2c_driver(ad7291_driver);
}
if (st->chip_info->has_vdrive) {
- reg = devm_regulator_get(&spi->dev, "vdrive");
- if (IS_ERR(reg))
- return PTR_ERR(reg);
-
- ret = regulator_enable(reg);
- if (ret)
- return ret;
-
- ret = devm_add_action_or_reset(&spi->dev, ad7476_reg_disable,
- reg);
+ ret = devm_regulator_get_enable(&spi->dev, "vdrive");
if (ret)
return ret;
}
.validate_device = iio_trigger_validate_own_device,
};
-static void ad7606_regulator_disable(void *data)
-{
- struct ad7606_state *st = data;
-
- regulator_disable(st->reg);
-}
-
int ad7606_probe(struct device *dev, int irq, void __iomem *base_address,
const char *name, unsigned int id,
const struct ad7606_bus_ops *bops)
st->scale_avail = ad7606_scale_avail;
st->num_scales = ARRAY_SIZE(ad7606_scale_avail);
- st->reg = devm_regulator_get(dev, "avcc");
- if (IS_ERR(st->reg))
- return PTR_ERR(st->reg);
-
- ret = regulator_enable(st->reg);
- if (ret) {
- dev_err(dev, "Failed to enable specified AVcc supply\n");
- return ret;
- }
-
- ret = devm_add_action_or_reset(dev, ad7606_regulator_disable, st);
+ ret = devm_regulator_get_enable(dev, "avcc");
if (ret)
- return ret;
+ return dev_err_probe(dev, ret,
+ "Failed to enable specified AVcc supply\n");
st->chip_info = &ad7606_chip_info_tbl[id];
* struct ad7606_state - driver instance specific data
* @dev pointer to kernel device
* @chip_info entry in the table of chips that describes this device
- * @reg regulator info for the power supply of the device
* @bops bus operations (SPI or parallel)
* @range voltage range selection, selects which scale to apply
* @oversampling oversampling selection
struct ad7606_state {
struct device *dev;
const struct ad7606_chip_info *chip_info;
- struct regulator *reg;
const struct ad7606_bus_ops *bops;
unsigned int range[16];
unsigned int oversampling;
if (irq < 0)
return irq;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- addr = devm_ioremap_resource(&pdev->dev, res);
+ addr = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
if (IS_ERR(addr))
return PTR_ERR(addr);
#include <linux/types.h>
#include <linux/err.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/bitops.h>
#include <linux/iio/iio.h>
const struct ad799x_chip_config *chip_config;
struct regulator *reg;
struct regulator *vref;
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
unsigned id;
u16 config;
ret = iio_device_claim_direct_mode(indio_dev);
if (ret)
return ret;
+ mutex_lock(&st->lock);
ret = ad799x_scan_direct(st, chan->scan_index);
+ mutex_unlock(&st->lock);
iio_device_release_direct_mode(indio_dev);
if (ret < 0)
if (ret)
return ret;
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&st->lock);
+
ret = i2c_smbus_read_byte_data(st->client, AD7998_CYCLE_TMR_REG);
if (ret < 0)
goto error_ret_mutex;
ret = len;
error_ret_mutex:
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&st->lock);
return ret;
}
if (ret)
return ret;
+ mutex_lock(&st->lock);
+
if (state)
st->config |= BIT(chan->scan_index) << AD799X_CHANNEL_SHIFT;
else
st->config &= ~AD7998_ALERT_EN;
ret = ad799x_write_config(st, st->config);
+ mutex_unlock(&st->lock);
iio_device_release_direct_mode(indio_dev);
return ret;
}
if (val < 0 || val > GENMASK(chan->scan_type.realbits - 1, 0))
return -EINVAL;
- mutex_lock(&indio_dev->mlock);
ret = i2c_smbus_write_word_swapped(st->client,
ad799x_threshold_reg(chan, dir, info),
val << chan->scan_type.shift);
- mutex_unlock(&indio_dev->mlock);
return ret;
}
int ret;
struct ad799x_state *st = iio_priv(indio_dev);
- mutex_lock(&indio_dev->mlock);
ret = i2c_smbus_read_word_swapped(st->client,
ad799x_threshold_reg(chan, dir, info));
- mutex_unlock(&indio_dev->mlock);
if (ret < 0)
return ret;
*val = (ret >> chan->scan_type.shift) &
},
};
-static int ad799x_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ad799x_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
int ret;
int extra_config = 0;
struct ad799x_state *st;
if (ret)
goto error_cleanup_ring;
}
+
+ mutex_init(&st->lock);
+
ret = iio_device_register(indio_dev);
if (ret)
goto error_cleanup_ring;
.name = "ad799x",
.pm = pm_sleep_ptr(&ad799x_pm_ops),
},
- .probe = ad799x_probe,
+ .probe_new = ad799x_probe,
.remove = ad799x_remove,
.id_table = ad799x_id,
};
info = of_device_get_match_data(&spi->dev);
if (!info)
+ info = (void *)spi_get_device_id(spi)->driver_data;
+ if (!info)
return -ENODEV;
conv = devm_adi_axi_adc_conv_register(&spi->dev, sizeof(*st));
};
MODULE_DEVICE_TABLE(of, ad9467_of_match);
+static const struct spi_device_id ad9467_ids[] = {
+ { "ad9265", (kernel_ulong_t)&ad9467_chip_tbl[ID_AD9265] },
+ { "ad9434", (kernel_ulong_t)&ad9467_chip_tbl[ID_AD9434] },
+ { "ad9467", (kernel_ulong_t)&ad9467_chip_tbl[ID_AD9467] },
+ {}
+};
+MODULE_DEVICE_TABLE(spi, ad9467_ids);
+
static struct spi_driver ad9467_driver = {
.driver = {
.name = "ad9467",
.of_match_table = ad9467_of_match,
},
.probe = ad9467_probe,
+ .id_table = ad9467_ids,
};
module_spi_driver(ad9467_driver);
unsigned int data_reg;
int ret = 0;
- if (iio_buffer_enabled(indio_dev))
- return -EBUSY;
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
- mutex_lock(&indio_dev->mlock);
ad_sigma_delta_set_channel(sigma_delta, chan->address);
spi_bus_lock(sigma_delta->spi->master);
ad_sigma_delta_set_mode(sigma_delta, AD_SD_MODE_IDLE);
sigma_delta->bus_locked = false;
spi_bus_unlock(sigma_delta->spi->master);
- mutex_unlock(&indio_dev->mlock);
+ iio_device_release_direct_mode(indio_dev);
if (ret)
return ret;
return scnprintf(buf, PAGE_SIZE, "%d\n", st->dma_st.watermark);
}
-static ssize_t hwfifo_watermark_min_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- return sysfs_emit(buf, "%s\n", "2");
-}
-
-static ssize_t hwfifo_watermark_max_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- return sysfs_emit(buf, "%s\n", AT91_HWFIFO_MAX_SIZE_STR);
-}
-
static IIO_DEVICE_ATTR(hwfifo_enabled, 0444,
at91_adc_get_fifo_state, NULL, 0);
static IIO_DEVICE_ATTR(hwfifo_watermark, 0444,
at91_adc_get_watermark, NULL, 0);
-static IIO_DEVICE_ATTR_RO(hwfifo_watermark_min, 0);
-static IIO_DEVICE_ATTR_RO(hwfifo_watermark_max, 0);
-
-static const struct attribute *at91_adc_fifo_attributes[] = {
- &iio_dev_attr_hwfifo_watermark_min.dev_attr.attr,
- &iio_dev_attr_hwfifo_watermark_max.dev_attr.attr,
- &iio_dev_attr_hwfifo_watermark.dev_attr.attr,
- &iio_dev_attr_hwfifo_enabled.dev_attr.attr,
+
+IIO_STATIC_CONST_DEVICE_ATTR(hwfifo_watermark_min, "2");
+IIO_STATIC_CONST_DEVICE_ATTR(hwfifo_watermark_max, AT91_HWFIFO_MAX_SIZE_STR);
+
+static const struct iio_dev_attr *at91_adc_fifo_attributes[] = {
+ &iio_dev_attr_hwfifo_watermark_min,
+ &iio_dev_attr_hwfifo_watermark_max,
+ &iio_dev_attr_hwfifo_watermark,
+ &iio_dev_attr_hwfifo_enabled,
NULL,
};
struct iio_dev *indio)
{
struct at91_adc_state *st = iio_priv(indio);
- const struct attribute **fifo_attrs;
+ const struct iio_dev_attr **fifo_attrs;
int ret;
if (st->selected_trig->hw_trig)
#include <linux/dmi.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/kernel.h>
#include <linux/device.h>
#include <linux/regmap.h>
struct axp288_adc_info {
int irq;
struct regmap *regmap;
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
bool ts_enabled;
};
int ret;
struct axp288_adc_info *info = iio_priv(indio_dev);
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&info->lock);
switch (mask) {
case IIO_CHAN_INFO_RAW:
if (axp288_adc_set_ts(info, AXP288_ADC_TS_CURRENT_ON_ONDEMAND,
default:
ret = -EINVAL;
}
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&info->lock);
return ret;
}
if (ret < 0)
return ret;
+ mutex_init(&info->lock);
+
return devm_iio_device_register(&pdev->dev, indio_dev);
}
return 0;
}
+static void cc10001_reg_disable(void *priv)
+{
+ regulator_disable(priv);
+}
+
+static void cc10001_pd_cb(void *priv)
+{
+ cc10001_adc_power_down(priv);
+}
+
static int cc10001_adc_probe(struct platform_device *pdev)
{
- struct device_node *node = pdev->dev.of_node;
+ struct device *dev = &pdev->dev;
+ struct device_node *node = dev->of_node;
struct cc10001_adc_device *adc_dev;
unsigned long adc_clk_rate;
struct iio_dev *indio_dev;
unsigned long channel_map;
int ret;
- indio_dev = devm_iio_device_alloc(&pdev->dev, sizeof(*adc_dev));
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*adc_dev));
if (indio_dev == NULL)
return -ENOMEM;
channel_map &= ~ret;
}
- adc_dev->reg = devm_regulator_get(&pdev->dev, "vref");
+ adc_dev->reg = devm_regulator_get(dev, "vref");
if (IS_ERR(adc_dev->reg))
return PTR_ERR(adc_dev->reg);
if (ret)
return ret;
- indio_dev->name = dev_name(&pdev->dev);
+ ret = devm_add_action_or_reset(dev, cc10001_reg_disable, adc_dev->reg);
+ if (ret)
+ return ret;
+
+ indio_dev->name = dev_name(dev);
indio_dev->info = &cc10001_adc_info;
indio_dev->modes = INDIO_DIRECT_MODE;
adc_dev->reg_base = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(adc_dev->reg_base)) {
- ret = PTR_ERR(adc_dev->reg_base);
- goto err_disable_reg;
- }
+ if (IS_ERR(adc_dev->reg_base))
+ return PTR_ERR(adc_dev->reg_base);
- adc_dev->adc_clk = devm_clk_get(&pdev->dev, "adc");
+ adc_dev->adc_clk = devm_clk_get_enabled(dev, "adc");
if (IS_ERR(adc_dev->adc_clk)) {
- dev_err(&pdev->dev, "failed to get the clock\n");
- ret = PTR_ERR(adc_dev->adc_clk);
- goto err_disable_reg;
- }
-
- ret = clk_prepare_enable(adc_dev->adc_clk);
- if (ret) {
- dev_err(&pdev->dev, "failed to enable the clock\n");
- goto err_disable_reg;
+ dev_err(dev, "failed to get/enable the clock\n");
+ return PTR_ERR(adc_dev->adc_clk);
}
adc_clk_rate = clk_get_rate(adc_dev->adc_clk);
if (!adc_clk_rate) {
- ret = -EINVAL;
- dev_err(&pdev->dev, "null clock rate!\n");
- goto err_disable_clk;
+ dev_err(dev, "null clock rate!\n");
+ return -EINVAL;
}
adc_dev->eoc_delay_ns = NSEC_PER_SEC / adc_clk_rate;
if (adc_dev->shared)
cc10001_adc_power_up(adc_dev);
+ ret = devm_add_action_or_reset(dev, cc10001_pd_cb, adc_dev);
+ if (ret)
+ return ret;
/* Setup the ADC channels available on the device */
ret = cc10001_adc_channel_init(indio_dev, channel_map);
if (ret < 0)
- goto err_disable_clk;
+ return ret;
mutex_init(&adc_dev->lock);
- ret = iio_triggered_buffer_setup(indio_dev, NULL,
- &cc10001_adc_trigger_h, NULL);
- if (ret < 0)
- goto err_disable_clk;
-
- ret = iio_device_register(indio_dev);
+ ret = devm_iio_triggered_buffer_setup(dev, indio_dev, NULL,
+ &cc10001_adc_trigger_h, NULL);
if (ret < 0)
- goto err_cleanup_buffer;
-
- platform_set_drvdata(pdev, indio_dev);
-
- return 0;
-
-err_cleanup_buffer:
- iio_triggered_buffer_cleanup(indio_dev);
-err_disable_clk:
- clk_disable_unprepare(adc_dev->adc_clk);
-err_disable_reg:
- regulator_disable(adc_dev->reg);
- return ret;
-}
-
-static int cc10001_adc_remove(struct platform_device *pdev)
-{
- struct iio_dev *indio_dev = platform_get_drvdata(pdev);
- struct cc10001_adc_device *adc_dev = iio_priv(indio_dev);
-
- cc10001_adc_power_down(adc_dev);
- iio_device_unregister(indio_dev);
- iio_triggered_buffer_cleanup(indio_dev);
- clk_disable_unprepare(adc_dev->adc_clk);
- regulator_disable(adc_dev->reg);
+ return ret;
- return 0;
+ return devm_iio_device_register(dev, indio_dev);
}
static const struct of_device_id cc10001_adc_dt_ids[] = {
.of_match_table = cc10001_adc_dt_ids,
},
.probe = cc10001_adc_probe,
- .remove = cc10001_adc_remove,
};
module_platform_driver(cc10001_adc_driver);
#include <linux/kernel.h>
#include <linux/mod_devicetable.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
struct device *dev;
void __iomem *regs;
struct clk *clk;
-
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
u32 vref_uv;
u32 value;
u32 channel;
switch (mask) {
case IIO_CHAN_INFO_RAW:
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&info->lock);
reinit_completion(&info->completion);
channel = chan->channel & 0x03;
ret = wait_for_completion_interruptible_timeout
(&info->completion, IMX7D_ADC_TIMEOUT);
if (ret == 0) {
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&info->lock);
return -ETIMEDOUT;
}
if (ret < 0) {
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&info->lock);
return ret;
}
*val = info->value;
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&info->lock);
return IIO_VAL_INT;
case IIO_CHAN_INFO_SCALE:
if (ret)
return ret;
+ mutex_init(&info->lock);
+
ret = devm_iio_device_register(dev, indio_dev);
if (ret) {
dev_err(&pdev->dev, "Couldn't register the device.\n");
return ina2xx_set_calibration(chip);
}
-static int ina2xx_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ina2xx_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct ina2xx_chip_info *chip;
struct iio_dev *indio_dev;
unsigned int val;
.name = KBUILD_MODNAME,
.of_match_table = ina2xx_of_match,
},
- .probe = ina2xx_probe,
+ .probe_new = ina2xx_probe,
.remove = ina2xx_remove,
.id_table = ina2xx_id,
};
#include <linux/io.h>
#include <linux/module.h>
#include <linux/mod_devicetable.h>
+#include <linux/mutex.h>
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
struct clk *clk;
struct completion completion;
struct regulator *vref;
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
u32 value;
};
switch (mask) {
case IIO_CHAN_INFO_RAW:
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&st->lock);
ret = clk_prepare_enable(st->clk);
if (ret) {
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&st->lock);
return ret;
}
/* Measurement setup */
wait_for_completion(&st->completion); /* set by ISR */
clk_disable_unprepare(st->clk);
*val = st->value;
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&st->lock);
return IIO_VAL_INT;
iodev->modes = INDIO_DIRECT_MODE;
iodev->num_channels = ARRAY_SIZE(lpc32xx_adc_iio_channels);
+ mutex_init(&st->lock);
+
retval = devm_iio_device_register(&pdev->dev, iodev);
if (retval)
return retval;
.read_raw = ltc2471_read_raw,
};
-static int ltc2471_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ltc2471_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct iio_dev *indio_dev;
struct ltc2471_data *data;
int ret;
.driver = {
.name = "ltc2471",
},
- .probe = ltc2471_i2c_probe,
+ .probe_new = ltc2471_i2c_probe,
.id_table = ltc2471_i2c_id,
};
.read_raw = ltc2485_read_raw,
};
-static int ltc2485_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ltc2485_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct iio_dev *indio_dev;
struct ltc2485_data *data;
int ret;
.driver = {
.name = "ltc2485",
},
- .probe = ltc2485_probe,
+ .probe_new = ltc2485_probe,
.id_table = ltc2485_id,
};
module_i2c_driver(ltc2485_driver);
#include <linux/iio/iio.h>
#include <linux/iio/driver.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/regulator/consumer.h>
#include "ltc2497.h"
switch (mask) {
case IIO_CHAN_INFO_RAW:
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&ddata->lock);
ret = ltc2497core_read(ddata, chan->address, val);
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&ddata->lock);
if (ret < 0)
return ret;
ddata->addr_prev = LTC2497_CONFIG_DEFAULT;
ddata->time_prev = ktime_get();
+ mutex_init(&ddata->lock);
+
ret = iio_device_register(indio_dev);
if (ret < 0)
goto err_array_unregister;
return ret;
}
-static int ltc2497_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ltc2497_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
const struct ltc2497_chip_info *chip_info;
struct iio_dev *indio_dev;
struct ltc2497_driverdata *st;
.name = "ltc2497",
.of_match_table = ltc2497_of_match,
},
- .probe = ltc2497_probe,
+ .probe_new = ltc2497_probe,
.remove = ltc2497_remove,
.id_table = ltc2497_id,
};
struct ltc2497core_driverdata {
struct regulator *ref;
ktime_t time_prev;
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
const struct ltc2497_chip_info *chip_info;
u8 addr_prev;
int (*result_and_measure)(struct ltc2497core_driverdata *ddata,
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * MAX11410 SPI ADC driver
+ *
+ * Copyright 2022 Analog Devices Inc.
+ */
+#include <asm-generic/unaligned.h>
+#include <linux/bitfield.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/regmap.h>
+#include <linux/regulator/consumer.h>
+#include <linux/spi/spi.h>
+
+#include <linux/iio/buffer.h>
+#include <linux/iio/sysfs.h>
+#include <linux/iio/trigger.h>
+#include <linux/iio/trigger_consumer.h>
+#include <linux/iio/triggered_buffer.h>
+
+#define MAX11410_REG_CONV_START 0x01
+#define MAX11410_CONV_TYPE_SINGLE 0x00
+#define MAX11410_CONV_TYPE_CONTINUOUS 0x01
+#define MAX11410_REG_CAL_START 0x03
+#define MAX11410_CAL_START_SELF 0x00
+#define MAX11410_CAL_START_PGA 0x01
+#define MAX11410_REG_GPIO_CTRL(ch) ((ch) ? 0x05 : 0x04)
+#define MAX11410_GPIO_INTRB 0xC1
+#define MAX11410_REG_FILTER 0x08
+#define MAX11410_FILTER_RATE_MASK GENMASK(3, 0)
+#define MAX11410_FILTER_RATE_MAX 0x0F
+#define MAX11410_FILTER_LINEF_MASK GENMASK(5, 4)
+#define MAX11410_FILTER_50HZ BIT(5)
+#define MAX11410_FILTER_60HZ BIT(4)
+#define MAX11410_REG_CTRL 0x09
+#define MAX11410_CTRL_REFSEL_MASK GENMASK(2, 0)
+#define MAX11410_CTRL_VREFN_BUF_BIT BIT(3)
+#define MAX11410_CTRL_VREFP_BUF_BIT BIT(4)
+#define MAX11410_CTRL_FORMAT_BIT BIT(5)
+#define MAX11410_CTRL_UNIPOLAR_BIT BIT(6)
+#define MAX11410_REG_MUX_CTRL0 0x0B
+#define MAX11410_REG_PGA 0x0E
+#define MAX11410_PGA_GAIN_MASK GENMASK(2, 0)
+#define MAX11410_PGA_SIG_PATH_MASK GENMASK(5, 4)
+#define MAX11410_PGA_SIG_PATH_BUFFERED 0x00
+#define MAX11410_PGA_SIG_PATH_BYPASS 0x01
+#define MAX11410_PGA_SIG_PATH_PGA 0x02
+#define MAX11410_REG_DATA0 0x30
+#define MAX11410_REG_STATUS 0x38
+#define MAX11410_STATUS_CONV_READY_BIT BIT(0)
+#define MAX11410_STATUS_CAL_READY_BIT BIT(2)
+
+#define MAX11410_REFSEL_AVDD_AGND 0x03
+#define MAX11410_REFSEL_MAX 0x06
+#define MAX11410_SIG_PATH_MAX 0x02
+#define MAX11410_CHANNEL_INDEX_MAX 0x0A
+#define MAX11410_AINP_AVDD 0x0A
+#define MAX11410_AINN_GND 0x0A
+
+#define MAX11410_CONVERSION_TIMEOUT_MS 2000
+#define MAX11410_CALIB_TIMEOUT_MS 2000
+
+#define MAX11410_SCALE_AVAIL_SIZE 8
+
+enum max11410_filter {
+ MAX11410_FILTER_FIR5060,
+ MAX11410_FILTER_FIR50,
+ MAX11410_FILTER_FIR60,
+ MAX11410_FILTER_SINC4,
+};
+
+static const u8 max11410_sampling_len[] = {
+ [MAX11410_FILTER_FIR5060] = 5,
+ [MAX11410_FILTER_FIR50] = 6,
+ [MAX11410_FILTER_FIR60] = 6,
+ [MAX11410_FILTER_SINC4] = 10,
+};
+
+static const int max11410_sampling_rates[4][10][2] = {
+ [MAX11410_FILTER_FIR5060] = {
+ { 1, 100000 },
+ { 2, 100000 },
+ { 4, 200000 },
+ { 8, 400000 },
+ { 16, 800000 }
+ },
+ [MAX11410_FILTER_FIR50] = {
+ { 1, 300000 },
+ { 2, 700000 },
+ { 5, 300000 },
+ { 10, 700000 },
+ { 21, 300000 },
+ { 40 }
+ },
+ [MAX11410_FILTER_FIR60] = {
+ { 1, 300000 },
+ { 2, 700000 },
+ { 5, 300000 },
+ { 10, 700000 },
+ { 21, 300000 },
+ { 40 }
+ },
+ [MAX11410_FILTER_SINC4] = {
+ { 4 },
+ { 10 },
+ { 20 },
+ { 40 },
+ { 60 },
+ { 120 },
+ { 240 },
+ { 480 },
+ { 960 },
+ { 1920 }
+ }
+};
+
+struct max11410_channel_config {
+ u32 settling_time_us;
+ u32 *scale_avail;
+ u8 refsel;
+ u8 sig_path;
+ u8 gain;
+ bool bipolar;
+ bool buffered_vrefp;
+ bool buffered_vrefn;
+};
+
+struct max11410_state {
+ struct spi_device *spi_dev;
+ struct iio_trigger *trig;
+ struct completion completion;
+ struct mutex lock; /* Prevent changing channel config during sampling */
+ struct regmap *regmap;
+ struct regulator *avdd;
+ struct regulator *vrefp[3];
+ struct regulator *vrefn[3];
+ struct max11410_channel_config *channels;
+ int irq;
+ struct {
+ u32 data __aligned(IIO_DMA_MINALIGN);
+ s64 ts __aligned(8);
+ } scan;
+};
+
+static const struct iio_chan_spec chanspec_template = {
+ .type = IIO_VOLTAGE,
+ .indexed = 1,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_SCALE) |
+ BIT(IIO_CHAN_INFO_OFFSET),
+ .info_mask_shared_by_all = BIT(IIO_CHAN_INFO_SAMP_FREQ),
+ .info_mask_shared_by_all_available = BIT(IIO_CHAN_INFO_SAMP_FREQ),
+ .scan_type = {
+ .sign = 's',
+ .realbits = 24,
+ .storagebits = 32,
+ .endianness = IIO_LE,
+ },
+};
+
+static unsigned int max11410_reg_size(unsigned int reg)
+{
+ /* Registers from 0x00 to 0x10 are 1 byte, the rest are 3 bytes long. */
+ return reg <= 0x10 ? 1 : 3;
+}
+
+static int max11410_write_reg(struct max11410_state *st, unsigned int reg,
+ unsigned int val)
+{
+ /* This driver only needs to write 8-bit registers */
+ if (max11410_reg_size(reg) != 1)
+ return -EINVAL;
+
+ return regmap_write(st->regmap, reg, val);
+}
+
+static int max11410_read_reg(struct max11410_state *st, unsigned int reg,
+ int *val)
+{
+ int ret;
+
+ if (max11410_reg_size(reg) == 3) {
+ ret = regmap_bulk_read(st->regmap, reg, &st->scan.data, 3);
+ if (ret)
+ return ret;
+
+ *val = get_unaligned_be24(&st->scan.data);
+ return 0;
+ }
+
+ return regmap_read(st->regmap, reg, val);
+}
+
+static struct regulator *max11410_get_vrefp(struct max11410_state *st,
+ u8 refsel)
+{
+ refsel = refsel % 4;
+ if (refsel == 3)
+ return st->avdd;
+
+ return st->vrefp[refsel];
+}
+
+static struct regulator *max11410_get_vrefn(struct max11410_state *st,
+ u8 refsel)
+{
+ if (refsel > 2)
+ return NULL;
+
+ return st->vrefn[refsel];
+}
+
+static const struct regmap_config regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+ .max_register = 0x39,
+};
+
+static ssize_t max11410_notch_en_show(struct device *dev,
+ struct device_attribute *devattr,
+ char *buf)
+{
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
+ struct max11410_state *state = iio_priv(indio_dev);
+ struct iio_dev_attr *iio_attr = to_iio_dev_attr(devattr);
+ unsigned int val;
+ int ret;
+
+ ret = max11410_read_reg(state, MAX11410_REG_FILTER, &val);
+ if (ret)
+ return ret;
+
+ switch (iio_attr->address) {
+ case 0:
+ val = !FIELD_GET(MAX11410_FILTER_50HZ, val);
+ break;
+ case 1:
+ val = !FIELD_GET(MAX11410_FILTER_60HZ, val);
+ break;
+ case 2:
+ val = FIELD_GET(MAX11410_FILTER_LINEF_MASK, val) == 3;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return sysfs_emit(buf, "%d\n", val);
+}
+
+static ssize_t max11410_notch_en_store(struct device *dev,
+ struct device_attribute *devattr,
+ const char *buf, size_t count)
+{
+ struct iio_dev_attr *iio_attr = to_iio_dev_attr(devattr);
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
+ struct max11410_state *state = iio_priv(indio_dev);
+ unsigned int filter_bits;
+ bool enable;
+ int ret;
+
+ ret = kstrtobool(buf, &enable);
+ if (ret)
+ return ret;
+
+ switch (iio_attr->address) {
+ case 0:
+ filter_bits = MAX11410_FILTER_50HZ;
+ break;
+ case 1:
+ filter_bits = MAX11410_FILTER_60HZ;
+ break;
+ case 2:
+ default:
+ filter_bits = MAX11410_FILTER_50HZ | MAX11410_FILTER_60HZ;
+ enable = !enable;
+ break;
+ }
+
+ if (enable)
+ ret = regmap_clear_bits(state->regmap, MAX11410_REG_FILTER,
+ filter_bits);
+ else
+ ret = regmap_set_bits(state->regmap, MAX11410_REG_FILTER,
+ filter_bits);
+
+ if (ret)
+ return ret;
+
+ return count;
+}
+
+static ssize_t in_voltage_filter2_notch_center_show(struct device *dev,
+ struct device_attribute *devattr,
+ char *buf)
+{
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
+ struct max11410_state *state = iio_priv(indio_dev);
+ int ret, reg, rate, filter;
+
+ ret = regmap_read(state->regmap, MAX11410_REG_FILTER, ®);
+ if (ret)
+ return ret;
+
+ rate = FIELD_GET(MAX11410_FILTER_RATE_MASK, reg);
+ rate = clamp_val(rate, 0,
+ max11410_sampling_len[MAX11410_FILTER_SINC4] - 1);
+ filter = max11410_sampling_rates[MAX11410_FILTER_SINC4][rate][0];
+
+ return sysfs_emit(buf, "%d\n", filter);
+}
+
+static IIO_CONST_ATTR(in_voltage_filter0_notch_center, "50");
+static IIO_CONST_ATTR(in_voltage_filter1_notch_center, "60");
+static IIO_DEVICE_ATTR_RO(in_voltage_filter2_notch_center, 2);
+
+static IIO_DEVICE_ATTR(in_voltage_filter0_notch_en, 0644,
+ max11410_notch_en_show, max11410_notch_en_store, 0);
+static IIO_DEVICE_ATTR(in_voltage_filter1_notch_en, 0644,
+ max11410_notch_en_show, max11410_notch_en_store, 1);
+static IIO_DEVICE_ATTR(in_voltage_filter2_notch_en, 0644,
+ max11410_notch_en_show, max11410_notch_en_store, 2);
+
+static struct attribute *max11410_attributes[] = {
+ &iio_const_attr_in_voltage_filter0_notch_center.dev_attr.attr,
+ &iio_const_attr_in_voltage_filter1_notch_center.dev_attr.attr,
+ &iio_dev_attr_in_voltage_filter2_notch_center.dev_attr.attr,
+ &iio_dev_attr_in_voltage_filter0_notch_en.dev_attr.attr,
+ &iio_dev_attr_in_voltage_filter1_notch_en.dev_attr.attr,
+ &iio_dev_attr_in_voltage_filter2_notch_en.dev_attr.attr,
+ NULL
+};
+
+static const struct attribute_group max11410_attribute_group = {
+ .attrs = max11410_attributes,
+};
+
+static int max11410_set_input_mux(struct max11410_state *st, u8 ainp, u8 ainn)
+{
+ if (ainp > MAX11410_CHANNEL_INDEX_MAX ||
+ ainn > MAX11410_CHANNEL_INDEX_MAX)
+ return -EINVAL;
+
+ return max11410_write_reg(st, MAX11410_REG_MUX_CTRL0,
+ (ainp << 4) | ainn);
+}
+
+static int max11410_configure_channel(struct max11410_state *st,
+ struct iio_chan_spec const *chan)
+{
+ struct max11410_channel_config cfg = st->channels[chan->address];
+ unsigned int regval;
+ int ret;
+
+ if (chan->differential)
+ ret = max11410_set_input_mux(st, chan->channel, chan->channel2);
+ else
+ ret = max11410_set_input_mux(st, chan->channel,
+ MAX11410_AINN_GND);
+
+ if (ret)
+ return ret;
+
+ regval = FIELD_PREP(MAX11410_CTRL_VREFP_BUF_BIT, cfg.buffered_vrefp) |
+ FIELD_PREP(MAX11410_CTRL_VREFN_BUF_BIT, cfg.buffered_vrefn) |
+ FIELD_PREP(MAX11410_CTRL_REFSEL_MASK, cfg.refsel) |
+ FIELD_PREP(MAX11410_CTRL_UNIPOLAR_BIT, cfg.bipolar ? 0 : 1);
+ ret = regmap_update_bits(st->regmap, MAX11410_REG_CTRL,
+ MAX11410_CTRL_REFSEL_MASK |
+ MAX11410_CTRL_VREFP_BUF_BIT |
+ MAX11410_CTRL_VREFN_BUF_BIT |
+ MAX11410_CTRL_UNIPOLAR_BIT, regval);
+ if (ret)
+ return ret;
+
+ regval = FIELD_PREP(MAX11410_PGA_SIG_PATH_MASK, cfg.sig_path) |
+ FIELD_PREP(MAX11410_PGA_GAIN_MASK, cfg.gain);
+ ret = regmap_write(st->regmap, MAX11410_REG_PGA, regval);
+ if (ret)
+ return ret;
+
+ if (cfg.settling_time_us)
+ fsleep(cfg.settling_time_us);
+
+ return 0;
+}
+
+static int max11410_sample(struct max11410_state *st, int *sample_raw,
+ struct iio_chan_spec const *chan)
+{
+ int val, ret;
+
+ ret = max11410_configure_channel(st, chan);
+ if (ret)
+ return ret;
+
+ if (st->irq > 0)
+ reinit_completion(&st->completion);
+
+ /* Start Conversion */
+ ret = max11410_write_reg(st, MAX11410_REG_CONV_START,
+ MAX11410_CONV_TYPE_SINGLE);
+ if (ret)
+ return ret;
+
+ if (st->irq > 0) {
+ /* Wait for an interrupt. */
+ ret = wait_for_completion_timeout(&st->completion,
+ msecs_to_jiffies(MAX11410_CONVERSION_TIMEOUT_MS));
+ if (!ret)
+ return -ETIMEDOUT;
+ } else {
+ /* Wait for status register Conversion Ready flag */
+ ret = read_poll_timeout(max11410_read_reg, ret,
+ ret || (val & MAX11410_STATUS_CONV_READY_BIT),
+ 5000, MAX11410_CONVERSION_TIMEOUT_MS * 1000,
+ true, st, MAX11410_REG_STATUS, &val);
+ if (ret)
+ return ret;
+ }
+
+ /* Read ADC Data */
+ return max11410_read_reg(st, MAX11410_REG_DATA0, sample_raw);
+}
+
+static int max11410_get_scale(struct max11410_state *state,
+ struct max11410_channel_config cfg)
+{
+ struct regulator *vrefp, *vrefn;
+ int scale;
+
+ vrefp = max11410_get_vrefp(state, cfg.refsel);
+
+ scale = regulator_get_voltage(vrefp) / 1000;
+ vrefn = max11410_get_vrefn(state, cfg.refsel);
+ if (vrefn)
+ scale -= regulator_get_voltage(vrefn) / 1000;
+
+ if (cfg.bipolar)
+ scale *= 2;
+
+ return scale >> cfg.gain;
+}
+
+static int max11410_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long info)
+{
+ struct max11410_state *state = iio_priv(indio_dev);
+ struct max11410_channel_config cfg = state->channels[chan->address];
+ int ret, reg_val, filter, rate;
+
+ switch (info) {
+ case IIO_CHAN_INFO_SCALE:
+ *val = max11410_get_scale(state, cfg);
+ *val2 = chan->scan_type.realbits;
+ return IIO_VAL_FRACTIONAL_LOG2;
+ case IIO_CHAN_INFO_OFFSET:
+ if (cfg.bipolar)
+ *val = -BIT(chan->scan_type.realbits - 1);
+ else
+ *val = 0;
+
+ return IIO_VAL_INT;
+ case IIO_CHAN_INFO_RAW:
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
+
+ mutex_lock(&state->lock);
+
+ ret = max11410_sample(state, ®_val, chan);
+
+ mutex_unlock(&state->lock);
+
+ iio_device_release_direct_mode(indio_dev);
+
+ if (ret)
+ return ret;
+
+ *val = reg_val;
+
+ return IIO_VAL_INT;
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ ret = regmap_read(state->regmap, MAX11410_REG_FILTER, ®_val);
+ if (ret)
+ return ret;
+
+ filter = FIELD_GET(MAX11410_FILTER_LINEF_MASK, reg_val);
+ rate = reg_val & MAX11410_FILTER_RATE_MASK;
+ if (rate >= max11410_sampling_len[filter])
+ rate = max11410_sampling_len[filter] - 1;
+
+ *val = max11410_sampling_rates[filter][rate][0];
+ *val2 = max11410_sampling_rates[filter][rate][1];
+
+ return IIO_VAL_INT_PLUS_MICRO;
+ }
+ return -EINVAL;
+}
+
+static int max11410_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ struct max11410_state *st = iio_priv(indio_dev);
+ int i, ret, reg_val, filter, gain;
+ u32 *scale_avail;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_SCALE:
+ scale_avail = st->channels[chan->address].scale_avail;
+ if (!scale_avail)
+ return -EOPNOTSUPP;
+
+ /* Accept values in range 0.000001 <= scale < 1.000000 */
+ if (val != 0 || val2 == 0)
+ return -EINVAL;
+
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
+
+ /* Convert from INT_PLUS_MICRO to FRACTIONAL_LOG2 */
+ val2 = val2 * DIV_ROUND_CLOSEST(BIT(24), 1000000);
+ val2 = DIV_ROUND_CLOSEST(scale_avail[0], val2);
+ gain = order_base_2(val2);
+
+ st->channels[chan->address].gain = clamp_val(gain, 0, 7);
+
+ iio_device_release_direct_mode(indio_dev);
+
+ return 0;
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
+
+ mutex_lock(&st->lock);
+
+ ret = regmap_read(st->regmap, MAX11410_REG_FILTER, ®_val);
+ if (ret)
+ goto out;
+
+ filter = FIELD_GET(MAX11410_FILTER_LINEF_MASK, reg_val);
+
+ for (i = 0; i < max11410_sampling_len[filter]; ++i) {
+ if (val == max11410_sampling_rates[filter][i][0] &&
+ val2 == max11410_sampling_rates[filter][i][1])
+ break;
+ }
+ if (i == max11410_sampling_len[filter]) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = regmap_write_bits(st->regmap, MAX11410_REG_FILTER,
+ MAX11410_FILTER_RATE_MASK, i);
+
+out:
+ mutex_unlock(&st->lock);
+ iio_device_release_direct_mode(indio_dev);
+
+ return ret;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int max11410_read_avail(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ const int **vals, int *type, int *length,
+ long info)
+{
+ struct max11410_state *st = iio_priv(indio_dev);
+ struct max11410_channel_config cfg;
+ int ret, reg_val, filter;
+
+ switch (info) {
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ ret = regmap_read(st->regmap, MAX11410_REG_FILTER, ®_val);
+ if (ret)
+ return ret;
+
+ filter = FIELD_GET(MAX11410_FILTER_LINEF_MASK, reg_val);
+
+ *vals = (const int *)max11410_sampling_rates[filter];
+ *length = max11410_sampling_len[filter] * 2;
+ *type = IIO_VAL_INT_PLUS_MICRO;
+
+ return IIO_AVAIL_LIST;
+ case IIO_CHAN_INFO_SCALE:
+ cfg = st->channels[chan->address];
+
+ if (!cfg.scale_avail)
+ return -EINVAL;
+
+ *vals = cfg.scale_avail;
+ *length = MAX11410_SCALE_AVAIL_SIZE * 2;
+ *type = IIO_VAL_FRACTIONAL_LOG2;
+
+ return IIO_AVAIL_LIST;
+ }
+ return -EINVAL;
+}
+
+static const struct iio_info max11410_info = {
+ .read_raw = max11410_read_raw,
+ .write_raw = max11410_write_raw,
+ .read_avail = max11410_read_avail,
+ .attrs = &max11410_attribute_group,
+};
+
+static irqreturn_t max11410_trigger_handler(int irq, void *p)
+{
+ struct iio_poll_func *pf = p;
+ struct iio_dev *indio_dev = pf->indio_dev;
+ struct max11410_state *st = iio_priv(indio_dev);
+ int ret;
+
+ ret = max11410_read_reg(st, MAX11410_REG_DATA0, &st->scan.data);
+ if (ret) {
+ dev_err(&indio_dev->dev, "cannot read data\n");
+ goto out;
+ }
+
+ iio_push_to_buffers_with_timestamp(indio_dev, &st->scan,
+ iio_get_time_ns(indio_dev));
+
+out:
+ iio_trigger_notify_done(indio_dev->trig);
+
+ return IRQ_HANDLED;
+}
+
+static int max11410_buffer_postenable(struct iio_dev *indio_dev)
+{
+ struct max11410_state *st = iio_priv(indio_dev);
+ int scan_ch, ret;
+
+ scan_ch = ffs(*indio_dev->active_scan_mask) - 1;
+
+ ret = max11410_configure_channel(st, &indio_dev->channels[scan_ch]);
+ if (ret)
+ return ret;
+
+ /* Start continuous conversion. */
+ return max11410_write_reg(st, MAX11410_REG_CONV_START,
+ MAX11410_CONV_TYPE_CONTINUOUS);
+}
+
+static int max11410_buffer_predisable(struct iio_dev *indio_dev)
+{
+ struct max11410_state *st = iio_priv(indio_dev);
+
+ /* Stop continuous conversion. */
+ return max11410_write_reg(st, MAX11410_REG_CONV_START,
+ MAX11410_CONV_TYPE_SINGLE);
+}
+
+static const struct iio_buffer_setup_ops max11410_buffer_ops = {
+ .postenable = &max11410_buffer_postenable,
+ .predisable = &max11410_buffer_predisable,
+ .validate_scan_mask = &iio_validate_scan_mask_onehot,
+};
+
+static const struct iio_trigger_ops max11410_trigger_ops = {
+ .validate_device = iio_trigger_validate_own_device,
+};
+
+static irqreturn_t max11410_interrupt(int irq, void *dev_id)
+{
+ struct iio_dev *indio_dev = dev_id;
+ struct max11410_state *st = iio_priv(indio_dev);
+
+ if (iio_buffer_enabled(indio_dev))
+ iio_trigger_poll_chained(st->trig);
+ else
+ complete(&st->completion);
+
+ return IRQ_HANDLED;
+};
+
+static int max11410_parse_channels(struct max11410_state *st,
+ struct iio_dev *indio_dev)
+{
+ struct iio_chan_spec chanspec = chanspec_template;
+ struct device *dev = &st->spi_dev->dev;
+ struct max11410_channel_config *cfg;
+ struct iio_chan_spec *channels;
+ struct fwnode_handle *child;
+ u32 reference, sig_path;
+ const char *node_name;
+ u32 inputs[2], scale;
+ unsigned int num_ch;
+ int chan_idx = 0;
+ int ret, i;
+
+ num_ch = device_get_child_node_count(dev);
+ if (num_ch == 0)
+ return dev_err_probe(&indio_dev->dev, -ENODEV,
+ "FW has no channels defined\n");
+
+ /* Reserve space for soft timestamp channel */
+ num_ch++;
+ channels = devm_kcalloc(dev, num_ch, sizeof(*channels), GFP_KERNEL);
+ if (!channels)
+ return -ENOMEM;
+
+ st->channels = devm_kcalloc(dev, num_ch, sizeof(*st->channels),
+ GFP_KERNEL);
+ if (!st->channels)
+ return -ENOMEM;
+
+ device_for_each_child_node(dev, child) {
+ node_name = fwnode_get_name(child);
+ if (fwnode_property_present(child, "diff-channels")) {
+ ret = fwnode_property_read_u32_array(child,
+ "diff-channels",
+ inputs,
+ ARRAY_SIZE(inputs));
+
+ chanspec.differential = 1;
+ } else {
+ ret = fwnode_property_read_u32(child, "reg", &inputs[0]);
+
+ inputs[1] = 0;
+ chanspec.differential = 0;
+ }
+ if (ret) {
+ fwnode_handle_put(child);
+ return ret;
+ }
+
+ if (inputs[0] > MAX11410_CHANNEL_INDEX_MAX ||
+ inputs[1] > MAX11410_CHANNEL_INDEX_MAX) {
+ fwnode_handle_put(child);
+ return dev_err_probe(&indio_dev->dev, -EINVAL,
+ "Invalid channel index for %s, should be less than %d\n",
+ node_name,
+ MAX11410_CHANNEL_INDEX_MAX + 1);
+ }
+
+ cfg = &st->channels[chan_idx];
+
+ reference = MAX11410_REFSEL_AVDD_AGND;
+ fwnode_property_read_u32(child, "adi,reference", &reference);
+ if (reference > MAX11410_REFSEL_MAX) {
+ fwnode_handle_put(child);
+ return dev_err_probe(&indio_dev->dev, -EINVAL,
+ "Invalid adi,reference value for %s, should be less than %d.\n",
+ node_name, MAX11410_REFSEL_MAX + 1);
+ }
+
+ if (!max11410_get_vrefp(st, reference) ||
+ (!max11410_get_vrefn(st, reference) && reference <= 2)) {
+ fwnode_handle_put(child);
+ return dev_err_probe(&indio_dev->dev, -EINVAL,
+ "Invalid VREF configuration for %s, either specify corresponding VREF regulators or change adi,reference property.\n",
+ node_name);
+ }
+
+ sig_path = MAX11410_PGA_SIG_PATH_BUFFERED;
+ fwnode_property_read_u32(child, "adi,input-mode", &sig_path);
+ if (sig_path > MAX11410_SIG_PATH_MAX) {
+ fwnode_handle_put(child);
+ return dev_err_probe(&indio_dev->dev, -EINVAL,
+ "Invalid adi,input-mode value for %s, should be less than %d.\n",
+ node_name, MAX11410_SIG_PATH_MAX + 1);
+ }
+
+ fwnode_property_read_u32(child, "settling-time-us",
+ &cfg->settling_time_us);
+ cfg->bipolar = fwnode_property_read_bool(child, "bipolar");
+ cfg->buffered_vrefp = fwnode_property_read_bool(child, "adi,buffered-vrefp");
+ cfg->buffered_vrefn = fwnode_property_read_bool(child, "adi,buffered-vrefn");
+ cfg->refsel = reference;
+ cfg->sig_path = sig_path;
+ cfg->gain = 0;
+
+ /* Enable scale_available property if input mode is PGA */
+ if (sig_path == MAX11410_PGA_SIG_PATH_PGA) {
+ __set_bit(IIO_CHAN_INFO_SCALE,
+ &chanspec.info_mask_separate_available);
+ cfg->scale_avail = devm_kcalloc(dev, MAX11410_SCALE_AVAIL_SIZE * 2,
+ sizeof(*cfg->scale_avail),
+ GFP_KERNEL);
+ if (!cfg->scale_avail) {
+ fwnode_handle_put(child);
+ return -ENOMEM;
+ }
+
+ scale = max11410_get_scale(st, *cfg);
+ for (i = 0; i < MAX11410_SCALE_AVAIL_SIZE; i++) {
+ cfg->scale_avail[2 * i] = scale >> i;
+ cfg->scale_avail[2 * i + 1] = chanspec.scan_type.realbits;
+ }
+ } else {
+ __clear_bit(IIO_CHAN_INFO_SCALE,
+ &chanspec.info_mask_separate_available);
+ }
+
+ chanspec.address = chan_idx;
+ chanspec.scan_index = chan_idx;
+ chanspec.channel = inputs[0];
+ chanspec.channel2 = inputs[1];
+
+ channels[chan_idx] = chanspec;
+ chan_idx++;
+ }
+
+ channels[chan_idx] = (struct iio_chan_spec)IIO_CHAN_SOFT_TIMESTAMP(chan_idx);
+
+ indio_dev->num_channels = chan_idx + 1;
+ indio_dev->channels = channels;
+
+ return 0;
+}
+
+static void max11410_disable_reg(void *reg)
+{
+ regulator_disable(reg);
+}
+
+static int max11410_init_vref(struct device *dev,
+ struct regulator **vref,
+ const char *id)
+{
+ struct regulator *reg;
+ int ret;
+
+ reg = devm_regulator_get_optional(dev, id);
+ if (PTR_ERR(reg) == -ENODEV) {
+ *vref = NULL;
+ return 0;
+ } else if (IS_ERR(reg)) {
+ return PTR_ERR(reg);
+ }
+ ret = regulator_enable(reg);
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "Failed to enable regulator %s\n", id);
+
+ *vref = reg;
+ return devm_add_action_or_reset(dev, max11410_disable_reg, reg);
+}
+
+static int max11410_calibrate(struct max11410_state *st, u32 cal_type)
+{
+ int ret, val;
+
+ ret = max11410_write_reg(st, MAX11410_REG_CAL_START, cal_type);
+ if (ret)
+ return ret;
+
+ /* Wait for status register Calibration Ready flag */
+ return read_poll_timeout(max11410_read_reg, ret,
+ ret || (val & MAX11410_STATUS_CAL_READY_BIT),
+ 50000, MAX11410_CALIB_TIMEOUT_MS * 1000, true,
+ st, MAX11410_REG_STATUS, &val);
+}
+
+static int max11410_self_calibrate(struct max11410_state *st)
+{
+ int ret, i;
+
+ ret = regmap_write_bits(st->regmap, MAX11410_REG_FILTER,
+ MAX11410_FILTER_RATE_MASK,
+ FIELD_PREP(MAX11410_FILTER_RATE_MASK,
+ MAX11410_FILTER_RATE_MAX));
+ if (ret)
+ return ret;
+
+ ret = max11410_calibrate(st, MAX11410_CAL_START_SELF);
+ if (ret)
+ return ret;
+
+ ret = regmap_write_bits(st->regmap, MAX11410_REG_PGA,
+ MAX11410_PGA_SIG_PATH_MASK,
+ FIELD_PREP(MAX11410_PGA_SIG_PATH_MASK,
+ MAX11410_PGA_SIG_PATH_PGA));
+ if (ret)
+ return ret;
+
+ /* PGA calibrations */
+ for (i = 1; i < 8; ++i) {
+ ret = regmap_write_bits(st->regmap, MAX11410_REG_PGA,
+ MAX11410_PGA_GAIN_MASK, i);
+ if (ret)
+ return ret;
+
+ ret = max11410_calibrate(st, MAX11410_CAL_START_PGA);
+ if (ret)
+ return ret;
+ }
+
+ /* Cleanup */
+ ret = regmap_write_bits(st->regmap, MAX11410_REG_PGA,
+ MAX11410_PGA_GAIN_MASK, 0);
+ if (ret)
+ return ret;
+
+ ret = regmap_write_bits(st->regmap, MAX11410_REG_FILTER,
+ MAX11410_FILTER_RATE_MASK, 0);
+ if (ret)
+ return ret;
+
+ return regmap_write_bits(st->regmap, MAX11410_REG_PGA,
+ MAX11410_PGA_SIG_PATH_MASK,
+ FIELD_PREP(MAX11410_PGA_SIG_PATH_MASK,
+ MAX11410_PGA_SIG_PATH_BUFFERED));
+}
+
+static int max11410_probe(struct spi_device *spi)
+{
+ const char *vrefp_regs[] = { "vref0p", "vref1p", "vref2p" };
+ const char *vrefn_regs[] = { "vref0n", "vref1n", "vref2n" };
+ struct device *dev = &spi->dev;
+ struct max11410_state *st;
+ struct iio_dev *indio_dev;
+ int ret, irqs[2];
+ int i;
+
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*st));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ st = iio_priv(indio_dev);
+ st->spi_dev = spi;
+ init_completion(&st->completion);
+ mutex_init(&st->lock);
+
+ indio_dev->name = "max11410";
+ indio_dev->modes = INDIO_DIRECT_MODE;
+ indio_dev->info = &max11410_info;
+
+ st->regmap = devm_regmap_init_spi(spi, ®map_config);
+ if (IS_ERR(st->regmap))
+ return dev_err_probe(dev, PTR_ERR(st->regmap),
+ "regmap initialization failed\n");
+
+ ret = max11410_init_vref(dev, &st->avdd, "avdd");
+ if (ret)
+ return ret;
+
+ for (i = 0; i < ARRAY_SIZE(vrefp_regs); i++) {
+ ret = max11410_init_vref(dev, &st->vrefp[i], vrefp_regs[i]);
+ if (ret)
+ return ret;
+
+ ret = max11410_init_vref(dev, &st->vrefn[i], vrefn_regs[i]);
+ if (ret)
+ return ret;
+ }
+
+ /*
+ * Regulators must be configured before parsing channels for
+ * validating "adi,reference" property of each channel.
+ */
+ ret = max11410_parse_channels(st, indio_dev);
+ if (ret)
+ return ret;
+
+ irqs[0] = fwnode_irq_get_byname(dev_fwnode(dev), "gpio0");
+ irqs[1] = fwnode_irq_get_byname(dev_fwnode(dev), "gpio1");
+
+ if (irqs[0] > 0) {
+ st->irq = irqs[0];
+ ret = regmap_write(st->regmap, MAX11410_REG_GPIO_CTRL(0),
+ MAX11410_GPIO_INTRB);
+ } else if (irqs[1] > 0) {
+ st->irq = irqs[1];
+ ret = regmap_write(st->regmap, MAX11410_REG_GPIO_CTRL(1),
+ MAX11410_GPIO_INTRB);
+ } else if (spi->irq > 0) {
+ return dev_err_probe(dev, -ENODEV,
+ "no interrupt name specified");
+ }
+
+ if (ret)
+ return ret;
+
+ ret = regmap_set_bits(st->regmap, MAX11410_REG_CTRL,
+ MAX11410_CTRL_FORMAT_BIT);
+ if (ret)
+ return ret;
+
+ ret = devm_iio_triggered_buffer_setup(dev, indio_dev, NULL,
+ &max11410_trigger_handler,
+ &max11410_buffer_ops);
+ if (ret)
+ return ret;
+
+ if (st->irq > 0) {
+ st->trig = devm_iio_trigger_alloc(dev, "%s-dev%d",
+ indio_dev->name,
+ iio_device_id(indio_dev));
+ if (!st->trig)
+ return -ENOMEM;
+
+ st->trig->ops = &max11410_trigger_ops;
+ ret = devm_iio_trigger_register(dev, st->trig);
+ if (ret)
+ return ret;
+
+ ret = devm_request_threaded_irq(dev, st->irq, NULL,
+ &max11410_interrupt,
+ IRQF_ONESHOT, "max11410",
+ indio_dev);
+ if (ret)
+ return ret;
+ }
+
+ ret = max11410_self_calibrate(st);
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "cannot perform device self calibration\n");
+
+ return devm_iio_device_register(dev, indio_dev);
+}
+
+static const struct of_device_id max11410_spi_of_id[] = {
+ { .compatible = "adi,max11410" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, max11410_spi_of_id);
+
+static const struct spi_device_id max11410_id[] = {
+ { "max11410" },
+ { }
+};
+MODULE_DEVICE_TABLE(spi, max11410_id);
+
+static struct spi_driver max11410_driver = {
+ .driver = {
+ .name = "max11410",
+ .of_match_table = max11410_spi_of_id,
+ },
+ .probe = max11410_probe,
+ .id_table = max11410_id,
+};
+module_spi_driver(max11410_driver);
+
+MODULE_AUTHOR("David Jung <David.Jung@analog.com>");
+MODULE_AUTHOR("Ibrahim Tilki <Ibrahim.Tilki@analog.com>");
+MODULE_DESCRIPTION("Analog Devices MAX11410 ADC");
+MODULE_LICENSE("GPL");
struct max1241 {
struct spi_device *spi;
struct mutex lock;
- struct regulator *vdd;
struct regulator *vref;
struct gpio_desc *shutdown;
.read_raw = max1241_read_raw,
};
-static void max1241_disable_vdd_action(void *data)
-{
- struct max1241 *adc = data;
- struct device *dev = &adc->spi->dev;
- int err;
-
- err = regulator_disable(adc->vdd);
- if (err)
- dev_err(dev, "could not disable vdd regulator.\n");
-}
-
static void max1241_disable_vref_action(void *data)
{
struct max1241 *adc = data;
adc->spi = spi;
mutex_init(&adc->lock);
- adc->vdd = devm_regulator_get(dev, "vdd");
- if (IS_ERR(adc->vdd))
- return dev_err_probe(dev, PTR_ERR(adc->vdd),
- "failed to get vdd regulator\n");
-
- ret = regulator_enable(adc->vdd);
+ ret = devm_regulator_get_enable(dev, "vdd");
if (ret)
- return ret;
-
- ret = devm_add_action_or_reset(dev, max1241_disable_vdd_action, adc);
- if (ret) {
- dev_err(dev, "could not set up vdd regulator cleanup action\n");
- return ret;
- }
+ return dev_err_probe(dev, ret,
+ "failed to get/enable vdd regulator\n");
adc->vref = devm_regulator_get(dev, "vref");
if (IS_ERR(adc->vref))
* @chip_info: chip model specific constants, available modes, etc.
* @current_mode: the scan mode of this chip
* @requestedmask: a valid requested set of channels
- * @reg: supply regulator
* @lock: lock to ensure state is consistent
* @monitor_on: whether monitor mode is enabled
* @monitor_speed: parameter corresponding to device monitor speed setting
const struct max1363_chip_info *chip_info;
const struct max1363_mode *current_mode;
u32 requestedmask;
- struct regulator *reg;
struct mutex lock;
/* Using monitor modes and buffer at the same time is
regulator_disable(reg);
}
-static int max1363_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max1363_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
int ret;
struct max1363_state *st;
struct iio_dev *indio_dev;
st = iio_priv(indio_dev);
mutex_init(&st->lock);
- st->reg = devm_regulator_get(&client->dev, "vcc");
- if (IS_ERR(st->reg))
- return PTR_ERR(st->reg);
-
- ret = regulator_enable(st->reg);
- if (ret)
- return ret;
-
- ret = devm_add_action_or_reset(&client->dev, max1363_reg_disable, st->reg);
+ ret = devm_regulator_get_enable(&client->dev, "vcc");
if (ret)
return ret;
.name = "max1363",
.of_match_table = max1363_of_match,
},
- .probe = max1363_probe,
+ .probe_new = max1363_probe,
.id_table = max1363_id,
};
module_i2c_driver(max1363_driver);
};
MODULE_DEVICE_TABLE(of, max9611_of_table);
-static int max9611_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max9611_probe(struct i2c_client *client)
{
const char * const shunt_res_prop = "shunt-resistor-micro-ohms";
struct max9611_dev *max9611;
.name = DRIVER_NAME,
.of_match_table = max9611_of_table,
},
- .probe = max9611_probe,
+ .probe_new = max9611_probe,
};
module_i2c_driver(max9611_driver);
.attrs = &mcp3422_attribute_group,
};
-static int mcp3422_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mcp3422_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct iio_dev *indio_dev;
struct mcp3422 *adc;
int err;
.name = "mcp3422",
.of_match_table = mcp3422_of_match,
},
- .probe = mcp3422_probe,
+ .probe_new = mcp3422_probe,
.id_table = mcp3422_id,
};
module_i2c_driver(mcp3422_driver);
#define MCP3911_REG_MOD 0x06
#define MCP3911_REG_PHASE 0x07
#define MCP3911_REG_GAIN 0x09
+#define MCP3911_GAIN_MASK(ch) (GENMASK(2, 0) << 3 * ch)
+#define MCP3911_GAIN_VAL(ch, val) ((val << 3 * ch) & MCP3911_GAIN_MASK(ch))
#define MCP3911_REG_STATUSCOM 0x0a
#define MCP3911_STATUSCOM_DRHIZ BIT(12)
#define MCP3911_REG_MASK GENMASK(4, 1)
#define MCP3911_NUM_CHANNELS 2
+#define MCP3911_NUM_SCALES 6
static const int mcp3911_osr_table[] = { 32, 64, 128, 256, 512, 1024, 2048, 4096 };
+static u32 mcp3911_scale_table[MCP3911_NUM_SCALES][2];
struct mcp3911 {
struct spi_device *spi;
struct clk *clki;
u32 dev_addr;
struct iio_trigger *trig;
+ u32 gain[MCP3911_NUM_CHANNELS];
struct {
u32 channels[MCP3911_NUM_CHANNELS];
s64 ts __aligned(8);
*vals = mcp3911_osr_table;
*length = ARRAY_SIZE(mcp3911_osr_table);
return IIO_AVAIL_LIST;
+ case IIO_CHAN_INFO_SCALE:
+ *type = IIO_VAL_INT_PLUS_NANO;
+ *vals = (int *)mcp3911_scale_table;
+ *length = ARRAY_SIZE(mcp3911_scale_table) * 2;
+ return IIO_AVAIL_LIST;
default:
return -EINVAL;
}
break;
case IIO_CHAN_INFO_SCALE:
- if (adc->vref) {
- ret = regulator_get_voltage(adc->vref);
- if (ret < 0) {
- dev_err(indio_dev->dev.parent,
- "failed to get vref voltage: %d\n",
- ret);
- goto out;
- }
-
- *val = ret / 1000;
- } else {
- *val = MCP3911_INT_VREF_MV;
- }
-
- /*
- * For 24bit Conversion
- * Raw = ((Voltage)/(Vref) * 2^23 * Gain * 1.5
- * Voltage = Raw * (Vref)/(2^23 * Gain * 1.5)
- */
-
- /* val2 = (2^23 * 1.5) */
- *val2 = 12582912;
- ret = IIO_VAL_FRACTIONAL;
+ *val = mcp3911_scale_table[ilog2(adc->gain[channel->channel])][0];
+ *val2 = mcp3911_scale_table[ilog2(adc->gain[channel->channel])][1];
+ ret = IIO_VAL_INT_PLUS_NANO;
break;
}
mutex_lock(&adc->lock);
switch (mask) {
+ case IIO_CHAN_INFO_SCALE:
+ for (int i = 0; i < MCP3911_NUM_SCALES; i++) {
+ if (val == mcp3911_scale_table[i][0] &&
+ val2 == mcp3911_scale_table[i][1]) {
+
+ adc->gain[channel->channel] = BIT(i);
+ ret = mcp3911_update(adc, MCP3911_REG_GAIN,
+ MCP3911_GAIN_MASK(channel->channel),
+ MCP3911_GAIN_VAL(channel->channel, i), 1);
+ }
+ }
+ break;
case IIO_CHAN_INFO_OFFSET:
if (val2 != 0) {
ret = -EINVAL;
return ret;
}
+static int mcp3911_calc_scale_table(struct mcp3911 *adc)
+{
+ u32 ref = MCP3911_INT_VREF_MV;
+ u32 div;
+ int ret;
+ u64 tmp;
+
+ if (adc->vref) {
+ ret = regulator_get_voltage(adc->vref);
+ if (ret < 0) {
+ dev_err(&adc->spi->dev,
+ "failed to get vref voltage: %d\n",
+ ret);
+ return ret;
+ }
+
+ ref = ret / 1000;
+ }
+
+ /*
+ * For 24-bit Conversion
+ * Raw = ((Voltage)/(Vref) * 2^23 * Gain * 1.5
+ * Voltage = Raw * (Vref)/(2^23 * Gain * 1.5)
+ *
+ * ref = Reference voltage
+ * div = (2^23 * 1.5 * gain) = 12582912 * gain
+ */
+ for (int i = 0; i < MCP3911_NUM_SCALES; i++) {
+ div = 12582912 * BIT(i);
+ tmp = div_s64((s64)ref * 1000000000LL, div);
+
+ mcp3911_scale_table[i][0] = 0;
+ mcp3911_scale_table[i][1] = tmp;
+ }
+
+ return 0;
+}
+
#define MCP3911_CHAN(idx) { \
.type = IIO_VOLTAGE, \
.indexed = 1, \
.info_mask_separate = BIT(IIO_CHAN_INFO_RAW) | \
BIT(IIO_CHAN_INFO_OFFSET) | \
BIT(IIO_CHAN_INFO_SCALE), \
- .info_mask_shared_by_type_available = \
+ .info_mask_shared_by_type_available = \
BIT(IIO_CHAN_INFO_OVERSAMPLING_RATIO), \
+ .info_mask_separate_available = \
+ BIT(IIO_CHAN_INFO_SCALE), \
.scan_type = { \
.sign = 's', \
.realbits = 24, \
if (ret)
return ret;
+ ret = mcp3911_calc_scale_table(adc);
+ if (ret)
+ return ret;
+
+ /* Set gain to 1 for all channels */
+ for (int i = 0; i < MCP3911_NUM_CHANNELS; i++) {
+ adc->gain[i] = 1;
+ ret = mcp3911_update(adc, MCP3911_REG_GAIN,
+ MCP3911_GAIN_MASK(i),
+ MCP3911_GAIN_VAL(i, 0), 1);
+ if (ret)
+ return ret;
+ }
+
indio_dev->name = spi_get_device_id(spi)->name;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->info = &mcp3911_info;
#include <linux/io.h>
#include <linux/iio/iio.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/nvmem-consumer.h>
#include <linux/interrupt.h>
#include <linux/of.h>
struct clk *adc_div_clk;
struct clk_divider clk_div;
struct completion done;
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
int calibbias;
int calibscale;
struct regmap *tsc_regmap;
struct meson_sar_adc_priv *priv = iio_priv(indio_dev);
int val, ret;
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&priv->lock);
if (priv->param->has_bl30_integration) {
/* prevent BL30 from using the SAR ADC while we are using it */
!(val & MESON_SAR_ADC_DELAY_BL30_BUSY),
1, 10000);
if (ret) {
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&priv->lock);
return ret;
}
}
regmap_update_bits(priv->regmap, MESON_SAR_ADC_DELAY,
MESON_SAR_ADC_DELAY_KERNEL_BUSY, 0);
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&priv->lock);
}
static void meson_sar_adc_clear_fifo(struct iio_dev *indio_dev)
if (ret)
goto err;
+ mutex_init(&priv->lock);
+
ret = meson_sar_adc_hw_enable(indio_dev);
if (ret)
goto err;
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2022 Richtek Technology Corp.
+ *
+ * Author: ChiaEn Wu <chiaen_wu@richtek.com>
+ */
+
+#include <linux/bits.h>
+#include <linux/bitfield.h>
+#include <linux/iio/iio.h>
+#include <linux/kernel.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <linux/sysfs.h>
+#include <linux/units.h>
+
+#include <dt-bindings/iio/adc/mediatek,mt6370_adc.h>
+
+#define MT6370_REG_CHG_CTRL3 0x113
+#define MT6370_REG_CHG_CTRL7 0x117
+#define MT6370_REG_CHG_ADC 0x121
+#define MT6370_REG_ADC_DATA_H 0x14C
+
+#define MT6370_ADC_START_MASK BIT(0)
+#define MT6370_ADC_IN_SEL_MASK GENMASK(7, 4)
+#define MT6370_AICR_ICHG_MASK GENMASK(7, 2)
+
+#define MT6370_AICR_100_mA 0x0
+#define MT6370_AICR_150_mA 0x1
+#define MT6370_AICR_200_mA 0x2
+#define MT6370_AICR_250_mA 0x3
+#define MT6370_AICR_300_mA 0x4
+#define MT6370_AICR_350_mA 0x5
+
+#define MT6370_ICHG_100_mA 0x0
+#define MT6370_ICHG_200_mA 0x1
+#define MT6370_ICHG_300_mA 0x2
+#define MT6370_ICHG_400_mA 0x3
+#define MT6370_ICHG_500_mA 0x4
+#define MT6370_ICHG_600_mA 0x5
+#define MT6370_ICHG_700_mA 0x6
+#define MT6370_ICHG_800_mA 0x7
+
+#define ADC_CONV_TIME_MS 35
+#define ADC_CONV_POLLING_TIME_US 1000
+
+struct mt6370_adc_data {
+ struct device *dev;
+ struct regmap *regmap;
+ /*
+ * This mutex lock is for preventing the different ADC channels
+ * from being read at the same time.
+ */
+ struct mutex adc_lock;
+};
+
+static int mt6370_adc_read_channel(struct mt6370_adc_data *priv, int chan,
+ unsigned long addr, int *val)
+{
+ unsigned int reg_val;
+ __be16 be_val;
+ int ret;
+
+ mutex_lock(&priv->adc_lock);
+
+ reg_val = MT6370_ADC_START_MASK |
+ FIELD_PREP(MT6370_ADC_IN_SEL_MASK, addr);
+ ret = regmap_write(priv->regmap, MT6370_REG_CHG_ADC, reg_val);
+ if (ret)
+ goto adc_unlock;
+
+ msleep(ADC_CONV_TIME_MS);
+
+ ret = regmap_read_poll_timeout(priv->regmap,
+ MT6370_REG_CHG_ADC, reg_val,
+ !(reg_val & MT6370_ADC_START_MASK),
+ ADC_CONV_POLLING_TIME_US,
+ ADC_CONV_TIME_MS * MILLI * 3);
+ if (ret) {
+ dev_err(priv->dev, "Failed to read ADC register (%d)\n", ret);
+ goto adc_unlock;
+ }
+
+ ret = regmap_raw_read(priv->regmap, MT6370_REG_ADC_DATA_H,
+ &be_val, sizeof(be_val));
+ if (ret)
+ goto adc_unlock;
+
+ *val = be16_to_cpu(be_val);
+ ret = IIO_VAL_INT;
+
+adc_unlock:
+ mutex_unlock(&priv->adc_lock);
+
+ return ret;
+}
+
+static int mt6370_adc_read_scale(struct mt6370_adc_data *priv,
+ int chan, int *val1, int *val2)
+{
+ unsigned int reg_val;
+ int ret;
+
+ switch (chan) {
+ case MT6370_CHAN_VBAT:
+ case MT6370_CHAN_VSYS:
+ case MT6370_CHAN_CHG_VDDP:
+ *val1 = 5;
+ return IIO_VAL_INT;
+ case MT6370_CHAN_IBUS:
+ ret = regmap_read(priv->regmap, MT6370_REG_CHG_CTRL3, ®_val);
+ if (ret)
+ return ret;
+
+ reg_val = FIELD_GET(MT6370_AICR_ICHG_MASK, reg_val);
+ switch (reg_val) {
+ case MT6370_AICR_100_mA:
+ case MT6370_AICR_150_mA:
+ case MT6370_AICR_200_mA:
+ case MT6370_AICR_250_mA:
+ case MT6370_AICR_300_mA:
+ case MT6370_AICR_350_mA:
+ *val1 = 3350;
+ break;
+ default:
+ *val1 = 5000;
+ break;
+ }
+
+ *val2 = 100;
+
+ return IIO_VAL_FRACTIONAL;
+ case MT6370_CHAN_IBAT:
+ ret = regmap_read(priv->regmap, MT6370_REG_CHG_CTRL7, ®_val);
+ if (ret)
+ return ret;
+
+ reg_val = FIELD_GET(MT6370_AICR_ICHG_MASK, reg_val);
+ switch (reg_val) {
+ case MT6370_ICHG_100_mA:
+ case MT6370_ICHG_200_mA:
+ case MT6370_ICHG_300_mA:
+ case MT6370_ICHG_400_mA:
+ *val1 = 2375;
+ break;
+ case MT6370_ICHG_500_mA:
+ case MT6370_ICHG_600_mA:
+ case MT6370_ICHG_700_mA:
+ case MT6370_ICHG_800_mA:
+ *val1 = 2680;
+ break;
+ default:
+ *val1 = 5000;
+ break;
+ }
+
+ *val2 = 100;
+
+ return IIO_VAL_FRACTIONAL;
+ case MT6370_CHAN_VBUSDIV5:
+ *val1 = 25;
+ return IIO_VAL_INT;
+ case MT6370_CHAN_VBUSDIV2:
+ *val1 = 10;
+ return IIO_VAL_INT;
+ case MT6370_CHAN_TS_BAT:
+ *val1 = 25;
+ *val2 = 10000;
+ return IIO_VAL_FRACTIONAL;
+ case MT6370_CHAN_TEMP_JC:
+ *val1 = 2000;
+ return IIO_VAL_INT;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int mt6370_adc_read_offset(struct mt6370_adc_data *priv,
+ int chan, int *val)
+{
+ *val = -20;
+
+ return IIO_VAL_INT;
+}
+
+static int mt6370_adc_read_raw(struct iio_dev *iio_dev,
+ const struct iio_chan_spec *chan,
+ int *val, int *val2, long mask)
+{
+ struct mt6370_adc_data *priv = iio_priv(iio_dev);
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ return mt6370_adc_read_channel(priv, chan->channel,
+ chan->address, val);
+ case IIO_CHAN_INFO_SCALE:
+ return mt6370_adc_read_scale(priv, chan->channel, val, val2);
+ case IIO_CHAN_INFO_OFFSET:
+ return mt6370_adc_read_offset(priv, chan->channel, val);
+ default:
+ return -EINVAL;
+ }
+}
+
+static const char * const mt6370_channel_labels[MT6370_CHAN_MAX] = {
+ [MT6370_CHAN_VBUSDIV5] = "vbusdiv5",
+ [MT6370_CHAN_VBUSDIV2] = "vbusdiv2",
+ [MT6370_CHAN_VSYS] = "vsys",
+ [MT6370_CHAN_VBAT] = "vbat",
+ [MT6370_CHAN_TS_BAT] = "ts_bat",
+ [MT6370_CHAN_IBUS] = "ibus",
+ [MT6370_CHAN_IBAT] = "ibat",
+ [MT6370_CHAN_CHG_VDDP] = "chg_vddp",
+ [MT6370_CHAN_TEMP_JC] = "temp_jc",
+};
+
+static int mt6370_adc_read_label(struct iio_dev *iio_dev,
+ struct iio_chan_spec const *chan, char *label)
+{
+ return sysfs_emit(label, "%s\n", mt6370_channel_labels[chan->channel]);
+}
+
+static const struct iio_info mt6370_adc_iio_info = {
+ .read_raw = mt6370_adc_read_raw,
+ .read_label = mt6370_adc_read_label,
+};
+
+#define MT6370_ADC_CHAN(_idx, _type, _addr, _extra_info) { \
+ .type = _type, \
+ .channel = MT6370_CHAN_##_idx, \
+ .address = _addr, \
+ .scan_index = MT6370_CHAN_##_idx, \
+ .indexed = 1, \
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) | \
+ BIT(IIO_CHAN_INFO_SCALE) | \
+ _extra_info, \
+}
+
+static const struct iio_chan_spec mt6370_adc_channels[] = {
+ MT6370_ADC_CHAN(VBUSDIV5, IIO_VOLTAGE, 1, 0),
+ MT6370_ADC_CHAN(VBUSDIV2, IIO_VOLTAGE, 2, 0),
+ MT6370_ADC_CHAN(VSYS, IIO_VOLTAGE, 3, 0),
+ MT6370_ADC_CHAN(VBAT, IIO_VOLTAGE, 4, 0),
+ MT6370_ADC_CHAN(TS_BAT, IIO_VOLTAGE, 6, 0),
+ MT6370_ADC_CHAN(IBUS, IIO_CURRENT, 8, 0),
+ MT6370_ADC_CHAN(IBAT, IIO_CURRENT, 9, 0),
+ MT6370_ADC_CHAN(CHG_VDDP, IIO_VOLTAGE, 11, 0),
+ MT6370_ADC_CHAN(TEMP_JC, IIO_TEMP, 12, BIT(IIO_CHAN_INFO_OFFSET)),
+};
+
+static int mt6370_adc_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mt6370_adc_data *priv;
+ struct iio_dev *indio_dev;
+ struct regmap *regmap;
+ int ret;
+
+ regmap = dev_get_regmap(pdev->dev.parent, NULL);
+ if (!regmap)
+ return dev_err_probe(dev, -ENODEV, "Failed to get regmap\n");
+
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*priv));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ priv = iio_priv(indio_dev);
+ priv->dev = dev;
+ priv->regmap = regmap;
+ mutex_init(&priv->adc_lock);
+
+ ret = regmap_write(priv->regmap, MT6370_REG_CHG_ADC, 0);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to reset ADC\n");
+
+ indio_dev->name = "mt6370-adc";
+ indio_dev->info = &mt6370_adc_iio_info;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+ indio_dev->channels = mt6370_adc_channels;
+ indio_dev->num_channels = ARRAY_SIZE(mt6370_adc_channels);
+
+ return devm_iio_device_register(dev, indio_dev);
+}
+
+static const struct of_device_id mt6370_adc_of_id[] = {
+ { .compatible = "mediatek,mt6370-adc", },
+ {}
+};
+MODULE_DEVICE_TABLE(of, mt6370_adc_of_id);
+
+static struct platform_driver mt6370_adc_driver = {
+ .driver = {
+ .name = "mt6370-adc",
+ .of_match_table = mt6370_adc_of_id,
+ },
+ .probe = mt6370_adc_probe,
+};
+module_platform_driver(mt6370_adc_driver);
+
+MODULE_AUTHOR("ChiaEn Wu <chiaen_wu@richtek.com>");
+MODULE_DESCRIPTION("MT6370 ADC Driver");
+MODULE_LICENSE("GPL v2");
*/
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
#include <linux/io.h>
struct clk *clk;
struct completion completion;
struct regulator *vref;
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
int uv_vref;
struct reset_control *reset;
const struct rockchip_saradc_data *data;
switch (mask) {
case IIO_CHAN_INFO_RAW:
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&info->lock);
ret = rockchip_saradc_conversion(info, chan);
if (ret) {
rockchip_saradc_power_down(info);
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&info->lock);
return ret;
}
*val = info->last_val;
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&info->lock);
return IIO_VAL_INT;
case IIO_CHAN_INFO_SCALE:
*val = info->uv_vref / 1000;
int ret;
int i, j = 0;
- mutex_lock(&i_dev->mlock);
+ mutex_lock(&info->lock);
for_each_set_bit(i, i_dev->active_scan_mask, i_dev->masklength) {
const struct iio_chan_spec *chan = &i_dev->channels[i];
iio_push_to_buffers_with_timestamp(i_dev, &data, iio_get_time_ns(i_dev));
out:
- mutex_unlock(&i_dev->mlock);
+ mutex_unlock(&info->lock);
iio_trigger_notify_done(i_dev->trig);
if (ret)
return ret;
+ mutex_init(&info->lock);
+
return devm_iio_device_register(&pdev->dev, indio_dev);
}
#include <linux/hwspinlock.h>
#include <linux/iio/iio.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/nvmem-consumer.h>
#include <linux/of.h>
#include <linux/of_device.h>
struct device *dev;
struct regulator *volref;
struct regmap *regmap;
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
/*
* One hardware spinlock to synchronize between the multiple
* subsystems which will access the unique ADC controller.
switch (mask) {
case IIO_CHAN_INFO_RAW:
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&data->lock);
ret = sc27xx_adc_read(data, chan->channel, scale, &tmp);
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&data->lock);
if (ret)
return ret;
return IIO_VAL_INT;
case IIO_CHAN_INFO_PROCESSED:
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&data->lock);
ret = sc27xx_adc_read_processed(data, chan->channel, scale,
&tmp);
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&data->lock);
if (ret)
return ret;
indio_dev->info = &sc27xx_info;
indio_dev->channels = sc27xx_channels;
indio_dev->num_channels = ARRAY_SIZE(sc27xx_channels);
+
+ mutex_init(&sc27xx_data->lock);
+
ret = devm_iio_device_register(dev, indio_dev);
if (ret)
dev_err(dev, "could not register iio (ADC)");
#include <linux/regmap.h>
#include <linux/regulator/consumer.h>
#include <linux/slab.h>
+#include <linux/units.h>
#include "stm32-adc-core.h"
static const struct stm32_adc_common_regs stm32f4_adc_common_regs = {
.csr = STM32F4_ADC_CSR,
.ccr = STM32F4_ADC_CCR,
- .eoc_msk = { STM32F4_EOC1, STM32F4_EOC2, STM32F4_EOC3},
- .ovr_msk = { STM32F4_OVR1, STM32F4_OVR2, STM32F4_OVR3},
+ .eoc_msk = { STM32F4_EOC1, STM32F4_EOC2, STM32F4_EOC3 },
+ .ovr_msk = { STM32F4_OVR1, STM32F4_OVR2, STM32F4_OVR3 },
.ier = STM32F4_ADC_CR1,
.eocie_msk = STM32F4_EOCIE,
};
static const struct stm32_adc_common_regs stm32h7_adc_common_regs = {
.csr = STM32H7_ADC_CSR,
.ccr = STM32H7_ADC_CCR,
- .eoc_msk = { STM32H7_EOC_MST, STM32H7_EOC_SLV},
- .ovr_msk = { STM32H7_OVR_MST, STM32H7_OVR_SLV},
+ .eoc_msk = { STM32H7_EOC_MST, STM32H7_EOC_SLV },
+ .ovr_msk = { STM32H7_OVR_MST, STM32H7_OVR_SLV },
+ .ier = STM32H7_ADC_IER,
+ .eocie_msk = STM32H7_EOCIE,
+};
+
+/* STM32MP13 common registers definitions */
+static const struct stm32_adc_common_regs stm32mp13_adc_common_regs = {
+ .csr = STM32H7_ADC_CSR,
+ .ccr = STM32H7_ADC_CCR,
+ .eoc_msk = { STM32H7_EOC_MST },
+ .ovr_msk = { STM32H7_OVR_MST },
.ier = STM32H7_ADC_IER,
.eocie_msk = STM32H7_EOCIE,
};
.num_irqs = 2,
};
+static const struct stm32_adc_priv_cfg stm32mp13_adc_priv_cfg = {
+ .regs = &stm32mp13_adc_common_regs,
+ .clk_sel = stm32h7_adc_clk_sel,
+ .max_clk_rate_hz = 75 * HZ_PER_MHZ,
+ .ipid = STM32MP13_IPIDR_NUMBER,
+ .num_irqs = 1,
+};
+
static const struct of_device_id stm32_adc_of_match[] = {
{
.compatible = "st,stm32f4-adc-core",
.compatible = "st,stm32mp1-adc-core",
.data = (void *)&stm32mp1_adc_priv_cfg
}, {
+ .compatible = "st,stm32mp13-adc-core",
+ .data = (void *)&stm32mp13_adc_priv_cfg
+ }, {
},
};
MODULE_DEVICE_TABLE(of, stm32_adc_of_match);
#define STM32MP1_ADC_IPDR 0x3F8
#define STM32MP1_ADC_SIDR 0x3FC
+/* STM32MP13 - Registers for each ADC instance */
+#define STM32MP13_ADC_DIFSEL 0xB0
+#define STM32MP13_ADC_CALFACT 0xB4
+#define STM32MP13_ADC2_OR 0xC8
+
/* STM32H7 - common registers for all ADC instances */
#define STM32H7_ADC_CSR (STM32_ADCX_COMN_OFFSET + 0x00)
#define STM32H7_ADC_CCR (STM32_ADCX_COMN_OFFSET + 0x08)
#define STM32H7_LINCALRDYW3 BIT(24)
#define STM32H7_LINCALRDYW2 BIT(23)
#define STM32H7_LINCALRDYW1 BIT(22)
+#define STM32H7_LINCALRDYW_MASK GENMASK(27, 22)
#define STM32H7_ADCALLIN BIT(16)
#define STM32H7_BOOST BIT(8)
#define STM32H7_ADSTP BIT(4)
STM32H7_DMNGT_DMA_CIRC, /* DMA circular mode */
};
+/* STM32H7_ADC_DIFSEL - bit fields */
+#define STM32H7_DIFSEL_MASK GENMASK(19, 0)
+
/* STM32H7_ADC_CALFACT - bit fields */
#define STM32H7_CALFACT_D_SHIFT 16
#define STM32H7_CALFACT_D_MASK GENMASK(26, 16)
/* STM32MP1_ADC_SIDR - bit fields */
#define STM32MP1_SIDR_MASK GENMASK(31, 0)
+/* STM32MP13_ADC_CFGR specific bit fields */
+#define STM32MP13_DMAEN BIT(0)
+#define STM32MP13_DMACFG BIT(1)
+#define STM32MP13_DFSDMCFG BIT(2)
+#define STM32MP13_RES_SHIFT 3
+#define STM32MP13_RES_MASK GENMASK(4, 3)
+
+/* STM32MP13_ADC_DIFSEL - bit fields */
+#define STM32MP13_DIFSEL_MASK GENMASK(18, 0)
+
+/* STM32MP13_ADC_CALFACT - bit fields */
+#define STM32MP13_CALFACT_D_SHIFT 16
+#define STM32MP13_CALFACT_D_MASK GENMASK(22, 16)
+#define STM32MP13_CALFACT_S_SHIFT 0
+#define STM32MP13_CALFACT_S_MASK GENMASK(6, 0)
+
+/* STM32MP13_ADC2_OR - bit fields */
+#define STM32MP13_OP2 BIT(2)
+#define STM32MP13_OP1 BIT(1)
+#define STM32MP13_OP0 BIT(0)
+
#define STM32MP15_IPIDR_NUMBER 0x00110005
+#define STM32MP13_IPIDR_NUMBER 0x00110006
/**
* struct stm32_adc_common - stm32 ADC driver common data (for all instances)
*/
#include <linux/clk.h>
+#include <linux/debugfs.h>
#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/dmaengine.h>
enum stm32_adc_int_ch {
STM32_ADC_INT_CH_NONE = -1,
STM32_ADC_INT_CH_VDDCORE,
+ STM32_ADC_INT_CH_VDDCPU,
+ STM32_ADC_INT_CH_VDDQ_DDR,
STM32_ADC_INT_CH_VREFINT,
STM32_ADC_INT_CH_VBAT,
STM32_ADC_INT_CH_NB,
static const struct stm32_adc_ic stm32_adc_ic[STM32_ADC_INT_CH_NB] = {
{ "vddcore", STM32_ADC_INT_CH_VDDCORE },
+ { "vddcpu", STM32_ADC_INT_CH_VDDCPU },
+ { "vddq_ddr", STM32_ADC_INT_CH_VDDQ_DDR },
{ "vrefint", STM32_ADC_INT_CH_VREFINT },
{ "vbat", STM32_ADC_INT_CH_VBAT },
};
/**
* struct stm32_adc_calib - optional adc calibration data
- * @calfact_s: Calibration offset for single ended channels
- * @calfact_d: Calibration offset in differential
* @lincalfact: Linearity calibration factor
- * @calibrated: Indicates calibration status
+ * @lincal_saved: Indicates that linear calibration factors are saved
*/
struct stm32_adc_calib {
- u32 calfact_s;
- u32 calfact_d;
u32 lincalfact[STM32H7_LINCALFACT_NUM];
- bool calibrated;
+ bool lincal_saved;
};
/**
* @exten: trigger control register & bitfield
* @extsel: trigger selection register & bitfield
* @res: resolution selection register & bitfield
+ * @difsel: differential mode selection register & bitfield
* @smpr: smpr1 & smpr2 registers offset array
* @smp_bits: smpr1 & smpr2 index and bitfields
- * @or_vdd: option register & vddcore bitfield
+ * @or_vddcore: option register & vddcore bitfield
+ * @or_vddcpu: option register & vddcpu bitfield
+ * @or_vddq_ddr: option register & vddq_ddr bitfield
* @ccr_vbat: common register & vbat bitfield
* @ccr_vref: common register & vrefint bitfield
*/
const struct stm32_adc_regs exten;
const struct stm32_adc_regs extsel;
const struct stm32_adc_regs res;
+ const struct stm32_adc_regs difsel;
const u32 smpr[2];
const struct stm32_adc_regs *smp_bits;
- const struct stm32_adc_regs or_vdd;
+ const struct stm32_adc_regs or_vddcore;
+ const struct stm32_adc_regs or_vddcpu;
+ const struct stm32_adc_regs or_vddq_ddr;
const struct stm32_adc_regs ccr_vbat;
const struct stm32_adc_regs ccr_vref;
};
* @trigs: external trigger sources
* @clk_required: clock is required
* @has_vregready: vregready status flag presence
+ * @has_boostmode: boost mode support flag
+ * @has_linearcal: linear calibration support flag
+ * @has_presel: channel preselection support flag
* @prepare: optional prepare routine (power-up, enable)
* @start_conv: routine to start conversions
* @stop_conv: routine to stop conversions
* @unprepare: optional unprepare routine (disable, power-down)
* @irq_clear: routine to clear irqs
* @smp_cycles: programmable sampling time (ADC clock cycles)
- * @ts_vrefint_ns: vrefint minimum sampling time in ns
+ * @ts_int_ch: pointer to array of internal channels minimum sampling time in ns
*/
struct stm32_adc_cfg {
const struct stm32_adc_regspec *regs;
struct stm32_adc_trig_info *trigs;
bool clk_required;
bool has_vregready;
+ bool has_boostmode;
+ bool has_linearcal;
+ bool has_presel;
int (*prepare)(struct iio_dev *);
void (*start_conv)(struct iio_dev *, bool dma);
void (*stop_conv)(struct iio_dev *);
void (*unprepare)(struct iio_dev *);
void (*irq_clear)(struct iio_dev *indio_dev, u32 msk);
const unsigned int *smp_cycles;
- const unsigned int ts_vrefint_ns;
+ const unsigned int *ts_int_ch;
};
/**
.num_res = ARRAY_SIZE(stm32h7_adc_resolutions),
};
+/* stm32mp13 can have up to 19 channels */
+static const struct stm32_adc_info stm32mp13_adc_info = {
+ .max_channels = 19,
+ .resolutions = stm32f4_adc_resolutions,
+ .num_res = ARRAY_SIZE(stm32f4_adc_resolutions),
+};
+
/*
* stm32f4_sq - describe regular sequence registers
* - L: sequence len (register & bit field)
.extsel = { STM32H7_ADC_CFGR, STM32H7_EXTSEL_MASK,
STM32H7_EXTSEL_SHIFT },
.res = { STM32H7_ADC_CFGR, STM32H7_RES_MASK, STM32H7_RES_SHIFT },
+ .difsel = { STM32H7_ADC_DIFSEL, STM32H7_DIFSEL_MASK},
.smpr = { STM32H7_ADC_SMPR1, STM32H7_ADC_SMPR2 },
.smp_bits = stm32h7_smp_bits,
};
+/* STM32MP13 programmable sampling time (ADC clock cycles, rounded down) */
+static const unsigned int stm32mp13_adc_smp_cycles[STM32_ADC_MAX_SMP + 1] = {
+ 2, 6, 12, 24, 47, 92, 247, 640,
+};
+
+static const struct stm32_adc_regspec stm32mp13_adc_regspec = {
+ .dr = STM32H7_ADC_DR,
+ .ier_eoc = { STM32H7_ADC_IER, STM32H7_EOCIE },
+ .ier_ovr = { STM32H7_ADC_IER, STM32H7_OVRIE },
+ .isr_eoc = { STM32H7_ADC_ISR, STM32H7_EOC },
+ .isr_ovr = { STM32H7_ADC_ISR, STM32H7_OVR },
+ .sqr = stm32h7_sq,
+ .exten = { STM32H7_ADC_CFGR, STM32H7_EXTEN_MASK, STM32H7_EXTEN_SHIFT },
+ .extsel = { STM32H7_ADC_CFGR, STM32H7_EXTSEL_MASK,
+ STM32H7_EXTSEL_SHIFT },
+ .res = { STM32H7_ADC_CFGR, STM32MP13_RES_MASK, STM32MP13_RES_SHIFT },
+ .difsel = { STM32MP13_ADC_DIFSEL, STM32MP13_DIFSEL_MASK},
+ .smpr = { STM32H7_ADC_SMPR1, STM32H7_ADC_SMPR2 },
+ .smp_bits = stm32h7_smp_bits,
+ .or_vddcore = { STM32MP13_ADC2_OR, STM32MP13_OP0 },
+ .or_vddcpu = { STM32MP13_ADC2_OR, STM32MP13_OP1 },
+ .or_vddq_ddr = { STM32MP13_ADC2_OR, STM32MP13_OP2 },
+ .ccr_vbat = { STM32H7_ADC_CCR, STM32H7_VBATEN },
+ .ccr_vref = { STM32H7_ADC_CCR, STM32H7_VREFEN },
+};
+
static const struct stm32_adc_regspec stm32mp1_adc_regspec = {
.dr = STM32H7_ADC_DR,
.ier_eoc = { STM32H7_ADC_IER, STM32H7_EOCIE },
.extsel = { STM32H7_ADC_CFGR, STM32H7_EXTSEL_MASK,
STM32H7_EXTSEL_SHIFT },
.res = { STM32H7_ADC_CFGR, STM32H7_RES_MASK, STM32H7_RES_SHIFT },
+ .difsel = { STM32H7_ADC_DIFSEL, STM32H7_DIFSEL_MASK},
.smpr = { STM32H7_ADC_SMPR1, STM32H7_ADC_SMPR2 },
.smp_bits = stm32h7_smp_bits,
- .or_vdd = { STM32MP1_ADC2_OR, STM32MP1_VDDCOREEN },
+ .or_vddcore = { STM32MP1_ADC2_OR, STM32MP1_VDDCOREEN },
.ccr_vbat = { STM32H7_ADC_CCR, STM32H7_VBATEN },
.ccr_vref = { STM32H7_ADC_CCR, STM32H7_VREFEN },
};
switch (i) {
case STM32_ADC_INT_CH_VDDCORE:
dev_dbg(&indio_dev->dev, "Enable VDDCore\n");
- stm32_adc_set_bits(adc, adc->cfg->regs->or_vdd.reg,
- adc->cfg->regs->or_vdd.mask);
+ stm32_adc_set_bits(adc, adc->cfg->regs->or_vddcore.reg,
+ adc->cfg->regs->or_vddcore.mask);
+ break;
+ case STM32_ADC_INT_CH_VDDCPU:
+ dev_dbg(&indio_dev->dev, "Enable VDDCPU\n");
+ stm32_adc_set_bits(adc, adc->cfg->regs->or_vddcpu.reg,
+ adc->cfg->regs->or_vddcpu.mask);
+ break;
+ case STM32_ADC_INT_CH_VDDQ_DDR:
+ dev_dbg(&indio_dev->dev, "Enable VDDQ_DDR\n");
+ stm32_adc_set_bits(adc, adc->cfg->regs->or_vddq_ddr.reg,
+ adc->cfg->regs->or_vddq_ddr.mask);
break;
case STM32_ADC_INT_CH_VREFINT:
dev_dbg(&indio_dev->dev, "Enable VREFInt\n");
switch (i) {
case STM32_ADC_INT_CH_VDDCORE:
- stm32_adc_clr_bits(adc, adc->cfg->regs->or_vdd.reg,
- adc->cfg->regs->or_vdd.mask);
+ stm32_adc_clr_bits(adc, adc->cfg->regs->or_vddcore.reg,
+ adc->cfg->regs->or_vddcore.mask);
+ break;
+ case STM32_ADC_INT_CH_VDDCPU:
+ stm32_adc_clr_bits(adc, adc->cfg->regs->or_vddcpu.reg,
+ adc->cfg->regs->or_vddcpu.mask);
+ break;
+ case STM32_ADC_INT_CH_VDDQ_DDR:
+ stm32_adc_clr_bits(adc, adc->cfg->regs->or_vddq_ddr.reg,
+ adc->cfg->regs->or_vddq_ddr.mask);
break;
case STM32_ADC_INT_CH_VREFINT:
stm32_adc_clr_bits_common(adc, adc->cfg->regs->ccr_vref.reg,
if (ret)
dev_warn(&indio_dev->dev, "stop failed\n");
+ /* STM32H7_DMNGT_MASK covers STM32MP13_DMAEN & STM32MP13_DMACFG */
stm32_adc_clr_bits(adc, STM32H7_ADC_CFGR, STM32H7_DMNGT_MASK);
}
stm32_adc_set_bits(adc, adc->cfg->regs->isr_eoc.reg, msk);
}
+static void stm32mp13_adc_start_conv(struct iio_dev *indio_dev, bool dma)
+{
+ struct stm32_adc *adc = iio_priv(indio_dev);
+
+ if (dma)
+ stm32_adc_set_bits(adc, STM32H7_ADC_CFGR,
+ STM32MP13_DMAEN | STM32MP13_DMACFG);
+
+ stm32_adc_set_bits(adc, STM32H7_ADC_CR, STM32H7_ADSTART);
+}
+
static int stm32h7_adc_exit_pwr_down(struct iio_dev *indio_dev)
{
struct stm32_adc *adc = iio_priv(indio_dev);
stm32_adc_clr_bits(adc, STM32H7_ADC_CR, STM32H7_DEEPPWD);
stm32_adc_set_bits(adc, STM32H7_ADC_CR, STM32H7_ADVREGEN);
- if (adc->common->rate > STM32H7_BOOST_CLKRATE)
+ if (adc->cfg->has_boostmode &&
+ adc->common->rate > STM32H7_BOOST_CLKRATE)
stm32_adc_set_bits(adc, STM32H7_ADC_CR, STM32H7_BOOST);
/* Wait for startup time */
static void stm32h7_adc_enter_pwr_down(struct stm32_adc *adc)
{
- stm32_adc_clr_bits(adc, STM32H7_ADC_CR, STM32H7_BOOST);
+ if (adc->cfg->has_boostmode)
+ stm32_adc_clr_bits(adc, STM32H7_ADC_CR, STM32H7_BOOST);
/* Setting DEEPPWD disables ADC vreg and clears ADVREGEN */
stm32_adc_set_bits(adc, STM32H7_ADC_CR, STM32H7_DEEPPWD);
lincalrdyw_mask >>= 1;
}
-
- /* Read offset calibration */
- val = stm32_adc_readl(adc, STM32H7_ADC_CALFACT);
- adc->cal.calfact_s = (val & STM32H7_CALFACT_S_MASK);
- adc->cal.calfact_s >>= STM32H7_CALFACT_S_SHIFT;
- adc->cal.calfact_d = (val & STM32H7_CALFACT_D_MASK);
- adc->cal.calfact_d >>= STM32H7_CALFACT_D_SHIFT;
- adc->cal.calibrated = true;
+ adc->cal.lincal_saved = true;
return 0;
}
int i, ret;
u32 lincalrdyw_mask, val;
- val = (adc->cal.calfact_s << STM32H7_CALFACT_S_SHIFT) |
- (adc->cal.calfact_d << STM32H7_CALFACT_D_SHIFT);
- stm32_adc_writel(adc, STM32H7_ADC_CALFACT, val);
-
lincalrdyw_mask = STM32H7_LINCALRDYW6;
for (i = STM32H7_LINCALFACT_NUM - 1; i >= 0; i--) {
/*
/**
* stm32h7_adc_selfcalib() - Procedure to calibrate ADC
* @indio_dev: IIO device instance
+ * @do_lincal: linear calibration request flag
* Note: Must be called once ADC is out of power down.
+ *
+ * Run offset calibration unconditionally.
+ * Run linear calibration if requested & supported.
*/
-static int stm32h7_adc_selfcalib(struct iio_dev *indio_dev)
+static int stm32h7_adc_selfcalib(struct iio_dev *indio_dev, int do_lincal)
{
struct stm32_adc *adc = iio_priv(indio_dev);
int ret;
+ u32 msk = STM32H7_ADCALDIF;
u32 val;
- if (adc->cal.calibrated)
- return true;
-
+ if (adc->cfg->has_linearcal && do_lincal)
+ msk |= STM32H7_ADCALLIN;
/* ADC must be disabled for calibration */
stm32h7_adc_disable(indio_dev);
* - Offset calibration for single ended inputs
* - No linearity calibration (do it later, before reading it)
*/
- stm32_adc_clr_bits(adc, STM32H7_ADC_CR, STM32H7_ADCALDIF);
- stm32_adc_clr_bits(adc, STM32H7_ADC_CR, STM32H7_ADCALLIN);
+ stm32_adc_clr_bits(adc, STM32H7_ADC_CR, msk);
/* Start calibration, then wait for completion */
stm32_adc_set_bits(adc, STM32H7_ADC_CR, STM32H7_ADCAL);
!(val & STM32H7_ADCAL), 100,
STM32H7_ADC_CALIB_TIMEOUT_US);
if (ret) {
- dev_err(&indio_dev->dev, "calibration failed\n");
+ dev_err(&indio_dev->dev, "calibration (single-ended) error %d\n", ret);
goto out;
}
* - Linearity calibration (needs to be done only once for single/diff)
* will run simultaneously with offset calibration.
*/
- stm32_adc_set_bits(adc, STM32H7_ADC_CR,
- STM32H7_ADCALDIF | STM32H7_ADCALLIN);
+ stm32_adc_set_bits(adc, STM32H7_ADC_CR, msk);
stm32_adc_set_bits(adc, STM32H7_ADC_CR, STM32H7_ADCAL);
ret = stm32_adc_readl_poll_timeout(STM32H7_ADC_CR, val,
!(val & STM32H7_ADCAL), 100,
STM32H7_ADC_CALIB_TIMEOUT_US);
if (ret) {
- dev_err(&indio_dev->dev, "calibration failed\n");
+ dev_err(&indio_dev->dev, "calibration (diff%s) error %d\n",
+ (msk & STM32H7_ADCALLIN) ? "+linear" : "", ret);
goto out;
}
out:
- stm32_adc_clr_bits(adc, STM32H7_ADC_CR,
- STM32H7_ADCALDIF | STM32H7_ADCALLIN);
+ stm32_adc_clr_bits(adc, STM32H7_ADC_CR, msk);
return ret;
}
/**
+ * stm32h7_adc_check_selfcalib() - Check linear calibration status
+ * @indio_dev: IIO device instance
+ *
+ * Used to check if linear calibration has been done.
+ * Return true if linear calibration factors are already saved in private data
+ * or if a linear calibration has been done at boot stage.
+ */
+static int stm32h7_adc_check_selfcalib(struct iio_dev *indio_dev)
+{
+ struct stm32_adc *adc = iio_priv(indio_dev);
+ u32 val;
+
+ if (adc->cal.lincal_saved)
+ return true;
+
+ /*
+ * Check if linear calibration factors are available in ADC registers,
+ * by checking that all LINCALRDYWx bits are set.
+ */
+ val = stm32_adc_readl(adc, STM32H7_ADC_CR) & STM32H7_LINCALRDYW_MASK;
+ if (val == STM32H7_LINCALRDYW_MASK)
+ return true;
+
+ return false;
+}
+
+/**
* stm32h7_adc_prepare() - Leave power down mode to enable ADC.
* @indio_dev: IIO device instance
* Leave power down mode.
static int stm32h7_adc_prepare(struct iio_dev *indio_dev)
{
struct stm32_adc *adc = iio_priv(indio_dev);
- int calib, ret;
+ int lincal_done = false;
+ int ret;
ret = stm32h7_adc_exit_pwr_down(indio_dev);
if (ret)
return ret;
- ret = stm32h7_adc_selfcalib(indio_dev);
+ if (adc->cfg->has_linearcal)
+ lincal_done = stm32h7_adc_check_selfcalib(indio_dev);
+
+ /* Always run offset calibration. Run linear calibration only once */
+ ret = stm32h7_adc_selfcalib(indio_dev, !lincal_done);
if (ret < 0)
goto pwr_dwn;
- calib = ret;
stm32_adc_int_ch_enable(indio_dev);
- stm32_adc_writel(adc, STM32H7_ADC_DIFSEL, adc->difsel);
+ stm32_adc_writel(adc, adc->cfg->regs->difsel.reg, adc->difsel);
ret = stm32h7_adc_enable(indio_dev);
if (ret)
goto ch_disable;
- /* Either restore or read calibration result for future reference */
- if (calib)
- ret = stm32h7_adc_restore_selfcalib(indio_dev);
- else
- ret = stm32h7_adc_read_selfcalib(indio_dev);
- if (ret)
- goto disable;
+ if (adc->cfg->has_linearcal) {
+ if (!adc->cal.lincal_saved)
+ ret = stm32h7_adc_read_selfcalib(indio_dev);
+ else
+ ret = stm32h7_adc_restore_selfcalib(indio_dev);
+
+ if (ret)
+ goto disable;
+ }
- stm32_adc_writel(adc, STM32H7_ADC_PCSEL, adc->pcsel);
+ if (adc->cfg->has_presel)
+ stm32_adc_writel(adc, STM32H7_ADC_PCSEL, adc->pcsel);
return 0;
{
struct stm32_adc *adc = iio_priv(indio_dev);
- stm32_adc_writel(adc, STM32H7_ADC_PCSEL, 0);
+ if (adc->cfg->has_presel)
+ stm32_adc_writel(adc, STM32H7_ADC_PCSEL, 0);
stm32h7_adc_disable(indio_dev);
stm32_adc_int_ch_disable(adc);
stm32h7_adc_enter_pwr_down(adc);
{},
};
+static void stm32_adc_debugfs_init(struct iio_dev *indio_dev)
+{
+ struct stm32_adc *adc = iio_priv(indio_dev);
+ struct dentry *d = iio_get_debugfs_dentry(indio_dev);
+ struct stm32_adc_calib *cal = &adc->cal;
+ char buf[16];
+ unsigned int i;
+
+ if (!adc->cfg->has_linearcal)
+ return;
+
+ for (i = 0; i < STM32H7_LINCALFACT_NUM; i++) {
+ snprintf(buf, sizeof(buf), "lincalfact%d", i + 1);
+ debugfs_create_u32(buf, 0444, d, &cal->lincalfact[i]);
+ }
+}
+
static int stm32_adc_fw_get_resolution(struct iio_dev *indio_dev)
{
struct device *dev = &indio_dev->dev;
{
const struct stm32_adc_regs *smpr = &adc->cfg->regs->smp_bits[channel];
u32 period_ns, shift = smpr->shift, mask = smpr->mask;
- unsigned int smp, r = smpr->reg;
+ unsigned int i, smp, r = smpr->reg;
/*
- * For vrefint channel, ensure that the sampling time cannot
+ * For internal channels, ensure that the sampling time cannot
* be lower than the one specified in the datasheet
*/
- if (channel == adc->int_ch[STM32_ADC_INT_CH_VREFINT])
- smp_ns = max(smp_ns, adc->cfg->ts_vrefint_ns);
+ for (i = 0; i < STM32_ADC_INT_CH_NB; i++)
+ if (channel == adc->int_ch[i] && adc->int_ch[i] != STM32_ADC_INT_CH_NONE)
+ smp_ns = max(smp_ns, adc->cfg->ts_int_ch[i]);
/* Determine sampling time (ADC clock cycles) */
period_ns = NSEC_PER_SEC / adc->common->rate;
adc->pcsel |= BIT(chan->channel);
if (differential) {
/* pre-build diff channels mask */
- adc->difsel |= BIT(chan->channel);
+ adc->difsel |= BIT(chan->channel) & adc->cfg->regs->difsel.mask;
/* Also add negative input to pre-selected channels */
adc->pcsel |= BIT(chan->channel2);
}
for (i = 0; i < STM32_ADC_INT_CH_NB; i++) {
if (!strncmp(stm32_adc_ic[i].name, ch_name, STM32_ADC_CH_SZ)) {
+ /* Check internal channel availability */
+ switch (i) {
+ case STM32_ADC_INT_CH_VDDCORE:
+ if (!adc->cfg->regs->or_vddcore.reg)
+ dev_warn(&indio_dev->dev,
+ "%s channel not available\n", ch_name);
+ break;
+ case STM32_ADC_INT_CH_VDDCPU:
+ if (!adc->cfg->regs->or_vddcpu.reg)
+ dev_warn(&indio_dev->dev,
+ "%s channel not available\n", ch_name);
+ break;
+ case STM32_ADC_INT_CH_VDDQ_DDR:
+ if (!adc->cfg->regs->or_vddq_ddr.reg)
+ dev_warn(&indio_dev->dev,
+ "%s channel not available\n", ch_name);
+ break;
+ case STM32_ADC_INT_CH_VREFINT:
+ if (!adc->cfg->regs->ccr_vref.reg)
+ dev_warn(&indio_dev->dev,
+ "%s channel not available\n", ch_name);
+ break;
+ case STM32_ADC_INT_CH_VBAT:
+ if (!adc->cfg->regs->ccr_vbat.reg)
+ dev_warn(&indio_dev->dev,
+ "%s channel not available\n", ch_name);
+ break;
+ }
+
if (stm32_adc_ic[i].idx != STM32_ADC_INT_CH_VREFINT) {
adc->int_ch[i] = chan;
break;
pm_runtime_mark_last_busy(dev);
pm_runtime_put_autosuspend(dev);
+ if (IS_ENABLED(CONFIG_DEBUG_FS))
+ stm32_adc_debugfs_init(indio_dev);
+
return 0;
err_hw_stop:
struct stm32_adc *adc = iio_priv(indio_dev);
pm_runtime_get_sync(&pdev->dev);
+ /* iio_device_unregister() also removes debugfs entries */
iio_device_unregister(indio_dev);
stm32_adc_hw_stop(&pdev->dev);
pm_runtime_disable(&pdev->dev);
.irq_clear = stm32f4_adc_irq_clear,
};
+const unsigned int stm32_adc_min_ts_h7[] = { 0, 0, 0, 4300, 9000 };
+static_assert(ARRAY_SIZE(stm32_adc_min_ts_h7) == STM32_ADC_INT_CH_NB);
+
static const struct stm32_adc_cfg stm32h7_adc_cfg = {
.regs = &stm32h7_adc_regspec,
.adc_info = &stm32h7_adc_info,
.trigs = stm32h7_adc_trigs,
+ .has_boostmode = true,
+ .has_linearcal = true,
+ .has_presel = true,
.start_conv = stm32h7_adc_start_conv,
.stop_conv = stm32h7_adc_stop_conv,
.prepare = stm32h7_adc_prepare,
.unprepare = stm32h7_adc_unprepare,
.smp_cycles = stm32h7_adc_smp_cycles,
.irq_clear = stm32h7_adc_irq_clear,
+ .ts_int_ch = stm32_adc_min_ts_h7,
};
+const unsigned int stm32_adc_min_ts_mp1[] = { 100, 100, 100, 4300, 9800 };
+static_assert(ARRAY_SIZE(stm32_adc_min_ts_mp1) == STM32_ADC_INT_CH_NB);
+
static const struct stm32_adc_cfg stm32mp1_adc_cfg = {
.regs = &stm32mp1_adc_regspec,
.adc_info = &stm32h7_adc_info,
.trigs = stm32h7_adc_trigs,
.has_vregready = true,
+ .has_boostmode = true,
+ .has_linearcal = true,
+ .has_presel = true,
.start_conv = stm32h7_adc_start_conv,
.stop_conv = stm32h7_adc_stop_conv,
.prepare = stm32h7_adc_prepare,
.unprepare = stm32h7_adc_unprepare,
.smp_cycles = stm32h7_adc_smp_cycles,
.irq_clear = stm32h7_adc_irq_clear,
- .ts_vrefint_ns = 4300,
+ .ts_int_ch = stm32_adc_min_ts_mp1,
+};
+
+const unsigned int stm32_adc_min_ts_mp13[] = { 100, 0, 0, 4300, 9800 };
+static_assert(ARRAY_SIZE(stm32_adc_min_ts_mp13) == STM32_ADC_INT_CH_NB);
+
+static const struct stm32_adc_cfg stm32mp13_adc_cfg = {
+ .regs = &stm32mp13_adc_regspec,
+ .adc_info = &stm32mp13_adc_info,
+ .trigs = stm32h7_adc_trigs,
+ .start_conv = stm32mp13_adc_start_conv,
+ .stop_conv = stm32h7_adc_stop_conv,
+ .prepare = stm32h7_adc_prepare,
+ .unprepare = stm32h7_adc_unprepare,
+ .smp_cycles = stm32mp13_adc_smp_cycles,
+ .irq_clear = stm32h7_adc_irq_clear,
+ .ts_int_ch = stm32_adc_min_ts_mp13,
};
static const struct of_device_id stm32_adc_of_match[] = {
{ .compatible = "st,stm32f4-adc", .data = (void *)&stm32f4_adc_cfg },
{ .compatible = "st,stm32h7-adc", .data = (void *)&stm32h7_adc_cfg },
{ .compatible = "st,stm32mp1-adc", .data = (void *)&stm32mp1_adc_cfg },
+ { .compatible = "st,stm32mp13-adc", .data = (void *)&stm32mp13_adc_cfg },
{},
};
MODULE_DEVICE_TABLE(of, stm32_adc_of_match);
regulator_disable(reg);
}
-static int adc081c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int adc081c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct iio_dev *iio;
struct adc081c *adc;
const struct adcxx1c_model *model;
.of_match_table = adc081c_of_match,
.acpi_match_table = adc081c_acpi_match,
},
- .probe = adc081c_probe,
+ .probe_new = adc081c_probe,
.id_table = adc081c_id,
};
module_i2c_driver(adc081c_driver);
}
static const struct of_device_id adc128_of_match[] = {
- { .compatible = "ti,adc128s052", },
- { .compatible = "ti,adc122s021", },
- { .compatible = "ti,adc122s051", },
- { .compatible = "ti,adc122s101", },
- { .compatible = "ti,adc124s021", },
- { .compatible = "ti,adc124s051", },
- { .compatible = "ti,adc124s101", },
+ { .compatible = "ti,adc128s052", .data = (void*)0L, },
+ { .compatible = "ti,adc122s021", .data = (void*)1L, },
+ { .compatible = "ti,adc122s051", .data = (void*)1L, },
+ { .compatible = "ti,adc122s101", .data = (void*)1L, },
+ { .compatible = "ti,adc124s021", .data = (void*)2L, },
+ { .compatible = "ti,adc124s051", .data = (void*)2L, },
+ { .compatible = "ti,adc124s101", .data = (void*)2L, },
{ /* sentinel */ },
};
MODULE_DEVICE_TABLE(of, adc128_of_match);
mode << ADS1015_CFG_MOD_SHIFT);
}
-static int ads1015_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ads1015_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
const struct ads1015_chip_data *chip;
struct iio_dev *indio_dev;
struct ads1015_data *data;
.of_match_table = ads1015_of_match,
.pm = &ads1015_pm_ops,
},
- .probe = ads1015_probe,
+ .probe_new = ads1015_probe,
.remove = ads1015_remove,
.id_table = ads1015_id,
};
int ret;
info = device_get_match_data(&spi->dev);
+ if (!info)
+ info = (void *)spi_get_device_id(spi)->driver_data;
if (!info) {
dev_err(&spi->dev, "failed to get match data\n");
return -ENODEV;
};
MODULE_DEVICE_TABLE(of, ads131e08_of_match);
+static const struct spi_device_id ads131e08_ids[] = {
+ { "ads131e04", (kernel_ulong_t)&ads131e08_info_tbl[ads131e04] },
+ { "ads131e06", (kernel_ulong_t)&ads131e08_info_tbl[ads131e06] },
+ { "ads131e08", (kernel_ulong_t)&ads131e08_info_tbl[ads131e08] },
+ {}
+};
+MODULE_DEVICE_TABLE(spi, ads131e08_ids);
+
static struct spi_driver ads131e08_driver = {
.driver = {
.name = "ads131e08",
.of_match_table = ads131e08_of_match,
},
.probe = ads131e08_probe,
+ .id_table = ads131e08_ids,
};
module_spi_driver(ads131e08_driver);
#include <linux/mod_devicetable.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/property.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
void __iomem *regs;
struct clk *clk;
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
+
u32 vref_uv;
u32 value;
struct regulator *vref;
{
struct vf610_adc *info = iio_priv(indio_dev);
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&info->lock);
info->adc_feature.conv_mode = mode;
vf610_adc_calculate_rates(info);
vf610_adc_hw_init(info);
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&info->lock);
return 0;
}
.attrs = vf610_attributes,
};
+static int vf610_read_sample(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan, int *val)
+{
+ struct vf610_adc *info = iio_priv(indio_dev);
+ unsigned int hc_cfg;
+ int ret;
+
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
+
+ mutex_lock(&info->lock);
+ reinit_completion(&info->completion);
+ hc_cfg = VF610_ADC_ADCHC(chan->channel);
+ hc_cfg |= VF610_ADC_AIEN;
+ writel(hc_cfg, info->regs + VF610_REG_ADC_HC0);
+ ret = wait_for_completion_interruptible_timeout(&info->completion,
+ VF610_ADC_TIMEOUT);
+ if (ret == 0) {
+ ret = -ETIMEDOUT;
+ goto out_unlock;
+ }
+
+ if (ret < 0)
+ goto out_unlock;
+
+ switch (chan->type) {
+ case IIO_VOLTAGE:
+ *val = info->value;
+ break;
+ case IIO_TEMP:
+ /*
+ * Calculate in degree Celsius times 1000
+ * Using the typical sensor slope of 1.84 mV/°C
+ * and VREFH_ADC at 3.3V, V at 25°C of 699 mV
+ */
+ *val = 25000 - ((int)info->value - VF610_VTEMP25_3V3) *
+ 1000000 / VF610_TEMP_SLOPE_COEFF;
+
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+out_unlock:
+ mutex_unlock(&info->lock);
+ iio_device_release_direct_mode(indio_dev);
+
+ return ret;
+}
+
static int vf610_read_raw(struct iio_dev *indio_dev,
struct iio_chan_spec const *chan,
int *val,
long mask)
{
struct vf610_adc *info = iio_priv(indio_dev);
- unsigned int hc_cfg;
long ret;
switch (mask) {
case IIO_CHAN_INFO_RAW:
case IIO_CHAN_INFO_PROCESSED:
- mutex_lock(&indio_dev->mlock);
- if (iio_buffer_enabled(indio_dev)) {
- mutex_unlock(&indio_dev->mlock);
- return -EBUSY;
- }
-
- reinit_completion(&info->completion);
- hc_cfg = VF610_ADC_ADCHC(chan->channel);
- hc_cfg |= VF610_ADC_AIEN;
- writel(hc_cfg, info->regs + VF610_REG_ADC_HC0);
- ret = wait_for_completion_interruptible_timeout
- (&info->completion, VF610_ADC_TIMEOUT);
- if (ret == 0) {
- mutex_unlock(&indio_dev->mlock);
- return -ETIMEDOUT;
- }
- if (ret < 0) {
- mutex_unlock(&indio_dev->mlock);
+ ret = vf610_read_sample(indio_dev, chan, val);
+ if (ret < 0)
return ret;
- }
- switch (chan->type) {
- case IIO_VOLTAGE:
- *val = info->value;
- break;
- case IIO_TEMP:
- /*
- * Calculate in degree Celsius times 1000
- * Using the typical sensor slope of 1.84 mV/°C
- * and VREFH_ADC at 3.3V, V at 25°C of 699 mV
- */
- *val = 25000 - ((int)info->value - VF610_VTEMP25_3V3) *
- 1000000 / VF610_TEMP_SLOPE_COEFF;
-
- break;
- default:
- mutex_unlock(&indio_dev->mlock);
- return -EINVAL;
- }
-
- mutex_unlock(&indio_dev->mlock);
return IIO_VAL_INT;
case IIO_CHAN_INFO_SCALE:
goto error_iio_device_register;
}
+ mutex_init(&info->lock);
+
ret = iio_device_register(indio_dev);
if (ret) {
dev_err(&pdev->dev, "Couldn't register the device.\n");
menu "Analog to digital and digital to analog converters"
+config AD74115
+ tristate "Analog Devices AD74115H driver"
+ depends on GPIOLIB && SPI
+ select CRC8
+ select IIO_BUFFER
+ select IIO_TRIGGERED_BUFFER
+ select REGMAP_SPI
+ help
+ Say yes here to build support for Analog Devices AD74115H
+ single-channel software configurable input/output solution.
+
+ To compile this driver as a module, choose M here: the
+ module will be called ad74115.
+
config AD74413R
tristate "Analog Devices AD74412R/AD74413R driver"
depends on GPIOLIB && SPI
#
# When adding new entries keep the list in alphabetical order
+obj-$(CONFIG_AD74115) += ad74115.o
obj-$(CONFIG_AD74413R) += ad74413r.o
obj-$(CONFIG_STX104) += stx104.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2022 Analog Devices, Inc.
+ * Author: Cosmin Tanislav <cosmin.tanislav@analog.com>
+ */
+
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/crc8.h>
+#include <linux/device.h>
+#include <linux/gpio/driver.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/regmap.h>
+#include <linux/regulator/consumer.h>
+#include <linux/spi/spi.h>
+#include <linux/units.h>
+
+#include <asm/unaligned.h>
+
+#include <linux/iio/buffer.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/trigger.h>
+#include <linux/iio/trigger_consumer.h>
+#include <linux/iio/triggered_buffer.h>
+
+#define AD74115_NAME "ad74115"
+
+#define AD74115_CH_FUNC_SETUP_REG 0x01
+
+#define AD74115_ADC_CONFIG_REG 0x02
+#define AD74115_ADC_CONFIG_CONV2_RATE_MASK GENMASK(15, 13)
+#define AD74115_ADC_CONFIG_CONV1_RATE_MASK GENMASK(12, 10)
+#define AD74115_ADC_CONFIG_CONV2_RANGE_MASK GENMASK(9, 7)
+#define AD74115_ADC_CONFIG_CONV1_RANGE_MASK GENMASK(6, 4)
+
+#define AD74115_PWR_OPTIM_CONFIG_REG 0x03
+
+#define AD74115_DIN_CONFIG1_REG 0x04
+#define AD74115_DIN_COMPARATOR_EN_MASK BIT(13)
+#define AD74115_DIN_SINK_MASK GENMASK(11, 7)
+#define AD74115_DIN_DEBOUNCE_MASK GENMASK(4, 0)
+
+#define AD74115_DIN_CONFIG2_REG 0x05
+#define AD74115_COMP_THRESH_MASK GENMASK(6, 0)
+
+#define AD74115_OUTPUT_CONFIG_REG 0x06
+#define AD74115_OUTPUT_SLEW_EN_MASK GENMASK(6, 5)
+#define AD74115_OUTPUT_SLEW_LIN_STEP_MASK GENMASK(4, 3)
+#define AD74115_OUTPUT_SLEW_LIN_RATE_MASK GENMASK(2, 1)
+
+#define AD74115_RTD3W4W_CONFIG_REG 0x07
+
+#define AD74115_BURNOUT_CONFIG_REG 0x0a
+#define AD74115_BURNOUT_EXT2_EN_MASK BIT(10)
+#define AD74115_BURNOUT_EXT1_EN_MASK BIT(5)
+#define AD74115_BURNOUT_VIOUT_EN_MASK BIT(0)
+
+#define AD74115_DAC_CODE_REG 0x0b
+
+#define AD74115_DAC_ACTIVE_REG 0x0d
+
+#define AD74115_GPIO_CONFIG_X_REG(x) (0x35 + (x))
+#define AD74115_GPIO_CONFIG_GPI_DATA BIT(5)
+#define AD74115_GPIO_CONFIG_GPO_DATA BIT(4)
+#define AD74115_GPIO_CONFIG_SELECT_MASK GENMASK(2, 0)
+
+#define AD74115_CHARGE_PUMP_REG 0x3a
+
+#define AD74115_ADC_CONV_CTRL_REG 0x3b
+#define AD74115_ADC_CONV_SEQ_MASK GENMASK(13, 12)
+
+#define AD74115_DIN_COMP_OUT_REG 0x40
+
+#define AD74115_LIVE_STATUS_REG 0x42
+#define AD74115_ADC_DATA_RDY_MASK BIT(3)
+
+#define AD74115_READ_SELECT_REG 0x64
+
+#define AD74115_CMD_KEY_REG 0x78
+#define AD74115_CMD_KEY_RESET1 0x15fa
+#define AD74115_CMD_KEY_RESET2 0xaf51
+
+#define AD74115_CRC_POLYNOMIAL 0x7
+DECLARE_CRC8_TABLE(ad74115_crc8_table);
+
+#define AD74115_ADC_CODE_MAX ((int)GENMASK(15, 0))
+#define AD74115_ADC_CODE_HALF (AD74115_ADC_CODE_MAX / 2)
+
+#define AD74115_DAC_VOLTAGE_MAX 12000
+#define AD74115_DAC_CURRENT_MAX 25
+#define AD74115_DAC_CODE_MAX ((int)GENMASK(13, 0))
+#define AD74115_DAC_CODE_HALF (AD74115_DAC_CODE_MAX / 2)
+
+#define AD74115_COMP_THRESH_MAX 98
+
+#define AD74115_SENSE_RESISTOR_OHMS 100
+#define AD74115_REF_RESISTOR_OHMS 2100
+
+#define AD74115_DIN_SINK_LOW_STEP 120
+#define AD74115_DIN_SINK_HIGH_STEP 240
+#define AD74115_DIN_SINK_MAX 31
+
+#define AD74115_FRAME_SIZE 4
+#define AD74115_GPIO_NUM 4
+
+#define AD74115_CONV_TIME_US 1000000
+
+enum ad74115_dac_ch {
+ AD74115_DAC_CH_MAIN,
+ AD74115_DAC_CH_COMPARATOR,
+};
+
+enum ad74115_adc_ch {
+ AD74115_ADC_CH_CONV1,
+ AD74115_ADC_CH_CONV2,
+ AD74115_ADC_CH_NUM
+};
+
+enum ad74115_ch_func {
+ AD74115_CH_FUNC_HIGH_IMPEDANCE,
+ AD74115_CH_FUNC_VOLTAGE_OUTPUT,
+ AD74115_CH_FUNC_CURRENT_OUTPUT,
+ AD74115_CH_FUNC_VOLTAGE_INPUT,
+ AD74115_CH_FUNC_CURRENT_INPUT_EXT_POWER,
+ AD74115_CH_FUNC_CURRENT_INPUT_LOOP_POWER,
+ AD74115_CH_FUNC_2_WIRE_RESISTANCE_INPUT,
+ AD74115_CH_FUNC_3_4_WIRE_RESISTANCE_INPUT,
+ AD74115_CH_FUNC_DIGITAL_INPUT_LOGIC,
+ AD74115_CH_FUNC_DIGITAL_INPUT_LOOP_POWER,
+ AD74115_CH_FUNC_CURRENT_OUTPUT_HART,
+ AD74115_CH_FUNC_CURRENT_INPUT_EXT_POWER_HART,
+ AD74115_CH_FUNC_CURRENT_INPUT_LOOP_POWER_HART,
+ AD74115_CH_FUNC_MAX = AD74115_CH_FUNC_CURRENT_INPUT_LOOP_POWER_HART,
+ AD74115_CH_FUNC_NUM
+};
+
+enum ad74115_adc_range {
+ AD74115_ADC_RANGE_12V,
+ AD74115_ADC_RANGE_12V_BIPOLAR,
+ AD74115_ADC_RANGE_2_5V_BIPOLAR,
+ AD74115_ADC_RANGE_2_5V_NEG,
+ AD74115_ADC_RANGE_2_5V,
+ AD74115_ADC_RANGE_0_625V,
+ AD74115_ADC_RANGE_104MV_BIPOLAR,
+ AD74115_ADC_RANGE_12V_OTHER,
+ AD74115_ADC_RANGE_MAX = AD74115_ADC_RANGE_12V_OTHER,
+ AD74115_ADC_RANGE_NUM
+};
+
+enum ad74115_adc_conv_seq {
+ AD74115_ADC_CONV_SEQ_STANDBY = 0b00,
+ AD74115_ADC_CONV_SEQ_SINGLE = 0b01,
+ AD74115_ADC_CONV_SEQ_CONTINUOUS = 0b10,
+};
+
+enum ad74115_din_threshold_mode {
+ AD74115_DIN_THRESHOLD_MODE_AVDD,
+ AD74115_DIN_THRESHOLD_MODE_FIXED,
+ AD74115_DIN_THRESHOLD_MODE_MAX = AD74115_DIN_THRESHOLD_MODE_FIXED,
+};
+
+enum ad74115_slew_mode {
+ AD74115_SLEW_MODE_DISABLED,
+ AD74115_SLEW_MODE_LINEAR,
+ AD74115_SLEW_MODE_HART,
+};
+
+enum ad74115_slew_step {
+ AD74115_SLEW_STEP_0_8_PERCENT,
+ AD74115_SLEW_STEP_1_5_PERCENT,
+ AD74115_SLEW_STEP_6_1_PERCENT,
+ AD74115_SLEW_STEP_22_2_PERCENT,
+};
+
+enum ad74115_slew_rate {
+ AD74115_SLEW_RATE_4KHZ,
+ AD74115_SLEW_RATE_64KHZ,
+ AD74115_SLEW_RATE_150KHZ,
+ AD74115_SLEW_RATE_240KHZ,
+};
+
+enum ad74115_gpio_config {
+ AD74115_GPIO_CONFIG_OUTPUT_BUFFERED = 0b010,
+ AD74115_GPIO_CONFIG_INPUT = 0b011,
+};
+
+enum ad74115_gpio_mode {
+ AD74115_GPIO_MODE_LOGIC = 1,
+ AD74115_GPIO_MODE_SPECIAL = 2,
+};
+
+struct ad74115_channels {
+ struct iio_chan_spec *channels;
+ unsigned int num_channels;
+};
+
+struct ad74115_state {
+ struct spi_device *spi;
+ struct regmap *regmap;
+ struct iio_trigger *trig;
+ struct regulator *avdd;
+
+ /*
+ * Synchronize consecutive operations when doing a one-shot
+ * conversion and when updating the ADC samples SPI message.
+ */
+ struct mutex lock;
+ struct gpio_chip gc;
+ struct gpio_chip comp_gc;
+ int irq;
+
+ unsigned int avdd_mv;
+ unsigned long gpio_valid_mask;
+ bool dac_bipolar;
+ bool dac_hart_slew;
+ bool rtd_mode_4_wire;
+ enum ad74115_ch_func ch_func;
+ enum ad74115_din_threshold_mode din_threshold_mode;
+
+ struct completion adc_data_completion;
+ struct spi_message adc_samples_msg;
+ struct spi_transfer adc_samples_xfer[AD74115_ADC_CH_NUM + 1];
+
+ /*
+ * DMA (thus cache coherency maintenance) requires the
+ * transfer buffers to live in their own cache lines.
+ */
+ u8 reg_tx_buf[AD74115_FRAME_SIZE] __aligned(IIO_DMA_MINALIGN);
+ u8 reg_rx_buf[AD74115_FRAME_SIZE];
+ u8 adc_samples_tx_buf[AD74115_FRAME_SIZE * AD74115_ADC_CH_NUM];
+ u8 adc_samples_rx_buf[AD74115_FRAME_SIZE * AD74115_ADC_CH_NUM];
+};
+
+struct ad74115_fw_prop {
+ const char *name;
+ bool is_boolean;
+ bool negate;
+ unsigned int max;
+ unsigned int reg;
+ unsigned int mask;
+ const unsigned int *lookup_tbl;
+ unsigned int lookup_tbl_len;
+};
+
+#define AD74115_FW_PROP(_name, _max, _reg, _mask) \
+{ \
+ .name = (_name), \
+ .max = (_max), \
+ .reg = (_reg), \
+ .mask = (_mask), \
+}
+
+#define AD74115_FW_PROP_TBL(_name, _tbl, _reg, _mask) \
+{ \
+ .name = (_name), \
+ .reg = (_reg), \
+ .mask = (_mask), \
+ .lookup_tbl = (_tbl), \
+ .lookup_tbl_len = ARRAY_SIZE(_tbl), \
+}
+
+#define AD74115_FW_PROP_BOOL(_name, _reg, _mask) \
+{ \
+ .name = (_name), \
+ .is_boolean = true, \
+ .reg = (_reg), \
+ .mask = (_mask), \
+}
+
+#define AD74115_FW_PROP_BOOL_NEG(_name, _reg, _mask) \
+{ \
+ .name = (_name), \
+ .is_boolean = true, \
+ .negate = true, \
+ .reg = (_reg), \
+ .mask = (_mask), \
+}
+
+static const int ad74115_dac_rate_tbl[] = {
+ 0,
+ 4 * 8,
+ 4 * 15,
+ 4 * 61,
+ 4 * 222,
+ 64 * 8,
+ 64 * 15,
+ 64 * 61,
+ 64 * 222,
+ 150 * 8,
+ 150 * 15,
+ 150 * 61,
+ 150 * 222,
+ 240 * 8,
+ 240 * 15,
+ 240 * 61,
+ 240 * 222,
+};
+
+static const unsigned int ad74115_dac_rate_step_tbl[][3] = {
+ { AD74115_SLEW_MODE_DISABLED },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_0_8_PERCENT, AD74115_SLEW_RATE_4KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_1_5_PERCENT, AD74115_SLEW_RATE_4KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_6_1_PERCENT, AD74115_SLEW_RATE_4KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_22_2_PERCENT, AD74115_SLEW_RATE_4KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_0_8_PERCENT, AD74115_SLEW_RATE_64KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_1_5_PERCENT, AD74115_SLEW_RATE_64KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_6_1_PERCENT, AD74115_SLEW_RATE_64KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_22_2_PERCENT, AD74115_SLEW_RATE_64KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_0_8_PERCENT, AD74115_SLEW_RATE_150KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_1_5_PERCENT, AD74115_SLEW_RATE_150KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_6_1_PERCENT, AD74115_SLEW_RATE_150KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_22_2_PERCENT, AD74115_SLEW_RATE_150KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_0_8_PERCENT, AD74115_SLEW_RATE_240KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_1_5_PERCENT, AD74115_SLEW_RATE_240KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_6_1_PERCENT, AD74115_SLEW_RATE_240KHZ },
+ { AD74115_SLEW_MODE_LINEAR, AD74115_SLEW_STEP_22_2_PERCENT, AD74115_SLEW_RATE_240KHZ },
+};
+
+static const unsigned int ad74115_rtd_excitation_current_ua_tbl[] = {
+ 250, 500, 750, 1000
+};
+
+static const unsigned int ad74115_burnout_current_na_tbl[] = {
+ 0, 50, 0, 500, 1000, 0, 10000, 0
+};
+
+static const unsigned int ad74115_viout_burnout_current_na_tbl[] = {
+ 0, 0, 0, 0, 1000, 0, 10000, 0
+};
+
+static const unsigned int ad74115_gpio_mode_tbl[] = {
+ 0, 0, 0, 1, 2, 3, 4, 5
+};
+
+static const unsigned int ad74115_adc_conv_rate_tbl[] = {
+ 10, 20, 1200, 4800, 9600
+};
+
+static const unsigned int ad74115_debounce_tbl[] = {
+ 0, 13, 18, 24, 32, 42, 56, 75,
+ 100, 130, 180, 240, 320, 420, 560, 750,
+ 1000, 1300, 1800, 2400, 3200, 4200, 5600, 7500,
+ 10000, 13000, 18000, 24000, 32000, 42000, 56000, 75000,
+};
+
+static const unsigned int ad74115_adc_ch_data_regs_tbl[] = {
+ [AD74115_ADC_CH_CONV1] = 0x44,
+ [AD74115_ADC_CH_CONV2] = 0x46,
+};
+
+static const unsigned int ad74115_adc_ch_en_bit_tbl[] = {
+ [AD74115_ADC_CH_CONV1] = BIT(0),
+ [AD74115_ADC_CH_CONV2] = BIT(1),
+};
+
+static const bool ad74115_adc_bipolar_tbl[AD74115_ADC_RANGE_NUM] = {
+ [AD74115_ADC_RANGE_12V_BIPOLAR] = true,
+ [AD74115_ADC_RANGE_2_5V_BIPOLAR] = true,
+ [AD74115_ADC_RANGE_104MV_BIPOLAR] = true,
+};
+
+static const unsigned int ad74115_adc_conv_mul_tbl[AD74115_ADC_RANGE_NUM] = {
+ [AD74115_ADC_RANGE_12V] = 12000,
+ [AD74115_ADC_RANGE_12V_BIPOLAR] = 24000,
+ [AD74115_ADC_RANGE_2_5V_BIPOLAR] = 5000,
+ [AD74115_ADC_RANGE_2_5V_NEG] = 2500,
+ [AD74115_ADC_RANGE_2_5V] = 2500,
+ [AD74115_ADC_RANGE_0_625V] = 625,
+ [AD74115_ADC_RANGE_104MV_BIPOLAR] = 208,
+ [AD74115_ADC_RANGE_12V_OTHER] = 12000,
+};
+
+static const unsigned int ad74115_adc_gain_tbl[AD74115_ADC_RANGE_NUM][2] = {
+ [AD74115_ADC_RANGE_12V] = { 5, 24 },
+ [AD74115_ADC_RANGE_12V_BIPOLAR] = { 5, 24 },
+ [AD74115_ADC_RANGE_2_5V_BIPOLAR] = { 1, 1 },
+ [AD74115_ADC_RANGE_2_5V_NEG] = { 1, 1 },
+ [AD74115_ADC_RANGE_2_5V] = { 1, 1 },
+ [AD74115_ADC_RANGE_0_625V] = { 4, 1 },
+ [AD74115_ADC_RANGE_104MV_BIPOLAR] = { 24, 1 },
+ [AD74115_ADC_RANGE_12V_OTHER] = { 5, 24 },
+};
+
+static const int ad74115_adc_range_tbl[AD74115_ADC_RANGE_NUM][2] = {
+ [AD74115_ADC_RANGE_12V] = { 0, 12000000 },
+ [AD74115_ADC_RANGE_12V_BIPOLAR] = { -12000000, 12000000 },
+ [AD74115_ADC_RANGE_2_5V_BIPOLAR] = { -2500000, 2500000 },
+ [AD74115_ADC_RANGE_2_5V_NEG] = { -2500000, 0 },
+ [AD74115_ADC_RANGE_2_5V] = { 0, 2500000 },
+ [AD74115_ADC_RANGE_0_625V] = { 0, 625000 },
+ [AD74115_ADC_RANGE_104MV_BIPOLAR] = { -104000, 104000 },
+ [AD74115_ADC_RANGE_12V_OTHER] = { 0, 12000000 },
+};
+
+static int _ad74115_find_tbl_index(const unsigned int *tbl, unsigned int tbl_len,
+ unsigned int val, unsigned int *index)
+{
+ unsigned int i;
+
+ for (i = 0; i < tbl_len; i++)
+ if (val == tbl[i]) {
+ *index = i;
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+#define ad74115_find_tbl_index(tbl, val, index) \
+ _ad74115_find_tbl_index(tbl, ARRAY_SIZE(tbl), val, index)
+
+static int ad74115_crc(u8 *buf)
+{
+ return crc8(ad74115_crc8_table, buf, 3, 0);
+}
+
+static void ad74115_format_reg_write(u8 reg, u16 val, u8 *buf)
+{
+ buf[0] = reg;
+ put_unaligned_be16(val, &buf[1]);
+ buf[3] = ad74115_crc(buf);
+}
+
+static int ad74115_reg_write(void *context, unsigned int reg, unsigned int val)
+{
+ struct ad74115_state *st = context;
+
+ ad74115_format_reg_write(reg, val, st->reg_tx_buf);
+
+ return spi_write(st->spi, st->reg_tx_buf, AD74115_FRAME_SIZE);
+}
+
+static int ad74115_crc_check(struct ad74115_state *st, u8 *buf)
+{
+ struct device *dev = &st->spi->dev;
+ u8 expected_crc = ad74115_crc(buf);
+
+ if (buf[3] != expected_crc) {
+ dev_err(dev, "Bad CRC %02x for %02x%02x%02x, expected %02x\n",
+ buf[3], buf[0], buf[1], buf[2], expected_crc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int ad74115_reg_read(void *context, unsigned int reg, unsigned int *val)
+{
+ struct ad74115_state *st = context;
+ struct spi_transfer reg_read_xfer[] = {
+ {
+ .tx_buf = st->reg_tx_buf,
+ .len = sizeof(st->reg_tx_buf),
+ .cs_change = 1,
+ },
+ {
+ .rx_buf = st->reg_rx_buf,
+ .len = sizeof(st->reg_rx_buf),
+ },
+ };
+ int ret;
+
+ ad74115_format_reg_write(AD74115_READ_SELECT_REG, reg, st->reg_tx_buf);
+
+ ret = spi_sync_transfer(st->spi, reg_read_xfer, ARRAY_SIZE(reg_read_xfer));
+ if (ret)
+ return ret;
+
+ ret = ad74115_crc_check(st, st->reg_rx_buf);
+ if (ret)
+ return ret;
+
+ *val = get_unaligned_be16(&st->reg_rx_buf[1]);
+
+ return 0;
+}
+
+static const struct regmap_config ad74115_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 16,
+ .reg_read = ad74115_reg_read,
+ .reg_write = ad74115_reg_write,
+};
+
+static int ad74115_gpio_config_set(struct ad74115_state *st, unsigned int offset,
+ enum ad74115_gpio_config cfg)
+{
+ return regmap_update_bits(st->regmap, AD74115_GPIO_CONFIG_X_REG(offset),
+ AD74115_GPIO_CONFIG_SELECT_MASK,
+ FIELD_PREP(AD74115_GPIO_CONFIG_SELECT_MASK, cfg));
+}
+
+static int ad74115_gpio_init_valid_mask(struct gpio_chip *gc,
+ unsigned long *valid_mask,
+ unsigned int ngpios)
+{
+ struct ad74115_state *st = gpiochip_get_data(gc);
+
+ *valid_mask = st->gpio_valid_mask;
+
+ return 0;
+}
+
+static int ad74115_gpio_get_direction(struct gpio_chip *gc, unsigned int offset)
+{
+ struct ad74115_state *st = gpiochip_get_data(gc);
+ unsigned int val;
+ int ret;
+
+ ret = regmap_read(st->regmap, AD74115_GPIO_CONFIG_X_REG(offset), &val);
+ if (ret)
+ return ret;
+
+ return FIELD_GET(AD74115_GPIO_CONFIG_SELECT_MASK, val) == AD74115_GPIO_CONFIG_INPUT;
+}
+
+static int ad74115_gpio_direction_input(struct gpio_chip *gc, unsigned int offset)
+{
+ struct ad74115_state *st = gpiochip_get_data(gc);
+
+ return ad74115_gpio_config_set(st, offset, AD74115_GPIO_CONFIG_INPUT);
+}
+
+static int ad74115_gpio_direction_output(struct gpio_chip *gc, unsigned int offset,
+ int value)
+{
+ struct ad74115_state *st = gpiochip_get_data(gc);
+
+ return ad74115_gpio_config_set(st, offset, AD74115_GPIO_CONFIG_OUTPUT_BUFFERED);
+}
+
+static int ad74115_gpio_get(struct gpio_chip *gc, unsigned int offset)
+{
+ struct ad74115_state *st = gpiochip_get_data(gc);
+ unsigned int val;
+ int ret;
+
+ ret = regmap_read(st->regmap, AD74115_GPIO_CONFIG_X_REG(offset), &val);
+ if (ret)
+ return ret;
+
+ return FIELD_GET(AD74115_GPIO_CONFIG_GPI_DATA, val);
+}
+
+static void ad74115_gpio_set(struct gpio_chip *gc, unsigned int offset, int value)
+{
+ struct ad74115_state *st = gpiochip_get_data(gc);
+ struct device *dev = &st->spi->dev;
+ int ret;
+
+ ret = regmap_update_bits(st->regmap, AD74115_GPIO_CONFIG_X_REG(offset),
+ AD74115_GPIO_CONFIG_GPO_DATA,
+ FIELD_PREP(AD74115_GPIO_CONFIG_GPO_DATA, value));
+ if (ret)
+ dev_err(dev, "Failed to set GPIO %u output value, err: %d\n",
+ offset, ret);
+}
+
+static int ad74115_set_comp_debounce(struct ad74115_state *st, unsigned int val)
+{
+ unsigned int len = ARRAY_SIZE(ad74115_debounce_tbl);
+ unsigned int i;
+
+ for (i = 0; i < len; i++)
+ if (val <= ad74115_debounce_tbl[i])
+ break;
+
+ if (i == len)
+ i = len - 1;
+
+ return regmap_update_bits(st->regmap, AD74115_DIN_CONFIG1_REG,
+ AD74115_DIN_DEBOUNCE_MASK,
+ FIELD_PREP(AD74115_DIN_DEBOUNCE_MASK, val));
+}
+
+static int ad74115_comp_gpio_get_direction(struct gpio_chip *chip,
+ unsigned int offset)
+{
+ return GPIO_LINE_DIRECTION_IN;
+}
+
+static int ad74115_comp_gpio_set_config(struct gpio_chip *chip,
+ unsigned int offset,
+ unsigned long config)
+{
+ struct ad74115_state *st = gpiochip_get_data(chip);
+ u32 param = pinconf_to_config_param(config);
+ u32 arg = pinconf_to_config_argument(config);
+
+ switch (param) {
+ case PIN_CONFIG_INPUT_DEBOUNCE:
+ return ad74115_set_comp_debounce(st, arg);
+ default:
+ return -ENOTSUPP;
+ }
+}
+
+static int ad74115_comp_gpio_get(struct gpio_chip *chip, unsigned int offset)
+{
+ struct ad74115_state *st = gpiochip_get_data(chip);
+ unsigned int val;
+ int ret;
+
+ ret = regmap_read(st->regmap, AD74115_DIN_COMP_OUT_REG, &val);
+ if (ret)
+ return ret;
+
+ return !!val;
+}
+
+static irqreturn_t ad74115_trigger_handler(int irq, void *p)
+{
+ struct iio_poll_func *pf = p;
+ struct iio_dev *indio_dev = pf->indio_dev;
+ struct ad74115_state *st = iio_priv(indio_dev);
+ int ret;
+
+ ret = spi_sync(st->spi, &st->adc_samples_msg);
+ if (ret)
+ goto out;
+
+ iio_push_to_buffers(indio_dev, st->adc_samples_rx_buf);
+
+out:
+ iio_trigger_notify_done(indio_dev->trig);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t ad74115_adc_data_interrupt(int irq, void *data)
+{
+ struct iio_dev *indio_dev = data;
+ struct ad74115_state *st = iio_priv(indio_dev);
+
+ if (iio_buffer_enabled(indio_dev))
+ iio_trigger_poll(st->trig);
+ else
+ complete(&st->adc_data_completion);
+
+ return IRQ_HANDLED;
+}
+
+static int ad74115_set_adc_ch_en(struct ad74115_state *st,
+ enum ad74115_adc_ch channel, bool status)
+{
+ unsigned int mask = ad74115_adc_ch_en_bit_tbl[channel];
+
+ return regmap_update_bits(st->regmap, AD74115_ADC_CONV_CTRL_REG, mask,
+ status ? mask : 0);
+}
+
+static int ad74115_set_adc_conv_seq(struct ad74115_state *st,
+ enum ad74115_adc_conv_seq conv_seq)
+{
+ return regmap_update_bits(st->regmap, AD74115_ADC_CONV_CTRL_REG,
+ AD74115_ADC_CONV_SEQ_MASK,
+ FIELD_PREP(AD74115_ADC_CONV_SEQ_MASK, conv_seq));
+}
+
+static int ad74115_update_scan_mode(struct iio_dev *indio_dev,
+ const unsigned long *active_scan_mask)
+{
+ struct ad74115_state *st = iio_priv(indio_dev);
+ struct spi_transfer *xfer = st->adc_samples_xfer;
+ u8 *rx_buf = st->adc_samples_rx_buf;
+ u8 *tx_buf = st->adc_samples_tx_buf;
+ unsigned int i;
+ int ret = 0;
+
+ mutex_lock(&st->lock);
+
+ spi_message_init(&st->adc_samples_msg);
+
+ for_each_clear_bit(i, active_scan_mask, AD74115_ADC_CH_NUM) {
+ ret = ad74115_set_adc_ch_en(st, i, false);
+ if (ret)
+ goto out;
+ }
+
+ /*
+ * The read select register is used to select which register's value
+ * will be sent by the slave on the next SPI frame.
+ *
+ * Create an SPI message that, on each step, writes to the read select
+ * register to select the ADC result of the next enabled channel, and
+ * reads the ADC result of the previous enabled channel.
+ *
+ * Example:
+ * W: [WCH1] [WCH2] [WCH2] [WCH3] [ ]
+ * R: [ ] [RCH1] [RCH2] [RCH3] [RCH4]
+ */
+ for_each_set_bit(i, active_scan_mask, AD74115_ADC_CH_NUM) {
+ ret = ad74115_set_adc_ch_en(st, i, true);
+ if (ret)
+ goto out;
+
+ if (xfer == st->adc_samples_xfer)
+ xfer->rx_buf = NULL;
+ else
+ xfer->rx_buf = rx_buf;
+
+ xfer->tx_buf = tx_buf;
+ xfer->len = AD74115_FRAME_SIZE;
+ xfer->cs_change = 1;
+
+ ad74115_format_reg_write(AD74115_READ_SELECT_REG,
+ ad74115_adc_ch_data_regs_tbl[i], tx_buf);
+
+ spi_message_add_tail(xfer, &st->adc_samples_msg);
+
+ tx_buf += AD74115_FRAME_SIZE;
+ if (xfer != st->adc_samples_xfer)
+ rx_buf += AD74115_FRAME_SIZE;
+ xfer++;
+ }
+
+ xfer->rx_buf = rx_buf;
+ xfer->tx_buf = NULL;
+ xfer->len = AD74115_FRAME_SIZE;
+ xfer->cs_change = 0;
+
+ spi_message_add_tail(xfer, &st->adc_samples_msg);
+
+out:
+ mutex_unlock(&st->lock);
+
+ return ret;
+}
+
+static int ad74115_buffer_postenable(struct iio_dev *indio_dev)
+{
+ struct ad74115_state *st = iio_priv(indio_dev);
+
+ return ad74115_set_adc_conv_seq(st, AD74115_ADC_CONV_SEQ_CONTINUOUS);
+}
+
+static int ad74115_buffer_predisable(struct iio_dev *indio_dev)
+{
+ struct ad74115_state *st = iio_priv(indio_dev);
+ unsigned int i;
+ int ret;
+
+ mutex_lock(&st->lock);
+
+ ret = ad74115_set_adc_conv_seq(st, AD74115_ADC_CONV_SEQ_STANDBY);
+ if (ret)
+ goto out;
+
+ /*
+ * update_scan_mode() is not called in the disable path, disable all
+ * channels here.
+ */
+ for (i = 0; i < AD74115_ADC_CH_NUM; i++) {
+ ret = ad74115_set_adc_ch_en(st, i, false);
+ if (ret)
+ goto out;
+ }
+
+out:
+ mutex_unlock(&st->lock);
+
+ return ret;
+}
+
+static const struct iio_buffer_setup_ops ad74115_buffer_ops = {
+ .postenable = &ad74115_buffer_postenable,
+ .predisable = &ad74115_buffer_predisable,
+};
+
+static const struct iio_trigger_ops ad74115_trigger_ops = {
+ .validate_device = iio_trigger_validate_own_device,
+};
+
+static int ad74115_get_adc_rate(struct ad74115_state *st,
+ enum ad74115_adc_ch channel, int *val)
+{
+ unsigned int i;
+ int ret;
+
+ ret = regmap_read(st->regmap, AD74115_ADC_CONFIG_REG, &i);
+ if (ret)
+ return ret;
+
+ if (channel == AD74115_ADC_CH_CONV1)
+ i = FIELD_GET(AD74115_ADC_CONFIG_CONV1_RATE_MASK, i);
+ else
+ i = FIELD_GET(AD74115_ADC_CONFIG_CONV2_RATE_MASK, i);
+
+ *val = ad74115_adc_conv_rate_tbl[i];
+
+ return IIO_VAL_INT;
+}
+
+static int _ad74115_get_adc_code(struct ad74115_state *st,
+ enum ad74115_adc_ch channel, int *val)
+{
+ unsigned int uval;
+ int ret;
+
+ reinit_completion(&st->adc_data_completion);
+
+ ret = ad74115_set_adc_ch_en(st, channel, true);
+ if (ret)
+ return ret;
+
+ ret = ad74115_set_adc_conv_seq(st, AD74115_ADC_CONV_SEQ_SINGLE);
+ if (ret)
+ return ret;
+
+ if (st->irq) {
+ ret = wait_for_completion_timeout(&st->adc_data_completion,
+ msecs_to_jiffies(1000));
+ if (!ret)
+ return -ETIMEDOUT;
+ } else {
+ unsigned int regval, wait_time;
+ int rate;
+
+ ret = ad74115_get_adc_rate(st, channel, &rate);
+ if (ret < 0)
+ return ret;
+
+ wait_time = DIV_ROUND_CLOSEST(AD74115_CONV_TIME_US, rate);
+
+ ret = regmap_read_poll_timeout(st->regmap, AD74115_LIVE_STATUS_REG,
+ regval, regval & AD74115_ADC_DATA_RDY_MASK,
+ wait_time, 5 * wait_time);
+ if (ret)
+ return ret;
+
+ /*
+ * The ADC_DATA_RDY bit is W1C.
+ * See datasheet page 98, Table 62. Bit Descriptions for
+ * LIVE_STATUS.
+ * Although the datasheet mentions that the bit will auto-clear
+ * when writing to the ADC_CONV_CTRL register, this does not
+ * seem to happen.
+ */
+ ret = regmap_write_bits(st->regmap, AD74115_LIVE_STATUS_REG,
+ AD74115_ADC_DATA_RDY_MASK,
+ FIELD_PREP(AD74115_ADC_DATA_RDY_MASK, 1));
+ if (ret)
+ return ret;
+ }
+
+ ret = regmap_read(st->regmap, ad74115_adc_ch_data_regs_tbl[channel], &uval);
+ if (ret)
+ return ret;
+
+ ret = ad74115_set_adc_conv_seq(st, AD74115_ADC_CONV_SEQ_STANDBY);
+ if (ret)
+ return ret;
+
+ ret = ad74115_set_adc_ch_en(st, channel, false);
+ if (ret)
+ return ret;
+
+ *val = uval;
+
+ return IIO_VAL_INT;
+}
+
+static int ad74115_get_adc_code(struct iio_dev *indio_dev,
+ enum ad74115_adc_ch channel, int *val)
+{
+ struct ad74115_state *st = iio_priv(indio_dev);
+ int ret;
+
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
+
+ mutex_lock(&st->lock);
+ ret = _ad74115_get_adc_code(st, channel, val);
+ mutex_unlock(&st->lock);
+
+ iio_device_release_direct_mode(indio_dev);
+
+ return ret;
+}
+
+static int ad74115_adc_code_to_resistance(int code, int *val, int *val2)
+{
+ if (code == AD74115_ADC_CODE_MAX)
+ code--;
+
+ *val = code * AD74115_REF_RESISTOR_OHMS;
+ *val2 = AD74115_ADC_CODE_MAX - code;
+
+ return IIO_VAL_FRACTIONAL;
+}
+
+static int ad74115_set_dac_code(struct ad74115_state *st,
+ enum ad74115_dac_ch channel, int val)
+{
+ if (val < 0)
+ return -EINVAL;
+
+ if (channel == AD74115_DAC_CH_COMPARATOR) {
+ if (val > AD74115_COMP_THRESH_MAX)
+ return -EINVAL;
+
+ return regmap_update_bits(st->regmap, AD74115_DIN_CONFIG2_REG,
+ AD74115_COMP_THRESH_MASK,
+ FIELD_PREP(AD74115_COMP_THRESH_MASK, val));
+ }
+
+ if (val > AD74115_DAC_CODE_MAX)
+ return -EINVAL;
+
+ return regmap_write(st->regmap, AD74115_DAC_CODE_REG, val);
+}
+
+static int ad74115_get_dac_code(struct ad74115_state *st,
+ enum ad74115_dac_ch channel, int *val)
+{
+ unsigned int uval;
+ int ret;
+
+ if (channel == AD74115_DAC_CH_COMPARATOR)
+ return -EINVAL;
+
+ ret = regmap_read(st->regmap, AD74115_DAC_ACTIVE_REG, &uval);
+ if (ret)
+ return ret;
+
+ *val = uval;
+
+ return IIO_VAL_INT;
+}
+
+static int ad74115_set_adc_rate(struct ad74115_state *st,
+ enum ad74115_adc_ch channel, int val)
+{
+ unsigned int i;
+ int ret;
+
+ ret = ad74115_find_tbl_index(ad74115_adc_conv_rate_tbl, val, &i);
+ if (ret)
+ return ret;
+
+ if (channel == AD74115_ADC_CH_CONV1)
+ return regmap_update_bits(st->regmap, AD74115_ADC_CONFIG_REG,
+ AD74115_ADC_CONFIG_CONV1_RATE_MASK,
+ FIELD_PREP(AD74115_ADC_CONFIG_CONV1_RATE_MASK, i));
+
+ return regmap_update_bits(st->regmap, AD74115_ADC_CONFIG_REG,
+ AD74115_ADC_CONFIG_CONV2_RATE_MASK,
+ FIELD_PREP(AD74115_ADC_CONFIG_CONV2_RATE_MASK, i));
+}
+
+static int ad74115_get_dac_rate(struct ad74115_state *st, int *val)
+{
+ unsigned int i, en_val, step_val, rate_val, tmp;
+ int ret;
+
+ ret = regmap_read(st->regmap, AD74115_OUTPUT_CONFIG_REG, &tmp);
+ if (ret)
+ return ret;
+
+ en_val = FIELD_GET(AD74115_OUTPUT_SLEW_EN_MASK, tmp);
+ step_val = FIELD_GET(AD74115_OUTPUT_SLEW_LIN_STEP_MASK, tmp);
+ rate_val = FIELD_GET(AD74115_OUTPUT_SLEW_LIN_RATE_MASK, tmp);
+
+ for (i = 0; i < ARRAY_SIZE(ad74115_dac_rate_step_tbl); i++)
+ if (en_val == ad74115_dac_rate_step_tbl[i][0] &&
+ step_val == ad74115_dac_rate_step_tbl[i][1] &&
+ rate_val == ad74115_dac_rate_step_tbl[i][2])
+ break;
+
+ if (i == ARRAY_SIZE(ad74115_dac_rate_step_tbl))
+ return -EINVAL;
+
+ *val = ad74115_dac_rate_tbl[i];
+
+ return IIO_VAL_INT;
+}
+
+static int ad74115_set_dac_rate(struct ad74115_state *st, int val)
+{
+ unsigned int i, en_val, step_val, rate_val, mask, tmp;
+ int ret;
+
+ ret = ad74115_find_tbl_index(ad74115_dac_rate_tbl, val, &i);
+ if (ret)
+ return ret;
+
+ en_val = ad74115_dac_rate_step_tbl[i][0];
+ step_val = ad74115_dac_rate_step_tbl[i][1];
+ rate_val = ad74115_dac_rate_step_tbl[i][2];
+
+ mask = AD74115_OUTPUT_SLEW_EN_MASK;
+ mask |= AD74115_OUTPUT_SLEW_LIN_STEP_MASK;
+ mask |= AD74115_OUTPUT_SLEW_LIN_RATE_MASK;
+
+ tmp = FIELD_PREP(AD74115_OUTPUT_SLEW_EN_MASK, en_val);
+ tmp |= FIELD_PREP(AD74115_OUTPUT_SLEW_LIN_STEP_MASK, step_val);
+ tmp |= FIELD_PREP(AD74115_OUTPUT_SLEW_LIN_RATE_MASK, rate_val);
+
+ return regmap_update_bits(st->regmap, AD74115_OUTPUT_CONFIG_REG, mask, tmp);
+}
+
+static int ad74115_get_dac_scale(struct ad74115_state *st,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2)
+{
+ if (chan->channel == AD74115_DAC_CH_MAIN) {
+ if (chan->type == IIO_VOLTAGE) {
+ *val = AD74115_DAC_VOLTAGE_MAX;
+
+ if (st->dac_bipolar)
+ *val *= 2;
+
+ } else {
+ *val = AD74115_DAC_CURRENT_MAX;
+ }
+
+ *val2 = AD74115_DAC_CODE_MAX;
+ } else {
+ if (st->din_threshold_mode == AD74115_DIN_THRESHOLD_MODE_AVDD) {
+ *val = 196 * st->avdd_mv;
+ *val2 = 10 * AD74115_COMP_THRESH_MAX;
+ } else {
+ *val = 49000;
+ *val2 = AD74115_COMP_THRESH_MAX;
+ }
+ }
+
+ return IIO_VAL_FRACTIONAL;
+}
+
+static int ad74115_get_dac_offset(struct ad74115_state *st,
+ struct iio_chan_spec const *chan, int *val)
+{
+ if (chan->channel == AD74115_DAC_CH_MAIN) {
+ if (chan->type == IIO_VOLTAGE && st->dac_bipolar)
+ *val = -AD74115_DAC_CODE_HALF;
+ else
+ *val = 0;
+ } else {
+ if (st->din_threshold_mode == AD74115_DIN_THRESHOLD_MODE_AVDD)
+ *val = -48;
+ else
+ *val = -38;
+ }
+
+ return IIO_VAL_INT;
+}
+
+static int ad74115_get_adc_range(struct ad74115_state *st,
+ enum ad74115_adc_ch channel, unsigned int *val)
+{
+ int ret;
+
+ ret = regmap_read(st->regmap, AD74115_ADC_CONFIG_REG, val);
+ if (ret)
+ return ret;
+
+ if (channel == AD74115_ADC_CH_CONV1)
+ *val = FIELD_GET(AD74115_ADC_CONFIG_CONV1_RANGE_MASK, *val);
+ else
+ *val = FIELD_GET(AD74115_ADC_CONFIG_CONV2_RANGE_MASK, *val);
+
+ return 0;
+}
+
+static int ad74115_get_adc_resistance_scale(struct ad74115_state *st,
+ unsigned int range,
+ int *val, int *val2)
+{
+ *val = ad74115_adc_gain_tbl[range][1] * AD74115_REF_RESISTOR_OHMS;
+ *val2 = ad74115_adc_gain_tbl[range][0];
+
+ if (ad74115_adc_bipolar_tbl[range])
+ *val2 *= AD74115_ADC_CODE_HALF;
+ else
+ *val2 *= AD74115_ADC_CODE_MAX;
+
+ return IIO_VAL_FRACTIONAL;
+}
+
+static int ad74115_get_adc_scale(struct ad74115_state *st,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2)
+{
+ unsigned int range;
+ int ret;
+
+ ret = ad74115_get_adc_range(st, chan->channel, &range);
+ if (ret)
+ return ret;
+
+ if (chan->type == IIO_RESISTANCE)
+ return ad74115_get_adc_resistance_scale(st, range, val, val2);
+
+ *val = ad74115_adc_conv_mul_tbl[range];
+ *val2 = AD74115_ADC_CODE_MAX;
+
+ if (chan->type == IIO_CURRENT)
+ *val2 *= AD74115_SENSE_RESISTOR_OHMS;
+
+ return IIO_VAL_FRACTIONAL;
+}
+
+static int ad74115_get_adc_resistance_offset(struct ad74115_state *st,
+ unsigned int range,
+ int *val, int *val2)
+{
+ unsigned int d = 10 * AD74115_REF_RESISTOR_OHMS
+ * ad74115_adc_gain_tbl[range][1];
+
+ *val = 5;
+
+ if (ad74115_adc_bipolar_tbl[range])
+ *val -= AD74115_ADC_CODE_HALF;
+
+ *val *= d;
+
+ if (!st->rtd_mode_4_wire) {
+ /* Add 0.2 Ohm to the final result for 3-wire RTD. */
+ unsigned int v = 2 * ad74115_adc_gain_tbl[range][0];
+
+ if (ad74115_adc_bipolar_tbl[range])
+ v *= AD74115_ADC_CODE_HALF;
+ else
+ v *= AD74115_ADC_CODE_MAX;
+
+ *val += v;
+ }
+
+ *val2 = d;
+
+ return IIO_VAL_FRACTIONAL;
+}
+
+static int ad74115_get_adc_offset(struct ad74115_state *st,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2)
+{
+ unsigned int range;
+ int ret;
+
+ ret = ad74115_get_adc_range(st, chan->channel, &range);
+ if (ret)
+ return ret;
+
+ if (chan->type == IIO_RESISTANCE)
+ return ad74115_get_adc_resistance_offset(st, range, val, val2);
+
+ if (ad74115_adc_bipolar_tbl[range])
+ *val = -AD74115_ADC_CODE_HALF;
+ else if (range == AD74115_ADC_RANGE_2_5V_NEG)
+ *val = -AD74115_ADC_CODE_MAX;
+ else
+ *val = 0;
+
+ return IIO_VAL_INT;
+}
+
+static int ad74115_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long info)
+{
+ struct ad74115_state *st = iio_priv(indio_dev);
+ int ret;
+
+ switch (info) {
+ case IIO_CHAN_INFO_RAW:
+ if (chan->output)
+ return ad74115_get_dac_code(st, chan->channel, val);
+
+ return ad74115_get_adc_code(indio_dev, chan->channel, val);
+ case IIO_CHAN_INFO_PROCESSED:
+ ret = ad74115_get_adc_code(indio_dev, chan->channel, val);
+ if (ret)
+ return ret;
+
+ return ad74115_adc_code_to_resistance(*val, val, val2);
+ case IIO_CHAN_INFO_SCALE:
+ if (chan->output)
+ return ad74115_get_dac_scale(st, chan, val, val2);
+
+ return ad74115_get_adc_scale(st, chan, val, val2);
+ case IIO_CHAN_INFO_OFFSET:
+ if (chan->output)
+ return ad74115_get_dac_offset(st, chan, val);
+
+ return ad74115_get_adc_offset(st, chan, val, val2);
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ if (chan->output)
+ return ad74115_get_dac_rate(st, val);
+
+ return ad74115_get_adc_rate(st, chan->channel, val);
+ default:
+ return -EINVAL;
+ }
+}
+
+static int ad74115_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan, int val, int val2,
+ long info)
+{
+ struct ad74115_state *st = iio_priv(indio_dev);
+
+ switch (info) {
+ case IIO_CHAN_INFO_RAW:
+ if (!chan->output)
+ return -EINVAL;
+
+ return ad74115_set_dac_code(st, chan->channel, val);
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ if (chan->output)
+ return ad74115_set_dac_rate(st, val);
+
+ return ad74115_set_adc_rate(st, chan->channel, val);
+ default:
+ return -EINVAL;
+ }
+}
+
+static int ad74115_read_avail(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ const int **vals, int *type, int *length, long info)
+{
+ switch (info) {
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ if (chan->output) {
+ *vals = ad74115_dac_rate_tbl;
+ *length = ARRAY_SIZE(ad74115_dac_rate_tbl);
+ } else {
+ *vals = ad74115_adc_conv_rate_tbl;
+ *length = ARRAY_SIZE(ad74115_adc_conv_rate_tbl);
+ }
+
+ *type = IIO_VAL_INT;
+
+ return IIO_AVAIL_LIST;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int ad74115_reg_access(struct iio_dev *indio_dev, unsigned int reg,
+ unsigned int writeval, unsigned int *readval)
+{
+ struct ad74115_state *st = iio_priv(indio_dev);
+
+ if (readval)
+ return regmap_read(st->regmap, reg, readval);
+
+ return regmap_write(st->regmap, reg, writeval);
+}
+
+static const struct iio_info ad74115_info = {
+ .read_raw = ad74115_read_raw,
+ .write_raw = ad74115_write_raw,
+ .read_avail = ad74115_read_avail,
+ .update_scan_mode = ad74115_update_scan_mode,
+ .debugfs_reg_access = ad74115_reg_access,
+};
+
+#define AD74115_DAC_CHANNEL(_type, index) \
+ { \
+ .type = (_type), \
+ .channel = (index), \
+ .indexed = 1, \
+ .output = 1, \
+ .scan_index = -1, \
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) \
+ | BIT(IIO_CHAN_INFO_SCALE) \
+ | BIT(IIO_CHAN_INFO_OFFSET), \
+ }
+
+#define _AD74115_ADC_CHANNEL(_type, index, extra_mask_separate) \
+ { \
+ .type = (_type), \
+ .channel = (index), \
+ .indexed = 1, \
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) \
+ | BIT(IIO_CHAN_INFO_SAMP_FREQ) \
+ | (extra_mask_separate), \
+ .info_mask_separate_available = \
+ BIT(IIO_CHAN_INFO_SAMP_FREQ), \
+ .scan_index = index, \
+ .scan_type = { \
+ .sign = 'u', \
+ .realbits = 16, \
+ .storagebits = 32, \
+ .shift = 8, \
+ .endianness = IIO_BE, \
+ }, \
+ }
+
+#define AD74115_ADC_CHANNEL(_type, index) \
+ _AD74115_ADC_CHANNEL(_type, index, BIT(IIO_CHAN_INFO_SCALE) \
+ | BIT(IIO_CHAN_INFO_OFFSET))
+
+static struct iio_chan_spec ad74115_voltage_input_channels[] = {
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV1),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV2),
+};
+
+static struct iio_chan_spec ad74115_voltage_output_channels[] = {
+ AD74115_DAC_CHANNEL(IIO_VOLTAGE, AD74115_DAC_CH_MAIN),
+ AD74115_ADC_CHANNEL(IIO_CURRENT, AD74115_ADC_CH_CONV1),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV2),
+};
+
+static struct iio_chan_spec ad74115_current_input_channels[] = {
+ AD74115_ADC_CHANNEL(IIO_CURRENT, AD74115_ADC_CH_CONV1),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV2),
+};
+
+static struct iio_chan_spec ad74115_current_output_channels[] = {
+ AD74115_DAC_CHANNEL(IIO_CURRENT, AD74115_DAC_CH_MAIN),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV1),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV2),
+};
+
+static struct iio_chan_spec ad74115_2_wire_resistance_input_channels[] = {
+ _AD74115_ADC_CHANNEL(IIO_RESISTANCE, AD74115_ADC_CH_CONV1,
+ BIT(IIO_CHAN_INFO_PROCESSED)),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV2),
+};
+
+static struct iio_chan_spec ad74115_3_4_wire_resistance_input_channels[] = {
+ AD74115_ADC_CHANNEL(IIO_RESISTANCE, AD74115_ADC_CH_CONV1),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV2),
+};
+
+static struct iio_chan_spec ad74115_digital_input_logic_channels[] = {
+ AD74115_DAC_CHANNEL(IIO_VOLTAGE, AD74115_DAC_CH_COMPARATOR),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV1),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV2),
+};
+
+static struct iio_chan_spec ad74115_digital_input_loop_channels[] = {
+ AD74115_DAC_CHANNEL(IIO_CURRENT, AD74115_DAC_CH_MAIN),
+ AD74115_DAC_CHANNEL(IIO_VOLTAGE, AD74115_DAC_CH_COMPARATOR),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV1),
+ AD74115_ADC_CHANNEL(IIO_VOLTAGE, AD74115_ADC_CH_CONV2),
+};
+
+#define _AD74115_CHANNELS(_channels) \
+ { \
+ .channels = _channels, \
+ .num_channels = ARRAY_SIZE(_channels), \
+ }
+
+#define AD74115_CHANNELS(name) \
+ _AD74115_CHANNELS(ad74115_ ## name ## _channels)
+
+static const struct ad74115_channels ad74115_channels_map[AD74115_CH_FUNC_NUM] = {
+ [AD74115_CH_FUNC_HIGH_IMPEDANCE] = AD74115_CHANNELS(voltage_input),
+ [AD74115_CH_FUNC_VOLTAGE_INPUT] = AD74115_CHANNELS(voltage_input),
+
+ [AD74115_CH_FUNC_VOLTAGE_OUTPUT] = AD74115_CHANNELS(voltage_output),
+
+ [AD74115_CH_FUNC_CURRENT_INPUT_EXT_POWER] = AD74115_CHANNELS(current_input),
+ [AD74115_CH_FUNC_CURRENT_INPUT_LOOP_POWER] = AD74115_CHANNELS(current_input),
+ [AD74115_CH_FUNC_CURRENT_INPUT_EXT_POWER_HART] = AD74115_CHANNELS(current_input),
+ [AD74115_CH_FUNC_CURRENT_INPUT_LOOP_POWER_HART] = AD74115_CHANNELS(current_input),
+
+ [AD74115_CH_FUNC_CURRENT_OUTPUT] = AD74115_CHANNELS(current_output),
+ [AD74115_CH_FUNC_CURRENT_OUTPUT_HART] = AD74115_CHANNELS(current_output),
+
+ [AD74115_CH_FUNC_2_WIRE_RESISTANCE_INPUT] = AD74115_CHANNELS(2_wire_resistance_input),
+ [AD74115_CH_FUNC_3_4_WIRE_RESISTANCE_INPUT] = AD74115_CHANNELS(3_4_wire_resistance_input),
+
+ [AD74115_CH_FUNC_DIGITAL_INPUT_LOGIC] = AD74115_CHANNELS(digital_input_logic),
+
+ [AD74115_CH_FUNC_DIGITAL_INPUT_LOOP_POWER] = AD74115_CHANNELS(digital_input_loop),
+};
+
+#define AD74115_GPIO_MODE_FW_PROP(i) \
+{ \
+ .name = "adi,gpio" __stringify(i) "-mode", \
+ .reg = AD74115_GPIO_CONFIG_X_REG(i), \
+ .mask = AD74115_GPIO_CONFIG_SELECT_MASK, \
+ .lookup_tbl = ad74115_gpio_mode_tbl, \
+ .lookup_tbl_len = ARRAY_SIZE(ad74115_gpio_mode_tbl), \
+}
+
+static const struct ad74115_fw_prop ad74115_gpio_mode_fw_props[] = {
+ AD74115_GPIO_MODE_FW_PROP(0),
+ AD74115_GPIO_MODE_FW_PROP(1),
+ AD74115_GPIO_MODE_FW_PROP(2),
+ AD74115_GPIO_MODE_FW_PROP(3),
+};
+
+static const struct ad74115_fw_prop ad74115_din_threshold_mode_fw_prop =
+ AD74115_FW_PROP_BOOL("adi,digital-input-threshold-mode-fixed",
+ AD74115_DIN_CONFIG2_REG, BIT(7));
+
+static const struct ad74115_fw_prop ad74115_dac_bipolar_fw_prop =
+ AD74115_FW_PROP_BOOL("adi,dac-bipolar", AD74115_OUTPUT_CONFIG_REG, BIT(7));
+
+static const struct ad74115_fw_prop ad74115_ch_func_fw_prop =
+ AD74115_FW_PROP("adi,ch-func", AD74115_CH_FUNC_MAX,
+ AD74115_CH_FUNC_SETUP_REG, GENMASK(3, 0));
+
+static const struct ad74115_fw_prop ad74115_rtd_mode_fw_prop =
+ AD74115_FW_PROP_BOOL("adi,4-wire-rtd", AD74115_RTD3W4W_CONFIG_REG, BIT(3));
+
+static const struct ad74115_fw_prop ad74115_din_range_fw_prop =
+ AD74115_FW_PROP_BOOL("adi,digital-input-sink-range-high",
+ AD74115_DIN_CONFIG1_REG, BIT(12));
+
+static const struct ad74115_fw_prop ad74115_ext2_burnout_current_fw_prop =
+ AD74115_FW_PROP_TBL("adi,ext2-burnout-current-nanoamp",
+ ad74115_burnout_current_na_tbl,
+ AD74115_BURNOUT_CONFIG_REG, GENMASK(14, 12));
+
+static const struct ad74115_fw_prop ad74115_ext1_burnout_current_fw_prop =
+ AD74115_FW_PROP_TBL("adi,ext1-burnout-current-nanoamp",
+ ad74115_burnout_current_na_tbl,
+ AD74115_BURNOUT_CONFIG_REG, GENMASK(9, 7));
+
+static const struct ad74115_fw_prop ad74115_viout_burnout_current_fw_prop =
+ AD74115_FW_PROP_TBL("adi,viout-burnout-current-nanoamp",
+ ad74115_viout_burnout_current_na_tbl,
+ AD74115_BURNOUT_CONFIG_REG, GENMASK(4, 2));
+
+static const struct ad74115_fw_prop ad74115_fw_props[] = {
+ AD74115_FW_PROP("adi,conv2-mux", 3,
+ AD74115_ADC_CONFIG_REG, GENMASK(3, 2)),
+
+ AD74115_FW_PROP_BOOL_NEG("adi,sense-agnd-buffer-low-power",
+ AD74115_PWR_OPTIM_CONFIG_REG, BIT(4)),
+ AD74115_FW_PROP_BOOL_NEG("adi,lf-buffer-low-power",
+ AD74115_PWR_OPTIM_CONFIG_REG, BIT(3)),
+ AD74115_FW_PROP_BOOL_NEG("adi,hf-buffer-low-power",
+ AD74115_PWR_OPTIM_CONFIG_REG, BIT(2)),
+ AD74115_FW_PROP_BOOL_NEG("adi,ext2-buffer-low-power",
+ AD74115_PWR_OPTIM_CONFIG_REG, BIT(1)),
+ AD74115_FW_PROP_BOOL_NEG("adi,ext1-buffer-low-power",
+ AD74115_PWR_OPTIM_CONFIG_REG, BIT(0)),
+
+ AD74115_FW_PROP_BOOL("adi,comparator-invert",
+ AD74115_DIN_CONFIG1_REG, BIT(14)),
+ AD74115_FW_PROP_BOOL("adi,digital-input-debounce-mode-counter-reset",
+ AD74115_DIN_CONFIG1_REG, BIT(6)),
+
+ AD74115_FW_PROP_BOOL("adi,digital-input-unbuffered",
+ AD74115_DIN_CONFIG2_REG, BIT(10)),
+ AD74115_FW_PROP_BOOL("adi,digital-input-short-circuit-detection",
+ AD74115_DIN_CONFIG2_REG, BIT(9)),
+ AD74115_FW_PROP_BOOL("adi,digital-input-open-circuit-detection",
+ AD74115_DIN_CONFIG2_REG, BIT(8)),
+
+ AD74115_FW_PROP_BOOL("adi,dac-current-limit-low",
+ AD74115_OUTPUT_CONFIG_REG, BIT(0)),
+
+ AD74115_FW_PROP_BOOL("adi,3-wire-rtd-excitation-swap",
+ AD74115_RTD3W4W_CONFIG_REG, BIT(2)),
+ AD74115_FW_PROP_TBL("adi,rtd-excitation-current-microamp",
+ ad74115_rtd_excitation_current_ua_tbl,
+ AD74115_RTD3W4W_CONFIG_REG, GENMASK(1, 0)),
+
+ AD74115_FW_PROP_BOOL("adi,ext2-burnout-current-polarity-sourcing",
+ AD74115_BURNOUT_CONFIG_REG, BIT(11)),
+ AD74115_FW_PROP_BOOL("adi,ext1-burnout-current-polarity-sourcing",
+ AD74115_BURNOUT_CONFIG_REG, BIT(6)),
+ AD74115_FW_PROP_BOOL("adi,viout-burnout-current-polarity-sourcing",
+ AD74115_BURNOUT_CONFIG_REG, BIT(1)),
+
+ AD74115_FW_PROP_BOOL("adi,charge-pump",
+ AD74115_CHARGE_PUMP_REG, BIT(0)),
+};
+
+static int ad74115_apply_fw_prop(struct ad74115_state *st,
+ const struct ad74115_fw_prop *prop, u32 *retval)
+{
+ struct device *dev = &st->spi->dev;
+ u32 val = 0;
+ int ret;
+
+ if (prop->is_boolean) {
+ val = device_property_read_bool(dev, prop->name);
+ } else {
+ ret = device_property_read_u32(dev, prop->name, &val);
+ if (ret && prop->lookup_tbl)
+ val = prop->lookup_tbl[0];
+ }
+
+ *retval = val;
+
+ if (prop->negate)
+ val = !val;
+
+ if (prop->lookup_tbl)
+ ret = _ad74115_find_tbl_index(prop->lookup_tbl,
+ prop->lookup_tbl_len, val, &val);
+ else if (prop->max && val > prop->max)
+ ret = -EINVAL;
+ else
+ ret = 0;
+
+ if (ret)
+ return dev_err_probe(dev, -EINVAL,
+ "Invalid value %u for prop %s\n",
+ val, prop->name);
+
+ WARN(!prop->mask, "Prop %s mask is empty\n", prop->name);
+
+ val = (val << __ffs(prop->mask)) & prop->mask;
+
+ return regmap_update_bits(st->regmap, prop->reg, prop->mask, val);
+}
+
+static int ad74115_setup_adc_conv2_range(struct ad74115_state *st)
+{
+ unsigned int tbl_len = ARRAY_SIZE(ad74115_adc_range_tbl);
+ const char *prop_name = "adi,conv2-range-microvolt";
+ s32 vals[2] = {
+ ad74115_adc_range_tbl[0][0],
+ ad74115_adc_range_tbl[0][1],
+ };
+ struct device *dev = &st->spi->dev;
+ unsigned int i;
+
+ device_property_read_u32_array(dev, prop_name, vals, 2);
+
+ for (i = 0; i < tbl_len; i++)
+ if (vals[0] == ad74115_adc_range_tbl[i][0] &&
+ vals[1] == ad74115_adc_range_tbl[i][1])
+ break;
+
+ if (i == tbl_len)
+ return dev_err_probe(dev, -EINVAL,
+ "Invalid value %d, %d for prop %s\n",
+ vals[0], vals[1], prop_name);
+
+ return regmap_update_bits(st->regmap, AD74115_ADC_CONFIG_REG,
+ AD74115_ADC_CONFIG_CONV2_RANGE_MASK,
+ FIELD_PREP(AD74115_ADC_CONFIG_CONV2_RANGE_MASK, i));
+}
+
+static int ad74115_setup_iio_channels(struct iio_dev *indio_dev)
+{
+ struct ad74115_state *st = iio_priv(indio_dev);
+ struct device *dev = &st->spi->dev;
+ struct iio_chan_spec *channels;
+
+ channels = devm_kcalloc(dev, sizeof(*channels),
+ indio_dev->num_channels, GFP_KERNEL);
+ if (!channels)
+ return -ENOMEM;
+
+ indio_dev->channels = channels;
+
+ memcpy(channels, ad74115_channels_map[st->ch_func].channels,
+ sizeof(*channels) * ad74115_channels_map[st->ch_func].num_channels);
+
+ if (channels[0].output && channels[0].channel == AD74115_DAC_CH_MAIN &&
+ channels[0].type == IIO_VOLTAGE && !st->dac_hart_slew) {
+ channels[0].info_mask_separate |= BIT(IIO_CHAN_INFO_SAMP_FREQ);
+ channels[0].info_mask_separate_available |= BIT(IIO_CHAN_INFO_SAMP_FREQ);
+ }
+
+ return 0;
+}
+
+static int ad74115_setup_gpio_chip(struct ad74115_state *st)
+{
+ struct device *dev = &st->spi->dev;
+
+ if (!st->gpio_valid_mask)
+ return 0;
+
+ st->gc = (struct gpio_chip) {
+ .owner = THIS_MODULE,
+ .label = AD74115_NAME,
+ .base = -1,
+ .ngpio = AD74115_GPIO_NUM,
+ .parent = dev,
+ .can_sleep = true,
+ .init_valid_mask = ad74115_gpio_init_valid_mask,
+ .get_direction = ad74115_gpio_get_direction,
+ .direction_input = ad74115_gpio_direction_input,
+ .direction_output = ad74115_gpio_direction_output,
+ .get = ad74115_gpio_get,
+ .set = ad74115_gpio_set,
+ };
+
+ return devm_gpiochip_add_data(dev, &st->gc, st);
+}
+
+static int ad74115_setup_comp_gpio_chip(struct ad74115_state *st)
+{
+ struct device *dev = &st->spi->dev;
+ u32 val;
+ int ret;
+
+ ret = regmap_read(st->regmap, AD74115_DIN_CONFIG1_REG, &val);
+ if (ret)
+ return ret;
+
+ if (!(val & AD74115_DIN_COMPARATOR_EN_MASK))
+ return 0;
+
+ st->comp_gc = (struct gpio_chip) {
+ .owner = THIS_MODULE,
+ .label = AD74115_NAME,
+ .base = -1,
+ .ngpio = 1,
+ .parent = dev,
+ .can_sleep = true,
+ .get_direction = ad74115_comp_gpio_get_direction,
+ .get = ad74115_comp_gpio_get,
+ .set_config = ad74115_comp_gpio_set_config,
+ };
+
+ return devm_gpiochip_add_data(dev, &st->comp_gc, st);
+}
+
+static int ad74115_setup(struct iio_dev *indio_dev)
+{
+ struct ad74115_state *st = iio_priv(indio_dev);
+ struct device *dev = &st->spi->dev;
+ u32 val, din_range_high;
+ unsigned int i;
+ int ret;
+
+ ret = ad74115_apply_fw_prop(st, &ad74115_ch_func_fw_prop, &val);
+ if (ret)
+ return ret;
+
+ indio_dev->num_channels += ad74115_channels_map[val].num_channels;
+ st->ch_func = val;
+
+ ret = ad74115_setup_adc_conv2_range(st);
+ if (ret)
+ return ret;
+
+ val = device_property_read_bool(dev, "adi,dac-hart-slew");
+ if (val) {
+ st->dac_hart_slew = val;
+
+ ret = regmap_update_bits(st->regmap, AD74115_OUTPUT_CONFIG_REG,
+ AD74115_OUTPUT_SLEW_EN_MASK,
+ FIELD_PREP(AD74115_OUTPUT_SLEW_EN_MASK,
+ AD74115_SLEW_MODE_HART));
+ if (ret)
+ return ret;
+ }
+
+ ret = ad74115_apply_fw_prop(st, &ad74115_din_range_fw_prop,
+ &din_range_high);
+ if (ret)
+ return ret;
+
+ ret = device_property_read_u32(dev, "adi,digital-input-sink-microamp", &val);
+ if (!ret) {
+ if (din_range_high)
+ val = DIV_ROUND_CLOSEST(val, AD74115_DIN_SINK_LOW_STEP);
+ else
+ val = DIV_ROUND_CLOSEST(val, AD74115_DIN_SINK_HIGH_STEP);
+
+ if (val > AD74115_DIN_SINK_MAX)
+ val = AD74115_DIN_SINK_MAX;
+
+ ret = regmap_update_bits(st->regmap, AD74115_DIN_CONFIG1_REG,
+ AD74115_DIN_SINK_MASK,
+ FIELD_PREP(AD74115_DIN_SINK_MASK, val));
+ if (ret)
+ return ret;
+ }
+
+ ret = ad74115_apply_fw_prop(st, &ad74115_din_threshold_mode_fw_prop, &val);
+ if (ret)
+ return ret;
+
+ if (val == AD74115_DIN_THRESHOLD_MODE_AVDD) {
+ ret = regulator_get_voltage(st->avdd);
+ if (ret < 0)
+ return ret;
+
+ st->avdd_mv = ret / 1000;
+ }
+
+ st->din_threshold_mode = val;
+
+ ret = ad74115_apply_fw_prop(st, &ad74115_dac_bipolar_fw_prop, &val);
+ if (ret)
+ return ret;
+
+ st->dac_bipolar = val;
+
+ ret = ad74115_apply_fw_prop(st, &ad74115_rtd_mode_fw_prop, &val);
+ if (ret)
+ return ret;
+
+ st->rtd_mode_4_wire = val;
+
+ ret = ad74115_apply_fw_prop(st, &ad74115_ext2_burnout_current_fw_prop, &val);
+ if (ret)
+ return ret;
+
+ if (val) {
+ ret = regmap_update_bits(st->regmap, AD74115_BURNOUT_CONFIG_REG,
+ AD74115_BURNOUT_EXT2_EN_MASK,
+ FIELD_PREP(AD74115_BURNOUT_EXT2_EN_MASK, 1));
+ if (ret)
+ return ret;
+ }
+
+ ret = ad74115_apply_fw_prop(st, &ad74115_ext1_burnout_current_fw_prop, &val);
+ if (ret)
+ return ret;
+
+ if (val) {
+ ret = regmap_update_bits(st->regmap, AD74115_BURNOUT_CONFIG_REG,
+ AD74115_BURNOUT_EXT1_EN_MASK,
+ FIELD_PREP(AD74115_BURNOUT_EXT1_EN_MASK, 1));
+ if (ret)
+ return ret;
+ }
+
+ ret = ad74115_apply_fw_prop(st, &ad74115_viout_burnout_current_fw_prop, &val);
+ if (ret)
+ return ret;
+
+ if (val) {
+ ret = regmap_update_bits(st->regmap, AD74115_BURNOUT_CONFIG_REG,
+ AD74115_BURNOUT_VIOUT_EN_MASK,
+ FIELD_PREP(AD74115_BURNOUT_VIOUT_EN_MASK, 1));
+ if (ret)
+ return ret;
+ }
+
+ for (i = 0; i < AD74115_GPIO_NUM; i++) {
+ ret = ad74115_apply_fw_prop(st, &ad74115_gpio_mode_fw_props[i], &val);
+ if (ret)
+ return ret;
+
+ if (val == AD74115_GPIO_MODE_LOGIC)
+ st->gpio_valid_mask |= BIT(i);
+ }
+
+ for (i = 0; i < ARRAY_SIZE(ad74115_fw_props); i++) {
+ ret = ad74115_apply_fw_prop(st, &ad74115_fw_props[i], &val);
+ if (ret)
+ return ret;
+ }
+
+ ret = ad74115_setup_gpio_chip(st);
+ if (ret)
+ return ret;
+
+ ret = ad74115_setup_comp_gpio_chip(st);
+ if (ret)
+ return ret;
+
+ return ad74115_setup_iio_channels(indio_dev);
+}
+
+static int ad74115_reset(struct ad74115_state *st)
+{
+ struct device *dev = &st->spi->dev;
+ struct gpio_desc *reset_gpio;
+ int ret;
+
+ reset_gpio = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_HIGH);
+ if (IS_ERR(reset_gpio))
+ return dev_err_probe(dev, PTR_ERR(reset_gpio),
+ "Failed to find reset GPIO\n");
+
+ if (reset_gpio) {
+ fsleep(100);
+
+ gpiod_set_value_cansleep(reset_gpio, 0);
+ } else {
+ ret = regmap_write(st->regmap, AD74115_CMD_KEY_REG,
+ AD74115_CMD_KEY_RESET1);
+ if (ret)
+ return ret;
+
+ ret = regmap_write(st->regmap, AD74115_CMD_KEY_REG,
+ AD74115_CMD_KEY_RESET2);
+ if (ret)
+ return ret;
+ }
+
+ fsleep(1000);
+
+ return 0;
+}
+
+static void ad74115_regulator_disable(void *data)
+{
+ regulator_disable(data);
+}
+
+static int ad74115_setup_trigger(struct iio_dev *indio_dev)
+{
+ struct ad74115_state *st = iio_priv(indio_dev);
+ struct device *dev = &st->spi->dev;
+ int ret;
+
+ st->irq = fwnode_irq_get_byname(dev_fwnode(dev), "adc_rdy");
+
+ if (st->irq == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
+
+ if (st->irq < 0) {
+ st->irq = 0;
+ return 0;
+ }
+
+ ret = devm_request_irq(dev, st->irq, ad74115_adc_data_interrupt,
+ 0, AD74115_NAME, indio_dev);
+ if (ret)
+ return ret;
+
+ st->trig = devm_iio_trigger_alloc(dev, "%s-dev%d", AD74115_NAME,
+ iio_device_id(indio_dev));
+ if (!st->trig)
+ return -ENOMEM;
+
+ st->trig->ops = &ad74115_trigger_ops;
+ iio_trigger_set_drvdata(st->trig, st);
+
+ ret = devm_iio_trigger_register(dev, st->trig);
+ if (ret)
+ return ret;
+
+ indio_dev->trig = iio_trigger_get(st->trig);
+
+ return 0;
+}
+
+static int ad74115_probe(struct spi_device *spi)
+{
+ static const char * const regulator_names[] = {
+ "avcc", "dvcc", "dovdd", "refin",
+ };
+ struct device *dev = &spi->dev;
+ struct ad74115_state *st;
+ struct iio_dev *indio_dev;
+ int ret;
+
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*st));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ st = iio_priv(indio_dev);
+
+ st->spi = spi;
+ mutex_init(&st->lock);
+ init_completion(&st->adc_data_completion);
+
+ indio_dev->name = AD74115_NAME;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+ indio_dev->info = &ad74115_info;
+
+ st->avdd = devm_regulator_get(dev, "avdd");
+ if (IS_ERR(st->avdd))
+ return PTR_ERR(st->avdd);
+
+ ret = regulator_enable(st->avdd);
+ if (ret) {
+ dev_err(dev, "Failed to enable avdd regulator\n");
+ return ret;
+ }
+
+ ret = devm_add_action_or_reset(dev, ad74115_regulator_disable, st->avdd);
+ if (ret)
+ return ret;
+
+ ret = devm_regulator_bulk_get_enable(dev, ARRAY_SIZE(regulator_names),
+ regulator_names);
+ if (ret)
+ return ret;
+
+ st->regmap = devm_regmap_init(dev, NULL, st, &ad74115_regmap_config);
+ if (IS_ERR(st->regmap))
+ return PTR_ERR(st->regmap);
+
+ ret = ad74115_reset(st);
+ if (ret)
+ return ret;
+
+ ret = ad74115_setup(indio_dev);
+ if (ret)
+ return ret;
+
+ ret = ad74115_setup_trigger(indio_dev);
+ if (ret)
+ return ret;
+
+ ret = devm_iio_triggered_buffer_setup(dev, indio_dev, NULL,
+ ad74115_trigger_handler,
+ &ad74115_buffer_ops);
+ if (ret)
+ return ret;
+
+ return devm_iio_device_register(dev, indio_dev);
+}
+
+static int ad74115_unregister_driver(struct spi_driver *spi)
+{
+ spi_unregister_driver(spi);
+
+ return 0;
+}
+
+static int __init ad74115_register_driver(struct spi_driver *spi)
+{
+ crc8_populate_msb(ad74115_crc8_table, AD74115_CRC_POLYNOMIAL);
+
+ return spi_register_driver(spi);
+}
+
+static const struct spi_device_id ad74115_spi_id[] = {
+ { "ad74115h" },
+ { }
+};
+
+MODULE_DEVICE_TABLE(spi, ad74115_spi_id);
+
+static const struct of_device_id ad74115_dt_id[] = {
+ { .compatible = "adi,ad74115h" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, ad74115_dt_id);
+
+static struct spi_driver ad74115_driver = {
+ .driver = {
+ .name = "ad74115",
+ .of_match_table = ad74115_dt_id,
+ },
+ .probe = ad74115_probe,
+ .id_table = ad74115_spi_id,
+};
+
+module_driver(ad74115_driver,
+ ad74115_register_driver, ad74115_unregister_driver);
+
+MODULE_AUTHOR("Cosmin Tanislav <cosmin.tanislav@analog.com>");
+MODULE_DESCRIPTION("Analog Devices AD74115 ADDAC");
+MODULE_LICENSE("GPL");
struct regmap *regmap;
struct device *dev;
struct iio_trigger *trig;
+ struct gpio_desc *reset_gpio;
size_t adc_active_channels;
struct spi_message adc_samples_msg;
{
int ret;
+ if (st->reset_gpio) {
+ gpiod_set_value_cansleep(st->reset_gpio, 1);
+ fsleep(50);
+ gpiod_set_value_cansleep(st->reset_gpio, 0);
+ return 0;
+ }
+
ret = regmap_write(st->regmap, AD74413R_REG_CMD_KEY,
AD74413R_CMD_KEY_RESET1);
if (ret)
if (ret)
return ret;
- *val = voltage_offset * AD74413R_ADC_RESULT_MAX / voltage_range;
+ *val = voltage_offset * (int)AD74413R_ADC_RESULT_MAX / voltage_range;
return IIO_VAL_INT;
}
st->spi = spi;
st->dev = &spi->dev;
st->chip_info = device_get_match_data(&spi->dev);
+ if (!st->chip_info) {
+ const struct spi_device_id *id = spi_get_device_id(spi);
+
+ if (id)
+ st->chip_info =
+ (struct ad74413r_chip_info *)id->driver_data;
+ if (!st->chip_info)
+ return -EINVAL;
+ }
+
mutex_init(&st->lock);
init_completion(&st->adc_data_completion);
if (IS_ERR(st->regmap))
return PTR_ERR(st->regmap);
+ st->reset_gpio = devm_gpiod_get_optional(st->dev, "reset", GPIOD_OUT_LOW);
+ if (IS_ERR(st->reset_gpio))
+ return PTR_ERR(st->reset_gpio);
+
st->refin_reg = devm_regulator_get(st->dev, "refin");
if (IS_ERR(st->refin_reg))
return dev_err_probe(st->dev, PTR_ERR(st->refin_reg),
};
MODULE_DEVICE_TABLE(of, ad74413r_dt_id);
+static const struct spi_device_id ad74413r_spi_id[] = {
+ { .name = "ad74412r", .driver_data = (kernel_ulong_t)&ad74412r_chip_info_data },
+ { .name = "ad74413r", .driver_data = (kernel_ulong_t)&ad74413r_chip_info_data },
+ {}
+};
+MODULE_DEVICE_TABLE(spi, ad74413r_spi_id);
+
static struct spi_driver ad74413r_driver = {
.driver = {
.name = "ad74413r",
.of_match_table = ad74413r_dt_id,
},
.probe = ad74413r_probe,
+ .id_table = ad74413r_spi_id,
};
module_driver(ad74413r_driver,
};
struct hmc425a_state {
- struct regulator *reg;
struct mutex lock; /* protect sensor state */
struct hmc425a_chip_info *chip_info;
struct gpio_descs *gpios;
};
MODULE_DEVICE_TABLE(of, hmc425a_of_match);
-static void hmc425a_reg_disable(void *data)
-{
- struct hmc425a_state *st = data;
-
- regulator_disable(st->reg);
-}
-
static struct hmc425a_chip_info hmc425a_chip_info_tbl[] = {
[ID_HMC425A] = {
.name = "hmc425a",
return -ENODEV;
}
- st->reg = devm_regulator_get(&pdev->dev, "vcc-supply");
- if (IS_ERR(st->reg))
- return PTR_ERR(st->reg);
-
- ret = regulator_enable(st->reg);
- if (ret)
- return ret;
- ret = devm_add_action_or_reset(&pdev->dev, hmc425a_reg_disable, st);
+ ret = devm_regulator_get_enable(&pdev->dev, "vcc-supply");
if (ret)
return ret;
static IIO_DEVICE_ATTR(length_align_bytes, 0444,
iio_dmaengine_buffer_get_length_align, NULL, 0);
-static const struct attribute *iio_dmaengine_buffer_attrs[] = {
- &iio_dev_attr_length_align_bytes.dev_attr.attr,
+static const struct iio_dev_attr *iio_dmaengine_buffer_attrs[] = {
+ &iio_dev_attr_length_align_bytes,
NULL,
};
irqreturn_t (*thread)(int irq, void *p),
enum iio_buffer_direction direction,
const struct iio_buffer_setup_ops *setup_ops,
- const struct attribute **buffer_attrs)
+ const struct iio_dev_attr **buffer_attrs)
{
struct iio_buffer *buffer;
int ret;
irqreturn_t (*thread)(int irq, void *p),
enum iio_buffer_direction direction,
const struct iio_buffer_setup_ops *ops,
- const struct attribute **buffer_attrs)
+ const struct iio_dev_attr **buffer_attrs)
{
int ret;
int devm_iio_kfifo_buffer_setup_ext(struct device *dev,
struct iio_dev *indio_dev,
const struct iio_buffer_setup_ops *setup_ops,
- const struct attribute **buffer_attrs)
+ const struct iio_dev_attr **buffer_attrs)
{
struct iio_buffer *buffer;
.read_raw = &ad7150_read_raw,
};
-static void ad7150_reg_disable(void *data)
-{
- struct regulator *reg = data;
-
- regulator_disable(reg);
-}
-
-static int ad7150_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ad7150_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct ad7150_chip_info *chip;
struct iio_dev *indio_dev;
- struct regulator *reg;
int ret;
indio_dev = devm_iio_device_alloc(&client->dev, sizeof(*chip));
indio_dev->modes = INDIO_DIRECT_MODE;
- reg = devm_regulator_get(&client->dev, "vdd");
- if (IS_ERR(reg))
- return PTR_ERR(reg);
-
- ret = regulator_enable(reg);
- if (ret)
- return ret;
-
- ret = devm_add_action_or_reset(&client->dev, ad7150_reg_disable, reg);
+ ret = devm_regulator_get_enable(&client->dev, "vdd");
if (ret)
return ret;
.name = "ad7150",
.of_match_table = ad7150_of_match,
},
- .probe = ad7150_probe,
+ .probe_new = ad7150_probe,
.id_table = ad7150_id,
};
module_i2c_driver(ad7150_driver);
.write_raw = ad7746_write_raw,
};
-static int ad7746_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ad7746_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct ad7746_chip_info *chip;
struct iio_dev *indio_dev;
.name = KBUILD_MODNAME,
.of_match_table = ad7746_of_match,
},
- .probe = ad7746_probe,
+ .probe_new = ad7746_probe,
.id_table = ad7746_id,
};
module_i2c_driver(ad7746_driver);
.read_raw = ams_iaqcore_read_raw,
};
-static int ams_iaqcore_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ams_iaqcore_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct ams_iaqcore_data *data;
.name = "ams-iaq-core",
.of_match_table = ams_iaqcore_dt_ids,
},
- .probe = ams_iaqcore_probe,
+ .probe_new = ams_iaqcore_probe,
.id_table = ams_iaqcore_id,
};
module_i2c_driver(ams_iaqcore_driver);
};
MODULE_DEVICE_TABLE(of, atlas_ezo_dt_ids);
-static int atlas_ezo_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int atlas_ezo_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
const struct atlas_ezo_device *chip;
struct atlas_ezo_data *data;
struct iio_dev *indio_dev;
.name = ATLAS_EZO_DRV_NAME,
.of_match_table = atlas_ezo_dt_ids,
},
- .probe = atlas_ezo_probe,
+ .probe_new = atlas_ezo_probe,
.id_table = atlas_ezo_id,
};
module_i2c_driver(atlas_ezo_driver);
};
MODULE_DEVICE_TABLE(of, atlas_dt_ids);
-static int atlas_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int atlas_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct atlas_data *data;
struct atlas_device *chip;
struct iio_trigger *trig;
.of_match_table = atlas_dt_ids,
.pm = pm_ptr(&atlas_pm_ops),
},
- .probe = atlas_probe,
+ .probe_new = atlas_probe,
.remove = atlas_remove,
.id_table = atlas_id,
};
#include "bme680.h"
-static int bme680_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bme680_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct regmap *regmap;
const char *name = NULL;
.name = "bme680_i2c",
.of_match_table = bme680_of_i2c_match,
},
- .probe = bme680_i2c_probe,
+ .probe_new = bme680_i2c_probe,
.id_table = bme680_i2c_id,
};
module_i2c_driver(bme680_i2c_driver);
return 0;
}
-static int ccs811_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ccs811_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct iio_dev *indio_dev;
struct ccs811_data *data;
int ret;
.name = "ccs811",
.of_match_table = ccs811_dt_ids,
},
- .probe = ccs811_probe,
+ .probe_new = ccs811_probe,
.remove = ccs811_remove,
.id_table = ccs811_id,
};
return IRQ_HANDLED;
}
-static int scd4x_probe(struct i2c_client *client, const struct i2c_device_id *id)
+static int scd4x_probe(struct i2c_client *client)
{
static const unsigned long scd4x_scan_masks[] = { 0x07, 0x00 };
struct device *dev = &client->dev;
.of_match_table = scd4x_dt_ids,
.pm = pm_sleep_ptr(&scd4x_pm_ops),
},
- .probe = scd4x_probe,
+ .probe_new = scd4x_probe,
};
module_i2c_driver(scd4x_i2c_driver);
{ }
};
-static int sgp_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int sgp_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct iio_dev *indio_dev;
struct sgp_data *data;
.name = "sgp30",
.of_match_table = sgp_dt_ids,
},
- .probe = sgp_probe,
+ .probe_new = sgp_probe,
.remove = sgp_remove,
.id_table = sgp_id,
};
.write_raw = sgp40_write_raw,
};
-static int sgp40_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int sgp40_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct iio_dev *indio_dev;
struct sgp40_data *data;
.name = "sgp40",
.of_match_table = sgp40_dt_ids,
},
- .probe = sgp40_probe,
+ .probe_new = sgp40_probe,
.id_table = sgp40_id,
};
module_i2c_driver(sgp40_driver);
};
MODULE_DEVICE_TABLE(of, vz89x_dt_ids);
-static int vz89x_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int vz89x_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct iio_dev *indio_dev;
struct vz89x_data *data;
.name = "vz89x",
.of_match_table = vz89x_dt_ids,
},
- .probe = vz89x_probe,
+ .probe_new = vz89x_probe,
.id_table = vz89x_id,
};
module_i2c_driver(vz89x_driver);
static IIO_DEVICE_ATTR_RO(hwfifo_watermark_max, 0);
-static const struct attribute *cros_ec_sensor_fifo_attributes[] = {
- &iio_dev_attr_hwfifo_timeout.dev_attr.attr,
- &iio_dev_attr_hwfifo_watermark_max.dev_attr.attr,
+static const struct iio_dev_attr *cros_ec_sensor_fifo_attributes[] = {
+ &iio_dev_attr_hwfifo_timeout,
+ &iio_dev_attr_hwfifo_watermark_max,
NULL,
};
static IIO_DEVICE_ATTR(hwfifo_enabled, 0444,
_hid_sensor_get_fifo_state, NULL, 0);
-static const struct attribute *hid_sensor_fifo_attributes[] = {
- &iio_dev_attr_hwfifo_timeout.dev_attr.attr,
- &iio_dev_attr_hwfifo_enabled.dev_attr.attr,
+static const struct iio_dev_attr *hid_sensor_fifo_attributes[] = {
+ &iio_dev_attr_hwfifo_timeout,
+ &iio_dev_attr_hwfifo_enabled,
NULL,
};
int hid_sensor_setup_trigger(struct iio_dev *indio_dev, const char *name,
struct hid_sensor_common *attrb)
{
- const struct attribute **fifo_attrs;
+ const struct iio_dev_attr **fifo_attrs;
int ret;
struct iio_trigger *trig;
#include <linux/kernel.h>
#include <linux/kthread.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/scmi_protocol.h>
#include <linux/time.h>
#include <linux/types.h>
struct scmi_protocol_handle *ph;
const struct scmi_sensor_info *sensor_info;
struct iio_dev *indio_dev;
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
/* adding one additional channel for timestamp */
s64 iio_buf[SCMI_IIO_NUM_OF_AXIS + 1];
struct notifier_block sensor_update_nb;
struct iio_chan_spec const *chan, int val,
int val2, long mask)
{
+ struct scmi_iio_priv *sensor = iio_priv(iio_dev);
int err;
switch (mask) {
case IIO_CHAN_INFO_SAMP_FREQ:
- mutex_lock(&iio_dev->mlock);
+ mutex_lock(&sensor->lock);
err = scmi_iio_set_odr_val(iio_dev, val, val2);
- mutex_unlock(&iio_dev->mlock);
+ mutex_unlock(&sensor->lock);
return err;
default:
return -EINVAL;
sensor->sensor_info = sensor_info;
sensor->sensor_update_nb.notifier_call = scmi_iio_sensor_update_cb;
sensor->indio_dev = iiodev;
+ mutex_init(&sensor->lock);
/* adding one additional channel for timestamp */
iiodev->num_channels = sensor_info->num_axis + 1;
}
EXPORT_SYMBOL_NS(st_sensors_set_axis_enable, IIO_ST_SENSORS);
-static void st_reg_disable(void *reg)
-{
- regulator_disable(reg);
-}
int st_sensors_power_enable(struct iio_dev *indio_dev)
{
- struct st_sensor_data *pdata = iio_priv(indio_dev);
+ static const char * const regulator_names[] = { "vdd", "vddio" };
struct device *parent = indio_dev->dev.parent;
int err;
/* Regulators not mandatory, but if requested we should enable them. */
- pdata->vdd = devm_regulator_get(parent, "vdd");
- if (IS_ERR(pdata->vdd))
- return dev_err_probe(&indio_dev->dev, PTR_ERR(pdata->vdd),
- "unable to get Vdd supply\n");
-
- err = regulator_enable(pdata->vdd);
- if (err != 0) {
- dev_warn(&indio_dev->dev,
- "Failed to enable specified Vdd supply\n");
- return err;
- }
-
- err = devm_add_action_or_reset(parent, st_reg_disable, pdata->vdd);
+ err = devm_regulator_bulk_get_enable(parent,
+ ARRAY_SIZE(regulator_names),
+ regulator_names);
if (err)
- return err;
+ return dev_err_probe(&indio_dev->dev, err,
+ "unable to enable supplies\n");
- pdata->vdd_io = devm_regulator_get(parent, "vddio");
- if (IS_ERR(pdata->vdd_io))
- return dev_err_probe(&indio_dev->dev, PTR_ERR(pdata->vdd_io),
- "unable to get Vdd_IO supply\n");
-
- err = regulator_enable(pdata->vdd_io);
- if (err != 0) {
- dev_warn(&indio_dev->dev,
- "Failed to enable specified Vdd_IO supply\n");
- return err;
- }
-
- return devm_add_action_or_reset(parent, st_reg_disable, pdata->vdd_io);
+ return 0;
}
EXPORT_SYMBOL_NS(st_sensors_power_enable, IIO_ST_SENSORS);
return 0;
}
-static int ad5064_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int ad5064_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
return ad5064_probe(&i2c->dev, id->driver_data, id->name,
ad5064_i2c_write);
}
.driver = {
.name = "ad5064",
},
- .probe = ad5064_i2c_probe,
+ .probe_new = ad5064_i2c_probe,
.id_table = ad5064_i2c_ids,
};
#if IS_ENABLED(CONFIG_I2C)
-static int ad5380_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int ad5380_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct regmap *regmap;
regmap = devm_regmap_init_i2c(i2c, &ad5380_regmap_config);
.driver = {
.name = "ad5380",
},
- .probe = ad5380_i2c_probe,
+ .probe_new = ad5380_i2c_probe,
.remove = ad5380_i2c_remove,
.id_table = ad5380_i2c_ids,
};
},
};
-static int ad5446_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int ad5446_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
return ad5446_probe(&i2c->dev, id->name,
&ad5446_i2c_chip_info[id->driver_data]);
}
.driver = {
.name = "ad5446",
},
- .probe = ad5446_i2c_probe,
+ .probe_new = ad5446_i2c_probe,
.remove = ad5446_i2c_remove,
.id_table = ad5446_i2c_ids,
};
.gpio_read = ad5593r_gpio_read,
};
-static int ad5593r_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int ad5593r_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
if (!i2c_check_functionality(i2c->adapter,
I2C_FUNC_SMBUS_BYTE | I2C_FUNC_I2C))
return -EOPNOTSUPP;
.of_match_table = ad5593r_of_match,
.acpi_match_table = ad5593r_acpi_match,
},
- .probe = ad5593r_i2c_probe,
+ .probe_new = ad5593r_i2c_probe,
.remove = ad5593r_i2c_remove,
.id_table = ad5593r_i2c_ids,
};
return (ret != 3) ? -EIO : 0;
}
-static int ad5686_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int ad5686_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
return ad5686_probe(&i2c->dev, id->driver_data, id->name,
ad5686_i2c_write, ad5686_i2c_read);
}
.name = "ad5696",
.of_match_table = ad5686_of_match,
},
- .probe = ad5686_i2c_probe,
+ .probe_new = ad5686_i2c_probe,
.remove = ad5686_i2c_remove,
.id_table = ad5686_i2c_id,
};
.write_raw = ds4424_write_raw,
};
-static int ds4424_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ds4424_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct ds4424_data *data;
struct iio_dev *indio_dev;
int ret;
.of_match_table = ds4424_of_match,
.pm = pm_sleep_ptr(&ds4424_pm_ops),
},
- .probe = ds4424_probe,
+ .probe_new = ds4424_probe,
.remove = ds4424_remove,
.id_table = ds4424_id,
};
struct ltc2688_state {
struct spi_device *spi;
struct regmap *regmap;
- struct regulator_bulk_data regulators[2];
struct ltc2688_chan channels[LTC2688_DAC_CHANNELS];
struct iio_chan_spec *iio_chan;
/* lock to protect against multiple access to the device and shared data */
LTC2688_CONFIG_EXT_REF);
}
-static void ltc2688_disable_regulators(void *data)
-{
- struct ltc2688_state *st = data;
-
- regulator_bulk_disable(ARRAY_SIZE(st->regulators), st->regulators);
-}
-
static void ltc2688_disable_regulator(void *regulator)
{
regulator_disable(regulator);
static int ltc2688_probe(struct spi_device *spi)
{
+ static const char * const regulators[] = { "vcc", "iovcc" };
struct ltc2688_state *st;
struct iio_dev *indio_dev;
struct regulator *vref_reg;
return dev_err_probe(dev, PTR_ERR(st->regmap),
"Failed to init regmap");
- st->regulators[0].supply = "vcc";
- st->regulators[1].supply = "iovcc";
- ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(st->regulators),
- st->regulators);
- if (ret)
- return dev_err_probe(dev, ret, "Failed to get regulators\n");
-
- ret = regulator_bulk_enable(ARRAY_SIZE(st->regulators), st->regulators);
+ ret = devm_regulator_bulk_get_enable(dev, ARRAY_SIZE(regulators),
+ regulators);
if (ret)
return dev_err_probe(dev, ret, "Failed to enable regulators\n");
- ret = devm_add_action_or_reset(dev, ltc2688_disable_regulators, st);
- if (ret)
- return ret;
-
vref_reg = devm_regulator_get_optional(dev, "vref");
if (IS_ERR(vref_reg)) {
if (PTR_ERR(vref_reg) != -ENODEV)
M62332_CHANNEL(1)
};
-static int m62332_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int m62332_probe(struct i2c_client *client)
{
struct m62332_data *data;
struct iio_dev *indio_dev;
.name = "m62332",
.pm = pm_sleep_ptr(&m62332_pm_ops),
},
- .probe = m62332_probe,
+ .probe_new = m62332_probe,
.remove = m62332_remove,
.id_table = m62332_id,
};
MAX517_CHANNEL(7),
};
-static int max517_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max517_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct max517_data *data;
struct iio_dev *indio_dev;
struct max517_platform_data *platform_data = client->dev.platform_data;
.name = MAX517_DRV_NAME,
.pm = pm_sleep_ptr(&max517_pm_ops),
},
- .probe = max517_probe,
+ .probe_new = max517_probe,
.id_table = max517_id,
};
module_i2c_driver(max517_driver);
regulator_disable(reg);
}
-static int max5821_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max5821_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct max5821_data *data;
struct iio_dev *indio_dev;
u32 tmp;
.of_match_table = max5821_of_match,
.pm = pm_sleep_ptr(&max5821_pm_ops),
},
- .probe = max5821_probe,
+ .probe_new = max5821_probe,
.id_table = max5821_id,
};
module_i2c_driver(max5821_driver);
return 0;
}
-static int mcp4725_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mcp4725_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct mcp4725_data *data;
struct iio_dev *indio_dev;
struct mcp4725_platform_data *pdata, pdata_dt;
.of_match_table = mcp4725_of_match,
.pm = pm_sleep_ptr(&mcp4725_pm_ops),
},
- .probe = mcp4725_probe,
+ .probe_new = mcp4725_probe,
.remove = mcp4725_remove,
.id_table = mcp4725_id,
};
.write_raw_get_fmt = dac5571_write_raw_get_fmt,
};
-static int dac5571_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int dac5571_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
const struct dac5571_spec *spec;
struct dac5571_data *data;
.name = "ti-dac5571",
.of_match_table = dac5571_of_id,
},
- .probe = dac5571_probe,
+ .probe_new = dac5571_probe,
.remove = dac5571_remove,
.id_table = dac5571_id,
};
return ret;
hpf_band = FIELD_GET(ADMV8818_SW_IN_WR0_MSK, data);
- if (!hpf_band) {
+ if (!hpf_band || hpf_band > 4) {
*hpf_freq = 0;
return ret;
}
return ret;
lpf_band = FIELD_GET(ADMV8818_SW_OUT_WR0_MSK, data);
- if (!lpf_band) {
+ if (!lpf_band || lpf_band > 4) {
*lpf_freq = 0;
return ret;
}
To compile this driver as a module, choose M here: the
module will be called adf4371.
+config ADF4377
+ tristate "Analog Devices ADF4377 Microwave Wideband Synthesizer"
+ depends on SPI && COMMON_CLK
+ help
+ Say yes here to build support for Analog Devices ADF4377 Microwave
+ Wideband Synthesizer.
+
+ To compile this driver as a module, choose M here: the
+ module will be called adf4377.
+
config ADMV1013
tristate "Analog Devices ADMV1013 Microwave Upconverter"
depends on SPI && COMMON_CLK
obj-$(CONFIG_AD9523) += ad9523.o
obj-$(CONFIG_ADF4350) += adf4350.o
obj-$(CONFIG_ADF4371) += adf4371.o
+obj-$(CONFIG_ADF4377) += adf4377.o
obj-$(CONFIG_ADMV1013) += admv1013.o
obj-$(CONFIG_ADMV1014) += admv1014.o
obj-$(CONFIG_ADMV4420) += admv4420.o
struct ad9523_state {
struct spi_device *spi;
- struct regulator *reg;
struct ad9523_platform_data *pdata;
struct iio_chan_spec ad9523_channels[AD9523_NUM_CHAN];
struct gpio_desc *pwrdown_gpio;
return 0;
}
-static void ad9523_reg_disable(void *data)
-{
- struct regulator *reg = data;
-
- regulator_disable(reg);
-}
-
static int ad9523_probe(struct spi_device *spi)
{
struct ad9523_platform_data *pdata = spi->dev.platform_data;
mutex_init(&st->lock);
- st->reg = devm_regulator_get(&spi->dev, "vcc");
- if (!IS_ERR(st->reg)) {
- ret = regulator_enable(st->reg);
- if (ret)
- return ret;
-
- ret = devm_add_action_or_reset(&spi->dev, ad9523_reg_disable,
- st->reg);
- if (ret)
- return ret;
- }
+ ret = devm_regulator_get_enable(&spi->dev, "vcc");
+ if (ret)
+ return ret;
st->pwrdown_gpio = devm_gpiod_get_optional(&spi->dev, "powerdown",
GPIOD_OUT_HIGH);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * ADF4377 driver
+ *
+ * Copyright 2022 Analog Devices Inc.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/bits.h>
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/gpio/consumer.h>
+#include <linux/module.h>
+#include <linux/notifier.h>
+#include <linux/property.h>
+#include <linux/spi/spi.h>
+#include <linux/iio/iio.h>
+#include <linux/regmap.h>
+#include <linux/units.h>
+
+#include <asm/unaligned.h>
+
+/* ADF4377 REG0000 Map */
+#define ADF4377_0000_SOFT_RESET_R_MSK BIT(7)
+#define ADF4377_0000_LSB_FIRST_R_MSK BIT(6)
+#define ADF4377_0000_ADDRESS_ASC_R_MSK BIT(5)
+#define ADF4377_0000_SDO_ACTIVE_R_MSK BIT(4)
+#define ADF4377_0000_SDO_ACTIVE_MSK BIT(3)
+#define ADF4377_0000_ADDRESS_ASC_MSK BIT(2)
+#define ADF4377_0000_LSB_FIRST_MSK BIT(1)
+#define ADF4377_0000_SOFT_RESET_MSK BIT(0)
+
+/* ADF4377 REG0000 Bit Definition */
+#define ADF4377_0000_SDO_ACTIVE_SPI_3W 0x0
+#define ADF4377_0000_SDO_ACTIVE_SPI_4W 0x1
+
+#define ADF4377_0000_ADDR_ASC_AUTO_DECR 0x0
+#define ADF4377_0000_ADDR_ASC_AUTO_INCR 0x1
+
+#define ADF4377_0000_LSB_FIRST_MSB 0x0
+#define ADF4377_0000_LSB_FIRST_LSB 0x1
+
+#define ADF4377_0000_SOFT_RESET_N_OP 0x0
+#define ADF4377_0000_SOFT_RESET_EN 0x1
+
+/* ADF4377 REG0001 Map */
+#define ADF4377_0001_SINGLE_INSTR_MSK BIT(7)
+#define ADF4377_0001_MASTER_RB_CTRL_MSK BIT(5)
+
+/* ADF4377 REG0003 Bit Definition */
+#define ADF4377_0003_CHIP_TYPE 0x06
+
+/* ADF4377 REG0004 Bit Definition */
+#define ADF4377_0004_PRODUCT_ID_LSB 0x0005
+
+/* ADF4377 REG0005 Bit Definition */
+#define ADF4377_0005_PRODUCT_ID_MSB 0x0005
+
+/* ADF4377 REG000A Map */
+#define ADF4377_000A_SCRATCHPAD_MSK GENMASK(7, 0)
+
+/* ADF4377 REG000C Bit Definition */
+#define ADF4377_000C_VENDOR_ID_LSB 0x56
+
+/* ADF4377 REG000D Bit Definition */
+#define ADF4377_000D_VENDOR_ID_MSB 0x04
+
+/* ADF4377 REG000F Bit Definition */
+#define ADF4377_000F_R00F_RSV1_MSK GENMASK(7, 0)
+
+/* ADF4377 REG0010 Map*/
+#define ADF4377_0010_N_INT_LSB_MSK GENMASK(7, 0)
+
+/* ADF4377 REG0011 Map*/
+#define ADF4377_0011_EN_AUTOCAL_MSK BIT(7)
+#define ADF4377_0011_EN_RDBLR_MSK BIT(6)
+#define ADF4377_0011_DCLK_DIV2_MSK GENMASK(5, 4)
+#define ADF4377_0011_N_INT_MSB_MSK GENMASK(3, 0)
+
+/* ADF4377 REG0011 Bit Definition */
+#define ADF4377_0011_DCLK_DIV2_1 0x0
+#define ADF4377_0011_DCLK_DIV2_2 0x1
+#define ADF4377_0011_DCLK_DIV2_4 0x2
+#define ADF4377_0011_DCLK_DIV2_8 0x3
+
+/* ADF4377 REG0012 Map*/
+#define ADF4377_0012_CLKOUT_DIV_MSK GENMASK(7, 6)
+#define ADF4377_0012_R_DIV_MSK GENMASK(5, 0)
+
+/* ADF4377 REG0012 Bit Definition */
+#define ADF4377_0012_CLKOUT_DIV_1 0x0
+#define ADF4377_0012_CLKOUT_DIV_2 0x1
+#define ADF4377_0012_CLKOUT_DIV_4 0x2
+#define ADF4377_0012_CLKOUT_DIV_8 0x3
+
+/* ADF4377 REG0013 Map */
+#define ADF4377_0013_M_VCO_CORE_MSK GENMASK(5, 4)
+#define ADF4377_0013_VCO_BIAS_MSK GENMASK(3, 0)
+
+/* ADF4377 REG0013 Bit Definition */
+#define ADF4377_0013_M_VCO_0 0x0
+#define ADF4377_0013_M_VCO_1 0x1
+#define ADF4377_0013_M_VCO_2 0x2
+#define ADF4377_0013_M_VCO_3 0x3
+
+/* ADF4377 REG0014 Map */
+#define ADF4377_0014_M_VCO_BAND_MSK GENMASK(7, 0)
+
+/* ADF4377 REG0015 Map */
+#define ADF4377_0015_BLEED_I_LSB_MSK GENMASK(7, 6)
+#define ADF4377_0015_BLEED_POL_MSK BIT(5)
+#define ADF4377_0015_EN_BLEED_MSK BIT(4)
+#define ADF4377_0015_CP_I_MSK GENMASK(3, 0)
+
+/* ADF4377 REG0015 Bit Definition */
+#define ADF4377_CURRENT_SINK 0x0
+#define ADF4377_CURRENT_SOURCE 0x1
+
+#define ADF4377_0015_CP_0MA7 0x0
+#define ADF4377_0015_CP_0MA9 0x1
+#define ADF4377_0015_CP_1MA1 0x2
+#define ADF4377_0015_CP_1MA3 0x3
+#define ADF4377_0015_CP_1MA4 0x4
+#define ADF4377_0015_CP_1MA8 0x5
+#define ADF4377_0015_CP_2MA2 0x6
+#define ADF4377_0015_CP_2MA5 0x7
+#define ADF4377_0015_CP_2MA9 0x8
+#define ADF4377_0015_CP_3MA6 0x9
+#define ADF4377_0015_CP_4MA3 0xA
+#define ADF4377_0015_CP_5MA0 0xB
+#define ADF4377_0015_CP_5MA7 0xC
+#define ADF4377_0015_CP_7MA2 0xD
+#define ADF4377_0015_CP_8MA6 0xE
+#define ADF4377_0015_CP_10MA1 0xF
+
+/* ADF4377 REG0016 Map */
+#define ADF4377_0016_BLEED_I_MSB_MSK GENMASK(7, 0)
+
+/* ADF4377 REG0017 Map */
+#define ADF4377_0016_INV_CLKOUT_MSK BIT(7)
+#define ADF4377_0016_N_DEL_MSK GENMASK(6, 0)
+
+/* ADF4377 REG0018 Map */
+#define ADF4377_0018_CMOS_OV_MSK BIT(7)
+#define ADF4377_0018_R_DEL_MSK GENMASK(6, 0)
+
+/* ADF4377 REG0018 Bit Definition */
+#define ADF4377_0018_1V8_LOGIC 0x0
+#define ADF4377_0018_3V3_LOGIC 0x1
+
+/* ADF4377 REG0019 Map */
+#define ADF4377_0019_CLKOUT2_OP_MSK GENMASK(7, 6)
+#define ADF4377_0019_CLKOUT1_OP_MSK GENMASK(5, 4)
+#define ADF4377_0019_PD_CLK_MSK BIT(3)
+#define ADF4377_0019_PD_RDET_MSK BIT(2)
+#define ADF4377_0019_PD_ADC_MSK BIT(1)
+#define ADF4377_0019_PD_CALADC_MSK BIT(0)
+
+/* ADF4377 REG0019 Bit Definition */
+#define ADF4377_0019_CLKOUT_320MV 0x0
+#define ADF4377_0019_CLKOUT_420MV 0x1
+#define ADF4377_0019_CLKOUT_530MV 0x2
+#define ADF4377_0019_CLKOUT_640MV 0x3
+
+/* ADF4377 REG001A Map */
+#define ADF4377_001A_PD_ALL_MSK BIT(7)
+#define ADF4377_001A_PD_RDIV_MSK BIT(6)
+#define ADF4377_001A_PD_NDIV_MSK BIT(5)
+#define ADF4377_001A_PD_VCO_MSK BIT(4)
+#define ADF4377_001A_PD_LD_MSK BIT(3)
+#define ADF4377_001A_PD_PFDCP_MSK BIT(2)
+#define ADF4377_001A_PD_CLKOUT1_MSK BIT(1)
+#define ADF4377_001A_PD_CLKOUT2_MSK BIT(0)
+
+/* ADF4377 REG001B Map */
+#define ADF4377_001B_EN_LOL_MSK BIT(7)
+#define ADF4377_001B_LDWIN_PW_MSK BIT(6)
+#define ADF4377_001B_EN_LDWIN_MSK BIT(5)
+#define ADF4377_001B_LD_COUNT_MSK GENMASK(4, 0)
+
+/* ADF4377 REG001B Bit Definition */
+#define ADF4377_001B_LDWIN_PW_NARROW 0x0
+#define ADF4377_001B_LDWIN_PW_WIDE 0x1
+
+/* ADF4377 REG001C Map */
+#define ADF4377_001C_EN_DNCLK_MSK BIT(7)
+#define ADF4377_001C_EN_DRCLK_MSK BIT(6)
+#define ADF4377_001C_RST_LD_MSK BIT(2)
+#define ADF4377_001C_R01C_RSV1_MSK BIT(0)
+
+/* ADF4377 REG001C Bit Definition */
+#define ADF4377_001C_RST_LD_INACTIVE 0x0
+#define ADF4377_001C_RST_LD_ACTIVE 0x1
+
+#define ADF4377_001C_R01C_RSV1 0x1
+
+/* ADF4377 REG001D Map */
+#define ADF4377_001D_MUXOUT_MSK GENMASK(7, 4)
+#define ADF4377_001D_EN_CPTEST_MSK BIT(2)
+#define ADF4377_001D_CP_DOWN_MSK BIT(1)
+#define ADF4377_001D_CP_UP_MSK BIT(0)
+
+#define ADF4377_001D_EN_CPTEST_OFF 0x0
+#define ADF4377_001D_EN_CPTEST_ON 0x1
+
+#define ADF4377_001D_CP_DOWN_OFF 0x0
+#define ADF4377_001D_CP_DOWN_ON 0x1
+
+#define ADF4377_001D_CP_UP_OFF 0x0
+#define ADF4377_001D_CP_UP_ON 0x1
+
+/* ADF4377 REG001F Map */
+#define ADF4377_001F_BST_REF_MSK BIT(7)
+#define ADF4377_001F_FILT_REF_MSK BIT(6)
+#define ADF4377_001F_REF_SEL_MSK BIT(5)
+#define ADF4377_001F_R01F_RSV1_MSK GENMASK(4, 0)
+
+/* ADF4377 REG001F Bit Definition */
+#define ADF4377_001F_BST_LARGE_REF_IN 0x0
+#define ADF4377_001F_BST_SMALL_REF_IN 0x1
+
+#define ADF4377_001F_FILT_REF_OFF 0x0
+#define ADF4377_001F_FILT_REF_ON 0x1
+
+#define ADF4377_001F_REF_SEL_DMA 0x0
+#define ADF4377_001F_REF_SEL_LNA 0x1
+
+#define ADF4377_001F_R01F_RSV1 0x7
+
+/* ADF4377 REG0020 Map */
+#define ADF4377_0020_RST_SYS_MSK BIT(4)
+#define ADF4377_0020_EN_ADC_CLK_MSK BIT(3)
+#define ADF4377_0020_R020_RSV1_MSK BIT(0)
+
+/* ADF4377 REG0021 Bit Definition */
+#define ADF4377_0021_R021_RSV1 0xD3
+
+/* ADF4377 REG0022 Bit Definition */
+#define ADF4377_0022_R022_RSV1 0x32
+
+/* ADF4377 REG0023 Map */
+#define ADF4377_0023_CAT_CT_SEL BIT(7)
+#define ADF4377_0023_R023_RSV1_MSK GENMASK(6, 0)
+
+/* ADF4377 REG0023 Bit Definition */
+#define ADF4377_0023_R023_RSV1 0x18
+
+/* ADF4377 REG0024 Map */
+#define ADF4377_0024_DCLK_MODE_MSK BIT(2)
+
+/* ADF4377 REG0025 Map */
+#define ADF4377_0025_CLKODIV_DB_MSK BIT(7)
+#define ADF4377_0025_DCLK_DB_MSK BIT(6)
+#define ADF4377_0025_R025_RSV1_MSK GENMASK(5, 0)
+
+/* ADF4377 REG0025 Bit Definition */
+#define ADF4377_0025_R025_RSV1 0x16
+
+/* ADF4377 REG0026 Map */
+#define ADF4377_0026_VCO_BAND_DIV_MSK GENMASK(7, 0)
+
+/* ADF4377 REG0027 Map */
+#define ADF4377_0027_SYNTH_LOCK_TO_LSB_MSK GENMASK(7, 0)
+
+/* ADF4377 REG0028 Map */
+#define ADF4377_0028_O_VCO_DB_MSK BIT(7)
+#define ADF4377_0028_SYNTH_LOCK_TO_MSB_MSK GENMASK(6, 0)
+
+/* ADF4377 REG0029 Map */
+#define ADF4377_0029_VCO_ALC_TO_LSB_MSK GENMASK(7, 0)
+
+/* ADF4377 REG002A Map */
+#define ADF4377_002A_DEL_CTRL_DB_MSK BIT(7)
+#define ADF4377_002A_VCO_ALC_TO_MSB_MSK GENMASK(6, 0)
+
+/* ADF4377 REG002C Map */
+#define ADF4377_002C_R02C_RSV1 0xC0
+
+/* ADF4377 REG002D Map */
+#define ADF4377_002D_ADC_CLK_DIV_MSK GENMASK(7, 0)
+
+/* ADF4377 REG002E Map */
+#define ADF4377_002E_EN_ADC_CNV_MSK BIT(7)
+#define ADF4377_002E_EN_ADC_MSK BIT(1)
+#define ADF4377_002E_ADC_A_CONV_MSK BIT(0)
+
+/* ADF4377 REG002E Bit Definition */
+#define ADF4377_002E_ADC_A_CONV_ADC_ST_CNV 0x0
+#define ADF4377_002E_ADC_A_CONV_VCO_CALIB 0x1
+
+/* ADF4377 REG002F Map */
+#define ADF4377_002F_DCLK_DIV1_MSK GENMASK(1, 0)
+
+/* ADF4377 REG002F Bit Definition */
+#define ADF4377_002F_DCLK_DIV1_1 0x0
+#define ADF4377_002F_DCLK_DIV1_2 0x1
+#define ADF4377_002F_DCLK_DIV1_8 0x2
+#define ADF4377_002F_DCLK_DIV1_32 0x3
+
+/* ADF4377 REG0031 Bit Definition */
+#define ADF4377_0031_R031_RSV1 0x09
+
+/* ADF4377 REG0032 Map */
+#define ADF4377_0032_ADC_CLK_SEL_MSK BIT(6)
+#define ADF4377_0032_R032_RSV1_MSK GENMASK(5, 0)
+
+/* ADF4377 REG0032 Bit Definition */
+#define ADF4377_0032_ADC_CLK_SEL_N_OP 0x0
+#define ADF4377_0032_ADC_CLK_SEL_SPI_CLK 0x1
+
+#define ADF4377_0032_R032_RSV1 0x9
+
+/* ADF4377 REG0033 Bit Definition */
+#define ADF4377_0033_R033_RSV1 0x18
+
+/* ADF4377 REG0034 Bit Definition */
+#define ADF4377_0034_R034_RSV1 0x08
+
+/* ADF4377 REG003A Bit Definition */
+#define ADF4377_003A_R03A_RSV1 0x5D
+
+/* ADF4377 REG003B Bit Definition */
+#define ADF4377_003B_R03B_RSV1 0x2B
+
+/* ADF4377 REG003D Map */
+#define ADF4377_003D_O_VCO_BAND_MSK BIT(3)
+#define ADF4377_003D_O_VCO_CORE_MSK BIT(2)
+#define ADF4377_003D_O_VCO_BIAS_MSK BIT(1)
+
+/* ADF4377 REG003D Bit Definition */
+#define ADF4377_003D_O_VCO_BAND_VCO_CALIB 0x0
+#define ADF4377_003D_O_VCO_BAND_M_VCO 0x1
+
+#define ADF4377_003D_O_VCO_CORE_VCO_CALIB 0x0
+#define ADF4377_003D_O_VCO_CORE_M_VCO 0x1
+
+#define ADF4377_003D_O_VCO_BIAS_VCO_CALIB 0x0
+#define ADF4377_003D_O_VCO_BIAS_M_VCO 0x1
+
+/* ADF4377 REG0042 Map */
+#define ADF4377_0042_R042_RSV1 0x05
+
+/* ADF4377 REG0045 Map */
+#define ADF4377_0045_ADC_ST_CNV_MSK BIT(0)
+
+/* ADF4377 REG0049 Map */
+#define ADF4377_0049_EN_CLK2_MSK BIT(7)
+#define ADF4377_0049_EN_CLK1_MSK BIT(6)
+#define ADF4377_0049_REF_OK_MSK BIT(3)
+#define ADF4377_0049_ADC_BUSY_MSK BIT(2)
+#define ADF4377_0049_FSM_BUSY_MSK BIT(1)
+#define ADF4377_0049_LOCKED_MSK BIT(0)
+
+/* ADF4377 REG004B Map */
+#define ADF4377_004B_VCO_CORE_MSK GENMASK(1, 0)
+
+/* ADF4377 REG004C Map */
+#define ADF4377_004C_CHIP_TEMP_LSB_MSK GENMASK(7, 0)
+
+/* ADF4377 REG004D Map */
+#define ADF4377_004D_CHIP_TEMP_MSB_MSK BIT(0)
+
+/* ADF4377 REG004F Map */
+#define ADF4377_004F_VCO_BAND_MSK GENMASK(7, 0)
+
+/* ADF4377 REG0051 Map */
+#define ADF4377_0051_VCO_BIAS_MSK GENMASK(3, 0)
+
+/* ADF4377 REG0054 Map */
+#define ADF4377_0054_CHIP_VERSION_MSK GENMASK(7, 0)
+
+/* Specifications */
+#define ADF4377_SPI_READ_CMD BIT(7)
+#define ADF4377_MAX_VCO_FREQ (12800ULL * HZ_PER_MHZ)
+#define ADF4377_MIN_VCO_FREQ (6400ULL * HZ_PER_MHZ)
+#define ADF4377_MAX_REFIN_FREQ (1000 * HZ_PER_MHZ)
+#define ADF4377_MIN_REFIN_FREQ (10 * HZ_PER_MHZ)
+#define ADF4377_MAX_FREQ_PFD (500 * HZ_PER_MHZ)
+#define ADF4377_MIN_FREQ_PFD (3 * HZ_PER_MHZ)
+#define ADF4377_MAX_CLKPN_FREQ ADF4377_MAX_VCO_FREQ
+#define ADF4377_MIN_CLKPN_FREQ (ADF4377_MIN_VCO_FREQ / 8)
+#define ADF4377_FREQ_PFD_80MHZ (80 * HZ_PER_MHZ)
+#define ADF4377_FREQ_PFD_125MHZ (125 * HZ_PER_MHZ)
+#define ADF4377_FREQ_PFD_160MHZ (160 * HZ_PER_MHZ)
+#define ADF4377_FREQ_PFD_250MHZ (250 * HZ_PER_MHZ)
+#define ADF4377_FREQ_PFD_320MHZ (320 * HZ_PER_MHZ)
+
+enum {
+ ADF4377_FREQ,
+};
+
+enum muxout_select_mode {
+ ADF4377_MUXOUT_HIGH_Z = 0x0,
+ ADF4377_MUXOUT_LKDET = 0x1,
+ ADF4377_MUXOUT_LOW = 0x2,
+ ADF4377_MUXOUT_DIV_RCLK_2 = 0x4,
+ ADF4377_MUXOUT_DIV_NCLK_2 = 0x5,
+ ADF4377_MUXOUT_HIGH = 0x8,
+};
+
+struct adf4377_state {
+ struct spi_device *spi;
+ struct regmap *regmap;
+ struct clk *clkin;
+ /* Protect against concurrent accesses to the device and data content */
+ struct mutex lock;
+ struct notifier_block nb;
+ /* Reference Divider */
+ unsigned int ref_div_factor;
+ /* PFD Frequency */
+ unsigned int f_pfd;
+ /* Input Reference Clock */
+ unsigned int clkin_freq;
+ /* CLKOUT Divider */
+ u8 clkout_div_sel;
+ /* Feedback Divider (N) */
+ u16 n_int;
+ u16 synth_lock_timeout;
+ u16 vco_alc_timeout;
+ u16 adc_clk_div;
+ u16 vco_band_div;
+ u8 dclk_div1;
+ u8 dclk_div2;
+ u8 dclk_mode;
+ unsigned int f_div_rclk;
+ enum muxout_select_mode muxout_select;
+ struct gpio_desc *gpio_ce;
+ struct gpio_desc *gpio_enclk1;
+ struct gpio_desc *gpio_enclk2;
+ u8 buf[2] __aligned(IIO_DMA_MINALIGN);
+};
+
+static const char * const adf4377_muxout_modes[] = {
+ [ADF4377_MUXOUT_HIGH_Z] = "high_z",
+ [ADF4377_MUXOUT_LKDET] = "lock_detect",
+ [ADF4377_MUXOUT_LOW] = "muxout_low",
+ [ADF4377_MUXOUT_DIV_RCLK_2] = "f_div_rclk_2",
+ [ADF4377_MUXOUT_DIV_NCLK_2] = "f_div_nclk_2",
+ [ADF4377_MUXOUT_HIGH] = "muxout_high",
+};
+
+static const struct reg_sequence adf4377_reg_defaults[] = {
+ { 0x42, ADF4377_0042_R042_RSV1 },
+ { 0x3B, ADF4377_003B_R03B_RSV1 },
+ { 0x3A, ADF4377_003A_R03A_RSV1 },
+ { 0x34, ADF4377_0034_R034_RSV1 },
+ { 0x33, ADF4377_0033_R033_RSV1 },
+ { 0x32, ADF4377_0032_R032_RSV1 },
+ { 0x31, ADF4377_0031_R031_RSV1 },
+ { 0x2C, ADF4377_002C_R02C_RSV1 },
+ { 0x25, ADF4377_0025_R025_RSV1 },
+ { 0x23, ADF4377_0023_R023_RSV1 },
+ { 0x22, ADF4377_0022_R022_RSV1 },
+ { 0x21, ADF4377_0021_R021_RSV1 },
+ { 0x1f, ADF4377_001F_R01F_RSV1 },
+ { 0x1c, ADF4377_001C_R01C_RSV1 },
+};
+
+static const struct regmap_config adf4377_regmap_config = {
+ .reg_bits = 16,
+ .val_bits = 8,
+ .read_flag_mask = BIT(7),
+ .max_register = 0x54,
+};
+
+static int adf4377_reg_access(struct iio_dev *indio_dev,
+ unsigned int reg,
+ unsigned int write_val,
+ unsigned int *read_val)
+{
+ struct adf4377_state *st = iio_priv(indio_dev);
+
+ if (read_val)
+ return regmap_read(st->regmap, reg, read_val);
+
+ return regmap_write(st->regmap, reg, write_val);
+}
+
+static const struct iio_info adf4377_info = {
+ .debugfs_reg_access = &adf4377_reg_access,
+};
+
+static int adf4377_soft_reset(struct adf4377_state *st)
+{
+ unsigned int read_val;
+ int ret;
+
+ ret = regmap_update_bits(st->regmap, 0x0, ADF4377_0000_SOFT_RESET_MSK |
+ ADF4377_0000_SOFT_RESET_R_MSK,
+ FIELD_PREP(ADF4377_0000_SOFT_RESET_MSK, 1) |
+ FIELD_PREP(ADF4377_0000_SOFT_RESET_R_MSK, 1));
+ if (ret)
+ return ret;
+
+ return regmap_read_poll_timeout(st->regmap, 0x0, read_val,
+ !(read_val & (ADF4377_0000_SOFT_RESET_R_MSK |
+ ADF4377_0000_SOFT_RESET_R_MSK)), 200, 200 * 100);
+}
+
+static int adf4377_get_freq(struct adf4377_state *st, u64 *freq)
+{
+ unsigned int ref_div_factor, n_int;
+ u64 clkin_freq;
+ int ret;
+
+ mutex_lock(&st->lock);
+ ret = regmap_read(st->regmap, 0x12, &ref_div_factor);
+ if (ret)
+ goto exit;
+
+ ret = regmap_bulk_read(st->regmap, 0x10, st->buf, sizeof(st->buf));
+ if (ret)
+ goto exit;
+
+ clkin_freq = clk_get_rate(st->clkin);
+ ref_div_factor = FIELD_GET(ADF4377_0012_R_DIV_MSK, ref_div_factor);
+ n_int = FIELD_GET(ADF4377_0010_N_INT_LSB_MSK | ADF4377_0011_N_INT_MSB_MSK,
+ get_unaligned_le16(&st->buf));
+
+ *freq = div_u64(clkin_freq, ref_div_factor) * n_int;
+exit:
+ mutex_unlock(&st->lock);
+
+ return ret;
+}
+
+static int adf4377_set_freq(struct adf4377_state *st, u64 freq)
+{
+ unsigned int read_val;
+ u64 f_vco;
+ int ret;
+
+ mutex_lock(&st->lock);
+
+ if (freq > ADF4377_MAX_CLKPN_FREQ || freq < ADF4377_MIN_CLKPN_FREQ) {
+ ret = -EINVAL;
+ goto exit;
+ }
+
+ ret = regmap_update_bits(st->regmap, 0x1C, ADF4377_001C_EN_DNCLK_MSK |
+ ADF4377_001C_EN_DRCLK_MSK,
+ FIELD_PREP(ADF4377_001C_EN_DNCLK_MSK, 1) |
+ FIELD_PREP(ADF4377_001C_EN_DRCLK_MSK, 1));
+ if (ret)
+ goto exit;
+
+ ret = regmap_update_bits(st->regmap, 0x11, ADF4377_0011_EN_AUTOCAL_MSK |
+ ADF4377_0011_DCLK_DIV2_MSK,
+ FIELD_PREP(ADF4377_0011_EN_AUTOCAL_MSK, 1) |
+ FIELD_PREP(ADF4377_0011_DCLK_DIV2_MSK, st->dclk_div2));
+ if (ret)
+ goto exit;
+
+ ret = regmap_update_bits(st->regmap, 0x2E, ADF4377_002E_EN_ADC_CNV_MSK |
+ ADF4377_002E_EN_ADC_MSK |
+ ADF4377_002E_ADC_A_CONV_MSK,
+ FIELD_PREP(ADF4377_002E_EN_ADC_CNV_MSK, 1) |
+ FIELD_PREP(ADF4377_002E_EN_ADC_MSK, 1) |
+ FIELD_PREP(ADF4377_002E_ADC_A_CONV_MSK,
+ ADF4377_002E_ADC_A_CONV_VCO_CALIB));
+ if (ret)
+ goto exit;
+
+ ret = regmap_update_bits(st->regmap, 0x20, ADF4377_0020_EN_ADC_CLK_MSK,
+ FIELD_PREP(ADF4377_0020_EN_ADC_CLK_MSK, 1));
+ if (ret)
+ goto exit;
+
+ ret = regmap_update_bits(st->regmap, 0x2F, ADF4377_002F_DCLK_DIV1_MSK,
+ FIELD_PREP(ADF4377_002F_DCLK_DIV1_MSK, st->dclk_div1));
+ if (ret)
+ goto exit;
+
+ ret = regmap_update_bits(st->regmap, 0x24, ADF4377_0024_DCLK_MODE_MSK,
+ FIELD_PREP(ADF4377_0024_DCLK_MODE_MSK, st->dclk_mode));
+ if (ret)
+ goto exit;
+
+ ret = regmap_write(st->regmap, 0x27,
+ FIELD_PREP(ADF4377_0027_SYNTH_LOCK_TO_LSB_MSK,
+ st->synth_lock_timeout));
+ if (ret)
+ goto exit;
+
+ ret = regmap_update_bits(st->regmap, 0x28, ADF4377_0028_SYNTH_LOCK_TO_MSB_MSK,
+ FIELD_PREP(ADF4377_0028_SYNTH_LOCK_TO_MSB_MSK,
+ st->synth_lock_timeout >> 8));
+ if (ret)
+ goto exit;
+
+ ret = regmap_write(st->regmap, 0x29,
+ FIELD_PREP(ADF4377_0029_VCO_ALC_TO_LSB_MSK,
+ st->vco_alc_timeout));
+ if (ret)
+ goto exit;
+
+ ret = regmap_update_bits(st->regmap, 0x2A, ADF4377_002A_VCO_ALC_TO_MSB_MSK,
+ FIELD_PREP(ADF4377_002A_VCO_ALC_TO_MSB_MSK,
+ st->vco_alc_timeout >> 8));
+ if (ret)
+ goto exit;
+
+ ret = regmap_write(st->regmap, 0x26,
+ FIELD_PREP(ADF4377_0026_VCO_BAND_DIV_MSK, st->vco_band_div));
+ if (ret)
+ goto exit;
+
+ ret = regmap_write(st->regmap, 0x2D,
+ FIELD_PREP(ADF4377_002D_ADC_CLK_DIV_MSK, st->adc_clk_div));
+ if (ret)
+ goto exit;
+
+ st->clkout_div_sel = 0;
+
+ f_vco = freq;
+
+ while (f_vco < ADF4377_MIN_VCO_FREQ) {
+ f_vco <<= 1;
+ st->clkout_div_sel++;
+ }
+
+ st->n_int = div_u64(freq, st->f_pfd);
+
+ ret = regmap_update_bits(st->regmap, 0x11, ADF4377_0011_EN_RDBLR_MSK |
+ ADF4377_0011_N_INT_MSB_MSK,
+ FIELD_PREP(ADF4377_0011_EN_RDBLR_MSK, 0) |
+ FIELD_PREP(ADF4377_0011_N_INT_MSB_MSK, st->n_int >> 8));
+ if (ret)
+ goto exit;
+
+ ret = regmap_update_bits(st->regmap, 0x12, ADF4377_0012_R_DIV_MSK |
+ ADF4377_0012_CLKOUT_DIV_MSK,
+ FIELD_PREP(ADF4377_0012_CLKOUT_DIV_MSK, st->clkout_div_sel) |
+ FIELD_PREP(ADF4377_0012_R_DIV_MSK, st->ref_div_factor));
+ if (ret)
+ goto exit;
+
+ ret = regmap_write(st->regmap, 0x10,
+ FIELD_PREP(ADF4377_0010_N_INT_LSB_MSK, st->n_int));
+ if (ret)
+ goto exit;
+
+ ret = regmap_read_poll_timeout(st->regmap, 0x49, read_val,
+ !(read_val & (ADF4377_0049_FSM_BUSY_MSK)), 200, 200 * 100);
+ if (ret)
+ goto exit;
+
+ /* Disable EN_DNCLK, EN_DRCLK */
+ ret = regmap_update_bits(st->regmap, 0x1C, ADF4377_001C_EN_DNCLK_MSK |
+ ADF4377_001C_EN_DRCLK_MSK,
+ FIELD_PREP(ADF4377_001C_EN_DNCLK_MSK, 0) |
+ FIELD_PREP(ADF4377_001C_EN_DRCLK_MSK, 0));
+ if (ret)
+ goto exit;
+
+ /* Disable EN_ADC_CLK */
+ ret = regmap_update_bits(st->regmap, 0x20, ADF4377_0020_EN_ADC_CLK_MSK,
+ FIELD_PREP(ADF4377_0020_EN_ADC_CLK_MSK, 0));
+ if (ret)
+ goto exit;
+
+ /* Set output Amplitude */
+ ret = regmap_update_bits(st->regmap, 0x19, ADF4377_0019_CLKOUT2_OP_MSK |
+ ADF4377_0019_CLKOUT1_OP_MSK,
+ FIELD_PREP(ADF4377_0019_CLKOUT1_OP_MSK,
+ ADF4377_0019_CLKOUT_420MV) |
+ FIELD_PREP(ADF4377_0019_CLKOUT2_OP_MSK,
+ ADF4377_0019_CLKOUT_420MV));
+
+exit:
+ mutex_unlock(&st->lock);
+
+ return ret;
+}
+
+static void adf4377_gpio_init(struct adf4377_state *st)
+{
+ if (st->gpio_ce) {
+ gpiod_set_value(st->gpio_ce, 1);
+
+ /* Delay for SPI register bits to settle to their power-on reset state */
+ fsleep(200);
+ }
+
+ if (st->gpio_enclk1)
+ gpiod_set_value(st->gpio_enclk1, 1);
+
+ if (st->gpio_enclk2)
+ gpiod_set_value(st->gpio_enclk2, 1);
+}
+
+static int adf4377_init(struct adf4377_state *st)
+{
+ struct spi_device *spi = st->spi;
+ int ret;
+
+ adf4377_gpio_init(st);
+
+ ret = adf4377_soft_reset(st);
+ if (ret) {
+ dev_err(&spi->dev, "Failed to soft reset.\n");
+ return ret;
+ }
+
+ ret = regmap_multi_reg_write(st->regmap, adf4377_reg_defaults,
+ ARRAY_SIZE(adf4377_reg_defaults));
+ if (ret) {
+ dev_err(&spi->dev, "Failed to set default registers.\n");
+ return ret;
+ }
+
+ ret = regmap_update_bits(st->regmap, 0x00,
+ ADF4377_0000_SDO_ACTIVE_MSK | ADF4377_0000_SDO_ACTIVE_R_MSK,
+ FIELD_PREP(ADF4377_0000_SDO_ACTIVE_MSK,
+ ADF4377_0000_SDO_ACTIVE_SPI_4W) |
+ FIELD_PREP(ADF4377_0000_SDO_ACTIVE_R_MSK,
+ ADF4377_0000_SDO_ACTIVE_SPI_4W));
+ if (ret) {
+ dev_err(&spi->dev, "Failed to set 4-Wire Operation.\n");
+ return ret;
+ }
+
+ st->clkin_freq = clk_get_rate(st->clkin);
+
+ /* Power Up */
+ ret = regmap_write(st->regmap, 0x1a,
+ FIELD_PREP(ADF4377_001A_PD_ALL_MSK, 0) |
+ FIELD_PREP(ADF4377_001A_PD_RDIV_MSK, 0) |
+ FIELD_PREP(ADF4377_001A_PD_NDIV_MSK, 0) |
+ FIELD_PREP(ADF4377_001A_PD_VCO_MSK, 0) |
+ FIELD_PREP(ADF4377_001A_PD_LD_MSK, 0) |
+ FIELD_PREP(ADF4377_001A_PD_PFDCP_MSK, 0) |
+ FIELD_PREP(ADF4377_001A_PD_CLKOUT1_MSK, 0) |
+ FIELD_PREP(ADF4377_001A_PD_CLKOUT2_MSK, 0));
+ if (ret) {
+ dev_err(&spi->dev, "Failed to set power down registers.\n");
+ return ret;
+ }
+
+ /* Set Mux Output */
+ ret = regmap_update_bits(st->regmap, 0x1D,
+ ADF4377_001D_MUXOUT_MSK,
+ FIELD_PREP(ADF4377_001D_MUXOUT_MSK, st->muxout_select));
+ if (ret)
+ return ret;
+
+ /* Compute PFD */
+ st->ref_div_factor = 0;
+ do {
+ st->ref_div_factor++;
+ st->f_pfd = st->clkin_freq / st->ref_div_factor;
+ } while (st->f_pfd > ADF4377_MAX_FREQ_PFD);
+
+ if (st->f_pfd > ADF4377_MAX_FREQ_PFD || st->f_pfd < ADF4377_MIN_FREQ_PFD)
+ return -EINVAL;
+
+ st->f_div_rclk = st->f_pfd;
+
+ if (st->f_pfd <= ADF4377_FREQ_PFD_80MHZ) {
+ st->dclk_div1 = ADF4377_002F_DCLK_DIV1_1;
+ st->dclk_div2 = ADF4377_0011_DCLK_DIV2_1;
+ st->dclk_mode = 0;
+ } else if (st->f_pfd <= ADF4377_FREQ_PFD_125MHZ) {
+ st->dclk_div1 = ADF4377_002F_DCLK_DIV1_1;
+ st->dclk_div2 = ADF4377_0011_DCLK_DIV2_1;
+ st->dclk_mode = 1;
+ } else if (st->f_pfd <= ADF4377_FREQ_PFD_160MHZ) {
+ st->dclk_div1 = ADF4377_002F_DCLK_DIV1_2;
+ st->dclk_div2 = ADF4377_0011_DCLK_DIV2_1;
+ st->dclk_mode = 0;
+ st->f_div_rclk /= 2;
+ } else if (st->f_pfd <= ADF4377_FREQ_PFD_250MHZ) {
+ st->dclk_div1 = ADF4377_002F_DCLK_DIV1_2;
+ st->dclk_div2 = ADF4377_0011_DCLK_DIV2_1;
+ st->dclk_mode = 1;
+ st->f_div_rclk /= 2;
+ } else if (st->f_pfd <= ADF4377_FREQ_PFD_320MHZ) {
+ st->dclk_div1 = ADF4377_002F_DCLK_DIV1_2;
+ st->dclk_div2 = ADF4377_0011_DCLK_DIV2_2;
+ st->dclk_mode = 0;
+ st->f_div_rclk /= 4;
+ } else {
+ st->dclk_div1 = ADF4377_002F_DCLK_DIV1_2;
+ st->dclk_div2 = ADF4377_0011_DCLK_DIV2_2;
+ st->dclk_mode = 1;
+ st->f_div_rclk /= 4;
+ }
+
+ st->synth_lock_timeout = DIV_ROUND_UP(st->f_div_rclk, 50000);
+ st->vco_alc_timeout = DIV_ROUND_UP(st->f_div_rclk, 20000);
+ st->vco_band_div = DIV_ROUND_UP(st->f_div_rclk, 150000 * 16 * (1 << st->dclk_mode));
+ st->adc_clk_div = DIV_ROUND_UP((st->f_div_rclk / 400000 - 2), 4);
+
+ return 0;
+}
+
+static ssize_t adf4377_read(struct iio_dev *indio_dev, uintptr_t private,
+ const struct iio_chan_spec *chan, char *buf)
+{
+ struct adf4377_state *st = iio_priv(indio_dev);
+ u64 val = 0;
+ int ret;
+
+ switch ((u32)private) {
+ case ADF4377_FREQ:
+ ret = adf4377_get_freq(st, &val);
+ if (ret)
+ return ret;
+
+ return sysfs_emit(buf, "%llu\n", val);
+ default:
+ return -EINVAL;
+ }
+}
+
+static ssize_t adf4377_write(struct iio_dev *indio_dev, uintptr_t private,
+ const struct iio_chan_spec *chan, const char *buf,
+ size_t len)
+{
+ struct adf4377_state *st = iio_priv(indio_dev);
+ unsigned long long freq;
+ int ret;
+
+ switch ((u32)private) {
+ case ADF4377_FREQ:
+ ret = kstrtoull(buf, 10, &freq);
+ if (ret)
+ return ret;
+
+ ret = adf4377_set_freq(st, freq);
+ if (ret)
+ return ret;
+
+ return len;
+ default:
+ return -EINVAL;
+ }
+}
+
+#define _ADF4377_EXT_INFO(_name, _shared, _ident) { \
+ .name = _name, \
+ .read = adf4377_read, \
+ .write = adf4377_write, \
+ .private = _ident, \
+ .shared = _shared, \
+ }
+
+static const struct iio_chan_spec_ext_info adf4377_ext_info[] = {
+ /*
+ * Usually we use IIO_CHAN_INFO_FREQUENCY, but there are
+ * values > 2^32 in order to support the entire frequency range
+ * in Hz.
+ */
+ _ADF4377_EXT_INFO("frequency", IIO_SEPARATE, ADF4377_FREQ),
+ { }
+};
+
+static const struct iio_chan_spec adf4377_channels[] = {
+ {
+ .type = IIO_ALTVOLTAGE,
+ .indexed = 1,
+ .output = 1,
+ .channel = 0,
+ .ext_info = adf4377_ext_info,
+ },
+};
+
+static int adf4377_properties_parse(struct adf4377_state *st)
+{
+ struct spi_device *spi = st->spi;
+ const char *str;
+ int ret;
+
+ st->clkin = devm_clk_get_enabled(&spi->dev, "ref_in");
+ if (IS_ERR(st->clkin))
+ return dev_err_probe(&spi->dev, PTR_ERR(st->clkin),
+ "failed to get the reference input clock\n");
+
+ st->gpio_ce = devm_gpiod_get_optional(&st->spi->dev, "chip-enable",
+ GPIOD_OUT_LOW);
+ if (IS_ERR(st->gpio_ce))
+ return dev_err_probe(&spi->dev, PTR_ERR(st->gpio_ce),
+ "failed to get the CE GPIO\n");
+
+ st->gpio_enclk1 = devm_gpiod_get_optional(&st->spi->dev, "clk1-enable",
+ GPIOD_OUT_LOW);
+ if (IS_ERR(st->gpio_enclk1))
+ return dev_err_probe(&spi->dev, PTR_ERR(st->gpio_enclk1),
+ "failed to get the CE GPIO\n");
+
+ st->gpio_enclk2 = devm_gpiod_get_optional(&st->spi->dev, "clk2-enable",
+ GPIOD_OUT_LOW);
+ if (IS_ERR(st->gpio_enclk2))
+ return dev_err_probe(&spi->dev, PTR_ERR(st->gpio_enclk2),
+ "failed to get the CE GPIO\n");
+
+ ret = device_property_read_string(&spi->dev, "adi,muxout-select", &str);
+ if (ret) {
+ st->muxout_select = ADF4377_MUXOUT_HIGH_Z;
+ } else {
+ ret = match_string(adf4377_muxout_modes, ARRAY_SIZE(adf4377_muxout_modes), str);
+ if (ret < 0)
+ return ret;
+
+ st->muxout_select = ret;
+ }
+
+ return 0;
+}
+
+static int adf4377_freq_change(struct notifier_block *nb, unsigned long action, void *data)
+{
+ struct adf4377_state *st = container_of(nb, struct adf4377_state, nb);
+ int ret;
+
+ if (action == POST_RATE_CHANGE) {
+ mutex_lock(&st->lock);
+ ret = notifier_from_errno(adf4377_init(st));
+ mutex_unlock(&st->lock);
+ return ret;
+ }
+
+ return NOTIFY_OK;
+}
+
+static int adf4377_probe(struct spi_device *spi)
+{
+ struct iio_dev *indio_dev;
+ struct regmap *regmap;
+ struct adf4377_state *st;
+ int ret;
+
+ indio_dev = devm_iio_device_alloc(&spi->dev, sizeof(*st));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ regmap = devm_regmap_init_spi(spi, &adf4377_regmap_config);
+ if (IS_ERR(regmap))
+ return PTR_ERR(regmap);
+
+ st = iio_priv(indio_dev);
+
+ indio_dev->info = &adf4377_info;
+ indio_dev->name = "adf4377";
+ indio_dev->channels = adf4377_channels;
+ indio_dev->num_channels = ARRAY_SIZE(adf4377_channels);
+
+ st->regmap = regmap;
+ st->spi = spi;
+ mutex_init(&st->lock);
+
+ ret = adf4377_properties_parse(st);
+ if (ret)
+ return ret;
+
+ st->nb.notifier_call = adf4377_freq_change;
+ ret = devm_clk_notifier_register(&spi->dev, st->clkin, &st->nb);
+ if (ret)
+ return ret;
+
+ ret = adf4377_init(st);
+ if (ret)
+ return ret;
+
+ return devm_iio_device_register(&spi->dev, indio_dev);
+}
+
+static const struct spi_device_id adf4377_id[] = {
+ { "adf4377", 0 },
+ {}
+};
+MODULE_DEVICE_TABLE(spi, adf4377_id);
+
+static const struct of_device_id adf4377_of_match[] = {
+ { .compatible = "adi,adf4377" },
+ {}
+};
+MODULE_DEVICE_TABLE(of, adf4377_of_match);
+
+static struct spi_driver adf4377_driver = {
+ .driver = {
+ .name = "adf4377",
+ .of_match_table = adf4377_of_match,
+ },
+ .probe = adf4377_probe,
+ .id_table = adf4377_id,
+};
+module_spi_driver(adf4377_driver);
+
+MODULE_AUTHOR("Antoniu Miclaus <antoniu.miclaus@analog.com>");
+MODULE_DESCRIPTION("Analog Devices ADF4377");
+MODULE_LICENSE("GPL");
uint16_t prod_id;
int ret;
- ret = adis_initial_startup(&adis16136->adis);
+ ret = __adis_initial_startup(&adis16136->adis);
if (ret)
return ret;
return ret;
/* Get the device into a sane initial state */
- ret = adis_initial_startup(&adis16260->adis);
+ ret = __adis_initial_startup(&adis16260->adis);
if (ret)
return ret;
struct bmg160_data {
struct regmap *regmap;
- struct regulator_bulk_data regulators[2];
struct iio_trigger *dready_trig;
struct iio_trigger *motion_trig;
struct iio_mount_matrix orientation;
return dev_name(dev);
}
-static void bmg160_disable_regulators(void *d)
-{
- struct bmg160_data *data = d;
-
- regulator_bulk_disable(ARRAY_SIZE(data->regulators), data->regulators);
-}
-
int bmg160_core_probe(struct device *dev, struct regmap *regmap, int irq,
const char *name)
{
+ static const char * const regulators[] = { "vdd", "vddio" };
struct bmg160_data *data;
struct iio_dev *indio_dev;
int ret;
data->irq = irq;
data->regmap = regmap;
- data->regulators[0].supply = "vdd";
- data->regulators[1].supply = "vddio";
- ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(data->regulators),
- data->regulators);
+ ret = devm_regulator_bulk_get_enable(dev, ARRAY_SIZE(regulators),
+ regulators);
if (ret)
return dev_err_probe(dev, ret, "Failed to get regulators\n");
- ret = regulator_bulk_enable(ARRAY_SIZE(data->regulators),
- data->regulators);
- if (ret)
- return ret;
-
- ret = devm_add_action_or_reset(dev, bmg160_disable_regulators, data);
- if (ret)
- return ret;
-
ret = iio_read_mount_matrix(dev, &data->orientation);
if (ret)
return ret;
.max_register = 0x3f
};
-static int bmg160_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bmg160_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct regmap *regmap;
const char *name = NULL;
.of_match_table = bmg160_of_match,
.pm = &bmg160_pm_ops,
},
- .probe = bmg160_i2c_probe,
+ .probe_new = bmg160_i2c_probe,
.remove = bmg160_i2c_remove,
.id_table = bmg160_i2c_id,
};
return ret;
}
-EXPORT_SYMBOL_GPL(fxas21002c_core_probe);
+EXPORT_SYMBOL_NS_GPL(fxas21002c_core_probe, IIO_FXAS21002C);
void fxas21002c_core_remove(struct device *dev)
{
pm_runtime_disable(dev);
pm_runtime_set_suspended(dev);
}
-EXPORT_SYMBOL_GPL(fxas21002c_core_remove);
+EXPORT_SYMBOL_NS_GPL(fxas21002c_core_remove, IIO_FXAS21002C);
-static int __maybe_unused fxas21002c_suspend(struct device *dev)
+static int fxas21002c_suspend(struct device *dev)
{
struct fxas21002c_data *data = iio_priv(dev_get_drvdata(dev));
return 0;
}
-static int __maybe_unused fxas21002c_resume(struct device *dev)
+static int fxas21002c_resume(struct device *dev)
{
struct fxas21002c_data *data = iio_priv(dev_get_drvdata(dev));
int ret;
return fxas21002c_mode_set(data, data->prev_mode);
}
-static int __maybe_unused fxas21002c_runtime_suspend(struct device *dev)
+static int fxas21002c_runtime_suspend(struct device *dev)
{
struct fxas21002c_data *data = iio_priv(dev_get_drvdata(dev));
return fxas21002c_mode_set(data, FXAS21002C_MODE_READY);
}
-static int __maybe_unused fxas21002c_runtime_resume(struct device *dev)
+static int fxas21002c_runtime_resume(struct device *dev)
{
struct fxas21002c_data *data = iio_priv(dev_get_drvdata(dev));
return fxas21002c_mode_set(data, FXAS21002C_MODE_ACTIVE);
}
-const struct dev_pm_ops fxas21002c_pm_ops = {
- SET_SYSTEM_SLEEP_PM_OPS(fxas21002c_suspend, fxas21002c_resume)
- SET_RUNTIME_PM_OPS(fxas21002c_runtime_suspend,
- fxas21002c_runtime_resume, NULL)
+EXPORT_NS_GPL_DEV_PM_OPS(fxas21002c_pm_ops, IIO_FXAS21002C) = {
+ SYSTEM_SLEEP_PM_OPS(fxas21002c_suspend, fxas21002c_resume)
+ RUNTIME_PM_OPS(fxas21002c_runtime_suspend, fxas21002c_runtime_resume,
+ NULL)
};
-EXPORT_SYMBOL_GPL(fxas21002c_pm_ops);
MODULE_AUTHOR("Rui Miguel Silva <rui.silva@linaro.org>");
MODULE_LICENSE("GPL v2");
static struct i2c_driver fxas21002c_i2c_driver = {
.driver = {
.name = "fxas21002c_i2c",
- .pm = &fxas21002c_pm_ops,
+ .pm = pm_ptr(&fxas21002c_pm_ops),
.of_match_table = fxas21002c_i2c_of_match,
},
.probe_new = fxas21002c_i2c_probe,
MODULE_AUTHOR("Rui Miguel Silva <rui.silva@linaro.org>");
MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("FXAS21002C I2C Gyro driver");
+MODULE_IMPORT_NS(IIO_FXAS21002C);
static struct spi_driver fxas21002c_spi_driver = {
.driver = {
.name = "fxas21002c_spi",
- .pm = &fxas21002c_pm_ops,
+ .pm = pm_ptr(&fxas21002c_pm_ops),
.of_match_table = fxas21002c_spi_of_match,
},
.probe = fxas21002c_spi_probe,
MODULE_AUTHOR("Rui Miguel Silva <rui.silva@linaro.org>");
MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("FXAS21002C SPI Gyro driver");
+MODULE_IMPORT_NS(IIO_FXAS21002C);
#include <linux/slab.h>
#include <linux/stat.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/delay.h>
#include <linux/iio/iio.h>
int val2,
long mask)
{
+ struct itg3200 *st = iio_priv(indio_dev);
int ret;
u8 t;
if (val == 0 || val2 != 0)
return -EINVAL;
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&st->lock);
ret = itg3200_read_reg_8(indio_dev, ITG3200_REG_DLPF, &t);
if (ret) {
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&st->lock);
return ret;
}
t = ((t & ITG3200_DLPF_CFG_MASK) ? 1000u : 8000u) / val - 1;
ITG3200_REG_SAMPLE_RATE_DIV,
t);
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&st->lock);
return ret;
default:
static const unsigned long itg3200_available_scan_masks[] = { 0xffffffff, 0x0 };
-static int itg3200_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int itg3200_probe(struct i2c_client *client)
{
int ret;
struct itg3200 *st;
if (ret)
goto error_remove_trigger;
+ mutex_init(&st->lock);
+
ret = iio_device_register(indio_dev);
if (ret)
goto error_remove_trigger;
.pm = pm_sleep_ptr(&itg3200_pm_ops),
},
.id_table = itg3200_id,
- .probe = itg3200_probe,
+ .probe_new = itg3200_probe,
.remove = itg3200_remove,
};
return 0;
}
-static int mpu3050_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mpu3050_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct regmap *regmap;
const char *name;
struct mpu3050 *mpu3050;
MODULE_DEVICE_TABLE(of, mpu3050_i2c_of_match);
static struct i2c_driver mpu3050_i2c_driver = {
- .probe = mpu3050_i2c_probe,
+ .probe_new = mpu3050_i2c_probe,
.remove = mpu3050_i2c_remove,
.id_table = mpu3050_i2c_id,
.driver = {
};
MODULE_DEVICE_TABLE(of, st_gyro_of_match);
-static int st_gyro_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int st_gyro_i2c_probe(struct i2c_client *client)
{
const struct st_sensor_settings *settings;
struct st_sensor_data *gdata;
.name = "st-gyro-i2c",
.of_match_table = st_gyro_of_match,
},
- .probe = st_gyro_i2c_probe,
+ .probe_new = st_gyro_i2c_probe,
.id_table = st_gyro_id_table,
};
module_i2c_driver(st_gyro_driver);
static DEFINE_SIMPLE_DEV_PM_OPS(afe4404_pm_ops, afe4404_suspend,
afe4404_resume);
-static int afe4404_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int afe4404_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct afe4404_data *afe;
.of_match_table = afe4404_of_match,
.pm = pm_sleep_ptr(&afe4404_pm_ops),
},
- .probe = afe4404_probe,
+ .probe_new = afe4404_probe,
.remove = afe4404_remove,
.id_table = afe4404_ids,
};
* Temperature reading can only be acquired while engine
* is running
*/
- mutex_lock(&indio_dev->mlock);
-
- if (!iio_buffer_enabled(indio_dev))
+ if (iio_device_claim_buffer_mode(indio_dev)) {
+ /*
+ * Replacing -EBUSY or other error code
+ * returned by iio_device_claim_buffer_mode()
+ * because user space may rely on the current
+ * one.
+ */
ret = -EAGAIN;
- else {
+ } else {
ret = max30100_get_temp(data, val);
if (!ret)
ret = IIO_VAL_INT;
+ iio_device_release_buffer_mode(indio_dev);
}
-
- mutex_unlock(&indio_dev->mlock);
break;
case IIO_CHAN_INFO_SCALE:
*val = 1; /* 0.0625 */
.read_raw = max30100_read_raw,
};
-static int max30100_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max30100_probe(struct i2c_client *client)
{
struct max30100_data *data;
struct iio_dev *indio_dev;
.name = MAX30100_DRV_NAME,
.of_match_table = max30100_dt_ids,
},
- .probe = max30100_probe,
+ .probe_new = max30100_probe,
.remove = max30100_remove,
.id_table = max30100_id,
};
* Temperature reading can only be acquired when not in
* shutdown; leave shutdown briefly when buffer not running
*/
- mutex_lock(&indio_dev->mlock);
- if (!iio_buffer_enabled(indio_dev))
+any_mode_retry:
+ if (iio_device_claim_buffer_mode(indio_dev)) {
+ /*
+ * This one is a *bit* hacky. If we cannot claim buffer
+ * mode, then try direct mode so that we make sure
+ * things cannot concurrently change. And we just keep
+ * trying until we get one of the modes...
+ */
+ if (iio_device_claim_direct_mode(indio_dev))
+ goto any_mode_retry;
+
ret = max30102_get_temp(data, val, true);
- else
+ iio_device_release_direct_mode(indio_dev);
+ } else {
ret = max30102_get_temp(data, val, false);
- mutex_unlock(&indio_dev->mlock);
+ iio_device_release_buffer_mode(indio_dev);
+ }
if (ret)
return ret;
.read_raw = max30102_read_raw,
};
-static int max30102_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max30102_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct max30102_data *data;
struct iio_dev *indio_dev;
int ret;
.name = MAX30102_DRV_NAME,
.of_match_table = max30102_dt_ids,
},
- .probe = max30102_probe,
+ .probe_new = max30102_probe,
.remove = max30102_remove,
.id_table = max30102_id,
};
.read_raw = am2315_read_raw,
};
-static int am2315_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int am2315_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
.driver = {
.name = "am2315",
},
- .probe = am2315_probe,
+ .probe_new = am2315_probe,
.id_table = am2315_i2c_id,
};
.attrs = &hdc100x_attribute_group,
};
-static int hdc100x_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int hdc100x_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct hdc100x_data *data;
.of_match_table = hdc100x_dt_ids,
.acpi_match_table = hdc100x_acpi_match,
},
- .probe = hdc100x_probe,
+ .probe_new = hdc100x_probe,
.id_table = hdc100x_id,
};
module_i2c_driver(hdc100x_driver);
.attrs = &hdc2010_attribute_group,
};
-static int hdc2010_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int hdc2010_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct hdc2010_data *data;
.name = "hdc2010",
.of_match_table = hdc2010_dt_ids,
},
- .probe = hdc2010_probe,
+ .probe_new = hdc2010_probe,
.remove = hdc2010_remove,
.id_table = hdc2010_id,
};
#define HTS221_DEV_NAME "hts221"
#include <linux/iio/iio.h>
-#include <linux/regulator/consumer.h>
enum hts221_sensor_type {
HTS221_SENSOR_H,
const char *name;
struct device *dev;
struct regmap *regmap;
- struct regulator *vdd;
struct iio_trigger *trig;
int irq;
#include <linux/delay.h>
#include <linux/pm.h>
#include <linux/regmap.h>
+#include <linux/regulator/consumer.h>
#include <linux/bitfield.h>
#include "hts221.h"
static int hts221_init_regulators(struct device *dev)
{
- struct iio_dev *iio_dev = dev_get_drvdata(dev);
- struct hts221_hw *hw = iio_priv(iio_dev);
int err;
- hw->vdd = devm_regulator_get(dev, "vdd");
- if (IS_ERR(hw->vdd))
- return dev_err_probe(dev, PTR_ERR(hw->vdd),
- "failed to get vdd regulator\n");
-
- err = regulator_enable(hw->vdd);
- if (err) {
- dev_err(dev, "failed to enable vdd regulator: %d\n", err);
- return err;
- }
+ err = devm_regulator_get_enable(dev, "vdd");
+ if (err)
+ return dev_err_probe(dev, err, "failed to get vdd regulator\n");
msleep(50);
return 0;
}
-static void hts221_chip_uninit(void *data)
-{
- struct hts221_hw *hw = data;
-
- regulator_disable(hw->vdd);
-}
-
int hts221_probe(struct device *dev, int irq, const char *name,
struct regmap *regmap)
{
if (err)
return err;
- err = devm_add_action_or_reset(dev, hts221_chip_uninit, hw);
- if (err)
- return err;
-
err = hts221_check_whoami(hw);
if (err < 0)
return err;
.read_flag_mask = HTS221_I2C_AUTO_INCREMENT,
};
-static int hts221_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int hts221_i2c_probe(struct i2c_client *client)
{
struct regmap *regmap;
.of_match_table = hts221_i2c_of_match,
.acpi_match_table = ACPI_PTR(hts221_acpi_match),
},
- .probe = hts221_i2c_probe,
+ .probe_new = hts221_i2c_probe,
.id_table = hts221_i2c_id_table,
};
module_i2c_driver(hts221_driver);
.attrs = &htu21_attribute_group,
};
-static int htu21_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int htu21_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct ms_ht_dev *dev_data;
struct iio_dev *indio_dev;
int ret;
MODULE_DEVICE_TABLE(of, htu21_of_match);
static struct i2c_driver htu21_driver = {
- .probe = htu21_probe,
+ .probe_new = htu21_probe,
.id_table = htu21_id,
.driver = {
.name = "htu21",
.read_raw = si7005_read_raw,
};
-static int si7005_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int si7005_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct si7005_data *data;
.driver = {
.name = "si7005",
},
- .probe = si7005_probe,
+ .probe_new = si7005_probe,
.id_table = si7005_id,
};
module_i2c_driver(si7005_driver);
.read_raw = si7020_read_raw,
};
-static int si7020_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int si7020_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct i2c_client **data;
.name = "si7020",
.of_match_table = si7020_dt_ids,
},
- .probe = si7020_probe,
+ .probe_new = si7020_probe,
.id_table = si7020_id,
};
#endif
/**
- * adis_enable_irq() - Enable or disable data ready IRQ
+ * __adis_enable_irq() - Enable or disable data ready IRQ (unlocked)
* @adis: The adis device
* @enable: Whether to enable the IRQ
*
* Returns 0 on success, negative error code otherwise
*/
-int adis_enable_irq(struct adis *adis, bool enable)
+int __adis_enable_irq(struct adis *adis, bool enable)
{
- int ret = 0;
+ int ret;
u16 msc;
- mutex_lock(&adis->state_lock);
-
- if (adis->data->enable_irq) {
- ret = adis->data->enable_irq(adis, enable);
- goto out_unlock;
- }
+ if (adis->data->enable_irq)
+ return adis->data->enable_irq(adis, enable);
if (adis->data->unmasked_drdy) {
if (enable)
else
disable_irq(adis->spi->irq);
- goto out_unlock;
+ return 0;
}
ret = __adis_read_reg_16(adis, adis->data->msc_ctrl_reg, &msc);
if (ret)
- goto out_unlock;
+ return ret;
msc |= ADIS_MSC_CTRL_DATA_RDY_POL_HIGH;
msc &= ~ADIS_MSC_CTRL_DATA_RDY_DIO2;
else
msc &= ~ADIS_MSC_CTRL_DATA_RDY_EN;
- ret = __adis_write_reg_16(adis, adis->data->msc_ctrl_reg, msc);
-
-out_unlock:
- mutex_unlock(&adis->state_lock);
- return ret;
+ return __adis_write_reg_16(adis, adis->data->msc_ctrl_reg, msc);
}
-EXPORT_SYMBOL_NS(adis_enable_irq, IIO_ADISLIB);
+EXPORT_SYMBOL_NS(__adis_enable_irq, IIO_ADISLIB);
/**
* __adis_check_status() - Check the device for error conditions (unlocked)
* with 'IRQF_NO_AUTOEN' anyways.
*/
if (!adis->data->unmasked_drdy)
- adis_enable_irq(adis, false);
+ __adis_enable_irq(adis, false);
if (!adis->data->prod_id_reg)
return 0;
st->adis.spi->mode = SPI_MODE_3;
spi_setup(st->adis.spi);
- ret = adis_initial_startup(&st->adis);
+ ret = __adis_initial_startup(&st->adis);
if (ret)
return ret;
#include "bmi160.h"
-static int bmi160_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bmi160_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct regmap *regmap;
const char *name;
.acpi_match_table = bmi160_acpi_match,
.of_match_table = bmi160_of_match,
},
- .probe = bmi160_i2c_probe,
+ .probe_new = bmi160_i2c_probe,
.id_table = bmi160_i2c_id,
};
module_i2c_driver(bmi160_i2c_driver);
#include "fxos8700.h"
-static int fxos8700_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int fxos8700_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct regmap *regmap;
const char *name = NULL;
.acpi_match_table = ACPI_PTR(fxos8700_acpi_match),
.of_match_table = fxos8700_of_match,
},
- .probe = fxos8700_i2c_probe,
+ .probe_new = fxos8700_i2c_probe,
.id_table = fxos8700_i2c_id,
};
module_i2c_driver(fxos8700_i2c_driver);
INV_CHIP_ICM42602,
INV_CHIP_ICM42605,
INV_CHIP_ICM42622,
+ INV_CHIP_ICM42631,
INV_CHIP_NB,
};
#define INV_ICM42600_WHOAMI_ICM42602 0x41
#define INV_ICM42600_WHOAMI_ICM42605 0x42
#define INV_ICM42600_WHOAMI_ICM42622 0x46
+#define INV_ICM42600_WHOAMI_ICM42631 0x5C
/* User bank 1 (MSB 0x10) */
#define INV_ICM42600_REG_SENSOR_CONFIG0 0x1003
.ranges = inv_icm42600_regmap_ranges,
.num_ranges = ARRAY_SIZE(inv_icm42600_regmap_ranges),
};
-EXPORT_SYMBOL_GPL(inv_icm42600_regmap_config);
+EXPORT_SYMBOL_NS_GPL(inv_icm42600_regmap_config, IIO_ICM42600);
struct inv_icm42600_hw {
uint8_t whoami;
.name = "icm42622",
.conf = &inv_icm42600_default_conf,
},
+ [INV_CHIP_ICM42631] = {
+ .whoami = INV_ICM42600_WHOAMI_ICM42631,
+ .name = "icm42631",
+ .conf = &inv_icm42600_default_conf,
+ },
};
const struct iio_mount_matrix *
return devm_add_action_or_reset(dev, inv_icm42600_disable_pm, dev);
}
-EXPORT_SYMBOL_GPL(inv_icm42600_core_probe);
+EXPORT_SYMBOL_NS_GPL(inv_icm42600_core_probe, IIO_ICM42600);
/*
* Suspend saves sensors state and turns everything off.
* Check first if runtime suspend has not already done the job.
*/
-static int __maybe_unused inv_icm42600_suspend(struct device *dev)
+static int inv_icm42600_suspend(struct device *dev)
{
struct inv_icm42600_state *st = dev_get_drvdata(dev);
int ret;
* System resume gets the system back on and restores the sensors state.
* Manually put runtime power management in system active state.
*/
-static int __maybe_unused inv_icm42600_resume(struct device *dev)
+static int inv_icm42600_resume(struct device *dev)
{
struct inv_icm42600_state *st = dev_get_drvdata(dev);
int ret;
}
/* Runtime suspend will turn off sensors that are enabled by iio devices. */
-static int __maybe_unused inv_icm42600_runtime_suspend(struct device *dev)
+static int inv_icm42600_runtime_suspend(struct device *dev)
{
struct inv_icm42600_state *st = dev_get_drvdata(dev);
int ret;
}
/* Sensors are enabled by iio devices, no need to turn them back on here. */
-static int __maybe_unused inv_icm42600_runtime_resume(struct device *dev)
+static int inv_icm42600_runtime_resume(struct device *dev)
{
struct inv_icm42600_state *st = dev_get_drvdata(dev);
int ret;
return ret;
}
-const struct dev_pm_ops inv_icm42600_pm_ops = {
- SET_SYSTEM_SLEEP_PM_OPS(inv_icm42600_suspend, inv_icm42600_resume)
- SET_RUNTIME_PM_OPS(inv_icm42600_runtime_suspend,
- inv_icm42600_runtime_resume, NULL)
+EXPORT_NS_GPL_DEV_PM_OPS(inv_icm42600_pm_ops, IIO_ICM42600) = {
+ SYSTEM_SLEEP_PM_OPS(inv_icm42600_suspend, inv_icm42600_resume)
+ RUNTIME_PM_OPS(inv_icm42600_runtime_suspend,
+ inv_icm42600_runtime_resume, NULL)
};
-EXPORT_SYMBOL_GPL(inv_icm42600_pm_ops);
MODULE_AUTHOR("InvenSense, Inc.");
MODULE_DESCRIPTION("InvenSense ICM-426xx device driver");
}, {
.compatible = "invensense,icm42622",
.data = (void *)INV_CHIP_ICM42622,
+ }, {
+ .compatible = "invensense,icm42631",
+ .data = (void *)INV_CHIP_ICM42631,
},
{}
};
.driver = {
.name = "inv-icm42600-i2c",
.of_match_table = inv_icm42600_of_matches,
- .pm = &inv_icm42600_pm_ops,
+ .pm = pm_ptr(&inv_icm42600_pm_ops),
},
.probe_new = inv_icm42600_probe,
};
MODULE_AUTHOR("InvenSense, Inc.");
MODULE_DESCRIPTION("InvenSense ICM-426xx I2C driver");
MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(IIO_ICM42600);
}, {
.compatible = "invensense,icm42622",
.data = (void *)INV_CHIP_ICM42622,
+ }, {
+ .compatible = "invensense,icm42631",
+ .data = (void *)INV_CHIP_ICM42631,
},
{}
};
.driver = {
.name = "inv-icm42600-spi",
.of_match_table = inv_icm42600_of_matches,
- .pm = &inv_icm42600_pm_ops,
+ .pm = pm_ptr(&inv_icm42600_pm_ops),
},
.probe = inv_icm42600_probe,
};
MODULE_AUTHOR("InvenSense, Inc.");
MODULE_DESCRIPTION("InvenSense ICM-426xx SPI driver");
MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(IIO_ICM42600);
inv_mpu6050_set_power_itg(st, false);
return result;
}
-EXPORT_SYMBOL_GPL(inv_mpu_core_probe);
+EXPORT_SYMBOL_NS_GPL(inv_mpu_core_probe, IIO_MPU6050);
-static int __maybe_unused inv_mpu_resume(struct device *dev)
+static int inv_mpu_resume(struct device *dev)
{
struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct inv_mpu6050_state *st = iio_priv(indio_dev);
return result;
}
-static int __maybe_unused inv_mpu_suspend(struct device *dev)
+static int inv_mpu_suspend(struct device *dev)
{
struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct inv_mpu6050_state *st = iio_priv(indio_dev);
return result;
}
-static int __maybe_unused inv_mpu_runtime_suspend(struct device *dev)
+static int inv_mpu_runtime_suspend(struct device *dev)
{
struct inv_mpu6050_state *st = iio_priv(dev_get_drvdata(dev));
unsigned int sensors;
return ret;
}
-static int __maybe_unused inv_mpu_runtime_resume(struct device *dev)
+static int inv_mpu_runtime_resume(struct device *dev)
{
struct inv_mpu6050_state *st = iio_priv(dev_get_drvdata(dev));
int ret;
return inv_mpu6050_set_power_itg(st, true);
}
-const struct dev_pm_ops inv_mpu_pmops = {
- SET_SYSTEM_SLEEP_PM_OPS(inv_mpu_suspend, inv_mpu_resume)
- SET_RUNTIME_PM_OPS(inv_mpu_runtime_suspend, inv_mpu_runtime_resume, NULL)
+EXPORT_NS_GPL_DEV_PM_OPS(inv_mpu_pmops, IIO_MPU6050) = {
+ SYSTEM_SLEEP_PM_OPS(inv_mpu_suspend, inv_mpu_resume)
+ RUNTIME_PM_OPS(inv_mpu_runtime_suspend, inv_mpu_runtime_resume, NULL)
};
-EXPORT_SYMBOL_GPL(inv_mpu_pmops);
MODULE_AUTHOR("Invensense Corporation");
MODULE_DESCRIPTION("Invensense device MPU6050 driver");
/**
* inv_mpu_probe() - probe function.
* @client: i2c client.
- * @id: i2c device id.
*
* Returns 0 on success, a negative error code otherwise.
*/
-static int inv_mpu_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int inv_mpu_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
const void *match;
struct inv_mpu6050_state *st;
int result;
MODULE_DEVICE_TABLE(acpi, inv_acpi_match);
static struct i2c_driver inv_mpu_driver = {
- .probe = inv_mpu_probe,
+ .probe_new = inv_mpu_probe,
.remove = inv_mpu_remove,
.id_table = inv_mpu_id,
.driver = {
.of_match_table = inv_of_match,
.acpi_match_table = inv_acpi_match,
.name = "inv-mpu6050-i2c",
- .pm = &inv_mpu_pmops,
+ .pm = pm_ptr(&inv_mpu_pmops),
},
};
MODULE_AUTHOR("Invensense Corporation");
MODULE_DESCRIPTION("Invensense device MPU6050 driver");
MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(IIO_MPU6050);
.of_match_table = inv_of_match,
.acpi_match_table = inv_acpi_match,
.name = "inv-mpu6000-spi",
- .pm = &inv_mpu_pmops,
+ .pm = pm_ptr(&inv_mpu_pmops),
},
};
MODULE_AUTHOR("Adriana Reus <adriana.reus@intel.com>");
MODULE_DESCRIPTION("Invensense device MPU6000 driver");
MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(IIO_MPU6050);
return trig;
}
-static int kmx61_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int kmx61_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
int ret;
struct kmx61_data *data;
const char *name = NULL;
.acpi_match_table = ACPI_PTR(kmx61_acpi_match),
.pm = pm_ptr(&kmx61_pm_ops),
},
- .probe = kmx61_probe,
+ .probe_new = kmx61_probe,
.remove = kmx61_remove,
.id_table = kmx61_id,
};
sensor. Supported devices: lsm6ds3, lsm6ds3h, lsm6dsl, lsm6dsm,
ism330dlc, lsm6dso, lsm6dsox, asm330lhh, asm330lhhx, lsm6dsr,
lsm6ds3tr-c, ism330dhcx, lsm6dsrx, lsm6ds0, lsm6dsop, lsm6dstx,
- the accelerometer/gyroscope of lsm9ds1 and lsm6dst.
+ lsm6dsv, lsm6dsv16x, lsm6dso16is, ism330is, lsm6dst and the
+ accelerometer/gyroscope of lsm9ds1.
To compile this driver as a module, choose M here: the module
will be called st_lsm6dsx.
#define ST_LSM6DSOP_DEV_NAME "lsm6dsop"
#define ST_ASM330LHHX_DEV_NAME "asm330lhhx"
#define ST_LSM6DSTX_DEV_NAME "lsm6dstx"
+#define ST_LSM6DSV_DEV_NAME "lsm6dsv"
+#define ST_LSM6DSV16X_DEV_NAME "lsm6dsv16x"
+#define ST_LSM6DSO16IS_DEV_NAME "lsm6dso16is"
+#define ST_ISM330IS_DEV_NAME "ism330is"
enum st_lsm6dsx_hw_id {
ST_LSM6DS3_ID,
ST_LSM6DSOP_ID,
ST_ASM330LHHX_ID,
ST_LSM6DSTX_ID,
+ ST_LSM6DSV_ID,
+ ST_LSM6DSV16X_ID,
+ ST_LSM6DSO16IS_ID,
+ ST_ISM330IS_ID,
ST_LSM6DSX_MAX_ID,
};
* struct st_lsm6dsx_hw - ST IMU MEMS hw instance
* @dev: Pointer to instance of struct device (I2C or SPI).
* @regmap: Register map of the device.
- * @regulators: VDD/VDDIO voltage regulators.
* @irq: Device interrupt line (I2C or SPI).
* @fifo_lock: Mutex to prevent concurrent access to the hw FIFO.
* @conf_lock: Mutex to prevent concurrent FIFO configuration update.
struct st_lsm6dsx_hw {
struct device *dev;
struct regmap *regmap;
- struct regulator_bulk_data regulators[2];
int irq;
struct mutex fifo_lock;
struct {
__le16 channels[3];
s64 ts __aligned(8);
- } scan[3];
+ } scan[ST_LSM6DSX_ID_MAX];
};
static __maybe_unused const struct iio_event_spec st_lsm6dsx_event = {
int st_lsm6dsx_check_odr(struct st_lsm6dsx_sensor *sensor, u32 odr, u8 *val);
int st_lsm6dsx_shub_probe(struct st_lsm6dsx_hw *hw, const char *name);
int st_lsm6dsx_shub_set_enable(struct st_lsm6dsx_sensor *sensor, bool enable);
+int st_lsm6dsx_shub_read_output(struct st_lsm6dsx_hw *hw, u8 *data, int len);
int st_lsm6dsx_set_page(struct st_lsm6dsx_hw *hw, bool enable);
static inline int
return &hw->orientation;
}
+static inline int
+st_lsm6dsx_device_set_enable(struct st_lsm6dsx_sensor *sensor, bool enable)
+{
+ if (sensor->id == ST_LSM6DSX_ID_EXT0 ||
+ sensor->id == ST_LSM6DSX_ID_EXT1 ||
+ sensor->id == ST_LSM6DSX_ID_EXT2)
+ return st_lsm6dsx_shub_set_enable(sensor, enable);
+
+ return st_lsm6dsx_sensor_set_enable(sensor, enable);
+}
+
static const
struct iio_chan_spec_ext_info __maybe_unused st_lsm6dsx_accel_ext_info[] = {
IIO_MOUNT_MATRIX(IIO_SHARED_BY_ALL, st_lsm6dsx_get_mount_matrix),
* value of the decimation factor and ODR set for each FIFO data set.
*
* LSM6DSO/LSM6DSOX/ASM330LHH/ASM330LHHX/LSM6DSR/LSM6DSRX/ISM330DHCX/
- * LSM6DST/LSM6DSOP/LSM6DSTX:
+ * LSM6DST/LSM6DSOP/LSM6DSTX/LSM6DSV:
* The FIFO buffer can be configured to store data from gyroscope and
* accelerometer. Each sample is queued with a tag (1B) indicating data
* source (gyroscope, accelerometer, hw timer).
goto out;
}
- if (sensor->id == ST_LSM6DSX_ID_EXT0 ||
- sensor->id == ST_LSM6DSX_ID_EXT1 ||
- sensor->id == ST_LSM6DSX_ID_EXT2) {
- err = st_lsm6dsx_shub_set_enable(sensor, enable);
- if (err < 0)
- goto out;
- } else {
- err = st_lsm6dsx_sensor_set_enable(sensor, enable);
- if (err < 0)
- goto out;
- }
+ err = st_lsm6dsx_device_set_enable(sensor, enable);
+ if (err < 0)
+ goto out;
err = st_lsm6dsx_set_fifo_odr(sensor, enable);
if (err < 0)
* - FIFO size: 4KB
*
* - LSM6DSO/LSM6DSOX/ASM330LHH/ASM330LHHX/LSM6DSR/ISM330DHCX/LSM6DST/LSM6DSOP/
- * LSM6DSTX:
+ * LSM6DSTX/LSM6DSO16IS/ISM330IS:
* - Accelerometer/Gyroscope supported ODR [Hz]: 12.5, 26, 52, 104, 208, 416,
* 833
* - Accelerometer supported full-scale [g]: +-2/+-4/+-8/+-16
* - Gyroscope supported full-scale [dps]: +-125/+-245/+-500/+-1000/+-2000
* - FIFO size: 3KB
*
+ * - LSM6DSV/LSM6DSV16X:
+ * - Accelerometer/Gyroscope supported ODR [Hz]: 7.5, 15, 30, 60, 120, 240,
+ * 480, 960
+ * - Accelerometer supported full-scale [g]: +-2/+-4/+-8/+-16
+ * - Gyroscope supported full-scale [dps]: +-125/+-250/+-500/+-1000/+-2000
+ * - FIFO size: 3KB
+ *
* - LSM9DS1/LSM6DS0:
* - Accelerometer supported ODR [Hz]: 10, 50, 119, 238, 476, 952
* - Accelerometer supported full-scale [g]: +-2/+-4/+-8/+-16
#include <linux/iio/events.h>
#include <linux/iio/iio.h>
#include <linux/iio/sysfs.h>
+#include <linux/iio/triggered_buffer.h>
+#include <linux/iio/trigger_consumer.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/minmax.h>
.wakeup_src_x_mask = BIT(2),
},
},
+ {
+ .reset = {
+ .addr = 0x12,
+ .mask = BIT(0),
+ },
+ .boot = {
+ .addr = 0x12,
+ .mask = BIT(7),
+ },
+ .bdu = {
+ .addr = 0x12,
+ .mask = BIT(6),
+ },
+ .id = {
+ {
+ .hw_id = ST_LSM6DSV_ID,
+ .name = ST_LSM6DSV_DEV_NAME,
+ .wai = 0x70,
+ }, {
+ .hw_id = ST_LSM6DSV16X_ID,
+ .name = ST_LSM6DSV16X_DEV_NAME,
+ .wai = 0x70,
+ },
+ },
+ .channels = {
+ [ST_LSM6DSX_ID_ACC] = {
+ .chan = st_lsm6dsx_acc_channels,
+ .len = ARRAY_SIZE(st_lsm6dsx_acc_channels),
+ },
+ [ST_LSM6DSX_ID_GYRO] = {
+ .chan = st_lsm6dsx_gyro_channels,
+ .len = ARRAY_SIZE(st_lsm6dsx_gyro_channels),
+ },
+ },
+ .drdy_mask = {
+ .addr = 0x13,
+ .mask = BIT(3),
+ },
+ .odr_table = {
+ [ST_LSM6DSX_ID_ACC] = {
+ .reg = {
+ .addr = 0x10,
+ .mask = GENMASK(3, 0),
+ },
+ .odr_avl[0] = { 7500, 0x02 },
+ .odr_avl[1] = { 15000, 0x03 },
+ .odr_avl[2] = { 30000, 0x04 },
+ .odr_avl[3] = { 60000, 0x05 },
+ .odr_avl[4] = { 120000, 0x06 },
+ .odr_avl[5] = { 240000, 0x07 },
+ .odr_avl[6] = { 480000, 0x08 },
+ .odr_avl[7] = { 960000, 0x09 },
+ .odr_len = 8,
+ },
+ [ST_LSM6DSX_ID_GYRO] = {
+ .reg = {
+ .addr = 0x11,
+ .mask = GENMASK(3, 0),
+ },
+ .odr_avl[0] = { 7500, 0x02 },
+ .odr_avl[1] = { 15000, 0x03 },
+ .odr_avl[2] = { 30000, 0x04 },
+ .odr_avl[3] = { 60000, 0x05 },
+ .odr_avl[4] = { 120000, 0x06 },
+ .odr_avl[5] = { 240000, 0x07 },
+ .odr_avl[6] = { 480000, 0x08 },
+ .odr_avl[7] = { 960000, 0x09 },
+ .odr_len = 8,
+ },
+ },
+ .fs_table = {
+ [ST_LSM6DSX_ID_ACC] = {
+ .reg = {
+ .addr = 0x17,
+ .mask = GENMASK(1, 0),
+ },
+ .fs_avl[0] = { IIO_G_TO_M_S_2(61000), 0x0 },
+ .fs_avl[1] = { IIO_G_TO_M_S_2(122000), 0x1 },
+ .fs_avl[2] = { IIO_G_TO_M_S_2(244000), 0x2 },
+ .fs_avl[3] = { IIO_G_TO_M_S_2(488000), 0x3 },
+ .fs_len = 4,
+ },
+ [ST_LSM6DSX_ID_GYRO] = {
+ .reg = {
+ .addr = 0x15,
+ .mask = GENMASK(3, 0),
+ },
+ .fs_avl[0] = { IIO_DEGREE_TO_RAD(8750000), 0x1 },
+ .fs_avl[1] = { IIO_DEGREE_TO_RAD(17500000), 0x2 },
+ .fs_avl[2] = { IIO_DEGREE_TO_RAD(35000000), 0x3 },
+ .fs_avl[3] = { IIO_DEGREE_TO_RAD(70000000), 0x4 },
+ .fs_len = 4,
+ },
+ },
+ .irq_config = {
+ .irq1 = {
+ .addr = 0x0d,
+ .mask = BIT(3),
+ },
+ .irq2 = {
+ .addr = 0x0e,
+ .mask = BIT(3),
+ },
+ .lir = {
+ .addr = 0x56,
+ .mask = BIT(0),
+ },
+ .irq1_func = {
+ .addr = 0x5e,
+ .mask = BIT(5),
+ },
+ .irq2_func = {
+ .addr = 0x5f,
+ .mask = BIT(5),
+ },
+ .hla = {
+ .addr = 0x03,
+ .mask = BIT(4),
+ },
+ .od = {
+ .addr = 0x03,
+ .mask = BIT(3),
+ },
+ },
+ .batch = {
+ [ST_LSM6DSX_ID_ACC] = {
+ .addr = 0x09,
+ .mask = GENMASK(3, 0),
+ },
+ [ST_LSM6DSX_ID_GYRO] = {
+ .addr = 0x09,
+ .mask = GENMASK(7, 4),
+ },
+ },
+ .fifo_ops = {
+ .update_fifo = st_lsm6dsx_update_fifo,
+ .read_fifo = st_lsm6dsx_read_tagged_fifo,
+ .fifo_th = {
+ .addr = 0x07,
+ .mask = GENMASK(7, 0),
+ },
+ .fifo_diff = {
+ .addr = 0x1b,
+ .mask = GENMASK(8, 0),
+ },
+ .max_size = 512,
+ .th_wl = 1,
+ },
+ .ts_settings = {
+ .timer_en = {
+ .addr = 0x50,
+ .mask = BIT(6),
+ },
+ .decimator = {
+ .addr = 0x0a,
+ .mask = GENMASK(7, 6),
+ },
+ .freq_fine = 0x4f,
+ },
+ .shub_settings = {
+ .page_mux = {
+ .addr = 0x01,
+ .mask = BIT(6),
+ },
+ .master_en = {
+ .sec_page = true,
+ .addr = 0x14,
+ .mask = BIT(2),
+ },
+ .pullup_en = {
+ .addr = 0x03,
+ .mask = BIT(6),
+ },
+ .aux_sens = {
+ .addr = 0x14,
+ .mask = GENMASK(1, 0),
+ },
+ .wr_once = {
+ .addr = 0x14,
+ .mask = BIT(6),
+ },
+ .num_ext_dev = 3,
+ .shub_out = {
+ .sec_page = true,
+ .addr = 0x02,
+ },
+ .slv0_addr = 0x15,
+ .dw_slv0_addr = 0x21,
+ .batch_en = BIT(3),
+ },
+ .event_settings = {
+ .enable_reg = {
+ .addr = 0x50,
+ .mask = BIT(7),
+ },
+ .wakeup_reg = {
+ .addr = 0x5b,
+ .mask = GENMASK(5, 0),
+ },
+ .wakeup_src_reg = 0x45,
+ .wakeup_src_status_mask = BIT(3),
+ .wakeup_src_z_mask = BIT(0),
+ .wakeup_src_y_mask = BIT(1),
+ .wakeup_src_x_mask = BIT(2),
+ },
+ },
+ {
+ .reset = {
+ .addr = 0x12,
+ .mask = BIT(0),
+ },
+ .boot = {
+ .addr = 0x12,
+ .mask = BIT(7),
+ },
+ .bdu = {
+ .addr = 0x12,
+ .mask = BIT(6),
+ },
+ .id = {
+ {
+ .hw_id = ST_LSM6DSO16IS_ID,
+ .name = ST_LSM6DSO16IS_DEV_NAME,
+ .wai = 0x22,
+ }, {
+ .hw_id = ST_ISM330IS_ID,
+ .name = ST_ISM330IS_DEV_NAME,
+ .wai = 0x22,
+ }
+ },
+ .channels = {
+ [ST_LSM6DSX_ID_ACC] = {
+ .chan = st_lsm6dsx_acc_channels,
+ .len = ARRAY_SIZE(st_lsm6dsx_acc_channels),
+ },
+ [ST_LSM6DSX_ID_GYRO] = {
+ .chan = st_lsm6dsx_gyro_channels,
+ .len = ARRAY_SIZE(st_lsm6dsx_gyro_channels),
+ },
+ },
+ .odr_table = {
+ [ST_LSM6DSX_ID_ACC] = {
+ .reg = {
+ .addr = 0x10,
+ .mask = GENMASK(7, 4),
+ },
+ .odr_avl[0] = { 12500, 0x01 },
+ .odr_avl[1] = { 26000, 0x02 },
+ .odr_avl[2] = { 52000, 0x03 },
+ .odr_avl[3] = { 104000, 0x04 },
+ .odr_avl[4] = { 208000, 0x05 },
+ .odr_avl[5] = { 416000, 0x06 },
+ .odr_avl[6] = { 833000, 0x07 },
+ .odr_len = 7,
+ },
+ [ST_LSM6DSX_ID_GYRO] = {
+ .reg = {
+ .addr = 0x11,
+ .mask = GENMASK(7, 4),
+ },
+ .odr_avl[0] = { 12500, 0x01 },
+ .odr_avl[1] = { 26000, 0x02 },
+ .odr_avl[2] = { 52000, 0x03 },
+ .odr_avl[3] = { 104000, 0x04 },
+ .odr_avl[4] = { 208000, 0x05 },
+ .odr_avl[5] = { 416000, 0x06 },
+ .odr_avl[6] = { 833000, 0x07 },
+ .odr_len = 7,
+ },
+ },
+ .fs_table = {
+ [ST_LSM6DSX_ID_ACC] = {
+ .reg = {
+ .addr = 0x10,
+ .mask = GENMASK(3, 2),
+ },
+ .fs_avl[0] = { IIO_G_TO_M_S_2(61000), 0x0 },
+ .fs_avl[1] = { IIO_G_TO_M_S_2(122000), 0x2 },
+ .fs_avl[2] = { IIO_G_TO_M_S_2(244000), 0x3 },
+ .fs_avl[3] = { IIO_G_TO_M_S_2(488000), 0x1 },
+ .fs_len = 4,
+ },
+ [ST_LSM6DSX_ID_GYRO] = {
+ .reg = {
+ .addr = 0x11,
+ .mask = GENMASK(3, 2),
+ },
+ .fs_avl[0] = { IIO_DEGREE_TO_RAD(8750000), 0x0 },
+ .fs_avl[1] = { IIO_DEGREE_TO_RAD(17500000), 0x1 },
+ .fs_avl[2] = { IIO_DEGREE_TO_RAD(35000000), 0x2 },
+ .fs_avl[3] = { IIO_DEGREE_TO_RAD(70000000), 0x3 },
+ .fs_len = 4,
+ },
+ },
+ .irq_config = {
+ .hla = {
+ .addr = 0x12,
+ .mask = BIT(5),
+ },
+ .od = {
+ .addr = 0x12,
+ .mask = BIT(4),
+ },
+ },
+ .shub_settings = {
+ .page_mux = {
+ .addr = 0x01,
+ .mask = BIT(6),
+ },
+ .master_en = {
+ .sec_page = true,
+ .addr = 0x14,
+ .mask = BIT(2),
+ },
+ .pullup_en = {
+ .sec_page = true,
+ .addr = 0x14,
+ .mask = BIT(3),
+ },
+ .aux_sens = {
+ .addr = 0x14,
+ .mask = GENMASK(1, 0),
+ },
+ .wr_once = {
+ .addr = 0x14,
+ .mask = BIT(6),
+ },
+ .num_ext_dev = 3,
+ .shub_out = {
+ .sec_page = true,
+ .addr = 0x02,
+ },
+ .slv0_addr = 0x15,
+ .dw_slv0_addr = 0x21,
+ },
+ },
};
int st_lsm6dsx_set_page(struct st_lsm6dsx_hw *hw, bool enable)
return fifo_len || event ? IRQ_HANDLED : IRQ_NONE;
}
+static irqreturn_t st_lsm6dsx_sw_trigger_handler_thread(int irq,
+ void *private)
+{
+ struct iio_poll_func *pf = private;
+ struct iio_dev *iio_dev = pf->indio_dev;
+ struct st_lsm6dsx_sensor *sensor = iio_priv(iio_dev);
+ struct st_lsm6dsx_hw *hw = sensor->hw;
+
+ if (sensor->id == ST_LSM6DSX_ID_EXT0 ||
+ sensor->id == ST_LSM6DSX_ID_EXT1 ||
+ sensor->id == ST_LSM6DSX_ID_EXT2)
+ st_lsm6dsx_shub_read_output(hw,
+ (u8 *)hw->scan[sensor->id].channels,
+ sizeof(hw->scan[sensor->id].channels));
+ else
+ st_lsm6dsx_read_locked(hw, iio_dev->channels[0].address,
+ hw->scan[sensor->id].channels,
+ sizeof(hw->scan[sensor->id].channels));
+
+ iio_push_to_buffers_with_timestamp(iio_dev, &hw->scan[sensor->id],
+ iio_get_time_ns(iio_dev));
+ iio_trigger_notify_done(iio_dev->trig);
+
+ return IRQ_HANDLED;
+}
+
static int st_lsm6dsx_irq_setup(struct st_lsm6dsx_hw *hw)
{
struct st_sensors_platform_data *pdata;
return 0;
}
-static int st_lsm6dsx_init_regulators(struct device *dev)
+static int st_lsm6dsx_sw_buffer_preenable(struct iio_dev *iio_dev)
{
- struct st_lsm6dsx_hw *hw = dev_get_drvdata(dev);
- int err;
+ struct st_lsm6dsx_sensor *sensor = iio_priv(iio_dev);
- /* vdd-vddio power regulators */
- hw->regulators[0].supply = "vdd";
- hw->regulators[1].supply = "vddio";
- err = devm_regulator_bulk_get(dev, ARRAY_SIZE(hw->regulators),
- hw->regulators);
- if (err)
- return dev_err_probe(dev, err, "failed to get regulators\n");
+ return st_lsm6dsx_device_set_enable(sensor, true);
+}
- err = regulator_bulk_enable(ARRAY_SIZE(hw->regulators),
- hw->regulators);
- if (err) {
- dev_err(dev, "failed to enable regulators: %d\n", err);
- return err;
- }
+static int st_lsm6dsx_sw_buffer_postdisable(struct iio_dev *iio_dev)
+{
+ struct st_lsm6dsx_sensor *sensor = iio_priv(iio_dev);
- msleep(50);
+ return st_lsm6dsx_device_set_enable(sensor, false);
+}
+
+static const struct iio_buffer_setup_ops st_lsm6dsx_sw_buffer_ops = {
+ .preenable = st_lsm6dsx_sw_buffer_preenable,
+ .postdisable = st_lsm6dsx_sw_buffer_postdisable,
+};
+
+static int st_lsm6dsx_sw_buffers_setup(struct st_lsm6dsx_hw *hw)
+{
+ int i;
+
+ for (i = 0; i < ST_LSM6DSX_ID_MAX; i++) {
+ int err;
+
+ if (!hw->iio_devs[i])
+ continue;
+
+ err = devm_iio_triggered_buffer_setup(hw->dev,
+ hw->iio_devs[i], NULL,
+ st_lsm6dsx_sw_trigger_handler_thread,
+ &st_lsm6dsx_sw_buffer_ops);
+ if (err)
+ return err;
+ }
return 0;
}
-static void st_lsm6dsx_chip_uninit(void *data)
+static int st_lsm6dsx_init_regulators(struct device *dev)
{
- struct st_lsm6dsx_hw *hw = data;
+ /* vdd-vddio power regulators */
+ static const char * const regulators[] = { "vdd", "vddio" };
+ int err;
- regulator_bulk_disable(ARRAY_SIZE(hw->regulators), hw->regulators);
+ err = devm_regulator_bulk_get_enable(dev, ARRAY_SIZE(regulators),
+ regulators);
+ if (err)
+ return dev_err_probe(dev, err, "failed to enable regulators\n");
+
+ msleep(50);
+
+ return 0;
}
int st_lsm6dsx_probe(struct device *dev, int irq, int hw_id,
if (err)
return err;
- err = devm_add_action_or_reset(dev, st_lsm6dsx_chip_uninit, hw);
- if (err)
- return err;
-
hw->buff = devm_kzalloc(dev, ST_LSM6DSX_BUFF_SIZE, GFP_KERNEL);
if (!hw->buff)
return -ENOMEM;
return err;
}
+ if (!hw->irq || !hw->settings->fifo_ops.read_fifo) {
+ /*
+ * Rely on sw triggers (e.g. hr-timers) if irq pin is not
+ * connected of if the device does not support HW FIFO
+ */
+ err = st_lsm6dsx_sw_buffers_setup(hw);
+ if (err)
+ return err;
+ }
+
err = iio_read_mount_matrix(hw->dev, &hw->orientation);
if (err)
return err;
continue;
}
- if (sensor->id == ST_LSM6DSX_ID_EXT0 ||
- sensor->id == ST_LSM6DSX_ID_EXT1 ||
- sensor->id == ST_LSM6DSX_ID_EXT2)
- err = st_lsm6dsx_shub_set_enable(sensor, false);
- else
- err = st_lsm6dsx_sensor_set_enable(sensor, false);
+ err = st_lsm6dsx_device_set_enable(sensor, false);
if (err < 0)
return err;
if (!(hw->suspend_mask & BIT(sensor->id)))
continue;
- if (sensor->id == ST_LSM6DSX_ID_EXT0 ||
- sensor->id == ST_LSM6DSX_ID_EXT1 ||
- sensor->id == ST_LSM6DSX_ID_EXT2)
- err = st_lsm6dsx_shub_set_enable(sensor, true);
- else
- err = st_lsm6dsx_sensor_set_enable(sensor, true);
+ err = st_lsm6dsx_device_set_enable(sensor, true);
if (err < 0)
return err;
.val_bits = 8,
};
-static int st_lsm6dsx_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int st_lsm6dsx_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
int hw_id = id->driver_data;
struct regmap *regmap;
.compatible = "st,lsm6dstx",
.data = (void *)ST_LSM6DSTX_ID,
},
+ {
+ .compatible = "st,lsm6dsv",
+ .data = (void *)ST_LSM6DSV_ID,
+ },
+ {
+ .compatible = "st,lsm6dsv16x",
+ .data = (void *)ST_LSM6DSV16X_ID,
+ },
+ {
+ .compatible = "st,lsm6dso16is",
+ .data = (void *)ST_LSM6DSO16IS_ID,
+ },
+ {
+ .compatible = "st,ism330is",
+ .data = (void *)ST_ISM330IS_ID,
+ },
{},
};
MODULE_DEVICE_TABLE(of, st_lsm6dsx_i2c_of_match);
{ ST_LSM6DSOP_DEV_NAME, ST_LSM6DSOP_ID },
{ ST_ASM330LHHX_DEV_NAME, ST_ASM330LHHX_ID },
{ ST_LSM6DSTX_DEV_NAME, ST_LSM6DSTX_ID },
+ { ST_LSM6DSV_DEV_NAME, ST_LSM6DSV_ID },
+ { ST_LSM6DSV16X_DEV_NAME, ST_LSM6DSV16X_ID },
+ { ST_LSM6DSO16IS_DEV_NAME, ST_LSM6DSO16IS_ID },
+ { ST_ISM330IS_DEV_NAME, ST_ISM330IS_ID },
{},
};
MODULE_DEVICE_TABLE(i2c, st_lsm6dsx_i2c_id_table);
.pm = pm_sleep_ptr(&st_lsm6dsx_pm_ops),
.of_match_table = st_lsm6dsx_i2c_of_match,
},
- .probe = st_lsm6dsx_i2c_probe,
+ .probe_new = st_lsm6dsx_i2c_probe,
.id_table = st_lsm6dsx_i2c_id_table,
};
module_i2c_driver(st_lsm6dsx_driver);
*
* Read st_lsm6dsx i2c controller register
*/
-static int
-st_lsm6dsx_shub_read_output(struct st_lsm6dsx_hw *hw, u8 *data,
- int len)
+int st_lsm6dsx_shub_read_output(struct st_lsm6dsx_hw *hw, u8 *data, int len)
{
const struct st_lsm6dsx_shub_settings *hub_settings;
int err;
.compatible = "st,lsm6dstx",
.data = (void *)ST_LSM6DSTX_ID,
},
+ {
+ .compatible = "st,lsm6dsv",
+ .data = (void *)ST_LSM6DSV_ID,
+ },
+ {
+ .compatible = "st,lsm6dsv16x",
+ .data = (void *)ST_LSM6DSV16X_ID,
+ },
+ {
+ .compatible = "st,lsm6dso16is",
+ .data = (void *)ST_LSM6DSO16IS_ID,
+ },
+ {
+ .compatible = "st,ism330is",
+ .data = (void *)ST_ISM330IS_ID,
+ },
{},
};
MODULE_DEVICE_TABLE(of, st_lsm6dsx_spi_of_match);
{ ST_LSM6DSOP_DEV_NAME, ST_LSM6DSOP_ID },
{ ST_ASM330LHHX_DEV_NAME, ST_ASM330LHHX_ID },
{ ST_LSM6DSTX_DEV_NAME, ST_LSM6DSTX_ID },
+ { ST_LSM6DSV_DEV_NAME, ST_LSM6DSV_ID },
+ { ST_LSM6DSV16X_DEV_NAME, ST_LSM6DSV16X_ID },
+ { ST_LSM6DSO16IS_DEV_NAME, ST_LSM6DSO16IS_ID },
+ { ST_ISM330IS_DEV_NAME, ST_ISM330IS_ID },
{},
};
MODULE_DEVICE_TABLE(spi, st_lsm6dsx_spi_id_table);
#include "st_lsm9ds0.h"
-static int st_lsm9ds0_power_enable(struct device *dev, struct st_lsm9ds0 *lsm9ds0)
-{
- int ret;
-
- /* Regulators not mandatory, but if requested we should enable them. */
- lsm9ds0->vdd = devm_regulator_get(dev, "vdd");
- if (IS_ERR(lsm9ds0->vdd))
- return dev_err_probe(dev, PTR_ERR(lsm9ds0->vdd),
- "unable to get Vdd supply\n");
-
- ret = regulator_enable(lsm9ds0->vdd);
- if (ret) {
- dev_warn(dev, "Failed to enable specified Vdd supply\n");
- return ret;
- }
-
- lsm9ds0->vdd_io = devm_regulator_get(dev, "vddio");
- if (IS_ERR(lsm9ds0->vdd_io)) {
- regulator_disable(lsm9ds0->vdd);
- return dev_err_probe(dev, PTR_ERR(lsm9ds0->vdd_io),
- "unable to get Vdd_IO supply\n");
- }
- ret = regulator_enable(lsm9ds0->vdd_io);
- if (ret) {
- dev_warn(dev, "Failed to enable specified Vdd_IO supply\n");
- regulator_disable(lsm9ds0->vdd);
- return ret;
- }
-
- return 0;
-}
-
-static void st_lsm9ds0_power_disable(void *data)
-{
- struct st_lsm9ds0 *lsm9ds0 = data;
-
- regulator_disable(lsm9ds0->vdd_io);
- regulator_disable(lsm9ds0->vdd);
-}
-
-static int devm_st_lsm9ds0_power_enable(struct st_lsm9ds0 *lsm9ds0)
-{
- struct device *dev = lsm9ds0->dev;
- int ret;
-
- ret = st_lsm9ds0_power_enable(dev, lsm9ds0);
- if (ret)
- return ret;
-
- return devm_add_action_or_reset(dev, st_lsm9ds0_power_disable, lsm9ds0);
-}
-
static int st_lsm9ds0_probe_accel(struct st_lsm9ds0 *lsm9ds0, struct regmap *regmap)
{
const struct st_sensor_settings *settings;
data->sensor_settings = (struct st_sensor_settings *)settings;
data->irq = lsm9ds0->irq;
data->regmap = regmap;
- data->vdd = lsm9ds0->vdd;
- data->vdd_io = lsm9ds0->vdd_io;
return st_accel_common_probe(lsm9ds0->accel);
}
data->sensor_settings = (struct st_sensor_settings *)settings;
data->irq = lsm9ds0->irq;
data->regmap = regmap;
- data->vdd = lsm9ds0->vdd;
- data->vdd_io = lsm9ds0->vdd_io;
return st_magn_common_probe(lsm9ds0->magn);
}
int st_lsm9ds0_probe(struct st_lsm9ds0 *lsm9ds0, struct regmap *regmap)
{
+ struct device *dev = lsm9ds0->dev;
+ static const char * const regulator_names[] = { "vdd", "vddio" };
int ret;
- ret = devm_st_lsm9ds0_power_enable(lsm9ds0);
+ /* Regulators not mandatory, but if requested we should enable them. */
+ ret = devm_regulator_bulk_get_enable(dev, ARRAY_SIZE(regulator_names),
+ regulator_names);
if (ret)
- return ret;
+ return dev_err_probe(dev, ret,
+ "unable to enable Vdd supply\n");
/* Setup accelerometer device */
ret = st_lsm9ds0_probe_accel(lsm9ds0, regmap);
int ret;
bool state;
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev_opaque *iio_dev_opaque = to_iio_dev_opaque(indio_dev);
struct iio_dev_attr *this_attr = to_iio_dev_attr(attr);
struct iio_buffer *buffer = this_attr->buffer;
ret = kstrtobool(buf, &state);
if (ret < 0)
return ret;
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&iio_dev_opaque->mlock);
if (iio_buffer_is_active(buffer)) {
ret = -EBUSY;
goto error_ret;
}
error_ret:
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return ret < 0 ? ret : len;
{
int ret;
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev_opaque *iio_dev_opaque = to_iio_dev_opaque(indio_dev);
struct iio_buffer *buffer = to_iio_dev_attr(attr)->buffer;
bool state;
if (ret < 0)
return ret;
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&iio_dev_opaque->mlock);
if (iio_buffer_is_active(buffer)) {
ret = -EBUSY;
goto error_ret;
}
buffer->scan_timestamp = state;
error_ret:
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return ret ? ret : len;
}
const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev_opaque *iio_dev_opaque = to_iio_dev_opaque(indio_dev);
struct iio_buffer *buffer = to_iio_dev_attr(attr)->buffer;
unsigned int val;
int ret;
if (val == buffer->length)
return len;
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&iio_dev_opaque->mlock);
if (iio_buffer_is_active(buffer)) {
ret = -EBUSY;
} else {
if (buffer->length && buffer->length < buffer->watermark)
buffer->watermark = buffer->length;
out:
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return ret ? ret : len;
}
return -EINVAL;
mutex_lock(&iio_dev_opaque->info_exist_lock);
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&iio_dev_opaque->mlock);
if (insert_buffer && iio_buffer_is_active(insert_buffer))
insert_buffer = NULL;
ret = __iio_update_buffers(indio_dev, insert_buffer, remove_buffer);
out_unlock:
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
mutex_unlock(&iio_dev_opaque->info_exist_lock);
return ret;
int ret;
bool requested_state;
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev_opaque *iio_dev_opaque = to_iio_dev_opaque(indio_dev);
struct iio_buffer *buffer = to_iio_dev_attr(attr)->buffer;
bool inlist;
if (ret < 0)
return ret;
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&iio_dev_opaque->mlock);
/* Find out if it is in the list */
inlist = iio_buffer_is_active(buffer);
ret = __iio_update_buffers(indio_dev, NULL, buffer);
done:
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return (ret < 0) ? ret : len;
}
const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
+ struct iio_dev_opaque *iio_dev_opaque = to_iio_dev_opaque(indio_dev);
struct iio_buffer *buffer = to_iio_dev_attr(attr)->buffer;
unsigned int val;
int ret;
if (!val)
return -EINVAL;
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&iio_dev_opaque->mlock);
if (val > buffer->length) {
ret = -EINVAL;
buffer->watermark = val;
out:
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return ret ? ret : len;
}
{
struct iio_dev_opaque *iio_dev_opaque = to_iio_dev_opaque(indio_dev);
struct iio_dev_attr *p;
+ const struct iio_dev_attr *id_attr;
struct attribute **attr;
int ret, i, attrn, scan_el_attrcount, buffer_attrcount;
const struct iio_chan_spec *channels;
while (buffer->attrs[buffer_attrcount] != NULL)
buffer_attrcount++;
}
+ buffer_attrcount += ARRAY_SIZE(iio_buffer_attrs);
scan_el_attrcount = 0;
INIT_LIST_HEAD(&buffer->buffer_attr_list);
}
}
- attrn = buffer_attrcount + scan_el_attrcount + ARRAY_SIZE(iio_buffer_attrs);
+ attrn = buffer_attrcount + scan_el_attrcount;
attr = kcalloc(attrn + 1, sizeof(*attr), GFP_KERNEL);
if (!attr) {
ret = -ENOMEM;
attr[2] = &dev_attr_watermark_ro.attr;
if (buffer->attrs)
- memcpy(&attr[ARRAY_SIZE(iio_buffer_attrs)], buffer->attrs,
- sizeof(struct attribute *) * buffer_attrcount);
+ for (i = 0, id_attr = buffer->attrs[i];
+ (id_attr = buffer->attrs[i]); i++)
+ attr[ARRAY_SIZE(iio_buffer_attrs) + i] =
+ (struct attribute *)&id_attr->dev_attr.attr;
- buffer_attrcount += ARRAY_SIZE(iio_buffer_attrs);
buffer->buffer_group.attrs = attr;
for (i = 0; i < buffer_attrcount; i++) {
struct iio_dev_opaque *iio_dev_opaque = to_iio_dev_opaque(indio_dev);
const struct iio_event_interface *ev_int = iio_dev_opaque->event_interface;
- ret = mutex_lock_interruptible(&indio_dev->mlock);
+ ret = mutex_lock_interruptible(&iio_dev_opaque->mlock);
if (ret)
return ret;
if ((ev_int && iio_event_enabled(ev_int)) ||
iio_buffer_enabled(indio_dev)) {
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return -EBUSY;
}
iio_dev_opaque->clock_id = clock_id;
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return 0;
}
indio_dev->dev.type = &iio_device_type;
indio_dev->dev.bus = &iio_bus_type;
device_initialize(&indio_dev->dev);
- mutex_init(&indio_dev->mlock);
+ mutex_init(&iio_dev_opaque->mlock);
mutex_init(&iio_dev_opaque->info_exist_lock);
INIT_LIST_HEAD(&iio_dev_opaque->channel_attr_list);
INIT_LIST_HEAD(&iio_dev_opaque->ioctl_handlers);
lockdep_register_key(&iio_dev_opaque->mlock_key);
- lockdep_set_class(&indio_dev->mlock, &iio_dev_opaque->mlock_key);
+ lockdep_set_class(&iio_dev_opaque->mlock, &iio_dev_opaque->mlock_key);
return indio_dev;
}
*/
int iio_device_claim_direct_mode(struct iio_dev *indio_dev)
{
- mutex_lock(&indio_dev->mlock);
+ struct iio_dev_opaque *iio_dev_opaque = to_iio_dev_opaque(indio_dev);
+
+ mutex_lock(&iio_dev_opaque->mlock);
if (iio_buffer_enabled(indio_dev)) {
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return -EBUSY;
}
return 0;
*/
void iio_device_release_direct_mode(struct iio_dev *indio_dev)
{
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&to_iio_dev_opaque(indio_dev)->mlock);
}
EXPORT_SYMBOL_GPL(iio_device_release_direct_mode);
/**
+ * iio_device_claim_buffer_mode - Keep device in buffer mode
+ * @indio_dev: the iio_dev associated with the device
+ *
+ * If the device is in buffer mode it is guaranteed to stay
+ * that way until iio_device_release_buffer_mode() is called.
+ *
+ * Use with iio_device_release_buffer_mode().
+ *
+ * Returns: 0 on success, -EBUSY on failure.
+ */
+int iio_device_claim_buffer_mode(struct iio_dev *indio_dev)
+{
+ struct iio_dev_opaque *iio_dev_opaque = to_iio_dev_opaque(indio_dev);
+
+ mutex_lock(&iio_dev_opaque->mlock);
+
+ if (iio_buffer_enabled(indio_dev))
+ return 0;
+
+ mutex_unlock(&iio_dev_opaque->mlock);
+ return -EBUSY;
+}
+EXPORT_SYMBOL_GPL(iio_device_claim_buffer_mode);
+
+/**
+ * iio_device_release_buffer_mode - releases claim on buffer mode
+ * @indio_dev: the iio_dev associated with the device
+ *
+ * Release the claim. Device is no longer guaranteed to stay
+ * in buffer mode.
+ *
+ * Use with iio_device_claim_buffer_mode().
+ */
+void iio_device_release_buffer_mode(struct iio_dev *indio_dev)
+{
+ mutex_unlock(&to_iio_dev_opaque(indio_dev)->mlock);
+}
+EXPORT_SYMBOL_GPL(iio_device_release_buffer_mode);
+
+/**
* iio_device_get_current_mode() - helper function providing read-only access to
* the opaque @currentmode variable
* @indio_dev: IIO device structure for device
if (ev_int == NULL)
return -ENODEV;
- fd = mutex_lock_interruptible(&indio_dev->mlock);
+ fd = mutex_lock_interruptible(&iio_dev_opaque->mlock);
if (fd)
return fd;
}
unlock:
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return fd;
}
ret = iio_device_register_sysfs_group(indio_dev, &ev_int->group);
if (ret)
- goto error_free_setup_event_lines;
+ goto error_free_group_attrs;
ev_int->ioctl_handler.ioctl = iio_event_ioctl;
iio_device_ioctl_handler_register(&iio_dev_opaque->indio_dev,
return 0;
+error_free_group_attrs:
+ kfree(ev_int->group.attrs);
error_free_setup_event_lines:
iio_free_chan_devattr_list(&ev_int->dev_attr_list);
kfree(ev_int);
return -EINVAL;
iio_dev_opaque = to_iio_dev_opaque(indio_dev);
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&iio_dev_opaque->mlock);
WARN_ON(iio_dev_opaque->trig_readonly);
indio_dev->trig = iio_trigger_get(trig);
iio_dev_opaque->trig_readonly = true;
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return 0;
}
struct iio_trigger *trig;
int ret;
- mutex_lock(&indio_dev->mlock);
+ mutex_lock(&iio_dev_opaque->mlock);
if (iio_dev_opaque->currentmode == INDIO_BUFFER_TRIGGERED) {
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return -EBUSY;
}
if (iio_dev_opaque->trig_readonly) {
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
return -EPERM;
}
- mutex_unlock(&indio_dev->mlock);
+ mutex_unlock(&iio_dev_opaque->mlock);
trig = iio_trigger_acquire_by_name(buf);
if (oldtrig == trig) {
.write_raw = adjd_s311_write_raw,
};
-static int adjd_s311_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int adjd_s311_probe(struct i2c_client *client)
{
struct adjd_s311_data *data;
struct iio_dev *indio_dev;
.driver = {
.name = ADJD_S311_DRV_NAME,
},
- .probe = adjd_s311_probe,
+ .probe_new = adjd_s311_probe,
.id_table = adjd_s311_id,
};
module_i2c_driver(adjd_s311_driver);
ADUX1020_MODE_INT_MASK, ADUX1020_MODE_INT_DISABLE);
}
-static int adux1020_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int adux1020_probe(struct i2c_client *client)
{
struct adux1020_data *data;
struct iio_dev *indio_dev;
.name = ADUX1020_DRV_NAME,
.of_match_table = adux1020_of_match,
},
- .probe = adux1020_probe,
+ .probe_new = adux1020_probe,
.id_table = adux1020_id,
};
module_i2c_driver(adux1020_driver);
.attrs = &al3010_attribute_group,
};
-static int al3010_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int al3010_probe(struct i2c_client *client)
{
struct al3010_data *data;
struct iio_dev *indio_dev;
.of_match_table = al3010_of_match,
.pm = pm_sleep_ptr(&al3010_pm_ops),
},
- .probe = al3010_probe,
+ .probe_new = al3010_probe,
.id_table = al3010_id,
};
module_i2c_driver(al3010_driver);
.attrs = &al3320a_attribute_group,
};
-static int al3320a_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int al3320a_probe(struct i2c_client *client)
{
struct al3320a_data *data;
struct iio_dev *indio_dev;
.of_match_table = al3320a_of_match,
.pm = pm_sleep_ptr(&al3320a_pm_ops),
},
- .probe = al3320a_probe,
+ .probe_new = al3320a_probe,
.id_table = al3320a_id,
};
return IRQ_HANDLED;
}
-static int apds9300_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int apds9300_probe(struct i2c_client *client)
{
struct apds9300_data *data;
struct iio_dev *indio_dev;
.name = APDS9300_DRV_NAME,
.pm = pm_sleep_ptr(&apds9300_pm_ops),
},
- .probe = apds9300_probe,
+ .probe_new = apds9300_probe,
.remove = apds9300_remove,
.id_table = apds9300_id,
};
{
.type = IIO_EV_TYPE_THRESH,
.dir = IIO_EV_DIR_RISING,
- .mask_separate = BIT(IIO_EV_INFO_VALUE) |
- BIT(IIO_EV_INFO_ENABLE),
+ .mask_separate = BIT(IIO_EV_INFO_VALUE),
},
{
.type = IIO_EV_TYPE_THRESH,
.dir = IIO_EV_DIR_FALLING,
- .mask_separate = BIT(IIO_EV_INFO_VALUE) |
- BIT(IIO_EV_INFO_ENABLE),
+ .mask_separate = BIT(IIO_EV_INFO_VALUE),
+ },
+ {
+ .type = IIO_EV_TYPE_THRESH,
+ .mask_separate = BIT(IIO_EV_INFO_ENABLE),
},
};
{
.type = IIO_EV_TYPE_THRESH,
.dir = IIO_EV_DIR_RISING,
- .mask_separate = BIT(IIO_EV_INFO_VALUE) |
- BIT(IIO_EV_INFO_ENABLE),
+ .mask_separate = BIT(IIO_EV_INFO_VALUE),
},
{
.type = IIO_EV_TYPE_THRESH,
.dir = IIO_EV_DIR_FALLING,
- .mask_separate = BIT(IIO_EV_INFO_VALUE) |
- BIT(IIO_EV_INFO_ENABLE),
+ .mask_separate = BIT(IIO_EV_INFO_VALUE),
+ },
+ {
+ .type = IIO_EV_TYPE_THRESH,
+ .mask_separate = BIT(IIO_EV_INFO_ENABLE),
},
};
return apds9960_set_powermode(data, 1);
}
-static int apds9960_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int apds9960_probe(struct i2c_client *client)
{
struct apds9960_data *data;
struct iio_dev *indio_dev;
.pm = &apds9960_pm_ops,
.acpi_match_table = apds9960_acpi_match,
},
- .probe = apds9960_probe,
+ .probe_new = apds9960_probe,
.remove = apds9960_remove,
.id_table = apds9960_id,
};
}
};
-static int bh1750_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bh1750_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
int ret, usec;
struct bh1750_data *data;
struct iio_dev *indio_dev;
.of_match_table = bh1750_of_match,
.pm = pm_sleep_ptr(&bh1750_pm_ops),
},
- .probe = bh1750_probe,
+ .probe_new = bh1750_probe,
.remove = bh1750_remove,
.id_table = bh1750_id,
}
};
-static int bh1780_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bh1780_probe(struct i2c_client *client)
{
int ret;
struct bh1780_data *bh1780;
MODULE_DEVICE_TABLE(of, of_bh1780_match);
static struct i2c_driver bh1780_driver = {
- .probe = bh1780_probe,
+ .probe_new = bh1780_probe,
.remove = bh1780_remove,
.id_table = bh1780_id,
.driver = {
.attrs = &cm3232_attribute_group,
};
-static int cm3232_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int cm3232_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct cm3232_chip *chip;
struct iio_dev *indio_dev;
int ret;
.pm = pm_sleep_ptr(&cm3232_pm_ops),
},
.id_table = cm3232_id,
- .probe = cm3232_probe,
+ .probe_new = cm3232_probe,
.remove = cm3232_remove,
};
.attrs = &cm3323_attribute_group,
};
-static int cm3323_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int cm3323_probe(struct i2c_client *client)
{
struct cm3323_data *data;
struct iio_dev *indio_dev;
.name = CM3323_DRV_NAME,
.of_match_table = cm3323_of_match,
},
- .probe = cm3323_probe,
+ .probe_new = cm3323_probe,
.id_table = cm3323_id,
};
.attrs = &cm36651_attribute_group,
};
-static int cm36651_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int cm36651_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct cm36651_data *cm36651;
struct iio_dev *indio_dev;
int ret;
.name = "cm36651",
.of_match_table = cm36651_of_match,
},
- .probe = cm36651_probe,
+ .probe_new = cm36651_probe,
.remove = cm36651_remove,
.id_table = cm36651_id,
};
.reg_write = gp2ap002_regmap_i2c_write,
};
-static int gp2ap002_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int gp2ap002_probe(struct i2c_client *client)
{
struct gp2ap002 *gp2ap002;
struct iio_dev *indio_dev;
.of_match_table = gp2ap002_of_match,
.pm = pm_ptr(&gp2ap002_dev_pm_ops),
},
- .probe = gp2ap002_probe,
+ .probe_new = gp2ap002_probe,
.remove = gp2ap002_remove,
.id_table = gp2ap002_id_table,
};
.predisable = &gp2ap020a00f_buffer_predisable,
};
-static int gp2ap020a00f_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int gp2ap020a00f_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct gp2ap020a00f_data *data;
struct iio_dev *indio_dev;
struct regmap *regmap;
.name = GP2A_I2C_NAME,
.of_match_table = gp2ap020a00f_of_match,
},
- .probe = gp2ap020a00f_probe,
+ .probe_new = gp2ap020a00f_probe,
.remove = gp2ap020a00f_remove,
.id_table = gp2ap020a00f_id,
};
pr_err("failed to disable isl29018's VCC regulator!\n");
}
-static int isl29018_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int isl29018_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct isl29018_chip *chip;
struct iio_dev *indio_dev;
int err;
.pm = pm_sleep_ptr(&isl29018_pm_ops),
.of_match_table = isl29018_of_match,
},
- .probe = isl29018_probe,
+ .probe_new = isl29018_probe,
.id_table = isl29018_id,
};
module_i2c_driver(isl29018_driver);
.cache_type = REGCACHE_RBTREE,
};
-static int isl29028_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int isl29028_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct isl29028_chip *chip;
struct iio_dev *indio_dev;
int ret;
.pm = pm_ptr(&isl29028_pm_ops),
.of_match_table = isl29028_of_match,
},
- .probe = isl29028_probe,
+ .probe_new = isl29028_probe,
.remove = isl29028_remove,
.id_table = isl29028_id,
};
.predisable = isl29125_buffer_predisable,
};
-static int isl29125_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int isl29125_probe(struct i2c_client *client)
{
struct isl29125_data *data;
struct iio_dev *indio_dev;
.name = ISL29125_DRV_NAME,
.pm = pm_sleep_ptr(&isl29125_pm_ops),
},
- .probe = isl29125_probe,
+ .probe_new = isl29125_probe,
.remove = isl29125_remove,
.id_table = isl29125_id,
};
.volatile_reg = jsa1212_is_volatile_reg,
};
-static int jsa1212_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int jsa1212_probe(struct i2c_client *client)
{
struct jsa1212_data *data;
struct iio_dev *indio_dev;
.pm = pm_sleep_ptr(&jsa1212_pm_ops),
.acpi_match_table = ACPI_PTR(jsa1212_acpi_match),
},
- .probe = jsa1212_probe,
+ .probe_new = jsa1212_probe,
.remove = jsa1212_remove,
.id_table = jsa1212_id,
};
struct ltr501_data {
struct i2c_client *client;
- struct regulator_bulk_data regulators[2];
struct mutex lock_als, lock_ps;
const struct ltr501_chip_info *chip_info;
u8 als_contr, ps_contr;
.volatile_reg = ltr501_is_volatile_reg,
};
-static void ltr501_disable_regulators(void *d)
-{
- struct ltr501_data *data = d;
-
- regulator_bulk_disable(ARRAY_SIZE(data->regulators), data->regulators);
-}
-
static int ltr501_powerdown(struct ltr501_data *data)
{
return ltr501_write_contr(data, data->als_contr &
return dev_name(dev);
}
-static int ltr501_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ltr501_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
+ static const char * const regulator_names[] = { "vdd", "vddio" };
struct ltr501_data *data;
struct iio_dev *indio_dev;
struct regmap *regmap;
mutex_init(&data->lock_als);
mutex_init(&data->lock_ps);
- data->regulators[0].supply = "vdd";
- data->regulators[1].supply = "vddio";
- ret = devm_regulator_bulk_get(&client->dev,
- ARRAY_SIZE(data->regulators),
- data->regulators);
+ ret = devm_regulator_bulk_get_enable(&client->dev,
+ ARRAY_SIZE(regulator_names),
+ regulator_names);
if (ret)
return dev_err_probe(&client->dev, ret,
"Failed to get regulators\n");
- ret = regulator_bulk_enable(ARRAY_SIZE(data->regulators),
- data->regulators);
- if (ret)
- return ret;
-
- ret = devm_add_action_or_reset(&client->dev,
- ltr501_disable_regulators, data);
- if (ret)
- return ret;
-
data->reg_it = devm_regmap_field_alloc(&client->dev, regmap,
reg_field_it);
if (IS_ERR(data->reg_it)) {
.pm = pm_sleep_ptr(<r501_pm_ops),
.acpi_match_table = ACPI_PTR(ltr_acpi_match),
},
- .probe = ltr501_probe,
+ .probe_new = ltr501_probe,
.remove = ltr501_remove,
.id_table = ltr501_id,
};
},
};
-static int lv0104cs_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int lv0104cs_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct lv0104cs_private *lv0104cs;
.name = "lv0104cs",
},
.id_table = lv0104cs_id,
- .probe = lv0104cs_probe,
+ .probe_new = lv0104cs_probe,
};
module_i2c_driver(lv0104cs_i2c_driver);
return IRQ_HANDLED;
}
-static int max44000_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max44000_probe(struct i2c_client *client)
{
struct max44000_data *data;
struct iio_dev *indio_dev;
.name = MAX44000_DRV_NAME,
.acpi_match_table = ACPI_PTR(max44000_acpi_match),
},
- .probe = max44000_probe,
+ .probe_new = max44000_probe,
.id_table = max44000_id,
};
struct noa1305_priv {
struct i2c_client *client;
struct regmap *regmap;
- struct regulator *vin_reg;
};
static int noa1305_measure(struct noa1305_priv *priv)
.writeable_reg = noa1305_writable_reg,
};
-static void noa1305_reg_remove(void *data)
-{
- struct noa1305_priv *priv = data;
-
- regulator_disable(priv->vin_reg);
-}
-
-static int noa1305_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int noa1305_probe(struct i2c_client *client)
{
struct noa1305_priv *priv;
struct iio_dev *indio_dev;
priv = iio_priv(indio_dev);
- priv->vin_reg = devm_regulator_get(&client->dev, "vin");
- if (IS_ERR(priv->vin_reg))
- return dev_err_probe(&client->dev, PTR_ERR(priv->vin_reg),
+ ret = devm_regulator_get_enable(&client->dev, "vin");
+ if (ret)
+ return dev_err_probe(&client->dev, ret,
"get regulator vin failed\n");
- ret = regulator_enable(priv->vin_reg);
- if (ret) {
- dev_err(&client->dev, "enable regulator vin failed\n");
- return ret;
- }
-
- ret = devm_add_action_or_reset(&client->dev, noa1305_reg_remove, priv);
- if (ret) {
- dev_err(&client->dev, "addition of devm action failed\n");
- return ret;
- }
-
i2c_set_clientdata(client, indio_dev);
priv->client = client;
priv->regmap = regmap;
.name = NOA1305_DRIVER_NAME,
.of_match_table = noa1305_of_match,
},
- .probe = noa1305_probe,
+ .probe_new = noa1305_probe,
.id_table = noa1305_ids,
};
return IRQ_HANDLED;
}
-static int opt3001_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int opt3001_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
MODULE_DEVICE_TABLE(of, opt3001_of_match);
static struct i2c_driver opt3001_driver = {
- .probe = opt3001_probe,
+ .probe_new = opt3001_probe,
.remove = opt3001_remove,
.id_table = opt3001_id,
return ret;
}
-static int pa12203001_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int pa12203001_probe(struct i2c_client *client)
{
struct pa12203001_data *data;
struct iio_dev *indio_dev;
.pm = &pa12203001_pm_ops,
.acpi_match_table = ACPI_PTR(pa12203001_acpi_match),
},
- .probe = pa12203001_probe,
+ .probe_new = pa12203001_probe,
.remove = pa12203001_remove,
.id_table = pa12203001_id,
.volatile_reg = rpr0521_is_volatile_reg,
};
-static int rpr0521_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int rpr0521_probe(struct i2c_client *client)
{
struct rpr0521_data *data;
struct iio_dev *indio_dev;
.pm = pm_ptr(&rpr0521_pm_ops),
.acpi_match_table = ACPI_PTR(rpr0521_acpi_match),
},
- .probe = rpr0521_probe,
+ .probe_new = rpr0521_probe,
.remove = rpr0521_remove,
.id_table = rpr0521_id,
};
return 0;
}
-static int si1133_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int si1133_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct si1133_data *data;
struct iio_dev *iio_dev;
int err;
.driver = {
.name = "si1133",
},
- .probe = si1133_probe,
+ .probe_new = si1133_probe,
.id_table = si1133_ids,
};
return 0;
}
-static int si1145_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int si1145_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct si1145_data *data;
struct iio_dev *indio_dev;
u8 part_id, rev_id, seq_id;
.driver = {
.name = "si1145",
},
- .probe = si1145_probe,
+ .probe_new = si1145_probe,
.id_table = si1145_ids,
};
.read_flag_mask = UVIS25_I2C_AUTO_INCREMENT,
};
-static int st_uvis25_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int st_uvis25_i2c_probe(struct i2c_client *client)
{
struct regmap *regmap;
.pm = pm_sleep_ptr(&st_uvis25_pm_ops),
.of_match_table = st_uvis25_i2c_of_match,
},
- .probe = st_uvis25_i2c_probe,
+ .probe_new = st_uvis25_i2c_probe,
.id_table = st_uvis25_i2c_id_table,
};
module_i2c_driver(st_uvis25_driver);
return IRQ_HANDLED;
}
-static int stk3310_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int stk3310_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
.pm = pm_sleep_ptr(&stk3310_pm_ops),
.acpi_match_table = ACPI_PTR(stk3310_acpi_id),
},
- .probe = stk3310_probe,
+ .probe_new = stk3310_probe,
.remove = stk3310_remove,
.id_table = stk3310_i2c_id,
};
tcs3414_powerdown(data);
}
-static int tcs3414_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tcs3414_probe(struct i2c_client *client)
{
struct tcs3414_data *data;
struct iio_dev *indio_dev;
.name = TCS3414_DRV_NAME,
.pm = pm_sleep_ptr(&tcs3414_pm_ops),
},
- .probe = tcs3414_probe,
+ .probe_new = tcs3414_probe,
.id_table = tcs3414_id,
};
module_i2c_driver(tcs3414_driver);
.attrs = &tcs3472_attribute_group,
};
-static int tcs3472_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tcs3472_probe(struct i2c_client *client)
{
struct tcs3472_data *data;
struct iio_dev *indio_dev;
.name = TCS3472_DRV_NAME,
.pm = pm_sleep_ptr(&tcs3472_pm_ops),
},
- .probe = tcs3472_probe,
+ .probe_new = tcs3472_probe,
.remove = tcs3472_remove,
.id_table = tcs3472_id,
};
.write_event_config = &tsl2563_write_interrupt_config,
};
-static int tsl2563_probe(struct i2c_client *client,
- const struct i2c_device_id *device_id)
+static int tsl2563_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct tsl2563_chip *chip;
.of_match_table = tsl2563_of_match,
.pm = pm_sleep_ptr(&tsl2563_pm_ops),
},
- .probe = tsl2563_probe,
+ .probe_new = tsl2563_probe,
.remove = tsl2563_remove,
.id_table = tsl2563_id,
};
.write_raw = tsl2583_write_raw,
};
-static int tsl2583_probe(struct i2c_client *clientp,
- const struct i2c_device_id *idp)
+static int tsl2583_probe(struct i2c_client *clientp)
{
int ret;
struct tsl2583_chip *chip;
.of_match_table = tsl2583_of_match,
},
.id_table = tsl2583_idtable,
- .probe = tsl2583_probe,
+ .probe_new = tsl2583_probe,
.remove = tsl2583_remove,
};
module_i2c_driver(tsl2583_driver);
},
};
-static int tsl2772_probe(struct i2c_client *clientp,
- const struct i2c_device_id *id)
+static int tsl2772_probe(struct i2c_client *clientp)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(clientp);
struct iio_dev *indio_dev;
struct tsl2772_chip *chip;
int ret;
.pm = &tsl2772_pm_ops,
},
.id_table = tsl2772_idtable,
- .probe = tsl2772_probe,
+ .probe_new = tsl2772_probe,
};
module_i2c_driver(tsl2772_driver);
}
}
-static int tsl4531_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tsl4531_probe(struct i2c_client *client)
{
struct tsl4531_data *data;
struct iio_dev *indio_dev;
.name = TSL4531_DRV_NAME,
.pm = pm_sleep_ptr(&tsl4531_pm_ops),
},
- .probe = tsl4531_probe,
+ .probe_new = tsl4531_probe,
.remove = tsl4531_remove,
.id_table = tsl4531_id,
};
return IRQ_HANDLED;
}
-static int us5182d_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int us5182d_probe(struct i2c_client *client)
{
struct us5182d_data *data;
struct iio_dev *indio_dev;
.of_match_table = us5182d_of_match,
.acpi_match_table = ACPI_PTR(us5182d_acpi_match),
},
- .probe = us5182d_probe,
+ .probe_new = us5182d_probe,
.remove = us5182d_remove,
.id_table = us5182d_id,
* interrupts (VCNL4040, VCNL4200)
*/
+#include <linux/bitfield.h>
#include <linux/module.h>
#include <linux/i2c.h>
#include <linux/err.h>
#define VCNL4000_PROX_EN BIT(1) /* start proximity measurement */
#define VCNL4000_SELF_TIMED_EN BIT(0) /* start self-timed measurement */
+#define VCNL4040_ALS_CONF_ALS_SHUTDOWN BIT(0)
+#define VCNL4040_PS_CONF1_PS_SHUTDOWN BIT(0)
+#define VCNL4040_PS_CONF2_PS_IT GENMASK(3, 1) /* Proximity integration time */
+
/* Bit masks for interrupt registers. */
#define VCNL4010_INT_THR_SEL BIT(0) /* Select threshold interrupt source */
#define VCNL4010_INT_THR_EN BIT(1) /* Threshold interrupt type */
{250, 0},
};
+static const int vcnl4040_ps_it_times[][2] = {
+ {0, 100},
+ {0, 150},
+ {0, 200},
+ {0, 250},
+ {0, 300},
+ {0, 350},
+ {0, 400},
+ {0, 800},
+};
+
#define VCNL4000_SLEEP_DELAY_MS 2000 /* before we enter pm_runtime_suspend */
enum vcnl4000_device_ids {
return data->chip_spec->set_power_state(data, true);
};
+static ssize_t vcnl4000_write_als_enable(struct vcnl4000_data *data, bool en)
+{
+ int ret;
+
+ mutex_lock(&data->vcnl4000_lock);
+
+ ret = i2c_smbus_read_word_data(data->client, VCNL4200_AL_CONF);
+ if (ret < 0)
+ goto out;
+
+ if (en)
+ ret &= ~VCNL4040_ALS_CONF_ALS_SHUTDOWN;
+ else
+ ret |= VCNL4040_ALS_CONF_ALS_SHUTDOWN;
+
+ ret = i2c_smbus_write_word_data(data->client, VCNL4200_AL_CONF, ret);
+
+out:
+ mutex_unlock(&data->vcnl4000_lock);
+
+ return ret;
+}
+
+static ssize_t vcnl4000_write_ps_enable(struct vcnl4000_data *data, bool en)
+{
+ int ret;
+
+ mutex_lock(&data->vcnl4000_lock);
+
+ ret = i2c_smbus_read_word_data(data->client, VCNL4200_PS_CONF1);
+ if (ret < 0)
+ goto out;
+
+ if (en)
+ ret &= ~VCNL4040_PS_CONF1_PS_SHUTDOWN;
+ else
+ ret |= VCNL4040_PS_CONF1_PS_SHUTDOWN;
+
+ ret = i2c_smbus_write_word_data(data->client, VCNL4200_PS_CONF1, ret);
+
+out:
+ mutex_unlock(&data->vcnl4000_lock);
+
+ return ret;
+}
+
static int vcnl4200_set_power_state(struct vcnl4000_data *data, bool on)
{
- u16 val = on ? 0 /* power on */ : 1 /* shut down */;
int ret;
- ret = i2c_smbus_write_word_data(data->client, VCNL4200_AL_CONF, val);
+ ret = vcnl4000_write_als_enable(data, on);
if (ret < 0)
return ret;
- ret = i2c_smbus_write_word_data(data->client, VCNL4200_PS_CONF1, val);
+ ret = vcnl4000_write_ps_enable(data, on);
if (ret < 0)
return ret;
return ret;
}
+static int vcnl4040_read_ps_it(struct vcnl4000_data *data, int *val, int *val2)
+{
+ int ret;
+
+ ret = i2c_smbus_read_word_data(data->client, VCNL4200_PS_CONF1);
+ if (ret < 0)
+ return ret;
+
+ ret = FIELD_GET(VCNL4040_PS_CONF2_PS_IT, ret);
+
+ if (ret >= ARRAY_SIZE(vcnl4040_ps_it_times))
+ return -EINVAL;
+
+ *val = vcnl4040_ps_it_times[ret][0];
+ *val2 = vcnl4040_ps_it_times[ret][1];
+
+ return 0;
+}
+
+static ssize_t vcnl4040_write_ps_it(struct vcnl4000_data *data, int val)
+{
+ unsigned int i;
+ int ret, index = -1;
+ u16 regval;
+
+ for (i = 0; i < ARRAY_SIZE(vcnl4040_ps_it_times); i++) {
+ if (val == vcnl4040_ps_it_times[i][1]) {
+ index = i;
+ break;
+ }
+ }
+
+ if (index < 0)
+ return -EINVAL;
+
+ mutex_lock(&data->vcnl4000_lock);
+
+ ret = i2c_smbus_read_word_data(data->client, VCNL4200_PS_CONF1);
+ if (ret < 0)
+ goto out;
+
+ regval = (ret & ~VCNL4040_PS_CONF2_PS_IT) |
+ FIELD_PREP(VCNL4040_PS_CONF2_PS_IT, index);
+ ret = i2c_smbus_write_word_data(data->client, VCNL4200_PS_CONF1,
+ regval);
+
+out:
+ mutex_unlock(&data->vcnl4000_lock);
+ return ret;
+}
+
static int vcnl4000_read_raw(struct iio_dev *indio_dev,
struct iio_chan_spec const *chan,
int *val, int *val2, long mask)
*val = 0;
*val2 = data->al_scale;
return IIO_VAL_INT_PLUS_MICRO;
+ case IIO_CHAN_INFO_INT_TIME:
+ if (chan->type != IIO_PROXIMITY)
+ return -EINVAL;
+ ret = vcnl4040_read_ps_it(data, val, val2);
+ if (ret < 0)
+ return ret;
+ return IIO_VAL_INT_PLUS_MICRO;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int vcnl4040_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ struct vcnl4000_data *data = iio_priv(indio_dev);
+
+ switch (mask) {
+ case IIO_CHAN_INFO_INT_TIME:
+ if (val != 0)
+ return -EINVAL;
+ if (chan->type != IIO_PROXIMITY)
+ return -EINVAL;
+ return vcnl4040_write_ps_it(data, val2);
+ default:
+ return -EINVAL;
+ }
+}
+
+static int vcnl4040_read_avail(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ const int **vals, int *type, int *length,
+ long mask)
+{
+ switch (mask) {
+ case IIO_CHAN_INFO_INT_TIME:
+ *vals = (int *)vcnl4040_ps_it_times;
+ *type = IIO_VAL_INT_PLUS_MICRO;
+ *length = 2 * ARRAY_SIZE(vcnl4040_ps_it_times);
+ return IIO_AVAIL_LIST;
default:
return -EINVAL;
}
IIO_CHAN_SOFT_TIMESTAMP(1),
};
+static const struct iio_chan_spec vcnl4040_channels[] = {
+ {
+ .type = IIO_LIGHT,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_SCALE),
+ }, {
+ .type = IIO_PROXIMITY,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_INT_TIME),
+ .info_mask_separate_available = BIT(IIO_CHAN_INFO_INT_TIME),
+ .ext_info = vcnl4000_ext_info,
+ }
+};
+
static const struct iio_info vcnl4000_info = {
.read_raw = vcnl4000_read_raw,
};
.write_event_config = vcnl4010_write_event_config,
};
+static const struct iio_info vcnl4040_info = {
+ .read_raw = vcnl4000_read_raw,
+ .write_raw = vcnl4040_write_raw,
+ .read_avail = vcnl4040_read_avail,
+};
+
static const struct vcnl4000_chip_spec vcnl4000_chip_spec_cfg[] = {
[VCNL4000] = {
.prod = "VCNL4000",
.measure_light = vcnl4200_measure_light,
.measure_proximity = vcnl4200_measure_proximity,
.set_power_state = vcnl4200_set_power_state,
- .channels = vcnl4000_channels,
- .num_channels = ARRAY_SIZE(vcnl4000_channels),
- .info = &vcnl4000_info,
+ .channels = vcnl4040_channels,
+ .num_channels = ARRAY_SIZE(vcnl4040_channels),
+ .info = &vcnl4040_info,
.irq_support = false,
},
[VCNL4200] = {
return devm_iio_trigger_register(&client->dev, trigger);
}
-static int vcnl4000_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int vcnl4000_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct vcnl4000_data *data;
struct iio_dev *indio_dev;
int ret;
.pm = pm_ptr(&vcnl4000_pm_ops),
.of_match_table = vcnl_4000_of_match,
},
- .probe = vcnl4000_probe,
+ .probe_new = vcnl4000_probe,
.id_table = vcnl4000_id,
.remove = vcnl4000_remove,
};
return ret;
}
-static int vcnl4035_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int vcnl4035_probe(struct i2c_client *client)
{
struct vcnl4035_data *data;
struct iio_dev *indio_dev;
.pm = pm_ptr(&vcnl4035_pm_ops),
.of_match_table = vcnl4035_of_match,
},
- .probe = vcnl4035_probe,
+ .probe_new = vcnl4035_probe,
.remove = vcnl4035_remove,
.id_table = vcnl4035_id,
};
return ret;
}
-static int veml6030_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int veml6030_probe(struct i2c_client *client)
{
int ret;
struct veml6030_data *data;
.of_match_table = veml6030_of_match,
.pm = pm_ptr(&veml6030_pm_ops),
},
- .probe = veml6030_probe,
+ .probe_new = veml6030_probe,
.id_table = veml6030_id,
};
module_i2c_driver(veml6030_driver);
.read_raw = veml6070_read_raw,
};
-static int veml6070_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int veml6070_probe(struct i2c_client *client)
{
struct veml6070_data *data;
struct iio_dev *indio_dev;
.driver = {
.name = VEML6070_DRV_NAME,
},
- .probe = veml6070_probe,
+ .probe_new = veml6070_probe,
.remove = veml6070_remove,
.id_table = veml6070_id,
};
return vl6180_hold(data, false);
}
-static int vl6180_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int vl6180_probe(struct i2c_client *client)
{
struct vl6180_data *data;
struct iio_dev *indio_dev;
.name = VL6180_DRV_NAME,
.of_match_table = vl6180_of_match,
},
- .probe = vl6180_probe,
+ .probe_new = vl6180_probe,
.id_table = vl6180_id,
};
.attrs = &zopt2201_attribute_group,
};
-static int zopt2201_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int zopt2201_probe(struct i2c_client *client)
{
struct zopt2201_data *data;
struct iio_dev *indio_dev;
.driver = {
.name = ZOPT2201_DRV_NAME,
},
- .probe = zopt2201_probe,
+ .probe_new = zopt2201_probe,
.id_table = zopt2201_id,
};
.precious_reg = ak8974_precious_reg,
};
-static int ak8974_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int ak8974_probe(struct i2c_client *i2c)
{
struct iio_dev *indio_dev;
struct ak8974 *ak8974;
.pm = pm_ptr(&ak8974_dev_pm_ops),
.of_match_table = ak8974_of_match,
},
- .probe = ak8974_probe,
+ .probe_new = ak8974_probe,
.remove = ak8974_remove,
.id_table = ak8974_id,
};
return IRQ_HANDLED;
}
-static int ak8975_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ak8975_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct ak8975_data *data;
struct iio_dev *indio_dev;
struct gpio_desc *eoc_gpiod;
.of_match_table = ak8975_of_match,
.acpi_match_table = ak_acpi_match,
},
- .probe = ak8975_probe,
+ .probe_new = ak8975_probe,
.remove = ak8975_remove,
.id_table = ak8975_id,
};
#include "bmc150_magn.h"
-static int bmc150_magn_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bmc150_magn_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct regmap *regmap;
const char *name = NULL;
.acpi_match_table = ACPI_PTR(bmc150_magn_acpi_match),
.pm = &bmc150_magn_pm_ops,
},
- .probe = bmc150_magn_i2c_probe,
+ .probe_new = bmc150_magn_i2c_probe,
.remove = bmc150_magn_i2c_remove,
.id_table = bmc150_magn_i2c_id,
};
.cache_type = REGCACHE_RBTREE,
};
-static int hmc5843_i2c_probe(struct i2c_client *cli,
- const struct i2c_device_id *id)
+static int hmc5843_i2c_probe(struct i2c_client *cli)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(cli);
struct regmap *regmap = devm_regmap_init_i2c(cli,
&hmc5843_i2c_regmap_config);
if (IS_ERR(regmap))
.of_match_table = hmc5843_of_match,
},
.id_table = hmc5843_id,
- .probe = hmc5843_i2c_probe,
+ .probe_new = hmc5843_i2c_probe,
.remove = hmc5843_i2c_remove,
};
module_i2c_driver(hmc5843_driver);
static const unsigned long mag3110_scan_masks[] = {0x7, 0xf, 0};
-static int mag3110_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mag3110_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct mag3110_data *data;
struct iio_dev *indio_dev;
int ret;
.of_match_table = mag3110_of_match,
.pm = pm_sleep_ptr(&mag3110_pm_ops),
},
- .probe = mag3110_probe,
+ .probe_new = mag3110_probe,
.remove = mag3110_remove,
.id_table = mag3110_id,
};
.num_reg_defaults = ARRAY_SIZE(mmc35240_reg_defaults),
};
-static int mmc35240_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mmc35240_probe(struct i2c_client *client)
{
struct mmc35240_data *data;
struct iio_dev *indio_dev;
.pm = pm_sleep_ptr(&mmc35240_pm_ops),
.acpi_match_table = ACPI_PTR(mmc35240_acpi_match),
},
- .probe = mmc35240_probe,
+ .probe_new = mmc35240_probe,
.id_table = mmc35240_id,
};
};
MODULE_DEVICE_TABLE(of, st_magn_of_match);
-static int st_magn_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int st_magn_i2c_probe(struct i2c_client *client)
{
const struct st_sensor_settings *settings;
struct st_sensor_data *mdata;
.name = "st-magn-i2c",
.of_match_table = st_magn_of_match,
},
- .probe = st_magn_i2c_probe,
+ .probe_new = st_magn_i2c_probe,
.id_table = st_magn_id_table,
};
module_i2c_driver(st_magn_driver);
},
};
-static int yas5xx_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int yas5xx_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct iio_dev *indio_dev;
struct device *dev = &i2c->dev;
struct yas5xx *yas5xx;
.of_match_table = yas5xx_of_match,
.pm = pm_ptr(&yas5xx_dev_pm_ops),
},
- .probe = yas5xx_probe,
+ .probe_new = yas5xx_probe,
.remove = yas5xx_remove,
.id_table = yas5xx_id,
};
}
mux->control = devm_mux_control_get(dev, NULL);
- if (IS_ERR(mux->control)) {
- if (PTR_ERR(mux->control) != -EPROBE_DEFER)
- dev_err(dev, "failed to get control-mux\n");
- return PTR_ERR(mux->control);
- }
+ if (IS_ERR(mux->control))
+ return dev_err_probe(dev, PTR_ERR(mux->control),
+ "failed to get control-mux\n");
i = 0;
for (state = 0; state < all_children; state++) {
return 0;
}
-static int ad5272_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ad5272_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct iio_dev *indio_dev;
struct ad5272_data *data;
.name = "ad5272",
.of_match_table = ad5272_dt_ids,
},
- .probe = ad5272_probe,
+ .probe_new = ad5272_probe,
.id_table = ad5272_id,
};
.read_avail = ds1803_read_avail,
};
-static int ds1803_probe(struct i2c_client *client, const struct i2c_device_id *id)
+static int ds1803_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct ds1803_data *data;
struct iio_dev *indio_dev;
.name = "ds1803",
.of_match_table = ds1803_dt_ids,
},
- .probe = ds1803_probe,
+ .probe_new = ds1803_probe,
.id_table = ds1803_id,
};
.write_raw = max5432_write_raw,
};
-static int max5432_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max5432_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
struct iio_dev *indio_dev;
.name = "max5432",
.of_match_table = max5432_dt_ids,
},
- .probe = max5432_probe,
+ .probe_new = max5432_probe,
};
module_i2c_driver(max5432_driver);
.write_raw = tpl0102_write_raw,
};
-static int tpl0102_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tpl0102_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct tpl0102_data *data;
struct iio_dev *indio_dev;
.driver = {
.name = "tpl0102",
},
- .probe = tpl0102_probe,
+ .probe_new = tpl0102_probe,
.id_table = tpl0102_id,
};
.predisable = lmp91000_buffer_predisable,
};
-static int lmp91000_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int lmp91000_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
struct lmp91000_data *data;
.name = LMP91000_DRV_NAME,
.of_match_table = lmp91000_of_match,
},
- .probe = lmp91000_probe,
+ .probe_new = lmp91000_probe,
.remove = lmp91000_remove,
.id_table = lmp91000_id,
};
state->offset -= ABP060MG_NUM_COUNTS >> 1;
}
-static int abp060mg_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int abp060mg_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct iio_dev *indio_dev;
struct abp_state *state;
unsigned long cfg_id = id->driver_data;
.driver = {
.name = "abp060mg",
},
- .probe = abp060mg_probe,
+ .probe_new = abp060mg_probe,
.id_table = abp060mg_id_table,
};
module_i2c_driver(abp060mg_driver);
#include "bmp280.h"
-static int bmp280_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bmp280_i2c_probe(struct i2c_client *client)
{
struct regmap *regmap;
const struct regmap_config *regmap_config;
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
switch (id->driver_data) {
case BMP180_CHIP_ID:
.of_match_table = bmp280_of_i2c_match,
.pm = pm_ptr(&bmp280_dev_pm_ops),
},
- .probe = bmp280_i2c_probe,
+ .probe_new = bmp280_i2c_probe,
.id_table = bmp280_i2c_id,
};
module_i2c_driver(bmp280_i2c_driver);
return IRQ_HANDLED;
};
-static int dlh_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int dlh_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct dlh_state *st;
struct iio_dev *indio_dev;
int ret;
.name = "dlhl60d",
.of_match_table = dlh_of_match,
},
- .probe = dlh_probe,
+ .probe_new = dlh_probe,
.id_table = dlh_id,
};
module_i2c_driver(dlh_driver);
.write_raw = dps310_write_raw,
};
-static int dps310_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int dps310_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct dps310_data *data;
struct iio_dev *iio;
int rc;
.name = DPS310_DEV_NAME,
.acpi_match_table = dps310_acpi_match,
},
- .probe = dps310_probe,
+ .probe_new = dps310_probe,
.id_table = dps310_id,
};
module_i2c_driver(dps310_driver);
.read_raw = &hp03_read_raw,
};
-static int hp03_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int hp03_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct iio_dev *indio_dev;
struct hp03_priv *priv;
.name = "hp03",
.of_match_table = hp03_of_match,
},
- .probe = hp03_probe,
+ .probe_new = hp03_probe,
.id_table = hp03_id,
};
module_i2c_driver(hp03_driver);
.write_raw = hp206c_write_raw,
};
-static int hp206c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int hp206c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct iio_dev *indio_dev;
struct hp206c_data *data;
int ret;
#endif
static struct i2c_driver hp206c_driver = {
- .probe = hp206c_probe,
+ .probe_new = hp206c_probe,
.id_table = hp206c_id,
.driver = {
.name = "hp206c",
pm_runtime_disable(dev);
}
-static int icp10100_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int icp10100_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct icp10100_state *st;
.pm = pm_ptr(&icp10100_pm),
.of_match_table = icp10100_of_match,
},
- .probe = icp10100_probe,
+ .probe_new = icp10100_probe,
.id_table = icp10100_id,
};
module_i2c_driver(icp10100_driver);
*
* Copyright (c) 2014 Peter Meerwald <pmeerw@pmeerw.net>
*
- * TODO: shutdown pin
+ * TODO: synchronization with system suspend
*/
#include <linux/module.h>
#include <linux/iio/iio.h>
#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
#include "mpl115.h"
s16 a0;
s16 b1, b2;
s16 c12;
+ struct gpio_desc *shutdown;
const struct mpl115_ops *ops;
};
switch (mask) {
case IIO_CHAN_INFO_PROCESSED:
+ pm_runtime_get_sync(data->dev);
ret = mpl115_comp_pressure(data, val, val2);
if (ret < 0)
return ret;
+ pm_runtime_mark_last_busy(data->dev);
+ pm_runtime_put_autosuspend(data->dev);
+
return IIO_VAL_INT_PLUS_MICRO;
case IIO_CHAN_INFO_RAW:
+ pm_runtime_get_sync(data->dev);
/* temperature -5.35 C / LSB, 472 LSB is 25 C */
ret = mpl115_read_temp(data);
if (ret < 0)
return ret;
+ pm_runtime_mark_last_busy(data->dev);
+ pm_runtime_put_autosuspend(data->dev);
*val = ret >> 6;
+
return IIO_VAL_INT;
case IIO_CHAN_INFO_OFFSET:
*val = -605;
if (ret)
return ret;
+ dev_set_drvdata(dev, indio_dev);
+
ret = data->ops->read(data->dev, MPL115_A0);
if (ret < 0)
return ret;
return ret;
data->c12 = ret;
+ data->shutdown = devm_gpiod_get_optional(dev, "shutdown",
+ GPIOD_OUT_LOW);
+ if (IS_ERR(data->shutdown))
+ return dev_err_probe(dev, PTR_ERR(data->shutdown),
+ "cannot get shutdown gpio\n");
+
+ if (data->shutdown) {
+ /* Enable runtime PM */
+ pm_runtime_get_noresume(dev);
+ pm_runtime_set_active(dev);
+ pm_runtime_enable(dev);
+
+ /*
+ * As the device takes 3 ms to come up with a fresh
+ * reading after power-on and 5 ms to actually power-on,
+ * do not shut it down unnecessarily. Set autosuspend to
+ * 2000 ms.
+ */
+ pm_runtime_set_autosuspend_delay(dev, 2000);
+ pm_runtime_use_autosuspend(dev);
+ pm_runtime_put(dev);
+
+ dev_dbg(dev, "low-power mode enabled");
+ } else
+ dev_dbg(dev, "low-power mode disabled");
+
return devm_iio_device_register(dev, indio_dev);
}
EXPORT_SYMBOL_NS_GPL(mpl115_probe, IIO_MPL115);
+static int mpl115_runtime_suspend(struct device *dev)
+{
+ struct mpl115_data *data = iio_priv(dev_get_drvdata(dev));
+
+ gpiod_set_value(data->shutdown, 1);
+
+ return 0;
+}
+
+static int mpl115_runtime_resume(struct device *dev)
+{
+ struct mpl115_data *data = iio_priv(dev_get_drvdata(dev));
+
+ gpiod_set_value(data->shutdown, 0);
+ usleep_range(5000, 6000);
+
+ return 0;
+}
+
+EXPORT_NS_RUNTIME_DEV_PM_OPS(mpl115_dev_pm_ops, mpl115_runtime_suspend,
+ mpl115_runtime_resume, NULL, IIO_MPL115);
+
MODULE_AUTHOR("Peter Meerwald <pmeerw@pmeerw.net>");
MODULE_DESCRIPTION("Freescale MPL115 pressure/temperature driver");
MODULE_LICENSE("GPL");
* Copyright (c) 2016 Akinobu Mita <akinobu.mita@gmail.com>
*/
+#include <linux/pm_runtime.h>
+
#ifndef _MPL115_H_
#define _MPL115_H_
int mpl115_probe(struct device *dev, const char *name,
const struct mpl115_ops *ops);
+/*PM ops */
+extern const struct dev_pm_ops mpl115_dev_pm_ops;
+
#endif
.write = mpl115_i2c_write,
};
-static int mpl115_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mpl115_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
if (!i2c_check_functionality(client->adapter, I2C_FUNC_SMBUS_WORD_DATA))
return -EOPNOTSUPP;
static struct i2c_driver mpl115_i2c_driver = {
.driver = {
.name = "mpl115",
+ .pm = pm_ptr(&mpl115_dev_pm_ops),
},
- .probe = mpl115_i2c_probe,
+ .probe_new = mpl115_i2c_probe,
.id_table = mpl115_i2c_id,
};
module_i2c_driver(mpl115_i2c_driver);
static struct spi_driver mpl115_spi_driver = {
.driver = {
.name = "mpl115",
+ .pm = pm_ptr(&mpl115_dev_pm_ops),
},
.probe = mpl115_spi_probe,
.id_table = mpl115_spi_ids,
.read_raw = &mpl3115_read_raw,
};
-static int mpl3115_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mpl3115_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct mpl3115_data *data;
struct iio_dev *indio_dev;
int ret;
.of_match_table = mpl3115_of_match,
.pm = pm_sleep_ptr(&mpl3115_pm_ops),
},
- .probe = mpl3115_probe,
+ .probe_new = mpl3115_probe,
.remove = mpl3115_remove,
.id_table = mpl3115_id,
};
return ms5611_i2c_read_adc(st, pressure);
}
-static int ms5611_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ms5611_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct ms5611_state *st;
struct iio_dev *indio_dev;
.of_match_table = ms5611_i2c_matches,
},
.id_table = ms5611_id,
- .probe = ms5611_i2c_probe,
+ .probe_new = ms5611_i2c_probe,
.remove = ms5611_i2c_remove,
};
module_i2c_driver(ms5611_driver);
.attrs = &ms5637_attribute_group,
};
-static int ms5637_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ms5637_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
const struct ms_tp_data *data;
struct ms_tp_dev *dev_data;
struct iio_dev *indio_dev;
MODULE_DEVICE_TABLE(of, ms5637_of_match);
static struct i2c_driver ms5637_driver = {
- .probe = ms5637_probe,
+ .probe_new = ms5637_probe,
.id_table = ms5637_id,
.driver = {
.name = "ms5637",
};
MODULE_DEVICE_TABLE(i2c, st_press_id_table);
-static int st_press_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int st_press_i2c_probe(struct i2c_client *client)
{
const struct st_sensor_settings *settings;
struct st_sensor_data *press_data;
.of_match_table = st_press_of_match,
.acpi_match_table = ACPI_PTR(st_press_acpi_match),
},
- .probe = st_press_i2c_probe,
+ .probe_new = st_press_i2c_probe,
.id_table = st_press_id_table,
};
module_i2c_driver(st_press_driver);
.attrs = &t5403_attribute_group,
};
-static int t5403_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int t5403_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct t5403_data *data;
struct iio_dev *indio_dev;
int ret;
.driver = {
.name = "t5403",
},
- .probe = t5403_probe,
+ .probe_new = t5403_probe,
.id_table = t5403_id,
};
module_i2c_driver(t5403_driver);
(ZPA2326_SA0(client->addr) << ZPA2326_DEVICE_ID_SA0_SHIFT));
}
-static int zpa2326_probe_i2c(struct i2c_client *client,
- const struct i2c_device_id *i2c_id)
+static int zpa2326_probe_i2c(struct i2c_client *client)
{
+ const struct i2c_device_id *i2c_id = i2c_client_get_device_id(client);
struct regmap *regmap;
regmap = devm_regmap_init_i2c(client, &zpa2326_regmap_i2c_config);
.of_match_table = zpa2326_i2c_matches,
.pm = ZPA2326_PM_OPS,
},
- .probe = zpa2326_probe_i2c,
+ .probe_new = zpa2326_probe_i2c,
.remove = zpa2326_remove_i2c,
.id_table = zpa2326_i2c_ids,
};
return IRQ_HANDLED;
}
-static int isl29501_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int isl29501_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct isl29501_private *isl29501;
.name = "isl29501",
},
.id_table = isl29501_id,
- .probe = isl29501_probe,
+ .probe_new = isl29501_probe,
};
module_i2c_driver(isl29501_driver);
.read_raw = mb1232_read_raw,
};
-static int mb1232_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mb1232_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct iio_dev *indio_dev;
struct mb1232_data *data;
int ret;
.name = "maxbotix-mb1232",
.of_match_table = of_mb1232_match,
},
- .probe = mb1232_probe,
+ .probe_new = mb1232_probe,
.id_table = mb1232_id,
};
module_i2c_driver(mb1232_driver);
.read_raw = lidar_read_raw,
};
-static int lidar_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int lidar_probe(struct i2c_client *client)
{
struct lidar_data *data;
struct iio_dev *indio_dev;
.of_match_table = lidar_dt_ids,
.pm = pm_ptr(&lidar_pm_ops),
},
- .probe = lidar_probe,
+ .probe_new = lidar_probe,
.remove = lidar_remove,
.id_table = lidar_id,
};
rfd77402_powerdown(client);
}
-static int rfd77402_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int rfd77402_probe(struct i2c_client *client)
{
struct rfd77402_data *data;
struct iio_dev *indio_dev;
.name = RFD77402_DRV_NAME,
.pm = pm_sleep_ptr(&rfd77402_pm_ops),
},
- .probe = rfd77402_probe,
+ .probe_new = rfd77402_probe,
.id_table = rfd77402_id,
};
.read_raw = srf08_read_raw,
};
-static int srf08_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int srf08_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct iio_dev *indio_dev;
struct srf08_data *data;
int ret;
.name = "srf08",
.of_match_table = of_srf08_match,
},
- .probe = srf08_probe,
+ .probe_new = srf08_probe,
.id_table = srf08_id,
};
module_i2c_driver(srf08_driver);
static const struct acpi_device_id sx9360_acpi_match[] = {
{ "STH9360", SX9360_WHOAMI_VALUE },
+ { "SAMM0208", SX9360_WHOAMI_VALUE },
{ }
};
MODULE_DEVICE_TABLE(acpi, sx9360_acpi_match);
}
}
-static int sx9500_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int sx9500_probe(struct i2c_client *client)
{
int ret;
struct iio_dev *indio_dev;
.of_match_table = of_match_ptr(sx9500_of_match),
.pm = pm_sleep_ptr(&sx9500_pm_ops),
},
- .probe = sx9500_probe,
+ .probe_new = sx9500_probe,
.remove = sx9500_remove,
.id_table = sx9500_id,
};
.postdisable = sx_common_buffer_postdisable,
};
-static void sx_common_regulator_disable(void *_data)
-{
- struct sx_common_data *data = _data;
-
- regulator_bulk_disable(ARRAY_SIZE(data->supplies), data->supplies);
-}
-
#define SX_COMMON_SOFT_RESET 0xde
static int sx_common_init_device(struct device *dev, struct iio_dev *indio_dev)
const struct sx_common_chip_info *chip_info,
const struct regmap_config *regmap_config)
{
+ static const char * const regulator_names[] = { "vdd", "svdd" };
struct device *dev = &client->dev;
struct iio_dev *indio_dev;
struct sx_common_data *data;
data->chip_info = chip_info;
data->client = client;
- data->supplies[0].supply = "vdd";
- data->supplies[1].supply = "svdd";
mutex_init(&data->mutex);
init_completion(&data->completion);
return dev_err_probe(dev, PTR_ERR(data->regmap),
"Could init register map\n");
- ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(data->supplies),
- data->supplies);
+ ret = devm_regulator_bulk_get_enable(dev, ARRAY_SIZE(regulator_names),
+ regulator_names);
if (ret)
return dev_err_probe(dev, ret, "Unable to get regulators\n");
- ret = regulator_bulk_enable(ARRAY_SIZE(data->supplies), data->supplies);
- if (ret)
- return dev_err_probe(dev, ret, "Unable to enable regulators\n");
-
/* Must wait for Tpor time after initial power up */
usleep_range(1000, 1100);
- ret = devm_add_action_or_reset(dev, sx_common_regulator_disable, data);
- if (ret)
- return dev_err_probe(dev, ret,
- "Unable to register regulators deleter\n");
-
ret = data->chip_info->ops.check_whoami(dev, indio_dev);
if (ret)
return dev_err_probe(dev, ret, "error reading WHOAMI\n");
* @trig: IIO trigger object.
* @regmap: Register map.
* @num_default_regs: Number of default registers to set at init.
- * @supplies: Power supplies object.
* @chan_prox_stat: Last reading of the proximity status for each channel.
* We only send an event to user space when this changes.
* @trigger_enabled: True when the device trigger is enabled.
struct iio_trigger *trig;
struct regmap *regmap;
- struct regulator_bulk_data supplies[2];
unsigned long chan_prox_stat;
bool trigger_enabled;
This driver can also be built as a module. If so, the module will
be called tsys02d.
+config MAX30208
+ tristate "Maxim MAX30208 digital temperature sensor"
+ depends on I2C
+ help
+ If you say yes here you get support for Maxim MAX30208
+ digital temperature sensor connected via I2C.
+
+ This driver can also be built as a module. If so, the module
+ will be called max30208.
+
config MAX31856
tristate "MAX31856 thermocouple sensor"
depends on SPI
obj-$(CONFIG_LTC2983) += ltc2983.o
obj-$(CONFIG_HID_SENSOR_TEMP) += hid-sensor-temperature.o
obj-$(CONFIG_MAXIM_THERMOCOUPLE) += maxim_thermocouple.o
+obj-$(CONFIG_MAX30208) += max30208.o
obj-$(CONFIG_MAX31856) += max31856.o
obj-$(CONFIG_MAX31865) += max31865.o
obj-$(CONFIG_MLX90614) += mlx90614.o
#define LTC2983_STATUS_REG 0x0000
#define LTC2983_TEMP_RES_START_REG 0x0010
#define LTC2983_TEMP_RES_END_REG 0x005F
+#define LTC2983_EEPROM_KEY_REG 0x00B0
+#define LTC2983_EEPROM_READ_STATUS_REG 0x00D0
#define LTC2983_GLOBAL_CONFIG_REG 0x00F0
#define LTC2983_MULT_CHANNEL_START_REG 0x00F4
#define LTC2983_MULT_CHANNEL_END_REG 0x00F7
+#define LTC2986_EEPROM_STATUS_REG 0x00F9
#define LTC2983_MUX_CONFIG_REG 0x00FF
#define LTC2983_CHAN_ASSIGN_START_REG 0x0200
#define LTC2983_CHAN_ASSIGN_END_REG 0x024F
#define LTC2983_CUST_SENS_TBL_END_REG 0x03CF
#define LTC2983_DIFFERENTIAL_CHAN_MIN 2
-#define LTC2983_MAX_CHANNELS_NR 20
#define LTC2983_MIN_CHANNELS_NR 1
#define LTC2983_SLEEP 0x97
#define LTC2983_CUSTOM_STEINHART_SIZE 24
#define LTC2983_CUSTOM_SENSOR_ENTRY_SZ 6
#define LTC2983_CUSTOM_STEINHART_ENTRY_SZ 4
+#define LTC2983_EEPROM_KEY 0xA53C0F5A
+#define LTC2983_EEPROM_WRITE_CMD 0x15
+#define LTC2983_EEPROM_READ_CMD 0x16
+#define LTC2983_EEPROM_STATUS_FAILURE_MASK GENMASK(3, 1)
+#define LTC2983_EEPROM_READ_FAILURE_MASK GENMASK(7, 0)
+
+#define LTC2983_EEPROM_WRITE_TIME_MS 2600
+#define LTC2983_EEPROM_READ_TIME_MS 20
+
#define LTC2983_CHAN_START_ADDR(chan) \
(((chan - 1) * 4) + LTC2983_CHAN_ASSIGN_START_REG)
#define LTC2983_CHAN_RES_ADDR(chan) \
LTC2983_SENSOR_DIODE = 28,
LTC2983_SENSOR_SENSE_RESISTOR = 29,
LTC2983_SENSOR_DIRECT_ADC = 30,
+ LTC2983_SENSOR_ACTIVE_TEMP = 31,
};
#define to_thermocouple(_sensor) \
#define to_adc(_sensor) \
container_of(_sensor, struct ltc2983_adc, sensor)
+#define to_temp(_sensor) \
+ container_of(_sensor, struct ltc2983_temp, sensor)
+
+struct ltc2983_chip_info {
+ unsigned int max_channels_nr;
+ bool has_temp;
+ bool has_eeprom;
+};
+
struct ltc2983_data {
+ const struct ltc2983_chip_info *info;
struct regmap *regmap;
struct spi_device *spi;
struct mutex lock;
* Holds the converted temperature
*/
__be32 temp __aligned(IIO_DMA_MINALIGN);
+ __be32 chan_val;
+ __be32 eeprom_key;
};
struct ltc2983_sensor {
bool single_ended;
};
+struct ltc2983_temp {
+ struct ltc2983_sensor sensor;
+ struct ltc2983_custom_sensor *custom;
+ bool single_ended;
+};
+
/*
* Convert to Q format numbers. These number's are integers where
* the number of integer and fractional bits are specified. The resolution
return 0;
}
-static int __ltc2983_chan_assign_common(const struct ltc2983_data *st,
+static int __ltc2983_chan_assign_common(struct ltc2983_data *st,
const struct ltc2983_sensor *sensor,
u32 chan_val)
{
u32 reg = LTC2983_CHAN_START_ADDR(sensor->chan);
- __be32 __chan_val;
chan_val |= LTC2983_CHAN_TYPE(sensor->type);
dev_dbg(&st->spi->dev, "Assign reg:0x%04X, val:0x%08X\n", reg,
chan_val);
- __chan_val = cpu_to_be32(chan_val);
- return regmap_bulk_write(st->regmap, reg, &__chan_val,
- sizeof(__chan_val));
+ st->chan_val = cpu_to_be32(chan_val);
+ return regmap_bulk_write(st->regmap, reg, &st->chan_val,
+ sizeof(st->chan_val));
}
static int __ltc2983_chan_custom_sensor_assign(struct ltc2983_data *st,
return __ltc2983_chan_assign_common(st, sensor, chan_val);
}
+static int ltc2983_temp_assign_chan(struct ltc2983_data *st,
+ const struct ltc2983_sensor *sensor)
+{
+ struct ltc2983_temp *temp = to_temp(sensor);
+ u32 chan_val;
+ int ret;
+
+ chan_val = LTC2983_ADC_SINGLE_ENDED(temp->single_ended);
+
+ ret = __ltc2983_chan_custom_sensor_assign(st, temp->custom, &chan_val);
+ if (ret)
+ return ret;
+
+ return __ltc2983_chan_assign_common(st, sensor, chan_val);
+}
+
static struct ltc2983_sensor *
ltc2983_thermocouple_new(const struct fwnode_handle *child, struct ltc2983_data *st,
const struct ltc2983_sensor *sensor)
if (rtd->sensor_config & LTC2983_RTD_4_WIRE_MASK) {
/* 4-wire */
u8 min = LTC2983_DIFFERENTIAL_CHAN_MIN,
- max = LTC2983_MAX_CHANNELS_NR;
+ max = st->info->max_channels_nr;
if (rtd->sensor_config & LTC2983_RTD_ROTATION_MASK)
- max = LTC2983_MAX_CHANNELS_NR - 1;
+ max = st->info->max_channels_nr - 1;
if (((rtd->sensor_config & LTC2983_RTD_KELVIN_R_SENSE_MASK)
== LTC2983_RTD_KELVIN_R_SENSE_MASK) &&
return &adc->sensor;
}
+static struct ltc2983_sensor *ltc2983_temp_new(struct fwnode_handle *child,
+ struct ltc2983_data *st,
+ const struct ltc2983_sensor *sensor)
+{
+ struct ltc2983_temp *temp;
+
+ temp = devm_kzalloc(&st->spi->dev, sizeof(*temp), GFP_KERNEL);
+ if (!temp)
+ return ERR_PTR(-ENOMEM);
+
+ if (fwnode_property_read_bool(child, "adi,single-ended"))
+ temp->single_ended = true;
+
+ if (!temp->single_ended &&
+ sensor->chan < LTC2983_DIFFERENTIAL_CHAN_MIN) {
+ dev_err(&st->spi->dev, "Invalid chan:%d for differential temp\n",
+ sensor->chan);
+ return ERR_PTR(-EINVAL);
+ }
+
+ temp->custom = __ltc2983_custom_sensor_new(st, child, "adi,custom-temp",
+ false, 4096, true);
+ if (IS_ERR(temp->custom))
+ return ERR_CAST(temp->custom);
+
+ /* set common parameters */
+ temp->sensor.assign_chan = ltc2983_temp_assign_chan;
+ temp->sensor.fault_handler = ltc2983_common_fault_handler;
+
+ return &temp->sensor;
+}
+
static int ltc2983_chan_read(struct ltc2983_data *st,
const struct ltc2983_sensor *sensor, int *val)
{
/* check if we have a valid channel */
if (sensor.chan < LTC2983_MIN_CHANNELS_NR ||
- sensor.chan > LTC2983_MAX_CHANNELS_NR) {
+ sensor.chan > st->info->max_channels_nr) {
ret = -EINVAL;
dev_err(dev, "chan:%d must be from %u to %u\n", sensor.chan,
- LTC2983_MIN_CHANNELS_NR, LTC2983_MAX_CHANNELS_NR);
+ LTC2983_MIN_CHANNELS_NR, st->info->max_channels_nr);
goto put_child;
} else if (channel_avail_mask & BIT(sensor.chan)) {
ret = -EINVAL;
st->iio_channels--;
} else if (sensor.type == LTC2983_SENSOR_DIRECT_ADC) {
st->sensors[chan] = ltc2983_adc_new(child, st, &sensor);
+ } else if (st->info->has_temp &&
+ sensor.type == LTC2983_SENSOR_ACTIVE_TEMP) {
+ st->sensors[chan] = ltc2983_temp_new(child, st, &sensor);
} else {
dev_err(dev, "Unknown sensor type %d\n", sensor.type);
ret = -EINVAL;
return ret;
}
+static int ltc2983_eeprom_cmd(struct ltc2983_data *st, unsigned int cmd,
+ unsigned int wait_time, unsigned int status_reg,
+ unsigned long status_fail_mask)
+{
+ unsigned long time;
+ unsigned int val;
+ int ret;
+
+ ret = regmap_bulk_write(st->regmap, LTC2983_EEPROM_KEY_REG,
+ &st->eeprom_key, sizeof(st->eeprom_key));
+ if (ret)
+ return ret;
+
+ reinit_completion(&st->completion);
+
+ ret = regmap_write(st->regmap, LTC2983_STATUS_REG,
+ LTC2983_STATUS_START(true) | cmd);
+ if (ret)
+ return ret;
+
+ time = wait_for_completion_timeout(&st->completion,
+ msecs_to_jiffies(wait_time));
+ if (!time) {
+ dev_err(&st->spi->dev, "EEPROM command timed out\n");
+ return -ETIMEDOUT;
+ }
+
+ ret = regmap_read(st->regmap, status_reg, &val);
+ if (ret)
+ return ret;
+
+ if (val & status_fail_mask) {
+ dev_err(&st->spi->dev, "EEPROM command failed: 0x%02X\n", val);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int ltc2983_setup(struct ltc2983_data *st, bool assign_iio)
{
u32 iio_chan_t = 0, iio_chan_v = 0, chan, iio_idx = 0, status;
if (ret)
return ret;
+ if (st->info->has_eeprom && !assign_iio) {
+ ret = ltc2983_eeprom_cmd(st, LTC2983_EEPROM_READ_CMD,
+ LTC2983_EEPROM_READ_TIME_MS,
+ LTC2983_EEPROM_READ_STATUS_REG,
+ LTC2983_EEPROM_READ_FAILURE_MASK);
+ if (!ret)
+ return 0;
+ }
+
for (chan = 0; chan < st->num_channels; chan++) {
u32 chan_type = 0, *iio_chan;
static const struct regmap_range ltc2983_reg_ranges[] = {
regmap_reg_range(LTC2983_STATUS_REG, LTC2983_STATUS_REG),
regmap_reg_range(LTC2983_TEMP_RES_START_REG, LTC2983_TEMP_RES_END_REG),
+ regmap_reg_range(LTC2983_EEPROM_KEY_REG, LTC2983_EEPROM_KEY_REG),
+ regmap_reg_range(LTC2983_EEPROM_READ_STATUS_REG,
+ LTC2983_EEPROM_READ_STATUS_REG),
regmap_reg_range(LTC2983_GLOBAL_CONFIG_REG, LTC2983_GLOBAL_CONFIG_REG),
regmap_reg_range(LTC2983_MULT_CHANNEL_START_REG,
LTC2983_MULT_CHANNEL_END_REG),
+ regmap_reg_range(LTC2986_EEPROM_STATUS_REG, LTC2986_EEPROM_STATUS_REG),
regmap_reg_range(LTC2983_MUX_CONFIG_REG, LTC2983_MUX_CONFIG_REG),
regmap_reg_range(LTC2983_CHAN_ASSIGN_START_REG,
LTC2983_CHAN_ASSIGN_END_REG),
st = iio_priv(indio_dev);
+ st->info = device_get_match_data(&spi->dev);
+ if (!st->info)
+ st->info = (void *)spi_get_device_id(spi)->driver_data;
+ if (!st->info)
+ return -ENODEV;
+
st->regmap = devm_regmap_init_spi(spi, <c2983_regmap_config);
if (IS_ERR(st->regmap)) {
dev_err(&spi->dev, "Failed to initialize regmap\n");
mutex_init(&st->lock);
init_completion(&st->completion);
st->spi = spi;
+ st->eeprom_key = cpu_to_be32(LTC2983_EEPROM_KEY);
spi_set_drvdata(spi, st);
ret = ltc2983_parse_dt(st);
return ret;
}
+ if (st->info->has_eeprom) {
+ ret = ltc2983_eeprom_cmd(st, LTC2983_EEPROM_WRITE_CMD,
+ LTC2983_EEPROM_WRITE_TIME_MS,
+ LTC2986_EEPROM_STATUS_REG,
+ LTC2983_EEPROM_STATUS_FAILURE_MASK);
+ if (ret)
+ return ret;
+ }
+
indio_dev->name = name;
indio_dev->num_channels = st->iio_channels;
indio_dev->channels = st->iio_chan;
static DEFINE_SIMPLE_DEV_PM_OPS(ltc2983_pm_ops, ltc2983_suspend,
ltc2983_resume);
+static const struct ltc2983_chip_info ltc2983_chip_info_data = {
+ .max_channels_nr = 20,
+};
+
+static const struct ltc2983_chip_info ltc2984_chip_info_data = {
+ .max_channels_nr = 20,
+ .has_eeprom = true,
+};
+
+static const struct ltc2983_chip_info ltc2986_chip_info_data = {
+ .max_channels_nr = 10,
+ .has_temp = true,
+ .has_eeprom = true,
+};
+
static const struct spi_device_id ltc2983_id_table[] = {
- { "ltc2983" },
+ { "ltc2983", (kernel_ulong_t)<c2983_chip_info_data },
+ { "ltc2984", (kernel_ulong_t)<c2984_chip_info_data },
+ { "ltc2986", (kernel_ulong_t)<c2986_chip_info_data },
+ { "ltm2985", (kernel_ulong_t)<c2986_chip_info_data },
{},
};
MODULE_DEVICE_TABLE(spi, ltc2983_id_table);
static const struct of_device_id ltc2983_of_match[] = {
- { .compatible = "adi,ltc2983" },
+ { .compatible = "adi,ltc2983", .data = <c2983_chip_info_data },
+ { .compatible = "adi,ltc2984", .data = <c2984_chip_info_data },
+ { .compatible = "adi,ltc2986", .data = <c2986_chip_info_data },
+ { .compatible = "adi,ltm2985", .data = <c2986_chip_info_data },
{},
};
MODULE_DEVICE_TABLE(of, ltc2983_of_match);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+
+/*
+ * Copyright (c) Rajat Khandelwal <rajat.khandelwal@linux.intel.com>
+ *
+ * Maxim MAX30208 digital temperature sensor with 0.1°C accuracy
+ * (7-bit I2C slave address (0x50 - 0x53))
+ */
+
+#include <linux/bitops.h>
+#include <linux/delay.h>
+#include <linux/iio/iio.h>
+#include <linux/i2c.h>
+#include <linux/module.h>
+#include <linux/types.h>
+
+#define MAX30208_STATUS 0x00
+#define MAX30208_STATUS_TEMP_RDY BIT(0)
+#define MAX30208_INT_ENABLE 0x01
+#define MAX30208_INT_ENABLE_TEMP_RDY BIT(0)
+
+#define MAX30208_FIFO_OVF_CNTR 0x06
+#define MAX30208_FIFO_DATA_CNTR 0x07
+#define MAX30208_FIFO_DATA 0x08
+
+#define MAX30208_FIFO_CONFIG 0x0a
+#define MAX30208_FIFO_CONFIG_RO BIT(1)
+
+#define MAX30208_SYSTEM_CTRL 0x0c
+#define MAX30208_SYSTEM_CTRL_RESET 0x01
+
+#define MAX30208_TEMP_SENSOR_SETUP 0x14
+#define MAX30208_TEMP_SENSOR_SETUP_CONV BIT(0)
+
+struct max30208_data {
+ struct i2c_client *client;
+ struct iio_dev *indio_dev;
+ struct mutex lock; /* Lock to prevent concurrent reads of temperature readings */
+};
+
+static const struct iio_chan_spec max30208_channels[] = {
+ {
+ .type = IIO_TEMP,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) | BIT(IIO_CHAN_INFO_SCALE),
+ },
+};
+
+/**
+ * max30208_request() - Request a reading
+ * @data: Struct comprising member elements of the device
+ *
+ * Requests a reading from the device and waits until the conversion is ready.
+ */
+static int max30208_request(struct max30208_data *data)
+{
+ /*
+ * Sensor can take up to 500 ms to respond so execute a total of
+ * 10 retries to give the device sufficient time.
+ */
+ int retries = 10;
+ u8 regval;
+ int ret;
+
+ ret = i2c_smbus_read_byte_data(data->client, MAX30208_TEMP_SENSOR_SETUP);
+ if (ret < 0)
+ return ret;
+
+ regval = ret | MAX30208_TEMP_SENSOR_SETUP_CONV;
+
+ ret = i2c_smbus_write_byte_data(data->client, MAX30208_TEMP_SENSOR_SETUP, regval);
+ if (ret)
+ return ret;
+
+ while (retries--) {
+ ret = i2c_smbus_read_byte_data(data->client, MAX30208_STATUS);
+ if (ret < 0)
+ return ret;
+
+ if (ret & MAX30208_STATUS_TEMP_RDY)
+ return 0;
+
+ msleep(50);
+ }
+ dev_err(&data->client->dev, "Temperature conversion failed\n");
+
+ return -ETIMEDOUT;
+}
+
+static int max30208_update_temp(struct max30208_data *data)
+{
+ u8 data_count;
+ int ret;
+
+ mutex_lock(&data->lock);
+
+ ret = max30208_request(data);
+ if (ret)
+ goto unlock;
+
+ ret = i2c_smbus_read_byte_data(data->client, MAX30208_FIFO_OVF_CNTR);
+ if (ret < 0)
+ goto unlock;
+ else if (!ret) {
+ ret = i2c_smbus_read_byte_data(data->client, MAX30208_FIFO_DATA_CNTR);
+ if (ret < 0)
+ goto unlock;
+
+ data_count = ret;
+ } else
+ data_count = 1;
+
+ while (data_count) {
+ ret = i2c_smbus_read_word_swapped(data->client, MAX30208_FIFO_DATA);
+ if (ret < 0)
+ goto unlock;
+
+ data_count--;
+ }
+
+unlock:
+ mutex_unlock(&data->lock);
+ return ret;
+}
+
+/**
+ * max30208_config_setup() - Set up FIFO configuration register
+ * @data: Struct comprising member elements of the device
+ *
+ * Sets the rollover bit to '1' to enable overwriting FIFO during overflow.
+ */
+static int max30208_config_setup(struct max30208_data *data)
+{
+ u8 regval;
+ int ret;
+
+ ret = i2c_smbus_read_byte_data(data->client, MAX30208_FIFO_CONFIG);
+ if (ret < 0)
+ return ret;
+
+ regval = ret | MAX30208_FIFO_CONFIG_RO;
+
+ ret = i2c_smbus_write_byte_data(data->client, MAX30208_FIFO_CONFIG, regval);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int max30208_read(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ struct max30208_data *data = iio_priv(indio_dev);
+ int ret;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ ret = max30208_update_temp(data);
+ if (ret < 0)
+ return ret;
+
+ *val = sign_extend32(ret, 15);
+ return IIO_VAL_INT;
+
+ case IIO_CHAN_INFO_SCALE:
+ *val = 5;
+ return IIO_VAL_INT;
+
+ default:
+ return -EINVAL;
+ }
+}
+
+static const struct iio_info max30208_info = {
+ .read_raw = max30208_read,
+};
+
+static int max30208_probe(struct i2c_client *i2c)
+{
+ struct device *dev = &i2c->dev;
+ struct max30208_data *data;
+ struct iio_dev *indio_dev;
+ int ret;
+
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*data));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ data = iio_priv(indio_dev);
+ data->client = i2c;
+ mutex_init(&data->lock);
+
+ indio_dev->name = "max30208";
+ indio_dev->channels = max30208_channels;
+ indio_dev->num_channels = ARRAY_SIZE(max30208_channels);
+ indio_dev->info = &max30208_info;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+
+ ret = i2c_smbus_write_byte_data(data->client, MAX30208_SYSTEM_CTRL,
+ MAX30208_SYSTEM_CTRL_RESET);
+ if (ret) {
+ dev_err(dev, "Failure in performing reset\n");
+ return ret;
+ }
+
+ msleep(50);
+
+ ret = max30208_config_setup(data);
+ if (ret)
+ return ret;
+
+ ret = devm_iio_device_register(dev, indio_dev);
+ if (ret) {
+ dev_err(dev, "Failed to register IIO device\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static const struct i2c_device_id max30208_id_table[] = {
+ { "max30208" },
+ { }
+};
+MODULE_DEVICE_TABLE(i2c, max30208_id_table);
+
+static const struct acpi_device_id max30208_acpi_match[] = {
+ { "MAX30208" },
+ { }
+};
+MODULE_DEVICE_TABLE(acpi, max30208_acpi_match);
+
+static const struct of_device_id max30208_of_match[] = {
+ { .compatible = "maxim,max30208" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, max30208_of_match);
+
+static struct i2c_driver max30208_driver = {
+ .driver = {
+ .name = "max30208",
+ .of_match_table = max30208_of_match,
+ .acpi_match_table = max30208_acpi_match,
+ },
+ .probe_new = max30208_probe,
+ .id_table = max30208_id_table,
+};
+module_i2c_driver(max30208_driver);
+
+MODULE_AUTHOR("Rajat Khandelwal <rajat.khandelwal@linux.intel.com>");
+MODULE_DESCRIPTION("Maxim MAX30208 digital temperature sensor");
+MODULE_LICENSE("GPL");
return (ret & MLX90614_CONFIG_DUAL_MASK) ? 1 : 0;
}
-static int mlx90614_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mlx90614_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct iio_dev *indio_dev;
struct mlx90614_data *data;
int ret;
.of_match_table = mlx90614_of_match,
.pm = pm_ptr(&mlx90614_pm_ops),
},
- .probe = mlx90614_probe,
+ .probe_new = mlx90614_probe,
.remove = mlx90614_remove,
.id_table = mlx90614_id,
};
*
* Driver for the Melexis MLX90632 I2C 16-bit IR thermopile sensor
*/
+#include <linux/bitfield.h>
#include <linux/delay.h>
+#include <linux/device.h>
#include <linux/err.h>
#include <linux/gpio/consumer.h>
#include <linux/i2c.h>
#include <linux/iopoll.h>
+#include <linux/jiffies.h>
#include <linux/kernel.h>
#include <linux/limits.h>
#include <linux/mod_devicetable.h>
#define MLX90632_EE_Ha 0x2481 /* Ha customer calib value reg 16bit */
#define MLX90632_EE_Hb 0x2482 /* Hb customer calib value reg 16bit */
+#define MLX90632_EE_MEDICAL_MEAS1 0x24E1 /* Medical measurement 1 16bit */
+#define MLX90632_EE_MEDICAL_MEAS2 0x24E2 /* Medical measurement 2 16bit */
+#define MLX90632_EE_EXTENDED_MEAS1 0x24F1 /* Extended measurement 1 16bit */
+#define MLX90632_EE_EXTENDED_MEAS2 0x24F2 /* Extended measurement 2 16bit */
+#define MLX90632_EE_EXTENDED_MEAS3 0x24F3 /* Extended measurement 3 16bit */
+
/* Register addresses - volatile */
#define MLX90632_REG_I2C_ADDR 0x3000 /* Chip I2C address register */
#define MLX90632_REG_CONTROL 0x3001 /* Control Register address */
#define MLX90632_CFG_PWR_MASK GENMASK(2, 1) /* PowerMode Mask */
#define MLX90632_CFG_MTYP_MASK GENMASK(8, 4) /* Meas select Mask */
+#define MLX90632_CFG_SOB_MASK BIT(11)
/* PowerModes statuses */
#define MLX90632_PWR_STATUS(ctrl_val) (ctrl_val << 1)
#define MLX90632_PWR_STATUS_HALT MLX90632_PWR_STATUS(0) /* hold */
-#define MLX90632_PWR_STATUS_SLEEP_STEP MLX90632_PWR_STATUS(1) /* sleep step*/
+#define MLX90632_PWR_STATUS_SLEEP_STEP MLX90632_PWR_STATUS(1) /* sleep step */
#define MLX90632_PWR_STATUS_STEP MLX90632_PWR_STATUS(2) /* step */
-#define MLX90632_PWR_STATUS_CONTINUOUS MLX90632_PWR_STATUS(3) /* continuous*/
+#define MLX90632_PWR_STATUS_CONTINUOUS MLX90632_PWR_STATUS(3) /* continuous */
+
+#define MLX90632_EE_RR GENMASK(10, 8) /* Only Refresh Rate bits */
+#define MLX90632_REFRESH_RATE(ee_val) FIELD_GET(MLX90632_EE_RR, ee_val)
+ /* Extract Refresh Rate from ee register */
+#define MLX90632_REFRESH_RATE_STATUS(refresh_rate) (refresh_rate << 8)
/* Measurement types */
#define MLX90632_MTYP_MEDICAL 0
#define MLX90632_REF_12 12LL /* ResCtrlRef value of Ch 1 or Ch 2 */
#define MLX90632_REF_3 12LL /* ResCtrlRef value of Channel 3 */
#define MLX90632_MAX_MEAS_NUM 31 /* Maximum measurements in list */
-#define MLX90632_SLEEP_DELAY_MS 3000 /* Autosleep delay */
+#define MLX90632_SLEEP_DELAY_MS 6000 /* Autosleep delay */
#define MLX90632_EXTENDED_LIMIT 27000 /* Extended mode raw value limit */
+#define MLX90632_MEAS_MAX_TIME 2000 /* Max measurement time in ms for the lowest refresh rate */
/**
* struct mlx90632_data - private data for the MLX90632 device
* @object_ambient_temperature: Ambient temperature at object (might differ of
* the ambient temperature of sensor.
* @regulator: Regulator of the device
+ * @powerstatus: Current POWER status of the device
+ * @interaction_ts: Timestamp of the last temperature read that is used
+ * for power management in jiffies
*/
struct mlx90632_data {
struct i2c_client *client;
u8 mtyp;
u32 object_ambient_temperature;
struct regulator *regulator;
+ int powerstatus;
+ unsigned long interaction_ts;
};
static const struct regmap_range mlx90632_volatile_reg_range[] = {
regmap_reg_range(MLX90632_EE_VERSION, MLX90632_EE_Ka),
regmap_reg_range(MLX90632_EE_CTRL, MLX90632_EE_I2C_ADDR),
regmap_reg_range(MLX90632_EE_Ha, MLX90632_EE_Hb),
+ regmap_reg_range(MLX90632_EE_MEDICAL_MEAS1, MLX90632_EE_MEDICAL_MEAS2),
+ regmap_reg_range(MLX90632_EE_EXTENDED_MEAS1, MLX90632_EE_EXTENDED_MEAS3),
regmap_reg_range(MLX90632_REG_I2C_ADDR, MLX90632_REG_CONTROL),
regmap_reg_range(MLX90632_REG_I2C_CMD, MLX90632_REG_I2C_CMD),
regmap_reg_range(MLX90632_REG_STATUS, MLX90632_REG_STATUS),
.cache_type = REGCACHE_RBTREE,
};
-static s32 mlx90632_pwr_set_sleep_step(struct regmap *regmap)
+static int mlx90632_pwr_set_sleep_step(struct regmap *regmap)
{
- return regmap_update_bits(regmap, MLX90632_REG_CONTROL,
- MLX90632_CFG_PWR_MASK,
- MLX90632_PWR_STATUS_SLEEP_STEP);
+ struct mlx90632_data *data =
+ iio_priv(dev_get_drvdata(regmap_get_device(regmap)));
+ int ret;
+
+ if (data->powerstatus == MLX90632_PWR_STATUS_SLEEP_STEP)
+ return 0;
+
+ ret = regmap_write_bits(regmap, MLX90632_REG_CONTROL, MLX90632_CFG_PWR_MASK,
+ MLX90632_PWR_STATUS_SLEEP_STEP);
+ if (ret < 0)
+ return ret;
+
+ data->powerstatus = MLX90632_PWR_STATUS_SLEEP_STEP;
+ return 0;
}
-static s32 mlx90632_pwr_continuous(struct regmap *regmap)
+static int mlx90632_pwr_continuous(struct regmap *regmap)
{
- return regmap_update_bits(regmap, MLX90632_REG_CONTROL,
- MLX90632_CFG_PWR_MASK,
- MLX90632_PWR_STATUS_CONTINUOUS);
+ struct mlx90632_data *data =
+ iio_priv(dev_get_drvdata(regmap_get_device(regmap)));
+ int ret;
+
+ if (data->powerstatus == MLX90632_PWR_STATUS_CONTINUOUS)
+ return 0;
+
+ ret = regmap_write_bits(regmap, MLX90632_REG_CONTROL, MLX90632_CFG_PWR_MASK,
+ MLX90632_PWR_STATUS_CONTINUOUS);
+ if (ret < 0)
+ return ret;
+
+ data->powerstatus = MLX90632_PWR_STATUS_CONTINUOUS;
+ return 0;
}
/**
usleep_range(150, 200);
}
+static int mlx90632_get_measurement_time(struct regmap *regmap, u16 meas)
+{
+ unsigned int reg;
+ int ret;
+
+ ret = regmap_read(regmap, meas, ®);
+ if (ret < 0)
+ return ret;
+
+ return MLX90632_MEAS_MAX_TIME >> FIELD_GET(MLX90632_EE_RR, reg);
+}
+
+static int mlx90632_calculate_dataset_ready_time(struct mlx90632_data *data)
+{
+ unsigned int refresh_time;
+ int ret;
+
+ if (data->mtyp == MLX90632_MTYP_MEDICAL) {
+ ret = mlx90632_get_measurement_time(data->regmap,
+ MLX90632_EE_MEDICAL_MEAS1);
+ if (ret < 0)
+ return ret;
+
+ refresh_time = ret;
+
+ ret = mlx90632_get_measurement_time(data->regmap,
+ MLX90632_EE_MEDICAL_MEAS2);
+ if (ret < 0)
+ return ret;
+
+ refresh_time += ret;
+ } else {
+ ret = mlx90632_get_measurement_time(data->regmap,
+ MLX90632_EE_EXTENDED_MEAS1);
+ if (ret < 0)
+ return ret;
+
+ refresh_time = ret;
+
+ ret = mlx90632_get_measurement_time(data->regmap,
+ MLX90632_EE_EXTENDED_MEAS2);
+ if (ret < 0)
+ return ret;
+
+ refresh_time += ret;
+
+ ret = mlx90632_get_measurement_time(data->regmap,
+ MLX90632_EE_EXTENDED_MEAS3);
+ if (ret < 0)
+ return ret;
+
+ refresh_time += ret;
+ }
+
+ return refresh_time;
+}
+
/**
* mlx90632_perform_measurement() - Trigger and retrieve current measurement cycle
* @data: pointer to mlx90632_data object containing regmap information
return (reg_status & MLX90632_STAT_CYCLE_POS) >> 2;
}
-static int mlx90632_set_meas_type(struct regmap *regmap, u8 type)
+/**
+ * mlx90632_perform_measurement_burst() - Trigger and retrieve current measurement
+ * cycle in step sleep mode
+ * @data: pointer to mlx90632_data object containing regmap information
+ *
+ * Perform a measurement and return 2 as measurement cycle position reported
+ * by sensor. This is a blocking function for amount dependent on the sensor
+ * refresh rate.
+ */
+static int mlx90632_perform_measurement_burst(struct mlx90632_data *data)
+{
+ unsigned int reg_status;
+ int ret;
+
+ ret = regmap_write_bits(data->regmap, MLX90632_REG_CONTROL,
+ MLX90632_CFG_SOB_MASK, MLX90632_CFG_SOB_MASK);
+ if (ret < 0)
+ return ret;
+
+ ret = mlx90632_calculate_dataset_ready_time(data);
+ if (ret < 0)
+ return ret;
+
+ msleep(ret); /* Wait minimum time for dataset to be ready */
+
+ ret = regmap_read_poll_timeout(data->regmap, MLX90632_REG_STATUS,
+ reg_status,
+ (reg_status & MLX90632_STAT_BUSY) == 0,
+ 10000, 100 * 10000);
+ if (ret < 0) {
+ dev_err(&data->client->dev, "data not ready");
+ return -ETIMEDOUT;
+ }
+
+ return 2;
+}
+
+static int mlx90632_set_meas_type(struct mlx90632_data *data, u8 type)
{
+ int current_powerstatus;
int ret;
- if ((type != MLX90632_MTYP_MEDICAL) && (type != MLX90632_MTYP_EXTENDED))
- return -EINVAL;
+ if (data->mtyp == type)
+ return 0;
+
+ current_powerstatus = data->powerstatus;
+ ret = mlx90632_pwr_continuous(data->regmap);
+ if (ret < 0)
+ return ret;
- ret = regmap_write(regmap, MLX90632_REG_I2C_CMD, MLX90632_RESET_CMD);
+ ret = regmap_write(data->regmap, MLX90632_REG_I2C_CMD, MLX90632_RESET_CMD);
if (ret < 0)
return ret;
mlx90632_reset_delay();
- ret = regmap_write_bits(regmap, MLX90632_REG_CONTROL,
+ ret = regmap_update_bits(data->regmap, MLX90632_REG_CONTROL,
(MLX90632_CFG_MTYP_MASK | MLX90632_CFG_PWR_MASK),
(MLX90632_MTYP_STATUS(type) | MLX90632_PWR_STATUS_HALT));
if (ret < 0)
return ret;
- return mlx90632_pwr_continuous(regmap);
+ data->mtyp = type;
+ data->powerstatus = MLX90632_PWR_STATUS_HALT;
+
+ if (current_powerstatus == MLX90632_PWR_STATUS_SLEEP_STEP)
+ return mlx90632_pwr_set_sleep_step(data->regmap);
+
+ return mlx90632_pwr_continuous(data->regmap);
}
static int mlx90632_channel_new_select(int perform_ret, uint8_t *channel_new,
*channel_old = 1;
break;
default:
- return -EINVAL;
+ return -ECHRNG;
}
return 0;
static int mlx90632_read_ambient_raw(struct regmap *regmap,
s16 *ambient_new_raw, s16 *ambient_old_raw)
{
- int ret;
unsigned int read_tmp;
+ int ret;
ret = regmap_read(regmap, MLX90632_RAM_3(1), &read_tmp);
if (ret < 0)
int perform_measurement_ret,
s16 *object_new_raw, s16 *object_old_raw)
{
- int ret;
unsigned int read_tmp;
- s16 read;
- u8 channel = 0;
u8 channel_old = 0;
+ u8 channel = 0;
+ s16 read;
+ int ret;
ret = mlx90632_channel_new_select(perform_measurement_ret, &channel,
&channel_old);
s16 *ambient_new_raw, s16 *ambient_old_raw,
s16 *object_new_raw, s16 *object_old_raw)
{
- s32 ret, measurement;
+ s32 measurement;
+ int ret;
mutex_lock(&data->lock);
- measurement = mlx90632_perform_measurement(data);
- if (measurement < 0) {
- ret = measurement;
+ ret = mlx90632_set_meas_type(data, MLX90632_MTYP_MEDICAL);
+ if (ret < 0)
+ goto read_unlock;
+
+ switch (data->powerstatus) {
+ case MLX90632_PWR_STATUS_CONTINUOUS:
+ ret = mlx90632_perform_measurement(data);
+ if (ret < 0)
+ goto read_unlock;
+
+ break;
+ case MLX90632_PWR_STATUS_SLEEP_STEP:
+ ret = mlx90632_perform_measurement_burst(data);
+ if (ret < 0)
+ goto read_unlock;
+
+ break;
+ default:
+ ret = -EOPNOTSUPP;
goto read_unlock;
}
+
+ measurement = ret; /* If we came here ret holds the measurement position */
+
ret = mlx90632_read_ambient_raw(data->regmap, ambient_new_raw,
ambient_old_raw);
if (ret < 0)
s32 ret, meas;
mutex_lock(&data->lock);
- ret = mlx90632_set_meas_type(data->regmap, MLX90632_MTYP_EXTENDED);
+ ret = mlx90632_set_meas_type(data, MLX90632_MTYP_EXTENDED);
if (ret < 0)
goto read_unlock;
- ret = read_poll_timeout(mlx90632_perform_measurement, meas, meas == 19,
- 50000, 800000, false, data);
- if (ret != 0)
+ switch (data->powerstatus) {
+ case MLX90632_PWR_STATUS_CONTINUOUS:
+ ret = read_poll_timeout(mlx90632_perform_measurement, meas, meas == 19,
+ 50000, 800000, false, data);
+ if (ret)
+ goto read_unlock;
+ break;
+ case MLX90632_PWR_STATUS_SLEEP_STEP:
+ ret = mlx90632_perform_measurement_burst(data);
+ if (ret < 0)
+ goto read_unlock;
+ break;
+ default:
+ ret = -EOPNOTSUPP;
goto read_unlock;
+ }
ret = mlx90632_read_object_raw_extended(data->regmap, object_new_raw);
if (ret < 0)
ret = mlx90632_read_ambient_raw_extended(data->regmap, ambient_new_raw, ambient_old_raw);
read_unlock:
- (void) mlx90632_set_meas_type(data->regmap, MLX90632_MTYP_MEDICAL);
-
mutex_unlock(&data->lock);
return ret;
}
static int mlx90632_read_ee_register(struct regmap *regmap, u16 reg_lsb,
s32 *reg_value)
{
- s32 ret;
unsigned int read;
u32 value;
+ int ret;
ret = regmap_read(regmap, reg_lsb, &read);
if (ret < 0)
static int mlx90632_calc_object_dsp105(struct mlx90632_data *data, int *val)
{
- s32 ret;
+ s16 ambient_new_raw, ambient_old_raw, object_new_raw, object_old_raw;
s32 Ea, Eb, Fa, Fb, Ga;
unsigned int read_tmp;
- s16 Ha, Hb, Gb, Ka;
- s16 ambient_new_raw, ambient_old_raw, object_new_raw, object_old_raw;
s64 object, ambient;
+ s16 Ha, Hb, Gb, Ka;
+ int ret;
ret = mlx90632_read_ee_register(data->regmap, MLX90632_EE_Ea, &Ea);
if (ret < 0)
static int mlx90632_calc_ambient_dsp105(struct mlx90632_data *data, int *val)
{
- s32 ret;
+ s16 ambient_new_raw, ambient_old_raw;
unsigned int read_tmp;
s32 PT, PR, PG, PO;
+ int ret;
s16 Gb;
- s16 ambient_new_raw, ambient_old_raw;
ret = mlx90632_read_ee_register(data->regmap, MLX90632_EE_P_R, &PR);
if (ret < 0)
return ret;
}
+static int mlx90632_get_refresh_rate(struct mlx90632_data *data,
+ int *refresh_rate)
+{
+ unsigned int meas1;
+ int ret;
+
+ ret = regmap_read(data->regmap, MLX90632_EE_MEDICAL_MEAS1, &meas1);
+ if (ret < 0)
+ return ret;
+
+ *refresh_rate = MLX90632_REFRESH_RATE(meas1);
+
+ return ret;
+}
+
+static const int mlx90632_freqs[][2] = {
+ {0, 500000},
+ {1, 0},
+ {2, 0},
+ {4, 0},
+ {8, 0},
+ {16, 0},
+ {32, 0},
+ {64, 0}
+};
+
+/**
+ * mlx90632_pm_interraction_wakeup() - Measure time between user interactions to change powermode
+ * @data: pointer to mlx90632_data object containing interaction_ts information
+ *
+ * Switch to continuous mode when interaction is faster than MLX90632_MEAS_MAX_TIME. Update the
+ * interaction_ts for each function call with the jiffies to enable measurement between function
+ * calls. Initial value of the interaction_ts needs to be set before this function call.
+ */
+static int mlx90632_pm_interraction_wakeup(struct mlx90632_data *data)
+{
+ unsigned long now;
+ int ret;
+
+ now = jiffies;
+ if (time_in_range(now, data->interaction_ts,
+ data->interaction_ts +
+ msecs_to_jiffies(MLX90632_MEAS_MAX_TIME + 100))) {
+ if (data->powerstatus == MLX90632_PWR_STATUS_SLEEP_STEP) {
+ ret = mlx90632_pwr_continuous(data->regmap);
+ if (ret < 0)
+ return ret;
+ }
+ }
+
+ data->interaction_ts = now;
+
+ return 0;
+}
+
static int mlx90632_read_raw(struct iio_dev *indio_dev,
struct iio_chan_spec const *channel, int *val,
int *val2, long mask)
{
struct mlx90632_data *data = iio_priv(indio_dev);
int ret;
+ int cr;
+
+ pm_runtime_get_sync(&data->client->dev);
+ ret = mlx90632_pm_interraction_wakeup(data);
+ if (ret < 0)
+ goto mlx90632_read_raw_pm;
switch (mask) {
case IIO_CHAN_INFO_PROCESSED:
case IIO_MOD_TEMP_AMBIENT:
ret = mlx90632_calc_ambient_dsp105(data, val);
if (ret < 0)
- return ret;
- return IIO_VAL_INT;
+ goto mlx90632_read_raw_pm;
+
+ ret = IIO_VAL_INT;
+ break;
case IIO_MOD_TEMP_OBJECT:
ret = mlx90632_calc_object_dsp105(data, val);
if (ret < 0)
- return ret;
- return IIO_VAL_INT;
+ goto mlx90632_read_raw_pm;
+
+ ret = IIO_VAL_INT;
+ break;
default:
- return -EINVAL;
+ ret = -EINVAL;
+ break;
}
+ break;
case IIO_CHAN_INFO_CALIBEMISSIVITY:
if (data->emissivity == 1000) {
*val = 1;
*val = 0;
*val2 = data->emissivity * 1000;
}
- return IIO_VAL_INT_PLUS_MICRO;
+ ret = IIO_VAL_INT_PLUS_MICRO;
+ break;
case IIO_CHAN_INFO_CALIBAMBIENT:
*val = data->object_ambient_temperature;
- return IIO_VAL_INT;
+ ret = IIO_VAL_INT;
+ break;
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ ret = mlx90632_get_refresh_rate(data, &cr);
+ if (ret < 0)
+ goto mlx90632_read_raw_pm;
+
+ *val = mlx90632_freqs[cr][0];
+ *val2 = mlx90632_freqs[cr][1];
+ ret = IIO_VAL_INT_PLUS_MICRO;
+ break;
default:
- return -EINVAL;
+ ret = -EINVAL;
+ break;
}
+
+mlx90632_read_raw_pm:
+ pm_runtime_mark_last_busy(&data->client->dev);
+ pm_runtime_put_autosuspend(&data->client->dev);
+ return ret;
}
static int mlx90632_write_raw(struct iio_dev *indio_dev,
}
}
+static int mlx90632_read_avail(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ const int **vals, int *type, int *length,
+ long mask)
+{
+ switch (mask) {
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ *vals = (int *)mlx90632_freqs;
+ *type = IIO_VAL_INT_PLUS_MICRO;
+ *length = 2 * ARRAY_SIZE(mlx90632_freqs);
+ return IIO_AVAIL_LIST;
+ default:
+ return -EINVAL;
+ }
+}
+
static const struct iio_chan_spec mlx90632_channels[] = {
{
.type = IIO_TEMP,
.modified = 1,
.channel2 = IIO_MOD_TEMP_AMBIENT,
.info_mask_separate = BIT(IIO_CHAN_INFO_PROCESSED),
+ .info_mask_shared_by_all = BIT(IIO_CHAN_INFO_SAMP_FREQ),
+ .info_mask_shared_by_all_available = BIT(IIO_CHAN_INFO_SAMP_FREQ),
},
{
.type = IIO_TEMP,
.channel2 = IIO_MOD_TEMP_OBJECT,
.info_mask_separate = BIT(IIO_CHAN_INFO_PROCESSED) |
BIT(IIO_CHAN_INFO_CALIBEMISSIVITY) | BIT(IIO_CHAN_INFO_CALIBAMBIENT),
+ .info_mask_shared_by_all = BIT(IIO_CHAN_INFO_SAMP_FREQ),
+ .info_mask_shared_by_all_available = BIT(IIO_CHAN_INFO_SAMP_FREQ),
},
};
static const struct iio_info mlx90632_info = {
.read_raw = mlx90632_read_raw,
.write_raw = mlx90632_write_raw,
+ .read_avail = mlx90632_read_avail,
};
-static int mlx90632_sleep(struct mlx90632_data *data)
+static void mlx90632_sleep(void *_data)
+{
+ struct mlx90632_data *data = _data;
+
+ mlx90632_pwr_set_sleep_step(data->regmap);
+}
+
+static int mlx90632_suspend(struct mlx90632_data *data)
{
regcache_mark_dirty(data->regmap);
- dev_dbg(&data->client->dev, "Requesting sleep");
+ dev_dbg(&data->client->dev, "Requesting suspend");
return mlx90632_pwr_set_sleep_step(data->regmap);
}
return ret;
}
-static int mlx90632_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mlx90632_probe(struct i2c_client *client)
{
- struct iio_dev *indio_dev;
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct mlx90632_data *mlx90632;
+ struct iio_dev *indio_dev;
struct regmap *regmap;
- int ret;
unsigned int read;
+ int ret;
indio_dev = devm_iio_device_alloc(&client->dev, sizeof(*mlx90632));
if (!indio_dev) {
mlx90632->client = client;
mlx90632->regmap = regmap;
mlx90632->mtyp = MLX90632_MTYP_MEDICAL;
+ mlx90632->powerstatus = MLX90632_PWR_STATUS_HALT;
mutex_init(&mlx90632->lock);
indio_dev->name = id->name;
return ret;
}
+ ret = devm_add_action_or_reset(&client->dev, mlx90632_sleep, mlx90632);
+ if (ret < 0) {
+ dev_err(&client->dev, "Failed to setup low power cleanup action %d\n",
+ ret);
+ return ret;
+ }
+
ret = regmap_read(mlx90632->regmap, MLX90632_EE_VERSION, &read);
if (ret < 0) {
dev_err(&client->dev, "read of version failed: %d\n", ret);
mlx90632->emissivity = 1000;
mlx90632->object_ambient_temperature = 25000; /* 25 degrees milliCelsius */
+ mlx90632->interaction_ts = jiffies; /* Set initial value */
- pm_runtime_disable(&client->dev);
- ret = pm_runtime_set_active(&client->dev);
- if (ret < 0) {
- mlx90632_sleep(mlx90632);
+ pm_runtime_get_noresume(&client->dev);
+ pm_runtime_set_active(&client->dev);
+
+ ret = devm_pm_runtime_enable(&client->dev);
+ if (ret)
return ret;
- }
- pm_runtime_enable(&client->dev);
+
pm_runtime_set_autosuspend_delay(&client->dev, MLX90632_SLEEP_DELAY_MS);
pm_runtime_use_autosuspend(&client->dev);
+ pm_runtime_put_autosuspend(&client->dev);
- return iio_device_register(indio_dev);
-}
-
-static void mlx90632_remove(struct i2c_client *client)
-{
- struct iio_dev *indio_dev = i2c_get_clientdata(client);
- struct mlx90632_data *data = iio_priv(indio_dev);
-
- iio_device_unregister(indio_dev);
-
- pm_runtime_disable(&client->dev);
- pm_runtime_set_suspended(&client->dev);
- pm_runtime_put_noidle(&client->dev);
-
- mlx90632_sleep(data);
+ return devm_iio_device_register(&client->dev, indio_dev);
}
static const struct i2c_device_id mlx90632_id[] = {
};
MODULE_DEVICE_TABLE(of, mlx90632_of_match);
-static int __maybe_unused mlx90632_pm_suspend(struct device *dev)
+static int mlx90632_pm_suspend(struct device *dev)
{
- struct iio_dev *indio_dev = i2c_get_clientdata(to_i2c_client(dev));
- struct mlx90632_data *data = iio_priv(indio_dev);
+ struct mlx90632_data *data = iio_priv(dev_get_drvdata(dev));
+ int ret;
- return mlx90632_sleep(data);
+ ret = mlx90632_suspend(data);
+ if (ret < 0)
+ return ret;
+
+ ret = regulator_disable(data->regulator);
+ if (ret < 0)
+ dev_err(regmap_get_device(data->regmap),
+ "Failed to disable power regulator: %d\n", ret);
+
+ return ret;
}
-static int __maybe_unused mlx90632_pm_resume(struct device *dev)
+static int mlx90632_pm_resume(struct device *dev)
{
- struct iio_dev *indio_dev = i2c_get_clientdata(to_i2c_client(dev));
- struct mlx90632_data *data = iio_priv(indio_dev);
+ struct mlx90632_data *data = iio_priv(dev_get_drvdata(dev));
+ int ret;
+
+ ret = mlx90632_enable_regulator(data);
+ if (ret < 0)
+ return ret;
return mlx90632_wakeup(data);
}
-static UNIVERSAL_DEV_PM_OPS(mlx90632_pm_ops, mlx90632_pm_suspend,
- mlx90632_pm_resume, NULL);
+static int mlx90632_pm_runtime_suspend(struct device *dev)
+{
+ struct mlx90632_data *data = iio_priv(dev_get_drvdata(dev));
+
+ return mlx90632_pwr_set_sleep_step(data->regmap);
+}
+
+static const struct dev_pm_ops mlx90632_pm_ops = {
+ SYSTEM_SLEEP_PM_OPS(mlx90632_pm_suspend, mlx90632_pm_resume)
+ RUNTIME_PM_OPS(mlx90632_pm_runtime_suspend, NULL, NULL)
+};
static struct i2c_driver mlx90632_driver = {
.driver = {
.name = "mlx90632",
.of_match_table = mlx90632_of_match,
- .pm = &mlx90632_pm_ops,
+ .pm = pm_ptr(&mlx90632_pm_ops),
},
- .probe = mlx90632_probe,
- .remove = mlx90632_remove,
+ .probe_new = mlx90632_probe,
.id_table = mlx90632_id,
};
module_i2c_driver(mlx90632_driver);
tmp006_power(dev, false);
}
-static int tmp006_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tmp006_probe(struct i2c_client *client)
{
struct iio_dev *indio_dev;
struct tmp006_data *data;
.name = "tmp006",
.pm = pm_sleep_ptr(&tmp006_pm_ops),
},
- .probe = tmp006_probe,
+ .probe_new = tmp006_probe,
.id_table = tmp006_id,
};
module_i2c_driver(tmp006_driver);
tmp007_powerdown(data);
}
-static int tmp007_probe(struct i2c_client *client,
- const struct i2c_device_id *tmp007_id)
+static int tmp007_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *tmp007_id = i2c_client_get_device_id(client);
struct tmp007_data *data;
struct iio_dev *indio_dev;
int ret;
.of_match_table = tmp007_of_match,
.pm = pm_sleep_ptr(&tmp007_pm_ops),
},
- .probe = tmp007_probe,
+ .probe_new = tmp007_probe,
.id_table = tmp007_id,
};
module_i2c_driver(tmp007_driver);
return devm_iio_device_register(dev, indio_dev);
}
-static int tsys01_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tsys01_i2c_probe(struct i2c_client *client)
{
struct tsys01_dev *dev_data;
struct iio_dev *indio_dev;
MODULE_DEVICE_TABLE(of, tsys01_of_match);
static struct i2c_driver tsys01_driver = {
- .probe = tsys01_i2c_probe,
+ .probe_new = tsys01_i2c_probe,
.id_table = tsys01_id,
.driver = {
.name = "tsys01",
.attrs = &tsys02d_attribute_group,
};
-static int tsys02d_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tsys02d_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct ms_ht_dev *dev_data;
struct iio_dev *indio_dev;
int ret;
MODULE_DEVICE_TABLE(i2c, tsys02d_id);
static struct i2c_driver tsys02d_driver = {
- .probe = tsys02d_probe,
+ .probe_new = tsys02d_probe,
.id_table = tsys02d_id,
.driver = {
.name = "tsys02d",
}
if (foundit) {
ret = -EINVAL;
- goto out1;
+ goto err_unlock;
}
t = kmalloc(sizeof(*t), GFP_KERNEL);
if (t == NULL) {
ret = -ENOMEM;
- goto out1;
+ goto err_unlock;
}
t->id = id;
t->trig = iio_trigger_alloc(&iio_sysfs_trig_dev, "sysfstrig%d", id);
if (!t->trig) {
ret = -ENOMEM;
- goto free_t;
+ goto err_free_sys_trig;
}
t->trig->dev.groups = iio_sysfs_trigger_attr_groups;
ret = iio_trigger_register(t->trig);
if (ret)
- goto out2;
+ goto err_free_trig;
list_add(&t->l, &iio_sysfs_trig_list);
__module_get(THIS_MODULE);
mutex_unlock(&iio_sysfs_trig_list_mut);
return 0;
-out2:
+err_free_trig:
iio_trigger_free(t->trig);
-free_t:
+err_free_sys_trig:
kfree(t);
-out1:
+err_unlock:
mutex_unlock(&iio_sysfs_trig_list_mut);
return ret;
}
kfree_rcu(dev, rcu_head);
}
-static int ib_device_uevent(struct device *device,
+static int ib_device_uevent(const struct device *device,
struct kobj_uevent_env *env)
{
if (add_uevent_var(env, "NAME=%s", dev_name(device)))
return 0;
}
-static const void *net_namespace(struct device *d)
+static const void *net_namespace(const struct device *d)
{
- struct ib_core_device *coredev =
+ const struct ib_core_device *coredev =
container_of(d, struct ib_core_device, dev);
return read_pnet(&coredev->rdma_net);
};
ATTRIBUTE_GROUPS(umad_class_dev);
-static char *umad_devnode(struct device *dev, umode_t *mode)
+static char *umad_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "infiniband/%s", dev_name(dev));
}
put_device(&uverbs_dev->dev);
}
-static char *uverbs_devnode(struct device *dev, umode_t *mode)
+static char *uverbs_devnode(const struct device *dev, umode_t *mode)
{
if (mode)
*mode = 0666;
return hfi1_class_name;
}
-static char *hfi1_devnode(struct device *dev, umode_t *mode)
+static char *hfi1_devnode(const struct device *dev, umode_t *mode)
{
if (mode)
*mode = 0600;
return hfi1_class_name_user;
}
-static char *hfi1_user_devnode(struct device *dev, umode_t *mode)
+static char *hfi1_user_devnode(const struct device *dev, umode_t *mode)
{
if (mode)
*mode = 0666;
return ret;
}
-static enum resp_states atomic_write_reply(struct rxe_qp *qp,
- struct rxe_pkt_info *pkt)
+#ifdef CONFIG_64BIT
+static enum resp_states do_atomic_write(struct rxe_qp *qp,
+ struct rxe_pkt_info *pkt)
{
- u64 src, *dst;
- struct resp_res *res = qp->resp.res;
struct rxe_mr *mr = qp->resp.mr;
int payload = payload_size(pkt);
+ u64 src, *dst;
- if (!res) {
- res = rxe_prepare_res(qp, pkt, RXE_ATOMIC_WRITE_MASK);
- qp->resp.res = res;
- }
-
- if (!res->replay) {
-#ifdef CONFIG_64BIT
- if (mr->state != RXE_MR_STATE_VALID)
- return RESPST_ERR_RKEY_VIOLATION;
-
- memcpy(&src, payload_addr(pkt), payload);
+ if (mr->state != RXE_MR_STATE_VALID)
+ return RESPST_ERR_RKEY_VIOLATION;
- dst = iova_to_vaddr(mr, qp->resp.va + qp->resp.offset, payload);
- /* check vaddr is 8 bytes aligned. */
- if (!dst || (uintptr_t)dst & 7)
- return RESPST_ERR_MISALIGNED_ATOMIC;
+ memcpy(&src, payload_addr(pkt), payload);
- /* Do atomic write after all prior operations have completed */
- smp_store_release(dst, src);
+ dst = iova_to_vaddr(mr, qp->resp.va + qp->resp.offset, payload);
+ /* check vaddr is 8 bytes aligned. */
+ if (!dst || (uintptr_t)dst & 7)
+ return RESPST_ERR_MISALIGNED_ATOMIC;
- /* decrease resp.resid to zero */
- qp->resp.resid -= sizeof(payload);
+ /* Do atomic write after all prior operations have completed */
+ smp_store_release(dst, src);
- qp->resp.msn++;
+ /* decrease resp.resid to zero */
+ qp->resp.resid -= sizeof(payload);
- /* next expected psn, read handles this separately */
- qp->resp.psn = (pkt->psn + 1) & BTH_PSN_MASK;
- qp->resp.ack_psn = qp->resp.psn;
+ qp->resp.msn++;
- qp->resp.opcode = pkt->opcode;
- qp->resp.status = IB_WC_SUCCESS;
+ /* next expected psn, read handles this separately */
+ qp->resp.psn = (pkt->psn + 1) & BTH_PSN_MASK;
+ qp->resp.ack_psn = qp->resp.psn;
- return RESPST_ACKNOWLEDGE;
+ qp->resp.opcode = pkt->opcode;
+ qp->resp.status = IB_WC_SUCCESS;
+ return RESPST_ACKNOWLEDGE;
+}
#else
- return RESPST_ERR_UNSUPPORTED_OPCODE;
+static enum resp_states do_atomic_write(struct rxe_qp *qp,
+ struct rxe_pkt_info *pkt)
+{
+ return RESPST_ERR_UNSUPPORTED_OPCODE;
+}
#endif /* CONFIG_64BIT */
+
+static enum resp_states atomic_write_reply(struct rxe_qp *qp,
+ struct rxe_pkt_info *pkt)
+{
+ struct resp_res *res = qp->resp.res;
+
+ if (!res) {
+ res = rxe_prepare_res(qp, pkt, RXE_ATOMIC_WRITE_MASK);
+ qp->resp.res = res;
}
- return RESPST_ACKNOWLEDGE;
+ if (res->replay)
+ return RESPST_ACKNOWLEDGE;
+ return do_atomic_write(qp, pkt);
}
static struct sk_buff *prepare_ack_packet(struct rxe_qp *qp,
dma_addr_t paddr = siw_pbl_get_buffer(pbl, offset, NULL, idx);
if (paddr)
- return virt_to_page((void *)paddr);
+ return virt_to_page((void *)(uintptr_t)paddr);
return NULL;
}
#endif
};
-static char *input_devnode(struct device *dev, umode_t *mode)
+static char *input_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "input/%s", dev_name(dev));
}
To compile this driver as a module, choose M here. The module will
be called tps65218-pwrbutton.
+config INPUT_TPS65219_PWRBUTTON
+ tristate "TPS65219 Power button driver"
+ depends on MFD_TPS65219
+ help
+ Say Y here if you want to enable power button reporting for
+ TPS65219 Power Management IC devices.
+
+ To compile this driver as a module, choose M here. The module will
+ be called tps65219-pwrbutton.
+
config INPUT_AXP20X_PEK
tristate "X-Powers AXP20X power button driver"
depends on MFD_AXP20X
To compile this driver as a module, choose M here: the module
will be called da9063_onkey.
-config INPUT_DM355EVM
- tristate "TI DaVinci DM355 EVM Keypad and IR Remote"
- depends on MFD_DM355EVM_MSP
- select INPUT_SPARSEKMAP
- help
- Supports the pushbuttons and IR remote used with
- the DM355 EVM board.
-
- To compile this driver as a module, choose M here: the
- module will be called dm355evm_keys.
-
config INPUT_WM831X_ON
tristate "WM831X ON pin"
depends on MFD_WM831X
obj-$(CONFIG_INPUT_DA9052_ONKEY) += da9052_onkey.o
obj-$(CONFIG_INPUT_DA9055_ONKEY) += da9055_onkey.o
obj-$(CONFIG_INPUT_DA9063_ONKEY) += da9063_onkey.o
-obj-$(CONFIG_INPUT_DM355EVM) += dm355evm_keys.o
obj-$(CONFIG_INPUT_E3X0_BUTTON) += e3x0-button.o
obj-$(CONFIG_INPUT_DRV260X_HAPTICS) += drv260x.o
obj-$(CONFIG_INPUT_DRV2665_HAPTICS) += drv2665.o
obj-$(CONFIG_INPUT_SPARCSPKR) += sparcspkr.o
obj-$(CONFIG_INPUT_STPMIC1_ONKEY) += stpmic1_onkey.o
obj-$(CONFIG_INPUT_TPS65218_PWRBUTTON) += tps65218-pwrbutton.o
+obj-$(CONFIG_INPUT_TPS65219_PWRBUTTON) += tps65219-pwrbutton.o
obj-$(CONFIG_INPUT_TWL4030_PWRBUTTON) += twl4030-pwrbutton.o
obj-$(CONFIG_INPUT_TWL4030_VIBRA) += twl4030-vibra.o
obj-$(CONFIG_INPUT_TWL6040_VIBRA) += twl6040-vibra.o
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * dm355evm_keys.c - support buttons and IR remote on DM355 EVM board
- *
- * Copyright (c) 2008 by David Brownell
- */
-#include <linux/kernel.h>
-#include <linux/slab.h>
-#include <linux/input.h>
-#include <linux/input/sparse-keymap.h>
-#include <linux/platform_device.h>
-#include <linux/interrupt.h>
-
-#include <linux/mfd/dm355evm_msp.h>
-#include <linux/module.h>
-
-
-/*
- * The MSP430 firmware on the DM355 EVM monitors on-board pushbuttons
- * and an IR receptor used for the remote control. When any key is
- * pressed, or its autorepeat kicks in, an event is sent. This driver
- * read those events from the small (32 event) queue and reports them.
- *
- * Note that physically there can only be one of these devices.
- *
- * This driver was tested with firmware revision A4.
- */
-struct dm355evm_keys {
- struct input_dev *input;
- struct device *dev;
-};
-
-/* These initial keycodes can be remapped */
-static const struct key_entry dm355evm_keys[] = {
- /*
- * Pushbuttons on the EVM board ... note that the labels for these
- * are SW10/SW11/etc on the PC board. The left/right orientation
- * comes only from the firmware's documentation, and presumes the
- * power connector is immediately in front of you and the IR sensor
- * is to the right. (That is, rotate the board counter-clockwise
- * by 90 degrees from the SW10/etc and "DM355 EVM" labels.)
- */
- { KE_KEY, 0x00d8, { KEY_OK } }, /* SW12 */
- { KE_KEY, 0x00b8, { KEY_UP } }, /* SW13 */
- { KE_KEY, 0x00e8, { KEY_DOWN } }, /* SW11 */
- { KE_KEY, 0x0078, { KEY_LEFT } }, /* SW14 */
- { KE_KEY, 0x00f0, { KEY_RIGHT } }, /* SW10 */
-
- /*
- * IR buttons ... codes assigned to match the universal remote
- * provided with the EVM (Philips PM4S) using DVD code 0020.
- *
- * These event codes match firmware documentation, but other
- * remote controls could easily send more RC5-encoded events.
- * The PM4S manual was used in several cases to help select
- * a keycode reflecting the intended usage.
- *
- * RC5 codes are 14 bits, with two start bits (0x3 prefix)
- * and a toggle bit (masked out below).
- */
- { KE_KEY, 0x300c, { KEY_POWER } }, /* NOTE: docs omit this */
- { KE_KEY, 0x3000, { KEY_NUMERIC_0 } },
- { KE_KEY, 0x3001, { KEY_NUMERIC_1 } },
- { KE_KEY, 0x3002, { KEY_NUMERIC_2 } },
- { KE_KEY, 0x3003, { KEY_NUMERIC_3 } },
- { KE_KEY, 0x3004, { KEY_NUMERIC_4 } },
- { KE_KEY, 0x3005, { KEY_NUMERIC_5 } },
- { KE_KEY, 0x3006, { KEY_NUMERIC_6 } },
- { KE_KEY, 0x3007, { KEY_NUMERIC_7 } },
- { KE_KEY, 0x3008, { KEY_NUMERIC_8 } },
- { KE_KEY, 0x3009, { KEY_NUMERIC_9 } },
- { KE_KEY, 0x3022, { KEY_ENTER } },
- { KE_KEY, 0x30ec, { KEY_MODE } }, /* "tv/vcr/..." */
- { KE_KEY, 0x300f, { KEY_SELECT } }, /* "info" */
- { KE_KEY, 0x3020, { KEY_CHANNELUP } }, /* "up" */
- { KE_KEY, 0x302e, { KEY_MENU } }, /* "in/out" */
- { KE_KEY, 0x3011, { KEY_VOLUMEDOWN } }, /* "left" */
- { KE_KEY, 0x300d, { KEY_MUTE } }, /* "ok" */
- { KE_KEY, 0x3010, { KEY_VOLUMEUP } }, /* "right" */
- { KE_KEY, 0x301e, { KEY_SUBTITLE } }, /* "cc" */
- { KE_KEY, 0x3021, { KEY_CHANNELDOWN } },/* "down" */
- { KE_KEY, 0x3022, { KEY_PREVIOUS } },
- { KE_KEY, 0x3026, { KEY_SLEEP } },
- { KE_KEY, 0x3172, { KEY_REWIND } }, /* NOTE: docs wrongly say 0x30ca */
- { KE_KEY, 0x3175, { KEY_PLAY } },
- { KE_KEY, 0x3174, { KEY_FASTFORWARD } },
- { KE_KEY, 0x3177, { KEY_RECORD } },
- { KE_KEY, 0x3176, { KEY_STOP } },
- { KE_KEY, 0x3169, { KEY_PAUSE } },
-};
-
-/*
- * Because we communicate with the MSP430 using I2C, and all I2C calls
- * in Linux sleep, we use a threaded IRQ handler. The IRQ itself is
- * active low, but we go through the GPIO controller so we can trigger
- * on falling edges and not worry about enabling/disabling the IRQ in
- * the keypress handling path.
- */
-static irqreturn_t dm355evm_keys_irq(int irq, void *_keys)
-{
- static u16 last_event;
- struct dm355evm_keys *keys = _keys;
- const struct key_entry *ke;
- unsigned int keycode;
- int status;
- u16 event;
-
- /* For simplicity we ignore INPUT_COUNT and just read
- * events until we get the "queue empty" indicator.
- * Reading INPUT_LOW decrements the count.
- */
- for (;;) {
- status = dm355evm_msp_read(DM355EVM_MSP_INPUT_HIGH);
- if (status < 0) {
- dev_dbg(keys->dev, "input high err %d\n",
- status);
- break;
- }
- event = status << 8;
-
- status = dm355evm_msp_read(DM355EVM_MSP_INPUT_LOW);
- if (status < 0) {
- dev_dbg(keys->dev, "input low err %d\n",
- status);
- break;
- }
- event |= status;
- if (event == 0xdead)
- break;
-
- /* Press and release a button: two events, same code.
- * Press and hold (autorepeat), then release: N events
- * (N > 2), same code. For RC5 buttons the toggle bits
- * distinguish (for example) "1-autorepeat" from "1 1";
- * but PCB buttons don't support that bit.
- *
- * So we must synthesize release events. We do that by
- * mapping events to a press/release event pair; then
- * to avoid adding extra events, skip the second event
- * of each pair.
- */
- if (event == last_event) {
- last_event = 0;
- continue;
- }
- last_event = event;
-
- /* ignore the RC5 toggle bit */
- event &= ~0x0800;
-
- /* find the key, or report it as unknown */
- ke = sparse_keymap_entry_from_scancode(keys->input, event);
- keycode = ke ? ke->keycode : KEY_UNKNOWN;
- dev_dbg(keys->dev,
- "input event 0x%04x--> keycode %d\n",
- event, keycode);
-
- /* report press + release */
- input_report_key(keys->input, keycode, 1);
- input_sync(keys->input);
- input_report_key(keys->input, keycode, 0);
- input_sync(keys->input);
- }
-
- return IRQ_HANDLED;
-}
-
-/*----------------------------------------------------------------------*/
-
-static int dm355evm_keys_probe(struct platform_device *pdev)
-{
- struct dm355evm_keys *keys;
- struct input_dev *input;
- int irq;
- int error;
-
- keys = devm_kzalloc(&pdev->dev, sizeof (*keys), GFP_KERNEL);
- if (!keys)
- return -ENOMEM;
-
- input = devm_input_allocate_device(&pdev->dev);
- if (!input)
- return -ENOMEM;
-
- keys->dev = &pdev->dev;
- keys->input = input;
-
- input->name = "DM355 EVM Controls";
- input->phys = "dm355evm/input0";
-
- input->id.bustype = BUS_I2C;
- input->id.product = 0x0355;
- input->id.version = dm355evm_msp_read(DM355EVM_MSP_FIRMREV);
-
- error = sparse_keymap_setup(input, dm355evm_keys, NULL);
- if (error)
- return error;
-
- /* REVISIT: flush the event queue? */
-
- /* set up "threaded IRQ handler" */
- irq = platform_get_irq(pdev, 0);
- if (irq < 0)
- return irq;
-
- error = devm_request_threaded_irq(&pdev->dev, irq,
- NULL, dm355evm_keys_irq,
- IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
- dev_name(&pdev->dev), keys);
- if (error)
- return error;
-
- /* register */
- error = input_register_device(input);
- if (error)
- return error;
-
- return 0;
-}
-
-/* REVISIT: add suspend/resume when DaVinci supports it. The IRQ should
- * be able to wake up the system. When device_may_wakeup(&pdev->dev), call
- * enable_irq_wake() on suspend, and disable_irq_wake() on resume.
- */
-
-/*
- * I2C is used to talk to the MSP430, but this platform device is
- * exposed by an MFD driver that manages I2C communications.
- */
-static struct platform_driver dm355evm_keys_driver = {
- .probe = dm355evm_keys_probe,
- .driver = {
- .name = "dm355evm_keys",
- },
-};
-module_platform_driver(dm355evm_keys_driver);
-
-MODULE_LICENSE("GPL");
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+//
+// Driver for TPS65219 Push Button
+//
+// Copyright (C) 2022 BayLibre Incorporated - https://www.baylibre.com/
+
+#include <linux/init.h>
+#include <linux/input.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/mfd/tps65219.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <linux/slab.h>
+
+struct tps65219_pwrbutton {
+ struct device *dev;
+ struct input_dev *idev;
+ char phys[32];
+};
+
+static irqreturn_t tps65219_pb_push_irq(int irq, void *_pwr)
+{
+ struct tps65219_pwrbutton *pwr = _pwr;
+
+ input_report_key(pwr->idev, KEY_POWER, 1);
+ pm_wakeup_event(pwr->dev, 0);
+ input_sync(pwr->idev);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t tps65219_pb_release_irq(int irq, void *_pwr)
+{
+ struct tps65219_pwrbutton *pwr = _pwr;
+
+ input_report_key(pwr->idev, KEY_POWER, 0);
+ input_sync(pwr->idev);
+
+ return IRQ_HANDLED;
+}
+
+static int tps65219_pb_probe(struct platform_device *pdev)
+{
+ struct tps65219 *tps = dev_get_drvdata(pdev->dev.parent);
+ struct device *dev = &pdev->dev;
+ struct tps65219_pwrbutton *pwr;
+ struct input_dev *idev;
+ int error;
+ int push_irq;
+ int release_irq;
+
+ pwr = devm_kzalloc(dev, sizeof(*pwr), GFP_KERNEL);
+ if (!pwr)
+ return -ENOMEM;
+
+ idev = devm_input_allocate_device(dev);
+ if (!idev)
+ return -ENOMEM;
+
+ idev->name = pdev->name;
+ snprintf(pwr->phys, sizeof(pwr->phys), "%s/input0",
+ pdev->name);
+ idev->phys = pwr->phys;
+ idev->id.bustype = BUS_I2C;
+
+ input_set_capability(idev, EV_KEY, KEY_POWER);
+
+ pwr->dev = dev;
+ pwr->idev = idev;
+ device_init_wakeup(dev, true);
+
+ push_irq = platform_get_irq(pdev, 0);
+ if (push_irq < 0)
+ return -EINVAL;
+
+ release_irq = platform_get_irq(pdev, 1);
+ if (release_irq < 0)
+ return -EINVAL;
+
+ error = devm_request_threaded_irq(dev, push_irq, NULL,
+ tps65219_pb_push_irq,
+ IRQF_ONESHOT,
+ dev->init_name, pwr);
+ if (error) {
+ dev_err(dev, "failed to request push IRQ #%d: %d\n", push_irq,
+ error);
+ return error;
+ }
+
+ error = devm_request_threaded_irq(dev, release_irq, NULL,
+ tps65219_pb_release_irq,
+ IRQF_ONESHOT,
+ dev->init_name, pwr);
+ if (error) {
+ dev_err(dev, "failed to request release IRQ #%d: %d\n",
+ release_irq, error);
+ return error;
+ }
+
+ error = input_register_device(idev);
+ if (error) {
+ dev_err(dev, "Can't register power button: %d\n", error);
+ return error;
+ }
+
+ /* Enable interrupts for the pushbutton */
+ regmap_clear_bits(tps->regmap, TPS65219_REG_MASK_CONFIG,
+ TPS65219_REG_MASK_INT_FOR_PB_MASK);
+
+ /* Set PB/EN/VSENSE pin to be a pushbutton */
+ regmap_update_bits(tps->regmap, TPS65219_REG_MFP_2_CONFIG,
+ TPS65219_MFP_2_EN_PB_VSENSE_MASK, TPS65219_MFP_2_PB);
+
+ return 0;
+}
+
+static int tps65219_pb_remove(struct platform_device *pdev)
+{
+ struct tps65219 *tps = dev_get_drvdata(pdev->dev.parent);
+
+ /* Disable interrupt for the pushbutton */
+ return regmap_update_bits(tps->regmap, TPS65219_REG_MASK_CONFIG,
+ TPS65219_REG_MASK_INT_FOR_PB_MASK,
+ TPS65219_REG_MASK_INT_FOR_PB_MASK);
+}
+
+static const struct platform_device_id tps65219_pwrbtn_id_table[] = {
+ { "tps65219-pwrbutton", },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(platform, tps65219_pwrbtn_id_table);
+
+static struct platform_driver tps65219_pb_driver = {
+ .probe = tps65219_pb_probe,
+ .remove = tps65219_pb_remove,
+ .driver = {
+ .name = "tps65219_pwrbutton",
+ },
+ .id_table = tps65219_pwrbtn_id_table,
+};
+module_platform_driver(tps65219_pb_driver);
+
+MODULE_DESCRIPTION("TPS65219 Power Button");
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Markus Schneider-Pargmann <msp@baylibre.com");
}
mmio = devm_ioremap_resource(dev, res);
-
- if (IS_ERR(mmio)) {
- dev_err(dev, "Cannot ioremap interconnect bus resource\n");
+ if (IS_ERR(mmio))
return PTR_ERR(mmio);
- }
qp->regmap = devm_regmap_init_mmio(dev, mmio, desc->regmap_cfg);
if (IS_ERR(qp->regmap)) {
/* EPSS Register offsets */
#define EPSS_LUT_ROW_SIZE 4
+#define EPSS_REG_L3_VOTE 0x90
#define EPSS_REG_FREQ_LUT 0x100
#define EPSS_REG_PERF_STATE 0x320
unsigned int reg_perf_state;
};
+enum {
+ OSM_L3_MASTER_NODE = 10000,
+ OSM_L3_SLAVE_NODE,
+};
+
#define DEFINE_QNODE(_name, _id, _buswidth, ...) \
static const struct qcom_osm_l3_node _name = { \
.name = #_name, \
.links = { __VA_ARGS__ }, \
}
-DEFINE_QNODE(sdm845_osm_apps_l3, SDM845_MASTER_OSM_L3_APPS, 16, SDM845_SLAVE_OSM_L3);
-DEFINE_QNODE(sdm845_osm_l3, SDM845_SLAVE_OSM_L3, 16);
-
-static const struct qcom_osm_l3_node * const sdm845_osm_l3_nodes[] = {
- [MASTER_OSM_L3_APPS] = &sdm845_osm_apps_l3,
- [SLAVE_OSM_L3] = &sdm845_osm_l3,
-};
+DEFINE_QNODE(osm_l3_master, OSM_L3_MASTER_NODE, 16, OSM_L3_SLAVE_NODE);
+DEFINE_QNODE(osm_l3_slave, OSM_L3_SLAVE_NODE, 16);
-static const struct qcom_osm_l3_desc sdm845_icc_osm_l3 = {
- .nodes = sdm845_osm_l3_nodes,
- .num_nodes = ARRAY_SIZE(sdm845_osm_l3_nodes),
- .lut_row_size = OSM_LUT_ROW_SIZE,
- .reg_freq_lut = OSM_REG_FREQ_LUT,
- .reg_perf_state = OSM_REG_PERF_STATE,
+static const struct qcom_osm_l3_node * const osm_l3_nodes[] = {
+ [MASTER_OSM_L3_APPS] = &osm_l3_master,
+ [SLAVE_OSM_L3] = &osm_l3_slave,
};
-DEFINE_QNODE(sc7180_osm_apps_l3, SC7180_MASTER_OSM_L3_APPS, 16, SC7180_SLAVE_OSM_L3);
-DEFINE_QNODE(sc7180_osm_l3, SC7180_SLAVE_OSM_L3, 16);
+DEFINE_QNODE(epss_l3_master, OSM_L3_MASTER_NODE, 32, OSM_L3_SLAVE_NODE);
+DEFINE_QNODE(epss_l3_slave, OSM_L3_SLAVE_NODE, 32);
-static const struct qcom_osm_l3_node * const sc7180_osm_l3_nodes[] = {
- [MASTER_OSM_L3_APPS] = &sc7180_osm_apps_l3,
- [SLAVE_OSM_L3] = &sc7180_osm_l3,
+static const struct qcom_osm_l3_node * const epss_l3_nodes[] = {
+ [MASTER_EPSS_L3_APPS] = &epss_l3_master,
+ [SLAVE_EPSS_L3_SHARED] = &epss_l3_slave,
};
-static const struct qcom_osm_l3_desc sc7180_icc_osm_l3 = {
- .nodes = sc7180_osm_l3_nodes,
- .num_nodes = ARRAY_SIZE(sc7180_osm_l3_nodes),
+static const struct qcom_osm_l3_desc osm_l3 = {
+ .nodes = osm_l3_nodes,
+ .num_nodes = ARRAY_SIZE(osm_l3_nodes),
.lut_row_size = OSM_LUT_ROW_SIZE,
.reg_freq_lut = OSM_REG_FREQ_LUT,
.reg_perf_state = OSM_REG_PERF_STATE,
};
-DEFINE_QNODE(sc7280_epss_apps_l3, SC7280_MASTER_EPSS_L3_APPS, 32, SC7280_SLAVE_EPSS_L3);
-DEFINE_QNODE(sc7280_epss_l3, SC7280_SLAVE_EPSS_L3, 32);
-
-static const struct qcom_osm_l3_node * const sc7280_epss_l3_nodes[] = {
- [MASTER_EPSS_L3_APPS] = &sc7280_epss_apps_l3,
- [SLAVE_EPSS_L3_SHARED] = &sc7280_epss_l3,
-};
-
-static const struct qcom_osm_l3_desc sc7280_icc_epss_l3 = {
- .nodes = sc7280_epss_l3_nodes,
- .num_nodes = ARRAY_SIZE(sc7280_epss_l3_nodes),
+static const struct qcom_osm_l3_desc epss_l3_perf_state = {
+ .nodes = epss_l3_nodes,
+ .num_nodes = ARRAY_SIZE(epss_l3_nodes),
.lut_row_size = EPSS_LUT_ROW_SIZE,
.reg_freq_lut = EPSS_REG_FREQ_LUT,
.reg_perf_state = EPSS_REG_PERF_STATE,
};
-DEFINE_QNODE(sc8180x_osm_apps_l3, SC8180X_MASTER_OSM_L3_APPS, 32, SC8180X_SLAVE_OSM_L3);
-DEFINE_QNODE(sc8180x_osm_l3, SC8180X_SLAVE_OSM_L3, 32);
-
-static const struct qcom_osm_l3_node * const sc8180x_osm_l3_nodes[] = {
- [MASTER_OSM_L3_APPS] = &sc8180x_osm_apps_l3,
- [SLAVE_OSM_L3] = &sc8180x_osm_l3,
-};
-
-static const struct qcom_osm_l3_desc sc8180x_icc_osm_l3 = {
- .nodes = sc8180x_osm_l3_nodes,
- .num_nodes = ARRAY_SIZE(sc8180x_osm_l3_nodes),
- .lut_row_size = OSM_LUT_ROW_SIZE,
- .reg_freq_lut = OSM_REG_FREQ_LUT,
- .reg_perf_state = OSM_REG_PERF_STATE,
-};
-
-DEFINE_QNODE(sm8150_osm_apps_l3, SM8150_MASTER_OSM_L3_APPS, 32, SM8150_SLAVE_OSM_L3);
-DEFINE_QNODE(sm8150_osm_l3, SM8150_SLAVE_OSM_L3, 32);
-
-static const struct qcom_osm_l3_node * const sm8150_osm_l3_nodes[] = {
- [MASTER_OSM_L3_APPS] = &sm8150_osm_apps_l3,
- [SLAVE_OSM_L3] = &sm8150_osm_l3,
-};
-
-static const struct qcom_osm_l3_desc sm8150_icc_osm_l3 = {
- .nodes = sm8150_osm_l3_nodes,
- .num_nodes = ARRAY_SIZE(sm8150_osm_l3_nodes),
- .lut_row_size = OSM_LUT_ROW_SIZE,
- .reg_freq_lut = OSM_REG_FREQ_LUT,
- .reg_perf_state = OSM_REG_PERF_STATE,
-};
-
-DEFINE_QNODE(sm8250_epss_apps_l3, SM8250_MASTER_EPSS_L3_APPS, 32, SM8250_SLAVE_EPSS_L3);
-DEFINE_QNODE(sm8250_epss_l3, SM8250_SLAVE_EPSS_L3, 32);
-
-static const struct qcom_osm_l3_node * const sm8250_epss_l3_nodes[] = {
- [MASTER_EPSS_L3_APPS] = &sm8250_epss_apps_l3,
- [SLAVE_EPSS_L3_SHARED] = &sm8250_epss_l3,
-};
-
-static const struct qcom_osm_l3_desc sm8250_icc_epss_l3 = {
- .nodes = sm8250_epss_l3_nodes,
- .num_nodes = ARRAY_SIZE(sm8250_epss_l3_nodes),
+static const struct qcom_osm_l3_desc epss_l3_l3_vote = {
+ .nodes = epss_l3_nodes,
+ .num_nodes = ARRAY_SIZE(epss_l3_nodes),
.lut_row_size = EPSS_LUT_ROW_SIZE,
.reg_freq_lut = EPSS_REG_FREQ_LUT,
- .reg_perf_state = EPSS_REG_PERF_STATE,
+ .reg_perf_state = EPSS_REG_L3_VOTE,
};
static int qcom_osm_l3_set(struct icc_node *src, struct icc_node *dst)
struct qcom_osm_l3_icc_provider *qp;
struct icc_provider *provider;
const struct qcom_osm_l3_node *qn;
- struct icc_node *n;
unsigned int index;
- u32 agg_peak = 0;
- u32 agg_avg = 0;
u64 rate;
qn = src->data;
provider = src->provider;
qp = to_osm_l3_provider(provider);
- list_for_each_entry(n, &provider->nodes, node_list)
- provider->aggregate(n, 0, n->avg_bw, n->peak_bw,
- &agg_avg, &agg_peak);
-
- rate = max(agg_avg, agg_peak);
- rate = icc_units_to_bps(rate);
+ rate = icc_units_to_bps(dst->peak_bw);
do_div(rate, qn->buswidth);
for (index = 0; index < qp->max_state - 1; index++) {
}
static const struct of_device_id osm_l3_of_match[] = {
- { .compatible = "qcom,sc7180-osm-l3", .data = &sc7180_icc_osm_l3 },
- { .compatible = "qcom,sc7280-epss-l3", .data = &sc7280_icc_epss_l3 },
- { .compatible = "qcom,sdm845-osm-l3", .data = &sdm845_icc_osm_l3 },
- { .compatible = "qcom,sm8150-osm-l3", .data = &sm8150_icc_osm_l3 },
- { .compatible = "qcom,sc8180x-osm-l3", .data = &sc8180x_icc_osm_l3 },
- { .compatible = "qcom,sm8250-epss-l3", .data = &sm8250_icc_epss_l3 },
+ { .compatible = "qcom,epss-l3", .data = &epss_l3_l3_vote },
+ { .compatible = "qcom,osm-l3", .data = &osm_l3 },
+ { .compatible = "qcom,sc7180-osm-l3", .data = &osm_l3 },
+ { .compatible = "qcom,sc7280-epss-l3", .data = &epss_l3_perf_state },
+ { .compatible = "qcom,sdm845-osm-l3", .data = &osm_l3 },
+ { .compatible = "qcom,sm8150-osm-l3", .data = &osm_l3 },
+ { .compatible = "qcom,sc8180x-osm-l3", .data = &osm_l3 },
+ { .compatible = "qcom,sm8250-epss-l3", .data = &epss_l3_perf_state },
{ }
};
MODULE_DEVICE_TABLE(of, osm_l3_of_match);
.num_bcms = ARRAY_SIZE(gem_noc_bcms),
};
-static struct qcom_icc_bcm *mc_virt_bcms[] = {
+static struct qcom_icc_bcm * const mc_virt_bcms[] = {
&bcm_acv,
&bcm_mc0,
};
[SLAVE_QUP_CORE_1] = &qup_core_slave_2,
};
-static const struct qcom_icc_desc sc7180_qup_virt = {
+static const struct qcom_icc_desc sc7180_qup_virt = {
.nodes = qup_virt_nodes,
.num_nodes = ARRAY_SIZE(qup_virt_nodes),
.bcms = qup_virt_bcms,
&bcm_qup0,
};
-static struct qcom_icc_node *qup_virt_nodes[] = {
+static struct qcom_icc_node * const qup_virt_nodes[] = {
[MASTER_QUP_CORE_0] = &mas_qup_core_0,
[MASTER_QUP_CORE_1] = &mas_qup_core_1,
[MASTER_QUP_CORE_2] = &mas_qup_core_2,
#define LOOP_TIMEOUT 2000000
-#define IVRS_GET_SBDF_ID(seg, bus, dev, fd) (((seg & 0xffff) << 16) | ((bus & 0xff) << 8) \
+#define IVRS_GET_SBDF_ID(seg, bus, dev, fn) (((seg & 0xffff) << 16) | ((bus & 0xff) << 8) \
| ((dev & 0x1f) << 3) | (fn & 0x7))
/*
static int __init parse_ivrs_ioapic(char *str)
{
u32 seg = 0, bus, dev, fn;
- int ret, id, i;
+ int id, i;
u32 devid;
- ret = sscanf(str, "[%d]=%x:%x.%x", &id, &bus, &dev, &fn);
- if (ret != 4) {
- ret = sscanf(str, "[%d]=%x:%x:%x.%x", &id, &seg, &bus, &dev, &fn);
- if (ret != 5) {
- pr_err("Invalid command line: ivrs_ioapic%s\n", str);
- return 1;
- }
+ if (sscanf(str, "=%d@%x:%x.%x", &id, &bus, &dev, &fn) == 4 ||
+ sscanf(str, "=%d@%x:%x:%x.%x", &id, &seg, &bus, &dev, &fn) == 5)
+ goto found;
+
+ if (sscanf(str, "[%d]=%x:%x.%x", &id, &bus, &dev, &fn) == 4 ||
+ sscanf(str, "[%d]=%x:%x:%x.%x", &id, &seg, &bus, &dev, &fn) == 5) {
+ pr_warn("ivrs_ioapic%s option format deprecated; use ivrs_ioapic=%d@%04x:%02x:%02x.%d instead\n",
+ str, id, seg, bus, dev, fn);
+ goto found;
}
+ pr_err("Invalid command line: ivrs_ioapic%s\n", str);
+ return 1;
+
+found:
if (early_ioapic_map_size == EARLY_MAP_SIZE) {
pr_err("Early IOAPIC map overflow - ignoring ivrs_ioapic%s\n",
str);
static int __init parse_ivrs_hpet(char *str)
{
u32 seg = 0, bus, dev, fn;
- int ret, id, i;
+ int id, i;
u32 devid;
- ret = sscanf(str, "[%d]=%x:%x.%x", &id, &bus, &dev, &fn);
- if (ret != 4) {
- ret = sscanf(str, "[%d]=%x:%x:%x.%x", &id, &seg, &bus, &dev, &fn);
- if (ret != 5) {
- pr_err("Invalid command line: ivrs_hpet%s\n", str);
- return 1;
- }
+ if (sscanf(str, "=%d@%x:%x.%x", &id, &bus, &dev, &fn) == 4 ||
+ sscanf(str, "=%d@%x:%x:%x.%x", &id, &seg, &bus, &dev, &fn) == 5)
+ goto found;
+
+ if (sscanf(str, "[%d]=%x:%x.%x", &id, &bus, &dev, &fn) == 4 ||
+ sscanf(str, "[%d]=%x:%x:%x.%x", &id, &seg, &bus, &dev, &fn) == 5) {
+ pr_warn("ivrs_hpet%s option format deprecated; use ivrs_hpet=%d@%04x:%02x:%02x.%d instead\n",
+ str, id, seg, bus, dev, fn);
+ goto found;
}
+ pr_err("Invalid command line: ivrs_hpet%s\n", str);
+ return 1;
+
+found:
if (early_hpet_map_size == EARLY_MAP_SIZE) {
pr_err("Early HPET map overflow - ignoring ivrs_hpet%s\n",
str);
static int __init parse_ivrs_acpihid(char *str)
{
u32 seg = 0, bus, dev, fn;
- char *hid, *uid, *p;
+ char *hid, *uid, *p, *addr;
char acpiid[ACPIHID_UID_LEN + ACPIHID_HID_LEN] = {0};
- int ret, i;
-
- ret = sscanf(str, "[%x:%x.%x]=%s", &bus, &dev, &fn, acpiid);
- if (ret != 4) {
- ret = sscanf(str, "[%x:%x:%x.%x]=%s", &seg, &bus, &dev, &fn, acpiid);
- if (ret != 5) {
- pr_err("Invalid command line: ivrs_acpihid(%s)\n", str);
- return 1;
+ int i;
+
+ addr = strchr(str, '@');
+ if (!addr) {
+ if (sscanf(str, "[%x:%x.%x]=%s", &bus, &dev, &fn, acpiid) == 4 ||
+ sscanf(str, "[%x:%x:%x.%x]=%s", &seg, &bus, &dev, &fn, acpiid) == 5) {
+ pr_warn("ivrs_acpihid%s option format deprecated; use ivrs_acpihid=%s@%04x:%02x:%02x.%d instead\n",
+ str, acpiid, seg, bus, dev, fn);
+ goto found;
}
+ goto not_found;
}
+ /* We have the '@', make it the terminator to get just the acpiid */
+ *addr++ = 0;
+
+ if (sscanf(str, "=%s", acpiid) != 1)
+ goto not_found;
+
+ if (sscanf(addr, "%x:%x.%x", &bus, &dev, &fn) == 3 ||
+ sscanf(addr, "%x:%x:%x.%x", &seg, &bus, &dev, &fn) == 4)
+ goto found;
+
+not_found:
+ pr_err("Invalid command line: ivrs_acpihid%s\n", str);
+ return 1;
+
+found:
p = acpiid;
hid = strsep(&p, ":");
uid = p;
return 1;
}
+ /*
+ * Ignore leading zeroes after ':', so e.g., AMDI0095:00
+ * will match AMDI0095:0 in the second strcmp in acpi_dev_hid_uid_match
+ */
+ while (*uid == '0' && *(uid + 1))
+ uid++;
+
i = early_acpihid_map_size++;
memcpy(early_acpihid_map[i].hid, hid, strlen(hid));
memcpy(early_acpihid_map[i].uid, uid, strlen(uid));
static void iommu_poll_ga_log(struct amd_iommu *iommu)
{
- u32 head, tail, cnt = 0;
+ u32 head, tail;
if (iommu->ga_log == NULL)
return;
u64 log_entry;
raw = (u64 *)(iommu->ga_log + head);
- cnt++;
/* Avoid memcpy function-call overhead */
log_entry = *raw;
put_device_state(dev_state);
out:
+ pci_dev_put(pdev);
return ret;
}
if (pasid_state->mm == NULL)
goto out_free;
- mmu_notifier_register(&pasid_state->mn, mm);
+ ret = mmu_notifier_register(&pasid_state->mn, mm);
+ if (ret)
+ goto out_free;
ret = set_pasid_state(dev_state, pasid_state, pasid);
if (ret)
reg = arm_smmu_cb_read(smmu, i, ARM_SMMU_CB_ACTLR);
reg &= ~ARM_MMU500_ACTLR_CPRE;
arm_smmu_cb_write(smmu, i, ARM_SMMU_CB_ACTLR, reg);
+ reg = arm_smmu_cb_read(smmu, i, ARM_SMMU_CB_ACTLR);
+ if (reg & ARM_MMU500_ACTLR_CPRE)
+ dev_warn_once(smmu->dev, "Failed to disable prefetcher [errata #841119 and #826419], check ACR.CACHE_LOCK\n");
}
return 0;
#include "arm-smmu.h"
#include "arm-smmu-qcom.h"
-enum qcom_smmu_impl_reg_offset {
- QCOM_SMMU_TBU_PWR_STATUS,
- QCOM_SMMU_STATS_SYNC_INV_TBU_ACK,
- QCOM_SMMU_MMU2QSS_AND_SAFE_WAIT_CNTR,
-};
-
-struct qcom_smmu_config {
- const u32 *reg_offset;
-};
-
void qcom_smmu_tlb_sync_debug(struct arm_smmu_device *smmu)
{
int ret;
tbu_pwr_status, sync_inv_ack, sync_inv_progress);
}
}
-
-/* Implementation Defined Register Space 0 register offsets */
-static const u32 qcom_smmu_impl0_reg_offset[] = {
- [QCOM_SMMU_TBU_PWR_STATUS] = 0x2204,
- [QCOM_SMMU_STATS_SYNC_INV_TBU_ACK] = 0x25dc,
- [QCOM_SMMU_MMU2QSS_AND_SAFE_WAIT_CNTR] = 0x2670,
-};
-
-static const struct qcom_smmu_config qcm2290_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct qcom_smmu_config sc7180_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct qcom_smmu_config sc7280_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct qcom_smmu_config sc8180x_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct qcom_smmu_config sc8280xp_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct qcom_smmu_config sm6125_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct qcom_smmu_config sm6350_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct qcom_smmu_config sm8150_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct qcom_smmu_config sm8250_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct qcom_smmu_config sm8350_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct qcom_smmu_config sm8450_smmu_cfg = {
- .reg_offset = qcom_smmu_impl0_reg_offset,
-};
-
-static const struct of_device_id __maybe_unused qcom_smmu_impl_debug_match[] = {
- { .compatible = "qcom,msm8998-smmu-v2" },
- { .compatible = "qcom,qcm2290-smmu-500", .data = &qcm2290_smmu_cfg },
- { .compatible = "qcom,sc7180-smmu-500", .data = &sc7180_smmu_cfg },
- { .compatible = "qcom,sc7280-smmu-500", .data = &sc7280_smmu_cfg},
- { .compatible = "qcom,sc8180x-smmu-500", .data = &sc8180x_smmu_cfg },
- { .compatible = "qcom,sc8280xp-smmu-500", .data = &sc8280xp_smmu_cfg },
- { .compatible = "qcom,sdm630-smmu-v2" },
- { .compatible = "qcom,sdm845-smmu-500" },
- { .compatible = "qcom,sm6125-smmu-500", .data = &sm6125_smmu_cfg},
- { .compatible = "qcom,sm6350-smmu-500", .data = &sm6350_smmu_cfg},
- { .compatible = "qcom,sm8150-smmu-500", .data = &sm8150_smmu_cfg },
- { .compatible = "qcom,sm8250-smmu-500", .data = &sm8250_smmu_cfg },
- { .compatible = "qcom,sm8350-smmu-500", .data = &sm8350_smmu_cfg },
- { .compatible = "qcom,sm8450-smmu-500", .data = &sm8450_smmu_cfg },
- { }
-};
-
-const void *qcom_smmu_impl_data(struct arm_smmu_device *smmu)
-{
- const struct of_device_id *match;
- const struct device_node *np = smmu->dev->of_node;
-
- match = of_match_node(qcom_smmu_impl_debug_match, np);
- if (!match)
- return NULL;
-
- return match->data;
-}
{
int ret;
+ arm_mmu500_reset(smmu);
+
/*
* To address performance degradation in non-real time clients,
* such as USB and UFS, turn off wait-for-safe on sdm845 based boards,
return ret;
}
-static int qcom_smmu500_reset(struct arm_smmu_device *smmu)
-{
- const struct device_node *np = smmu->dev->of_node;
-
- arm_mmu500_reset(smmu);
-
- if (of_device_is_compatible(np, "qcom,sdm845-smmu-500"))
- return qcom_sdm845_smmu500_reset(smmu);
+static const struct arm_smmu_impl qcom_smmu_v2_impl = {
+ .init_context = qcom_smmu_init_context,
+ .cfg_probe = qcom_smmu_cfg_probe,
+ .def_domain_type = qcom_smmu_def_domain_type,
+ .write_s2cr = qcom_smmu_write_s2cr,
+ .tlb_sync = qcom_smmu_tlb_sync,
+};
- return 0;
-}
+static const struct arm_smmu_impl qcom_smmu_500_impl = {
+ .init_context = qcom_smmu_init_context,
+ .cfg_probe = qcom_smmu_cfg_probe,
+ .def_domain_type = qcom_smmu_def_domain_type,
+ .reset = arm_mmu500_reset,
+ .write_s2cr = qcom_smmu_write_s2cr,
+ .tlb_sync = qcom_smmu_tlb_sync,
+};
-static const struct arm_smmu_impl qcom_smmu_impl = {
+static const struct arm_smmu_impl sdm845_smmu_500_impl = {
.init_context = qcom_smmu_init_context,
.cfg_probe = qcom_smmu_cfg_probe,
.def_domain_type = qcom_smmu_def_domain_type,
- .reset = qcom_smmu500_reset,
+ .reset = qcom_sdm845_smmu500_reset,
.write_s2cr = qcom_smmu_write_s2cr,
.tlb_sync = qcom_smmu_tlb_sync,
};
-static const struct arm_smmu_impl qcom_adreno_smmu_impl = {
+static const struct arm_smmu_impl qcom_adreno_smmu_v2_impl = {
+ .init_context = qcom_adreno_smmu_init_context,
+ .def_domain_type = qcom_smmu_def_domain_type,
+ .alloc_context_bank = qcom_adreno_smmu_alloc_context_bank,
+ .write_sctlr = qcom_adreno_smmu_write_sctlr,
+ .tlb_sync = qcom_smmu_tlb_sync,
+};
+
+static const struct arm_smmu_impl qcom_adreno_smmu_500_impl = {
.init_context = qcom_adreno_smmu_init_context,
.def_domain_type = qcom_smmu_def_domain_type,
- .reset = qcom_smmu500_reset,
+ .reset = arm_mmu500_reset,
.alloc_context_bank = qcom_adreno_smmu_alloc_context_bank,
.write_sctlr = qcom_adreno_smmu_write_sctlr,
.tlb_sync = qcom_smmu_tlb_sync,
};
static struct arm_smmu_device *qcom_smmu_create(struct arm_smmu_device *smmu,
- const struct arm_smmu_impl *impl)
+ const struct qcom_smmu_match_data *data)
{
+ const struct device_node *np = smmu->dev->of_node;
+ const struct arm_smmu_impl *impl;
struct qcom_smmu *qsmmu;
+ if (!data)
+ return ERR_PTR(-EINVAL);
+
+ if (np && of_device_is_compatible(np, "qcom,adreno-smmu"))
+ impl = data->adreno_impl;
+ else
+ impl = data->impl;
+
+ if (!impl)
+ return smmu;
+
/* Check to make sure qcom_scm has finished probing */
if (!qcom_scm_is_available())
return ERR_PTR(-EPROBE_DEFER);
return ERR_PTR(-ENOMEM);
qsmmu->smmu.impl = impl;
- qsmmu->cfg = qcom_smmu_impl_data(smmu);
+ qsmmu->cfg = data->cfg;
return &qsmmu->smmu;
}
+/* Implementation Defined Register Space 0 register offsets */
+static const u32 qcom_smmu_impl0_reg_offset[] = {
+ [QCOM_SMMU_TBU_PWR_STATUS] = 0x2204,
+ [QCOM_SMMU_STATS_SYNC_INV_TBU_ACK] = 0x25dc,
+ [QCOM_SMMU_MMU2QSS_AND_SAFE_WAIT_CNTR] = 0x2670,
+};
+
+static const struct qcom_smmu_config qcom_smmu_impl0_cfg = {
+ .reg_offset = qcom_smmu_impl0_reg_offset,
+};
+
+/*
+ * It is not yet possible to use MDP SMMU with the bypass quirk on the msm8996,
+ * there are not enough context banks.
+ */
+static const struct qcom_smmu_match_data msm8996_smmu_data = {
+ .impl = NULL,
+ .adreno_impl = &qcom_adreno_smmu_v2_impl,
+};
+
+static const struct qcom_smmu_match_data qcom_smmu_v2_data = {
+ .impl = &qcom_smmu_v2_impl,
+ .adreno_impl = &qcom_adreno_smmu_v2_impl,
+};
+
+static const struct qcom_smmu_match_data sdm845_smmu_500_data = {
+ .impl = &sdm845_smmu_500_impl,
+ /*
+ * No need for adreno impl here. On sdm845 the Adreno SMMU is handled
+ * by the separate sdm845-smmu-v2 device.
+ */
+ /* Also no debug configuration. */
+};
+
+static const struct qcom_smmu_match_data qcom_smmu_500_impl0_data = {
+ .impl = &qcom_smmu_500_impl,
+ .adreno_impl = &qcom_adreno_smmu_500_impl,
+ .cfg = &qcom_smmu_impl0_cfg,
+};
+
+/*
+ * Do not add any more qcom,SOC-smmu-500 entries to this list, unless they need
+ * special handling and can not be covered by the qcom,smmu-500 entry.
+ */
static const struct of_device_id __maybe_unused qcom_smmu_impl_of_match[] = {
- { .compatible = "qcom,msm8998-smmu-v2" },
- { .compatible = "qcom,qcm2290-smmu-500" },
- { .compatible = "qcom,sc7180-smmu-500" },
- { .compatible = "qcom,sc7280-smmu-500" },
- { .compatible = "qcom,sc8180x-smmu-500" },
- { .compatible = "qcom,sc8280xp-smmu-500" },
- { .compatible = "qcom,sdm630-smmu-v2" },
- { .compatible = "qcom,sdm845-smmu-500" },
- { .compatible = "qcom,sm6125-smmu-500" },
- { .compatible = "qcom,sm6350-smmu-500" },
- { .compatible = "qcom,sm6375-smmu-500" },
- { .compatible = "qcom,sm8150-smmu-500" },
- { .compatible = "qcom,sm8250-smmu-500" },
- { .compatible = "qcom,sm8350-smmu-500" },
- { .compatible = "qcom,sm8450-smmu-500" },
+ { .compatible = "qcom,msm8996-smmu-v2", .data = &msm8996_smmu_data },
+ { .compatible = "qcom,msm8998-smmu-v2", .data = &qcom_smmu_v2_data },
+ { .compatible = "qcom,qcm2290-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,qdu1000-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sc7180-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sc7280-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sc8180x-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sc8280xp-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sdm630-smmu-v2", .data = &qcom_smmu_v2_data },
+ { .compatible = "qcom,sdm845-smmu-v2", .data = &qcom_smmu_v2_data },
+ { .compatible = "qcom,sdm845-smmu-500", .data = &sdm845_smmu_500_data },
+ { .compatible = "qcom,sm6115-smmu-500", .data = &qcom_smmu_500_impl0_data},
+ { .compatible = "qcom,sm6125-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sm6350-smmu-v2", .data = &qcom_smmu_v2_data },
+ { .compatible = "qcom,sm6350-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sm6375-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sm8150-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sm8250-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sm8350-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,sm8450-smmu-500", .data = &qcom_smmu_500_impl0_data },
+ { .compatible = "qcom,smmu-500", .data = &qcom_smmu_500_impl0_data },
{ }
};
struct arm_smmu_device *qcom_smmu_impl_init(struct arm_smmu_device *smmu)
{
const struct device_node *np = smmu->dev->of_node;
+ const struct of_device_id *match;
#ifdef CONFIG_ACPI
if (np == NULL) {
/* Match platform for ACPI boot */
if (acpi_match_platform_list(qcom_acpi_platlist) >= 0)
- return qcom_smmu_create(smmu, &qcom_smmu_impl);
+ return qcom_smmu_create(smmu, &qcom_smmu_500_impl0_data);
}
#endif
- /*
- * Do not change this order of implementation, i.e., first adreno
- * smmu impl and then apss smmu since we can have both implementing
- * arm,mmu-500 in which case we will miss setting adreno smmu specific
- * features if the order is changed.
- */
- if (of_device_is_compatible(np, "qcom,adreno-smmu"))
- return qcom_smmu_create(smmu, &qcom_adreno_smmu_impl);
-
- if (of_match_node(qcom_smmu_impl_of_match, np))
- return qcom_smmu_create(smmu, &qcom_smmu_impl);
+ match = of_match_node(qcom_smmu_impl_of_match, np);
+ if (match)
+ return qcom_smmu_create(smmu, match->data);
return smmu;
}
u32 stall_enabled;
};
+enum qcom_smmu_impl_reg_offset {
+ QCOM_SMMU_TBU_PWR_STATUS,
+ QCOM_SMMU_STATS_SYNC_INV_TBU_ACK,
+ QCOM_SMMU_MMU2QSS_AND_SAFE_WAIT_CNTR,
+};
+
+struct qcom_smmu_config {
+ const u32 *reg_offset;
+};
+
+struct qcom_smmu_match_data {
+ const struct qcom_smmu_config *cfg;
+ const struct arm_smmu_impl *impl;
+ const struct arm_smmu_impl *adreno_impl;
+};
+
#ifdef CONFIG_ARM_SMMU_QCOM_DEBUG
void qcom_smmu_tlb_sync_debug(struct arm_smmu_device *smmu);
-const void *qcom_smmu_impl_data(struct arm_smmu_device *smmu);
#else
static inline void qcom_smmu_tlb_sync_debug(struct arm_smmu_device *smmu) { }
-static inline const void *qcom_smmu_impl_data(struct arm_smmu_device *smmu)
-{
- return NULL;
-}
#endif
#endif /* _ARM_SMMU_QCOM_H */
}
static int qcom_iommu_map(struct iommu_domain *domain, unsigned long iova,
- phys_addr_t paddr, size_t size, int prot, gfp_t gfp)
+ phys_addr_t paddr, size_t pgsize, size_t pgcount,
+ int prot, gfp_t gfp, size_t *mapped)
{
int ret;
unsigned long flags;
return -ENODEV;
spin_lock_irqsave(&qcom_domain->pgtbl_lock, flags);
- ret = ops->map(ops, iova, paddr, size, prot, GFP_ATOMIC);
+ ret = ops->map_pages(ops, iova, paddr, pgsize, pgcount, prot, GFP_ATOMIC, mapped);
spin_unlock_irqrestore(&qcom_domain->pgtbl_lock, flags);
return ret;
}
static size_t qcom_iommu_unmap(struct iommu_domain *domain, unsigned long iova,
- size_t size, struct iommu_iotlb_gather *gather)
+ size_t pgsize, size_t pgcount,
+ struct iommu_iotlb_gather *gather)
{
size_t ret;
unsigned long flags;
*/
pm_runtime_get_sync(qcom_domain->iommu->dev);
spin_lock_irqsave(&qcom_domain->pgtbl_lock, flags);
- ret = ops->unmap(ops, iova, size, gather);
+ ret = ops->unmap_pages(ops, iova, pgsize, pgcount, gather);
spin_unlock_irqrestore(&qcom_domain->pgtbl_lock, flags);
pm_runtime_put_sync(qcom_domain->iommu->dev);
.default_domain_ops = &(const struct iommu_domain_ops) {
.attach_dev = qcom_iommu_attach_dev,
.detach_dev = qcom_iommu_detach_dev,
- .map = qcom_iommu_map,
- .unmap = qcom_iommu_unmap,
+ .map_pages = qcom_iommu_map,
+ .unmap_pages = qcom_iommu_unmap,
.flush_iotlb_all = qcom_iommu_flush_iotlb_all,
.iotlb_sync = qcom_iommu_iotlb_sync,
.iova_to_phys = qcom_iommu_iova_to_phys,
if (ret)
return ret;
- ret = iommu_device_register(&data->iommu, &exynos_iommu_ops, dev);
- if (ret)
- goto err_iommu_register;
-
platform_set_drvdata(pdev, data);
if (PG_ENT_SHIFT < 0) {
pm_runtime_enable(dev);
+ ret = iommu_device_register(&data->iommu, &exynos_iommu_ops, dev);
+ if (ret)
+ goto err_dma_set_mask;
+
return 0;
err_dma_set_mask:
- iommu_device_unregister(&data->iommu);
-err_iommu_register:
iommu_device_sysfs_remove(&data->iommu);
return ret;
}
return -ENOMEM;
}
- ret = platform_driver_register(&exynos_sysmmu_driver);
- if (ret) {
- pr_err("%s: Failed to register driver\n", __func__);
- goto err_reg_driver;
- }
-
zero_lv2_table = kmem_cache_zalloc(lv2table_kmem_cache, GFP_KERNEL);
if (zero_lv2_table == NULL) {
pr_err("%s: Failed to allocate zero level2 page table\n",
goto err_zero_lv2;
}
+ ret = platform_driver_register(&exynos_sysmmu_driver);
+ if (ret) {
+ pr_err("%s: Failed to register driver\n", __func__);
+ goto err_reg_driver;
+ }
+
return 0;
-err_zero_lv2:
- platform_driver_unregister(&exynos_sysmmu_driver);
err_reg_driver:
+ platform_driver_unregister(&exynos_sysmmu_driver);
+err_zero_lv2:
kmem_cache_destroy(lv2table_kmem_cache);
return ret;
}
of_get_address(dev->of_node, 0, &size, NULL);
irq = irq_of_parse_and_map(dev->of_node, 0);
- if (irq == NO_IRQ) {
+ if (!irq) {
dev_warn(dev, "no interrupts listed in PAMU node\n");
goto error;
}
ret = create_csd(ppaact_phys, mem_size, csd_port_id);
if (ret) {
dev_err(dev, "could not create coherence subdomain\n");
- return ret;
+ goto error;
}
}
return 0;
error:
- if (irq != NO_IRQ)
+ if (irq)
free_irq(irq, data);
kfree_sensitive(data);
#define for_each_rmrr_units(rmrr) \
list_for_each_entry(rmrr, &dmar_rmrr_units, list)
-static void dmar_remove_one_dev_info(struct device *dev);
+static void device_block_translation(struct device *dev);
+static void intel_iommu_domain_free(struct iommu_domain *domain);
int dmar_disabled = !IS_ENABLED(CONFIG_INTEL_IOMMU_DEFAULT_ON);
int intel_iommu_sm = IS_ENABLED(CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON);
return domain->domain.type == IOMMU_DOMAIN_IDENTITY;
}
-static inline bool domain_use_first_level(struct dmar_domain *domain)
-{
- return domain->flags & DOMAIN_FLAG_USE_FIRST_LEVEL;
-}
-
static inline int domain_pfn_supported(struct dmar_domain *domain,
unsigned long pfn)
{
rcu_read_lock();
for_each_active_iommu(iommu, drhd) {
if (iommu != skip) {
- if (domain && domain_use_first_level(domain)) {
+ if (domain && domain->use_first_level) {
if (!cap_fl1gp_support(iommu->cap))
mask = 0x1;
} else {
* paging and 57-bits with 5-level paging). Hence, skip bit
* [N-1].
*/
- if (domain_use_first_level(domain))
+ if (domain->use_first_level)
domain->domain.geometry.aperture_end = __DOMAIN_MAX_ADDR(domain->gaw - 1);
else
domain->domain.geometry.aperture_end = __DOMAIN_MAX_ADDR(domain->gaw);
clflush_cache_range(addr, size);
}
-static int device_context_mapped(struct intel_iommu *iommu, u8 bus, u8 devfn)
-{
- struct context_entry *context;
- int ret = 0;
-
- spin_lock(&iommu->lock);
- context = iommu_context_addr(iommu, bus, devfn, 0);
- if (context)
- ret = context_present(context);
- spin_unlock(&iommu->lock);
- return ret;
-}
-
static void free_context_table(struct intel_iommu *iommu)
{
struct context_entry *context;
domain_flush_cache(domain, tmp_page, VTD_PAGE_SIZE);
pteval = ((uint64_t)virt_to_dma_pfn(tmp_page) << VTD_PAGE_SHIFT) | DMA_PTE_READ | DMA_PTE_WRITE;
- if (domain_use_first_level(domain))
+ if (domain->use_first_level)
pteval |= DMA_FL_PTE_XD | DMA_FL_PTE_US | DMA_FL_PTE_ACCESS;
if (cmpxchg64(&pte->val, 0ULL, pteval))
{
struct pci_dev *pdev;
- if (!info || !dev_is_pci(info->dev))
+ if (!dev_is_pci(info->dev))
return;
pdev = to_pci_dev(info->dev);
}
}
-static void iommu_disable_dev_iotlb(struct device_domain_info *info)
+static void iommu_disable_pci_caps(struct device_domain_info *info)
{
struct pci_dev *pdev;
if (ih)
ih = 1 << 6;
- if (domain_use_first_level(domain)) {
+ if (domain->use_first_level) {
qi_flush_piotlb(iommu, did, PASID_RID2PASID, addr, pages, ih);
} else {
unsigned long bitmask = aligned_pages - 1;
* It's a non-present to present mapping. Only flush if caching mode
* and second level.
*/
- if (cap_caching_mode(iommu->cap) && !domain_use_first_level(domain))
+ if (cap_caching_mode(iommu->cap) && !domain->use_first_level)
iommu_flush_iotlb_psi(iommu, domain, pfn, pages, 0, 1);
else
iommu_flush_write_buffer(iommu);
struct intel_iommu *iommu = info->iommu;
u16 did = domain_id_iommu(dmar_domain, iommu);
- if (domain_use_first_level(dmar_domain))
+ if (dmar_domain->use_first_level)
qi_flush_piotlb(iommu, did, PASID_RID2PASID, 0, -1, 0);
else
iommu->flush.flush_iotlb(iommu, did, 0, 0,
domain->nid = NUMA_NO_NODE;
if (first_level_by_default(type))
- domain->flags |= DOMAIN_FLAG_USE_FIRST_LEVEL;
+ domain->use_first_level = true;
domain->has_iotlb_device = false;
INIT_LIST_HEAD(&domain->devices);
spin_lock_init(&domain->lock);
} else {
iommu_flush_write_buffer(iommu);
}
- iommu_enable_pci_caps(info);
ret = 0;
&domain_context_mapping_cb, &data);
}
-static int domain_context_mapped_cb(struct pci_dev *pdev,
- u16 alias, void *opaque)
-{
- struct intel_iommu *iommu = opaque;
-
- return !device_context_mapped(iommu, PCI_BUS_NUM(alias), alias & 0xff);
-}
-
-static int domain_context_mapped(struct device *dev)
-{
- struct intel_iommu *iommu;
- u8 bus, devfn;
-
- iommu = device_to_iommu(dev, &bus, &devfn);
- if (!iommu)
- return -ENODEV;
-
- if (!dev_is_pci(dev))
- return device_context_mapped(iommu, bus, devfn);
-
- return !pci_for_each_dma_alias(to_pci_dev(dev),
- domain_context_mapped_cb, iommu);
-}
-
/* Returns a number of VTD pages, but aligned to MM page size */
static inline unsigned long aligned_nrpages(unsigned long host_addr,
size_t size)
attr = prot & (DMA_PTE_READ | DMA_PTE_WRITE | DMA_PTE_SNP);
attr |= DMA_FL_PTE_PRESENT;
- if (domain_use_first_level(domain)) {
+ if (domain->use_first_level) {
attr |= DMA_FL_PTE_XD | DMA_FL_PTE_US | DMA_FL_PTE_ACCESS;
if (prot & DMA_PTE_WRITE)
attr |= DMA_FL_PTE_DIRTY;
return 0;
}
-static int domain_add_dev_info(struct dmar_domain *domain, struct device *dev)
+static int dmar_domain_attach_device(struct dmar_domain *domain,
+ struct device *dev)
{
struct device_domain_info *info = dev_iommu_priv_get(dev);
struct intel_iommu *iommu;
/* PASID table is mandatory for a PCI device in scalable mode. */
if (sm_supported(iommu) && !dev_is_real_dma_subdevice(dev)) {
- ret = intel_pasid_alloc_table(dev);
- if (ret) {
- dev_err(dev, "PASID table allocation failed\n");
- dmar_remove_one_dev_info(dev);
- return ret;
- }
-
/* Setup the PASID entry for requests without PASID: */
if (hw_pass_through && domain_type_is_si(domain))
ret = intel_pasid_setup_pass_through(iommu, domain,
dev, PASID_RID2PASID);
- else if (domain_use_first_level(domain))
+ else if (domain->use_first_level)
ret = domain_setup_first_level(iommu, domain, dev,
PASID_RID2PASID);
else
dev, PASID_RID2PASID);
if (ret) {
dev_err(dev, "Setup RID2PASID failed\n");
- dmar_remove_one_dev_info(dev);
+ device_block_translation(dev);
return ret;
}
}
ret = domain_context_mapping(domain, dev);
if (ret) {
dev_err(dev, "Domain context map failed\n");
- dmar_remove_one_dev_info(dev);
+ device_block_translation(dev);
return ret;
}
+ iommu_enable_pci_caps(info);
+
return 0;
}
intel_pasid_tear_down_entry(iommu, info->dev,
PASID_RID2PASID, false);
- iommu_disable_dev_iotlb(info);
+ iommu_disable_pci_caps(info);
domain_context_clear(info);
- intel_pasid_free_table(info->dev);
}
spin_lock_irqsave(&domain->lock, flags);
info->domain = NULL;
}
+/*
+ * Clear the page table pointer in context or pasid table entries so that
+ * all DMA requests without PASID from the device are blocked. If the page
+ * table has been set, clean up the data structures.
+ */
+static void device_block_translation(struct device *dev)
+{
+ struct device_domain_info *info = dev_iommu_priv_get(dev);
+ struct intel_iommu *iommu = info->iommu;
+ unsigned long flags;
+
+ iommu_disable_pci_caps(info);
+ if (!dev_is_real_dma_subdevice(dev)) {
+ if (sm_supported(iommu))
+ intel_pasid_tear_down_entry(iommu, dev,
+ PASID_RID2PASID, false);
+ else
+ domain_context_clear(info);
+ }
+
+ if (!info->domain)
+ return;
+
+ spin_lock_irqsave(&info->domain->lock, flags);
+ list_del(&info->link);
+ spin_unlock_irqrestore(&info->domain->lock, flags);
+
+ domain_detach_iommu(info->domain, iommu);
+ info->domain = NULL;
+}
+
static int md_domain_init(struct dmar_domain *domain, int guest_width)
{
int adjust_width;
return 0;
}
+static int blocking_domain_attach_dev(struct iommu_domain *domain,
+ struct device *dev)
+{
+ device_block_translation(dev);
+ return 0;
+}
+
+static struct iommu_domain blocking_domain = {
+ .ops = &(const struct iommu_domain_ops) {
+ .attach_dev = blocking_domain_attach_dev,
+ .free = intel_iommu_domain_free
+ }
+};
+
static struct iommu_domain *intel_iommu_domain_alloc(unsigned type)
{
struct dmar_domain *dmar_domain;
struct iommu_domain *domain;
switch (type) {
+ case IOMMU_DOMAIN_BLOCKED:
+ return &blocking_domain;
case IOMMU_DOMAIN_DMA:
case IOMMU_DOMAIN_DMA_FQ:
case IOMMU_DOMAIN_UNMANAGED:
static void intel_iommu_domain_free(struct iommu_domain *domain)
{
- if (domain != &si_domain->domain)
+ if (domain != &si_domain->domain && domain != &blocking_domain)
domain_exit(to_dmar_domain(domain));
}
static int intel_iommu_attach_device(struct iommu_domain *domain,
struct device *dev)
{
+ struct device_domain_info *info = dev_iommu_priv_get(dev);
int ret;
if (domain->type == IOMMU_DOMAIN_UNMANAGED &&
return -EPERM;
}
- /* normally dev is not mapped */
- if (unlikely(domain_context_mapped(dev))) {
- struct device_domain_info *info = dev_iommu_priv_get(dev);
-
- if (info->domain)
- dmar_remove_one_dev_info(dev);
- }
+ if (info->domain)
+ device_block_translation(dev);
ret = prepare_domain_attach_device(domain, dev);
if (ret)
return ret;
- return domain_add_dev_info(to_dmar_domain(domain), dev);
-}
-
-static void intel_iommu_detach_device(struct iommu_domain *domain,
- struct device *dev)
-{
- dmar_remove_one_dev_info(dev);
+ return dmar_domain_attach_device(to_dmar_domain(domain), dev);
}
static int intel_iommu_map(struct iommu_domain *domain,
* Second level page table supports per-PTE snoop control. The
* iommu_map() interface will handle this by setting SNP bit.
*/
- if (!domain_use_first_level(domain)) {
+ if (!domain->use_first_level) {
domain->set_pte_snp = true;
return;
}
struct device_domain_info *info;
struct intel_iommu *iommu;
u8 bus, devfn;
+ int ret;
iommu = device_to_iommu(dev, &bus, &devfn);
if (!iommu || !iommu->iommu.ops)
dev_iommu_priv_set(dev, info);
+ if (sm_supported(iommu) && !dev_is_real_dma_subdevice(dev)) {
+ ret = intel_pasid_alloc_table(dev);
+ if (ret) {
+ dev_err(dev, "PASID table allocation failed\n");
+ dev_iommu_priv_set(dev, NULL);
+ kfree(info);
+ return ERR_PTR(ret);
+ }
+ }
+
return &iommu->iommu;
}
struct device_domain_info *info = dev_iommu_priv_get(dev);
dmar_remove_one_dev_info(dev);
+ intel_pasid_free_table(dev);
dev_iommu_priv_set(dev, NULL);
kfree(info);
set_dma_ops(dev, NULL);
#endif
.default_domain_ops = &(const struct iommu_domain_ops) {
.attach_dev = intel_iommu_attach_device,
- .detach_dev = intel_iommu_detach_device,
.map_pages = intel_iommu_map_pages,
.unmap_pages = intel_iommu_unmap_pages,
.iotlb_sync_map = intel_iommu_iotlb_sync_map,
u64 hi;
};
-/*
- * When VT-d works in the scalable mode, it allows DMA translation to
- * happen through either first level or second level page table. This
- * bit marks that the DMA translation for the domain goes through the
- * first level page table, otherwise, it goes through the second level.
- */
-#define DOMAIN_FLAG_USE_FIRST_LEVEL BIT(1)
-
struct iommu_domain_info {
struct intel_iommu *iommu;
unsigned int refcnt; /* Refcount of devices per iommu */
u8 iommu_coherency: 1; /* indicate coherency of iommu access */
u8 force_snooping : 1; /* Create IOPTEs with snoop control */
u8 set_pte_snp:1;
+ u8 use_first_level:1; /* DMA translation for the domain goes
+ * through the first level page table,
+ * otherwise, goes through the second
+ * level.
+ */
spinlock_t lock; /* Protect device tracking lists */
struct list_head devices; /* all devices' list */
/* adjusted guest address width, 0 is level 2 30-bit */
int agaw;
-
- int flags; /* flags to find out type of domain */
int iommu_superpage;/* Level of superpages supported:
0 == 4KiB (no superpages), 1 == 2MiB,
2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
index = irq_iommu->irte_index + irq_iommu->sub_handle;
irte = &iommu->ir_table->base[index];
-#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE)
if ((irte->pst == 1) || (irte_modified->pst == 1)) {
bool ret;
* same as the old value.
*/
WARN_ON(!ret);
- } else
-#endif
- {
- set_64bit(&irte->low, irte_modified->low);
- set_64bit(&irte->high, irte_modified->high);
+ } else {
+ WRITE_ONCE(irte->low, irte_modified->low);
+ WRITE_ONCE(irte->high, irte_modified->high);
}
__iommu_flush_cache(iommu, irte, sizeof(*irte));
end = start + (1 << irq_iommu->irte_mask);
for (entry = start; entry < end; entry++) {
- set_64bit(&entry->low, 0);
- set_64bit(&entry->high, 0);
+ WRITE_ONCE(entry->low, 0);
+ WRITE_ONCE(entry->high, 0);
}
bitmap_release_region(iommu->ir_table->bitmap, index,
irq_iommu->irte_mask);
iova += pgsize;
paddr += pgsize;
- if (mapped)
- *mapped += pgsize;
+ *mapped += pgsize;
}
/*
* Synchronise all PTE updates for the new mapping before there's
return ret;
}
-static int arm_v7s_map(struct io_pgtable_ops *ops, unsigned long iova,
- phys_addr_t paddr, size_t size, int prot, gfp_t gfp)
-{
- return arm_v7s_map_pages(ops, iova, paddr, size, 1, prot, gfp, NULL);
-}
-
static void arm_v7s_free_pgtable(struct io_pgtable *iop)
{
struct arm_v7s_io_pgtable *data = io_pgtable_to_data(iop);
return unmapped;
}
-static size_t arm_v7s_unmap(struct io_pgtable_ops *ops, unsigned long iova,
- size_t size, struct iommu_iotlb_gather *gather)
-{
- return arm_v7s_unmap_pages(ops, iova, size, 1, gather);
-}
-
static phys_addr_t arm_v7s_iova_to_phys(struct io_pgtable_ops *ops,
unsigned long iova)
{
goto out_free_data;
data->iop.ops = (struct io_pgtable_ops) {
- .map = arm_v7s_map,
.map_pages = arm_v7s_map_pages,
- .unmap = arm_v7s_unmap,
.unmap_pages = arm_v7s_unmap_pages,
.iova_to_phys = arm_v7s_iova_to_phys,
};
};
unsigned int iova, size, iova_start;
unsigned int i, loopnr = 0;
+ size_t mapped;
selftest_running = true;
iova = 0;
for_each_set_bit(i, &cfg.pgsize_bitmap, BITS_PER_LONG) {
size = 1UL << i;
- if (ops->map(ops, iova, iova, size, IOMMU_READ |
- IOMMU_WRITE |
- IOMMU_NOEXEC |
- IOMMU_CACHE, GFP_KERNEL))
+ if (ops->map_pages(ops, iova, iova, size, 1,
+ IOMMU_READ | IOMMU_WRITE |
+ IOMMU_NOEXEC | IOMMU_CACHE,
+ GFP_KERNEL, &mapped))
return __FAIL(ops);
/* Overlapping mappings */
- if (!ops->map(ops, iova, iova + size, size,
- IOMMU_READ | IOMMU_NOEXEC, GFP_KERNEL))
+ if (!ops->map_pages(ops, iova, iova + size, size, 1,
+ IOMMU_READ | IOMMU_NOEXEC, GFP_KERNEL,
+ &mapped))
return __FAIL(ops);
if (ops->iova_to_phys(ops, iova + 42) != (iova + 42))
size = 1UL << __ffs(cfg.pgsize_bitmap);
while (i < loopnr) {
iova_start = i * SZ_16M;
- if (ops->unmap(ops, iova_start + size, size, NULL) != size)
+ if (ops->unmap_pages(ops, iova_start + size, size, 1, NULL) != size)
return __FAIL(ops);
/* Remap of partial unmap */
- if (ops->map(ops, iova_start + size, size, size, IOMMU_READ, GFP_KERNEL))
+ if (ops->map_pages(ops, iova_start + size, size, size, 1,
+ IOMMU_READ, GFP_KERNEL, &mapped))
return __FAIL(ops);
if (ops->iova_to_phys(ops, iova_start + size + 42)
for_each_set_bit(i, &cfg.pgsize_bitmap, BITS_PER_LONG) {
size = 1UL << i;
- if (ops->unmap(ops, iova, size, NULL) != size)
+ if (ops->unmap_pages(ops, iova, size, 1, NULL) != size)
return __FAIL(ops);
if (ops->iova_to_phys(ops, iova + 42))
return __FAIL(ops);
/* Remap full block */
- if (ops->map(ops, iova, iova, size, IOMMU_WRITE, GFP_KERNEL))
+ if (ops->map_pages(ops, iova, iova, size, 1, IOMMU_WRITE,
+ GFP_KERNEL, &mapped))
return __FAIL(ops);
if (ops->iova_to_phys(ops, iova + 42) != (iova + 42))
max_entries = ARM_LPAE_PTES_PER_TABLE(data) - map_idx_start;
num_entries = min_t(int, pgcount, max_entries);
ret = arm_lpae_init_pte(data, iova, paddr, prot, lvl, num_entries, ptep);
- if (!ret && mapped)
+ if (!ret)
*mapped += num_entries * size;
return ret;
return ret;
}
-static int arm_lpae_map(struct io_pgtable_ops *ops, unsigned long iova,
- phys_addr_t paddr, size_t size, int iommu_prot, gfp_t gfp)
-{
- return arm_lpae_map_pages(ops, iova, paddr, size, 1, iommu_prot, gfp,
- NULL);
-}
-
static void __arm_lpae_free_pgtable(struct arm_lpae_io_pgtable *data, int lvl,
arm_lpae_iopte *ptep)
{
data->start_level, ptep);
}
-static size_t arm_lpae_unmap(struct io_pgtable_ops *ops, unsigned long iova,
- size_t size, struct iommu_iotlb_gather *gather)
-{
- return arm_lpae_unmap_pages(ops, iova, size, 1, gather);
-}
-
static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops,
unsigned long iova)
{
data->pgd_bits = va_bits - (data->bits_per_level * (levels - 1));
data->iop.ops = (struct io_pgtable_ops) {
- .map = arm_lpae_map,
.map_pages = arm_lpae_map_pages,
- .unmap = arm_lpae_unmap,
.unmap_pages = arm_lpae_unmap_pages,
.iova_to_phys = arm_lpae_iova_to_phys,
};
int i, j;
unsigned long iova;
- size_t size;
+ size_t size, mapped;
struct io_pgtable_ops *ops;
selftest_running = true;
for_each_set_bit(j, &cfg->pgsize_bitmap, BITS_PER_LONG) {
size = 1UL << j;
- if (ops->map(ops, iova, iova, size, IOMMU_READ |
- IOMMU_WRITE |
- IOMMU_NOEXEC |
- IOMMU_CACHE, GFP_KERNEL))
+ if (ops->map_pages(ops, iova, iova, size, 1,
+ IOMMU_READ | IOMMU_WRITE |
+ IOMMU_NOEXEC | IOMMU_CACHE,
+ GFP_KERNEL, &mapped))
return __FAIL(ops, i);
/* Overlapping mappings */
- if (!ops->map(ops, iova, iova + size, size,
- IOMMU_READ | IOMMU_NOEXEC, GFP_KERNEL))
+ if (!ops->map_pages(ops, iova, iova + size, size, 1,
+ IOMMU_READ | IOMMU_NOEXEC,
+ GFP_KERNEL, &mapped))
return __FAIL(ops, i);
if (ops->iova_to_phys(ops, iova + 42) != (iova + 42))
/* Partial unmap */
size = 1UL << __ffs(cfg->pgsize_bitmap);
- if (ops->unmap(ops, SZ_1G + size, size, NULL) != size)
+ if (ops->unmap_pages(ops, SZ_1G + size, size, 1, NULL) != size)
return __FAIL(ops, i);
/* Remap of partial unmap */
- if (ops->map(ops, SZ_1G + size, size, size, IOMMU_READ, GFP_KERNEL))
+ if (ops->map_pages(ops, SZ_1G + size, size, size, 1,
+ IOMMU_READ, GFP_KERNEL, &mapped))
return __FAIL(ops, i);
if (ops->iova_to_phys(ops, SZ_1G + size + 42) != (size + 42))
for_each_set_bit(j, &cfg->pgsize_bitmap, BITS_PER_LONG) {
size = 1UL << j;
- if (ops->unmap(ops, iova, size, NULL) != size)
+ if (ops->unmap_pages(ops, iova, size, 1, NULL) != size)
return __FAIL(ops, i);
if (ops->iova_to_phys(ops, iova + 42))
return __FAIL(ops, i);
/* Remap full block */
- if (ops->map(ops, iova, iova, size, IOMMU_WRITE, GFP_KERNEL))
+ if (ops->map_pages(ops, iova, iova, size, 1,
+ IOMMU_WRITE, GFP_KERNEL, &mapped))
return __FAIL(ops, i);
if (ops->iova_to_phys(ops, iova + 42) != (iova + 42))
const struct iommu_ops *ops = dev->bus->iommu_ops;
struct iommu_device *iommu_dev;
struct iommu_group *group;
+ static DEFINE_MUTEX(iommu_probe_device_lock);
int ret;
if (!ops)
return -ENODEV;
-
- if (!dev_iommu_get(dev))
- return -ENOMEM;
+ /*
+ * Serialise to avoid races between IOMMU drivers registering in
+ * parallel and/or the "replay" calls from ACPI/OF code via client
+ * driver probe. Once the latter have been cleaned up we should
+ * probably be able to use device_lock() here to minimise the scope,
+ * but for now enforcing a simple global ordering is fine.
+ */
+ mutex_lock(&iommu_probe_device_lock);
+ if (!dev_iommu_get(dev)) {
+ ret = -ENOMEM;
+ goto err_unlock;
+ }
if (!try_module_get(ops->owner)) {
ret = -EINVAL;
ret = PTR_ERR(group);
goto out_release;
}
- iommu_group_put(group);
+ mutex_lock(&group->mutex);
if (group_list && !group->default_domain && list_empty(&group->entry))
list_add_tail(&group->entry, group_list);
+ mutex_unlock(&group->mutex);
+ iommu_group_put(group);
+ mutex_unlock(&iommu_probe_device_lock);
iommu_device_link(iommu_dev, dev);
return 0;
err_free:
dev_iommu_free(dev);
+err_unlock:
+ mutex_unlock(&iommu_probe_device_lock);
+
return ret;
}
return ret;
list_for_each_entry_safe(group, next, &group_list, entry) {
+ mutex_lock(&group->mutex);
+
/* Remove item from the list */
list_del_init(&group->entry);
- mutex_lock(&group->mutex);
-
/* Try to allocate default domain */
probe_alloc_default_domain(bus, group);
}
static int ipmmu_map(struct iommu_domain *io_domain, unsigned long iova,
- phys_addr_t paddr, size_t size, int prot, gfp_t gfp)
+ phys_addr_t paddr, size_t pgsize, size_t pgcount,
+ int prot, gfp_t gfp, size_t *mapped)
{
struct ipmmu_vmsa_domain *domain = to_vmsa_domain(io_domain);
- if (!domain)
- return -ENODEV;
-
- return domain->iop->map(domain->iop, iova, paddr, size, prot, gfp);
+ return domain->iop->map_pages(domain->iop, iova, paddr, pgsize, pgcount,
+ prot, gfp, mapped);
}
static size_t ipmmu_unmap(struct iommu_domain *io_domain, unsigned long iova,
- size_t size, struct iommu_iotlb_gather *gather)
+ size_t pgsize, size_t pgcount,
+ struct iommu_iotlb_gather *gather)
{
struct ipmmu_vmsa_domain *domain = to_vmsa_domain(io_domain);
- return domain->iop->unmap(domain->iop, iova, size, gather);
+ return domain->iop->unmap_pages(domain->iop, iova, pgsize, pgcount, gather);
}
static void ipmmu_flush_iotlb_all(struct iommu_domain *io_domain)
.default_domain_ops = &(const struct iommu_domain_ops) {
.attach_dev = ipmmu_attach_device,
.detach_dev = ipmmu_detach_device,
- .map = ipmmu_map,
- .unmap = ipmmu_unmap,
+ .map_pages = ipmmu_map,
+ .unmap_pages = ipmmu_unmap,
.flush_iotlb_all = ipmmu_flush_iotlb_all,
.iotlb_sync = ipmmu_iotlb_sync,
.iova_to_phys = ipmmu_iova_to_phys,
}
static int msm_iommu_map(struct iommu_domain *domain, unsigned long iova,
- phys_addr_t pa, size_t len, int prot, gfp_t gfp)
+ phys_addr_t pa, size_t pgsize, size_t pgcount,
+ int prot, gfp_t gfp, size_t *mapped)
{
struct msm_priv *priv = to_msm_priv(domain);
unsigned long flags;
int ret;
spin_lock_irqsave(&priv->pgtlock, flags);
- ret = priv->iop->map(priv->iop, iova, pa, len, prot, GFP_ATOMIC);
+ ret = priv->iop->map_pages(priv->iop, iova, pa, pgsize, pgcount, prot,
+ GFP_ATOMIC, mapped);
spin_unlock_irqrestore(&priv->pgtlock, flags);
return ret;
}
static size_t msm_iommu_unmap(struct iommu_domain *domain, unsigned long iova,
- size_t len, struct iommu_iotlb_gather *gather)
+ size_t pgsize, size_t pgcount,
+ struct iommu_iotlb_gather *gather)
{
struct msm_priv *priv = to_msm_priv(domain);
unsigned long flags;
+ size_t ret;
spin_lock_irqsave(&priv->pgtlock, flags);
- len = priv->iop->unmap(priv->iop, iova, len, gather);
+ ret = priv->iop->unmap_pages(priv->iop, iova, pgsize, pgcount, gather);
spin_unlock_irqrestore(&priv->pgtlock, flags);
- return len;
+ return ret;
}
static phys_addr_t msm_iommu_iova_to_phys(struct iommu_domain *domain,
.default_domain_ops = &(const struct iommu_domain_ops) {
.attach_dev = msm_iommu_attach_dev,
.detach_dev = msm_iommu_detach_dev,
- .map = msm_iommu_map,
- .unmap = msm_iommu_unmap,
+ .map_pages = msm_iommu_map,
+ .unmap_pages = msm_iommu_unmap,
/*
* Nothing is needed here, the barrier to guarantee
* completion of the tlb sync operation is implicitly
#define F_MMU_INT_ID_SUB_COMM_ID(a) (((a) >> 7) & 0x3)
#define F_MMU_INT_ID_COMM_ID_EXT(a) (((a) >> 10) & 0x7)
#define F_MMU_INT_ID_SUB_COMM_ID_EXT(a) (((a) >> 7) & 0x7)
+/* Macro for 5 bits length port ID field (default) */
#define F_MMU_INT_ID_LARB_ID(a) (((a) >> 7) & 0x7)
#define F_MMU_INT_ID_PORT_ID(a) (((a) >> 2) & 0x1f)
+/* Macro for 6 bits length port ID field */
+#define F_MMU_INT_ID_LARB_ID_WID_6(a) (((a) >> 8) & 0x7)
+#define F_MMU_INT_ID_PORT_ID_WID_6(a) (((a) >> 2) & 0x3f)
#define MTK_PROTECT_PA_ALIGN 256
#define MTK_IOMMU_BANK_SZ 0x1000
#define IFA_IOMMU_PCIE_SUPPORT BIT(16)
#define PGTABLE_PA_35_EN BIT(17)
#define TF_PORT_TO_ADDR_MT8173 BIT(18)
+#define INT_ID_PORT_WIDTH_6 BIT(19)
#define MTK_IOMMU_HAS_FLAG_MASK(pdata, _x, mask) \
((((pdata)->flags) & (mask)) == (_x))
M4U_MT8186,
M4U_MT8192,
M4U_MT8195,
+ M4U_MT8365,
};
struct mtk_iommu_iova_region {
struct device *smicomm_dev;
struct mtk_iommu_bank_data *bank;
-
- struct dma_iommu_mapping *mapping; /* For mtk_iommu_v1.c */
struct regmap *pericfg;
-
struct mutex mutex; /* Protect m4u_group/m4u_dom above */
/*
fault_pa |= (u64)pa34_32 << 32;
if (MTK_IOMMU_IS_TYPE(plat_data, MTK_IOMMU_TYPE_MM)) {
- fault_port = F_MMU_INT_ID_PORT_ID(regval);
if (MTK_IOMMU_HAS_FLAG(plat_data, HAS_SUB_COMM_2BITS)) {
fault_larb = F_MMU_INT_ID_COMM_ID(regval);
sub_comm = F_MMU_INT_ID_SUB_COMM_ID(regval);
+ fault_port = F_MMU_INT_ID_PORT_ID(regval);
} else if (MTK_IOMMU_HAS_FLAG(plat_data, HAS_SUB_COMM_3BITS)) {
fault_larb = F_MMU_INT_ID_COMM_ID_EXT(regval);
sub_comm = F_MMU_INT_ID_SUB_COMM_ID_EXT(regval);
+ fault_port = F_MMU_INT_ID_PORT_ID(regval);
+ } else if (MTK_IOMMU_HAS_FLAG(plat_data, INT_ID_PORT_WIDTH_6)) {
+ fault_port = F_MMU_INT_ID_PORT_ID_WID_6(regval);
+ fault_larb = F_MMU_INT_ID_LARB_ID_WID_6(regval);
} else {
+ fault_port = F_MMU_INT_ID_PORT_ID(regval);
fault_larb = F_MMU_INT_ID_LARB_ID(regval);
}
fault_larb = data->plat_data->larbid_remap[fault_larb][sub_comm];
}
- if (report_iommu_fault(&dom->domain, bank->parent_dev, fault_iova,
+ if (!dom || report_iommu_fault(&dom->domain, bank->parent_dev, fault_iova,
write ? IOMMU_FAULT_WRITE : IOMMU_FAULT_READ)) {
dev_err_ratelimited(
bank->parent_dev,
}
static int mtk_iommu_map(struct iommu_domain *domain, unsigned long iova,
- phys_addr_t paddr, size_t size, int prot, gfp_t gfp)
+ phys_addr_t paddr, size_t pgsize, size_t pgcount,
+ int prot, gfp_t gfp, size_t *mapped)
{
struct mtk_iommu_domain *dom = to_mtk_domain(domain);
paddr |= BIT_ULL(32);
/* Synchronize with the tlb_lock */
- return dom->iop->map(dom->iop, iova, paddr, size, prot, gfp);
+ return dom->iop->map_pages(dom->iop, iova, paddr, pgsize, pgcount, prot, gfp, mapped);
}
static size_t mtk_iommu_unmap(struct iommu_domain *domain,
- unsigned long iova, size_t size,
+ unsigned long iova, size_t pgsize, size_t pgcount,
struct iommu_iotlb_gather *gather)
{
struct mtk_iommu_domain *dom = to_mtk_domain(domain);
- iommu_iotlb_gather_add_range(gather, iova, size);
- return dom->iop->unmap(dom->iop, iova, size, gather);
+ iommu_iotlb_gather_add_range(gather, iova, pgsize * pgcount);
+ return dom->iop->unmap_pages(dom->iop, iova, pgsize, pgcount, gather);
}
static void mtk_iommu_flush_iotlb_all(struct iommu_domain *domain)
.default_domain_ops = &(const struct iommu_domain_ops) {
.attach_dev = mtk_iommu_attach_device,
.detach_dev = mtk_iommu_detach_device,
- .map = mtk_iommu_map,
- .unmap = mtk_iommu_unmap,
+ .map_pages = mtk_iommu_map,
+ .unmap_pages = mtk_iommu_unmap,
.flush_iotlb_all = mtk_iommu_flush_iotlb_all,
.iotlb_sync = mtk_iommu_iotlb_sync,
.iotlb_sync_map = mtk_iommu_sync_map,
static int mtk_iommu_mm_dts_parse(struct device *dev, struct component_match **match,
struct mtk_iommu_data *data)
{
- struct device_node *larbnode, *smicomm_node, *smi_subcomm_node;
- struct platform_device *plarbdev;
+ struct device_node *larbnode, *frst_avail_smicomm_node = NULL;
+ struct platform_device *plarbdev, *pcommdev;
struct device_link *link;
int i, larb_nr, ret;
larb_nr = of_count_phandle_with_args(dev->of_node, "mediatek,larbs", NULL);
if (larb_nr < 0)
return larb_nr;
+ if (larb_nr == 0 || larb_nr > MTK_LARB_NR_MAX)
+ return -EINVAL;
for (i = 0; i < larb_nr; i++) {
+ struct device_node *smicomm_node, *smi_subcomm_node;
u32 id;
larbnode = of_parse_phandle(dev->of_node, "mediatek,larbs", i);
- if (!larbnode)
- return -EINVAL;
+ if (!larbnode) {
+ ret = -EINVAL;
+ goto err_larbdev_put;
+ }
if (!of_device_is_available(larbnode)) {
of_node_put(larbnode);
ret = of_property_read_u32(larbnode, "mediatek,larb-id", &id);
if (ret)/* The id is consecutive if there is no this property */
id = i;
+ if (id >= MTK_LARB_NR_MAX) {
+ of_node_put(larbnode);
+ ret = -EINVAL;
+ goto err_larbdev_put;
+ }
plarbdev = of_find_device_by_node(larbnode);
+ of_node_put(larbnode);
if (!plarbdev) {
- of_node_put(larbnode);
- return -ENODEV;
+ ret = -ENODEV;
+ goto err_larbdev_put;
}
- if (!plarbdev->dev.driver) {
- of_node_put(larbnode);
- return -EPROBE_DEFER;
+ if (data->larb_imu[id].dev) {
+ platform_device_put(plarbdev);
+ ret = -EEXIST;
+ goto err_larbdev_put;
}
data->larb_imu[id].dev = &plarbdev->dev;
- component_match_add_release(dev, match, component_release_of,
- component_compare_of, larbnode);
+ if (!plarbdev->dev.driver) {
+ ret = -EPROBE_DEFER;
+ goto err_larbdev_put;
+ }
+
+ /* Get smi-(sub)-common dev from the last larb. */
+ smi_subcomm_node = of_parse_phandle(larbnode, "mediatek,smi", 0);
+ if (!smi_subcomm_node) {
+ ret = -EINVAL;
+ goto err_larbdev_put;
+ }
+
+ /*
+ * It may have two level smi-common. the node is smi-sub-common if it
+ * has a new mediatek,smi property. otherwise it is smi-commmon.
+ */
+ smicomm_node = of_parse_phandle(smi_subcomm_node, "mediatek,smi", 0);
+ if (smicomm_node)
+ of_node_put(smi_subcomm_node);
+ else
+ smicomm_node = smi_subcomm_node;
+
+ /*
+ * All the larbs that connect to one IOMMU must connect with the same
+ * smi-common.
+ */
+ if (!frst_avail_smicomm_node) {
+ frst_avail_smicomm_node = smicomm_node;
+ } else if (frst_avail_smicomm_node != smicomm_node) {
+ dev_err(dev, "mediatek,smi property is not right @larb%d.", id);
+ of_node_put(smicomm_node);
+ ret = -EINVAL;
+ goto err_larbdev_put;
+ } else {
+ of_node_put(smicomm_node);
+ }
+
+ component_match_add(dev, match, component_compare_dev, &plarbdev->dev);
+ platform_device_put(plarbdev);
}
- /* Get smi-(sub)-common dev from the last larb. */
- smi_subcomm_node = of_parse_phandle(larbnode, "mediatek,smi", 0);
- if (!smi_subcomm_node)
+ if (!frst_avail_smicomm_node)
return -EINVAL;
- /*
- * It may have two level smi-common. the node is smi-sub-common if it
- * has a new mediatek,smi property. otherwise it is smi-commmon.
- */
- smicomm_node = of_parse_phandle(smi_subcomm_node, "mediatek,smi", 0);
- if (smicomm_node)
- of_node_put(smi_subcomm_node);
- else
- smicomm_node = smi_subcomm_node;
-
- plarbdev = of_find_device_by_node(smicomm_node);
- of_node_put(smicomm_node);
- data->smicomm_dev = &plarbdev->dev;
+ pcommdev = of_find_device_by_node(frst_avail_smicomm_node);
+ of_node_put(frst_avail_smicomm_node);
+ if (!pcommdev)
+ return -ENODEV;
+ data->smicomm_dev = &pcommdev->dev;
link = device_link_add(data->smicomm_dev, dev,
DL_FLAG_STATELESS | DL_FLAG_PM_RUNTIME);
+ platform_device_put(pcommdev);
if (!link) {
dev_err(dev, "Unable to link %s.\n", dev_name(data->smicomm_dev));
return -EINVAL;
}
return 0;
+
+err_larbdev_put:
+ for (i = MTK_LARB_NR_MAX - 1; i >= 0; i--) {
+ if (!data->larb_imu[i].dev)
+ continue;
+ put_device(data->larb_imu[i].dev);
+ }
+ return ret;
}
static int mtk_iommu_probe(struct platform_device *pdev)
banks_num = data->plat_data->banks_num;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res)
+ return -EINVAL;
if (resource_size(res) < banks_num * MTK_IOMMU_BANK_SZ) {
dev_err(dev, "banknr %d. res %pR is not enough.\n", banks_num, res);
return -EINVAL;
{4, MTK_INVALID_LARBID, MTK_INVALID_LARBID, MTK_INVALID_LARBID, 6}},
};
+static const struct mtk_iommu_plat_data mt8365_data = {
+ .m4u_plat = M4U_MT8365,
+ .flags = RESET_AXI | INT_ID_PORT_WIDTH_6,
+ .inv_sel_reg = REG_MMU_INV_SEL_GEN1,
+ .banks_num = 1,
+ .banks_enable = {true},
+ .iova_region = single_domain,
+ .iova_region_nr = ARRAY_SIZE(single_domain),
+ .larbid_remap = {{0}, {1}, {2}, {3}, {4}, {5}}, /* Linear mapping. */
+};
+
static const struct of_device_id mtk_iommu_of_ids[] = {
{ .compatible = "mediatek,mt2712-m4u", .data = &mt2712_data},
{ .compatible = "mediatek,mt6779-m4u", .data = &mt6779_data},
{ .compatible = "mediatek,mt8195-iommu-infra", .data = &mt8195_data_infra},
{ .compatible = "mediatek,mt8195-iommu-vdo", .data = &mt8195_data_vdo},
{ .compatible = "mediatek,mt8195-iommu-vpp", .data = &mt8195_data_vpp},
+ { .compatible = "mediatek,mt8365-m4u", .data = &mt8365_data},
{}
};
}
static int mtk_iommu_v1_map(struct iommu_domain *domain, unsigned long iova,
- phys_addr_t paddr, size_t size, int prot, gfp_t gfp)
+ phys_addr_t paddr, size_t pgsize, size_t pgcount,
+ int prot, gfp_t gfp, size_t *mapped)
{
struct mtk_iommu_v1_domain *dom = to_mtk_domain(domain);
- unsigned int page_num = size >> MT2701_IOMMU_PAGE_SHIFT;
unsigned long flags;
unsigned int i;
u32 *pgt_base_iova = dom->pgt_va + (iova >> MT2701_IOMMU_PAGE_SHIFT);
u32 pabase = (u32)paddr;
- int map_size = 0;
spin_lock_irqsave(&dom->pgtlock, flags);
- for (i = 0; i < page_num; i++) {
- if (pgt_base_iova[i]) {
- memset(pgt_base_iova, 0, i * sizeof(u32));
+ for (i = 0; i < pgcount; i++) {
+ if (pgt_base_iova[i])
break;
- }
pgt_base_iova[i] = pabase | F_DESC_VALID | F_DESC_NONSEC;
pabase += MT2701_IOMMU_PAGE_SIZE;
- map_size += MT2701_IOMMU_PAGE_SIZE;
}
spin_unlock_irqrestore(&dom->pgtlock, flags);
- mtk_iommu_v1_tlb_flush_range(dom->data, iova, size);
+ *mapped = i * MT2701_IOMMU_PAGE_SIZE;
+ mtk_iommu_v1_tlb_flush_range(dom->data, iova, *mapped);
- return map_size == size ? 0 : -EEXIST;
+ return i == pgcount ? 0 : -EEXIST;
}
static size_t mtk_iommu_v1_unmap(struct iommu_domain *domain, unsigned long iova,
- size_t size, struct iommu_iotlb_gather *gather)
+ size_t pgsize, size_t pgcount,
+ struct iommu_iotlb_gather *gather)
{
struct mtk_iommu_v1_domain *dom = to_mtk_domain(domain);
unsigned long flags;
u32 *pgt_base_iova = dom->pgt_va + (iova >> MT2701_IOMMU_PAGE_SHIFT);
- unsigned int page_num = size >> MT2701_IOMMU_PAGE_SHIFT;
+ size_t size = pgcount * MT2701_IOMMU_PAGE_SIZE;
spin_lock_irqsave(&dom->pgtlock, flags);
- memset(pgt_base_iova, 0, page_num * sizeof(u32));
+ memset(pgt_base_iova, 0, pgcount * sizeof(u32));
spin_unlock_irqrestore(&dom->pgtlock, flags);
mtk_iommu_v1_tlb_flush_range(dom->data, iova, size);
.release_device = mtk_iommu_v1_release_device,
.def_domain_type = mtk_iommu_v1_def_domain_type,
.device_group = generic_device_group,
- .pgsize_bitmap = ~0UL << MT2701_IOMMU_PAGE_SHIFT,
+ .pgsize_bitmap = MT2701_IOMMU_PAGE_SIZE,
.owner = THIS_MODULE,
.default_domain_ops = &(const struct iommu_domain_ops) {
.attach_dev = mtk_iommu_v1_attach_device,
.detach_dev = mtk_iommu_v1_detach_device,
- .map = mtk_iommu_v1_map,
- .unmap = mtk_iommu_v1_unmap,
+ .map_pages = mtk_iommu_v1_map,
+ .unmap_pages = mtk_iommu_v1_unmap,
.iova_to_phys = mtk_iommu_v1_iova_to_phys,
.free = mtk_iommu_v1_domain_free,
}
* 11:9 - Page address bit 34:32
* 8:4 - Page address bit 39:35
* 3 - Security
- * 2 - Readable
- * 1 - Writable
+ * 2 - Writable
+ * 1 - Readable
* 0 - 1 if Page @ Page address is valid
*/
-#define RK_PTE_PAGE_READABLE_V2 BIT(2)
-#define RK_PTE_PAGE_WRITABLE_V2 BIT(1)
static u32 rk_mk_pte_v2(phys_addr_t page, int prot)
{
u32 flags = 0;
- flags |= (prot & IOMMU_READ) ? RK_PTE_PAGE_READABLE_V2 : 0;
- flags |= (prot & IOMMU_WRITE) ? RK_PTE_PAGE_WRITABLE_V2 : 0;
+ flags |= (prot & IOMMU_READ) ? RK_PTE_PAGE_READABLE : 0;
+ flags |= (prot & IOMMU_WRITE) ? RK_PTE_PAGE_WRITABLE : 0;
return rk_mk_dte_v2(page) | flags;
}
#include <linux/iommu.h>
#include <linux/iommu-helper.h>
#include <linux/sizes.h>
+#include <linux/rculist.h>
+#include <linux/rcupdate.h>
#include <asm/pci_dma.h>
-/*
- * Physically contiguous memory regions can be mapped with 4 KiB alignment,
- * we allow all page sizes that are an order of 4KiB (no special large page
- * support so far).
- */
-#define S390_IOMMU_PGSIZES (~0xFFFUL)
-
static const struct iommu_ops s390_iommu_ops;
struct s390_domain {
struct iommu_domain domain;
struct list_head devices;
unsigned long *dma_table;
- spinlock_t dma_table_lock;
spinlock_t list_lock;
-};
-
-struct s390_domain_device {
- struct list_head list;
- struct zpci_dev *zdev;
+ struct rcu_head rcu;
};
static struct s390_domain *to_s390_domain(struct iommu_domain *dom)
kfree(s390_domain);
return NULL;
}
+ s390_domain->domain.geometry.force_aperture = true;
+ s390_domain->domain.geometry.aperture_start = 0;
+ s390_domain->domain.geometry.aperture_end = ZPCI_TABLE_SIZE_RT - 1;
- spin_lock_init(&s390_domain->dma_table_lock);
spin_lock_init(&s390_domain->list_lock);
- INIT_LIST_HEAD(&s390_domain->devices);
+ INIT_LIST_HEAD_RCU(&s390_domain->devices);
return &s390_domain->domain;
}
-static void s390_domain_free(struct iommu_domain *domain)
+static void s390_iommu_rcu_free_domain(struct rcu_head *head)
{
- struct s390_domain *s390_domain = to_s390_domain(domain);
+ struct s390_domain *s390_domain = container_of(head, struct s390_domain, rcu);
dma_cleanup_tables(s390_domain->dma_table);
kfree(s390_domain);
}
+static void s390_domain_free(struct iommu_domain *domain)
+{
+ struct s390_domain *s390_domain = to_s390_domain(domain);
+
+ rcu_read_lock();
+ WARN_ON(!list_empty(&s390_domain->devices));
+ rcu_read_unlock();
+
+ call_rcu(&s390_domain->rcu, s390_iommu_rcu_free_domain);
+}
+
+static void __s390_iommu_detach_device(struct zpci_dev *zdev)
+{
+ struct s390_domain *s390_domain = zdev->s390_domain;
+ unsigned long flags;
+
+ if (!s390_domain)
+ return;
+
+ spin_lock_irqsave(&s390_domain->list_lock, flags);
+ list_del_rcu(&zdev->iommu_list);
+ spin_unlock_irqrestore(&s390_domain->list_lock, flags);
+
+ zpci_unregister_ioat(zdev, 0);
+ zdev->s390_domain = NULL;
+ zdev->dma_table = NULL;
+}
+
static int s390_iommu_attach_device(struct iommu_domain *domain,
struct device *dev)
{
struct s390_domain *s390_domain = to_s390_domain(domain);
struct zpci_dev *zdev = to_zpci_dev(dev);
- struct s390_domain_device *domain_device;
unsigned long flags;
- int cc, rc;
+ u8 status;
+ int cc;
if (!zdev)
return -ENODEV;
- domain_device = kzalloc(sizeof(*domain_device), GFP_KERNEL);
- if (!domain_device)
- return -ENOMEM;
-
- if (zdev->dma_table && !zdev->s390_domain) {
- cc = zpci_dma_exit_device(zdev);
- if (cc) {
- rc = -EIO;
- goto out_free;
- }
- }
+ if (WARN_ON(domain->geometry.aperture_start > zdev->end_dma ||
+ domain->geometry.aperture_end < zdev->start_dma))
+ return -EINVAL;
if (zdev->s390_domain)
- zpci_unregister_ioat(zdev, 0);
+ __s390_iommu_detach_device(zdev);
+ else if (zdev->dma_table)
+ zpci_dma_exit_device(zdev);
- zdev->dma_table = s390_domain->dma_table;
cc = zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma,
- virt_to_phys(zdev->dma_table));
- if (cc) {
- rc = -EIO;
- goto out_restore;
- }
+ virt_to_phys(s390_domain->dma_table), &status);
+ /*
+ * If the device is undergoing error recovery the reset code
+ * will re-establish the new domain.
+ */
+ if (cc && status != ZPCI_PCI_ST_FUNC_NOT_AVAIL)
+ return -EIO;
+ zdev->dma_table = s390_domain->dma_table;
- spin_lock_irqsave(&s390_domain->list_lock, flags);
- /* First device defines the DMA range limits */
- if (list_empty(&s390_domain->devices)) {
- domain->geometry.aperture_start = zdev->start_dma;
- domain->geometry.aperture_end = zdev->end_dma;
- domain->geometry.force_aperture = true;
- /* Allow only devices with identical DMA range limits */
- } else if (domain->geometry.aperture_start != zdev->start_dma ||
- domain->geometry.aperture_end != zdev->end_dma) {
- rc = -EINVAL;
- spin_unlock_irqrestore(&s390_domain->list_lock, flags);
- goto out_restore;
- }
- domain_device->zdev = zdev;
+ zdev->dma_table = s390_domain->dma_table;
zdev->s390_domain = s390_domain;
- list_add(&domain_device->list, &s390_domain->devices);
+
+ spin_lock_irqsave(&s390_domain->list_lock, flags);
+ list_add_rcu(&zdev->iommu_list, &s390_domain->devices);
spin_unlock_irqrestore(&s390_domain->list_lock, flags);
return 0;
-
-out_restore:
- if (!zdev->s390_domain) {
- zpci_dma_init_device(zdev);
- } else {
- zdev->dma_table = zdev->s390_domain->dma_table;
- zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma,
- virt_to_phys(zdev->dma_table));
- }
-out_free:
- kfree(domain_device);
-
- return rc;
}
static void s390_iommu_detach_device(struct iommu_domain *domain,
struct device *dev)
{
- struct s390_domain *s390_domain = to_s390_domain(domain);
struct zpci_dev *zdev = to_zpci_dev(dev);
- struct s390_domain_device *domain_device, *tmp;
- unsigned long flags;
- int found = 0;
- if (!zdev)
- return;
+ WARN_ON(zdev->s390_domain != to_s390_domain(domain));
- spin_lock_irqsave(&s390_domain->list_lock, flags);
- list_for_each_entry_safe(domain_device, tmp, &s390_domain->devices,
- list) {
- if (domain_device->zdev == zdev) {
- list_del(&domain_device->list);
- kfree(domain_device);
- found = 1;
- break;
- }
+ __s390_iommu_detach_device(zdev);
+ zpci_dma_init_device(zdev);
+}
+
+static void s390_iommu_get_resv_regions(struct device *dev,
+ struct list_head *list)
+{
+ struct zpci_dev *zdev = to_zpci_dev(dev);
+ struct iommu_resv_region *region;
+
+ if (zdev->start_dma) {
+ region = iommu_alloc_resv_region(0, zdev->start_dma, 0,
+ IOMMU_RESV_RESERVED, GFP_KERNEL);
+ if (!region)
+ return;
+ list_add_tail(®ion->list, list);
}
- spin_unlock_irqrestore(&s390_domain->list_lock, flags);
- if (found && (zdev->s390_domain == s390_domain)) {
- zdev->s390_domain = NULL;
- zpci_unregister_ioat(zdev, 0);
- zpci_dma_init_device(zdev);
+ if (zdev->end_dma < ZPCI_TABLE_SIZE_RT - 1) {
+ region = iommu_alloc_resv_region(zdev->end_dma + 1,
+ ZPCI_TABLE_SIZE_RT - zdev->end_dma - 1,
+ 0, IOMMU_RESV_RESERVED, GFP_KERNEL);
+ if (!region)
+ return;
+ list_add_tail(®ion->list, list);
}
}
zdev = to_zpci_dev(dev);
+ if (zdev->start_dma > zdev->end_dma ||
+ zdev->start_dma > ZPCI_TABLE_SIZE_RT - 1)
+ return ERR_PTR(-EINVAL);
+
+ if (zdev->end_dma > ZPCI_TABLE_SIZE_RT - 1)
+ zdev->end_dma = ZPCI_TABLE_SIZE_RT - 1;
+
return &zdev->iommu_dev;
}
static void s390_iommu_release_device(struct device *dev)
{
struct zpci_dev *zdev = to_zpci_dev(dev);
- struct iommu_domain *domain;
/*
- * This is a workaround for a scenario where the IOMMU API common code
- * "forgets" to call the detach_dev callback: After binding a device
- * to vfio-pci and completing the VFIO_SET_IOMMU ioctl (which triggers
- * the attach_dev), removing the device via
- * "echo 1 > /sys/bus/pci/devices/.../remove" won't trigger detach_dev,
- * only release_device will be called via the BUS_NOTIFY_REMOVED_DEVICE
- * notifier.
- *
- * So let's call detach_dev from here if it hasn't been called before.
+ * release_device is expected to detach any domain currently attached
+ * to the device, but keep it attached to other devices in the group.
*/
- if (zdev && zdev->s390_domain) {
- domain = iommu_get_domain_for_dev(dev);
- if (domain)
- s390_iommu_detach_device(domain, dev);
+ if (zdev)
+ __s390_iommu_detach_device(zdev);
+}
+
+static void s390_iommu_flush_iotlb_all(struct iommu_domain *domain)
+{
+ struct s390_domain *s390_domain = to_s390_domain(domain);
+ struct zpci_dev *zdev;
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(zdev, &s390_domain->devices, iommu_list) {
+ zpci_refresh_trans((u64)zdev->fh << 32, zdev->start_dma,
+ zdev->end_dma - zdev->start_dma + 1);
}
+ rcu_read_unlock();
}
-static int s390_iommu_update_trans(struct s390_domain *s390_domain,
- phys_addr_t pa, dma_addr_t dma_addr,
- size_t size, int flags)
+static void s390_iommu_iotlb_sync(struct iommu_domain *domain,
+ struct iommu_iotlb_gather *gather)
{
- struct s390_domain_device *domain_device;
- phys_addr_t page_addr = pa & PAGE_MASK;
- dma_addr_t start_dma_addr = dma_addr;
- unsigned long irq_flags, nr_pages, i;
- unsigned long *entry;
- int rc = 0;
+ struct s390_domain *s390_domain = to_s390_domain(domain);
+ size_t size = gather->end - gather->start + 1;
+ struct zpci_dev *zdev;
- if (dma_addr < s390_domain->domain.geometry.aperture_start ||
- dma_addr + size > s390_domain->domain.geometry.aperture_end)
- return -EINVAL;
+ /* If gather was never added to there is nothing to flush */
+ if (!gather->end)
+ return;
- nr_pages = PAGE_ALIGN(size) >> PAGE_SHIFT;
- if (!nr_pages)
- return 0;
+ rcu_read_lock();
+ list_for_each_entry_rcu(zdev, &s390_domain->devices, iommu_list) {
+ zpci_refresh_trans((u64)zdev->fh << 32, gather->start,
+ size);
+ }
+ rcu_read_unlock();
+}
+
+static void s390_iommu_iotlb_sync_map(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
+{
+ struct s390_domain *s390_domain = to_s390_domain(domain);
+ struct zpci_dev *zdev;
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(zdev, &s390_domain->devices, iommu_list) {
+ if (!zdev->tlb_refresh)
+ continue;
+ zpci_refresh_trans((u64)zdev->fh << 32,
+ iova, size);
+ }
+ rcu_read_unlock();
+}
+
+static int s390_iommu_validate_trans(struct s390_domain *s390_domain,
+ phys_addr_t pa, dma_addr_t dma_addr,
+ unsigned long nr_pages, int flags)
+{
+ phys_addr_t page_addr = pa & PAGE_MASK;
+ unsigned long *entry;
+ unsigned long i;
+ int rc;
- spin_lock_irqsave(&s390_domain->dma_table_lock, irq_flags);
for (i = 0; i < nr_pages; i++) {
entry = dma_walk_cpu_trans(s390_domain->dma_table, dma_addr);
- if (!entry) {
+ if (unlikely(!entry)) {
rc = -ENOMEM;
goto undo_cpu_trans;
}
dma_addr += PAGE_SIZE;
}
- spin_lock(&s390_domain->list_lock);
- list_for_each_entry(domain_device, &s390_domain->devices, list) {
- rc = zpci_refresh_trans((u64) domain_device->zdev->fh << 32,
- start_dma_addr, nr_pages * PAGE_SIZE);
- if (rc)
+ return 0;
+
+undo_cpu_trans:
+ while (i-- > 0) {
+ dma_addr -= PAGE_SIZE;
+ entry = dma_walk_cpu_trans(s390_domain->dma_table,
+ dma_addr);
+ if (!entry)
break;
+ dma_update_cpu_trans(entry, 0, ZPCI_PTE_INVALID);
}
- spin_unlock(&s390_domain->list_lock);
-undo_cpu_trans:
- if (rc && ((flags & ZPCI_PTE_VALID_MASK) == ZPCI_PTE_VALID)) {
- flags = ZPCI_PTE_INVALID;
- while (i-- > 0) {
- page_addr -= PAGE_SIZE;
- dma_addr -= PAGE_SIZE;
- entry = dma_walk_cpu_trans(s390_domain->dma_table,
- dma_addr);
- if (!entry)
- break;
- dma_update_cpu_trans(entry, page_addr, flags);
+ return rc;
+}
+
+static int s390_iommu_invalidate_trans(struct s390_domain *s390_domain,
+ dma_addr_t dma_addr, unsigned long nr_pages)
+{
+ unsigned long *entry;
+ unsigned long i;
+ int rc = 0;
+
+ for (i = 0; i < nr_pages; i++) {
+ entry = dma_walk_cpu_trans(s390_domain->dma_table, dma_addr);
+ if (unlikely(!entry)) {
+ rc = -EINVAL;
+ break;
}
+ dma_update_cpu_trans(entry, 0, ZPCI_PTE_INVALID);
+ dma_addr += PAGE_SIZE;
}
- spin_unlock_irqrestore(&s390_domain->dma_table_lock, irq_flags);
return rc;
}
-static int s390_iommu_map(struct iommu_domain *domain, unsigned long iova,
- phys_addr_t paddr, size_t size, int prot, gfp_t gfp)
+static int s390_iommu_map_pages(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t pgsize, size_t pgcount,
+ int prot, gfp_t gfp, size_t *mapped)
{
struct s390_domain *s390_domain = to_s390_domain(domain);
+ size_t size = pgcount << __ffs(pgsize);
int flags = ZPCI_PTE_VALID, rc = 0;
+ if (pgsize != SZ_4K)
+ return -EINVAL;
+
+ if (iova < s390_domain->domain.geometry.aperture_start ||
+ (iova + size - 1) > s390_domain->domain.geometry.aperture_end)
+ return -EINVAL;
+
+ if (!IS_ALIGNED(iova | paddr, pgsize))
+ return -EINVAL;
+
if (!(prot & IOMMU_READ))
return -EINVAL;
if (!(prot & IOMMU_WRITE))
flags |= ZPCI_TABLE_PROTECTED;
- rc = s390_iommu_update_trans(s390_domain, paddr, iova,
- size, flags);
+ rc = s390_iommu_validate_trans(s390_domain, paddr, iova,
+ pgcount, flags);
+ if (!rc)
+ *mapped = size;
return rc;
}
dma_addr_t iova)
{
struct s390_domain *s390_domain = to_s390_domain(domain);
- unsigned long *sto, *pto, *rto, flags;
+ unsigned long *rto, *sto, *pto;
+ unsigned long ste, pte, rte;
unsigned int rtx, sx, px;
phys_addr_t phys = 0;
px = calc_px(iova);
rto = s390_domain->dma_table;
- spin_lock_irqsave(&s390_domain->dma_table_lock, flags);
- if (rto && reg_entry_isvalid(rto[rtx])) {
- sto = get_rt_sto(rto[rtx]);
- if (sto && reg_entry_isvalid(sto[sx])) {
- pto = get_st_pto(sto[sx]);
- if (pto && pt_entry_isvalid(pto[px]))
- phys = pto[px] & ZPCI_PTE_ADDR_MASK;
+ rte = READ_ONCE(rto[rtx]);
+ if (reg_entry_isvalid(rte)) {
+ sto = get_rt_sto(rte);
+ ste = READ_ONCE(sto[sx]);
+ if (reg_entry_isvalid(ste)) {
+ pto = get_st_pto(ste);
+ pte = READ_ONCE(pto[px]);
+ if (pt_entry_isvalid(pte))
+ phys = pte & ZPCI_PTE_ADDR_MASK;
}
}
- spin_unlock_irqrestore(&s390_domain->dma_table_lock, flags);
return phys;
}
-static size_t s390_iommu_unmap(struct iommu_domain *domain,
- unsigned long iova, size_t size,
- struct iommu_iotlb_gather *gather)
+static size_t s390_iommu_unmap_pages(struct iommu_domain *domain,
+ unsigned long iova,
+ size_t pgsize, size_t pgcount,
+ struct iommu_iotlb_gather *gather)
{
struct s390_domain *s390_domain = to_s390_domain(domain);
- int flags = ZPCI_PTE_INVALID;
- phys_addr_t paddr;
+ size_t size = pgcount << __ffs(pgsize);
int rc;
- paddr = s390_iommu_iova_to_phys(domain, iova);
- if (!paddr)
+ if (WARN_ON(iova < s390_domain->domain.geometry.aperture_start ||
+ (iova + size - 1) > s390_domain->domain.geometry.aperture_end))
return 0;
- rc = s390_iommu_update_trans(s390_domain, paddr, iova,
- size, flags);
+ rc = s390_iommu_invalidate_trans(s390_domain, iova, pgcount);
if (rc)
return 0;
+ iommu_iotlb_gather_add_range(gather, iova, size);
+
return size;
}
.probe_device = s390_iommu_probe_device,
.release_device = s390_iommu_release_device,
.device_group = generic_device_group,
- .pgsize_bitmap = S390_IOMMU_PGSIZES,
+ .pgsize_bitmap = SZ_4K,
+ .get_resv_regions = s390_iommu_get_resv_regions,
.default_domain_ops = &(const struct iommu_domain_ops) {
.attach_dev = s390_iommu_attach_device,
.detach_dev = s390_iommu_detach_device,
- .map = s390_iommu_map,
- .unmap = s390_iommu_unmap,
+ .map_pages = s390_iommu_map_pages,
+ .unmap_pages = s390_iommu_unmap_pages,
+ .flush_iotlb_all = s390_iommu_flush_iotlb_all,
+ .iotlb_sync = s390_iommu_iotlb_sync,
+ .iotlb_sync_map = s390_iommu_iotlb_sync_map,
.iova_to_phys = s390_iommu_iova_to_phys,
.free = s390_domain_free,
}
}
static int sprd_iommu_map(struct iommu_domain *domain, unsigned long iova,
- phys_addr_t paddr, size_t size, int prot, gfp_t gfp)
+ phys_addr_t paddr, size_t pgsize, size_t pgcount,
+ int prot, gfp_t gfp, size_t *mapped)
{
struct sprd_iommu_domain *dom = to_sprd_domain(domain);
- unsigned int page_num = size >> SPRD_IOMMU_PAGE_SHIFT;
+ size_t size = pgcount * SPRD_IOMMU_PAGE_SIZE;
unsigned long flags;
unsigned int i;
u32 *pgt_base_iova;
pgt_base_iova = dom->pgt_va + ((iova - start) >> SPRD_IOMMU_PAGE_SHIFT);
spin_lock_irqsave(&dom->pgtlock, flags);
- for (i = 0; i < page_num; i++) {
+ for (i = 0; i < pgcount; i++) {
pgt_base_iova[i] = pabase >> SPRD_IOMMU_PAGE_SHIFT;
pabase += SPRD_IOMMU_PAGE_SIZE;
}
spin_unlock_irqrestore(&dom->pgtlock, flags);
+ *mapped = size;
return 0;
}
static size_t sprd_iommu_unmap(struct iommu_domain *domain, unsigned long iova,
- size_t size, struct iommu_iotlb_gather *iotlb_gather)
+ size_t pgsize, size_t pgcount,
+ struct iommu_iotlb_gather *iotlb_gather)
{
struct sprd_iommu_domain *dom = to_sprd_domain(domain);
unsigned long flags;
u32 *pgt_base_iova;
- unsigned int page_num = size >> SPRD_IOMMU_PAGE_SHIFT;
+ size_t size = pgcount * SPRD_IOMMU_PAGE_SIZE;
unsigned long start = domain->geometry.aperture_start;
unsigned long end = domain->geometry.aperture_end;
if (iova < start || (iova + size) > (end + 1))
- return -EINVAL;
+ return 0;
pgt_base_iova = dom->pgt_va + ((iova - start) >> SPRD_IOMMU_PAGE_SHIFT);
spin_lock_irqsave(&dom->pgtlock, flags);
- memset(pgt_base_iova, 0, page_num * sizeof(u32));
+ memset(pgt_base_iova, 0, pgcount * sizeof(u32));
spin_unlock_irqrestore(&dom->pgtlock, flags);
- return 0;
+ return size;
}
static void sprd_iommu_sync_map(struct iommu_domain *domain,
.probe_device = sprd_iommu_probe_device,
.device_group = sprd_iommu_device_group,
.of_xlate = sprd_iommu_of_xlate,
- .pgsize_bitmap = ~0UL << SPRD_IOMMU_PAGE_SHIFT,
+ .pgsize_bitmap = SPRD_IOMMU_PAGE_SIZE,
.owner = THIS_MODULE,
.default_domain_ops = &(const struct iommu_domain_ops) {
.attach_dev = sprd_iommu_attach_device,
.detach_dev = sprd_iommu_detach_device,
- .map = sprd_iommu_map,
- .unmap = sprd_iommu_unmap,
+ .map_pages = sprd_iommu_map,
+ .unmap_pages = sprd_iommu_unmap,
.iotlb_sync_map = sprd_iommu_sync_map,
.iotlb_sync = sprd_iommu_sync,
.iova_to_phys = sprd_iommu_iova_to_phys,
#include <linux/types.h>
#define IOMMU_RESET_REG 0x010
+#define IOMMU_RESET_RELEASE_ALL 0xffffffff
#define IOMMU_ENABLE_REG 0x020
#define IOMMU_ENABLE_ENABLE BIT(0)
#define NUM_PT_ENTRIES 256
#define PT_SIZE (NUM_PT_ENTRIES * PT_ENTRY_SIZE)
+#define SPAGE_SIZE 4096
+
struct sun50i_iommu {
struct iommu_device iommu;
enum sun50i_iommu_aci aci;
u32 flags = 0;
- if (prot & (IOMMU_READ | IOMMU_WRITE))
+ if ((prot & (IOMMU_READ | IOMMU_WRITE)) == (IOMMU_READ | IOMMU_WRITE))
aci = SUN50I_IOMMU_ACI_RD_WR;
else if (prot & IOMMU_READ)
aci = SUN50I_IOMMU_ACI_RD;
dma_sync_single_for_device(iommu->dev, dma, size, DMA_TO_DEVICE);
}
+static void sun50i_iommu_zap_iova(struct sun50i_iommu *iommu,
+ unsigned long iova)
+{
+ u32 reg;
+ int ret;
+
+ iommu_write(iommu, IOMMU_TLB_IVLD_ADDR_REG, iova);
+ iommu_write(iommu, IOMMU_TLB_IVLD_ADDR_MASK_REG, GENMASK(31, 12));
+ iommu_write(iommu, IOMMU_TLB_IVLD_ENABLE_REG,
+ IOMMU_TLB_IVLD_ENABLE_ENABLE);
+
+ ret = readl_poll_timeout_atomic(iommu->base + IOMMU_TLB_IVLD_ENABLE_REG,
+ reg, !reg, 1, 2000);
+ if (ret)
+ dev_warn(iommu->dev, "TLB invalidation timed out!\n");
+}
+
+static void sun50i_iommu_zap_ptw_cache(struct sun50i_iommu *iommu,
+ unsigned long iova)
+{
+ u32 reg;
+ int ret;
+
+ iommu_write(iommu, IOMMU_PC_IVLD_ADDR_REG, iova);
+ iommu_write(iommu, IOMMU_PC_IVLD_ENABLE_REG,
+ IOMMU_PC_IVLD_ENABLE_ENABLE);
+
+ ret = readl_poll_timeout_atomic(iommu->base + IOMMU_PC_IVLD_ENABLE_REG,
+ reg, !reg, 1, 2000);
+ if (ret)
+ dev_warn(iommu->dev, "PTW cache invalidation timed out!\n");
+}
+
+static void sun50i_iommu_zap_range(struct sun50i_iommu *iommu,
+ unsigned long iova, size_t size)
+{
+ assert_spin_locked(&iommu->iommu_lock);
+
+ iommu_write(iommu, IOMMU_AUTO_GATING_REG, 0);
+
+ sun50i_iommu_zap_iova(iommu, iova);
+ sun50i_iommu_zap_iova(iommu, iova + SPAGE_SIZE);
+ if (size > SPAGE_SIZE) {
+ sun50i_iommu_zap_iova(iommu, iova + size);
+ sun50i_iommu_zap_iova(iommu, iova + size + SPAGE_SIZE);
+ }
+ sun50i_iommu_zap_ptw_cache(iommu, iova);
+ sun50i_iommu_zap_ptw_cache(iommu, iova + SZ_1M);
+ if (size > SZ_1M) {
+ sun50i_iommu_zap_ptw_cache(iommu, iova + size);
+ sun50i_iommu_zap_ptw_cache(iommu, iova + size + SZ_1M);
+ }
+
+ iommu_write(iommu, IOMMU_AUTO_GATING_REG, IOMMU_AUTO_GATING_ENABLE);
+}
+
static int sun50i_iommu_flush_all_tlb(struct sun50i_iommu *iommu)
{
u32 reg;
spin_unlock_irqrestore(&iommu->iommu_lock, flags);
}
+static void sun50i_iommu_iotlb_sync_map(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
+{
+ struct sun50i_iommu_domain *sun50i_domain = to_sun50i_domain(domain);
+ struct sun50i_iommu *iommu = sun50i_domain->iommu;
+ unsigned long flags;
+
+ spin_lock_irqsave(&iommu->iommu_lock, flags);
+ sun50i_iommu_zap_range(iommu, iova, size);
+ spin_unlock_irqrestore(&iommu->iommu_lock, flags);
+}
+
static void sun50i_iommu_iotlb_sync(struct iommu_domain *domain,
struct iommu_iotlb_gather *gather)
{
sun50i_iommu_free_page_table(iommu, drop_pt);
}
- sun50i_table_flush(sun50i_domain, page_table, PT_SIZE);
+ sun50i_table_flush(sun50i_domain, page_table, NUM_PT_ENTRIES);
sun50i_table_flush(sun50i_domain, dte_addr, 1);
return page_table;
struct sun50i_iommu_domain *sun50i_domain;
if (type != IOMMU_DOMAIN_DMA &&
- type != IOMMU_DOMAIN_IDENTITY &&
type != IOMMU_DOMAIN_UNMANAGED)
return NULL;
.attach_dev = sun50i_iommu_attach_device,
.detach_dev = sun50i_iommu_detach_device,
.flush_iotlb_all = sun50i_iommu_flush_iotlb_all,
+ .iotlb_sync_map = sun50i_iommu_iotlb_sync_map,
.iotlb_sync = sun50i_iommu_iotlb_sync,
.iova_to_phys = sun50i_iommu_iova_to_phys,
.map = sun50i_iommu_map,
report_iommu_fault(iommu->domain, iommu->dev, iova, prot);
else
dev_err(iommu->dev, "Page fault while iommu not attached to any domain?\n");
+
+ sun50i_iommu_zap_range(iommu, iova, SPAGE_SIZE);
}
static phys_addr_t sun50i_iommu_handle_pt_irq(struct sun50i_iommu *iommu,
static irqreturn_t sun50i_iommu_irq(int irq, void *dev_id)
{
+ u32 status, l1_status, l2_status, resets;
struct sun50i_iommu *iommu = dev_id;
- u32 status;
spin_lock(&iommu->iommu_lock);
return IRQ_NONE;
}
+ l1_status = iommu_read(iommu, IOMMU_L1PG_INT_REG);
+ l2_status = iommu_read(iommu, IOMMU_L2PG_INT_REG);
+
if (status & IOMMU_INT_INVALID_L2PG)
sun50i_iommu_handle_pt_irq(iommu,
IOMMU_INT_ERR_ADDR_L2_REG,
iommu_write(iommu, IOMMU_INT_CLR_REG, status);
- iommu_write(iommu, IOMMU_RESET_REG, ~status);
- iommu_write(iommu, IOMMU_RESET_REG, status);
+ resets = (status | l1_status | l2_status) & IOMMU_INT_MASTER_MASK;
+ iommu_write(iommu, IOMMU_RESET_REG, ~resets);
+ iommu_write(iommu, IOMMU_RESET_REG, IOMMU_RESET_RELEASE_ALL);
spin_unlock(&iommu->iommu_lock);
hfcm_l1callback(struct dchannel *dch, u_int cmd)
{
struct hfc_multi *hc = dch->hw;
+ struct sk_buff_head free_queue;
u_long flags;
switch (cmd) {
l1_event(dch->l1, HW_POWERUP_IND);
break;
case HW_DEACT_REQ:
+ __skb_queue_head_init(&free_queue);
/* start deactivation */
spin_lock_irqsave(&hc->lock, flags);
if (hc->ctype == HFC_TYPE_E1) {
plxsd_checksync(hc, 0);
}
}
- skb_queue_purge(&dch->squeue);
+ skb_queue_splice_init(&dch->squeue, &free_queue);
if (dch->tx_skb) {
- dev_kfree_skb(dch->tx_skb);
+ __skb_queue_tail(&free_queue, dch->tx_skb);
dch->tx_skb = NULL;
}
dch->tx_idx = 0;
if (dch->rx_skb) {
- dev_kfree_skb(dch->rx_skb);
+ __skb_queue_tail(&free_queue, dch->rx_skb);
dch->rx_skb = NULL;
}
test_and_clear_bit(FLG_TX_BUSY, &dch->Flags);
if (test_and_clear_bit(FLG_BUSY_TIMER, &dch->Flags))
del_timer(&dch->timer);
spin_unlock_irqrestore(&hc->lock, flags);
+ __skb_queue_purge(&free_queue);
break;
case HW_POWERUP_REQ:
spin_lock_irqsave(&hc->lock, flags);
case PH_DEACTIVATE_REQ:
test_and_clear_bit(FLG_L2_ACTIVATED, &dch->Flags);
if (dch->dev.D.protocol != ISDN_P_TE_S0) {
+ struct sk_buff_head free_queue;
+
+ __skb_queue_head_init(&free_queue);
spin_lock_irqsave(&hc->lock, flags);
if (debug & DEBUG_HFCMULTI_MSG)
printk(KERN_DEBUG
/* deactivate */
dch->state = 1;
}
- skb_queue_purge(&dch->squeue);
+ skb_queue_splice_init(&dch->squeue, &free_queue);
if (dch->tx_skb) {
- dev_kfree_skb(dch->tx_skb);
+ __skb_queue_tail(&free_queue, dch->tx_skb);
dch->tx_skb = NULL;
}
dch->tx_idx = 0;
if (dch->rx_skb) {
- dev_kfree_skb(dch->rx_skb);
+ __skb_queue_tail(&free_queue, dch->rx_skb);
dch->rx_skb = NULL;
}
test_and_clear_bit(FLG_TX_BUSY, &dch->Flags);
#endif
ret = 0;
spin_unlock_irqrestore(&hc->lock, flags);
+ __skb_queue_purge(&free_queue);
} else
ret = l1_event(dch->l1, hh->prim);
break;
test_and_clear_bit(FLG_L2_ACTIVATED, &dch->Flags);
spin_lock_irqsave(&hc->lock, flags);
if (hc->hw.protocol == ISDN_P_NT_S0) {
+ struct sk_buff_head free_queue;
+
+ __skb_queue_head_init(&free_queue);
/* prepare deactivation */
Write_hfc(hc, HFCPCI_STATES, 0x40);
- skb_queue_purge(&dch->squeue);
+ skb_queue_splice_init(&dch->squeue, &free_queue);
if (dch->tx_skb) {
- dev_kfree_skb(dch->tx_skb);
+ __skb_queue_tail(&free_queue, dch->tx_skb);
dch->tx_skb = NULL;
}
dch->tx_idx = 0;
if (dch->rx_skb) {
- dev_kfree_skb(dch->rx_skb);
+ __skb_queue_tail(&free_queue, dch->rx_skb);
dch->rx_skb = NULL;
}
test_and_clear_bit(FLG_TX_BUSY, &dch->Flags);
hc->hw.mst_m &= ~HFCPCI_MASTER;
Write_hfc(hc, HFCPCI_MST_MODE, hc->hw.mst_m);
ret = 0;
+ spin_unlock_irqrestore(&hc->lock, flags);
+ __skb_queue_purge(&free_queue);
} else {
ret = l1_event(dch->l1, hh->prim);
+ spin_unlock_irqrestore(&hc->lock, flags);
}
- spin_unlock_irqrestore(&hc->lock, flags);
break;
}
if (!ret)
test_and_clear_bit(FLG_L2_ACTIVATED, &dch->Flags);
if (hw->protocol == ISDN_P_NT_S0) {
+ struct sk_buff_head free_queue;
+
+ __skb_queue_head_init(&free_queue);
hfcsusb_ph_command(hw, HFC_L1_DEACTIVATE_NT);
spin_lock_irqsave(&hw->lock, flags);
- skb_queue_purge(&dch->squeue);
+ skb_queue_splice_init(&dch->squeue, &free_queue);
if (dch->tx_skb) {
- dev_kfree_skb(dch->tx_skb);
+ __skb_queue_tail(&free_queue, dch->tx_skb);
dch->tx_skb = NULL;
}
dch->tx_idx = 0;
if (dch->rx_skb) {
- dev_kfree_skb(dch->rx_skb);
+ __skb_queue_tail(&free_queue, dch->rx_skb);
dch->rx_skb = NULL;
}
test_and_clear_bit(FLG_TX_BUSY, &dch->Flags);
spin_unlock_irqrestore(&hw->lock, flags);
+ __skb_queue_purge(&free_queue);
#ifdef FIXME
if (test_and_clear_bit(FLG_L1_BUSY, &dch->Flags))
dchannel_sched_event(&hc->dch, D_CLEARBUSY);
printk("\n");
}
- dev_kfree_skb(tx_skb);
+ dev_consume_skb_irq(tx_skb);
tx_skb = NULL;
if (fifo->dch && get_next_dframe(fifo->dch))
tx_skb = fifo->dch->tx_skb;
};
ATTRIBUTE_GROUPS(mISDN);
-static int mISDN_uevent(struct device *dev, struct kobj_uevent_env *env)
+static int mISDN_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
- struct mISDNdevice *mdev = dev_to_mISDN(dev);
+ const struct mISDNdevice *mdev = dev_to_mISDN(dev);
if (!mdev)
return 0;
return ret;
switch (color) {
case RED:
- return scnprintf(buf, PAGE_SIZE, "%02X\n", data->red);
+ return sysfs_emit(buf, "%02X\n", data->red);
case GREEN:
- return scnprintf(buf, PAGE_SIZE, "%02X\n", data->green);
+ return sysfs_emit(buf, "%02X\n", data->green);
case BLUE:
- return scnprintf(buf, PAGE_SIZE, "%02X\n", data->blue);
+ return sysfs_emit(buf, "%02X\n", data->blue);
default:
return -EINVAL;
}
static ssize_t test_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- return scnprintf(buf, PAGE_SIZE,
+ return sysfs_emit(buf,
"#Write into test to start test sequence!#\n");
}
#define IS31FL3190_CURRENT_uA_MIN 5000
#define IS31FL3190_CURRENT_uA_DEFAULT 42000
#define IS31FL3190_CURRENT_uA_MAX 42000
+#define IS31FL3190_CURRENT_SHIFT 2
#define IS31FL3190_CURRENT_MASK GENMASK(4, 2)
#define IS31FL3190_CURRENT_5_mA 0x02
#define IS31FL3190_CURRENT_10_mA 0x01
is31fl3196_db_to_gain(is31->audio_gain_db));
else
regmap_update_bits(is31->regmap, IS31FL3190_CURRENT, IS31FL3190_CURRENT_MASK,
- is31fl3190_microamp_to_cs(dev, aggregated_led_microamp));
+ is31fl3190_microamp_to_cs(dev, aggregated_led_microamp) << IS31FL3190_CURRENT_SHIFT);
for (i = 0; i < is31->cdef->num_leds; i++) {
struct is31fl319x_led *led = &is31->leds[i];
struct led_classdev *led_cdev = dev_get_drvdata(dev);
struct lm3533_led *led = to_lm3533_led(led_cdev);
- return scnprintf(buf, PAGE_SIZE, "%d\n", led->id);
+ return sysfs_emit(buf, "%d\n", led->id);
}
/*
if (ret)
return ret;
- return scnprintf(buf, PAGE_SIZE, "%x\n", val);
+ return sysfs_emit(buf, "%x\n", val);
}
static ssize_t show_risetime(struct device *dev,
channel = (val & LM3533_REG_CTRLBANK_BCONF_ALS_CHANNEL_MASK) + 1;
- return scnprintf(buf, PAGE_SIZE, "%u\n", channel);
+ return sysfs_emit(buf, "%u\n", channel);
}
static ssize_t store_als_channel(struct device *dev,
enable = val & LM3533_REG_CTRLBANK_BCONF_ALS_EN_MASK;
- return scnprintf(buf, PAGE_SIZE, "%d\n", enable);
+ return sysfs_emit(buf, "%d\n", enable);
}
static ssize_t store_als_en(struct device *dev,
else
linear = 0;
- return scnprintf(buf, PAGE_SIZE, "%x\n", linear);
+ return sysfs_emit(buf, "%x\n", linear);
}
static ssize_t store_linear(struct device *dev,
if (ret)
return ret;
- return scnprintf(buf, PAGE_SIZE, "%u\n", val);
+ return sysfs_emit(buf, "%u\n", val);
}
static ssize_t store_pwm(struct device *dev,
ret = lp5521_run_selftest(chip, buf);
mutex_unlock(&chip->lock);
- return scnprintf(buf, PAGE_SIZE, "%s\n", ret ? "FAIL" : "OK");
+ return sysfs_emit(buf, "%s\n", ret ? "FAIL" : "OK");
}
/* device attributes */
struct lp55xx_led *led = i2c_get_clientdata(to_i2c_client(dev));
struct lp55xx_chip *chip = led->chip;
struct lp55xx_platform_data *pdata = chip->pdata;
- int i, ret, pos = 0;
- u8 status, adc, vdd;
+ int ret, pos = 0;
+ u8 status, adc, vdd, i;
mutex_lock(&chip->lock);
vdd--; /* There may be some fluctuation in measurement */
- for (i = 0; i < LP5523_MAX_LEDS; i++) {
- /* Skip non-existing channels */
+ for (i = 0; i < pdata->num_channels; i++) {
+ /* Skip disabled channels */
if (pdata->led_config[i].led_current == 0)
continue;
/* Set default current */
- lp55xx_write(chip, LP5523_REG_LED_CURRENT_BASE + i,
+ lp55xx_write(chip, LP5523_REG_LED_CURRENT_BASE + led->chan_nr,
pdata->led_config[i].led_current);
- lp55xx_write(chip, LP5523_REG_LED_PWM_BASE + i, 0xff);
+ lp55xx_write(chip, LP5523_REG_LED_PWM_BASE + led->chan_nr,
+ 0xff);
/* let current stabilize 2 - 4ms before measurements start */
usleep_range(2000, 4000);
lp55xx_write(chip, LP5523_REG_LED_TEST_CTRL,
- LP5523_EN_LEDTEST | i);
+ LP5523_EN_LEDTEST | led->chan_nr);
/* ADC conversion time is 2.7 ms typically */
usleep_range(3000, 6000);
ret = lp55xx_read(chip, LP5523_REG_STATUS, &status);
goto fail;
if (!(status & LP5523_LEDTEST_DONE))
- usleep_range(3000, 6000);/* Was not ready. Wait. */
+ usleep_range(3000, 6000); /* Was not ready. Wait. */
ret = lp55xx_read(chip, LP5523_REG_LED_TEST_ADC, &adc);
if (ret < 0)
goto fail;
if (adc >= vdd || adc < LP5523_ADC_SHORTCIRC_LIM)
- pos += sprintf(buf + pos, "LED %d FAIL\n", i);
+ pos += sprintf(buf + pos, "LED %d FAIL\n",
+ led->chan_nr);
- lp55xx_write(chip, LP5523_REG_LED_PWM_BASE + i, 0x00);
+ lp55xx_write(chip, LP5523_REG_LED_PWM_BASE + led->chan_nr,
+ 0x00);
/* Restore current */
- lp55xx_write(chip, LP5523_REG_LED_CURRENT_BASE + i,
- led->led_current);
+ lp55xx_write(chip, LP5523_REG_LED_CURRENT_BASE + led->chan_nr,
+ led->led_current);
led++;
}
if (pos == 0)
{
struct lp55xx_led *led = dev_to_lp55xx_led(dev);
- return scnprintf(buf, PAGE_SIZE, "%d\n", led->led_current);
+ return sysfs_emit(buf, "%d\n", led->led_current);
}
static ssize_t led_current_store(struct device *dev,
{
struct lp55xx_led *led = dev_to_lp55xx_led(dev);
- return scnprintf(buf, PAGE_SIZE, "%d\n", led->max_current);
+ return sysfs_emit(buf, "%d\n", led->max_current);
}
static DEVICE_ATTR_RW(led_current);
struct mc_subled *mc_led_info;
struct led_classdev *led_cdev;
char name[32];
- int i, j = 0;
+ int i;
int ret;
if (chan >= max_channel) {
pdata->led_config[chan].color_id[i];
mc_led_info[i].channel =
pdata->led_config[chan].output_num[i];
- j++;
}
led->mc_cdev.subled_info = mc_led_info;
char name[20];
int ret = 0;
- if (pdata == NULL) {
- dev_err(&pdev->dev, "no platform data\n");
- return -ENODEV;
- }
-
led = devm_kzalloc(&pdev->dev, sizeof(*led), GFP_KERNEL);
if (led == NULL)
return -ENOMEM;
led->iodev = iodev;
/* initialize mode and brightness according to platform_data */
- if (pdata->led_pdata) {
+ if (pdata && pdata->led_pdata) {
u8 mode = 0, brightness = 0;
mode = pdata->led_pdata->mode[led->id];
return (bits + 7) / 8;
}
-/* 4 bits per LED selector register */
-static inline int pca95xx_num_led_regs(int bits)
-{
- return (bits + 3) / 4;
-}
-
/*
* Return an LED selector register value based on an existing one, with
* the appropriate 2-bit state value set for the given LED number (0-3).
lpg_lut_sync(lpg, lut_mask);
}
-static void lpg_brightness_single_set(struct led_classdev *cdev,
- enum led_brightness value)
+static int lpg_brightness_single_set(struct led_classdev *cdev,
+ enum led_brightness value)
{
struct lpg_led *led = container_of(cdev, struct lpg_led, cdev);
struct mc_subled info;
lpg_brightness_set(led, cdev, &info);
mutex_unlock(&led->lpg->lock);
+
+ return 0;
}
-static void lpg_brightness_mc_set(struct led_classdev *cdev,
- enum led_brightness value)
+static int lpg_brightness_mc_set(struct led_classdev *cdev,
+ enum led_brightness value)
{
struct led_classdev_mc *mc = lcdev_to_mccdev(cdev);
struct lpg_led *led = container_of(mc, struct lpg_led, mcdev);
lpg_brightness_set(led, cdev, mc->subled_info);
mutex_unlock(&led->lpg->lock);
+
+ return 0;
}
static int lpg_blink_set(struct lpg_led *led,
return ret;
}
-static void lpg_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int lpg_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct lpg *lpg = container_of(chip, struct lpg, pwm);
struct lpg_channel *chan = &lpg->channels[pwm->hwpwm];
ret = regmap_read(lpg->map, chan->base + LPG_SIZE_CLK_REG, &val);
if (ret)
- return;
+ return ret;
refclk = lpg_clk_rates[val & PWM_CLK_SELECT_MASK];
if (refclk) {
ret = regmap_read(lpg->map, chan->base + LPG_PREDIV_CLK_REG, &val);
if (ret)
- return;
+ return ret;
pre_div = lpg_pre_divs[FIELD_GET(PWM_FREQ_PRE_DIV_MASK, val)];
m = FIELD_GET(PWM_FREQ_EXP_MASK, val);
ret = regmap_bulk_read(lpg->map, chan->base + PWM_VALUE_REG, &pwm_value, sizeof(pwm_value));
if (ret)
- return;
+ return ret;
state->period = DIV_ROUND_UP_ULL((u64)NSEC_PER_SEC * LPG_RESOLUTION * pre_div * (1 << m), refclk);
state->duty_cycle = DIV_ROUND_UP_ULL((u64)NSEC_PER_SEC * pwm_value * pre_div * (1 << m), refclk);
ret = regmap_read(lpg->map, chan->base + PWM_ENABLE_CONTROL_REG, &val);
if (ret)
- return;
+ return ret;
state->enabled = FIELD_GET(LPG_ENABLE_CONTROL_OUTPUT, val);
state->polarity = PWM_POLARITY_NORMAL;
if (state->duty_cycle > state->period)
state->duty_cycle = state->period;
+
+ return 0;
}
static const struct pwm_ops lpg_pwm_ops = {
led->mcdev.num_colors = num_channels;
cdev = &led->mcdev.led_cdev;
- cdev->brightness_set = lpg_brightness_mc_set;
+ cdev->brightness_set_blocking = lpg_brightness_mc_set;
cdev->blink_set = lpg_blink_mc_set;
/* Register pattern accessors only if we have a LUT block */
return ret;
cdev = &led->cdev;
- cdev->brightness_set = lpg_brightness_single_set;
+ cdev->brightness_set_blocking = lpg_brightness_single_set;
cdev->blink_set = lpg_blink_single_set;
/* Register pattern accessors only if we have a LUT block */
else
cdev->brightness = LED_OFF;
- cdev->brightness_set(cdev, cdev->brightness);
+ cdev->brightness_set_blocking(cdev, cdev->brightness);
init_data.fwnode = of_fwnode_handle(np);
mutex_unlock(&data->lock);
- return scnprintf(buf, PAGE_SIZE, "%d\n", repeat);
+ return sysfs_emit(buf, "%d\n", repeat);
}
static ssize_t repeat_store(struct device *dev, struct device_attribute *attr,
if ((adb_handler[i].original_address == default_id) &&
(!handler_id || (handler_id == adb_handler[i].handler_id) ||
try_handler_change(i, handler_id))) {
- if (adb_handler[i].handler != 0) {
+ if (adb_handler[i].handler) {
pr_err("Two handlers for ADB device %d\n",
default_id);
continue;
goto out;
}
state = kmalloc(sizeof(struct adbdev_state), GFP_KERNEL);
- if (state == 0) {
+ if (!state) {
ret = -ENOMEM;
goto out;
}
AMS_CMD_START,
};
-static int ams_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id);
+static int ams_i2c_probe(struct i2c_client *client);
static void ams_i2c_remove(struct i2c_client *client);
static const struct i2c_device_id ams_id[] = {
.driver = {
.name = "ams",
},
- .probe = ams_i2c_probe,
+ .probe_new = ams_i2c_probe,
.remove = ams_i2c_remove,
.id_table = ams_id,
};
*z = ams_i2c_read(AMS_DATAZ);
}
-static int ams_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ams_i2c_probe(struct i2c_client *client)
{
int vmaj, vmin;
int result;
/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _AMS_H
+#define _AMS_H
+
#include <linux/i2c.h>
#include <linux/input.h>
#include <linux/kthread.h>
extern int ams_input_init(void);
extern void ams_input_exit(void);
+
+#endif /* _AMS_H */
unsigned int irq;
adbs = of_find_compatible_node(NULL, "adb", "chrp,adb0");
- if (adbs == 0)
+ if (!adbs)
return -ENXIO;
if (of_address_to_resource(adbs, 0, &r)) {
return -ENXIO;
}
adb = ioremap(r.start, sizeof(struct adb_regs));
+ if (!adb) {
+ of_node_put(adbs);
+ return -ENOMEM;
+ }
out_8(&adb->ctrl.r, 0);
out_8(&adb->intr.r, 0);
req->reply_len = 0;
spin_lock_irqsave(&macio_lock, flags);
- if (current_req != 0) {
+ if (current_req) {
last_req->next = req;
last_req = req;
} else {
spin_lock(&macio_lock);
if (in_8(&adb->intr.r) & TAG) {
handled = 1;
- if ((req = current_req) != 0) {
+ req = current_req;
+ if (req) {
/* put the current request in */
for (i = 0; i < req->nbytes; ++i)
out_8(&adb->data[i].r, req->data[i]);
if (of_device_register(&dev->ofdev) != 0) {
printk(KERN_DEBUG"macio: device registration error for %s!\n",
dev_name(&dev->ofdev.dev));
- kfree(dev);
+ put_device(&dev->ofdev.dev);
return NULL;
}
}
-static int probe_thermostat(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int probe_thermostat(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device_node *np = client->dev.of_node;
struct thermostat* th;
const __be32 *prop;
.driver = {
.name = "therm_adt746x",
},
- .probe = probe_thermostat,
+ .probe_new = probe_thermostat,
.remove = remove_thermostat,
.id_table = therm_adt746x_id,
};
MODULE_DEVICE_TABLE(i2c, therm_windtunnel_id);
static int
-do_probe(struct i2c_client *cl, const struct i2c_device_id *id)
+do_probe(struct i2c_client *cl)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(cl);
struct i2c_adapter *adapter = cl->adapter;
int ret = 0;
.driver = {
.name = "therm_windtunnel",
},
- .probe = do_probe,
+ .probe_new = do_probe,
.remove = do_remove,
.id_table = therm_windtunnel_id,
};
static int __pmu_backlight_update_status(struct backlight_device *bd)
{
struct adb_request req;
- int level = bd->props.brightness;
-
-
- if (bd->props.power != FB_BLANK_UNBLANK ||
- bd->props.fb_blank != FB_BLANK_UNBLANK)
- level = 0;
+ int level = backlight_get_brightness(bd);
if (level > 0) {
int pmulevel = pmu_backlight_get_level_brightness(level);
static void pmu_start(void);
static irqreturn_t via_pmu_interrupt(int irq, void *arg);
static irqreturn_t gpio1_interrupt(int irq, void *arg);
+#ifdef CONFIG_PROC_FS
static int pmu_info_proc_show(struct seq_file *m, void *v);
static int pmu_irqstats_proc_show(struct seq_file *m, void *v);
static int pmu_battery_proc_show(struct seq_file *m, void *v);
+#endif
static void pmu_pass_intr(unsigned char *data, int len);
static const struct proc_ops pmu_options_proc_ops;
2, PMU_SMART_BATTERY_STATE, pmu_cur_battery+1);
}
+#ifdef CONFIG_PROC_FS
static int pmu_info_proc_show(struct seq_file *m, void *v)
{
seq_printf(m, "PMU driver version : %d\n", PMU_DRIVER_VERSION);
.proc_release = single_release,
.proc_write = pmu_options_proc_write,
};
+#endif
#ifdef CONFIG_ADB
/* Send an ADB command */
pv->config = config;
}
-static int wf_ad7417_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int wf_ad7417_probe(struct i2c_client *client)
{
struct wf_ad7417_priv *pv;
const struct mpu_data *mpu;
.name = "wf_ad7417",
.of_match_table = wf_ad7417_of_id,
},
- .probe = wf_ad7417_probe,
+ .probe_new = wf_ad7417_probe,
.remove = wf_ad7417_remove,
.id_table = wf_ad7417_id,
};
return 0;
}
-static int wf_fcu_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int wf_fcu_probe(struct i2c_client *client)
{
struct wf_fcu_priv *pv;
.name = "wf_fcu",
.of_match_table = wf_fcu_of_id,
},
- .probe = wf_fcu_probe,
+ .probe_new = wf_fcu_probe,
.remove = wf_fcu_remove,
.id_table = wf_fcu_id,
};
.owner = THIS_MODULE,
};
-static int wf_lm75_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
-{
+static int wf_lm75_probe(struct i2c_client *client)
+{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct wf_lm75_sensor *lm;
int rc, ds1775;
const char *name, *loc;
.name = "wf_lm75",
.of_match_table = wf_lm75_of_id,
},
- .probe = wf_lm75_probe,
+ .probe_new = wf_lm75_probe,
.remove = wf_lm75_remove,
.id_table = wf_lm75_id,
};
.owner = THIS_MODULE,
};
-static int wf_lm87_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int wf_lm87_probe(struct i2c_client *client)
{
struct wf_lm87_sensor *lm;
const char *name = NULL, *loc;
.name = "wf_lm87",
.of_match_table = wf_lm87_of_id,
},
- .probe = wf_lm87_probe,
+ .probe_new = wf_lm87_probe,
.remove = wf_lm87_remove,
.id_table = wf_lm87_id,
};
.owner = THIS_MODULE,
};
-static int wf_max6690_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int wf_max6690_probe(struct i2c_client *client)
{
const char *name, *loc;
struct wf_6690_sensor *max;
.name = "wf_max6690",
.of_match_table = wf_max6690_of_id,
},
- .probe = wf_max6690_probe,
+ .probe_new = wf_max6690_probe,
.remove = wf_max6690_remove,
.id_table = wf_max6690_id,
};
/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _WINDFARM_PID_H
+#define _WINDFARM_PID_H
+
/*
* Windfarm PowerMac thermal control. Generic PID helpers
*
extern void wf_cpu_pid_init(struct wf_cpu_pid_state *st,
struct wf_cpu_pid_param *param);
extern s32 wf_cpu_pid_run(struct wf_cpu_pid_state *st, s32 power, s32 temp);
+
+#endif /* _WINDFARM_PID_H */
/* First, locate the PID params in SMU SBD */
hdr = smu_get_sdb_partition(SMU_SDB_CPUPIDDATA_ID, NULL);
- if (hdr == 0) {
+ if (!hdr) {
printk(KERN_WARNING "pm121: CPU PID fan config not found.\n");
goto fail;
}
const struct smu_sdbp_header *hdr;
hdr = smu_get_sdb_partition(SMU_SDB_SENSORTREE_ID, NULL);
- if (hdr != 0) {
+ if (hdr) {
struct smu_sdbp_sensortree *st =
(struct smu_sdbp_sensortree *)&hdr[1];
pm121_mach_model = st->model_id;
/* First, locate the PID params in SMU SBD */
hdr = smu_get_sdb_partition(SMU_SDB_CPUPIDDATA_ID, NULL);
- if (hdr == 0) {
+ if (!hdr) {
printk(KERN_WARNING "windfarm: CPU PID fan config not found "
"max fan speed\n");
goto fail;
const struct smu_sdbp_header *hdr;
hdr = smu_get_sdb_partition(SMU_SDB_SENSORTREE_ID, NULL);
- if (hdr != 0) {
+ if (hdr) {
struct smu_sdbp_sensortree *st =
(struct smu_sdbp_sensortree *)&hdr[1];
wf_smu_mach_model = st->model_id;
/* First, locate the PID params in SMU SBD */
hdr = smu_get_sdb_partition(SMU_SDB_CPUPIDDATA_ID, NULL);
- if (hdr == 0) {
+ if (!hdr) {
printk(KERN_WARNING "windfarm: CPU PID fan config not found "
"max fan speed\n");
goto fail;
return -ENODEV;
/* Look for RPM fans */
- for (fans = NULL; (fans = of_get_next_child(smu, fans)) != NULL;)
+ for_each_child_of_node(smu, fans)
if (of_node_name_eq(fans, "rpm-fans") ||
of_device_is_compatible(fans, "smu-rpm-fans"))
break;
- for (fan = NULL;
- fans && (fan = of_get_next_child(fans, fan)) != NULL;) {
+ for_each_child_of_node(fans, fan) {
struct smu_fan_control *fct;
fct = smu_fan_create(fan, 0);
/* Look for PWM fans */
- for (fans = NULL; (fans = of_get_next_child(smu, fans)) != NULL;)
+ for_each_child_of_node(smu, fans)
if (of_node_name_eq(fans, "pwm-fans"))
break;
- for (fan = NULL;
- fans && (fan = of_get_next_child(fans, fan)) != NULL;) {
+ for_each_child_of_node(fans, fan) {
struct smu_fan_control *fct;
fct = smu_fan_create(fan, 1);
.owner = THIS_MODULE,
};
-static int wf_sat_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int wf_sat_probe(struct i2c_client *client)
{
struct device_node *dev = client->dev.of_node;
struct wf_sat *sat;
.name = "wf_smu_sat",
.of_match_table = wf_sat_of_id,
},
- .probe = wf_sat_probe,
+ .probe_new = wf_sat_probe,
.remove = wf_sat_remove,
.id_table = wf_sat_id,
};
config TI_MESSAGE_MANAGER
tristate "Texas Instruments Message Manager Driver"
depends on ARCH_KEYSTONE || ARCH_K3
+ default ARCH_K3
help
An implementation of Message Manager slave driver for Keystone
and K3 architecture SoCs from Texas Instruments. Message Manager
int ret = -EINVAL;
reg = devm_of_iomap(dev, dev->of_node, 0, NULL);
- if (!reg)
- return -ENOMEM;
+ if (IS_ERR(reg))
+ return PTR_ERR(reg);
mhu = devm_kzalloc(dev, sizeof(*mhu), GFP_KERNEL);
if (!mhu)
/*
* Microchip PolarFire SoC (MPFS) system controller/mailbox controller driver
*
- * Copyright (c) 2020 Microchip Corporation. All rights reserved.
+ * Copyright (c) 2020-2022 Microchip Corporation. All rights reserved.
*
* Author: Conor Dooley <conor.dooley@microchip.com>
*
#define SCB_STATUS_NOTIFY_MASK BIT(SCB_STATUS_NOTIFY)
#define SCB_STATUS_POS (16)
-#define SCB_STATUS_MASK GENMASK_ULL(SCB_STATUS_POS + SCB_MASK_WIDTH, SCB_STATUS_POS)
+#define SCB_STATUS_MASK GENMASK(SCB_STATUS_POS + SCB_MASK_WIDTH - 1, SCB_STATUS_POS)
struct mpfs_mbox {
struct mbox_controller controller;
struct mpfs_mbox *mbox = (struct mpfs_mbox *)chan->con_priv;
struct mpfs_mss_response *response = mbox->response;
u16 num_words = ALIGN((response->resp_size), (4)) / 4U;
- u32 i;
+ u32 i, status;
if (!response->resp_msg) {
dev_err(mbox->dev, "failed to assign memory for response %d\n", -ENOMEM);
return;
}
+ /*
+ * The status is stored in bits 31:16 of the SERVICES_SR register.
+ * It is only valid when BUSY == 0.
+ * We should *never* get an interrupt while the controller is
+ * still in the busy state. If we do, something has gone badly
+ * wrong & the content of the mailbox would not be valid.
+ */
+ if (mpfs_mbox_busy(mbox)) {
+ dev_err(mbox->dev, "got an interrupt but system controller is busy\n");
+ response->resp_status = 0xDEAD;
+ return;
+ }
+
+ status = readl_relaxed(mbox->ctrl_base + SERVICES_SR_OFFSET);
+
+ /*
+ * If the status of the individual servers is non-zero, the service has
+ * failed. The contents of the mailbox at this point are not be valid,
+ * so don't bother reading them. Set the status so that the driver
+ * implementing the service can handle the result.
+ */
+ response->resp_status = (status & SCB_STATUS_MASK) >> SCB_STATUS_POS;
+ if (response->resp_status)
+ return;
+
if (!mpfs_mbox_busy(mbox)) {
for (i = 0; i < num_words; i++) {
response->resp_msg[i] =
#define CMDQ_THR_PRIORITY 0x40
#define GCE_GCTL_VALUE 0x48
+#define GCE_CTRL_BY_SW GENMASK(2, 0)
+#define GCE_DDR_EN GENMASK(18, 16)
#define CMDQ_THR_ACTIVE_SLOT_CYCLES 0x3200
#define CMDQ_THR_ENABLED 0x1
struct mbox_controller mbox;
void __iomem *base;
int irq;
- u32 thread_nr;
u32 irq_mask;
+ const struct gce_plat *pdata;
struct cmdq_thread *thread;
struct clk_bulk_data clocks[CMDQ_GCE_NUM_MAX];
bool suspended;
- u8 shift_pa;
- bool control_by_sw;
- u32 gce_num;
};
struct gce_plat {
u32 thread_nr;
u8 shift;
bool control_by_sw;
+ bool sw_ddr_en;
u32 gce_num;
};
+static void cmdq_sw_ddr_enable(struct cmdq *cmdq, bool enable)
+{
+ WARN_ON(clk_bulk_enable(cmdq->pdata->gce_num, cmdq->clocks));
+
+ if (enable)
+ writel(GCE_DDR_EN | GCE_CTRL_BY_SW, cmdq->base + GCE_GCTL_VALUE);
+ else
+ writel(GCE_CTRL_BY_SW, cmdq->base + GCE_GCTL_VALUE);
+
+ clk_bulk_disable(cmdq->pdata->gce_num, cmdq->clocks);
+}
+
u8 cmdq_get_shift_pa(struct mbox_chan *chan)
{
struct cmdq *cmdq = container_of(chan->mbox, struct cmdq, mbox);
- return cmdq->shift_pa;
+ return cmdq->pdata->shift;
}
EXPORT_SYMBOL(cmdq_get_shift_pa);
static void cmdq_init(struct cmdq *cmdq)
{
int i;
+ u32 gctl_regval = 0;
+
+ WARN_ON(clk_bulk_enable(cmdq->pdata->gce_num, cmdq->clocks));
+ if (cmdq->pdata->control_by_sw)
+ gctl_regval = GCE_CTRL_BY_SW;
+ if (cmdq->pdata->sw_ddr_en)
+ gctl_regval |= GCE_DDR_EN;
+
+ if (gctl_regval)
+ writel(gctl_regval, cmdq->base + GCE_GCTL_VALUE);
- WARN_ON(clk_bulk_enable(cmdq->gce_num, cmdq->clocks));
- if (cmdq->control_by_sw)
- writel(0x7, cmdq->base + GCE_GCTL_VALUE);
writel(CMDQ_THR_ACTIVE_SLOT_CYCLES, cmdq->base + CMDQ_THR_SLOT_CYCLES);
for (i = 0; i <= CMDQ_MAX_EVENT; i++)
writel(i, cmdq->base + CMDQ_SYNC_TOKEN_UPDATE);
- clk_bulk_disable(cmdq->gce_num, cmdq->clocks);
+ clk_bulk_disable(cmdq->pdata->gce_num, cmdq->clocks);
}
static int cmdq_thread_reset(struct cmdq *cmdq, struct cmdq_thread *thread)
prev_task->pkt->cmd_buf_size, DMA_TO_DEVICE);
prev_task_base[CMDQ_NUM_CMD(prev_task->pkt) - 1] =
(u64)CMDQ_JUMP_BY_PA << 32 |
- (task->pa_base >> task->cmdq->shift_pa);
+ (task->pa_base >> task->cmdq->pdata->shift);
dma_sync_single_for_device(dev, prev_task->pa_base,
prev_task->pkt->cmd_buf_size, DMA_TO_DEVICE);
next_task = list_first_entry_or_null(&thread->task_busy_list,
struct cmdq_task, list_entry);
if (next_task)
- writel(next_task->pa_base >> cmdq->shift_pa,
+ writel(next_task->pa_base >> cmdq->pdata->shift,
thread->base + CMDQ_THR_CURR_ADDR);
cmdq_thread_resume(thread);
}
else
return;
- curr_pa = readl(thread->base + CMDQ_THR_CURR_ADDR) << cmdq->shift_pa;
+ curr_pa = readl(thread->base + CMDQ_THR_CURR_ADDR) << cmdq->pdata->shift;
list_for_each_entry_safe(task, tmp, &thread->task_busy_list,
list_entry) {
if (list_empty(&thread->task_busy_list)) {
cmdq_thread_disable(cmdq, thread);
- clk_bulk_disable(cmdq->gce_num, cmdq->clocks);
+ clk_bulk_disable(cmdq->pdata->gce_num, cmdq->clocks);
}
}
if (!(irq_status ^ cmdq->irq_mask))
return IRQ_NONE;
- for_each_clear_bit(bit, &irq_status, cmdq->thread_nr) {
+ for_each_clear_bit(bit, &irq_status, cmdq->pdata->thread_nr) {
struct cmdq_thread *thread = &cmdq->thread[bit];
spin_lock_irqsave(&thread->chan->lock, flags);
cmdq->suspended = true;
- for (i = 0; i < cmdq->thread_nr; i++) {
+ for (i = 0; i < cmdq->pdata->thread_nr; i++) {
thread = &cmdq->thread[i];
if (!list_empty(&thread->task_busy_list)) {
task_running = true;
if (task_running)
dev_warn(dev, "exist running task(s) in suspend\n");
- clk_bulk_unprepare(cmdq->gce_num, cmdq->clocks);
+ if (cmdq->pdata->sw_ddr_en)
+ cmdq_sw_ddr_enable(cmdq, false);
+
+ clk_bulk_unprepare(cmdq->pdata->gce_num, cmdq->clocks);
return 0;
}
{
struct cmdq *cmdq = dev_get_drvdata(dev);
- WARN_ON(clk_bulk_prepare(cmdq->gce_num, cmdq->clocks));
+ WARN_ON(clk_bulk_prepare(cmdq->pdata->gce_num, cmdq->clocks));
cmdq->suspended = false;
+
+ if (cmdq->pdata->sw_ddr_en)
+ cmdq_sw_ddr_enable(cmdq, true);
+
return 0;
}
{
struct cmdq *cmdq = platform_get_drvdata(pdev);
- clk_bulk_unprepare(cmdq->gce_num, cmdq->clocks);
+ if (cmdq->pdata->sw_ddr_en)
+ cmdq_sw_ddr_enable(cmdq, false);
+
+ clk_bulk_unprepare(cmdq->pdata->gce_num, cmdq->clocks);
return 0;
}
task->pkt = pkt;
if (list_empty(&thread->task_busy_list)) {
- WARN_ON(clk_bulk_enable(cmdq->gce_num, cmdq->clocks));
+ WARN_ON(clk_bulk_enable(cmdq->pdata->gce_num, cmdq->clocks));
/*
* The thread reset will clear thread related register to 0,
*/
WARN_ON(cmdq_thread_reset(cmdq, thread) < 0);
- writel(task->pa_base >> cmdq->shift_pa,
+ writel(task->pa_base >> cmdq->pdata->shift,
thread->base + CMDQ_THR_CURR_ADDR);
- writel((task->pa_base + pkt->cmd_buf_size) >> cmdq->shift_pa,
+ writel((task->pa_base + pkt->cmd_buf_size) >> cmdq->pdata->shift,
thread->base + CMDQ_THR_END_ADDR);
writel(thread->priority, thread->base + CMDQ_THR_PRIORITY);
} else {
WARN_ON(cmdq_thread_suspend(cmdq, thread) < 0);
curr_pa = readl(thread->base + CMDQ_THR_CURR_ADDR) <<
- cmdq->shift_pa;
+ cmdq->pdata->shift;
end_pa = readl(thread->base + CMDQ_THR_END_ADDR) <<
- cmdq->shift_pa;
+ cmdq->pdata->shift;
/* check boundary */
if (curr_pa == end_pa - CMDQ_INST_SIZE ||
curr_pa == end_pa) {
/* set to this task directly */
- writel(task->pa_base >> cmdq->shift_pa,
+ writel(task->pa_base >> cmdq->pdata->shift,
thread->base + CMDQ_THR_CURR_ADDR);
} else {
cmdq_task_insert_into_thread(task);
smp_mb(); /* modify jump before enable thread */
}
- writel((task->pa_base + pkt->cmd_buf_size) >> cmdq->shift_pa,
+ writel((task->pa_base + pkt->cmd_buf_size) >> cmdq->pdata->shift,
thread->base + CMDQ_THR_END_ADDR);
cmdq_thread_resume(thread);
}
}
cmdq_thread_disable(cmdq, thread);
- clk_bulk_disable(cmdq->gce_num, cmdq->clocks);
+ clk_bulk_disable(cmdq->pdata->gce_num, cmdq->clocks);
done:
/*
cmdq_thread_resume(thread);
cmdq_thread_disable(cmdq, thread);
- clk_bulk_disable(cmdq->gce_num, cmdq->clocks);
+ clk_bulk_disable(cmdq->pdata->gce_num, cmdq->clocks);
out:
spin_unlock_irqrestore(&thread->chan->lock, flags);
struct device *dev = &pdev->dev;
struct cmdq *cmdq;
int err, i;
- struct gce_plat *plat_data;
struct device_node *phandle = dev->of_node;
struct device_node *node;
int alias_id = 0;
if (cmdq->irq < 0)
return cmdq->irq;
- plat_data = (struct gce_plat *)of_device_get_match_data(dev);
- if (!plat_data) {
+ cmdq->pdata = device_get_match_data(dev);
+ if (!cmdq->pdata) {
dev_err(dev, "failed to get match data\n");
return -EINVAL;
}
- cmdq->thread_nr = plat_data->thread_nr;
- cmdq->shift_pa = plat_data->shift;
- cmdq->control_by_sw = plat_data->control_by_sw;
- cmdq->gce_num = plat_data->gce_num;
- cmdq->irq_mask = GENMASK(cmdq->thread_nr - 1, 0);
- err = devm_request_irq(dev, cmdq->irq, cmdq_irq_handler, IRQF_SHARED,
- "mtk_cmdq", cmdq);
- if (err < 0) {
- dev_err(dev, "failed to register ISR (%d)\n", err);
- return err;
- }
+ cmdq->irq_mask = GENMASK(cmdq->pdata->thread_nr - 1, 0);
dev_dbg(dev, "cmdq device: addr:0x%p, va:0x%p, irq:%d\n",
dev, cmdq->base, cmdq->irq);
- if (cmdq->gce_num > 1) {
+ if (cmdq->pdata->gce_num > 1) {
for_each_child_of_node(phandle->parent, node) {
alias_id = of_alias_get_id(node, clk_name);
- if (alias_id >= 0 && alias_id < cmdq->gce_num) {
+ if (alias_id >= 0 && alias_id < cmdq->pdata->gce_num) {
cmdq->clocks[alias_id].id = clk_names[alias_id];
cmdq->clocks[alias_id].clk = of_clk_get(node, 0);
if (IS_ERR(cmdq->clocks[alias_id].clk)) {
}
cmdq->mbox.dev = dev;
- cmdq->mbox.chans = devm_kcalloc(dev, cmdq->thread_nr,
+ cmdq->mbox.chans = devm_kcalloc(dev, cmdq->pdata->thread_nr,
sizeof(*cmdq->mbox.chans), GFP_KERNEL);
if (!cmdq->mbox.chans)
return -ENOMEM;
- cmdq->mbox.num_chans = cmdq->thread_nr;
+ cmdq->mbox.num_chans = cmdq->pdata->thread_nr;
cmdq->mbox.ops = &cmdq_mbox_chan_ops;
cmdq->mbox.of_xlate = cmdq_xlate;
cmdq->mbox.txdone_irq = false;
cmdq->mbox.txdone_poll = false;
- cmdq->thread = devm_kcalloc(dev, cmdq->thread_nr,
+ cmdq->thread = devm_kcalloc(dev, cmdq->pdata->thread_nr,
sizeof(*cmdq->thread), GFP_KERNEL);
if (!cmdq->thread)
return -ENOMEM;
- for (i = 0; i < cmdq->thread_nr; i++) {
+ for (i = 0; i < cmdq->pdata->thread_nr; i++) {
cmdq->thread[i].base = cmdq->base + CMDQ_THR_BASE +
CMDQ_THR_SIZE * i;
INIT_LIST_HEAD(&cmdq->thread[i].task_busy_list);
platform_set_drvdata(pdev, cmdq);
- WARN_ON(clk_bulk_prepare(cmdq->gce_num, cmdq->clocks));
+ WARN_ON(clk_bulk_prepare(cmdq->pdata->gce_num, cmdq->clocks));
cmdq_init(cmdq);
+ err = devm_request_irq(dev, cmdq->irq, cmdq_irq_handler, IRQF_SHARED,
+ "mtk_cmdq", cmdq);
+ if (err < 0) {
+ dev_err(dev, "failed to register ISR (%d)\n", err);
+ return err;
+ }
+
return 0;
}
.gce_num = 2
};
+static const struct gce_plat gce_plat_v7 = {
+ .thread_nr = 24,
+ .shift = 3,
+ .control_by_sw = true,
+ .sw_ddr_en = true,
+ .gce_num = 1
+};
+
static const struct of_device_id cmdq_of_ids[] = {
{.compatible = "mediatek,mt8173-gce", .data = (void *)&gce_plat_v2},
{.compatible = "mediatek,mt8183-gce", .data = (void *)&gce_plat_v3},
+ {.compatible = "mediatek,mt8186-gce", .data = (void *)&gce_plat_v7},
{.compatible = "mediatek,mt6779-gce", .data = (void *)&gce_plat_v4},
{.compatible = "mediatek,mt8192-gce", .data = (void *)&gce_plat_v5},
{.compatible = "mediatek,mt8195-gce", .data = (void *)&gce_plat_v6},
{ .compatible = "qcom,sc8180x-apss-shared", .data = &apps_shared_apcs_data },
{ .compatible = "qcom,sdm660-apcs-hmss-global", .data = &msm8994_apcs_data },
{ .compatible = "qcom,sdm845-apss-shared", .data = &apps_shared_apcs_data },
+ { .compatible = "qcom,sm4250-apcs-hmss-global", .data = &msm8994_apcs_data },
{ .compatible = "qcom,sm6125-apcs-hmss-global", .data = &msm8994_apcs_data },
{ .compatible = "qcom,sm8150-apss-shared", .data = &apps_shared_apcs_data },
{ .compatible = "qcom,sm6115-apcs-hmss-global", .data = &msm8994_apcs_data },
static int rockchip_mbox_probe(struct platform_device *pdev)
{
struct rockchip_mbox *mb;
- const struct of_device_id *match;
const struct rockchip_mbox_data *drv_data;
struct resource *res;
int ret, irq, i;
if (!pdev->dev.of_node)
return -ENODEV;
- match = of_match_node(rockchip_mbox_of_match, pdev->dev.of_node);
- drv_data = (const struct rockchip_mbox_data *)match->data;
+ drv_data = (const struct rockchip_mbox_data *) device_get_match_data(&pdev->dev);
mb = devm_kzalloc(&pdev->dev, sizeof(*mb), GFP_KERNEL);
if (!mb)
ret = device_register(&ipi_mbox->dev);
if (ret) {
dev_err(dev, "Failed to register ipi mbox dev.\n");
+ put_device(&ipi_mbox->dev);
return ret;
}
mdev = &ipi_mbox->dev;
ipi_mbox = &pdata->ipi_mboxes[i];
if (ipi_mbox->dev.parent) {
mbox_controller_unregister(&ipi_mbox->mbox);
- device_unregister(&ipi_mbox->dev);
+ if (device_is_registered(&ipi_mbox->dev))
+ device_unregister(&ipi_mbox->dev);
}
}
}
get_device(dev);
ret = mdrv->probe(mdev, found_id);
- if (ret)
+ if (ret) {
module_put(carrier_mod);
+ put_device(dev);
+ }
return ret;
}
return 0;
err:
- mcb_free_dev(mdev);
+ put_device(&mdev->dev);
return ret;
}
EXPORT_SYMBOL_GPL(dvb_module_release);
#endif
-static int dvb_uevent(struct device *dev, struct kobj_uevent_env *env)
+static int dvb_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
- struct dvb_device *dvbdev = dev_get_drvdata(dev);
+ const struct dvb_device *dvbdev = dev_get_drvdata(dev);
add_uevent_var(env, "DVB_ADAPTER_NUM=%d", dvbdev->adapter->num);
add_uevent_var(env, "DVB_DEVICE_TYPE=%s", dnames[dvbdev->type]);
return 0;
}
-static char *dvb_devnode(struct device *dev, umode_t *mode)
+static char *dvb_devnode(const struct device *dev, umode_t *mode)
{
- struct dvb_device *dvbdev = dev_get_drvdata(dev);
+ const struct dvb_device *dvbdev = dev_get_drvdata(dev);
return kasprintf(GFP_KERNEL, "dvb/adapter%d/%s%d",
dvbdev->adapter->num, dnames[dvbdev->type], dvbdev->id);
.release = ddb_release,
};
-static char *ddb_devnode(struct device *device, umode_t *mode)
+static char *ddb_devnode(const struct device *device, umode_t *mode)
{
- struct ddb *dev = dev_get_drvdata(device);
+ const struct ddb *dev = dev_get_drvdata(device);
return kasprintf(GFP_KERNEL, "ddbridge/card%d", dev->nr);
}
enabled = !bridge->source_parallel.expected;
break;
default:
- break;
+ return -EINVAL;
}
source->subdev = remote_subdev;
{
struct sun6i_csi_device *csi_dev = video_drvdata(file);
struct sun6i_csi_capture *capture = &csi_dev->capture;
- int ret = 0;
+ int ret;
if (mutex_lock_interruptible(&capture->lock))
return -ERESTARTSYS;
return -ENODEV;
if (!on) {
- ret = v4l2_subdev_call(source_subdev, video, s_stream, 0);
+ v4l2_subdev_call(source_subdev, video, s_stream, 0);
+ ret = 0;
goto disable;
}
return 0;
disable:
- if (!on)
- ret = 0;
phy_power_off(dphy);
sun6i_mipi_csi2_disable(csi2_dev);
return -ENODEV;
if (!on) {
- ret = v4l2_subdev_call(source_subdev, video, s_stream, 0);
+ v4l2_subdev_call(source_subdev, video, s_stream, 0);
+ ret = 0;
goto disable;
}
return 0;
disable:
- if (!on)
- ret = 0;
phy_power_off(dphy);
sun8i_a83t_mipi_csi2_disable(csi2_dev);
}
/* class for /sys/class/rc */
-static char *rc_devnode(struct device *dev, umode_t *mode)
+static char *rc_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "rc/%s", dev_name(dev));
}
* then return an error.
*/
if (strlen(ctrl->p_new.p_char) == ctrl->maximum && last)
+ ctrl->is_new = 1;
return -ERANGE;
}
return ret;
.num_regs = 4,
.status_base = PM800_INT_STATUS1,
- .mask_base = PM800_INT_ENA_1,
+ .unmask_base = PM800_INT_ENA_1,
.ack_base = PM800_INT_STATUS1,
- .mask_invert = 1,
};
static int pm800_pages_init(struct pm80x_chip *chip)
return ret;
}
-static int pm800_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int pm800_probe(struct i2c_client *client)
{
int ret = 0;
struct pm80x_chip *chip;
static struct i2c_driver pm800_driver = {
.driver = {
.name = "88PM800",
- .pm = &pm80x_pm_ops,
+ .pm = pm_sleep_ptr(&pm80x_pm_ops),
},
- .probe = pm800_probe,
+ .probe_new = pm800_probe,
.remove = pm800_remove,
.id_table = pm80x_id_table,
};
return ret;
}
-static int pm805_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int pm805_probe(struct i2c_client *client)
{
int ret = 0;
struct pm80x_chip *chip;
static struct i2c_driver pm805_driver = {
.driver = {
.name = "88PM805",
- .pm = &pm80x_pm_ops,
+ .pm = pm_sleep_ptr(&pm80x_pm_ops),
},
- .probe = pm805_probe,
+ .probe_new = pm805_probe,
.remove = pm805_remove,
.id_table = pm80x_id_table,
};
}
EXPORT_SYMBOL_GPL(pm80x_deinit);
-#ifdef CONFIG_PM_SLEEP
static int pm80x_suspend(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
return 0;
}
-#endif
-SIMPLE_DEV_PM_OPS(pm80x_pm_ops, pm80x_suspend, pm80x_resume);
-EXPORT_SYMBOL_GPL(pm80x_pm_ops);
+EXPORT_GPL_SIMPLE_DEV_PM_OPS(pm80x_pm_ops, pm80x_suspend, pm80x_resume);
MODULE_DESCRIPTION("I2C Driver for Marvell 88PM80x");
MODULE_AUTHOR("Qiao Zhou <zhouqiao@marvell.com>");
}
}
-#ifdef CONFIG_PM_SLEEP
static int pm860x_suspend(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
disable_irq_wake(chip->core_irq);
return 0;
}
-#endif
-static SIMPLE_DEV_PM_OPS(pm860x_pm_ops, pm860x_suspend, pm860x_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(pm860x_pm_ops, pm860x_suspend, pm860x_resume);
static const struct i2c_device_id pm860x_id_table[] = {
{ "88PM860x", 0 },
static struct i2c_driver pm860x_driver = {
.driver = {
.name = "88PM860x",
- .pm = &pm860x_pm_ops,
+ .pm = pm_sleep_ptr(&pm860x_pm_ops),
.of_match_table = pm860x_dt_ids,
},
.probe_new = pm860x_probe,
help
Support for the AS3711 PMIC from AMS
+config MFD_SMPRO
+ tristate "Ampere Computing SMpro core driver"
+ depends on I2C
+ select MFD_CORE
+ select REGMAP_I2C
+ help
+ Say yes here to enable SMpro driver support for Ampere's Altra
+ processor family.
+
+ Ampere's Altra SMpro exposes an I2C regmap interface that can
+ be accessed by child devices.
+
config MFD_AS3722
tristate "ams AS3722 Power Management IC"
select MFD_CORE
HTC Magician devices, respectively. Actual functionality is
handled by the leds-pasic3 and ds1wm drivers.
-config HTC_I2CPLD
- bool "HTC I2C PLD chip support"
- depends on I2C=y && GPIOLIB
- help
- If you say yes here you get support for the supposed CPLD
- found on omap850 HTC devices like the HTC Wizard and HTC Herald.
- This device provides input and output GPIOs through an I2C
- interface to one or more sub-chips.
-
config MFD_INTEL_QUARK_I2C_GPIO
tristate "Intel Quark MFD I2C GPIO"
depends on PCI
bool "Support for Crystal Cove PMIC"
depends on HAS_IOMEM && I2C=y && GPIOLIB && COMMON_CLK
depends on (X86 && ACPI) || COMPILE_TEST
- depends on I2C_DESIGNWARE_PLATFORM=y
+ depends on I2C_DESIGNWARE_PLATFORM=y || COMPILE_TEST
select MFD_CORE
select REGMAP_I2C
select REGMAP_IRQ
config MFD_MAX77620
bool "Maxim Semiconductor MAX77620 and MAX20024 PMIC Support"
depends on I2C=y
- depends on OF || COMPILE_TEST
+ depends on OF
select MFD_CORE
select REGMAP_I2C
select REGMAP_IRQ
config MFD_MAX77650
tristate "Maxim MAX77650/77651 PMIC Support"
depends on I2C
- depends on OF || COMPILE_TEST
+ depends on OF
select MFD_CORE
select REGMAP_I2C
select REGMAP_IRQ
config MFD_MAX77686
tristate "Maxim Semiconductor MAX77686/802 PMIC Support"
depends on I2C
- depends on OF || COMPILE_TEST
+ depends on OF
select MFD_CORE
select REGMAP_I2C
select REGMAP_IRQ
config MFD_MAX77714
tristate "Maxim Semiconductor MAX77714 PMIC Support"
depends on I2C
- depends on OF || COMPILE_TEST
+ depends on OF
select MFD_CORE
select REGMAP_I2C
help
config MFD_CPCAP
tristate "Support for Motorola CPCAP"
depends on SPI
- depends on OF || COMPILE_TEST
+ depends on OF
select MFD_CORE
select REGMAP_SPI
select REGMAP_IRQ
config MFD_NTXEC
tristate "Netronix embedded controller (EC)"
- depends on OF || COMPILE_TEST
+ depends on OF
depends on I2C
select REGMAP_I2C
select MFD_CORE
config MFD_SEC_CORE
tristate "Samsung Electronics PMIC Series Support"
depends on I2C=y
- depends on OF || COMPILE_TEST
+ depends on OF
select MFD_CORE
select REGMAP_I2C
select REGMAP_IRQ
Select this option to enable accessing system control registers
via regmap.
-config MFD_DAVINCI_VOICECODEC
- tristate
- select MFD_CORE
- select REGMAP_MMIO
-
config MFD_TI_AM335X_TSCADC
tristate "TI ADC / Touch Screen chip support"
select MFD_CORE
To compile this driver as a module, choose M here: the
module will be called ti_am335x_tscadc.
-config MFD_DM355EVM_MSP
- bool "TI DaVinci DM355 EVM microcontroller"
- depends on I2C=y && MACH_DAVINCI_DM355_EVM
- help
- This driver supports the MSP430 microcontroller used on these
- boards. MSP430 firmware manages resets and power sequencing,
- inputs from buttons and the IR remote, LEDs, an RTC, and more.
-
config MFD_LP3943
tristate "TI/National Semiconductor LP3943 MFD Driver"
depends on I2C
OMAP USB Host drivers.
config MFD_PALMAS
- bool "TI Palmas series chips"
+ tristate "TI Palmas series chips"
select MFD_CORE
select REGMAP_I2C
select REGMAP_IRQ
This driver can also be built as a module. If so, the module
will be called tps65218.
+config MFD_TPS65219
+ tristate "TI TPS65219 Power Management IC"
+ depends on I2C && OF
+ select MFD_CORE
+ select REGMAP_I2C
+ select REGMAP_IRQ
+ help
+ If you say yes here you get support for the TPS65219 series of Power
+ Management ICs. These include voltage regulators, GPIOs and
+ push/power button that is often used in portable devices.
+
+ This driver can also be built as a module. If so, the module
+ will be called tps65219.
+
config MFD_TPS6586X
bool "TI TPS6586x Power Management chips"
depends on I2C=y
depends on I2C=y
depends on OF
select REGMAP_I2C
+ select REGMAP_IRQ
select MFD_CORE
help
Select this option to get support for the ROHM BD9576MUF and
config MFD_STMFX
tristate "Support for STMicroelectronics Multi-Function eXpander (STMFX)"
depends on I2C
- depends on OF || COMPILE_TEST
+ depends on OF
select MFD_CORE
select REGMAP_I2C
help
obj-$(CONFIG_MFD_GATEWORKS_GSC) += gateworks-gsc.o
obj-$(CONFIG_HTC_PASIC3) += htc-pasic3.o
-obj-$(CONFIG_HTC_I2CPLD) += htc-i2cpld.o
obj-$(CONFIG_MFD_TI_LP873X) += lp873x.o
obj-$(CONFIG_MFD_TI_LP87565) += lp87565.o
-
-obj-$(CONFIG_MFD_DAVINCI_VOICECODEC) += davinci_voicecodec.o
-obj-$(CONFIG_MFD_DM355EVM_MSP) += dm355evm_msp.o
obj-$(CONFIG_MFD_TI_AM335X_TSCADC) += ti_am335x_tscadc.o
obj-$(CONFIG_MFD_STA2X11) += sta2x11-mfd.o
obj-$(CONFIG_MFD_TPS65086) += tps65086.o
obj-$(CONFIG_MFD_TPS65217) += tps65217.o
obj-$(CONFIG_MFD_TPS65218) += tps65218.o
+obj-$(CONFIG_MFD_TPS65219) += tps65219.o
obj-$(CONFIG_MFD_TPS65910) += tps65910.o
obj-$(CONFIG_MFD_TPS65912) += tps65912-core.o
obj-$(CONFIG_MFD_TPS65912_I2C) += tps65912-i2c.o
obj-$(CONFIG_SGI_MFD_IOC3) += ioc3.o
obj-$(CONFIG_MFD_SIMPLE_MFD_I2C) += simple-mfd-i2c.o
+obj-$(CONFIG_MFD_SMPRO) += smpro-core.o
obj-$(CONFIG_MFD_INTEL_M10_BMC) += intel-m10-bmc.o
obj-$(CONFIG_MFD_ATC260X) += atc260x-core.o
}
#endif /* CONFIG_DEBUG_FS */
-static int aat2870_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int aat2870_i2c_probe(struct i2c_client *client)
{
struct aat2870_platform_data *pdata = dev_get_platdata(&client->dev);
struct aat2870_data *aat2870;
return ret;
}
-#ifdef CONFIG_PM_SLEEP
static int aat2870_i2c_suspend(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
return 0;
}
-#endif /* CONFIG_PM_SLEEP */
-static SIMPLE_DEV_PM_OPS(aat2870_pm_ops, aat2870_i2c_suspend,
- aat2870_i2c_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(aat2870_pm_ops, aat2870_i2c_suspend,
+ aat2870_i2c_resume);
static const struct i2c_device_id aat2870_i2c_id_table[] = {
{ "aat2870", 0 },
static struct i2c_driver aat2870_i2c_driver = {
.driver = {
.name = "aat2870",
- .pm = &aat2870_pm_ops,
+ .pm = pm_sleep_ptr(&aat2870_pm_ops),
.suppress_bind_attrs = true,
},
- .probe = aat2870_i2c_probe,
+ .probe_new = aat2870_i2c_probe,
.id_table = aat2870_i2c_id_table,
};
.val_bits = 8,
};
-static int act8945a_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int act8945a_i2c_probe(struct i2c_client *i2c)
{
int ret;
struct regmap *regmap;
.name = "act8945a",
.of_match_table = of_match_ptr(act8945a_of_match),
},
- .probe = act8945a_i2c_probe,
+ .probe_new = act8945a_i2c_probe,
.id_table = act8945a_i2c_id,
};
return device_for_each_child(chip->dev, NULL, __remove_subdev);
}
-static int adp5520_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int adp5520_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct adp5520_platform_data *pdata = dev_get_platdata(&client->dev);
struct platform_device *pdev;
struct adp5520_chip *chip;
return ret;
}
-#ifdef CONFIG_PM_SLEEP
static int adp5520_suspend(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
adp5520_write(chip->dev, ADP5520_MODE_STATUS, chip->mode);
return 0;
}
-#endif
-static SIMPLE_DEV_PM_OPS(adp5520_pm, adp5520_suspend, adp5520_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(adp5520_pm, adp5520_suspend, adp5520_resume);
static const struct i2c_device_id adp5520_id[] = {
{ "pmic-adp5520", ID_ADP5520 },
static struct i2c_driver adp5520_driver = {
.driver = {
.name = "adp5520",
- .pm = &adp5520_pm,
+ .pm = pm_sleep_ptr(&adp5520_pm),
.suppress_bind_attrs = true,
},
- .probe = adp5520_probe,
+ .probe_new = adp5520_probe,
.id_table = adp5520_id,
};
builtin_i2c_driver(adp5520_driver);
return 0;
}
-#ifdef CONFIG_PM
static int arizona_isolate_dcvdd(struct arizona *arizona)
{
int ret;
return 0;
}
-#endif
-#ifdef CONFIG_PM_SLEEP
static int arizona_suspend(struct device *dev)
{
struct arizona *arizona = dev_get_drvdata(dev);
return 0;
}
-#endif
-const struct dev_pm_ops arizona_pm_ops = {
- SET_RUNTIME_PM_OPS(arizona_runtime_suspend,
- arizona_runtime_resume,
- NULL)
- SET_SYSTEM_SLEEP_PM_OPS(arizona_suspend, arizona_resume)
- SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(arizona_suspend_noirq,
- arizona_resume_noirq)
+EXPORT_GPL_DEV_PM_OPS(arizona_pm_ops) = {
+ RUNTIME_PM_OPS(arizona_runtime_suspend,
+ arizona_runtime_resume,
+ NULL)
+ SYSTEM_SLEEP_PM_OPS(arizona_suspend, arizona_resume)
+ NOIRQ_SYSTEM_SLEEP_PM_OPS(arizona_suspend_noirq,
+ arizona_resume_noirq)
};
-EXPORT_SYMBOL_GPL(arizona_pm_ops);
#ifdef CONFIG_OF
static int arizona_of_get_core_pdata(struct arizona *arizona)
#include "arizona.h"
-static int arizona_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int arizona_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
const void *match_data;
struct arizona *arizona;
const struct regmap_config *regmap_config = NULL;
static struct i2c_driver arizona_i2c_driver = {
.driver = {
.name = "arizona",
- .pm = &arizona_pm_ops,
+ .pm = pm_ptr(&arizona_pm_ops),
.of_match_table = of_match_ptr(arizona_i2c_of_match),
},
- .probe = arizona_i2c_probe,
+ .probe_new = arizona_i2c_probe,
.remove = arizona_i2c_remove,
.id_table = arizona_i2c_id,
};
static struct spi_driver arizona_spi_driver = {
.driver = {
.name = "arizona",
- .pm = &arizona_pm_ops,
+ .pm = pm_ptr(&arizona_pm_ops),
.of_match_table = of_match_ptr(arizona_spi_of_match),
.acpi_match_table = ACPI_PTR(arizona_acpi_match),
},
};
#endif
-static int as3711_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int as3711_i2c_probe(struct i2c_client *client)
{
struct as3711 *as3711;
struct as3711_platform_data *pdata;
.name = "as3711",
.of_match_table = of_match_ptr(as3711_of_match),
},
- .probe = as3711_i2c_probe,
+ .probe_new = as3711_i2c_probe,
.id_table = as3711_i2c_id,
};
return 0;
}
-static int as3722_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int as3722_i2c_probe(struct i2c_client *i2c)
{
struct as3722 *as3722;
unsigned long irq_flags;
.of_match_table = as3722_of_match,
.pm = &as3722_pm_ops,
},
- .probe = as3722_i2c_probe,
+ .probe_new = as3722_i2c_probe,
.id_table = as3722_i2c_id,
};
.num_irqs = ARRAY_SIZE(atc2603c_regmap_irqs),
.num_regs = 1,
.status_base = ATC2603C_INTS_PD,
- .mask_base = ATC2603C_INTS_MSK,
- .mask_invert = true,
+ .unmask_base = ATC2603C_INTS_MSK,
};
static const struct regmap_irq_chip atc2609a_regmap_irq_chip = {
.num_irqs = ARRAY_SIZE(atc2609a_regmap_irqs),
.num_regs = 1,
.status_base = ATC2609A_INTS_PD,
- .mask_base = ATC2609A_INTS_MSK,
- .mask_invert = true,
+ .unmask_base = ATC2609A_INTS_MSK,
};
static const struct resource atc2603c_onkey_resources[] = {
#include <linux/of.h>
#include <linux/regmap.h>
-static int atc260x_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int atc260x_i2c_probe(struct i2c_client *client)
{
struct atc260x *atc260x;
struct regmap_config regmap_cfg;
.name = "atc260x",
.of_match_table = of_match_ptr(atc260x_i2c_of_match),
},
- .probe = atc260x_i2c_probe,
+ .probe_new = atc260x_i2c_probe,
};
module_i2c_driver(atc260x_i2c_driver);
#include <linux/regmap.h>
#include <linux/slab.h>
-static int axp20x_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int axp20x_i2c_probe(struct i2c_client *i2c)
{
struct axp20x_dev *axp20x;
int ret;
.of_match_table = of_match_ptr(axp20x_i2c_of_match),
.acpi_match_table = ACPI_PTR(axp20x_i2c_acpi_match),
},
- .probe = axp20x_i2c_probe,
+ .probe_new = axp20x_i2c_probe,
.remove = axp20x_i2c_remove,
.id_table = axp20x_i2c_id,
};
.name = "axp152_irq_chip",
.status_base = AXP152_IRQ1_STATE,
.ack_base = AXP152_IRQ1_STATE,
- .mask_base = AXP152_IRQ1_EN,
- .mask_invert = true,
+ .unmask_base = AXP152_IRQ1_EN,
.init_ack_masked = true,
.irqs = axp152_regmap_irqs,
.num_irqs = ARRAY_SIZE(axp152_regmap_irqs),
.name = "axp20x_irq_chip",
.status_base = AXP20X_IRQ1_STATE,
.ack_base = AXP20X_IRQ1_STATE,
- .mask_base = AXP20X_IRQ1_EN,
- .mask_invert = true,
+ .unmask_base = AXP20X_IRQ1_EN,
.init_ack_masked = true,
.irqs = axp20x_regmap_irqs,
.num_irqs = ARRAY_SIZE(axp20x_regmap_irqs),
.name = "axp22x_irq_chip",
.status_base = AXP20X_IRQ1_STATE,
.ack_base = AXP20X_IRQ1_STATE,
- .mask_base = AXP20X_IRQ1_EN,
- .mask_invert = true,
+ .unmask_base = AXP20X_IRQ1_EN,
.init_ack_masked = true,
.irqs = axp22x_regmap_irqs,
.num_irqs = ARRAY_SIZE(axp22x_regmap_irqs),
.name = "axp288_irq_chip",
.status_base = AXP20X_IRQ1_STATE,
.ack_base = AXP20X_IRQ1_STATE,
- .mask_base = AXP20X_IRQ1_EN,
- .mask_invert = true,
+ .unmask_base = AXP20X_IRQ1_EN,
.init_ack_masked = true,
.irqs = axp288_regmap_irqs,
.num_irqs = ARRAY_SIZE(axp288_regmap_irqs),
.name = "axp803",
.status_base = AXP20X_IRQ1_STATE,
.ack_base = AXP20X_IRQ1_STATE,
- .mask_base = AXP20X_IRQ1_EN,
- .mask_invert = true,
+ .unmask_base = AXP20X_IRQ1_EN,
.init_ack_masked = true,
.irqs = axp803_regmap_irqs,
.num_irqs = ARRAY_SIZE(axp803_regmap_irqs),
.name = "axp806",
.status_base = AXP20X_IRQ1_STATE,
.ack_base = AXP20X_IRQ1_STATE,
- .mask_base = AXP20X_IRQ1_EN,
- .mask_invert = true,
+ .unmask_base = AXP20X_IRQ1_EN,
.init_ack_masked = true,
.irqs = axp806_regmap_irqs,
.num_irqs = ARRAY_SIZE(axp806_regmap_irqs),
.name = "axp809",
.status_base = AXP20X_IRQ1_STATE,
.ack_base = AXP20X_IRQ1_STATE,
- .mask_base = AXP20X_IRQ1_EN,
- .mask_invert = true,
+ .unmask_base = AXP20X_IRQ1_EN,
.init_ack_masked = true,
.irqs = axp809_regmap_irqs,
.num_irqs = ARRAY_SIZE(axp809_regmap_irqs),
AXP20X_OFF);
/* Give capacitors etc. time to drain to avoid kernel panic msg. */
- msleep(500);
+ mdelay(500);
}
int axp20x_match_device(struct axp20x_dev *axp20x)
.cache_type = REGCACHE_RBTREE,
};
-static int bcm590xx_i2c_probe(struct i2c_client *i2c_pri,
- const struct i2c_device_id *id)
+static int bcm590xx_i2c_probe(struct i2c_client *i2c_pri)
{
struct bcm590xx *bcm590xx;
int ret;
.name = "bcm590xx",
.of_match_table = bcm590xx_of_match,
},
- .probe = bcm590xx_i2c_probe,
+ .probe_new = bcm590xx_i2c_probe,
.id_table = bcm590xx_i2c_id,
};
module_i2c_driver(bcm590xx_i2c_driver);
return 0;
}
-static int bd9571mwv_probe(struct i2c_client *client,
- const struct i2c_device_id *ids)
+static int bd9571mwv_probe(struct i2c_client *client)
{
const struct regmap_config *regmap_config;
const struct regmap_irq_chip *irq_chip;
.name = "bd9571mwv",
.of_match_table = bd9571mwv_of_match_table,
},
- .probe = bd9571mwv_probe,
+ .probe_new = bd9571mwv_probe,
.id_table = bd9571mwv_id_table,
};
module_i2c_driver(bd9571mwv_driver);
return ret;
}
-static int da903x_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int da903x_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct da903x_platform_data *pdata = dev_get_platdata(&client->dev);
struct da903x_chip *chip;
unsigned int tmp;
.driver = {
.name = "da903x",
},
- .probe = da903x_probe,
+ .probe_new = da903x_probe,
.remove = da903x_remove,
.id_table = da903x_id_table,
};
};
#endif
-static int da9052_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int da9052_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct da9052 *da9052;
int ret;
}
static struct i2c_driver da9052_i2c_driver = {
- .probe = da9052_i2c_probe,
+ .probe_new = da9052_i2c_probe,
.remove = da9052_i2c_remove,
.id_table = da9052_i2c_id,
.driver = {
#include <linux/mfd/da9055/core.h>
-static int da9055_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int da9055_i2c_probe(struct i2c_client *i2c)
{
struct da9055 *da9055;
int ret;
};
static struct i2c_driver da9055_i2c_driver = {
- .probe = da9055_i2c_probe,
+ .probe_new = da9055_i2c_probe,
.remove = da9055_i2c_remove,
.id_table = da9055_i2c_id,
.driver = {
};
MODULE_DEVICE_TABLE(of, da9062_dt_ids);
-static int da9062_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int da9062_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct da9062 *chip;
unsigned int irq_base;
const struct mfd_cell *cell;
.name = "da9062",
.of_match_table = da9062_dt_ids,
},
- .probe = da9062_i2c_probe,
+ .probe_new = da9062_i2c_probe,
.remove = da9062_i2c_remove,
.id_table = da9062_i2c_id,
};
{ }
};
MODULE_DEVICE_TABLE(of, da9063_dt_ids);
-static int da9063_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int da9063_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct da9063 *da9063;
int ret;
.name = "da9063",
.of_match_table = da9063_dt_ids,
},
- .probe = da9063_i2c_probe,
+ .probe_new = da9063_i2c_probe,
.id_table = da9063_i2c_id,
};
},
};
-static int da9150_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int da9150_probe(struct i2c_client *client)
{
struct da9150 *da9150;
struct da9150_pdata *pdata = dev_get_platdata(&client->dev);
.name = "da9150",
.of_match_table = da9150_of_match,
},
- .probe = da9150_probe,
+ .probe_new = da9150_probe,
.remove = da9150_remove,
.shutdown = da9150_shutdown,
.id_table = da9150_i2c_id,
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * DaVinci Voice Codec Core Interface for TI platforms
- *
- * Copyright (C) 2010 Texas Instruments, Inc
- *
- * Author: Miguel Aguilar <miguel.aguilar@ridgerun.com>
- */
-
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/device.h>
-#include <linux/slab.h>
-#include <linux/delay.h>
-#include <linux/io.h>
-#include <linux/clk.h>
-#include <linux/regmap.h>
-
-#include <sound/pcm.h>
-
-#include <linux/mfd/davinci_voicecodec.h>
-
-static const struct regmap_config davinci_vc_regmap = {
- .reg_bits = 32,
- .val_bits = 32,
-};
-
-static int __init davinci_vc_probe(struct platform_device *pdev)
-{
- struct davinci_vc *davinci_vc;
- struct resource *res;
- struct mfd_cell *cell = NULL;
- dma_addr_t fifo_base;
- int ret;
-
- davinci_vc = devm_kzalloc(&pdev->dev,
- sizeof(struct davinci_vc), GFP_KERNEL);
- if (!davinci_vc)
- return -ENOMEM;
-
- davinci_vc->clk = devm_clk_get(&pdev->dev, NULL);
- if (IS_ERR(davinci_vc->clk)) {
- dev_dbg(&pdev->dev,
- "could not get the clock for voice codec\n");
- return -ENODEV;
- }
- clk_enable(davinci_vc->clk);
-
- davinci_vc->base = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
- if (IS_ERR(davinci_vc->base)) {
- ret = PTR_ERR(davinci_vc->base);
- goto fail;
- }
- fifo_base = (dma_addr_t)res->start;
-
- davinci_vc->regmap = devm_regmap_init_mmio(&pdev->dev,
- davinci_vc->base,
- &davinci_vc_regmap);
- if (IS_ERR(davinci_vc->regmap)) {
- ret = PTR_ERR(davinci_vc->regmap);
- goto fail;
- }
-
- res = platform_get_resource(pdev, IORESOURCE_DMA, 0);
- if (!res) {
- dev_err(&pdev->dev, "no DMA resource\n");
- ret = -ENXIO;
- goto fail;
- }
-
- davinci_vc->davinci_vcif.dma_tx_channel = res->start;
- davinci_vc->davinci_vcif.dma_tx_addr = fifo_base + DAVINCI_VC_WFIFO;
-
- res = platform_get_resource(pdev, IORESOURCE_DMA, 1);
- if (!res) {
- dev_err(&pdev->dev, "no DMA resource\n");
- ret = -ENXIO;
- goto fail;
- }
-
- davinci_vc->davinci_vcif.dma_rx_channel = res->start;
- davinci_vc->davinci_vcif.dma_rx_addr = fifo_base + DAVINCI_VC_RFIFO;
-
- davinci_vc->dev = &pdev->dev;
- davinci_vc->pdev = pdev;
-
- /* Voice codec interface client */
- cell = &davinci_vc->cells[DAVINCI_VC_VCIF_CELL];
- cell->name = "davinci-vcif";
- cell->platform_data = davinci_vc;
- cell->pdata_size = sizeof(*davinci_vc);
-
- /* Voice codec CQ93VC client */
- cell = &davinci_vc->cells[DAVINCI_VC_CQ93VC_CELL];
- cell->name = "cq93vc-codec";
- cell->platform_data = davinci_vc;
- cell->pdata_size = sizeof(*davinci_vc);
-
- ret = mfd_add_devices(&pdev->dev, pdev->id, davinci_vc->cells,
- DAVINCI_VC_CELLS, NULL, 0, NULL);
- if (ret != 0) {
- dev_err(&pdev->dev, "fail to register client devices\n");
- goto fail;
- }
-
- return 0;
-
-fail:
- clk_disable(davinci_vc->clk);
-
- return ret;
-}
-
-static int davinci_vc_remove(struct platform_device *pdev)
-{
- struct davinci_vc *davinci_vc = platform_get_drvdata(pdev);
-
- mfd_remove_devices(&pdev->dev);
-
- clk_disable(davinci_vc->clk);
-
- return 0;
-}
-
-static struct platform_driver davinci_vc_driver = {
- .driver = {
- .name = "davinci_voicecodec",
- },
- .remove = davinci_vc_remove,
-};
-
-module_platform_driver_probe(davinci_vc_driver, davinci_vc_probe);
-
-MODULE_AUTHOR("Miguel Aguilar");
-MODULE_DESCRIPTION("Texas Instruments DaVinci Voice Codec Core Interface");
-MODULE_LICENSE("GPL");
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * dm355evm_msp.c - driver for MSP430 firmware on DM355EVM board
- *
- * Copyright (C) 2008 David Brownell
- */
-
-#include <linux/init.h>
-#include <linux/mutex.h>
-#include <linux/platform_device.h>
-#include <linux/clk.h>
-#include <linux/module.h>
-#include <linux/err.h>
-#include <linux/gpio.h>
-#include <linux/gpio/machine.h>
-#include <linux/leds.h>
-#include <linux/i2c.h>
-#include <linux/mfd/dm355evm_msp.h>
-
-
-/*
- * The DM355 is a DaVinci chip with video support but no C64+ DSP. Its
- * EVM board has an MSP430 programmed with firmware for various board
- * support functions. This driver exposes some of them directly, and
- * supports other drivers (e.g. RTC, input) for more complex access.
- *
- * Because this firmware is entirely board-specific, this file embeds
- * knowledge that would be passed as platform_data in a generic driver.
- *
- * This driver was tested with firmware revision A4.
- */
-
-#if IS_ENABLED(CONFIG_INPUT_DM355EVM)
-#define msp_has_keyboard() true
-#else
-#define msp_has_keyboard() false
-#endif
-
-#if IS_ENABLED(CONFIG_LEDS_GPIO)
-#define msp_has_leds() true
-#else
-#define msp_has_leds() false
-#endif
-
-#if IS_ENABLED(CONFIG_RTC_DRV_DM355EVM)
-#define msp_has_rtc() true
-#else
-#define msp_has_rtc() false
-#endif
-
-#if IS_ENABLED(CONFIG_VIDEO_TVP514X)
-#define msp_has_tvp() true
-#else
-#define msp_has_tvp() false
-#endif
-
-
-/*----------------------------------------------------------------------*/
-
-/* REVISIT for paranoia's sake, retry reads/writes on error */
-
-static struct i2c_client *msp430;
-
-/**
- * dm355evm_msp_write - Writes a register in dm355evm_msp
- * @value: the value to be written
- * @reg: register address
- *
- * Returns result of operation - 0 is success, else negative errno
- */
-int dm355evm_msp_write(u8 value, u8 reg)
-{
- return i2c_smbus_write_byte_data(msp430, reg, value);
-}
-EXPORT_SYMBOL(dm355evm_msp_write);
-
-/**
- * dm355evm_msp_read - Reads a register from dm355evm_msp
- * @reg: register address
- *
- * Returns result of operation - value, or negative errno
- */
-int dm355evm_msp_read(u8 reg)
-{
- return i2c_smbus_read_byte_data(msp430, reg);
-}
-EXPORT_SYMBOL(dm355evm_msp_read);
-
-/*----------------------------------------------------------------------*/
-
-/*
- * Many of the msp430 pins are just used as fixed-direction GPIOs.
- * We could export a few more of them this way, if we wanted.
- */
-#define MSP_GPIO(bit, reg) ((DM355EVM_MSP_ ## reg) << 3 | (bit))
-
-static const u8 msp_gpios[] = {
- /* eight leds */
- MSP_GPIO(0, LED), MSP_GPIO(1, LED),
- MSP_GPIO(2, LED), MSP_GPIO(3, LED),
- MSP_GPIO(4, LED), MSP_GPIO(5, LED),
- MSP_GPIO(6, LED), MSP_GPIO(7, LED),
- /* SW6 and the NTSC/nPAL jumper */
- MSP_GPIO(0, SWITCH1), MSP_GPIO(1, SWITCH1),
- MSP_GPIO(2, SWITCH1), MSP_GPIO(3, SWITCH1),
- MSP_GPIO(4, SWITCH1),
- /* switches on MMC/SD sockets */
- /*
- * Note: EVMDM355_ECP_VA4.pdf suggests that Bit 2 and 4 should be
- * checked for card detection. However on the EVM bit 1 and 3 gives
- * this status, for 0 and 1 instance respectively. The pdf also
- * suggests that Bit 1 and 3 should be checked for write protection.
- * However on the EVM bit 2 and 4 gives this status,for 0 and 1
- * instance respectively.
- */
- MSP_GPIO(2, SDMMC), MSP_GPIO(1, SDMMC), /* mmc0 WP, nCD */
- MSP_GPIO(4, SDMMC), MSP_GPIO(3, SDMMC), /* mmc1 WP, nCD */
-};
-
-static struct gpio_led evm_leds[] = {
- { .name = "dm355evm::ds14",
- .default_trigger = "heartbeat", },
- { .name = "dm355evm::ds15",
- .default_trigger = "mmc0", },
- { .name = "dm355evm::ds16",
- /* could also be a CE-ATA drive */
- .default_trigger = "mmc1", },
- { .name = "dm355evm::ds17",
- .default_trigger = "nand-disk", },
- { .name = "dm355evm::ds18", },
- { .name = "dm355evm::ds19", },
- { .name = "dm355evm::ds20", },
- { .name = "dm355evm::ds21", },
-};
-
-static struct gpio_led_platform_data evm_led_data = {
- .num_leds = ARRAY_SIZE(evm_leds),
- .leds = evm_leds,
-};
-
-static struct gpiod_lookup_table evm_leds_gpio_table = {
- .dev_id = "leds-gpio",
- .table = {
- /*
- * These GPIOs are on the dm355evm_msp
- * GPIO chip at index 0..7
- */
- GPIO_LOOKUP_IDX("dm355evm_msp", 0, NULL,
- 0, GPIO_ACTIVE_LOW),
- GPIO_LOOKUP_IDX("dm355evm_msp", 1, NULL,
- 1, GPIO_ACTIVE_LOW),
- GPIO_LOOKUP_IDX("dm355evm_msp", 2, NULL,
- 2, GPIO_ACTIVE_LOW),
- GPIO_LOOKUP_IDX("dm355evm_msp", 3, NULL,
- 3, GPIO_ACTIVE_LOW),
- GPIO_LOOKUP_IDX("dm355evm_msp", 4, NULL,
- 4, GPIO_ACTIVE_LOW),
- GPIO_LOOKUP_IDX("dm355evm_msp", 5, NULL,
- 5, GPIO_ACTIVE_LOW),
- GPIO_LOOKUP_IDX("dm355evm_msp", 6, NULL,
- 6, GPIO_ACTIVE_LOW),
- GPIO_LOOKUP_IDX("dm355evm_msp", 7, NULL,
- 7, GPIO_ACTIVE_LOW),
- { },
- },
-};
-
-#define MSP_GPIO_REG(offset) (msp_gpios[(offset)] >> 3)
-#define MSP_GPIO_MASK(offset) BIT(msp_gpios[(offset)] & 0x07)
-
-static int msp_gpio_in(struct gpio_chip *chip, unsigned offset)
-{
- switch (MSP_GPIO_REG(offset)) {
- case DM355EVM_MSP_SWITCH1:
- case DM355EVM_MSP_SWITCH2:
- case DM355EVM_MSP_SDMMC:
- return 0;
- default:
- return -EINVAL;
- }
-}
-
-static u8 msp_led_cache;
-
-static int msp_gpio_get(struct gpio_chip *chip, unsigned offset)
-{
- int reg, status;
-
- reg = MSP_GPIO_REG(offset);
- status = dm355evm_msp_read(reg);
- if (status < 0)
- return status;
- if (reg == DM355EVM_MSP_LED)
- msp_led_cache = status;
- return !!(status & MSP_GPIO_MASK(offset));
-}
-
-static int msp_gpio_out(struct gpio_chip *chip, unsigned offset, int value)
-{
- int mask, bits;
-
- /* NOTE: there are some other signals that could be
- * packaged as output GPIOs, but they aren't as useful
- * as the LEDs ... so for now we don't.
- */
- if (MSP_GPIO_REG(offset) != DM355EVM_MSP_LED)
- return -EINVAL;
-
- mask = MSP_GPIO_MASK(offset);
- bits = msp_led_cache;
-
- bits &= ~mask;
- if (value)
- bits |= mask;
- msp_led_cache = bits;
-
- return dm355evm_msp_write(bits, DM355EVM_MSP_LED);
-}
-
-static void msp_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
-{
- msp_gpio_out(chip, offset, value);
-}
-
-static struct gpio_chip dm355evm_msp_gpio = {
- .label = "dm355evm_msp",
- .owner = THIS_MODULE,
- .direction_input = msp_gpio_in,
- .get = msp_gpio_get,
- .direction_output = msp_gpio_out,
- .set = msp_gpio_set,
- .base = -EINVAL, /* dynamic assignment */
- .ngpio = ARRAY_SIZE(msp_gpios),
- .can_sleep = true,
-};
-
-/*----------------------------------------------------------------------*/
-
-static struct device *add_child(struct i2c_client *client, const char *name,
- void *pdata, unsigned pdata_len,
- bool can_wakeup, int irq)
-{
- struct platform_device *pdev;
- int status;
-
- pdev = platform_device_alloc(name, -1);
- if (!pdev)
- return ERR_PTR(-ENOMEM);
-
- device_init_wakeup(&pdev->dev, can_wakeup);
- pdev->dev.parent = &client->dev;
-
- if (pdata) {
- status = platform_device_add_data(pdev, pdata, pdata_len);
- if (status < 0) {
- dev_dbg(&pdev->dev, "can't add platform_data\n");
- goto put_device;
- }
- }
-
- if (irq) {
- struct resource r = {
- .start = irq,
- .flags = IORESOURCE_IRQ,
- };
-
- status = platform_device_add_resources(pdev, &r, 1);
- if (status < 0) {
- dev_dbg(&pdev->dev, "can't add irq\n");
- goto put_device;
- }
- }
-
- status = platform_device_add(pdev);
- if (status)
- goto put_device;
-
- return &pdev->dev;
-
-put_device:
- platform_device_put(pdev);
- dev_err(&client->dev, "failed to add device %s\n", name);
- return ERR_PTR(status);
-}
-
-static int add_children(struct i2c_client *client)
-{
- static const struct {
- int offset;
- char *label;
- } config_inputs[] = {
- /* 8 == right after the LEDs */
- { 8 + 0, "sw6_1", },
- { 8 + 1, "sw6_2", },
- { 8 + 2, "sw6_3", },
- { 8 + 3, "sw6_4", },
- { 8 + 4, "NTSC/nPAL", },
- };
-
- struct device *child;
- int status;
- int i;
-
- /* GPIO-ish stuff */
- dm355evm_msp_gpio.parent = &client->dev;
- status = gpiochip_add_data(&dm355evm_msp_gpio, NULL);
- if (status < 0)
- return status;
-
- /* LED output */
- if (msp_has_leds()) {
- gpiod_add_lookup_table(&evm_leds_gpio_table);
- /* NOTE: these are the only fully programmable LEDs
- * on the board, since GPIO-61/ds22 (and many signals
- * going to DC7) must be used for AEMIF address lines
- * unless the top 1 GB of NAND is unused...
- */
- child = add_child(client, "leds-gpio",
- &evm_led_data, sizeof(evm_led_data),
- false, 0);
- if (IS_ERR(child))
- return PTR_ERR(child);
- }
-
- /* configuration inputs */
- for (i = 0; i < ARRAY_SIZE(config_inputs); i++) {
- int gpio = dm355evm_msp_gpio.base + config_inputs[i].offset;
-
- gpio_request_one(gpio, GPIOF_IN, config_inputs[i].label);
-
- /* make it easy for userspace to see these */
- gpio_export(gpio, false);
- }
-
- /* MMC/SD inputs -- right after the last config input */
- if (dev_get_platdata(&client->dev)) {
- void (*mmcsd_setup)(unsigned) = dev_get_platdata(&client->dev);
-
- mmcsd_setup(dm355evm_msp_gpio.base + 8 + 5);
- }
-
- /* RTC is a 32 bit counter, no alarm */
- if (msp_has_rtc()) {
- child = add_child(client, "rtc-dm355evm",
- NULL, 0, false, 0);
- if (IS_ERR(child))
- return PTR_ERR(child);
- }
-
- /* input from buttons and IR remote (uses the IRQ) */
- if (msp_has_keyboard()) {
- child = add_child(client, "dm355evm_keys",
- NULL, 0, true, client->irq);
- if (IS_ERR(child))
- return PTR_ERR(child);
- }
-
- return 0;
-}
-
-/*----------------------------------------------------------------------*/
-
-static void dm355evm_command(unsigned command)
-{
- int status;
-
- status = dm355evm_msp_write(command, DM355EVM_MSP_COMMAND);
- if (status < 0)
- dev_err(&msp430->dev, "command %d failure %d\n",
- command, status);
-}
-
-static void dm355evm_power_off(void)
-{
- dm355evm_command(MSP_COMMAND_POWEROFF);
-}
-
-static void dm355evm_msp_remove(struct i2c_client *client)
-{
- pm_power_off = NULL;
- msp430 = NULL;
-}
-
-static int
-dm355evm_msp_probe(struct i2c_client *client, const struct i2c_device_id *id)
-{
- int status;
- const char *video = msp_has_tvp() ? "TVP5146" : "imager";
-
- if (msp430)
- return -EBUSY;
- msp430 = client;
-
- /* display revision status; doubles as sanity check */
- status = dm355evm_msp_read(DM355EVM_MSP_FIRMREV);
- if (status < 0)
- goto fail;
- dev_info(&client->dev, "firmware v.%02X, %s as video-in\n",
- status, video);
-
- /* mux video input: either tvp5146 or some external imager */
- status = dm355evm_msp_write(msp_has_tvp() ? 0 : MSP_VIDEO_IMAGER,
- DM355EVM_MSP_VIDEO_IN);
- if (status < 0)
- dev_warn(&client->dev, "error %d muxing %s as video-in\n",
- status, video);
-
- /* init LED cache, and turn off the LEDs */
- msp_led_cache = 0xff;
- dm355evm_msp_write(msp_led_cache, DM355EVM_MSP_LED);
-
- /* export capabilities we support */
- status = add_children(client);
- if (status < 0)
- goto fail;
-
- /* PM hookup */
- pm_power_off = dm355evm_power_off;
-
- return 0;
-
-fail:
- /* FIXME remove children ... */
- dm355evm_msp_remove(client);
- return status;
-}
-
-static const struct i2c_device_id dm355evm_msp_ids[] = {
- { "dm355evm_msp", 0 },
- { /* end of list */ },
-};
-MODULE_DEVICE_TABLE(i2c, dm355evm_msp_ids);
-
-static struct i2c_driver dm355evm_msp_driver = {
- .driver.name = "dm355evm_msp",
- .id_table = dm355evm_msp_ids,
- .probe = dm355evm_msp_probe,
- .remove = dm355evm_msp_remove,
-};
-
-static int __init dm355evm_msp_init(void)
-{
- return i2c_add_driver(&dm355evm_msp_driver);
-}
-subsys_initcall(dm355evm_msp_init);
-
-static void __exit dm355evm_msp_exit(void)
-{
- i2c_del_driver(&dm355evm_msp_driver);
-}
-module_exit(dm355evm_msp_exit);
-
-MODULE_DESCRIPTION("Interface to MSP430 firmware on DM355EVM");
-MODULE_LICENSE("GPL");
{
struct device *dev = &pdev->dev;
struct mx25_tsadc *tsadc;
- struct resource *res;
int ret;
void __iomem *iomem;
if (!tsadc)
return -ENOMEM;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- iomem = devm_ioremap_resource(dev, res);
+ iomem = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
if (IS_ERR(iomem))
return PTR_ERR(iomem);
.num_irqs = ARRAY_SIZE(gsc_irqs),
.num_regs = 1,
.status_base = GSC_IRQ_STATUS,
- .mask_base = GSC_IRQ_ENABLE,
- .mask_invert = true,
+ .unmask_base = GSC_IRQ_ENABLE,
.ack_base = GSC_IRQ_STATUS,
.ack_invert = true,
};
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * htc-i2cpld.c
- * Chip driver for an unknown CPLD chip found on omap850 HTC devices like
- * the HTC Wizard and HTC Herald.
- * The cpld is located on the i2c bus and acts as an input/output GPIO
- * extender.
- *
- * Copyright (C) 2009 Cory Maccarrone <darkstar6262@gmail.com>
- *
- * Based on work done in the linwizard project
- * Copyright (C) 2008-2009 Angelo Arrifano <miknix@gmail.com>
- */
-
-#include <linux/kernel.h>
-#include <linux/init.h>
-#include <linux/interrupt.h>
-#include <linux/platform_device.h>
-#include <linux/i2c.h>
-#include <linux/irq.h>
-#include <linux/spinlock.h>
-#include <linux/htcpld.h>
-#include <linux/gpio/driver.h>
-#include <linux/gpio/machine.h>
-#include <linux/gpio/consumer.h>
-#include <linux/slab.h>
-
-struct htcpld_chip {
- spinlock_t lock;
-
- /* chip info */
- u8 reset;
- u8 addr;
- struct device *dev;
- struct i2c_client *client;
-
- /* Output details */
- u8 cache_out;
- struct gpio_chip chip_out;
-
- /* Input details */
- u8 cache_in;
- struct gpio_chip chip_in;
-
- u16 irqs_enabled;
- uint irq_start;
- int nirqs;
-
- unsigned int flow_type;
- /*
- * Work structure to allow for setting values outside of any
- * possible interrupt context
- */
- struct work_struct set_val_work;
-};
-
-struct htcpld_data {
- /* irq info */
- u16 irqs_enabled;
- uint irq_start;
- int nirqs;
- uint chained_irq;
- struct gpio_desc *int_reset_gpio_hi;
- struct gpio_desc *int_reset_gpio_lo;
-
- /* htcpld info */
- struct htcpld_chip *chip;
- unsigned int nchips;
-};
-
-/* There does not appear to be a way to proactively mask interrupts
- * on the htcpld chip itself. So, we simply ignore interrupts that
- * aren't desired. */
-static void htcpld_mask(struct irq_data *data)
-{
- struct htcpld_chip *chip = irq_data_get_irq_chip_data(data);
- chip->irqs_enabled &= ~(1 << (data->irq - chip->irq_start));
- pr_debug("HTCPLD mask %d %04x\n", data->irq, chip->irqs_enabled);
-}
-static void htcpld_unmask(struct irq_data *data)
-{
- struct htcpld_chip *chip = irq_data_get_irq_chip_data(data);
- chip->irqs_enabled |= 1 << (data->irq - chip->irq_start);
- pr_debug("HTCPLD unmask %d %04x\n", data->irq, chip->irqs_enabled);
-}
-
-static int htcpld_set_type(struct irq_data *data, unsigned int flags)
-{
- struct htcpld_chip *chip = irq_data_get_irq_chip_data(data);
-
- if (flags & ~IRQ_TYPE_SENSE_MASK)
- return -EINVAL;
-
- /* We only allow edge triggering */
- if (flags & (IRQ_TYPE_LEVEL_LOW|IRQ_TYPE_LEVEL_HIGH))
- return -EINVAL;
-
- chip->flow_type = flags;
- return 0;
-}
-
-static struct irq_chip htcpld_muxed_chip = {
- .name = "htcpld",
- .irq_mask = htcpld_mask,
- .irq_unmask = htcpld_unmask,
- .irq_set_type = htcpld_set_type,
-};
-
-/* To properly dispatch IRQ events, we need to read from the
- * chip. This is an I2C action that could possibly sleep
- * (which is bad in interrupt context) -- so we use a threaded
- * interrupt handler to get around that.
- */
-static irqreturn_t htcpld_handler(int irq, void *dev)
-{
- struct htcpld_data *htcpld = dev;
- unsigned int i;
- unsigned long flags;
- int irqpin;
-
- if (!htcpld) {
- pr_debug("htcpld is null in ISR\n");
- return IRQ_HANDLED;
- }
-
- /*
- * For each chip, do a read of the chip and trigger any interrupts
- * desired. The interrupts will be triggered from LSB to MSB (i.e.
- * bit 0 first, then bit 1, etc.)
- *
- * For chips that have no interrupt range specified, just skip 'em.
- */
- for (i = 0; i < htcpld->nchips; i++) {
- struct htcpld_chip *chip = &htcpld->chip[i];
- struct i2c_client *client;
- int val;
- unsigned long uval, old_val;
-
- if (!chip) {
- pr_debug("chip %d is null in ISR\n", i);
- continue;
- }
-
- if (chip->nirqs == 0)
- continue;
-
- client = chip->client;
- if (!client) {
- pr_debug("client %d is null in ISR\n", i);
- continue;
- }
-
- /* Scan the chip */
- val = i2c_smbus_read_byte_data(client, chip->cache_out);
- if (val < 0) {
- /* Throw a warning and skip this chip */
- dev_warn(chip->dev, "Unable to read from chip: %d\n",
- val);
- continue;
- }
-
- uval = (unsigned long)val;
-
- spin_lock_irqsave(&chip->lock, flags);
-
- /* Save away the old value so we can compare it */
- old_val = chip->cache_in;
-
- /* Write the new value */
- chip->cache_in = uval;
-
- spin_unlock_irqrestore(&chip->lock, flags);
-
- /*
- * For each bit in the data (starting at bit 0), trigger
- * associated interrupts.
- */
- for (irqpin = 0; irqpin < chip->nirqs; irqpin++) {
- unsigned oldb, newb, type = chip->flow_type;
-
- irq = chip->irq_start + irqpin;
-
- /* Run the IRQ handler, but only if the bit value
- * changed, and the proper flags are set */
- oldb = (old_val >> irqpin) & 1;
- newb = (uval >> irqpin) & 1;
-
- if ((!oldb && newb && (type & IRQ_TYPE_EDGE_RISING)) ||
- (oldb && !newb && (type & IRQ_TYPE_EDGE_FALLING))) {
- pr_debug("fire IRQ %d\n", irqpin);
- generic_handle_irq(irq);
- }
- }
- }
-
- /*
- * In order to continue receiving interrupts, the int_reset_gpio must
- * be asserted.
- */
- if (htcpld->int_reset_gpio_hi)
- gpiod_set_value(htcpld->int_reset_gpio_hi, 1);
- if (htcpld->int_reset_gpio_lo)
- gpiod_set_value(htcpld->int_reset_gpio_lo, 0);
-
- return IRQ_HANDLED;
-}
-
-/*
- * The GPIO set routines can be called from interrupt context, especially if,
- * for example they're attached to the led-gpio framework and a trigger is
- * enabled. As such, we declared work above in the htcpld_chip structure,
- * and that work is scheduled in the set routine. The kernel can then run
- * the I2C functions, which will sleep, in process context.
- */
-static void htcpld_chip_set(struct gpio_chip *chip, unsigned offset, int val)
-{
- struct i2c_client *client;
- struct htcpld_chip *chip_data = gpiochip_get_data(chip);
- unsigned long flags;
-
- client = chip_data->client;
- if (!client)
- return;
-
- spin_lock_irqsave(&chip_data->lock, flags);
- if (val)
- chip_data->cache_out |= (1 << offset);
- else
- chip_data->cache_out &= ~(1 << offset);
- spin_unlock_irqrestore(&chip_data->lock, flags);
-
- schedule_work(&(chip_data->set_val_work));
-}
-
-static void htcpld_chip_set_ni(struct work_struct *work)
-{
- struct htcpld_chip *chip_data;
- struct i2c_client *client;
-
- chip_data = container_of(work, struct htcpld_chip, set_val_work);
- client = chip_data->client;
- i2c_smbus_read_byte_data(client, chip_data->cache_out);
-}
-
-static int htcpld_chip_get(struct gpio_chip *chip, unsigned offset)
-{
- struct htcpld_chip *chip_data = gpiochip_get_data(chip);
- u8 cache;
-
- if (!strncmp(chip->label, "htcpld-out", 10)) {
- cache = chip_data->cache_out;
- } else if (!strncmp(chip->label, "htcpld-in", 9)) {
- cache = chip_data->cache_in;
- } else
- return -EINVAL;
-
- return (cache >> offset) & 1;
-}
-
-static int htcpld_direction_output(struct gpio_chip *chip,
- unsigned offset, int value)
-{
- htcpld_chip_set(chip, offset, value);
- return 0;
-}
-
-static int htcpld_direction_input(struct gpio_chip *chip,
- unsigned offset)
-{
- /*
- * No-op: this function can only be called on the input chip.
- * We do however make sure the offset is within range.
- */
- return (offset < chip->ngpio) ? 0 : -EINVAL;
-}
-
-static int htcpld_chip_to_irq(struct gpio_chip *chip, unsigned offset)
-{
- struct htcpld_chip *chip_data = gpiochip_get_data(chip);
-
- if (offset < chip_data->nirqs)
- return chip_data->irq_start + offset;
- else
- return -EINVAL;
-}
-
-static void htcpld_chip_reset(struct i2c_client *client)
-{
- struct htcpld_chip *chip_data = i2c_get_clientdata(client);
- if (!chip_data)
- return;
-
- i2c_smbus_read_byte_data(
- client, (chip_data->cache_out = chip_data->reset));
-}
-
-static int htcpld_setup_chip_irq(
- struct platform_device *pdev,
- int chip_index)
-{
- struct htcpld_data *htcpld;
- struct htcpld_chip *chip;
- unsigned int irq, irq_end;
-
- /* Get the platform and driver data */
- htcpld = platform_get_drvdata(pdev);
- chip = &htcpld->chip[chip_index];
-
- /* Setup irq handlers */
- irq_end = chip->irq_start + chip->nirqs;
- for (irq = chip->irq_start; irq < irq_end; irq++) {
- irq_set_chip_and_handler(irq, &htcpld_muxed_chip,
- handle_simple_irq);
- irq_set_chip_data(irq, chip);
- irq_clear_status_flags(irq, IRQ_NOREQUEST | IRQ_NOPROBE);
- }
-
- return 0;
-}
-
-static int htcpld_register_chip_i2c(
- struct platform_device *pdev,
- int chip_index)
-{
- struct htcpld_data *htcpld;
- struct device *dev = &pdev->dev;
- struct htcpld_core_platform_data *pdata;
- struct htcpld_chip *chip;
- struct htcpld_chip_platform_data *plat_chip_data;
- struct i2c_adapter *adapter;
- struct i2c_client *client;
- struct i2c_board_info info;
-
- /* Get the platform and driver data */
- pdata = dev_get_platdata(dev);
- htcpld = platform_get_drvdata(pdev);
- chip = &htcpld->chip[chip_index];
- plat_chip_data = &pdata->chip[chip_index];
-
- adapter = i2c_get_adapter(pdata->i2c_adapter_id);
- if (!adapter) {
- /* Eek, no such I2C adapter! Bail out. */
- dev_warn(dev, "Chip at i2c address 0x%x: Invalid i2c adapter %d\n",
- plat_chip_data->addr, pdata->i2c_adapter_id);
- return -ENODEV;
- }
-
- if (!i2c_check_functionality(adapter, I2C_FUNC_SMBUS_READ_BYTE_DATA)) {
- dev_warn(dev, "i2c adapter %d non-functional\n",
- pdata->i2c_adapter_id);
- i2c_put_adapter(adapter);
- return -EINVAL;
- }
-
- memset(&info, 0, sizeof(struct i2c_board_info));
- info.addr = plat_chip_data->addr;
- strscpy(info.type, "htcpld-chip", I2C_NAME_SIZE);
- info.platform_data = chip;
-
- /* Add the I2C device. This calls the probe() function. */
- client = i2c_new_client_device(adapter, &info);
- if (IS_ERR(client)) {
- /* I2C device registration failed, contineu with the next */
- dev_warn(dev, "Unable to add I2C device for 0x%x\n",
- plat_chip_data->addr);
- i2c_put_adapter(adapter);
- return PTR_ERR(client);
- }
-
- i2c_set_clientdata(client, chip);
- snprintf(client->name, I2C_NAME_SIZE, "Chip_0x%x", client->addr);
- chip->client = client;
-
- /* Reset the chip */
- htcpld_chip_reset(client);
- chip->cache_in = i2c_smbus_read_byte_data(client, chip->cache_out);
-
- return 0;
-}
-
-static void htcpld_unregister_chip_i2c(
- struct platform_device *pdev,
- int chip_index)
-{
- struct htcpld_data *htcpld;
- struct htcpld_chip *chip;
-
- /* Get the platform and driver data */
- htcpld = platform_get_drvdata(pdev);
- chip = &htcpld->chip[chip_index];
-
- i2c_unregister_device(chip->client);
-}
-
-static int htcpld_register_chip_gpio(
- struct platform_device *pdev,
- int chip_index)
-{
- struct htcpld_data *htcpld;
- struct device *dev = &pdev->dev;
- struct htcpld_core_platform_data *pdata;
- struct htcpld_chip *chip;
- struct htcpld_chip_platform_data *plat_chip_data;
- struct gpio_chip *gpio_chip;
- int ret = 0;
-
- /* Get the platform and driver data */
- pdata = dev_get_platdata(dev);
- htcpld = platform_get_drvdata(pdev);
- chip = &htcpld->chip[chip_index];
- plat_chip_data = &pdata->chip[chip_index];
-
- /* Setup the GPIO chips */
- gpio_chip = &(chip->chip_out);
- gpio_chip->label = "htcpld-out";
- gpio_chip->parent = dev;
- gpio_chip->owner = THIS_MODULE;
- gpio_chip->get = htcpld_chip_get;
- gpio_chip->set = htcpld_chip_set;
- gpio_chip->direction_input = NULL;
- gpio_chip->direction_output = htcpld_direction_output;
- gpio_chip->base = plat_chip_data->gpio_out_base;
- gpio_chip->ngpio = plat_chip_data->num_gpios;
-
- gpio_chip = &(chip->chip_in);
- gpio_chip->label = "htcpld-in";
- gpio_chip->parent = dev;
- gpio_chip->owner = THIS_MODULE;
- gpio_chip->get = htcpld_chip_get;
- gpio_chip->set = NULL;
- gpio_chip->direction_input = htcpld_direction_input;
- gpio_chip->direction_output = NULL;
- gpio_chip->to_irq = htcpld_chip_to_irq;
- gpio_chip->base = plat_chip_data->gpio_in_base;
- gpio_chip->ngpio = plat_chip_data->num_gpios;
-
- /* Add the GPIO chips */
- ret = gpiochip_add_data(&(chip->chip_out), chip);
- if (ret) {
- dev_warn(dev, "Unable to register output GPIOs for 0x%x: %d\n",
- plat_chip_data->addr, ret);
- return ret;
- }
-
- ret = gpiochip_add_data(&(chip->chip_in), chip);
- if (ret) {
- dev_warn(dev, "Unable to register input GPIOs for 0x%x: %d\n",
- plat_chip_data->addr, ret);
- gpiochip_remove(&(chip->chip_out));
- return ret;
- }
-
- return 0;
-}
-
-static int htcpld_setup_chips(struct platform_device *pdev)
-{
- struct htcpld_data *htcpld;
- struct device *dev = &pdev->dev;
- struct htcpld_core_platform_data *pdata;
- int i;
-
- /* Get the platform and driver data */
- pdata = dev_get_platdata(dev);
- htcpld = platform_get_drvdata(pdev);
-
- /* Setup each chip's output GPIOs */
- htcpld->nchips = pdata->num_chip;
- htcpld->chip = devm_kcalloc(dev,
- htcpld->nchips,
- sizeof(struct htcpld_chip),
- GFP_KERNEL);
- if (!htcpld->chip)
- return -ENOMEM;
-
- /* Add the chips as best we can */
- for (i = 0; i < htcpld->nchips; i++) {
- int ret;
-
- /* Setup the HTCPLD chips */
- htcpld->chip[i].reset = pdata->chip[i].reset;
- htcpld->chip[i].cache_out = pdata->chip[i].reset;
- htcpld->chip[i].cache_in = 0;
- htcpld->chip[i].dev = dev;
- htcpld->chip[i].irq_start = pdata->chip[i].irq_base;
- htcpld->chip[i].nirqs = pdata->chip[i].num_irqs;
-
- INIT_WORK(&(htcpld->chip[i].set_val_work), &htcpld_chip_set_ni);
- spin_lock_init(&(htcpld->chip[i].lock));
-
- /* Setup the interrupts for the chip */
- if (htcpld->chained_irq) {
- ret = htcpld_setup_chip_irq(pdev, i);
- if (ret)
- continue;
- }
-
- /* Register the chip with I2C */
- ret = htcpld_register_chip_i2c(pdev, i);
- if (ret)
- continue;
-
-
- /* Register the chips with the GPIO subsystem */
- ret = htcpld_register_chip_gpio(pdev, i);
- if (ret) {
- /* Unregister the chip from i2c and continue */
- htcpld_unregister_chip_i2c(pdev, i);
- continue;
- }
-
- dev_info(dev, "Registered chip at 0x%x\n", pdata->chip[i].addr);
- }
-
- return 0;
-}
-
-static int htcpld_core_probe(struct platform_device *pdev)
-{
- struct htcpld_data *htcpld;
- struct device *dev = &pdev->dev;
- struct htcpld_core_platform_data *pdata;
- struct resource *res;
- int ret = 0;
-
- if (!dev)
- return -ENODEV;
-
- pdata = dev_get_platdata(dev);
- if (!pdata) {
- dev_warn(dev, "Platform data not found for htcpld core!\n");
- return -ENXIO;
- }
-
- htcpld = devm_kzalloc(dev, sizeof(struct htcpld_data), GFP_KERNEL);
- if (!htcpld)
- return -ENOMEM;
-
- /* Find chained irq */
- res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
- if (res) {
- int flags;
- htcpld->chained_irq = res->start;
-
- /* Setup the chained interrupt handler */
- flags = IRQF_TRIGGER_FALLING | IRQF_TRIGGER_RISING |
- IRQF_ONESHOT;
- ret = request_threaded_irq(htcpld->chained_irq,
- NULL, htcpld_handler,
- flags, pdev->name, htcpld);
- if (ret) {
- dev_warn(dev, "Unable to setup chained irq handler: %d\n", ret);
- return ret;
- } else
- device_init_wakeup(dev, 0);
- }
-
- /* Set the driver data */
- platform_set_drvdata(pdev, htcpld);
-
- /* Setup the htcpld chips */
- ret = htcpld_setup_chips(pdev);
- if (ret)
- return ret;
-
- /* Request the GPIO(s) for the int reset and set them up */
- htcpld->int_reset_gpio_hi = gpiochip_request_own_desc(&htcpld->chip[2].chip_out,
- 7, "htcpld-core", GPIO_ACTIVE_HIGH,
- GPIOD_OUT_HIGH);
- if (IS_ERR(htcpld->int_reset_gpio_hi)) {
- /*
- * If it failed, that sucks, but we can probably
- * continue on without it.
- */
- htcpld->int_reset_gpio_hi = NULL;
- dev_warn(dev, "Unable to request int_reset_gpio_hi -- interrupts may not work\n");
- }
-
- htcpld->int_reset_gpio_lo = gpiochip_request_own_desc(&htcpld->chip[2].chip_out,
- 0, "htcpld-core", GPIO_ACTIVE_HIGH,
- GPIOD_OUT_LOW);
- if (IS_ERR(htcpld->int_reset_gpio_lo)) {
- /*
- * If it failed, that sucks, but we can probably
- * continue on without it.
- */
- htcpld->int_reset_gpio_lo = NULL;
- dev_warn(dev, "Unable to request int_reset_gpio_lo -- interrupts may not work\n");
- }
-
- dev_info(dev, "Initialized successfully\n");
- return 0;
-}
-
-/* The I2C Driver -- used internally */
-static const struct i2c_device_id htcpld_chip_id[] = {
- { "htcpld-chip", 0 },
- { }
-};
-
-static struct i2c_driver htcpld_chip_driver = {
- .driver = {
- .name = "htcpld-chip",
- },
- .id_table = htcpld_chip_id,
-};
-
-/* The Core Driver */
-static struct platform_driver htcpld_core_driver = {
- .driver = {
- .name = "i2c-htcpld",
- },
-};
-
-static int __init htcpld_core_init(void)
-{
- int ret;
-
- /* Register the I2C Chip driver */
- ret = i2c_add_driver(&htcpld_chip_driver);
- if (ret)
- return ret;
-
- /* Probe for our chips */
- return platform_driver_probe(&htcpld_core_driver, htcpld_core_probe);
-}
-device_initcall(htcpld_core_init);
{ .name = "khadas-mcu-user-mem", },
};
-static int khadas_mcu_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int khadas_mcu_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
struct khadas_mcu *ddata;
.name = "khadas-mcu-core",
.of_match_table = of_match_ptr(khadas_mcu_of_match),
},
- .probe = khadas_mcu_probe,
+ .probe_new = khadas_mcu_probe,
};
module_i2c_driver(khadas_mcu_driver);
.precious_reg = lm3533_precious_register,
};
-static int lm3533_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int lm3533_i2c_probe(struct i2c_client *i2c)
{
struct lm3533 *lm3533;
.name = "lm3533",
},
.id_table = lm3533_i2c_ids,
- .probe = lm3533_i2c_probe,
+ .probe_new = lm3533_i2c_probe,
.remove = lm3533_i2c_remove,
};
.max_register = LP3943_MAX_REGISTERS,
};
-static int lp3943_probe(struct i2c_client *cl, const struct i2c_device_id *id)
+static int lp3943_probe(struct i2c_client *cl)
{
struct lp3943 *lp3943;
struct device *dev = &cl->dev;
#endif
static struct i2c_driver lp3943_driver = {
- .probe = lp3943_probe,
+ .probe_new = lp3943_probe,
.driver = {
.name = "lp3943",
.of_match_table = of_match_ptr(lp3943_of_match),
{ .name = "lp873x-gpio", },
};
-static int lp873x_probe(struct i2c_client *client,
- const struct i2c_device_id *ids)
+static int lp873x_probe(struct i2c_client *client)
{
struct lp873x *lp873;
int ret;
.name = "lp873x",
.of_match_table = of_lp873x_match_table,
},
- .probe = lp873x_probe,
+ .probe_new = lp873x_probe,
.id_table = lp873x_id_table,
};
module_i2c_driver(lp873x_driver);
};
MODULE_DEVICE_TABLE(of, of_lp87565_match_table);
-static int lp87565_probe(struct i2c_client *client,
- const struct i2c_device_id *ids)
+static int lp87565_probe(struct i2c_client *client)
{
struct lp87565 *lp87565;
const struct of_device_id *of_id;
.name = "lp87565",
.of_match_table = of_lp87565_match_table,
},
- .probe = lp87565_probe,
+ .probe_new = lp87565_probe,
.shutdown = lp87565_shutdown,
.id_table = lp87565_id_table,
};
.max_register = MAX_LP8788_REGISTERS,
};
-static int lp8788_probe(struct i2c_client *cl, const struct i2c_device_id *id)
+static int lp8788_probe(struct i2c_client *cl)
{
struct lp8788 *lp;
struct lp8788_platform_data *pdata = dev_get_platdata(&cl->dev);
.driver = {
.name = "lp8788",
},
- .probe = lp8788_probe,
+ .probe_new = lp8788_probe,
.remove = lp8788_remove,
.id_table = lp8788_ids,
};
#include <linux/device.h>
#include <linux/delay.h>
#include <linux/err.h>
-#include <linux/gpio.h>
+#include <linux/gpio/consumer.h>
#include <linux/mfd/core.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/notifier.h>
#include <linux/of.h>
-#include <linux/of_gpio.h>
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
#include <linux/regmap.h>
#include "madera.h"
-static int madera_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int madera_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct madera *madera;
const struct regmap_config *regmap_16bit_config = NULL;
const struct regmap_config *regmap_32bit_config = NULL;
.pm = &madera_pm_ops,
.of_match_table = of_match_ptr(madera_of_match),
},
- .probe = madera_i2c_probe,
+ .probe_new = madera_i2c_probe,
.remove = madera_i2c_remove,
.id_table = madera_i2c_id,
};
static const struct regmap_irq_chip max14577_irq_chip = {
.name = "max14577",
.status_base = MAX14577_REG_INT1,
- .mask_base = MAX14577_REG_INTMASK1,
- .mask_invert = true,
+ .unmask_base = MAX14577_REG_INTMASK1,
.num_regs = 3,
.irqs = max14577_irqs,
.num_irqs = ARRAY_SIZE(max14577_irqs),
static const struct regmap_irq_chip max77836_muic_irq_chip = {
.name = "max77836-muic",
.status_base = MAX14577_REG_INT1,
- .mask_base = MAX14577_REG_INTMASK1,
- .mask_invert = true,
+ .unmask_base = MAX14577_REG_INTMASK1,
.num_regs = 3,
.irqs = max77836_muic_irqs,
.num_irqs = ARRAY_SIZE(max77836_muic_irqs),
.name = "max77836-pmic",
.status_base = MAX77836_PMIC_REG_TOPSYS_INT,
.mask_base = MAX77836_PMIC_REG_TOPSYS_INT_MASK,
- .mask_invert = false,
.num_regs = 1,
.irqs = max77836_pmic_irqs,
.num_irqs = ARRAY_SIZE(max77836_pmic_irqs),
i2c_unregister_device(max14577->i2c_pmic);
}
-static int max14577_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int max14577_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct max14577 *max14577;
struct max14577_platform_data *pdata = dev_get_platdata(&i2c->dev);
struct device_node *np = i2c->dev.of_node;
};
MODULE_DEVICE_TABLE(i2c, max14577_i2c_id);
-#ifdef CONFIG_PM_SLEEP
static int max14577_suspend(struct device *dev)
{
struct i2c_client *i2c = to_i2c_client(dev);
return 0;
}
-#endif /* CONFIG_PM_SLEEP */
-static SIMPLE_DEV_PM_OPS(max14577_pm, max14577_suspend, max14577_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(max14577_pm, max14577_suspend, max14577_resume);
static struct i2c_driver max14577_i2c_driver = {
.driver = {
.name = "max14577",
- .pm = &max14577_pm,
+ .pm = pm_sleep_ptr(&max14577_pm),
.of_match_table = max14577_dt_match,
},
- .probe = max14577_i2c_probe,
+ .probe_new = max14577_i2c_probe,
.remove = max14577_i2c_remove,
.id_table = max14577_i2c_id,
};
MAX77620_ONOFFCNFG1_SFT_RST);
}
-static int max77620_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max77620_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
const struct regmap_config *rmap_config;
struct max77620_chip *chip;
const struct mfd_cell *mfd_cells;
return 0;
}
-#ifdef CONFIG_PM_SLEEP
static int max77620_set_fps_period(struct max77620_chip *chip,
int fps_id, int time_period)
{
return 0;
}
-#endif
static const struct i2c_device_id max77620_id[] = {
{"max77620", MAX77620},
{},
};
-static const struct dev_pm_ops max77620_pm_ops = {
- SET_SYSTEM_SLEEP_PM_OPS(max77620_i2c_suspend, max77620_i2c_resume)
-};
+static DEFINE_SIMPLE_DEV_PM_OPS(max77620_pm_ops,
+ max77620_i2c_suspend, max77620_i2c_resume);
static struct i2c_driver max77620_driver = {
.driver = {
.name = "max77620",
- .pm = &max77620_pm_ops,
+ .pm = pm_sleep_ptr(&max77620_pm_ops),
},
- .probe = max77620_probe,
+ .probe_new = max77620_probe,
.id_table = max77620_id,
};
builtin_i2c_driver(max77620_driver);
.status_base = MAX77650_REG_INT_GLBL,
.mask_base = MAX77650_REG_INTM_GLBL,
.type_in_mask = true,
- .type_invert = true,
.init_ack_masked = true,
.clear_on_unmask = true,
};
return 0;
}
-#ifdef CONFIG_PM_SLEEP
static int max77686_suspend(struct device *dev)
{
struct i2c_client *i2c = to_i2c_client(dev);
return 0;
}
-#endif /* CONFIG_PM_SLEEP */
-static SIMPLE_DEV_PM_OPS(max77686_pm, max77686_suspend, max77686_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(max77686_pm, max77686_suspend, max77686_resume);
static struct i2c_driver max77686_i2c_driver = {
.driver = {
.name = "max77686",
- .pm = &max77686_pm,
+ .pm = pm_sleep_ptr(&max77686_pm),
.of_match_table = max77686_pmic_dt_match,
},
.probe_new = max77686_i2c_probe,
.name = "max77693-led",
.status_base = MAX77693_LED_REG_FLASH_INT,
.mask_base = MAX77693_LED_REG_FLASH_INT_MASK,
- .mask_invert = false,
.num_regs = 1,
.irqs = max77693_led_irqs,
.num_irqs = ARRAY_SIZE(max77693_led_irqs),
.name = "max77693-topsys",
.status_base = MAX77693_PMIC_REG_TOPSYS_INT,
.mask_base = MAX77693_PMIC_REG_TOPSYS_INT_MASK,
- .mask_invert = false,
.num_regs = 1,
.irqs = max77693_topsys_irqs,
.num_irqs = ARRAY_SIZE(max77693_topsys_irqs),
.name = "max77693-charger",
.status_base = MAX77693_CHG_REG_CHG_INT,
.mask_base = MAX77693_CHG_REG_CHG_INT_MASK,
- .mask_invert = false,
.num_regs = 1,
.irqs = max77693_charger_irqs,
.num_irqs = ARRAY_SIZE(max77693_charger_irqs),
static const struct regmap_irq_chip max77693_muic_irq_chip = {
.name = "max77693-muic",
.status_base = MAX77693_MUIC_REG_INT1,
- .mask_base = MAX77693_MUIC_REG_INTMASK1,
- .mask_invert = true,
+ .unmask_base = MAX77693_MUIC_REG_INTMASK1,
.num_regs = 3,
.irqs = max77693_muic_irqs,
.num_irqs = ARRAY_SIZE(max77693_muic_irqs),
.max_register = MAX77693_HAPTIC_REG_END,
};
-static int max77693_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int max77693_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct max77693_dev *max77693;
unsigned int reg_data;
int ret = 0;
.pm = &max77693_pm,
.of_match_table = of_match_ptr(max77693_dt_match),
},
- .probe = max77693_i2c_probe,
+ .probe_new = max77693_i2c_probe,
.remove = max77693_i2c_remove,
.id_table = max77693_i2c_id,
};
.name = "max77843",
.status_base = MAX77843_SYS_REG_SYSINTSRC,
.mask_base = MAX77843_SYS_REG_SYSINTMASK,
- .mask_invert = false,
.num_regs = 1,
.irqs = max77843_irqs,
.num_irqs = ARRAY_SIZE(max77843_irqs),
return ret;
}
-static int max77843_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int max77843_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct max77693_dev *max77843;
unsigned int reg_data;
int ret;
.of_match_table = max77843_dt_match,
.suppress_bind_attrs = true,
},
- .probe = max77843_probe,
+ .probe_new = max77843_probe,
.id_table = max77843_id,
};
MAX8907_MASK_POWER_OFF, MAX8907_MASK_POWER_OFF);
}
-static int max8907_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int max8907_i2c_probe(struct i2c_client *i2c)
{
struct max8907 *max8907;
int ret;
.name = "max8907",
.of_match_table = of_match_ptr(max8907_of_match),
},
- .probe = max8907_i2c_probe,
+ .probe_new = max8907_i2c_probe,
.remove = max8907_i2c_remove,
.id_table = max8907_i2c_id,
};
return 0;
}
-static int max8925_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max8925_probe(struct i2c_client *client)
{
struct max8925_platform_data *pdata = dev_get_platdata(&client->dev);
struct max8925_chip *chip;
i2c_unregister_device(chip->rtc);
}
-#ifdef CONFIG_PM_SLEEP
static int max8925_suspend(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
disable_irq_wake(chip->core_irq);
return 0;
}
-#endif
-static SIMPLE_DEV_PM_OPS(max8925_pm_ops, max8925_suspend, max8925_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(max8925_pm_ops,
+ max8925_suspend, max8925_resume);
static const struct of_device_id max8925_dt_ids[] = {
{ .compatible = "maxim,max8925", },
static struct i2c_driver max8925_driver = {
.driver = {
.name = "max8925",
- .pm = &max8925_pm_ops,
+ .pm = pm_sleep_ptr(&max8925_pm_ops),
.of_match_table = max8925_dt_ids,
},
- .probe = max8925_probe,
+ .probe_new = max8925_probe,
.remove = max8925_remove,
.id_table = max8925_id_table,
};
return id->driver_data;
}
-static int max8997_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int max8997_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct max8997_dev *max8997;
struct max8997_platform_data *pdata = dev_get_platdata(&i2c->dev);
int ret = 0;
.suppress_bind_attrs = true,
.of_match_table = of_match_ptr(max8997_pmic_dt_match),
},
- .probe = max8997_i2c_probe,
+ .probe_new = max8997_i2c_probe,
.id_table = max8997_i2c_id,
};
return id->driver_data;
}
-static int max8998_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int max8998_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct max8998_platform_data *pdata = dev_get_platdata(&i2c->dev);
struct max8998_dev *max8998;
int ret = 0;
.suppress_bind_attrs = true,
.of_match_table = of_match_ptr(max8998_dt_match),
},
- .probe = max8998_i2c_probe,
+ .probe_new = max8998_i2c_probe,
.id_table = max8998_i2c_id,
};
#include <linux/mfd/mc13xxx.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/of_gpio.h>
#include <linux/i2c.h>
#include <linux/err.h>
.cache_type = REGCACHE_NONE,
};
-static int mc13xxx_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int mc13xxx_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct mc13xxx *mc13xxx;
int ret;
.name = "mc13xxx",
.of_match_table = mc13xxx_dt_ids,
},
- .probe = mc13xxx_i2c_probe,
+ .probe_new = mc13xxx_i2c_probe,
.remove = mc13xxx_i2c_remove,
};
#include <linux/mfd/mc13xxx.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/of_gpio.h>
#include <linux/err.h>
#include <linux/spi/spi.h>
* result, the SS will negate before all of the data has been
* transferred to/from the peripheral."
* We workaround this by accessing the SPI controller with a
- * single transfert.
+ * single transfer.
*/
static struct regmap_bus regmap_mc13xxx_bus = {
return 0;
}
-#ifdef CONFIG_PM_SLEEP
static int mcp_sa11x0_suspend(struct device *dev)
{
struct mcp_sa11x0 *m = priv(dev_get_drvdata(dev));
return 0;
}
-#endif
static const struct dev_pm_ops mcp_sa11x0_pm_ops = {
-#ifdef CONFIG_PM_SLEEP
.suspend = mcp_sa11x0_suspend,
.freeze = mcp_sa11x0_suspend,
.poweroff = mcp_sa11x0_suspend,
.resume_noirq = mcp_sa11x0_resume,
.thaw_noirq = mcp_sa11x0_resume,
.restore_noirq = mcp_sa11x0_resume,
-#endif
};
static struct platform_driver mcp_sa11x0_driver = {
.remove = mcp_sa11x0_remove,
.driver = {
.name = DRIVER_NAME,
- .pm = &mcp_sa11x0_pm_ops,
+ .pm = pm_sleep_ptr(&mcp_sa11x0_pm_ops),
},
};
static struct i2c_driver menelaus_i2c_driver;
-static int menelaus_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int menelaus_probe(struct i2c_client *client)
{
struct menelaus_chip *menelaus;
int rev = 0;
.driver = {
.name = DRIVER_NAME,
},
- .probe = menelaus_probe,
+ .probe_new = menelaus_probe,
.remove = menelaus_remove,
.id_table = menelaus_id,
};
}
static int
-menf21bmc_probe(struct i2c_client *client, const struct i2c_device_id *ids)
+menf21bmc_probe(struct i2c_client *client)
{
int rev_major, rev_minor, rev_main;
int ret;
static struct i2c_driver menf21bmc_driver = {
.driver.name = "menf21bmc",
.id_table = menf21bmc_id_table,
- .probe = menf21bmc_probe,
+ .probe_new = menf21bmc_probe,
};
module_i2c_driver(menf21bmc_driver);
.val_format_endian = REGMAP_ENDIAN_LITTLE,
};
-#ifdef CONFIG_PM_SLEEP
static int cpcap_suspend(struct device *dev)
{
struct spi_device *spi = to_spi_device(dev);
return 0;
}
-#endif
-static SIMPLE_DEV_PM_OPS(cpcap_pm, cpcap_suspend, cpcap_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(cpcap_pm, cpcap_suspend, cpcap_resume);
static const struct mfd_cell cpcap_mfd_devices[] = {
{
struct cpcap_ddata *cpcap;
int ret;
- match = of_match_device(of_match_ptr(cpcap_of_match), &spi->dev);
+ match = of_match_device(cpcap_of_match, &spi->dev);
if (!match)
return -ENODEV;
.driver = {
.name = "cpcap-core",
.of_match_table = cpcap_of_match,
- .pm = &cpcap_pm,
+ .pm = pm_sleep_ptr(&cpcap_pm),
},
.probe = cpcap_probe,
.id_table = cpcap_spi_ids,
struct mt6360_ddata *ddata = context;
u8 bank = *(u8 *)reg;
u8 reg_addr = *(u8 *)(reg + 1);
- struct i2c_client *i2c = ddata->i2c[bank];
+ struct i2c_client *i2c;
bool crc_needed = false;
u8 *buf;
int buf_len = MT6360_ALLOC_READ_SIZE(val_size);
u8 crc;
int ret;
+ if (bank >= MT6360_SLAVE_MAX)
+ return -EINVAL;
+
+ i2c = ddata->i2c[bank];
+
if (bank == MT6360_SLAVE_PMIC || bank == MT6360_SLAVE_LDO) {
crc_needed = true;
ret = mt6360_xlate_pmicldo_addr(®_addr, val_size);
struct mt6360_ddata *ddata = context;
u8 bank = *(u8 *)val;
u8 reg_addr = *(u8 *)(val + 1);
- struct i2c_client *i2c = ddata->i2c[bank];
+ struct i2c_client *i2c;
bool crc_needed = false;
u8 *buf;
int buf_len = MT6360_ALLOC_WRITE_SIZE(val_size);
int write_size = val_size - MT6360_REGMAP_REG_BYTE_SIZE;
int ret;
+ if (bank >= MT6360_SLAVE_MAX)
+ return -EINVAL;
+
+ i2c = ddata->i2c[bank];
+
if (bank == MT6360_SLAVE_PMIC || bank == MT6360_SLAVE_LDO) {
crc_needed = true;
ret = mt6360_xlate_pmicldo_addr(®_addr, val_size - MT6360_REGMAP_REG_BYTE_SIZE);
mt6397->irq_masks_cur[reg] |= BIT(shift);
}
-#ifdef CONFIG_PM_SLEEP
static int mt6397_irq_set_wake(struct irq_data *irq_data, unsigned int on)
{
struct mt6397_chip *mt6397 = irq_data_get_irq_chip_data(irq_data);
return 0;
}
-#else
-#define mt6397_irq_set_wake NULL
-#endif
static struct irq_chip mt6397_irq_chip = {
.name = "mt6397-irq",
.irq_bus_sync_unlock = mt6397_irq_sync_unlock,
.irq_enable = mt6397_irq_enable,
.irq_disable = mt6397_irq_disable,
- .irq_set_wake = mt6397_irq_set_wake,
+ .irq_set_wake = pm_sleep_ptr(mt6397_irq_set_wake),
};
static void mt6397_irq_handle_reg(struct mt6397_chip *mt6397, int reg,
};
MODULE_DEVICE_TABLE(of, of_palmas_match_tbl);
-static int palmas_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int palmas_i2c_probe(struct i2c_client *i2c)
{
struct palmas *palmas;
struct palmas_platform_data *pdata;
int ret = 0, i;
unsigned int reg, addr;
int slave;
- const struct of_device_id *match;
pdata = dev_get_platdata(&i2c->dev);
palmas->dev = &i2c->dev;
palmas->irq = i2c->irq;
- match = of_match_device(of_palmas_match_tbl, &i2c->dev);
-
- if (!match)
- return -ENODATA;
-
- driver_data = (struct palmas_driver_data *)match->data;
+ driver_data = (struct palmas_driver_data *) device_get_match_data(&i2c->dev);
palmas->features = *driver_data->features;
for (i = 0; i < PALMAS_NUM_CLIENTS; i++) {
.name = "palmas",
.of_match_table = of_palmas_match_tbl,
},
- .probe = palmas_i2c_probe,
+ .probe_new = palmas_i2c_probe,
.remove = palmas_i2c_remove,
.id_table = palmas_i2c_id,
};
}
}
-#ifdef CONFIG_PM_SLEEP
-static int pcf50633_suspend(struct device *dev)
-{
- struct i2c_client *client = to_i2c_client(dev);
- struct pcf50633 *pcf = i2c_get_clientdata(client);
-
- return pcf50633_irq_suspend(pcf);
-}
-
-static int pcf50633_resume(struct device *dev)
-{
- struct i2c_client *client = to_i2c_client(dev);
- struct pcf50633 *pcf = i2c_get_clientdata(client);
-
- return pcf50633_irq_resume(pcf);
-}
-#endif
-
-static SIMPLE_DEV_PM_OPS(pcf50633_pm, pcf50633_suspend, pcf50633_resume);
-
static const struct regmap_config pcf50633_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
};
-static int pcf50633_probe(struct i2c_client *client,
- const struct i2c_device_id *ids)
+static int pcf50633_probe(struct i2c_client *client)
{
struct pcf50633 *pcf;
struct platform_device *pdev;
static struct i2c_driver pcf50633_driver = {
.driver = {
.name = "pcf50633",
- .pm = &pcf50633_pm,
+ .pm = pm_sleep_ptr(&pcf50633_pm),
},
.id_table = pcf50633_id_table,
- .probe = pcf50633_probe,
+ .probe_new = pcf50633_probe,
.remove = pcf50633_remove,
};
* All rights reserved.
*/
+#include <linux/i2c.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
#include <linux/mutex.h>
return IRQ_HANDLED;
}
-#ifdef CONFIG_PM
-
-int pcf50633_irq_suspend(struct pcf50633 *pcf)
+static int pcf50633_suspend(struct device *dev)
{
+ struct i2c_client *client = to_i2c_client(dev);
+ struct pcf50633 *pcf = i2c_get_clientdata(client);
int ret;
int i;
u8 res[5];
return ret;
}
-int pcf50633_irq_resume(struct pcf50633 *pcf)
+static int pcf50633_resume(struct device *dev)
{
+ struct i2c_client *client = to_i2c_client(dev);
+ struct pcf50633 *pcf = i2c_get_clientdata(client);
int ret;
/* Write the saved mask registers */
return ret;
}
-#endif
+EXPORT_GPL_SIMPLE_DEV_PM_OPS(pcf50633_pm, pcf50633_suspend, pcf50633_resume);
int pcf50633_irq_init(struct pcf50633 *pcf, int irq)
{
dev = &client->dev;
regmap = devm_regmap_init_i2c(client, &qcom_mfd_regmap_cfg);
- if (!regmap)
- return -ENODEV;
+ if (IS_ERR(regmap))
+ return PTR_ERR(regmap);
i2c_set_clientdata(client, regmap);
};
static const struct of_device_id pm8xxx_id_table[] = {
- { .compatible = "qcom,pm8018", .data = &pm8xxx_data},
{ .compatible = "qcom,pm8058", .data = &pm8xxx_data},
{ .compatible = "qcom,pm8821", .data = &pm8821_data},
{ .compatible = "qcom,pm8921", .data = &pm8xxx_data},
init_completion(&rpm->ack);
/* Enable message RAM clock */
- rpm->ramclk = devm_clk_get(&pdev->dev, "ram");
+ rpm->ramclk = devm_clk_get_enabled(&pdev->dev, "ram");
if (IS_ERR(rpm->ramclk)) {
ret = PTR_ERR(rpm->ramclk);
if (ret == -EPROBE_DEFER)
*/
rpm->ramclk = NULL;
}
- clk_prepare_enable(rpm->ramclk); /* Accepts NULL */
irq_ack = platform_get_irq_byname(pdev, "ack");
if (irq_ack < 0)
if (ret)
dev_warn(&pdev->dev, "failed to mark wakeup irq as wakeup\n");
- return of_platform_populate(pdev->dev.of_node, NULL, NULL, &pdev->dev);
-}
-
-static int qcom_rpm_remove(struct platform_device *pdev)
-{
- struct qcom_rpm *rpm = dev_get_drvdata(&pdev->dev);
-
- of_platform_depopulate(&pdev->dev);
- clk_disable_unprepare(rpm->ramclk);
-
- return 0;
+ return devm_of_platform_populate(&pdev->dev);
}
static struct platform_driver qcom_rpm_driver = {
.probe = qcom_rpm_probe,
- .remove = qcom_rpm_remove,
.driver = {
.name = "qcom_rpm",
.of_match_table = qcom_rpm_of_match,
mutex_unlock(&rc5t583->irq_lock);
}
-#ifdef CONFIG_PM_SLEEP
+
static int rc5t583_irq_set_wake(struct irq_data *irq_data, unsigned int on)
{
struct rc5t583 *rc5t583 = irq_data_get_irq_chip_data(irq_data);
return irq_set_irq_wake(rc5t583->chip_irq, on);
}
-#else
-#define rc5t583_irq_set_wake NULL
-#endif
static irqreturn_t rc5t583_irq(int irq, void *data)
{
.irq_bus_lock = rc5t583_irq_lock,
.irq_bus_sync_unlock = rc5t583_irq_sync_unlock,
.irq_set_type = rc5t583_irq_set_type,
- .irq_set_wake = rc5t583_irq_set_wake,
+ .irq_set_wake = pm_sleep_ptr(rc5t583_irq_set_wake),
};
int rc5t583_irq_init(struct rc5t583 *rc5t583, int irq, int irq_base)
.cache_type = REGCACHE_RBTREE,
};
-static int rc5t583_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int rc5t583_i2c_probe(struct i2c_client *i2c)
{
struct rc5t583 *rc5t583;
struct rc5t583_platform_data *pdata = dev_get_platdata(&i2c->dev);
.driver = {
.name = "rc5t583",
},
- .probe = rc5t583_i2c_probe,
+ .probe_new = rc5t583_i2c_probe,
.id_table = rc5t583_i2c_id,
};
.val_bits = 16,
};
-static int retu_probe(struct i2c_client *i2c, const struct i2c_device_id *id)
+static int retu_probe(struct i2c_client *i2c)
{
struct retu_data const *rdat;
struct retu_dev *rdev;
.name = "retu-mfd",
.of_match_table = retu_of_match,
},
- .probe = retu_probe,
+ .probe_new = retu_probe,
.remove = retu_remove,
.id_table = retu_id,
};
};
static const struct mfd_cell rk805s[] = {
- { .name = "rk808-clkout", },
- { .name = "rk808-regulator", },
- { .name = "rk805-pinctrl", },
+ { .name = "rk808-clkout", .id = PLATFORM_DEVID_NONE, },
+ { .name = "rk808-regulator", .id = PLATFORM_DEVID_NONE, },
+ { .name = "rk805-pinctrl", .id = PLATFORM_DEVID_NONE, },
{
.name = "rk808-rtc",
.num_resources = ARRAY_SIZE(rtc_resources),
.resources = &rtc_resources[0],
+ .id = PLATFORM_DEVID_NONE,
},
{ .name = "rk805-pwrkey",
.num_resources = ARRAY_SIZE(rk805_key_resources),
.resources = &rk805_key_resources[0],
+ .id = PLATFORM_DEVID_NONE,
},
};
static const struct mfd_cell rk808s[] = {
- { .name = "rk808-clkout", },
- { .name = "rk808-regulator", },
+ { .name = "rk808-clkout", .id = PLATFORM_DEVID_NONE, },
+ { .name = "rk808-regulator", .id = PLATFORM_DEVID_NONE, },
{
.name = "rk808-rtc",
.num_resources = ARRAY_SIZE(rtc_resources),
.resources = rtc_resources,
+ .id = PLATFORM_DEVID_NONE,
},
};
static const struct mfd_cell rk817s[] = {
- { .name = "rk808-clkout",},
- { .name = "rk808-regulator",},
+ { .name = "rk808-clkout", .id = PLATFORM_DEVID_NONE, },
+ { .name = "rk808-regulator", .id = PLATFORM_DEVID_NONE, },
{
.name = "rk805-pwrkey",
.num_resources = ARRAY_SIZE(rk817_pwrkey_resources),
.resources = &rk817_pwrkey_resources[0],
+ .id = PLATFORM_DEVID_NONE,
},
{
.name = "rk808-rtc",
.num_resources = ARRAY_SIZE(rk817_rtc_resources),
.resources = &rk817_rtc_resources[0],
+ .id = PLATFORM_DEVID_NONE,
},
- { .name = "rk817-codec",},
+ { .name = "rk817-codec", .id = PLATFORM_DEVID_NONE, },
{
.name = "rk817-charger",
.num_resources = ARRAY_SIZE(rk817_charger_resources),
.resources = &rk817_charger_resources[0],
+ .id = PLATFORM_DEVID_NONE,
},
};
static const struct mfd_cell rk818s[] = {
- { .name = "rk808-clkout", },
- { .name = "rk808-regulator", },
+ { .name = "rk808-regulator", .id = PLATFORM_DEVID_NONE, },
{
.name = "rk808-rtc",
.num_resources = ARRAY_SIZE(rtc_resources),
.resources = rtc_resources,
+ .id = PLATFORM_DEVID_NONE,
},
};
};
MODULE_DEVICE_TABLE(of, rk808_of_match);
-static int rk808_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int rk808_probe(struct i2c_client *client)
{
struct device_node *np = client->dev.of_node;
struct rk808 *rk808;
.of_match_table = rk808_of_match,
.pm = &rk8xx_pm_ops,
},
- .probe = rk808_probe,
+ .probe_new = rk808_probe,
.remove = rk808_remove,
.shutdown = rk8xx_shutdown,
};
.num_irqs = ARRAY_SIZE(rc5t619_irqs),
.num_regs = 1,
.status_base = RN5T618_INTMON,
- .mask_base = RN5T618_INTEN,
- .mask_invert = true,
+ .unmask_base = RN5T618_INTEN,
};
static struct i2c_client *rn5t618_pm_power_off;
.irqs = &bd71828_irqs[0],
.num_irqs = ARRAY_SIZE(bd71828_irqs),
.status_base = BD71828_REG_INT_BUCK,
- .mask_base = BD71828_REG_INT_MASK_BUCK,
+ .unmask_base = BD71828_REG_INT_MASK_BUCK,
.ack_base = BD71828_REG_INT_BUCK,
- .mask_invert = true,
.init_ack_masked = true,
.num_regs = 12,
.num_main_regs = 1,
.irqs = &bd71815_irqs[0],
.num_irqs = ARRAY_SIZE(bd71815_irqs),
.status_base = BD71815_REG_INT_STAT_01,
- .mask_base = BD71815_REG_INT_EN_01,
+ .unmask_base = BD71815_REG_INT_EN_01,
.ack_base = BD71815_REG_INT_STAT_01,
- .mask_invert = true,
.init_ack_masked = true,
.num_regs = 12,
.num_main_regs = 1,
}
regmap = devm_regmap_init_i2c(i2c, regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(&i2c->dev, "Failed to initialize Regmap\n");
- return PTR_ERR(regmap);
- }
+ if (IS_ERR(regmap))
+ return dev_err_probe(&i2c->dev, PTR_ERR(regmap),
+ "Failed to initialize Regmap\n");
ret = devm_regmap_add_irq_chip(&i2c->dev, regmap, i2c->irq,
IRQF_ONESHOT, 0, irqchip, &irq_data);
- if (ret) {
- dev_err(&i2c->dev, "Failed to add IRQ chip\n");
- return ret;
- }
+ if (ret)
+ return dev_err_probe(&i2c->dev, ret,
+ "Failed to add IRQ chip\n");
dev_dbg(&i2c->dev, "Registered %d IRQs for chip\n",
irqchip->num_irqs);
if (button_irq) {
ret = regmap_irq_get_virq(irq_data, button_irq);
- if (ret < 0) {
- dev_err(&i2c->dev, "Failed to get the power-key IRQ\n");
- return ret;
- }
+ if (ret < 0)
+ return dev_err_probe(&i2c->dev, ret,
+ "Failed to get the power-key IRQ\n");
button.irq = ret;
}
ret = devm_mfd_add_devices(&i2c->dev, PLATFORM_DEVID_AUTO, mfd, cells,
NULL, 0, regmap_irq_get_domain(irq_data));
if (ret)
- dev_err(&i2c->dev, "Failed to create subdevices\n");
+ dev_err_probe(&i2c->dev, ret, "Failed to create subdevices\n");
return ret;
}
.mask_base = BD718XX_REG_MIRQ,
.ack_base = BD718XX_REG_IRQ,
.init_ack_masked = true,
- .mask_invert = false,
};
static const struct regmap_range pmic_status_range = {
return 0;
}
-static int bd718xx_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int bd718xx_i2c_probe(struct i2c_client *i2c)
{
struct regmap *regmap;
struct regmap_irq_chip_data *irq_data;
}
regmap = devm_regmap_init_i2c(i2c, &bd718xx_regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(&i2c->dev, "regmap initialization failed\n");
- return PTR_ERR(regmap);
- }
+ if (IS_ERR(regmap))
+ return dev_err_probe(&i2c->dev, PTR_ERR(regmap),
+ "regmap initialization failed\n");
ret = devm_regmap_add_irq_chip(&i2c->dev, regmap, i2c->irq,
IRQF_ONESHOT, 0, &bd718xx_irq_chip,
&irq_data);
- if (ret) {
- dev_err(&i2c->dev, "Failed to add irq_chip\n");
- return ret;
- }
+ if (ret)
+ return dev_err_probe(&i2c->dev, ret, "Failed to add irq_chip\n");
ret = bd718xx_init_press_duration(regmap, &i2c->dev);
if (ret)
ret = regmap_irq_get_virq(irq_data, BD718XX_INT_PWRBTN_S);
- if (ret < 0) {
- dev_err(&i2c->dev, "Failed to get the IRQ\n");
- return ret;
- }
+ if (ret < 0)
+ return dev_err_probe(&i2c->dev, ret, "Failed to get the IRQ\n");
button.irq = ret;
mfd, cells, NULL, 0,
regmap_irq_get_domain(irq_data));
if (ret)
- dev_err(&i2c->dev, "Failed to create subdevices\n");
+ dev_err_probe(&i2c->dev, ret, "Failed to create subdevices\n");
return ret;
}
.name = "rohm-bd718x7",
.of_match_table = bd718xx_of_match,
},
- .probe = bd718xx_i2c_probe,
+ .probe_new = bd718xx_i2c_probe,
};
static int __init bd718xx_i2c_init(void)
.irq_reg_stride = 1,
};
-static int bd957x_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int bd957x_i2c_probe(struct i2c_client *i2c)
{
int ret;
struct regmap *regmap;
}
regmap = devm_regmap_init_i2c(i2c, &bd957x_regmap);
- if (IS_ERR(regmap)) {
- dev_err(&i2c->dev, "Failed to initialize Regmap\n");
- return PTR_ERR(regmap);
- }
+ if (IS_ERR(regmap))
+ return dev_err_probe(&i2c->dev, PTR_ERR(regmap),
+ "Failed to initialize Regmap\n");
/*
* BD9576 behaves badly. It kepts IRQ line asserted for the whole
ret = devm_regmap_add_irq_chip(&i2c->dev, regmap, i2c->irq,
IRQF_ONESHOT, 0,
&bd9576_irq_chip, &irq_data);
- if (ret) {
- dev_err(&i2c->dev, "Failed to add IRQ chip\n");
- return ret;
- }
+ if (ret)
+ return dev_err_probe(&i2c->dev, ret,
+ "Failed to add IRQ chip\n");
+
domain = regmap_irq_get_domain(irq_data);
} else {
ret = regmap_update_bits(regmap, BD957X_REG_INT_MAIN_MASK,
ret = devm_mfd_add_devices(&i2c->dev, PLATFORM_DEVID_AUTO, cells,
num_cells, NULL, 0, domain);
if (ret)
- dev_err(&i2c->dev, "Failed to create subdevices\n");
+ dev_err_probe(&i2c->dev, ret, "Failed to create subdevices\n");
return ret;
}
.name = "rohm-bd957x",
.of_match_table = bd957x_of_match,
},
- .probe = &bd957x_i2c_probe,
+ .probe_new = &bd957x_i2c_probe,
};
module_i2c_driver(bd957x_drv);
.can_multi_write = true,
};
-static int rsmu_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int rsmu_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
const struct regmap_config *cfg;
struct rsmu_ddata *rsmu;
int ret;
.name = "rsmu-i2c",
.of_match_table = of_match_ptr(rsmu_i2c_of_match),
},
- .probe = rsmu_i2c_probe,
+ .probe_new = rsmu_i2c_probe,
.remove = rsmu_i2c_remove,
.id_table = rsmu_i2c_id,
};
static const struct regmap_irq_chip rt5033_irq_chip = {
.name = "rt5033",
.status_base = RT5033_REG_PMIC_IRQ_STAT,
- .mask_base = RT5033_REG_PMIC_IRQ_CTRL,
- .mask_invert = true,
+ .unmask_base = RT5033_REG_PMIC_IRQ_CTRL,
.num_regs = 1,
.irqs = rt5033_irqs,
.num_irqs = ARRAY_SIZE(rt5033_irqs),
.max_register = RT5033_REG_END,
};
-static int rt5033_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int rt5033_i2c_probe(struct i2c_client *i2c)
{
struct rt5033_dev *rt5033;
unsigned int dev_id;
.name = "rt5033",
.of_match_table = rt5033_dt_match,
},
- .probe = rt5033_i2c_probe,
+ .probe_new = rt5033_i2c_probe,
.id_table = rt5033_i2c_id,
};
module_i2c_driver(rt5033_driver);
static const struct regmap_irq_chip rt5120_irq_chip = {
.name = "rt5120-pmic",
.status_base = RT5120_REG_INTSTAT,
- .mask_base = RT5120_REG_INTENABLE,
+ .unmask_base = RT5120_REG_INTENABLE,
.ack_base = RT5120_REG_INTSTAT,
- .mask_invert = true,
.use_ack = true,
.num_regs = 1,
.irqs = rt5120_irqs,
return pd;
}
-static int sec_pmic_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int sec_pmic_probe(struct i2c_client *i2c)
{
const struct regmap_config *regmap;
struct sec_platform_data *pdata;
regmap_update_bits(sec_pmic->regmap_pmic, reg, mask, 0);
}
-#ifdef CONFIG_PM_SLEEP
static int sec_pmic_suspend(struct device *dev)
{
struct i2c_client *i2c = to_i2c_client(dev);
return 0;
}
-#endif /* CONFIG_PM_SLEEP */
-static SIMPLE_DEV_PM_OPS(sec_pmic_pm_ops, sec_pmic_suspend, sec_pmic_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(sec_pmic_pm_ops,
+ sec_pmic_suspend, sec_pmic_resume);
static struct i2c_driver sec_pmic_driver = {
.driver = {
.name = "sec_pmic",
- .pm = &sec_pmic_pm_ops,
+ .pm = pm_sleep_ptr(&sec_pmic_pm_ops),
.of_match_table = sec_dt_match,
},
- .probe = sec_pmic_probe,
+ .probe_new = sec_pmic_probe,
.shutdown = sec_pmic_shutdown,
};
module_i2c_driver(sec_pmic_driver);
}
EXPORT_SYMBOL_GPL(si476x_core_is_powered_up);
-static int si476x_core_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int si476x_core_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
int rval;
struct si476x_core *core;
struct si476x_platform_data *pdata;
.driver = {
.name = "si476x-core",
},
- .probe = si476x_core_probe,
+ .probe_new = si476x_core_probe,
.remove = si476x_core_remove,
.id_table = si476x_id,
};
.val_bits = 8,
};
-static int sky81452_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int sky81452_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
const struct sky81452_platform_data *pdata = dev_get_platdata(dev);
.name = "sky81452",
.of_match_table = of_match_ptr(sky81452_of_match),
},
- .probe = sky81452_probe,
+ .probe_new = sky81452_probe,
.id_table = sky81452_ids,
};
}
-#ifdef CONFIG_PM
-
/* power management support */
static void sm501_set_power(struct sm501_devdata *sm, int on)
return 0;
}
-#else
-#define sm501_plat_suspend NULL
-#define sm501_plat_resume NULL
-#endif
/* Initialisation data for PCI devices */
},
.probe = sm501_plat_probe,
.remove = sm501_plat_remove,
- .suspend = sm501_plat_suspend,
- .resume = sm501_plat_resume,
+ .suspend = pm_sleep_ptr(sm501_plat_suspend),
+ .resume = pm_sleep_ptr(sm501_plat_resume),
};
static int __init sm501_base_init(void)
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Ampere Altra Family SMPro core driver
+ * Copyright (c) 2022, Ampere Computing LLC
+ */
+
+#include <linux/i2c.h>
+#include <linux/kernel.h>
+#include <linux/mfd/core.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/regmap.h>
+
+/* Identification Registers */
+#define MANUFACTURER_ID_REG 0x02
+#define AMPERE_MANUFACTURER_ID 0xCD3A
+
+#define CORE_CE_ERR_DATA 0x82
+#define CORE_UE_ERR_DATA 0x85
+#define MEM_CE_ERR_DATA 0x92
+#define MEM_UE_ERR_DATA 0x95
+#define PCIE_CE_ERR_DATA 0xC2
+#define PCIE_UE_ERR_DATA 0xC5
+#define OTHER_CE_ERR_DATA 0xD2
+#define OTHER_UE_ERR_DATA 0xDA
+
+static int smpro_core_write(void *context, const void *data, size_t count)
+{
+ struct device *dev = context;
+ struct i2c_client *i2c = to_i2c_client(dev);
+ int ret;
+
+ ret = i2c_master_send(i2c, data, count);
+ if (unlikely(ret != count))
+ return (ret < 0) ? ret : -EIO;
+
+ return 0;
+}
+
+static int smpro_core_read(void *context, const void *reg, size_t reg_size,
+ void *val, size_t val_size)
+{
+ struct device *dev = context;
+ struct i2c_client *i2c = to_i2c_client(dev);
+ struct i2c_msg xfer[2];
+ unsigned char buf[2];
+ int ret;
+
+ xfer[0].addr = i2c->addr;
+ xfer[0].flags = 0;
+
+ buf[0] = *(u8 *)reg;
+ buf[1] = val_size;
+ xfer[0].len = 2;
+ xfer[0].buf = buf;
+
+ xfer[1].addr = i2c->addr;
+ xfer[1].flags = I2C_M_RD;
+ xfer[1].len = val_size;
+ xfer[1].buf = val;
+
+ ret = i2c_transfer(i2c->adapter, xfer, 2);
+ if (unlikely(ret != 2))
+ return (ret < 0) ? ret : -EIO;
+
+ return 0;
+}
+
+static const struct regmap_bus smpro_regmap_bus = {
+ .read = smpro_core_read,
+ .write = smpro_core_write,
+ .val_format_endian_default = REGMAP_ENDIAN_BIG,
+};
+
+static bool smpro_core_readable_noinc_reg(struct device *dev, unsigned int reg)
+{
+ return (reg == CORE_CE_ERR_DATA || reg == CORE_UE_ERR_DATA ||
+ reg == MEM_CE_ERR_DATA || reg == MEM_UE_ERR_DATA ||
+ reg == PCIE_CE_ERR_DATA || reg == PCIE_UE_ERR_DATA ||
+ reg == OTHER_CE_ERR_DATA || reg == OTHER_UE_ERR_DATA);
+}
+
+static const struct regmap_config smpro_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 16,
+ .readable_noinc_reg = smpro_core_readable_noinc_reg,
+};
+
+static const struct mfd_cell smpro_devs[] = {
+ MFD_CELL_NAME("smpro-hwmon"),
+ MFD_CELL_NAME("smpro-errmon"),
+ MFD_CELL_NAME("smpro-misc"),
+};
+
+static int smpro_core_probe(struct i2c_client *i2c)
+{
+ const struct regmap_config *config;
+ struct regmap *regmap;
+ unsigned int val;
+ int ret;
+
+ config = device_get_match_data(&i2c->dev);
+ if (!config)
+ return -EINVAL;
+
+ regmap = devm_regmap_init(&i2c->dev, &smpro_regmap_bus, &i2c->dev, config);
+ if (IS_ERR(regmap))
+ return PTR_ERR(regmap);
+
+ ret = regmap_read(regmap, MANUFACTURER_ID_REG, &val);
+ if (ret)
+ return ret;
+
+ if (val != AMPERE_MANUFACTURER_ID)
+ return -ENODEV;
+
+ return devm_mfd_add_devices(&i2c->dev, PLATFORM_DEVID_AUTO,
+ smpro_devs, ARRAY_SIZE(smpro_devs), NULL, 0, NULL);
+}
+
+static const struct of_device_id smpro_core_of_match[] = {
+ { .compatible = "ampere,smpro", .data = &smpro_regmap_config },
+ {}
+};
+MODULE_DEVICE_TABLE(of, smpro_core_of_match);
+
+static struct i2c_driver smpro_core_driver = {
+ .probe_new = smpro_core_probe,
+ .driver = {
+ .name = "smpro-core",
+ .of_match_table = smpro_core_of_match,
+ },
+};
+module_i2c_driver(smpro_core_driver);
+
+MODULE_AUTHOR("Quan Nguyen <quan@os.amperecomputing.com>");
+MODULE_DESCRIPTION("SMPRO CORE - I2C driver");
+MODULE_LICENSE("GPL");
ddata->irq_chip.name = dev_name(&spi->dev);
ddata->irq_chip.status_base =
pdata->irq_base + SPRD_PMIC_INT_MASK_STATUS;
- ddata->irq_chip.mask_base = pdata->irq_base + SPRD_PMIC_INT_EN;
+ ddata->irq_chip.unmask_base = pdata->irq_base + SPRD_PMIC_INT_EN;
ddata->irq_chip.ack_base = 0;
ddata->irq_chip.num_regs = 1;
ddata->irq_chip.num_irqs = pdata->num_irqs;
- ddata->irq_chip.mask_invert = true;
ddata->irqs = devm_kcalloc(&spi->dev,
pdata->num_irqs, sizeof(struct regmap_irq),
return 0;
}
-#ifdef CONFIG_PM_SLEEP
static int sprd_pmic_suspend(struct device *dev)
{
struct sprd_pmic *ddata = dev_get_drvdata(dev);
return 0;
}
-#endif
-static SIMPLE_DEV_PM_OPS(sprd_pmic_pm_ops, sprd_pmic_suspend, sprd_pmic_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(sprd_pmic_pm_ops,
+ sprd_pmic_suspend, sprd_pmic_resume);
static const struct of_device_id sprd_pmic_match[] = {
{ .compatible = "sprd,sc2730", .data = &sc2730_data },
.driver = {
.name = "sc27xx-pmic",
.of_match_table = sprd_pmic_match,
- .pm = &sprd_pmic_pm_ops,
+ .pm = pm_sleep_ptr(&sprd_pmic_pm_ops),
},
.probe = sprd_pmic_probe,
.id_table = sprd_pmic_spi_ids,
{
struct device *dev = &pdev->dev;
struct stm32_lptimer *ddata;
- struct resource *res;
void __iomem *mmio;
int ret;
if (!ddata)
return -ENOMEM;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- mmio = devm_ioremap_resource(dev, res);
+ mmio = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
if (IS_ERR(mmio))
return PTR_ERR(mmio);
}
}
-static int stmfx_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int stmfx_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
struct stmfx *stmfx;
stmfx_chip_exit(client);
}
-#ifdef CONFIG_PM_SLEEP
static int stmfx_suspend(struct device *dev)
{
struct stmfx *stmfx = dev_get_drvdata(dev);
return 0;
}
-#endif
-static SIMPLE_DEV_PM_OPS(stmfx_dev_pm_ops, stmfx_suspend, stmfx_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(stmfx_dev_pm_ops, stmfx_suspend, stmfx_resume);
static const struct of_device_id stmfx_of_match[] = {
{ .compatible = "st,stmfx-0300", },
.driver = {
.name = "stmfx-core",
.of_match_table = stmfx_of_match,
- .pm = &stmfx_dev_pm_ops,
+ .pm = pm_sleep_ptr(&stmfx_dev_pm_ops),
},
- .probe = stmfx_probe,
+ .probe_new = stmfx_probe,
.remove = stmfx_remove,
};
module_i2c_driver(stmfx_driver);
MODULE_DEVICE_TABLE(of, stmpe_of_match);
static int
-stmpe_i2c_probe(struct i2c_client *i2c, const struct i2c_device_id *id)
+stmpe_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
enum stmpe_partnum partnum;
const struct of_device_id *of_id;
static struct i2c_driver stmpe_i2c_driver = {
.driver = {
.name = "stmpe-i2c",
-#ifdef CONFIG_PM
- .pm = &stmpe_dev_pm_ops,
-#endif
+ .pm = pm_sleep_ptr(&stmpe_dev_pm_ops),
.of_match_table = stmpe_of_match,
},
- .probe = stmpe_i2c_probe,
+ .probe_new = stmpe_i2c_probe,
.remove = stmpe_i2c_remove,
.id_table = stmpe_i2c_id,
};
.driver = {
.name = "stmpe-spi",
.of_match_table = of_match_ptr(stmpe_spi_of_match),
-#ifdef CONFIG_PM
- .pm = &stmpe_dev_pm_ops,
-#endif
+ .pm = pm_sleep_ptr(&stmpe_dev_pm_ops),
},
.probe = stmpe_spi_probe,
.remove = stmpe_spi_remove,
mfd_remove_devices(stmpe->dev);
}
-#ifdef CONFIG_PM
static int stmpe_suspend(struct device *dev)
{
struct stmpe *stmpe = dev_get_drvdata(dev);
return 0;
}
-const struct dev_pm_ops stmpe_dev_pm_ops = {
- .suspend = stmpe_suspend,
- .resume = stmpe_resume,
-};
-#endif
+EXPORT_GPL_SIMPLE_DEV_PM_OPS(stmpe_dev_pm_ops,
+ stmpe_suspend, stmpe_resume);
static const struct regmap_irq_chip stpmic1_regmap_irq_chip = {
.name = "pmic_irq",
.status_base = INT_PENDING_R1,
- .mask_base = INT_CLEAR_MASK_R1,
- .unmask_base = INT_SET_MASK_R1,
+ .mask_base = INT_SET_MASK_R1,
+ .unmask_base = INT_CLEAR_MASK_R1,
+ .mask_unmask_non_inverted = true,
.ack_base = INT_CLEAR_R1,
.num_regs = STPMIC1_PMIC_NUM_IRQ_REGS,
.irqs = stpmic1_irqs,
.num_irqs = ARRAY_SIZE(stpmic1_irqs),
};
-static int stpmic1_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int stpmic1_probe(struct i2c_client *i2c)
{
struct stpmic1 *ddata;
struct device *dev = &i2c->dev;
return devm_of_platform_populate(dev);
}
-#ifdef CONFIG_PM_SLEEP
static int stpmic1_suspend(struct device *dev)
{
struct i2c_client *i2c = to_i2c_client(dev);
return 0;
}
-#endif
-static SIMPLE_DEV_PM_OPS(stpmic1_pm, stpmic1_suspend, stpmic1_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(stpmic1_pm, stpmic1_suspend, stpmic1_resume);
static const struct of_device_id stpmic1_of_match[] = {
{ .compatible = "st,stpmic1", },
.driver = {
.name = "stpmic1",
.of_match_table = of_match_ptr(stpmic1_of_match),
- .pm = &stpmic1_pm,
+ .pm = pm_sleep_ptr(&stpmic1_pm),
},
- .probe = stpmic1_probe,
+ .probe_new = stpmic1_probe,
};
module_i2c_driver(stpmic1_driver);
.val_bits = 8,
};
-static int stw481x_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int stw481x_probe(struct i2c_client *client)
{
struct stw481x *stw481x;
int ret;
.name = "stw481x",
.of_match_table = stw481x_match,
},
- .probe = stw481x_probe,
+ .probe_new = stw481x_probe,
.id_table = stw481x_id,
};
.name = "sun4i_gpadc_irq_chip",
.status_base = SUN4I_GPADC_INT_FIFOS,
.ack_base = SUN4I_GPADC_INT_FIFOS,
- .mask_base = SUN4I_GPADC_INT_FIFOC,
+ .unmask_base = SUN4I_GPADC_INT_FIFOC,
.init_ack_masked = true,
- .mask_invert = true,
.irqs = sun4i_gpadc_regmap_irq,
.num_irqs = ARRAY_SIZE(sun4i_gpadc_regmap_irq),
.num_regs = 1,
/*--------------------------------------------------------------------------*/
-#ifdef CONFIG_PM
static int t7l66xb_suspend(struct platform_device *dev, pm_message_t state)
{
struct t7l66xb *t7l66xb = platform_get_drvdata(dev);
return 0;
}
-#else
-#define t7l66xb_suspend NULL
-#define t7l66xb_resume NULL
-#endif
/*--------------------------------------------------------------------------*/
.driver = {
.name = "t7l66xb",
},
- .suspend = t7l66xb_suspend,
- .resume = t7l66xb_resume,
+ .suspend = pm_sleep_ptr(t7l66xb_suspend),
+ .resume = pm_sleep_ptr(t7l66xb_resume),
.probe = t7l66xb_probe,
.remove = t7l66xb_remove,
};
return pdata;
}
-static int tc3589x_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int tc3589x_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct device_node *np = i2c->dev.of_node;
struct tc3589x_platform_data *pdata = dev_get_platdata(&i2c->dev);
struct tc3589x *tc3589x;
mfd_remove_devices(tc3589x->dev);
}
-#ifdef CONFIG_PM_SLEEP
static int tc3589x_suspend(struct device *dev)
{
struct tc3589x *tc3589x = dev_get_drvdata(dev);
return ret;
}
-#endif
-static SIMPLE_DEV_PM_OPS(tc3589x_dev_pm_ops, tc3589x_suspend, tc3589x_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(tc3589x_dev_pm_ops,
+ tc3589x_suspend, tc3589x_resume);
static const struct i2c_device_id tc3589x_id[] = {
{ "tc35890", TC3589X_TC35890 },
static struct i2c_driver tc3589x_driver = {
.driver = {
.name = "tc3589x",
- .pm = &tc3589x_dev_pm_ops,
+ .pm = pm_sleep_ptr(&tc3589x_dev_pm_ops),
.of_match_table = of_match_ptr(tc3589x_match),
},
- .probe = tc3589x_probe,
+ .probe_new = tc3589x_probe,
.remove = tc3589x_remove,
.id_table = tc3589x_id,
};
/*--------------------------------------------------------------------------*/
-#ifdef CONFIG_PM
static int tc6387xb_suspend(struct platform_device *dev, pm_message_t state)
{
struct tc6387xb *tc6387xb = platform_get_drvdata(dev);
return 0;
}
-#else
-#define tc6387xb_suspend NULL
-#define tc6387xb_resume NULL
-#endif
/*--------------------------------------------------------------------------*/
},
.probe = tc6387xb_probe,
.remove = tc6387xb_remove,
- .suspend = tc6387xb_suspend,
- .resume = tc6387xb_resume,
+ .suspend = pm_sleep_ptr(tc6387xb_suspend),
+ .resume = pm_sleep_ptr(tc6387xb_resume),
};
module_platform_driver(tc6387xb_platform_driver);
return 0;
}
-#ifdef CONFIG_PM
static int tc6393xb_suspend(struct platform_device *dev, pm_message_t state)
{
struct tc6393xb_platform_data *tcpd = dev_get_platdata(&dev->dev);
return 0;
}
-#else
-#define tc6393xb_suspend NULL
-#define tc6393xb_resume NULL
-#endif
static struct platform_driver tc6393xb_driver = {
.probe = tc6393xb_probe,
.remove = tc6393xb_remove,
- .suspend = tc6393xb_suspend,
- .resume = tc6393xb_resume,
+ .suspend = pm_sleep_ptr(tc6393xb_suspend),
+ .resume = pm_sleep_ptr(tc6393xb_resume),
.driver = {
.name = "tc6393xb",
TI_LMU_DATA(lm3695, LM3695_MAX_REG);
TI_LMU_DATA(lm36274, LM36274_MAX_REG);
-static int ti_lmu_probe(struct i2c_client *cl, const struct i2c_device_id *id)
+static int ti_lmu_probe(struct i2c_client *cl)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(cl);
struct device *dev = &cl->dev;
const struct ti_lmu_data *data;
struct regmap_config regmap_cfg;
MODULE_DEVICE_TABLE(i2c, ti_lmu_ids);
static struct i2c_driver ti_lmu_driver = {
- .probe = ti_lmu_probe,
+ .probe_new = ti_lmu_probe,
.driver = {
.name = "ti-lmu",
.of_match_table = ti_lmu_of_match,
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/pci.h>
-#include <linux/msi.h>
#include <linux/mfd/core.h>
#include <linux/slab.h>
return pdata;
}
-static int tps6105x_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tps6105x_probe(struct i2c_client *client)
{
struct tps6105x *tps6105x;
struct tps6105x_platform_data *pdata;
.name = "tps6105x",
.of_match_table = tps6105x_of_match,
},
- .probe = tps6105x_probe,
+ .probe_new = tps6105x_probe,
.remove = tps6105x_remove,
.id_table = tps6105x_id,
};
the_tps = NULL;
}
-static int tps65010_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tps65010_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct tps65010 *tps;
int status;
struct tps65010_board *board = dev_get_platdata(&client->dev);
.driver = {
.name = "tps65010",
},
- .probe = tps65010_probe,
+ .probe_new = tps65010_probe,
.remove = tps65010_remove,
.id_table = tps65010_id,
};
return 0;
}
-static int tps6507x_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int tps6507x_i2c_probe(struct i2c_client *i2c)
{
struct tps6507x_dev *tps6507x;
.name = "tps6507x",
.of_match_table = of_match_ptr(tps6507x_of_match),
},
- .probe = tps6507x_i2c_probe,
+ .probe_new = tps6507x_i2c_probe,
.id_table = tps6507x_i2c_id,
};
};
MODULE_DEVICE_TABLE(of, tps65086_of_match_table);
-static int tps65086_probe(struct i2c_client *client,
- const struct i2c_device_id *ids)
+static int tps65086_probe(struct i2c_client *client)
{
struct tps65086 *tps;
unsigned int version;
.name = "tps65086",
.of_match_table = tps65086_of_match_table,
},
- .probe = tps65086_probe,
+ .probe_new = tps65086_probe,
.remove = tps65086_remove,
.id_table = tps65086_id_table,
};
.num_irqs = ARRAY_SIZE(tps65090_irqs),
.num_regs = NUM_INT_REG,
.status_base = TPS65090_REG_INTR_STS,
- .mask_base = TPS65090_REG_INTR_MASK,
- .mask_invert = true,
+ .unmask_base = TPS65090_REG_INTR_MASK,
};
static bool is_volatile_reg(struct device *dev, unsigned int reg)
};
#endif
-static int tps65090_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tps65090_i2c_probe(struct i2c_client *client)
{
struct tps65090_platform_data *pdata = dev_get_platdata(&client->dev);
int irq_base = 0;
.suppress_bind_attrs = true,
.of_match_table = of_match_ptr(tps65090_of_match),
},
- .probe = tps65090_i2c_probe,
+ .probe_new = tps65090_i2c_probe,
.id_table = tps65090_id_table,
};
return 0;
}
-static int tps65218_probe(struct i2c_client *client,
- const struct i2c_device_id *ids)
+static int tps65218_probe(struct i2c_client *client)
{
struct tps65218 *tps;
int ret;
.name = "tps65218",
.of_match_table = of_tps65218_match_table,
},
- .probe = tps65218_probe,
+ .probe_new = tps65218_probe,
.id_table = tps65218_id_table,
};
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+//
+// Driver for TPS65219 Integrated Power Management Integrated Chips (PMIC)
+//
+// Copyright (C) 2022 BayLibre Incorporated - https://www.baylibre.com/
+
+#include <linux/i2c.h>
+#include <linux/reboot.h>
+#include <linux/regmap.h>
+
+#include <linux/mfd/core.h>
+#include <linux/mfd/tps65219.h>
+
+static int tps65219_warm_reset(struct tps65219 *tps)
+{
+ return regmap_update_bits(tps->regmap, TPS65219_REG_MFP_CTRL,
+ TPS65219_MFP_WARM_RESET_I2C_CTRL_MASK,
+ TPS65219_MFP_WARM_RESET_I2C_CTRL_MASK);
+}
+
+static int tps65219_cold_reset(struct tps65219 *tps)
+{
+ return regmap_update_bits(tps->regmap, TPS65219_REG_MFP_CTRL,
+ TPS65219_MFP_COLD_RESET_I2C_CTRL_MASK,
+ TPS65219_MFP_COLD_RESET_I2C_CTRL_MASK);
+}
+
+static int tps65219_restart(struct notifier_block *this,
+ unsigned long reboot_mode, void *cmd)
+{
+ struct tps65219 *tps;
+
+ tps = container_of(this, struct tps65219, nb);
+
+ if (reboot_mode == REBOOT_WARM)
+ tps65219_warm_reset(tps);
+ else
+ tps65219_cold_reset(tps);
+
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block pmic_rst_restart_nb = {
+ .notifier_call = tps65219_restart,
+ .priority = 200,
+};
+
+static const struct resource tps65219_pwrbutton_resources[] = {
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_PB_FALLING_EDGE_DETECT, "falling"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_PB_RISING_EDGE_DETECT, "rising"),
+};
+
+static const struct resource tps65219_regulator_resources[] = {
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO3_SCG, "LDO3_SCG"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO3_OC, "LDO3_OC"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO3_UV, "LDO3_UV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO4_SCG, "LDO4_SCG"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO4_OC, "LDO4_OC"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO4_UV, "LDO4_UV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO1_SCG, "LDO1_SCG"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO1_OC, "LDO1_OC"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO1_UV, "LDO1_UV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO2_SCG, "LDO2_SCG"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO2_OC, "LDO2_OC"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO2_UV, "LDO2_UV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK3_SCG, "BUCK3_SCG"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK3_OC, "BUCK3_OC"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK3_NEG_OC, "BUCK3_NEG_OC"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK3_UV, "BUCK3_UV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK1_SCG, "BUCK1_SCG"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK1_OC, "BUCK1_OC"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK1_NEG_OC, "BUCK1_NEG_OC"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK1_UV, "BUCK1_UV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK2_SCG, "BUCK2_SCG"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK2_OC, "BUCK2_OC"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK2_NEG_OC, "BUCK2_NEG_OC"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK2_UV, "BUCK2_UV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK1_RV, "BUCK1_RV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK2_RV, "BUCK2_RV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK3_RV, "BUCK3_RV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO1_RV, "LDO1_RV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO2_RV, "LDO2_RV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO3_RV, "LDO3_RV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO4_RV, "LDO4_RV"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK1_RV_SD, "BUCK1_RV_SD"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK2_RV_SD, "BUCK2_RV_SD"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_BUCK3_RV_SD, "BUCK3_RV_SD"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO1_RV_SD, "LDO1_RV_SD"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO2_RV_SD, "LDO2_RV_SD"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO3_RV_SD, "LDO3_RV_SD"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_LDO4_RV_SD, "LDO4_RV_SD"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_TIMEOUT, "TIMEOUT"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_SENSOR_3_WARM, "SENSOR_3_WARM"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_SENSOR_2_WARM, "SENSOR_2_WARM"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_SENSOR_1_WARM, "SENSOR_1_WARM"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_SENSOR_0_WARM, "SENSOR_0_WARM"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_SENSOR_3_HOT, "SENSOR_3_HOT"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_SENSOR_2_HOT, "SENSOR_2_HOT"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_SENSOR_1_HOT, "SENSOR_1_HOT"),
+ DEFINE_RES_IRQ_NAMED(TPS65219_INT_SENSOR_0_HOT, "SENSOR_0_HOT"),
+};
+
+static const struct mfd_cell tps65219_cells[] = {
+ {
+ .name = "tps65219-regulator",
+ .resources = tps65219_regulator_resources,
+ .num_resources = ARRAY_SIZE(tps65219_regulator_resources),
+ },
+ { .name = "tps65219-gpios", },
+};
+
+static const struct mfd_cell tps65219_pwrbutton_cell = {
+ .name = "tps65219-pwrbutton",
+ .resources = tps65219_pwrbutton_resources,
+ .num_resources = ARRAY_SIZE(tps65219_pwrbutton_resources),
+};
+
+static const struct regmap_config tps65219_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+ .max_register = TPS65219_REG_FACTORY_CONFIG_2,
+};
+
+/*
+ * Mapping of main IRQ register bits to sub-IRQ register offsets so that we can
+ * access corect sub-IRQ registers based on bits that are set in main IRQ
+ * register.
+ */
+/* Timeout Residual Voltage Shutdown */
+static unsigned int bit0_offsets[] = { TPS65219_REG_INT_TO_RV_POS };
+static unsigned int bit1_offsets[] = { TPS65219_REG_INT_RV_POS }; /* Residual Voltage */
+static unsigned int bit2_offsets[] = { TPS65219_REG_INT_SYS_POS }; /* System */
+static unsigned int bit3_offsets[] = { TPS65219_REG_INT_BUCK_1_2_POS }; /* Buck 1-2 */
+static unsigned int bit4_offsets[] = { TPS65219_REG_INT_BUCK_3_POS }; /* Buck 3 */
+static unsigned int bit5_offsets[] = { TPS65219_REG_INT_LDO_1_2_POS }; /* LDO 1-2 */
+static unsigned int bit6_offsets[] = { TPS65219_REG_INT_LDO_3_4_POS }; /* LDO 3-4 */
+static unsigned int bit7_offsets[] = { TPS65219_REG_INT_PB_POS }; /* Power Button */
+
+static struct regmap_irq_sub_irq_map tps65219_sub_irq_offsets[] = {
+ REGMAP_IRQ_MAIN_REG_OFFSET(bit0_offsets),
+ REGMAP_IRQ_MAIN_REG_OFFSET(bit1_offsets),
+ REGMAP_IRQ_MAIN_REG_OFFSET(bit2_offsets),
+ REGMAP_IRQ_MAIN_REG_OFFSET(bit3_offsets),
+ REGMAP_IRQ_MAIN_REG_OFFSET(bit4_offsets),
+ REGMAP_IRQ_MAIN_REG_OFFSET(bit5_offsets),
+ REGMAP_IRQ_MAIN_REG_OFFSET(bit6_offsets),
+ REGMAP_IRQ_MAIN_REG_OFFSET(bit7_offsets),
+};
+
+#define TPS65219_REGMAP_IRQ_REG(int_name, register_position) \
+ REGMAP_IRQ_REG(int_name, register_position, int_name##_MASK)
+
+static struct regmap_irq tps65219_irqs[] = {
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO3_SCG, TPS65219_REG_INT_LDO_3_4_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO3_OC, TPS65219_REG_INT_LDO_3_4_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO3_UV, TPS65219_REG_INT_LDO_3_4_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO4_SCG, TPS65219_REG_INT_LDO_3_4_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO4_OC, TPS65219_REG_INT_LDO_3_4_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO4_UV, TPS65219_REG_INT_LDO_3_4_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO1_SCG, TPS65219_REG_INT_LDO_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO1_OC, TPS65219_REG_INT_LDO_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO1_UV, TPS65219_REG_INT_LDO_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO2_SCG, TPS65219_REG_INT_LDO_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO2_OC, TPS65219_REG_INT_LDO_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO2_UV, TPS65219_REG_INT_LDO_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK3_SCG, TPS65219_REG_INT_BUCK_3_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK3_OC, TPS65219_REG_INT_BUCK_3_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK3_NEG_OC, TPS65219_REG_INT_BUCK_3_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK3_UV, TPS65219_REG_INT_BUCK_3_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK2_SCG, TPS65219_REG_INT_BUCK_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK2_OC, TPS65219_REG_INT_BUCK_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK2_NEG_OC, TPS65219_REG_INT_BUCK_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK2_UV, TPS65219_REG_INT_BUCK_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK1_SCG, TPS65219_REG_INT_BUCK_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK1_OC, TPS65219_REG_INT_BUCK_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK1_NEG_OC, TPS65219_REG_INT_BUCK_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK1_UV, TPS65219_REG_INT_BUCK_1_2_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_SENSOR_3_WARM, TPS65219_REG_INT_SYS_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_SENSOR_2_WARM, TPS65219_REG_INT_SYS_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_SENSOR_1_WARM, TPS65219_REG_INT_SYS_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_SENSOR_0_WARM, TPS65219_REG_INT_SYS_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_SENSOR_3_HOT, TPS65219_REG_INT_SYS_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_SENSOR_2_HOT, TPS65219_REG_INT_SYS_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_SENSOR_1_HOT, TPS65219_REG_INT_SYS_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_SENSOR_0_HOT, TPS65219_REG_INT_SYS_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK1_RV, TPS65219_REG_INT_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK2_RV, TPS65219_REG_INT_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK3_RV, TPS65219_REG_INT_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO1_RV, TPS65219_REG_INT_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO2_RV, TPS65219_REG_INT_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO3_RV, TPS65219_REG_INT_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO4_RV, TPS65219_REG_INT_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK1_RV_SD, TPS65219_REG_INT_TO_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK2_RV_SD, TPS65219_REG_INT_TO_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_BUCK3_RV_SD, TPS65219_REG_INT_TO_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO1_RV_SD, TPS65219_REG_INT_TO_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO2_RV_SD, TPS65219_REG_INT_TO_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO3_RV_SD, TPS65219_REG_INT_TO_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_LDO4_RV_SD, TPS65219_REG_INT_TO_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_TIMEOUT, TPS65219_REG_INT_TO_RV_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_PB_FALLING_EDGE_DETECT, TPS65219_REG_INT_PB_POS),
+ TPS65219_REGMAP_IRQ_REG(TPS65219_INT_PB_RISING_EDGE_DETECT, TPS65219_REG_INT_PB_POS),
+};
+
+static struct regmap_irq_chip tps65219_irq_chip = {
+ .name = "tps65219_irq",
+ .main_status = TPS65219_REG_INT_SOURCE,
+ .num_main_regs = 1,
+ .num_main_status_bits = 8,
+ .irqs = tps65219_irqs,
+ .num_irqs = ARRAY_SIZE(tps65219_irqs),
+ .status_base = TPS65219_REG_INT_LDO_3_4,
+ .ack_base = TPS65219_REG_INT_LDO_3_4,
+ .clear_ack = 1,
+ .num_regs = 8,
+ .sub_reg_offsets = tps65219_sub_irq_offsets,
+};
+
+static int tps65219_probe(struct i2c_client *client)
+{
+ struct tps65219 *tps;
+ unsigned int chipid;
+ bool pwr_button;
+ int ret;
+
+ tps = devm_kzalloc(&client->dev, sizeof(*tps), GFP_KERNEL);
+ if (!tps)
+ return -ENOMEM;
+
+ i2c_set_clientdata(client, tps);
+
+ tps->dev = &client->dev;
+
+ tps->regmap = devm_regmap_init_i2c(client, &tps65219_regmap_config);
+ if (IS_ERR(tps->regmap)) {
+ ret = PTR_ERR(tps->regmap);
+ dev_err(tps->dev, "Failed to allocate register map: %d\n", ret);
+ return ret;
+ }
+
+ ret = devm_regmap_add_irq_chip(&client->dev, tps->regmap, client->irq,
+ IRQF_ONESHOT, 0, &tps65219_irq_chip,
+ &tps->irq_data);
+ if (ret)
+ return ret;
+
+ ret = regmap_read(tps->regmap, TPS65219_REG_TI_DEV_ID, &chipid);
+ if (ret) {
+ dev_err(tps->dev, "Failed to read device ID: %d\n", ret);
+ return ret;
+ }
+
+ ret = devm_mfd_add_devices(tps->dev, PLATFORM_DEVID_AUTO,
+ tps65219_cells, ARRAY_SIZE(tps65219_cells),
+ NULL, 0, regmap_irq_get_domain(tps->irq_data));
+ if (ret) {
+ dev_err(tps->dev, "Failed to add child devices: %d\n", ret);
+ return ret;
+ }
+
+ pwr_button = of_property_read_bool(tps->dev->of_node, "ti,power-button");
+ if (pwr_button) {
+ ret = devm_mfd_add_devices(tps->dev, PLATFORM_DEVID_AUTO,
+ &tps65219_pwrbutton_cell, 1, NULL, 0,
+ regmap_irq_get_domain(tps->irq_data));
+ if (ret) {
+ dev_err(tps->dev, "Failed to add power-button: %d\n", ret);
+ return ret;
+ }
+ }
+
+ tps->nb = pmic_rst_restart_nb;
+ ret = register_restart_handler(&tps->nb);
+ if (ret) {
+ dev_err(tps->dev, "cannot register restart handler, %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static const struct of_device_id of_tps65219_match_table[] = {
+ { .compatible = "ti,tps65219", },
+ {}
+};
+MODULE_DEVICE_TABLE(of, of_tps65219_match_table);
+
+static struct i2c_driver tps65219_driver = {
+ .driver = {
+ .name = "tps65219",
+ .of_match_table = of_tps65219_match_table,
+ },
+ .probe_new = tps65219_probe,
+};
+module_i2c_driver(tps65219_driver);
+
+MODULE_AUTHOR("Jerome Neanne <jneanne@baylibre.com>");
+MODULE_DESCRIPTION("TPS65219 power management IC driver");
+MODULE_LICENSE("GPL");
mutex_unlock(&tps6586x->irq_lock);
}
-#ifdef CONFIG_PM_SLEEP
static int tps6586x_irq_set_wake(struct irq_data *irq_data, unsigned int on)
{
struct tps6586x *tps6586x = irq_data_get_irq_chip_data(irq_data);
return irq_set_irq_wake(tps6586x->irq, on);
}
-#else
-#define tps6586x_irq_set_wake NULL
-#endif
static struct irq_chip tps6586x_irq_chip = {
.name = "tps6586x",
.irq_bus_sync_unlock = tps6586x_irq_sync_unlock,
.irq_disable = tps6586x_irq_disable,
.irq_enable = tps6586x_irq_enable,
- .irq_set_wake = tps6586x_irq_set_wake,
+ .irq_set_wake = pm_sleep_ptr(tps6586x_irq_set_wake),
};
static int tps6586x_irq_map(struct irq_domain *h, unsigned int virq,
dev_info(&client->dev, "Found %s, VERSIONCRC is %02x\n", name, version);
}
-static int tps6586x_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tps6586x_i2c_probe(struct i2c_client *client)
{
struct tps6586x_platform_data *pdata = dev_get_platdata(&client->dev);
struct tps6586x *tps6586x;
.of_match_table = of_match_ptr(tps6586x_of_match),
.pm = &tps6586x_pm_ops,
},
- .probe = tps6586x_i2c_probe,
+ .probe_new = tps6586x_i2c_probe,
.remove = tps6586x_i2c_remove,
.id_table = tps6586x_id_table,
};
DEVCTRL_DEV_OFF_MASK);
}
-static int tps65910_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int tps65910_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct tps65910 *tps65910;
struct tps65910_board *pmic_plat_data;
struct tps65910_board *of_pmic_plat_data = NULL;
.name = "tps65910",
.of_match_table = of_match_ptr(tps65910_of_match),
},
- .probe = tps65910_i2c_probe,
+ .probe_new = tps65910_i2c_probe,
.id_table = tps65910_i2c_id,
};
};
MODULE_DEVICE_TABLE(of, tps65912_i2c_of_match_table);
-static int tps65912_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *ids)
+static int tps65912_i2c_probe(struct i2c_client *client)
{
struct tps65912 *tps;
.name = "tps65912",
.of_match_table = tps65912_i2c_of_match_table,
},
- .probe = tps65912_i2c_probe,
+ .probe_new = tps65912_i2c_probe,
.remove = tps65912_i2c_remove,
.id_table = tps65912_i2c_id_table,
};
/* NOTE: This driver only handles a single twl4030/tps659x0 chip */
static int
-twl_probe(struct i2c_client *client, const struct i2c_device_id *id)
+twl_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device_node *node = client->dev.of_node;
struct platform_device *pdev;
const struct regmap_config *twl_regmap_config;
.driver.name = DRIVER_NAME,
.driver.pm = &twl_dev_pm_ops,
.id_table = twl_ids,
- .probe = twl_probe,
+ .probe_new = twl_probe,
.remove = twl_remove,
};
builtin_i2c_driver(twl_driver);
#include <linux/platform_device.h>
#include <linux/of.h>
#include <linux/of_irq.h>
-#include <linux/of_gpio.h>
#include <linux/of_platform.h>
-#include <linux/gpio.h>
+#include <linux/gpio/consumer.h>
#include <linux/delay.h>
#include <linux/i2c.h>
#include <linux/regmap.h>
{
int time_left;
- gpio_set_value(twl6040->audpwron, 1);
+ gpiod_set_value_cansleep(twl6040->audpwron, 1);
time_left = wait_for_completion_timeout(&twl6040->ready,
msecs_to_jiffies(144));
intid = twl6040_reg_read(twl6040, TWL6040_REG_INTID);
if (!(intid & TWL6040_READYINT)) {
dev_err(twl6040->dev, "automatic power-up failed\n");
- gpio_set_value(twl6040->audpwron, 0);
+ gpiod_set_value_cansleep(twl6040->audpwron, 0);
return -ETIMEDOUT;
}
}
/* Allow writes to the chip */
regcache_cache_only(twl6040->regmap, false);
- if (gpio_is_valid(twl6040->audpwron)) {
+ if (twl6040->audpwron) {
/* use automatic power-up sequence */
ret = twl6040_power_up_automatic(twl6040);
if (ret) {
if (--twl6040->power_count)
goto out;
- if (gpio_is_valid(twl6040->audpwron)) {
+ if (twl6040->audpwron) {
/* use AUDPWRON line */
- gpio_set_value(twl6040->audpwron, 0);
+ gpiod_set_value_cansleep(twl6040->audpwron, 0);
/* power-down sequence latency */
usleep_range(500, 700);
.mask_base = TWL6040_REG_INTMR,
};
-static int twl6040_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int twl6040_probe(struct i2c_client *client)
{
struct device_node *node = client->dev.of_node;
struct twl6040 *twl6040;
}
/* ERRATA: Automatic power-up is not possible in ES1.0 */
- if (twl6040_get_revid(twl6040) > TWL6040_REV_ES1_0)
- twl6040->audpwron = of_get_named_gpio(node,
- "ti,audpwron-gpio", 0);
- else
- twl6040->audpwron = -EINVAL;
-
- if (gpio_is_valid(twl6040->audpwron)) {
- ret = devm_gpio_request_one(&client->dev, twl6040->audpwron,
- GPIOF_OUT_INIT_LOW, "audpwron");
+ if (twl6040_get_revid(twl6040) > TWL6040_REV_ES1_0) {
+ twl6040->audpwron = devm_gpiod_get_optional(&client->dev,
+ "ti,audpwron",
+ GPIOD_OUT_LOW);
+ ret = PTR_ERR_OR_ZERO(twl6040->audpwron);
if (ret)
goto gpio_err;
+ gpiod_set_consumer_name(twl6040->audpwron, "audpwron");
+
/* Clear any pending interrupt */
twl6040_reg_read(twl6040, TWL6040_REG_INTID);
}
.driver = {
.name = "twl6040",
},
- .probe = twl6040_probe,
+ .probe_new = twl6040_probe,
.remove = twl6040_remove,
.id_table = twl6040_i2c_id,
};
mutex_unlock(&ucb1x00_mutex);
}
-#ifdef CONFIG_PM_SLEEP
static int ucb1x00_suspend(struct device *dev)
{
struct ucb1x00_plat_data *pdata = dev_get_platdata(dev);
mutex_unlock(&ucb1x00_mutex);
return 0;
}
-#endif
-static SIMPLE_DEV_PM_OPS(ucb1x00_pm_ops, ucb1x00_suspend, ucb1x00_resume);
+static DEFINE_SIMPLE_DEV_PM_OPS(ucb1x00_pm_ops,
+ ucb1x00_suspend, ucb1x00_resume);
static struct mcp_driver ucb1x00_driver = {
.drv = {
.name = "ucb1x00",
.owner = THIS_MODULE,
- .pm = &ucb1x00_pm_ops,
+ .pm = pm_sleep_ptr(&ucb1x00_pm_ops),
},
.probe = ucb1x00_probe,
.remove = ucb1x00_remove,
.name = "basic-mmio-gpio",
.of_compatible = "arm,vexpress-sysreg,sys_led",
.num_resources = 1,
- .resources = (struct resource []) {
- DEFINE_RES_MEM_NAMED(SYS_LED, 0x4, "dat"),
- },
+ .resources = &DEFINE_RES_MEM_NAMED(SYS_LED, 0x4, "dat"),
.platform_data = &vexpress_sysreg_sys_led_pdata,
.pdata_size = sizeof(vexpress_sysreg_sys_led_pdata),
}, {
.name = "basic-mmio-gpio",
.of_compatible = "arm,vexpress-sysreg,sys_mci",
.num_resources = 1,
- .resources = (struct resource []) {
- DEFINE_RES_MEM_NAMED(SYS_MCI, 0x4, "dat"),
- },
+ .resources = &DEFINE_RES_MEM_NAMED(SYS_MCI, 0x4, "dat"),
.platform_data = &vexpress_sysreg_sys_mci_pdata,
.pdata_size = sizeof(vexpress_sysreg_sys_mci_pdata),
}, {
.name = "basic-mmio-gpio",
.of_compatible = "arm,vexpress-sysreg,sys_flash",
.num_resources = 1,
- .resources = (struct resource []) {
- DEFINE_RES_MEM_NAMED(SYS_FLASH, 0x4, "dat"),
- },
+ .resources = &DEFINE_RES_MEM_NAMED(SYS_FLASH, 0x4, "dat"),
.platform_data = &vexpress_sysreg_sys_flash_pdata,
.pdata_size = sizeof(vexpress_sysreg_sys_flash_pdata),
}, {
.name = "vexpress-syscfg",
.num_resources = 1,
- .resources = (struct resource []) {
- DEFINE_RES_MEM(SYS_MISC, 0x4c),
- },
+ .resources = &DEFINE_RES_MEM(SYS_MISC, 0x4c),
}
};
WCD934X_REGMAP_IRQ_REG(WCD934X_IRQ_SOUNDWIRE, 2, BIT(4)),
};
+static const unsigned int wcd934x_config_regs[] = {
+ WCD934X_INTR_LEVEL0,
+};
+
static const struct regmap_irq_chip wcd934x_regmap_irq_chip = {
.name = "wcd934x_irq",
.status_base = WCD934X_INTR_PIN1_STATUS0,
.mask_base = WCD934X_INTR_PIN1_MASK0,
.ack_base = WCD934X_INTR_PIN1_CLEAR0,
- .type_base = WCD934X_INTR_LEVEL0,
- .num_type_reg = 4,
- .type_in_mask = false,
.num_regs = 4,
.irqs = wcd934x_irqs,
.num_irqs = ARRAY_SIZE(wcd934x_irqs),
+ .config_base = wcd934x_config_regs,
+ .num_config_bases = ARRAY_SIZE(wcd934x_config_regs),
+ .num_config_regs = 4,
+ .set_type_config = regmap_irq_set_type_config_simple,
};
static bool wcd934x_is_volatile_register(struct device *dev, unsigned int reg)
return 0;
}
-static int wl1273_core_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int wl1273_core_probe(struct i2c_client *client)
{
struct wl1273_fm_platform_data *pdata = dev_get_platdata(&client->dev);
struct wl1273_core *core;
.driver = {
.name = WL1273_FM_DRIVER_NAME,
},
- .probe = wl1273_core_probe,
+ .probe_new = wl1273_core_probe,
.id_table = wl1273_driver_id_table,
};
#include <linux/mfd/wm831x/core.h>
#include <linux/mfd/wm831x/pdata.h>
-static int wm831x_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int wm831x_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
struct wm831x_pdata *pdata = dev_get_platdata(&i2c->dev);
const struct of_device_id *of_id;
struct wm831x *wm831x;
.of_match_table = of_match_ptr(wm831x_of_match),
.suppress_bind_attrs = true,
},
- .probe = wm831x_i2c_probe,
+ .probe_new = wm831x_i2c_probe,
.id_table = wm831x_i2c_id,
};
#include <linux/regmap.h>
#include <linux/slab.h>
-static int wm8350_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int wm8350_i2c_probe(struct i2c_client *i2c)
{
struct wm8350 *wm8350;
struct wm8350_platform_data *pdata = dev_get_platdata(&i2c->dev);
.name = "wm8350",
.suppress_bind_attrs = true,
},
- .probe = wm8350_i2c_probe,
+ .probe_new = wm8350_i2c_probe,
.id_table = wm8350_i2c_id,
};
EXPORT_SYMBOL_GPL(wm8400_reset_codec_reg_cache);
#if IS_ENABLED(CONFIG_I2C)
-static int wm8400_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int wm8400_i2c_probe(struct i2c_client *i2c)
{
struct wm8400 *wm8400;
.driver = {
.name = "WM8400",
},
- .probe = wm8400_i2c_probe,
+ .probe_new = wm8400_i2c_probe,
.id_table = wm8400_i2c_id,
};
#endif
"SPKVDD2",
};
-#ifdef CONFIG_PM
static int wm8994_suspend(struct device *dev)
{
struct wm8994 *wm8994 = dev_get_drvdata(dev);
return ret;
}
-#endif
#ifdef CONFIG_REGULATOR
static int wm8994_ldo_in_use(struct wm8994_pdata *pdata, int ldo)
};
MODULE_DEVICE_TABLE(of, wm8994_of_match);
-static int wm8994_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int wm8994_i2c_probe(struct i2c_client *i2c)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(i2c);
const struct of_device_id *of_id;
struct wm8994 *wm8994;
int ret;
MODULE_DEVICE_TABLE(i2c, wm8994_i2c_id);
static const struct dev_pm_ops wm8994_pm_ops = {
- SET_RUNTIME_PM_OPS(wm8994_suspend, wm8994_resume, NULL)
+ RUNTIME_PM_OPS(wm8994_suspend, wm8994_resume, NULL)
};
static struct i2c_driver wm8994_i2c_driver = {
.driver = {
.name = "wm8994",
- .pm = &wm8994_pm_ops,
+ .pm = pm_ptr(&wm8994_pm_ops),
.of_match_table = wm8994_of_match,
},
- .probe = wm8994_i2c_probe,
+ .probe_new = wm8994_i2c_probe,
.remove = wm8994_i2c_remove,
.id_table = wm8994_i2c_id,
};
this feature will allow for direct communication between SSIs
based on a network adapter and DMA messaging.
+config SMPRO_ERRMON
+ tristate "Ampere Computing SMPro error monitor driver"
+ depends on MFD_SMPRO || COMPILE_TEST
+ help
+ Say Y here to get support for the SMpro error monitor function
+ provided by Ampere Computing's Altra and Altra Max SoCs. Upon
+ loading, the driver creates sysfs files which can be use to gather
+ multiple HW error data reported via read and write system calls.
+
+ To compile this driver as a module, say M here. The driver will be
+ called smpro-errmon.
+
+config SMPRO_MISC
+ tristate "Ampere Computing SMPro miscellaneous driver"
+ depends on MFD_SMPRO || COMPILE_TEST
+ help
+ Say Y here to get support for the SMpro error miscellalenous function
+ provided by Ampere Computing's Altra and Altra Max SoCs.
+
+ To compile this driver as a module, say M here. The driver will be
+ called smpro-misc.
+
config CS5535_MFGPT
tristate "CS5535/CS5536 Geode Multi-Function General Purpose Timer (MFGPT) support"
depends on MFD_CS5535
obj-$(CONFIG_KGDB_TESTS) += kgdbts.o
obj-$(CONFIG_SGI_XP) += sgi-xp/
obj-$(CONFIG_SGI_GRU) += sgi-gru/
+obj-$(CONFIG_SMPRO_ERRMON) += smpro-errmon.o
+obj-$(CONFIG_SMPRO_MISC) += smpro-misc.o
obj-$(CONFIG_CS5535_MFGPT) += cs5535-mfgpt.o
obj-$(CONFIG_GEHC_ACHC) += gehc-achc.o
obj-$(CONFIG_HP_ILO) += hpilo.o
return ret_val;
}
-static int apds9802als_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int apds9802als_probe(struct i2c_client *client)
{
int res;
struct als_data *data;
.name = DRIVER_NAME,
.pm = APDS9802ALS_PM_OPS,
},
- .probe = apds9802als_probe,
+ .probe_new = apds9802als_probe,
.remove = apds9802als_remove,
.id_table = apds9802als_id,
};
{.attrs = sysfs_attrs_ctrl },
};
-static int apds990x_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int apds990x_probe(struct i2c_client *client)
{
struct apds990x_chip *chip;
int err;
.name = "apds990x",
.pm = &apds990x_pm_ops,
},
- .probe = apds990x_probe,
+ .probe_new = apds990x_probe,
.remove = apds990x_remove,
.id_table = apds990x_id,
};
.attrs = sysfs_attrs
};
-static int bh1770_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bh1770_probe(struct i2c_client *client)
{
struct bh1770_chip *chip;
int err;
.name = "bh1770glc",
.pm = &bh1770_pm_ops,
},
- .probe = bh1770_probe,
+ .probe_new = bh1770_probe,
.remove = bh1770_remove,
.id_table = bh1770_id,
};
};
-static char *cxl_devnode(struct device *dev, umode_t *mode)
+static char *cxl_devnode(const struct device *dev, umode_t *mode)
{
if (cpu_has_feature(CPU_FTR_HVMODE) &&
CXL_DEVT_IS_CARD(dev->devt)) {
* if it returns an error!
*/
if ((rc = cxl_register_afu(afu)))
- goto err_put1;
+ goto err_put_dev;
if ((rc = cxl_sysfs_afu_add(afu)))
- goto err_put1;
+ goto err_del_dev;
/*
* pHyp doesn't expose the programming models supported by the
afu->modes_supported = CXL_MODE_DIRECTED;
if ((rc = cxl_afu_select_best_mode(afu)))
- goto err_put2;
+ goto err_remove_sysfs;
adapter->afu[afu->slice] = afu;
return 0;
-err_put2:
+err_remove_sysfs:
cxl_sysfs_afu_remove(afu);
-err_put1:
- device_unregister(&afu->dev);
+err_del_dev:
+ device_del(&afu->dev);
+err_put_dev:
+ put_device(&afu->dev);
free = false;
guest_release_serr_irq(afu);
err2:
* even if it returns an error!
*/
if ((rc = cxl_register_adapter(adapter)))
- goto err_put1;
+ goto err_put_dev;
if ((rc = cxl_sysfs_adapter_add(adapter)))
- goto err_put1;
+ goto err_del_dev;
/* release the context lock as the adapter is configured */
cxl_adapter_context_unlock(adapter);
return adapter;
-err_put1:
- device_unregister(&adapter->dev);
+err_del_dev:
+ device_del(&adapter->dev);
+err_put_dev:
+ put_device(&adapter->dev);
free = false;
cxl_guest_remove_chardev(adapter);
err1:
rc = get_phb_index(np, phb_index);
if (rc) {
pr_err("cxl: invalid phb index\n");
+ of_node_put(np);
return rc;
}
* if it returns an error!
*/
if ((rc = cxl_register_afu(afu)))
- goto err_put1;
+ goto err_put_dev;
if ((rc = cxl_sysfs_afu_add(afu)))
- goto err_put1;
+ goto err_del_dev;
adapter->afu[afu->slice] = afu;
return 0;
-err_put1:
+err_del_dev:
+ device_del(&afu->dev);
+err_put_dev:
pci_deconfigure_afu(afu);
cxl_debugfs_afu_remove(afu);
- device_unregister(&afu->dev);
+ put_device(&afu->dev);
return rc;
err_free_native:
* even if it returns an error!
*/
if ((rc = cxl_register_adapter(adapter)))
- goto err_put1;
+ goto err_put_dev;
if ((rc = cxl_sysfs_adapter_add(adapter)))
- goto err_put1;
+ goto err_del_dev;
/* Release the context lock as adapter is configured */
cxl_adapter_context_unlock(adapter);
return adapter;
-err_put1:
+err_del_dev:
+ device_del(&adapter->dev);
+err_put_dev:
/* This should mirror cxl_remove_adapter, except without the
* sysfs parts
*/
cxl_debugfs_adapter_remove(adapter);
cxl_deconfigure_adapter(adapter);
- device_unregister(&adapter->dev);
+ put_device(&adapter->dev);
return ERR_PTR(rc);
err_release:
}
}
-static resource_size_t cxl_pci_window_alignment(struct pci_bus *bus,
- unsigned long type)
-{
- return 1;
-}
-
static void cxl_pci_reset_secondary_bus(struct pci_dev *dev)
{
/* Should we do an AFU reset here ? */
.enable_device_hook = cxl_pci_enable_device_hook,
.disable_device = cxl_pci_disable_device,
.release_device = cxl_pci_disable_device,
- .window_alignment = cxl_pci_window_alignment,
.reset_secondary_bus = cxl_pci_reset_secondary_bus,
.setup_msi_irqs = cxl_setup_msi_irqs,
.teardown_msi_irqs = cxl_teardown_msi_irqs,
/*
* Called when a ds1682 device is matched with this driver
*/
-static int ds1682_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ds1682_probe(struct i2c_client *client)
{
int rc;
.name = "ds1682",
.of_match_table = ds1682_of_match,
},
- .probe = ds1682_probe,
+ .probe_new = ds1682_probe,
.remove = ds1682_remove,
.id_table = ds1682_id,
};
return 0;
}
-static int eeprom_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int eeprom_probe(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
struct eeprom_data *data;
.driver = {
.name = "eeprom",
},
- .probe = eeprom_probe,
+ .probe_new = eeprom_probe,
.remove = eeprom_remove,
.id_table = eeprom_id,
/*
* idt_probe() - IDT 89HPESx driver probe() callback method
*/
-static int idt_probe(struct i2c_client *client, const struct i2c_device_id *id)
+static int idt_probe(struct i2c_client *client)
{
struct idt_89hpesx_dev *pdev;
int ret;
.name = IDT_NAME,
.of_match_table = idt_of_match,
},
- .probe = idt_probe,
+ .probe_new = idt_probe,
.remove = idt_remove,
.id_table = idt_ids,
};
.read = max6875_read,
};
-static int max6875_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max6875_probe(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
struct max6875_data *data;
.driver = {
.name = "max6875",
},
- .probe = max6875_probe,
+ .probe_new = max6875_probe,
.remove = max6875_remove,
.id_table = max6875_id,
};
#include <linux/slab.h>
#include <linux/qcom_scm.h>
#include <uapi/misc/fastrpc.h>
+#include <linux/of_reserved_mem.h>
#define ADSP_DOMAIN_ID (0)
#define MDSP_DOMAIN_ID (1)
#define FASTRPC_DSP_UTILITIES_HANDLE 2
#define FASTRPC_CTXID_MASK (0xFF0)
#define INIT_FILELEN_MAX (2 * 1024 * 1024)
+#define INIT_FILE_NAMELEN_MAX (128)
#define FASTRPC_DEVICE_NAME "fastrpc"
+
+/* Add memory to static PD pool, protection thru XPU */
+#define ADSP_MMAP_HEAP_ADDR 4
+/* MAP static DMA buffer on DSP User PD */
+#define ADSP_MMAP_DMA_BUFFER 6
+/* Add memory to static PD pool protection thru hypervisor */
+#define ADSP_MMAP_REMOTE_HEAP_ADDR 8
+/* Add memory to userPD pool, for user heap */
#define ADSP_MMAP_ADD_PAGES 0x1000
+/* Add memory to userPD pool, for LLC heap */
+#define ADSP_MMAP_ADD_PAGES_LLC 0x3000,
+
#define DSP_UNSUPPORTED_API (0x80000414)
/* MAX NUMBER of DSP ATTRIBUTES SUPPORTED */
#define FASTRPC_MAX_DSP_ATTRIBUTES (256)
FASTRPC_BUILD_SCALARS(0, method, in, out, 0, 0)
#define FASTRPC_CREATE_PROCESS_NARGS 6
+#define FASTRPC_CREATE_STATIC_PROCESS_NARGS 3
/* Remote Method id table */
#define FASTRPC_RMID_INIT_ATTACH 0
#define FASTRPC_RMID_INIT_RELEASE 1
#define FASTRPC_RMID_INIT_MEM_UNMAP 11
/* Protection Domain(PD) ids */
-#define AUDIO_PD (0) /* also GUEST_OS PD? */
+#define ROOT_PD (0)
#define USER_PD (1)
#define SENSORS_PD (2)
u32 dsp_attributes[FASTRPC_MAX_DSP_ATTRIBUTES];
struct fastrpc_device *secure_fdevice;
struct fastrpc_device *fdevice;
+ struct fastrpc_buf *remote_heap;
+ struct list_head invoke_interrupted_mmaps;
bool secure;
bool unsigned_support;
+ u64 dma_mask;
};
struct fastrpc_device {
kfree(buf);
}
-static int fastrpc_buf_alloc(struct fastrpc_user *fl, struct device *dev,
+static int __fastrpc_buf_alloc(struct fastrpc_user *fl, struct device *dev,
u64 size, struct fastrpc_buf **obuf)
{
struct fastrpc_buf *buf;
return -ENOMEM;
}
+ *obuf = buf;
+
+ return 0;
+}
+
+static int fastrpc_buf_alloc(struct fastrpc_user *fl, struct device *dev,
+ u64 size, struct fastrpc_buf **obuf)
+{
+ int ret;
+ struct fastrpc_buf *buf;
+
+ ret = __fastrpc_buf_alloc(fl, dev, size, obuf);
+ if (ret)
+ return ret;
+
+ buf = *obuf;
+
if (fl->sctx && fl->sctx->sid)
buf->phys += ((u64)fl->sctx->sid << 32);
- *obuf = buf;
-
return 0;
}
+static int fastrpc_remote_heap_alloc(struct fastrpc_user *fl, struct device *dev,
+ u64 size, struct fastrpc_buf **obuf)
+{
+ struct device *rdev = &fl->cctx->rpdev->dev;
+
+ return __fastrpc_buf_alloc(fl, rdev, size, obuf);
+}
+
static void fastrpc_channel_ctx_free(struct kref *ref)
{
struct fastrpc_channel_ctx *cctx;
return -ENOMEM;
INIT_LIST_HEAD(&map->node);
+ kref_init(&map->refcount);
+
map->fl = fl;
map->fd = fd;
map->buf = dma_buf_get(fd);
map->size = len;
map->va = sg_virt(map->table->sgl);
map->len = len;
- kref_init(&map->refcount);
if (attr & FASTRPC_ATTR_SECUREMAP) {
/*
attach_err:
dma_buf_put(map->buf);
get_err:
- kfree(map);
+ fastrpc_map_put(map);
return err;
}
struct fastrpc_invoke_args *args)
{
struct fastrpc_invoke_ctx *ctx = NULL;
+ struct fastrpc_buf *buf, *b;
+
int err = 0;
if (!fl->sctx)
spin_unlock(&fl->lock);
fastrpc_context_put(ctx);
}
+
+ if (err == -ERESTARTSYS) {
+ list_for_each_entry_safe(buf, b, &fl->mmaps, node) {
+ list_del(&buf->node);
+ list_add_tail(&buf->node, &fl->cctx->invoke_interrupted_mmaps);
+ }
+ }
+
if (err)
dev_dbg(fl->sctx->dev, "Error: Invoke Failed %d\n", err);
return false;
}
+static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
+ char __user *argp)
+{
+ struct fastrpc_init_create_static init;
+ struct fastrpc_invoke_args *args;
+ struct fastrpc_phy_page pages[1];
+ char *name;
+ int err;
+ struct {
+ int pgid;
+ u32 namelen;
+ u32 pageslen;
+ } inbuf;
+ u32 sc;
+
+ args = kcalloc(FASTRPC_CREATE_STATIC_PROCESS_NARGS, sizeof(*args), GFP_KERNEL);
+ if (!args)
+ return -ENOMEM;
+
+ if (copy_from_user(&init, argp, sizeof(init))) {
+ err = -EFAULT;
+ goto err;
+ }
+
+ if (init.namelen > INIT_FILE_NAMELEN_MAX) {
+ err = -EINVAL;
+ goto err;
+ }
+
+ name = kzalloc(init.namelen, GFP_KERNEL);
+ if (!name) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ if (copy_from_user(name, (void __user *)(uintptr_t)init.name, init.namelen)) {
+ err = -EFAULT;
+ goto err_name;
+ }
+
+ if (!fl->cctx->remote_heap) {
+ err = fastrpc_remote_heap_alloc(fl, fl->sctx->dev, init.memlen,
+ &fl->cctx->remote_heap);
+ if (err)
+ goto err_name;
+
+ /* Map if we have any heap VMIDs associated with this ADSP Static Process. */
+ if (fl->cctx->vmcount) {
+ unsigned int perms = BIT(QCOM_SCM_VMID_HLOS);
+
+ err = qcom_scm_assign_mem(fl->cctx->remote_heap->phys,
+ (u64)fl->cctx->remote_heap->size, &perms,
+ fl->cctx->vmperms, fl->cctx->vmcount);
+ if (err) {
+ dev_err(fl->sctx->dev, "Failed to assign memory with phys 0x%llx size 0x%llx err %d",
+ fl->cctx->remote_heap->phys, fl->cctx->remote_heap->size, err);
+ goto err_map;
+ }
+ }
+ }
+
+ inbuf.pgid = fl->tgid;
+ inbuf.namelen = init.namelen;
+ inbuf.pageslen = 0;
+ fl->pd = USER_PD;
+
+ args[0].ptr = (u64)(uintptr_t)&inbuf;
+ args[0].length = sizeof(inbuf);
+ args[0].fd = -1;
+
+ args[1].ptr = (u64)(uintptr_t)name;
+ args[1].length = inbuf.namelen;
+ args[1].fd = -1;
+
+ pages[0].addr = fl->cctx->remote_heap->phys;
+ pages[0].size = fl->cctx->remote_heap->size;
+
+ args[2].ptr = (u64)(uintptr_t) pages;
+ args[2].length = sizeof(*pages);
+ args[2].fd = -1;
+
+ sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_CREATE_STATIC, 3, 0);
+
+ err = fastrpc_internal_invoke(fl, true, FASTRPC_INIT_HANDLE,
+ sc, args);
+ if (err)
+ goto err_invoke;
+
+ kfree(args);
+
+ return 0;
+err_invoke:
+ if (fl->cctx->vmcount) {
+ struct qcom_scm_vmperm perm;
+
+ perm.vmid = QCOM_SCM_VMID_HLOS;
+ perm.perm = QCOM_SCM_PERM_RWX;
+ err = qcom_scm_assign_mem(fl->cctx->remote_heap->phys,
+ (u64)fl->cctx->remote_heap->size,
+ &(fl->cctx->vmperms[0].vmid), &perm, 1);
+ if (err)
+ dev_err(fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d",
+ fl->cctx->remote_heap->phys, fl->cctx->remote_heap->size, err);
+ }
+err_map:
+ fastrpc_buf_free(fl->cctx->remote_heap);
+err_name:
+ kfree(name);
+err:
+ kfree(args);
+
+ return err;
+}
+
static int fastrpc_init_create_process(struct fastrpc_user *fl,
char __user *argp)
{
return 0;
}
-static int fastrpc_req_munmap_impl(struct fastrpc_user *fl,
- struct fastrpc_req_munmap *req)
+static int fastrpc_req_munmap_impl(struct fastrpc_user *fl, struct fastrpc_buf *buf)
{
struct fastrpc_invoke_args args[1] = { [0] = { 0 } };
- struct fastrpc_buf *buf = NULL, *iter, *b;
struct fastrpc_munmap_req_msg req_msg;
struct device *dev = fl->sctx->dev;
int err;
u32 sc;
- spin_lock(&fl->lock);
- list_for_each_entry_safe(iter, b, &fl->mmaps, node) {
- if ((iter->raddr == req->vaddrout) && (iter->size == req->size)) {
- buf = iter;
- break;
- }
- }
- spin_unlock(&fl->lock);
-
- if (!buf) {
- dev_err(dev, "mmap not in list\n");
- return -EINVAL;
- }
-
req_msg.pgid = fl->tgid;
req_msg.size = buf->size;
req_msg.vaddr = buf->raddr;
static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp)
{
+ struct fastrpc_buf *buf = NULL, *iter, *b;
struct fastrpc_req_munmap req;
+ struct device *dev = fl->sctx->dev;
if (copy_from_user(&req, argp, sizeof(req)))
return -EFAULT;
- return fastrpc_req_munmap_impl(fl, &req);
+ spin_lock(&fl->lock);
+ list_for_each_entry_safe(iter, b, &fl->mmaps, node) {
+ if ((iter->raddr == req.vaddrout) && (iter->size == req.size)) {
+ buf = iter;
+ break;
+ }
+ }
+ spin_unlock(&fl->lock);
+
+ if (!buf) {
+ dev_err(dev, "mmap\t\tpt 0x%09llx [len 0x%08llx] not in list\n",
+ req.vaddrout, req.size);
+ return -EINVAL;
+ }
+
+ return fastrpc_req_munmap_impl(fl, buf);
}
static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp)
struct fastrpc_buf *buf = NULL;
struct fastrpc_mmap_req_msg req_msg;
struct fastrpc_mmap_rsp_msg rsp_msg;
- struct fastrpc_req_munmap req_unmap;
struct fastrpc_phy_page pages;
struct fastrpc_req_mmap req;
struct device *dev = fl->sctx->dev;
if (copy_from_user(&req, argp, sizeof(req)))
return -EFAULT;
- if (req.flags != ADSP_MMAP_ADD_PAGES) {
+ if (req.flags != ADSP_MMAP_ADD_PAGES && req.flags != ADSP_MMAP_REMOTE_HEAP_ADDR) {
dev_err(dev, "flag not supported 0x%x\n", req.flags);
+
return -EINVAL;
}
/* let the client know the address to use */
req.vaddrout = rsp_msg.vaddr;
+ /* Add memory to static PD pool, protection thru hypervisor */
+ if (req.flags != ADSP_MMAP_REMOTE_HEAP_ADDR && fl->cctx->vmcount) {
+ struct qcom_scm_vmperm perm;
+ int err = 0;
+
+ perm.vmid = QCOM_SCM_VMID_HLOS;
+ perm.perm = QCOM_SCM_PERM_RWX;
+ err = qcom_scm_assign_mem(buf->phys, buf->size,
+ &(fl->cctx->vmperms[0].vmid), &perm, 1);
+ if (err) {
+ dev_err(fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d",
+ buf->phys, buf->size, err);
+ goto err_assign;
+ }
+ }
+
spin_lock(&fl->lock);
list_add_tail(&buf->node, &fl->mmaps);
spin_unlock(&fl->lock);
if (copy_to_user((void __user *)argp, &req, sizeof(req))) {
- /* unmap the memory and release the buffer */
- req_unmap.vaddrout = buf->raddr;
- req_unmap.size = buf->size;
- fastrpc_req_munmap_impl(fl, &req_unmap);
- return -EFAULT;
+ err = -EFAULT;
+ goto err_assign;
}
dev_dbg(dev, "mmap\t\tpt 0x%09lx OK [len 0x%08llx]\n",
return 0;
+err_assign:
+ fastrpc_req_munmap_impl(fl, buf);
err_invoke:
fastrpc_buf_free(buf);
err = fastrpc_invoke(fl, argp);
break;
case FASTRPC_IOCTL_INIT_ATTACH:
- err = fastrpc_init_attach(fl, AUDIO_PD);
+ err = fastrpc_init_attach(fl, ROOT_PD);
break;
case FASTRPC_IOCTL_INIT_ATTACH_SNS:
err = fastrpc_init_attach(fl, SENSORS_PD);
break;
+ case FASTRPC_IOCTL_INIT_CREATE_STATIC:
+ err = fastrpc_init_create_static_process(fl, argp);
+ break;
case FASTRPC_IOCTL_INIT_CREATE:
err = fastrpc_init_create_process(fl, argp);
break;
return -EINVAL;
}
+ if (of_reserved_mem_device_init_by_idx(rdev, rdev->of_node, 0))
+ dev_info(rdev, "no reserved DMA memory for FASTRPC\n");
+
vmcount = of_property_read_variable_u32_array(rdev->of_node,
"qcom,vmids", &vmids[0], 0, FASTRPC_MAX_VMIDS);
if (vmcount < 0)
kref_init(&data->refcount);
dev_set_drvdata(&rpdev->dev, data);
+ rdev->dma_mask = &data->dma_mask;
dma_set_mask_and_coherent(rdev, DMA_BIT_MASK(32));
INIT_LIST_HEAD(&data->users);
+ INIT_LIST_HEAD(&data->invoke_interrupted_mmaps);
spin_lock_init(&data->lock);
idr_init(&data->ctx_idr);
data->domain_id = domain_id;
static void fastrpc_rpmsg_remove(struct rpmsg_device *rpdev)
{
struct fastrpc_channel_ctx *cctx = dev_get_drvdata(&rpdev->dev);
+ struct fastrpc_buf *buf, *b;
struct fastrpc_user *user;
unsigned long flags;
if (cctx->secure_fdevice)
misc_deregister(&cctx->secure_fdevice->miscdev);
+ list_for_each_entry_safe(buf, b, &cctx->invoke_interrupted_mmaps, node)
+ list_del(&buf->node);
+
+ if (cctx->remote_heap)
+ fastrpc_buf_free(cctx->remote_heap);
+
of_platform_depopulate(&rpdev->dev);
cctx->rpdev = NULL;
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* IBM Accelerator Family 'GenWQE'
*
* (C) Copyright IBM Corp. 2013
* Default mode should be rw for everybody. Do not change default
* device name.
*/
-static char *genwqe_devnode(struct device *dev, umode_t *mode)
+static char *genwqe_devnode(const struct device *dev, umode_t *mode)
{
if (mode)
*mode = 0666;
*/
if (hl_cs_cmpl->encaps_signals)
kref_put(&hl_cs_cmpl->encaps_sig_hdl->refcount,
- hl_encaps_handle_do_release);
+ hl_encaps_release_handle_and_put_ctx);
}
- if ((cs->type == CS_TYPE_WAIT || cs->type == CS_TYPE_COLLECTIVE_WAIT)
- && cs->encaps_signals)
- kref_put(&cs->encaps_sig_hdl->refcount,
- hl_encaps_handle_do_release);
+ if ((cs->type == CS_TYPE_WAIT || cs->type == CS_TYPE_COLLECTIVE_WAIT) && cs->encaps_signals)
+ kref_put(&cs->encaps_sig_hdl->refcount, hl_encaps_release_handle_and_put_ctx);
out:
/* Must be called before hl_ctx_put because inside we use ctx to get
static void cs_timedout(struct work_struct *work)
{
struct hl_device *hdev;
- u64 event_mask;
+ u64 event_mask = 0x0;
int rc;
struct hl_cs *cs = container_of(work, struct hl_cs,
work_tdr.work);
if (rc) {
hdev->captured_err_info.cs_timeout.timestamp = ktime_get();
hdev->captured_err_info.cs_timeout.seq = cs->sequence;
-
- event_mask = device_reset ? (HL_NOTIFIER_EVENT_CS_TIMEOUT |
- HL_NOTIFIER_EVENT_DEVICE_RESET) : HL_NOTIFIER_EVENT_CS_TIMEOUT;
-
- hl_notifier_event_send_all(hdev, event_mask);
+ event_mask |= HL_NOTIFIER_EVENT_CS_TIMEOUT;
}
switch (cs->type) {
cs_put(cs);
- if (device_reset)
- hl_device_reset(hdev, HL_DRV_RESET_TDR);
+ if (device_reset) {
+ event_mask |= HL_NOTIFIER_EVENT_DEVICE_RESET;
+ hl_device_cond_reset(hdev, HL_DRV_RESET_TDR, event_mask);
+ } else if (event_mask) {
+ hl_notifier_event_send_all(hdev, event_mask);
+ }
}
static int allocate_cs(struct hl_device *hdev, struct hl_ctx *ctx,
hl_complete_job(hdev, job);
}
+/*
+ * release_reserved_encaps_signals() - release reserved encapsulated signals.
+ * @hdev: pointer to habanalabs device structure
+ *
+ * Release reserved encapsulated signals which weren't un-reserved, or for which a CS with
+ * encapsulated signals wasn't submitted and thus weren't released as part of CS roll-back.
+ * For these signals need also to put the refcount of the H/W SOB which was taken at the
+ * reservation.
+ */
+static void release_reserved_encaps_signals(struct hl_device *hdev)
+{
+ struct hl_ctx *ctx = hl_get_compute_ctx(hdev);
+ struct hl_cs_encaps_sig_handle *handle;
+ struct hl_encaps_signals_mgr *mgr;
+ u32 id;
+
+ if (!ctx)
+ return;
+
+ mgr = &ctx->sig_mgr;
+
+ idr_for_each_entry(&mgr->handles, handle, id)
+ if (handle->cs_seq == ULLONG_MAX)
+ kref_put(&handle->refcount, hl_encaps_release_handle_and_put_sob_ctx);
+
+ hl_ctx_put(ctx);
+}
+
void hl_cs_rollback_all(struct hl_device *hdev, bool skip_wq_flush)
{
int i;
}
force_complete_multi_cs(hdev);
+
+ release_reserved_encaps_signals(hdev);
}
static void
*/
handle->pre_sob_val = prop->next_sob_val - handle->count;
+ handle->cs_seq = ULLONG_MAX;
+
*signals_count = prop->next_sob_val;
hdev->asic_funcs->hw_queues_unlock(hdev);
/* We finished with the CS in this function, so put the ref */
cs_put(cs);
free_cs_chunk_array:
- if (!wait_cs_submitted && cs_encaps_signals && handle_found &&
- is_wait_cs)
- kref_put(&encaps_sig_hdl->refcount,
- hl_encaps_handle_do_release);
+ if (!wait_cs_submitted && cs_encaps_signals && handle_found && is_wait_cs)
+ kref_put(&encaps_sig_hdl->refcount, hl_encaps_release_handle_and_put_ctx);
kfree(cs_chunk_array);
out:
return rc;
#include <linux/slab.h>
-void hl_encaps_handle_do_release(struct kref *ref)
+static void encaps_handle_do_release(struct hl_cs_encaps_sig_handle *handle, bool put_hw_sob,
+ bool put_ctx)
{
- struct hl_cs_encaps_sig_handle *handle =
- container_of(ref, struct hl_cs_encaps_sig_handle, refcount);
struct hl_encaps_signals_mgr *mgr = &handle->ctx->sig_mgr;
+ if (put_hw_sob)
+ hw_sob_put(handle->hw_sob);
+
spin_lock(&mgr->lock);
idr_remove(&mgr->handles, handle->id);
spin_unlock(&mgr->lock);
- hl_ctx_put(handle->ctx);
+ if (put_ctx)
+ hl_ctx_put(handle->ctx);
+
kfree(handle);
}
-static void hl_encaps_handle_do_release_sob(struct kref *ref)
+void hl_encaps_release_handle_and_put_ctx(struct kref *ref)
{
struct hl_cs_encaps_sig_handle *handle =
- container_of(ref, struct hl_cs_encaps_sig_handle, refcount);
- struct hl_encaps_signals_mgr *mgr = &handle->ctx->sig_mgr;
+ container_of(ref, struct hl_cs_encaps_sig_handle, refcount);
- /* if we're here, then there was a signals reservation but cs with
- * encaps signals wasn't submitted, so need to put refcount
- * to hw_sob taken at the reservation.
- */
- hw_sob_put(handle->hw_sob);
+ encaps_handle_do_release(handle, false, true);
+}
- spin_lock(&mgr->lock);
- idr_remove(&mgr->handles, handle->id);
- spin_unlock(&mgr->lock);
+static void hl_encaps_release_handle_and_put_sob(struct kref *ref)
+{
+ struct hl_cs_encaps_sig_handle *handle =
+ container_of(ref, struct hl_cs_encaps_sig_handle, refcount);
- hl_ctx_put(handle->ctx);
- kfree(handle);
+ encaps_handle_do_release(handle, true, false);
+}
+
+void hl_encaps_release_handle_and_put_sob_ctx(struct kref *ref)
+{
+ struct hl_cs_encaps_sig_handle *handle =
+ container_of(ref, struct hl_cs_encaps_sig_handle, refcount);
+
+ encaps_handle_do_release(handle, true, true);
}
static void hl_encaps_sig_mgr_init(struct hl_encaps_signals_mgr *mgr)
idr_init(&mgr->handles);
}
-static void hl_encaps_sig_mgr_fini(struct hl_device *hdev,
- struct hl_encaps_signals_mgr *mgr)
+static void hl_encaps_sig_mgr_fini(struct hl_device *hdev, struct hl_encaps_signals_mgr *mgr)
{
struct hl_cs_encaps_sig_handle *handle;
struct idr *idp;
idp = &mgr->handles;
+ /* The IDR is expected to be empty at this stage, because any left signal should have been
+ * released as part of CS roll-back.
+ */
if (!idr_is_empty(idp)) {
- dev_warn(hdev->dev, "device released while some encaps signals handles are still allocated\n");
+ dev_warn(hdev->dev,
+ "device released while some encaps signals handles are still allocated\n");
idr_for_each_entry(idp, handle, id)
- kref_put(&handle->refcount,
- hl_encaps_handle_do_release_sob);
+ kref_put(&handle->refcount, hl_encaps_release_handle_and_put_sob);
}
idr_destroy(&mgr->handles);
dev_entry,
&hl_timeout_locked_fops);
+ debugfs_create_u32("device_release_watchdog_timeout",
+ 0644,
+ dev_entry->root,
+ &hdev->device_release_watchdog_timeout_sec);
+
for (i = 0, entry = dev_entry->entry_arr ; i < count ; i++, entry++) {
debugfs_create_file(hl_debugfs_list[i].name,
0444,
#include <linux/pci.h>
#include <linux/hwmon.h>
+#include <linux/vmalloc.h>
#include <trace/events/habanalabs.h>
-#define HL_RESET_DELAY_USEC 10000 /* 10ms */
+#define HL_RESET_DELAY_USEC 10000 /* 10ms */
+
+#define HL_DEVICE_RELEASE_WATCHDOG_TIMEOUT_SEC 5
enum dma_alloc_type {
DMA_ALLOC_COHERENT,
* @hdev: pointer to habanalabs device structure.
* @addr: the address the caller wants to access.
* @region: the PCI region.
+ * @new_bar_region_base: the new BAR region base address.
*
* @return: the old BAR base address on success, U64_MAX for failure.
* The caller should set it back to the old address after use.
* This function can be called also if the bar doesn't need to be set,
* in that case it just won't change the base.
*/
-static u64 hl_set_dram_bar(struct hl_device *hdev, u64 addr, struct pci_mem_region *region)
+static u64 hl_set_dram_bar(struct hl_device *hdev, u64 addr, struct pci_mem_region *region,
+ u64 *new_bar_region_base)
{
struct asic_fixed_properties *prop = &hdev->asic_prop;
u64 bar_base_addr, old_base;
old_base = hdev->asic_funcs->set_dram_bar_base(hdev, bar_base_addr);
/* in case of success we need to update the new BAR base */
- if (old_base != U64_MAX)
- region->region_base = bar_base_addr;
+ if ((old_base != U64_MAX) && new_bar_region_base)
+ *new_bar_region_base = bar_base_addr;
return old_base;
}
-static int hl_access_sram_dram_region(struct hl_device *hdev, u64 addr, u64 *val,
- enum debugfs_access_type acc_type, enum pci_region region_type)
+int hl_access_sram_dram_region(struct hl_device *hdev, u64 addr, u64 *val,
+ enum debugfs_access_type acc_type, enum pci_region region_type, bool set_dram_bar)
{
struct pci_mem_region *region = &hdev->pci_mem_region[region_type];
+ u64 old_base = 0, rc, bar_region_base = region->region_base;
void __iomem *acc_addr;
- u64 old_base = 0, rc;
- if (region_type == PCI_REGION_DRAM) {
- old_base = hl_set_dram_bar(hdev, addr, region);
+ if (set_dram_bar) {
+ old_base = hl_set_dram_bar(hdev, addr, region, &bar_region_base);
if (old_base == U64_MAX)
return -EIO;
}
- acc_addr = hdev->pcie_bar[region->bar_id] + addr - region->region_base +
- region->offset_in_bar;
+ acc_addr = hdev->pcie_bar[region->bar_id] + region->offset_in_bar +
+ (addr - bar_region_base);
+
switch (acc_type) {
case DEBUGFS_READ8:
*val = readb(acc_addr);
break;
}
- if (region_type == PCI_REGION_DRAM) {
- rc = hl_set_dram_bar(hdev, old_base, region);
+ if (set_dram_bar) {
+ rc = hl_set_dram_bar(hdev, old_base, region, NULL);
if (rc == U64_MAX)
return -EIO;
}
dma_addr_t dma_handle, enum dma_alloc_type alloc_type,
const char *caller)
{
+ /* this is needed to avoid warning on using freed pointer */
+ u64 store_cpu_addr = (u64) (uintptr_t) cpu_addr;
+
switch (alloc_type) {
case DMA_ALLOC_COHERENT:
hdev->asic_funcs->asic_dma_free_coherent(hdev, size, cpu_addr, dma_handle);
break;
}
- trace_habanalabs_dma_free(hdev->dev, (u64) (uintptr_t) cpu_addr, dma_handle, size, caller);
+ trace_habanalabs_dma_free(hdev->dev, store_cpu_addr, dma_handle, size, caller);
}
void *hl_asic_dma_alloc_coherent_caller(struct hl_device *hdev, size_t size, dma_addr_t *dma_handle,
case PCI_REGION_SRAM:
case PCI_REGION_DRAM:
return hl_access_sram_dram_region(hdev, addr, val, acc_type,
- region_type);
+ region_type, (region_type == PCI_REGION_DRAM));
default:
return -EFAULT;
}
}
}
+bool hl_ctrl_device_operational(struct hl_device *hdev,
+ enum hl_device_status *status)
+{
+ enum hl_device_status current_status;
+
+ current_status = hl_device_status(hdev);
+ if (status)
+ *status = current_status;
+
+ switch (current_status) {
+ case HL_DEVICE_STATUS_MALFUNCTION:
+ return false;
+ case HL_DEVICE_STATUS_IN_RESET:
+ case HL_DEVICE_STATUS_IN_RESET_AFTER_DEVICE_RELEASE:
+ case HL_DEVICE_STATUS_NEEDS_RESET:
+ case HL_DEVICE_STATUS_OPERATIONAL:
+ case HL_DEVICE_STATUS_IN_DEVICE_CREATION:
+ default:
+ return true;
+ }
+}
+
+static void print_idle_status_mask(struct hl_device *hdev, const char *message,
+ u64 idle_mask[HL_BUSY_ENGINES_MASK_EXT_SIZE])
+{
+ u32 pad_width[HL_BUSY_ENGINES_MASK_EXT_SIZE] = {};
+
+ BUILD_BUG_ON(HL_BUSY_ENGINES_MASK_EXT_SIZE != 4);
+
+ pad_width[3] = idle_mask[3] ? 16 : 0;
+ pad_width[2] = idle_mask[2] || pad_width[3] ? 16 : 0;
+ pad_width[1] = idle_mask[1] || pad_width[2] ? 16 : 0;
+ pad_width[0] = idle_mask[0] || pad_width[1] ? 16 : 0;
+
+ dev_err(hdev->dev, "%s (mask %0*llx_%0*llx_%0*llx_%0*llx)\n",
+ message, pad_width[3], idle_mask[3], pad_width[2], idle_mask[2],
+ pad_width[1], idle_mask[1], pad_width[0], idle_mask[0]);
+}
+
static void hpriv_release(struct kref *ref)
{
u64 idle_mask[HL_BUSY_ENGINES_MASK_EXT_SIZE] = {0};
- bool device_is_idle = true;
+ bool reset_device, device_is_idle = true;
struct hl_fpriv *hpriv;
struct hl_device *hdev;
mutex_destroy(&hpriv->ctx_lock);
mutex_destroy(&hpriv->restore_phase_mutex);
- if ((!hdev->pldm) && (hdev->pdev) &&
- (!hdev->asic_funcs->is_device_idle(hdev,
- idle_mask,
- HL_BUSY_ENGINES_MASK_EXT_SIZE, NULL))) {
- dev_err(hdev->dev,
- "device not idle after user context is closed (0x%llx_%llx)\n",
- idle_mask[1], idle_mask[0]);
+ /* Device should be reset if reset-upon-device-release is enabled, or if there is a pending
+ * reset that waits for device release.
+ */
+ reset_device = hdev->reset_upon_device_release || hdev->reset_info.watchdog_active;
- device_is_idle = false;
+ /* Unless device is reset in any case, check idle status and reset if device is not idle */
+ if (!reset_device && hdev->pdev && !hdev->pldm)
+ device_is_idle = hdev->asic_funcs->is_device_idle(hdev, idle_mask,
+ HL_BUSY_ENGINES_MASK_EXT_SIZE, NULL);
+ if (!device_is_idle) {
+ print_idle_status_mask(hdev, "device is not idle after user context is closed",
+ idle_mask);
+ reset_device = true;
}
/* We need to remove the user from the list to make sure the reset process won't
list_del(&hpriv->dev_node);
mutex_unlock(&hdev->fpriv_list_lock);
- if (!device_is_idle || hdev->reset_upon_device_release) {
+ if (reset_device) {
hl_device_reset(hdev, HL_DRV_RESET_DEV_RELEASE);
} else {
+ /* Scrubbing is handled within hl_device_reset(), so here need to do it directly */
int rc = hdev->asic_funcs->scrub_device_mem(hdev);
if (rc)
hdev->compute_ctx_in_release = 1;
- if (!hl_hpriv_put(hpriv))
- dev_notice(hdev->dev,
- "User process closed FD but device still in use\n");
+ if (!hl_hpriv_put(hpriv)) {
+ dev_notice(hdev->dev, "User process closed FD but device still in use\n");
+ hl_device_reset(hdev, HL_DRV_RESET_HARD);
+ }
hdev->last_open_session_duration_jif =
jiffies - hdev->last_successful_open_jif;
flags = device_reset_work->flags | HL_DRV_RESET_FROM_RESET_THR;
rc = hl_device_reset(hdev, flags);
+
if ((rc == -EBUSY) && !hdev->device_fini_pending) {
- dev_info(hdev->dev,
- "Could not reset device. will try again in %u seconds",
- HL_PENDING_RESET_PER_SEC);
+ struct hl_ctx *ctx = hl_get_compute_ctx(hdev);
- queue_delayed_work(device_reset_work->wq,
- &device_reset_work->reset_work,
- msecs_to_jiffies(HL_PENDING_RESET_PER_SEC * 1000));
+ if (ctx) {
+ /* The read refcount value should subtracted by one, because the read is
+ * protected with hl_get_compute_ctx().
+ */
+ dev_info(hdev->dev,
+ "Could not reset device (compute_ctx refcount %u). will try again in %u seconds",
+ kref_read(&ctx->refcount) - 1, HL_PENDING_RESET_PER_SEC);
+ hl_ctx_put(ctx);
+ } else {
+ dev_info(hdev->dev, "Could not reset device. will try again in %u seconds",
+ HL_PENDING_RESET_PER_SEC);
+ }
+
+ queue_delayed_work(hdev->reset_wq, &device_reset_work->reset_work,
+ msecs_to_jiffies(HL_PENDING_RESET_PER_SEC * 1000));
}
}
+static void device_release_watchdog_func(struct work_struct *work)
+{
+ struct hl_device_reset_work *device_release_watchdog_work =
+ container_of(work, struct hl_device_reset_work, reset_work.work);
+ struct hl_device *hdev = device_release_watchdog_work->hdev;
+ u32 flags;
+
+ dev_dbg(hdev->dev, "Device wasn't released in time. Initiate device reset.\n");
+
+ flags = device_release_watchdog_work->flags | HL_DRV_RESET_FROM_WD_THR;
+
+ hl_device_reset(hdev, flags);
+}
+
/*
* device_early_init - do some early initialization for the habanalabs device
*
gaudi2_set_asic_funcs(hdev);
strscpy(hdev->asic_name, "GAUDI2", sizeof(hdev->asic_name));
break;
- case ASIC_GAUDI2_SEC:
+ case ASIC_GAUDI2B:
gaudi2_set_asic_funcs(hdev);
- strscpy(hdev->asic_name, "GAUDI2 SEC", sizeof(hdev->asic_name));
+ strscpy(hdev->asic_name, "GAUDI2B", sizeof(hdev->asic_name));
+ break;
break;
default:
dev_err(hdev->dev, "Unrecognized ASIC type %d\n",
}
}
- hdev->eq_wq = alloc_workqueue("hl-events", WQ_UNBOUND, 0);
+ hdev->eq_wq = create_singlethread_workqueue("hl-events");
if (hdev->eq_wq == NULL) {
dev_err(hdev->dev, "Failed to allocate EQ workqueue\n");
rc = -ENOMEM;
goto free_cs_cmplt_wq;
}
- hdev->pf_wq = alloc_workqueue("hl-prefetch", WQ_UNBOUND, 0);
- if (!hdev->pf_wq) {
+ hdev->prefetch_wq = alloc_workqueue("hl-prefetch", WQ_UNBOUND, 0);
+ if (!hdev->prefetch_wq) {
dev_err(hdev->dev, "Failed to allocate MMU prefetch workqueue\n");
rc = -ENOMEM;
goto free_ts_free_wq;
GFP_KERNEL);
if (!hdev->hl_chip_info) {
rc = -ENOMEM;
- goto free_pf_wq;
+ goto free_prefetch_wq;
}
rc = hl_mmu_if_set_funcs(hdev);
hl_mem_mgr_init(hdev->dev, &hdev->kernel_mem_mgr);
- hdev->device_reset_work.wq =
- create_singlethread_workqueue("hl_device_reset");
- if (!hdev->device_reset_work.wq) {
+ hdev->reset_wq = create_singlethread_workqueue("hl_device_reset");
+ if (!hdev->reset_wq) {
rc = -ENOMEM;
dev_err(hdev->dev, "Failed to create device reset WQ\n");
goto free_cb_mgr;
}
- INIT_DELAYED_WORK(&hdev->device_reset_work.reset_work,
- device_hard_reset_pending);
+ INIT_DELAYED_WORK(&hdev->device_reset_work.reset_work, device_hard_reset_pending);
hdev->device_reset_work.hdev = hdev;
hdev->device_fini_pending = 0;
+ INIT_DELAYED_WORK(&hdev->device_release_watchdog_work.reset_work,
+ device_release_watchdog_func);
+ hdev->device_release_watchdog_work.hdev = hdev;
+
mutex_init(&hdev->send_cpu_message_lock);
mutex_init(&hdev->debug_lock);
INIT_LIST_HEAD(&hdev->cs_mirror_list);
hl_mem_mgr_fini(&hdev->kernel_mem_mgr);
free_chip_info:
kfree(hdev->hl_chip_info);
-free_pf_wq:
- destroy_workqueue(hdev->pf_wq);
+free_prefetch_wq:
+ destroy_workqueue(hdev->prefetch_wq);
free_ts_free_wq:
destroy_workqueue(hdev->ts_free_obj_wq);
free_cs_cmplt_wq:
kfree(hdev->hl_chip_info);
- destroy_workqueue(hdev->pf_wq);
+ destroy_workqueue(hdev->prefetch_wq);
destroy_workqueue(hdev->ts_free_obj_wq);
destroy_workqueue(hdev->cs_cmplt_wq);
destroy_workqueue(hdev->eq_wq);
- destroy_workqueue(hdev->device_reset_work.wq);
+ destroy_workqueue(hdev->reset_wq);
for (i = 0 ; i < hdev->asic_prop.completion_queues_count ; i++)
destroy_workqueue(hdev->cq_wq[i]);
int hl_device_utilization(struct hl_device *hdev, u32 *utilization)
{
- u64 max_power, curr_power, dc_power, dividend;
+ u64 max_power, curr_power, dc_power, dividend, divisor;
int rc;
max_power = hdev->max_power;
dc_power = hdev->asic_prop.dc_power_default;
+ divisor = max_power - dc_power;
+ if (!divisor) {
+ dev_warn(hdev->dev, "device utilization is not supported\n");
+ return -EOPNOTSUPP;
+ }
rc = hl_fw_cpucp_power_get(hdev, &curr_power);
if (rc)
curr_power = clamp(curr_power, dc_power, max_power);
dividend = (curr_power - dc_power) * 100;
- *utilization = (u32) div_u64(dividend, (max_power - dc_power));
+ *utilization = (u32) div_u64(dividend, divisor);
return 0;
}
hl_cs_rollback_all(hdev, skip_wq_flush);
/* flush the MMU prefetch workqueue */
- flush_workqueue(hdev->pf_wq);
+ flush_workqueue(hdev->prefetch_wq);
/* Release all pending user interrupts, each pending user interrupt
* holds a reference to user context
{
u32 cur_reset_trigger = HL_RESET_TRIGGER_DEFAULT;
+ /* No consecutive mechanism when user context exists */
+ if (hdev->is_compute_ctx_active)
+ return;
+
/*
* 'reset cause' is being updated here, because getting here
* means that it's the 1st time and the last time we're here
int hl_device_reset(struct hl_device *hdev, u32 flags)
{
bool hard_reset, from_hard_reset_thread, fw_reset, hard_instead_soft = false,
- reset_upon_device_release = false, schedule_hard_reset = false,
- skip_wq_flush, delay_reset;
+ reset_upon_device_release = false, schedule_hard_reset = false, delay_reset,
+ from_dev_release, from_watchdog_thread;
u64 idle_mask[HL_BUSY_ENGINES_MASK_EXT_SIZE] = {0};
struct hl_ctx *ctx;
int i, rc;
hard_reset = !!(flags & HL_DRV_RESET_HARD);
from_hard_reset_thread = !!(flags & HL_DRV_RESET_FROM_RESET_THR);
fw_reset = !!(flags & HL_DRV_RESET_BYPASS_REQ_TO_FW);
- skip_wq_flush = !!(flags & HL_DRV_RESET_DEV_RELEASE);
+ from_dev_release = !!(flags & HL_DRV_RESET_DEV_RELEASE);
delay_reset = !!(flags & HL_DRV_RESET_DELAY);
+ from_watchdog_thread = !!(flags & HL_DRV_RESET_FROM_WD_THR);
if (!hard_reset && !hdev->asic_prop.supports_compute_reset) {
hard_instead_soft = true;
spin_unlock(&hdev->reset_info.lock);
+ /* Cancel the device release watchdog work if required.
+ * In case of reset-upon-device-release while the release watchdog work is
+ * scheduled, do hard-reset instead of compute-reset.
+ */
+ if ((hard_reset || from_dev_release) && hdev->reset_info.watchdog_active) {
+ hdev->reset_info.watchdog_active = 0;
+ if (!from_watchdog_thread)
+ cancel_delayed_work_sync(
+ &hdev->device_release_watchdog_work.reset_work);
+
+ if (from_dev_release) {
+ flags |= HL_DRV_RESET_HARD;
+ flags &= ~HL_DRV_RESET_DEV_RELEASE;
+ hard_reset = true;
+ }
+ }
+
if (delay_reset)
usleep_range(HL_RESET_DELAY_USEC, HL_RESET_DELAY_USEC << 1);
* Because the reset function can't run from heartbeat work,
* we need to call the reset function from a dedicated work.
*/
- queue_delayed_work(hdev->device_reset_work.wq,
- &hdev->device_reset_work.reset_work, 0);
+ queue_delayed_work(hdev->reset_wq, &hdev->device_reset_work.reset_work, 0);
return 0;
}
- cleanup_resources(hdev, hard_reset, fw_reset, skip_wq_flush);
+ cleanup_resources(hdev, hard_reset, fw_reset, from_dev_release);
kill_processes:
if (hard_reset) {
/* If device is not idle fail the reset process */
if (!hdev->asic_funcs->is_device_idle(hdev, idle_mask,
- HL_BUSY_ENGINES_MASK_EXT_SIZE, NULL)) {
- dev_err(hdev->dev, "device is not idle (mask 0x%llx_%llx) after reset\n",
- idle_mask[1], idle_mask[0]);
+ HL_BUSY_ENGINES_MASK_EXT_SIZE, NULL)) {
+ print_idle_status_mask(hdev, "device is not idle after reset", idle_mask);
rc = -EIO;
goto out_err;
}
* the device will be operational although it shouldn't be
*/
hdev->asic_funcs->enable_events_from_fw(hdev);
- } else if (!reset_upon_device_release) {
- hdev->reset_info.compute_reset_cnt++;
- }
-
- if (schedule_hard_reset) {
- dev_info(hdev->dev, "Performing hard reset scheduled during compute reset\n");
- flags = hdev->reset_info.hard_reset_schedule_flags;
- hdev->reset_info.hard_reset_schedule_flags = 0;
- hdev->disabled = true;
- hard_reset = true;
- handle_reset_trigger(hdev, flags);
- goto again;
+ } else {
+ if (!reset_upon_device_release)
+ hdev->reset_info.compute_reset_cnt++;
+
+ if (schedule_hard_reset) {
+ dev_info(hdev->dev, "Performing hard reset scheduled during compute reset\n");
+ flags = hdev->reset_info.hard_reset_schedule_flags;
+ hdev->reset_info.hard_reset_schedule_flags = 0;
+ hdev->disabled = true;
+ hard_reset = true;
+ handle_reset_trigger(hdev, flags);
+ goto again;
+ }
}
return 0;
return rc;
}
+/*
+ * hl_device_cond_reset() - conditionally reset the device.
+ * @hdev: pointer to habanalabs device structure.
+ * @reset_flags: reset flags.
+ * @event_mask: events to notify user about.
+ *
+ * Conditionally reset the device, or alternatively schedule a watchdog work to reset the device
+ * unless another reset precedes it.
+ */
+int hl_device_cond_reset(struct hl_device *hdev, u32 flags, u64 event_mask)
+{
+ struct hl_ctx *ctx = NULL;
+
+ /* Device release watchdog is only for hard reset */
+ if (!(flags & HL_DRV_RESET_HARD) && hdev->asic_prop.allow_inference_soft_reset)
+ goto device_reset;
+
+ /* F/W reset cannot be postponed */
+ if (flags & HL_DRV_RESET_BYPASS_REQ_TO_FW)
+ goto device_reset;
+
+ /* Device release watchdog is relevant only if user exists and gets a reset notification */
+ if (!(event_mask & HL_NOTIFIER_EVENT_DEVICE_RESET)) {
+ dev_err(hdev->dev, "Resetting device without a reset indication to user\n");
+ goto device_reset;
+ }
+
+ ctx = hl_get_compute_ctx(hdev);
+ if (!ctx || !ctx->hpriv->notifier_event.eventfd)
+ goto device_reset;
+
+ /* Schedule the device release watchdog work unless reset is already in progress or if the
+ * work is already scheduled.
+ */
+ spin_lock(&hdev->reset_info.lock);
+ if (hdev->reset_info.in_reset) {
+ spin_unlock(&hdev->reset_info.lock);
+ goto device_reset;
+ }
+
+ if (hdev->reset_info.watchdog_active)
+ goto out;
+
+ hdev->device_release_watchdog_work.flags = flags;
+ dev_dbg(hdev->dev, "Device is going to be reset in %u sec unless being released\n",
+ hdev->device_release_watchdog_timeout_sec);
+ schedule_delayed_work(&hdev->device_release_watchdog_work.reset_work,
+ msecs_to_jiffies(hdev->device_release_watchdog_timeout_sec * 1000));
+ hdev->reset_info.watchdog_active = 1;
+out:
+ spin_unlock(&hdev->reset_info.lock);
+
+ hl_notifier_event_send_all(hdev, event_mask);
+
+ hl_ctx_put(ctx);
+
+ return 0;
+
+device_reset:
+ if (event_mask)
+ hl_notifier_event_send_all(hdev, event_mask);
+ if (ctx)
+ hl_ctx_put(ctx);
+
+ return hl_device_reset(hdev, flags);
+}
+
static void hl_notifier_event_send(struct hl_notifier_event *notifier_event, u64 event_mask)
{
mutex_lock(¬ifier_event->lock);
{
struct hl_fpriv *hpriv;
+ if (!event_mask) {
+ dev_warn(hdev->dev, "Skip sending zero event");
+ return;
+ }
+
mutex_lock(&hdev->fpriv_list_lock);
list_for_each_entry(hpriv, &hdev->fpriv_list, dev_node)
hdev->asic_funcs->state_dump_init(hdev);
+ hdev->device_release_watchdog_timeout_sec = HL_DEVICE_RELEASE_WATCHDOG_TIMEOUT_SEC;
+
hdev->memory_scrub_val = MEM_SCRUB_DEFAULT_VAL;
hl_debugfs_add_device(hdev);
}
}
+ cancel_delayed_work_sync(&hdev->device_release_watchdog_work.reset_work);
+
/* Disable PCI access from device F/W so it won't send us additional
* interrupts. We disable MSI/MSI-X at the halt_engines function and we
* can't have the F/W sending us interrupts after that. We need to
*/
dev_info(hdev->dev,
"Waiting for all processes to exit (timeout of %u seconds)",
- HL_PENDING_RESET_LONG_SEC);
+ HL_WAIT_PROCESS_KILL_ON_DEVICE_FINI);
- rc = device_kill_open_processes(hdev, HL_PENDING_RESET_LONG_SEC, false);
+ hdev->process_kill_trial_cnt = 0;
+ rc = device_kill_open_processes(hdev, HL_WAIT_PROCESS_KILL_ON_DEVICE_FINI, false);
if (rc) {
dev_crit(hdev->dev, "Failed to kill all open processes\n");
device_disable_open_processes(hdev, false);
}
+ hdev->process_kill_trial_cnt = 0;
rc = device_kill_open_processes(hdev, 0, true);
if (rc) {
dev_crit(hdev->dev, "Failed to kill all control device open processes\n");
hl_mmu_fini(hdev);
+ vfree(hdev->captured_err_info.pgf_info.user_mappings);
+
hl_eq_fini(hdev, &hdev->event_queue);
kfree(hdev->shadow_cs_queue);
{
writel(val, hdev->rmmio + reg);
}
+
+void hl_capture_razwi(struct hl_device *hdev, u64 addr, u16 *engine_id, u16 num_of_engines,
+ u8 flags)
+{
+ if (num_of_engines > HL_RAZWI_MAX_NUM_OF_ENGINES_PER_RTR) {
+ dev_err(hdev->dev,
+ "Number of possible razwi initiators (%u) exceeded limit (%u)\n",
+ num_of_engines, HL_RAZWI_MAX_NUM_OF_ENGINES_PER_RTR);
+ return;
+ }
+
+ /* In case it's the first razwi since the device was opened, capture its parameters */
+ if (atomic_cmpxchg(&hdev->captured_err_info.razwi_info_recorded, 0, 1))
+ return;
+
+ hdev->captured_err_info.razwi.timestamp = ktime_to_ns(ktime_get());
+ hdev->captured_err_info.razwi.addr = addr;
+ hdev->captured_err_info.razwi.num_of_possible_engines = num_of_engines;
+ memcpy(&hdev->captured_err_info.razwi.engine_id[0], &engine_id[0],
+ num_of_engines * sizeof(u16));
+ hdev->captured_err_info.razwi.flags = flags;
+}
+
+void hl_handle_razwi(struct hl_device *hdev, u64 addr, u16 *engine_id, u16 num_of_engines,
+ u8 flags, u64 *event_mask)
+{
+ hl_capture_razwi(hdev, addr, engine_id, num_of_engines, flags);
+
+ if (event_mask)
+ *event_mask |= HL_NOTIFIER_EVENT_RAZWI;
+}
+
+static void hl_capture_user_mappings(struct hl_device *hdev, bool is_pmmu)
+{
+ struct page_fault_info *pgf_info = &hdev->captured_err_info.pgf_info;
+ struct hl_vm_phys_pg_pack *phys_pg_pack = NULL;
+ struct hl_vm_hash_node *hnode;
+ struct hl_userptr *userptr;
+ enum vm_type *vm_type;
+ struct hl_ctx *ctx;
+ u32 map_idx = 0;
+ int i;
+
+ /* Reset previous session count*/
+ pgf_info->num_of_user_mappings = 0;
+
+ ctx = hl_get_compute_ctx(hdev);
+ if (!ctx) {
+ dev_err(hdev->dev, "Can't get user context for user mappings\n");
+ return;
+ }
+
+ mutex_lock(&ctx->mem_hash_lock);
+ hash_for_each(ctx->mem_hash, i, hnode, node) {
+ vm_type = hnode->ptr;
+ if (((*vm_type == VM_TYPE_USERPTR) && is_pmmu) ||
+ ((*vm_type == VM_TYPE_PHYS_PACK) && !is_pmmu))
+ pgf_info->num_of_user_mappings++;
+
+ }
+
+ if (!pgf_info->num_of_user_mappings)
+ goto finish;
+
+ /* In case we already allocated in previous session, need to release it before
+ * allocating new buffer.
+ */
+ vfree(pgf_info->user_mappings);
+ pgf_info->user_mappings =
+ vzalloc(pgf_info->num_of_user_mappings * sizeof(struct hl_user_mapping));
+ if (!pgf_info->user_mappings) {
+ pgf_info->num_of_user_mappings = 0;
+ goto finish;
+ }
+
+ hash_for_each(ctx->mem_hash, i, hnode, node) {
+ vm_type = hnode->ptr;
+ if ((*vm_type == VM_TYPE_USERPTR) && (is_pmmu)) {
+ userptr = hnode->ptr;
+ pgf_info->user_mappings[map_idx].dev_va = hnode->vaddr;
+ pgf_info->user_mappings[map_idx].size = userptr->size;
+ map_idx++;
+ } else if ((*vm_type == VM_TYPE_PHYS_PACK) && (!is_pmmu)) {
+ phys_pg_pack = hnode->ptr;
+ pgf_info->user_mappings[map_idx].dev_va = hnode->vaddr;
+ pgf_info->user_mappings[map_idx].size = phys_pg_pack->total_size;
+ map_idx++;
+ }
+ }
+finish:
+ mutex_unlock(&ctx->mem_hash_lock);
+ hl_ctx_put(ctx);
+}
+
+void hl_capture_page_fault(struct hl_device *hdev, u64 addr, u16 eng_id, bool is_pmmu)
+{
+ /* Capture only the first page fault */
+ if (atomic_cmpxchg(&hdev->captured_err_info.pgf_info_recorded, 0, 1))
+ return;
+
+ hdev->captured_err_info.pgf_info.pgf.timestamp = ktime_to_ns(ktime_get());
+ hdev->captured_err_info.pgf_info.pgf.addr = addr;
+ hdev->captured_err_info.pgf_info.pgf.engine_id = eng_id;
+ hl_capture_user_mappings(hdev, is_pmmu);
+}
+
+void hl_handle_page_fault(struct hl_device *hdev, u64 addr, u16 eng_id, bool is_pmmu,
+ u64 *event_mask)
+{
+ hl_capture_page_fault(hdev, addr, eng_id, is_pmmu);
+
+ if (event_mask)
+ *event_mask |= HL_NOTIFIER_EVENT_PAGE_FAULT;
+}
#include <linux/crc32.h>
#include <linux/slab.h>
#include <linux/ctype.h>
+#include <linux/vmalloc.h>
#define FW_FILE_MAX_SIZE 0x1400000 /* maximum size of 20MB */
if (!prop->supports_advanced_cpucp_rc) {
dev_dbg(hdev->dev, "F/W ERROR %d for CPU packet %d\n", rc, opcode);
+ rc = -EIO;
goto scrub_descriptor;
}
if (sts_val & CPU_BOOT_DEV_STS0_ENABLED)
dev_dbg(hdev->dev, "Device status0 %#x\n", sts_val);
- /* All warnings should go here in order not to reach the unknown error validation */
if (err_val & CPU_BOOT_ERR0_EEPROM_FAIL) {
- dev_warn(hdev->dev,
- "Device boot warning - EEPROM failure detected, default settings applied\n");
- /* This is a warning so we don't want it to disable the
- * device
- */
- err_val &= ~CPU_BOOT_ERR0_EEPROM_FAIL;
+ dev_err(hdev->dev, "Device boot error - EEPROM failure detected\n");
+ err_exists = true;
}
+ /* All warnings should go here in order not to reach the unknown error validation */
if (err_val & CPU_BOOT_ERR0_DRAM_SKIPPED) {
dev_warn(hdev->dev,
"Device boot warning - Skipped DRAM initialization\n");
/* first send clear command to clean former commands */
rc = hl_fw_dynamic_send_clear_cmd(hdev, fw_loader);
+ if (rc)
+ return rc;
/* send the actual command */
hl_fw_dynamic_send_cmd(hdev, fw_loader, cmd, size);
struct fw_load_mgr *fw_loader)
{
struct lkd_fw_comms_desc *fw_desc;
+ void __iomem *src, *temp_fw_desc;
struct pci_mem_region *region;
struct fw_response *response;
+ u16 fw_data_size;
enum pci_region region_id;
- void __iomem *src;
int rc;
fw_desc = &fw_loader->dynamic_loader.comm_desc;
fw_loader->dynamic_loader.fw_desc_valid = false;
src = hdev->pcie_bar[region->bar_id] + region->offset_in_bar +
response->ram_offset;
+
+ /*
+ * We do the copy of the fw descriptor in 2 phases:
+ * 1. copy the header + data info according to our lkd_fw_comms_desc definition.
+ * then we're able to read the actual data size provided by fw.
+ * this is needed for cases where data in descriptor was changed(add/remove)
+ * in embedded specs header file before updating lkd copy of the header file
+ * 2. copy descriptor to temporary buffer with aligned size and send it to validation
+ */
memcpy_fromio(fw_desc, src, sizeof(struct lkd_fw_comms_desc));
+ fw_data_size = le16_to_cpu(fw_desc->header.size);
+
+ temp_fw_desc = vzalloc(sizeof(struct comms_desc_header) + fw_data_size);
+ if (!temp_fw_desc)
+ return -ENOMEM;
- return hl_fw_dynamic_validate_descriptor(hdev, fw_loader, fw_desc);
+ memcpy_fromio(temp_fw_desc, src, sizeof(struct comms_desc_header) + fw_data_size);
+
+ rc = hl_fw_dynamic_validate_descriptor(hdev, fw_loader,
+ (struct lkd_fw_comms_desc *) temp_fw_desc);
+ vfree(temp_fw_desc);
+
+ return rc;
}
/**
struct fw_load_mgr *fw_loader)
{
struct cpu_dyn_regs *dyn_regs;
- int rc;
+ int rc, fw_error_rc;
dev_info(hdev->dev,
"Loading %sfirmware to device, may take some time...\n",
hl_fw_dynamic_update_linux_interrupt_if(hdev);
- return 0;
-
protocol_err:
- if (fw_loader->dynamic_loader.fw_desc_valid)
- fw_read_errors(hdev, le32_to_cpu(dyn_regs->cpu_boot_err0),
+ if (fw_loader->dynamic_loader.fw_desc_valid) {
+ fw_error_rc = fw_read_errors(hdev, le32_to_cpu(dyn_regs->cpu_boot_err0),
le32_to_cpu(dyn_regs->cpu_boot_err1),
le32_to_cpu(dyn_regs->cpu_boot_dev_sts0),
le32_to_cpu(dyn_regs->cpu_boot_dev_sts1));
+
+ if (fw_error_rc)
+ return fw_error_rc;
+ }
+
return rc;
}
int rc;
req_cpu_addr = hl_cpu_accessible_dma_pool_alloc(hdev, size, &req_dma_addr);
- if (!data) {
+ if (!req_cpu_addr) {
dev_err(hdev->dev,
"Failed to allocate DMA memory for CPU-CP packet %u\n", packet_id);
return -ENOMEM;
#define HL_MMAP_OFFSET_VALUE_MASK (0x1FFFFFFFFFFFull >> PAGE_SHIFT)
#define HL_MMAP_OFFSET_VALUE_GET(off) (off & HL_MMAP_OFFSET_VALUE_MASK)
-#define HL_PENDING_RESET_PER_SEC 10
-#define HL_PENDING_RESET_MAX_TRIALS 60 /* 10 minutes */
-#define HL_PENDING_RESET_LONG_SEC 60
+#define HL_PENDING_RESET_PER_SEC 10
+#define HL_PENDING_RESET_MAX_TRIALS 60 /* 10 minutes */
+#define HL_PENDING_RESET_LONG_SEC 60
+/*
+ * In device fini, wait 10 minutes for user processes to be terminated after we kill them.
+ * This is needed to prevent situation of clearing resources while user processes are still alive.
+ */
+#define HL_WAIT_PROCESS_KILL_ON_DEVICE_FINI 600
#define HL_HARD_RESET_MAX_TIMEOUT 120
#define HL_PLDM_HARD_RESET_MAX_TIMEOUT (HL_HARD_RESET_MAX_TIMEOUT * 3)
*
* - HL_DRV_RESET_DELAY
* Set if a delay should be added before the reset
+ *
+ * - HL_DRV_RESET_FROM_WD_THR
+ * Set if the caller is the device release watchdog thread
*/
#define HL_DRV_RESET_HARD (1 << 0)
#define HL_DRV_RESET_BYPASS_REQ_TO_FW (1 << 5)
#define HL_DRV_RESET_FW_FATAL_ERR (1 << 6)
#define HL_DRV_RESET_DELAY (1 << 7)
+#define HL_DRV_RESET_FROM_WD_THR (1 << 8)
/*
* Security
* @ASIC_GAUDI: Gaudi device (HL-2000).
* @ASIC_GAUDI_SEC: Gaudi secured device (HL-2000).
* @ASIC_GAUDI2: Gaudi2 device.
- * @ASIC_GAUDI2_SEC: Gaudi2 secured device.
+ * @ASIC_GAUDI2B: Gaudi2B device.
*/
enum hl_asic_type {
ASIC_INVALID,
ASIC_GAUDI,
ASIC_GAUDI_SEC,
ASIC_GAUDI2,
- ASIC_GAUDI2_SEC,
+ ASIC_GAUDI2B,
};
struct hl_cs_parser;
#define WREG32_AND(reg, and) WREG32_P(reg, 0, and)
#define WREG32_OR(reg, or) WREG32_P(reg, or, ~(or))
-#define RMWREG32(reg, val, mask) \
- do { \
- u32 tmp_ = RREG32(reg); \
- tmp_ &= ~(mask); \
- tmp_ |= ((val) << __ffs(mask)); \
- WREG32(reg, tmp_); \
- } while (0)
+#define RMWREG32_SHIFTED(reg, val, mask) WREG32_P(reg, val, ~(mask))
+
+#define RMWREG32(reg, val, mask) RMWREG32_SHIFTED(reg, (val) << __ffs(mask), mask)
#define RREG32_MASK(reg, mask) ((RREG32(reg) & mask) >> __ffs(mask))
break; \
(val) = __elbi_read; \
} else {\
- (val) = RREG32((u32)(addr)); \
+ (val) = RREG32(lower_32_bits(addr)); \
} \
if (cond) \
break; \
break; \
(val) = __elbi_read; \
} else {\
- (val) = RREG32((u32)(addr)); \
+ (val) = RREG32(lower_32_bits(addr)); \
} \
break; \
} \
if (__rc) \
break; \
} else { \
- __read_val = RREG32((u32)(addr_arr)[__arr_idx]); \
+ __read_val = RREG32(lower_32_bits(addr_arr[__arr_idx])); \
} \
if (__read_val == (expected_val)) \
__elem_bitmask &= ~BIT_ULL(__arr_idx); \
struct hwmon_chip_info;
/**
- * struct hl_device_reset_work - reset workqueue task wrapper.
- * @wq: work queue for device reset procedure.
+ * struct hl_device_reset_work - reset work wrapper.
* @reset_work: reset work to be done.
* @hdev: habanalabs device structure.
* @flags: reset flags.
*/
struct hl_device_reset_work {
- struct workqueue_struct *wq;
- struct delayed_work reset_work;
- struct hl_device *hdev;
- u32 flags;
+ struct delayed_work reset_work;
+ struct hl_device *hdev;
+ u32 flags;
};
/**
/**
* struct hl_prefetch_work - prefetch work structure handler
- * @pf_work: actual work struct.
+ * @prefetch_work: actual work struct.
* @ctx: compute context.
* @va: virtual address to pre-fetch.
* @size: pre-fetch size.
* @asid: ASID for maintenance operation.
*/
struct hl_prefetch_work {
- struct work_struct pf_work;
+ struct work_struct prefetch_work;
struct hl_ctx *ctx;
u64 va;
u64 size;
u64 seq;
};
-/**
- * struct razwi_info - info about last razwi error occurred.
- * @timestamp: razwi timestamp.
- * @write_enable: if set writing to razwi parameters in the structure is enabled.
- * otherwise - disabled, so the first (root cause) razwi will not be overwritten.
- * @addr: address that caused razwi.
- * @engine_id_1: engine id of the razwi initiator, if it was initiated by engine that does
- * not have engine id it will be set to U16_MAX.
- * @engine_id_2: second engine id of razwi initiator. Might happen that razwi have 2 possible
- * engines which one them caused the razwi. In that case, it will contain the
- * second possible engine id, otherwise it will be set to U16_MAX.
- * @non_engine_initiator: in case the initiator of the razwi does not have engine id.
- * @type: cause of razwi, page fault or access error, otherwise it will be set to U8_MAX.
- */
-struct razwi_info {
- ktime_t timestamp;
- atomic_t write_enable;
- u64 addr;
- u16 engine_id_1;
- u16 engine_id_2;
- u8 non_engine_initiator;
- u8 type;
-};
-
#define MAX_QMAN_STREAMS_INFO 4
#define OPCODE_INFO_MAX_ADDR_SIZE 8
/**
};
/**
+ * struct page_fault_info - info about page fault
+ * @pgf_info: page fault information.
+ * @user_mappings: buffer containing user mappings.
+ * @num_of_user_mappings: number of user mappings.
+ */
+struct page_fault_info {
+ struct hl_page_fault_info pgf;
+ struct hl_user_mapping *user_mappings;
+ u64 num_of_user_mappings;
+};
+
+/**
* struct hl_error_info - holds information collected during an error.
* @cs_timeout: CS timeout error information.
* @razwi: razwi information.
+ * @razwi_info_recorded: if set writing to razwi information is enabled.
+ * otherwise - disabled, so the first (root cause) razwi will not be
+ * overwritten.
* @undef_opcode: undefined opcode information
+ * @pgf_info: page fault information.
+ * @pgf_info_recorded: if set writing to page fault information is enabled.
+ * otherwise - disabled, so the first (root cause) page fault will not be
+ * overwritten.
*/
struct hl_error_info {
struct cs_timeout_info cs_timeout;
- struct razwi_info razwi;
+ struct hl_info_razwi_event razwi;
+ atomic_t razwi_info_recorded;
struct undefined_opcode_info undef_opcode;
+ struct page_fault_info pgf_info;
+ atomic_t pgf_info_recorded;
};
/**
* same cause.
* @skip_reset_on_timeout: Skip device reset if CS has timed out, wait for it to
* complete instead.
+ * @watchdog_active: true if a device release watchdog work is scheduled.
*/
struct hl_reset_info {
spinlock_t lock;
u8 in_compute_reset;
u8 needs_reset;
u8 hard_reset_pending;
-
u8 curr_reset_cause;
u8 prev_reset_trigger;
u8 reset_trigger_repeated;
-
u8 skip_reset_on_timeout;
+ u8 watchdog_active;
};
/**
* @dev_ctrl: related kernel device structure for the control device
* @work_heartbeat: delayed work for CPU-CP is-alive check.
* @device_reset_work: delayed work which performs hard reset
+ * @device_release_watchdog_work: watchdog work that performs hard reset if user doesn't release
+ * device upon certain error cases.
* @asic_name: ASIC specific name.
* @asic_type: ASIC specific type.
* @completion_queue: array of hl_cq.
* @cs_cmplt_wq: work queue of CS completions for executing work in process
* context.
* @ts_free_obj_wq: work queue for timestamp registration objects release.
- * @pf_wq: work queue for MMU pre-fetch operations.
+ * @prefetch_wq: work queue for MMU pre-fetch operations.
+ * @reset_wq: work queue for device reset procedure.
* @kernel_ctx: Kernel driver context structure.
* @kernel_queues: array of hl_hw_queue.
* @cs_mirror_list: CS mirror list for TDR.
* indicates which decoder engines are binned-out
* @edma_binning: contains mask of edma engines that is received from the f/w which
* indicates which edma engines are binned-out
+ * @device_release_watchdog_timeout_sec: device release watchdog timeout value in seconds.
* @id: device minor.
* @id_control: minor of the control device.
* @cdev_idx: char device index. Used for setting its name.
struct device *dev_ctrl;
struct delayed_work work_heartbeat;
struct hl_device_reset_work device_reset_work;
+ struct hl_device_reset_work device_release_watchdog_work;
char asic_name[HL_STR_MAX];
char status[HL_DEV_STS_MAX][HL_STR_MAX];
enum hl_asic_type asic_type;
struct workqueue_struct *eq_wq;
struct workqueue_struct *cs_cmplt_wq;
struct workqueue_struct *ts_free_obj_wq;
- struct workqueue_struct *pf_wq;
+ struct workqueue_struct *prefetch_wq;
+ struct workqueue_struct *reset_wq;
struct hl_ctx *kernel_ctx;
struct hl_hw_queue *kernel_queues;
struct list_head cs_mirror_list;
u32 high_pll;
u32 decoder_binning;
u32 edma_binning;
+ u32 device_release_watchdog_timeout_sec;
u16 id;
u16 id_control;
u16 cdev_idx;
int hl_dma_map_sgtable(struct hl_device *hdev, struct sg_table *sgt, enum dma_data_direction dir);
void hl_dma_unmap_sgtable(struct hl_device *hdev, struct sg_table *sgt,
enum dma_data_direction dir);
+int hl_access_sram_dram_region(struct hl_device *hdev, u64 addr, u64 *val,
+ enum debugfs_access_type acc_type, enum pci_region region_type, bool set_dram_bar);
int hl_access_cfg_region(struct hl_device *hdev, u64 addr, u64 *val,
enum debugfs_access_type acc_type);
int hl_access_dev_mem(struct hl_device *hdev, enum pci_region region_type,
int hl_device_open_ctrl(struct inode *inode, struct file *filp);
bool hl_device_operational(struct hl_device *hdev,
enum hl_device_status *status);
+bool hl_ctrl_device_operational(struct hl_device *hdev,
+ enum hl_device_status *status);
enum hl_device_status hl_device_status(struct hl_device *hdev);
int hl_device_set_debug_mode(struct hl_device *hdev, struct hl_ctx *ctx, bool enable);
int hl_hw_queues_create(struct hl_device *hdev);
int hl_device_suspend(struct hl_device *hdev);
int hl_device_resume(struct hl_device *hdev);
int hl_device_reset(struct hl_device *hdev, u32 flags);
+int hl_device_cond_reset(struct hl_device *hdev, u32 flags, u64 event_mask);
void hl_hpriv_get(struct hl_fpriv *hpriv);
int hl_hpriv_put(struct hl_fpriv *hpriv);
int hl_device_utilization(struct hl_device *hdev, u32 *utilization);
void hw_sob_get(struct hl_hw_sob *hw_sob);
void hw_sob_put(struct hl_hw_sob *hw_sob);
-void hl_encaps_handle_do_release(struct kref *ref);
+void hl_encaps_release_handle_and_put_ctx(struct kref *ref);
+void hl_encaps_release_handle_and_put_sob_ctx(struct kref *ref);
void hl_hw_queue_encaps_sig_set_sob_info(struct hl_device *hdev,
struct hl_cs *cs, struct hl_cs_job *job,
struct hl_cs_compl *cs_cmpl);
struct hl_mmap_mem_buf_behavior *behavior, gfp_t gfp,
void *args);
__printf(2, 3) void hl_engine_data_sprintf(struct engines_data *e, const char *fmt, ...);
+void hl_capture_razwi(struct hl_device *hdev, u64 addr, u16 *engine_id, u16 num_of_engines,
+ u8 flags);
+void hl_handle_razwi(struct hl_device *hdev, u64 addr, u16 *engine_id, u16 num_of_engines,
+ u8 flags, u64 *event_mask);
+void hl_capture_page_fault(struct hl_device *hdev, u64 addr, u16 eng_id, bool is_pmmu);
+void hl_handle_page_fault(struct hl_device *hdev, u64 addr, u16 eng_id, bool is_pmmu,
+ u64 *event_mask);
#ifdef CONFIG_DEBUG_FS
#define pr_fmt(fmt) "habanalabs: " fmt
#include "habanalabs.h"
+#include "../include/hw_ip/pci/pci_general.h"
#include <linux/pci.h>
#include <linux/aer.h>
/*
* get_asic_type - translate device id to asic type
*
- * @device: id of the PCI device
+ * @hdev: pointer to habanalabs device structure.
*
- * Translate device id to asic type.
+ * Translate device id and revision id to asic type.
* In case of unidentified device, return -1
*/
-static enum hl_asic_type get_asic_type(u16 device)
+static enum hl_asic_type get_asic_type(struct hl_device *hdev)
{
- enum hl_asic_type asic_type;
+ struct pci_dev *pdev = hdev->pdev;
+ enum hl_asic_type asic_type = ASIC_INVALID;
- switch (device) {
+ switch (pdev->device) {
case PCI_IDS_GOYA:
asic_type = ASIC_GOYA;
break;
asic_type = ASIC_GAUDI_SEC;
break;
case PCI_IDS_GAUDI2:
- asic_type = ASIC_GAUDI2;
+ switch (pdev->revision) {
+ case REV_ID_A:
+ asic_type = ASIC_GAUDI2;
+ break;
+ case REV_ID_B:
+ asic_type = ASIC_GAUDI2B;
+ break;
+ default:
+ break;
+ }
break;
default:
- asic_type = ASIC_INVALID;
break;
}
hl_debugfs_add_file(hpriv);
atomic_set(&hdev->captured_err_info.cs_timeout.write_enable, 1);
- atomic_set(&hdev->captured_err_info.razwi.write_enable, 1);
+ atomic_set(&hdev->captured_err_info.razwi_info_recorded, 0);
+ atomic_set(&hdev->captured_err_info.pgf_info_recorded, 0);
hdev->captured_err_info.undef_opcode.write_enable = true;
hdev->open_counter++;
mutex_lock(&hdev->fpriv_ctrl_list_lock);
- if (!hl_device_operational(hdev, NULL)) {
+ if (!hl_ctrl_device_operational(hdev, NULL)) {
dev_dbg_ratelimited(hdev->dev_ctrl,
- "Can't open %s because it is disabled or in reset\n",
+ "Can't open %s because it is disabled\n",
dev_name(hdev->dev_ctrl));
rc = -EPERM;
goto out_err;
/* First, we must find out which ASIC are we handling. This is needed
* to configure the behavior of the driver (kernel parameters)
*/
- hdev->asic_type = get_asic_type(pdev->device);
+ hdev->asic_type = get_asic_type(hdev);
if (hdev->asic_type == ASIC_INVALID) {
dev_err(&pdev->dev, "Unsupported ASIC\n");
rc = -ENODEV;
switch (state) {
case pci_channel_io_normal:
+ dev_warn(hdev->dev, "PCI normal state error detected\n");
return PCI_ERS_RESULT_CAN_RECOVER;
case pci_channel_io_frozen:
- dev_warn(hdev->dev, "frozen state error detected\n");
+ dev_warn(hdev->dev, "PCI frozen state error detected\n");
result = PCI_ERS_RESULT_NEED_RESET;
break;
case pci_channel_io_perm_failure:
- dev_warn(hdev->dev, "failure state error detected\n");
+ dev_warn(hdev->dev, "PCI failure state error detected\n");
result = PCI_ERS_RESULT_DISCONNECT;
break;
*/
static pci_ers_result_t hl_pci_err_slot_reset(struct pci_dev *pdev)
{
+ struct hl_device *hdev = pci_get_drvdata(pdev);
+
+ dev_warn(hdev->dev, "PCI slot reset detected\n");
+
return PCI_ERS_RESULT_RECOVERED;
}
#include <uapi/misc/habanalabs.h>
#include "habanalabs.h"
-#include <linux/kernel.h>
#include <linux/fs.h>
-#include <linux/uaccess.h>
+#include <linux/kernel.h>
+#include <linux/pci.h>
#include <linux/slab.h>
+#include <linux/uaccess.h>
#include <linux/vmalloc.h>
static u32 hl_debug_struct_size[HL_DEBUG_OP_TIMESTAMP + 1] = {
hw_ip.edma_enabled_mask = prop->edma_enabled_mask;
hw_ip.server_type = prop->server_type;
hw_ip.security_enabled = prop->fw_security_enabled;
+ hw_ip.revision_id = hdev->pdev->revision;
return copy_to_user(out, &hw_ip,
min((size_t) size, sizeof(hw_ip))) ? -EFAULT : 0;
return -EINVAL;
arr = hdev->asic_funcs->get_events_stat(hdev, aggregate, &size);
+ if (!arr) {
+ dev_err(hdev->dev, "Events info not supported\n");
+ return -EOPNOTSUPP;
+ }
return copy_to_user(out, arr, min(max_size, size)) ? -EFAULT : 0;
}
{
struct hl_device *hdev = hpriv->hdev;
u32 max_size = args->return_size;
- struct hl_info_razwi_event info = {0};
+ struct hl_info_razwi_event *info = &hdev->captured_err_info.razwi;
void __user *out = (void __user *) (uintptr_t) args->return_pointer;
if ((!max_size) || (!out))
return -EINVAL;
- info.timestamp = ktime_to_ns(hdev->captured_err_info.razwi.timestamp);
- info.addr = hdev->captured_err_info.razwi.addr;
- info.engine_id_1 = hdev->captured_err_info.razwi.engine_id_1;
- info.engine_id_2 = hdev->captured_err_info.razwi.engine_id_2;
- info.no_engine_id = hdev->captured_err_info.razwi.non_engine_initiator;
- info.error_type = hdev->captured_err_info.razwi.type;
-
- return copy_to_user(out, &info, min_t(size_t, max_size, sizeof(info))) ? -EFAULT : 0;
+ return copy_to_user(out, info, min_t(size_t, max_size, sizeof(struct hl_info_razwi_event)))
+ ? -EFAULT : 0;
}
static int undefined_opcode_info(struct hl_fpriv *hpriv, struct hl_info_args *args)
return rc;
}
+static int page_fault_info(struct hl_fpriv *hpriv, struct hl_info_args *args)
+{
+ struct hl_device *hdev = hpriv->hdev;
+ u32 max_size = args->return_size;
+ struct hl_page_fault_info *info = &hdev->captured_err_info.pgf_info.pgf;
+ void __user *out = (void __user *) (uintptr_t) args->return_pointer;
+
+ if ((!max_size) || (!out))
+ return -EINVAL;
+
+ return copy_to_user(out, info, min_t(size_t, max_size, sizeof(struct hl_page_fault_info)))
+ ? -EFAULT : 0;
+}
+
+static int user_mappings_info(struct hl_fpriv *hpriv, struct hl_info_args *args)
+{
+ void __user *out = (void __user *) (uintptr_t) args->return_pointer;
+ u32 user_buf_size = args->return_size;
+ struct hl_device *hdev = hpriv->hdev;
+ struct page_fault_info *pgf_info;
+ u64 actual_size;
+
+ pgf_info = &hdev->captured_err_info.pgf_info;
+ args->array_size = pgf_info->num_of_user_mappings;
+
+ if (!out)
+ return -EINVAL;
+
+ actual_size = pgf_info->num_of_user_mappings * sizeof(struct hl_user_mapping);
+ if (user_buf_size < actual_size)
+ return -ENOMEM;
+
+ return copy_to_user(out, pgf_info->user_mappings, min_t(size_t, user_buf_size, actual_size))
+ ? -EFAULT : 0;
+}
+
static int _hl_info_ioctl(struct hl_fpriv *hpriv, void *data,
struct device *dev)
{
case HL_INFO_GET_EVENTS:
return events_info(hpriv, args);
+ case HL_INFO_PAGE_FAULT_EVENT:
+ return page_fault_info(hpriv, args);
+
+ case HL_INFO_USER_MAPPINGS:
+ return user_mappings_info(hpriv, args);
+
+ case HL_INFO_UNREGISTER_EVENTFD:
+ return eventfd_unregister(hpriv, args);
+
default:
break;
}
case HL_INFO_REGISTER_EVENTFD:
return eventfd_register(hpriv, args);
- case HL_INFO_UNREGISTER_EVENTFD:
- return eventfd_unregister(hpriv, args);
-
case HL_INFO_ENGINE_STATUS:
return engine_status_info(hpriv, args);
hl_dmabuf = dmabuf->priv;
hdev = hl_dmabuf->ctx->hdev;
- rc = pci_p2pdma_distance_many(hdev->pdev, &attachment->dev, 1, true);
+ rc = pci_p2pdma_distance(hdev->pdev, attachment->dev, true);
if (rc < 0)
attachment->peer2peer = false;
/* Allocate the internal kernel buffer */
size = num_elements * sizeof(struct hl_user_pending_interrupt);
- p = vmalloc(size);
+ p = vzalloc(size);
if (!p)
goto free_user_buff;
/*
* PAGE_SIZE alignment
- * it is the callers responsibility to align the addresses if the
+ * it is the caller's responsibility to align the addresses if the
* page size is not a power of 2
*/
if (is_power_of_2(page_size)) {
- if (start & (PAGE_SIZE - 1)) {
- start &= PAGE_MASK;
- start += PAGE_SIZE;
- }
+ start = round_up(start, page_size);
/*
* The end of the range is inclusive, hence we need to align it
* to the end of the last full page in the range. For example if
* end = 0x3ff5 with page size 0x1000, we need to align it to
- * 0x2fff. The remainig 0xff5 bytes do not form a full page.
+ * 0x2fff. The remaining 0xff5 bytes do not form a full page.
*/
- if ((end + 1) & (PAGE_SIZE - 1))
- end = ((end + 1) & PAGE_MASK) - 1;
+ end = round_down(end + 1, page_size) - 1;
}
if (start >= end) {
hl_mmu_v1_set_funcs(hdev, &hdev->mmu_func[MMU_DR_PGT]);
break;
case ASIC_GAUDI2:
- case ASIC_GAUDI2_SEC:
+ case ASIC_GAUDI2B:
/* MMUs in Gaudi2 are always host resident */
hl_mmu_v2_hr_set_funcs(hdev, &hdev->mmu_func[MMU_HR_PGT]);
break;
static void hl_mmu_prefetch_work_function(struct work_struct *work)
{
- struct hl_prefetch_work *pfw = container_of(work, struct hl_prefetch_work, pf_work);
+ struct hl_prefetch_work *pfw = container_of(work, struct hl_prefetch_work, prefetch_work);
struct hl_ctx *ctx = pfw->ctx;
struct hl_device *hdev = ctx->hdev;
int hl_mmu_prefetch_cache_range(struct hl_ctx *ctx, u32 flags, u32 asid, u64 va, u64 size)
{
- struct hl_prefetch_work *handle_pf_work;
+ struct hl_prefetch_work *handle_prefetch_work;
- handle_pf_work = kmalloc(sizeof(*handle_pf_work), GFP_KERNEL);
- if (!handle_pf_work)
+ handle_prefetch_work = kmalloc(sizeof(*handle_prefetch_work), GFP_KERNEL);
+ if (!handle_prefetch_work)
return -ENOMEM;
- INIT_WORK(&handle_pf_work->pf_work, hl_mmu_prefetch_work_function);
- handle_pf_work->ctx = ctx;
- handle_pf_work->va = va;
- handle_pf_work->size = size;
- handle_pf_work->flags = flags;
- handle_pf_work->asid = asid;
+ INIT_WORK(&handle_prefetch_work->prefetch_work, hl_mmu_prefetch_work_function);
+ handle_prefetch_work->ctx = ctx;
+ handle_prefetch_work->va = va;
+ handle_prefetch_work->size = size;
+ handle_prefetch_work->flags = flags;
+ handle_prefetch_work->asid = asid;
/*
* as actual prefetch is done in a WQ we must get the context (and put it
* at the end of the work function)
*/
hl_ctx_get(ctx);
- queue_work(ctx->hdev->pf_wq, &handle_pf_work->pf_work);
+ queue_work(ctx->hdev->prefetch_wq, &handle_prefetch_work->prefetch_work);
return 0;
}
case ASIC_GAUDI2:
str = "GAUDI2";
break;
- case ASIC_GAUDI2_SEC:
- str = "GAUDI2 SEC";
+ case ASIC_GAUDI2B:
+ str = "GAUDI2B";
break;
default:
dev_err(hdev->dev, "Unrecognized ASIC type %d\n",
}
static const char *gaudi_get_razwi_initiator_dma_name(struct hl_device *hdev, u32 x_y,
- bool is_write, s32 *engine_id_1,
- s32 *engine_id_2)
+ bool is_write, u16 *engine_id_1,
+ u16 *engine_id_2)
{
u32 dma_id[2], dma_offset, err_cause[2], mask, i;
}
static const char *gaudi_get_razwi_initiator_name(struct hl_device *hdev, bool is_write,
- u32 *engine_id_1, u32 *engine_id_2)
+ u16 *engine_id_1, u16 *engine_id_2)
{
u32 val, x_y, axi_id;
return "unknown initiator";
}
-static void gaudi_print_and_get_razwi_info(struct hl_device *hdev, u32 *engine_id_1,
- u32 *engine_id_2)
+static void gaudi_print_and_get_razwi_info(struct hl_device *hdev, u16 *engine_id_1,
+ u16 *engine_id_2, bool *is_read, bool *is_write)
{
if (RREG32(mmMMU_UP_RAZWI_WRITE_VLD)) {
"RAZWI event caused by illegal write of %s\n",
gaudi_get_razwi_initiator_name(hdev, true, engine_id_1, engine_id_2));
WREG32(mmMMU_UP_RAZWI_WRITE_VLD, 0);
+ *is_write = true;
}
if (RREG32(mmMMU_UP_RAZWI_READ_VLD)) {
"RAZWI event caused by illegal read of %s\n",
gaudi_get_razwi_initiator_name(hdev, false, engine_id_1, engine_id_2));
WREG32(mmMMU_UP_RAZWI_READ_VLD, 0);
+ *is_read = true;
}
}
-static void gaudi_print_and_get_mmu_error_info(struct hl_device *hdev, u64 *addr, u8 *type)
+static void gaudi_print_and_get_mmu_error_info(struct hl_device *hdev, u64 *addr, u64 *event_mask)
{
struct gaudi_device *gaudi = hdev->asic_specific;
u32 val;
*addr |= RREG32(mmMMU_UP_PAGE_ERROR_CAPTURE_VA);
dev_err_ratelimited(hdev->dev, "MMU page fault on va 0x%llx\n", *addr);
- *type = HL_RAZWI_PAGE_FAULT;
+ hl_handle_page_fault(hdev, *addr, 0, true, event_mask);
WREG32(mmMMU_UP_PAGE_ERROR_CAPTURE, 0);
}
*addr |= RREG32(mmMMU_UP_ACCESS_ERROR_CAPTURE_VA);
dev_err_ratelimited(hdev->dev, "MMU access error on va 0x%llx\n", *addr);
- *type = HL_RAZWI_MMU_ACCESS_ERROR;
WREG32(mmMMU_UP_ACCESS_ERROR_CAPTURE, 0);
}
}
static void gaudi_print_irq_info(struct hl_device *hdev, u16 event_type,
- bool razwi)
+ bool razwi, u64 *event_mask)
{
- u32 engine_id_1, engine_id_2;
+ bool is_read = false, is_write = false;
+ u16 engine_id[2], num_of_razwi_eng = 0;
char desc[64] = "";
u64 razwi_addr = 0;
- u8 razwi_type;
- int rc;
+ u8 razwi_flags = 0;
/*
* Init engine id by default as not valid and only if razwi initiated from engine with
* engine id it will get valid value.
- * Init razwi type to default, will be changed only if razwi caused by page fault of
- * MMU access error
*/
- engine_id_1 = U16_MAX;
- engine_id_2 = U16_MAX;
- razwi_type = U8_MAX;
+ engine_id[0] = HL_RAZWI_NA_ENG_ID;
+ engine_id[1] = HL_RAZWI_NA_ENG_ID;
gaudi_get_event_desc(event_type, desc, sizeof(desc));
dev_err_ratelimited(hdev->dev, "Received H/W interrupt %d [\"%s\"]\n",
event_type, desc);
if (razwi) {
- gaudi_print_and_get_razwi_info(hdev, &engine_id_1, &engine_id_2);
- gaudi_print_and_get_mmu_error_info(hdev, &razwi_addr, &razwi_type);
-
- /* In case it's the first razwi, save its parameters*/
- rc = atomic_cmpxchg(&hdev->captured_err_info.razwi.write_enable, 1, 0);
- if (rc) {
- hdev->captured_err_info.razwi.timestamp = ktime_get();
- hdev->captured_err_info.razwi.addr = razwi_addr;
- hdev->captured_err_info.razwi.engine_id_1 = engine_id_1;
- hdev->captured_err_info.razwi.engine_id_2 = engine_id_2;
- /*
- * If first engine id holds non valid value the razwi initiator
- * does not have engine id
- */
- hdev->captured_err_info.razwi.non_engine_initiator =
- (engine_id_1 == U16_MAX);
- hdev->captured_err_info.razwi.type = razwi_type;
-
+ gaudi_print_and_get_razwi_info(hdev, &engine_id[0], &engine_id[1], &is_read,
+ &is_write);
+ gaudi_print_and_get_mmu_error_info(hdev, &razwi_addr, event_mask);
+
+ if (is_read)
+ razwi_flags |= HL_RAZWI_READ;
+ if (is_write)
+ razwi_flags |= HL_RAZWI_WRITE;
+
+ if (engine_id[0] != HL_RAZWI_NA_ENG_ID) {
+ if (engine_id[1] != HL_RAZWI_NA_ENG_ID)
+ num_of_razwi_eng = 2;
+ else
+ num_of_razwi_eng = 1;
}
+
+ hl_handle_razwi(hdev, razwi_addr, engine_id, num_of_razwi_eng, razwi_flags,
+ event_mask);
}
}
{
struct hl_hw_queue *q = &hdev->kernel_queues[GAUDI_QUEUE_ID_CPU_PQ];
- dev_err(hdev->dev, "Out of sync with FW, FW: pi=%u, ci=%u, LKD: pi=%u, ci=%u\n",
- sync_err->pi, sync_err->ci, q->pi, atomic_read(&q->ci));
+ dev_err(hdev->dev, "Out of sync with FW, FW: pi=%u, ci=%u, LKD: pi=%u, ci=%d\n",
+ le32_to_cpu(sync_err->pi), le32_to_cpu(sync_err->ci), q->pi, atomic_read(&q->ci));
}
static void gaudi_print_fw_alive_info(struct hl_device *hdev,
{
dev_err(hdev->dev,
"FW alive report: severity=%s, process_id=%u, thread_id=%u, uptime=%llu seconds\n",
- (fw_alive->severity == FW_ALIVE_SEVERITY_MINOR) ?
- "Minor" : "Critical", fw_alive->process_id,
- fw_alive->thread_id, fw_alive->uptime_seconds);
+ (fw_alive->severity == FW_ALIVE_SEVERITY_MINOR) ? "Minor" : "Critical",
+ le32_to_cpu(fw_alive->process_id),
+ le32_to_cpu(fw_alive->thread_id),
+ le64_to_cpu(fw_alive->uptime_seconds));
}
static void gaudi_print_nic_axi_irq_info(struct hl_device *hdev, u16 event_type,
case GAUDI_EVENT_HBM_0_DERR ... GAUDI_EVENT_HBM_3_DERR:
case GAUDI_EVENT_MMU_DERR:
case GAUDI_EVENT_NIC0_CS_DBG_DERR ... GAUDI_EVENT_NIC4_CS_DBG_DERR:
- gaudi_print_irq_info(hdev, event_type, true);
+ gaudi_print_irq_info(hdev, event_type, true, &event_mask);
gaudi_handle_ecc_event(hdev, event_type, &eq_entry->ecc_data);
event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR;
fw_fatal_err_flag = HL_DRV_RESET_FW_FATAL_ERR;
case GAUDI_EVENT_AXI_ECC:
case GAUDI_EVENT_L2_RAM_ECC:
case GAUDI_EVENT_PLL0 ... GAUDI_EVENT_PLL17:
- gaudi_print_irq_info(hdev, event_type, false);
+ gaudi_print_irq_info(hdev, event_type, false, &event_mask);
fw_fatal_err_flag = HL_DRV_RESET_FW_FATAL_ERR;
event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR;
goto reset_device;
case GAUDI_EVENT_HBM1_SPI_0:
case GAUDI_EVENT_HBM2_SPI_0:
case GAUDI_EVENT_HBM3_SPI_0:
- gaudi_print_irq_info(hdev, event_type, false);
+ gaudi_print_irq_info(hdev, event_type, false, &event_mask);
gaudi_hbm_read_interrupts(hdev,
gaudi_hbm_event_to_dev(event_type),
&eq_entry->hbm_ecc_data);
case GAUDI_EVENT_HBM1_SPI_1:
case GAUDI_EVENT_HBM2_SPI_1:
case GAUDI_EVENT_HBM3_SPI_1:
- gaudi_print_irq_info(hdev, event_type, false);
+ gaudi_print_irq_info(hdev, event_type, false, &event_mask);
gaudi_hbm_read_interrupts(hdev,
gaudi_hbm_event_to_dev(event_type),
&eq_entry->hbm_ecc_data);
* if the event is a TPC Assertion or a "real" TPC DEC.
*/
event_mask |= HL_NOTIFIER_EVENT_TPC_ASSERT;
- gaudi_print_irq_info(hdev, event_type, true);
+ gaudi_print_irq_info(hdev, event_type, true, &event_mask);
reset_required = gaudi_tpc_read_interrupts(hdev,
tpc_dec_event_to_tpc_id(event_type),
"AXI_SLV_DEC_Error");
case GAUDI_EVENT_TPC5_KRN_ERR:
case GAUDI_EVENT_TPC6_KRN_ERR:
case GAUDI_EVENT_TPC7_KRN_ERR:
- gaudi_print_irq_info(hdev, event_type, true);
+ gaudi_print_irq_info(hdev, event_type, true, &event_mask);
reset_required = gaudi_tpc_read_interrupts(hdev,
tpc_krn_event_to_tpc_id(event_type),
"KRN_ERR");
case GAUDI_EVENT_HBM_0_SERR ... GAUDI_EVENT_HBM_3_SERR:
fallthrough;
case GAUDI_EVENT_MMU_SERR:
- gaudi_print_irq_info(hdev, event_type, true);
+ gaudi_print_irq_info(hdev, event_type, true, &event_mask);
gaudi_handle_ecc_event(hdev, event_type, &eq_entry->ecc_data);
hl_fw_unmask_irq(hdev, event_type);
event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR;
case GAUDI_EVENT_CPU_AXI_SPLITTER:
case GAUDI_EVENT_PSOC_AXI_DEC:
case GAUDI_EVENT_PSOC_PRSTN_FALL:
- gaudi_print_irq_info(hdev, event_type, true);
+ gaudi_print_irq_info(hdev, event_type, true, &event_mask);
hl_fw_unmask_irq(hdev, event_type);
event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR;
break;
case GAUDI_EVENT_MMU_PAGE_FAULT:
case GAUDI_EVENT_MMU_WR_PERM:
- gaudi_print_irq_info(hdev, event_type, true);
+ gaudi_print_irq_info(hdev, event_type, true, &event_mask);
hl_fw_unmask_irq(hdev, event_type);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
case GAUDI_EVENT_NIC4_QM1:
case GAUDI_EVENT_DMA0_CORE ... GAUDI_EVENT_DMA7_CORE:
case GAUDI_EVENT_TPC0_QM ... GAUDI_EVENT_TPC7_QM:
- gaudi_print_irq_info(hdev, event_type, true);
+ gaudi_print_irq_info(hdev, event_type, true, &event_mask);
gaudi_handle_qman_err(hdev, event_type, &event_mask);
hl_fw_unmask_irq(hdev, event_type);
event_mask |= (HL_NOTIFIER_EVENT_USER_ENGINE_ERR | HL_NOTIFIER_EVENT_DEVICE_RESET);
break;
case GAUDI_EVENT_RAZWI_OR_ADC_SW:
- gaudi_print_irq_info(hdev, event_type, true);
+ gaudi_print_irq_info(hdev, event_type, true, &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
goto reset_device;
case GAUDI_EVENT_TPC6_BMON_SPMU:
case GAUDI_EVENT_TPC7_BMON_SPMU:
case GAUDI_EVENT_DMA_BM_CH0 ... GAUDI_EVENT_DMA_BM_CH7:
- gaudi_print_irq_info(hdev, event_type, false);
+ gaudi_print_irq_info(hdev, event_type, false, &event_mask);
hl_fw_unmask_irq(hdev, event_type);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
break;
case GAUDI_EVENT_DMA_IF_SEI_0 ... GAUDI_EVENT_DMA_IF_SEI_3:
- gaudi_print_irq_info(hdev, event_type, false);
+ gaudi_print_irq_info(hdev, event_type, false, &event_mask);
gaudi_print_sm_sei_info(hdev, event_type,
&eq_entry->sm_sei_data);
rc = hl_state_dump(hdev);
break;
case GAUDI_EVENT_DEV_RESET_REQ:
- gaudi_print_irq_info(hdev, event_type, false);
+ gaudi_print_irq_info(hdev, event_type, false, &event_mask);
event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR;
goto reset_device;
case GAUDI_EVENT_PKT_QUEUE_OUT_SYNC:
- gaudi_print_irq_info(hdev, event_type, false);
+ gaudi_print_irq_info(hdev, event_type, false, &event_mask);
gaudi_print_out_of_sync_info(hdev, &eq_entry->pkt_sync_err);
event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR;
goto reset_device;
case GAUDI_EVENT_FW_ALIVE_S:
- gaudi_print_irq_info(hdev, event_type, false);
+ gaudi_print_irq_info(hdev, event_type, false, &event_mask);
gaudi_print_fw_alive_info(hdev, &eq_entry->fw_alive);
event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR;
goto reset_device;
reset_required = false;
}
- /* despite reset doesn't execute. a notification on
- * occurred event needs to be sent here
- */
- hl_notifier_event_send_all(hdev, event_mask);
- if (reset_required)
- hl_device_reset(hdev, flags);
- else
+ if (reset_required) {
+ hl_device_cond_reset(hdev, flags, event_mask);
+ } else {
hl_fw_unmask_irq(hdev, event_type);
+ /* Notification on occurred event needs to be sent although reset is not executed */
+ if (event_mask)
+ hl_notifier_event_send_all(hdev, event_mask);
+ }
}
static void *gaudi_get_events_stat(struct hl_device *hdev, bool aggregate, u32 *size)
#define GAUDI2_VDEC_MSIX_ENTRIES (GAUDI2_IRQ_NUM_SHARED_DEC1_ABNRM - \
GAUDI2_IRQ_NUM_DCORE0_DEC0_NRM + 1)
+#define ENGINE_ID_DCORE_OFFSET (GAUDI2_DCORE1_ENGINE_ID_EDMA_0 - GAUDI2_DCORE0_ENGINE_ID_EDMA_0)
+
enum hl_pmmu_fatal_cause {
LATENCY_RD_OUT_FIFO_OVERRUN,
LATENCY_WR_OUT_FIFO_OVERRUN,
fw_loader->skip_bmc = false;
fw_loader->sram_bar_id = SRAM_CFG_BAR_ID;
fw_loader->dram_bar_id = DRAM_BAR_ID;
-
- if (hdev->asic_type == ASIC_GAUDI2 || hdev->asic_type == ASIC_GAUDI2_SEC)
- fw_loader->cpu_timeout = GAUDI2_CPU_TIMEOUT_USEC;
- else /* ASIC_GAUDI2_FPGA */
- fw_loader->cpu_timeout = GAUDI2_FPGA_CPU_TIMEOUT;
+ fw_loader->cpu_timeout = GAUDI2_CPU_TIMEOUT_USEC;
/* here we update initial values for few specific dynamic regs (as
* before reading the first descriptor from FW those value has to be
reg_val = FIELD_PREP(DCORE0_SYNC_MNGR_OBJS_MON_CONFIG_CQ_EN_MASK, 1);
WREG32(mmDCORE0_SYNC_MNGR_OBJS_MON_CONFIG_0 + (4 * i), reg_val);
- /* Init CQ0 DB */
- /* Configure the monitor to trigger MSI-X interrupt */
- /* TODO:
- * Remove the if statement when virtual MSI-X doorbell is supported in simulator (SW-93022)
- * and in F/W (SW-93024).
- */
- if (!hdev->pdev || hdev->asic_prop.fw_security_enabled) {
- u64 msix_db_reg = CFG_BASE + mmPCIE_DBI_MSIX_DOORBELL_OFF;
-
- WREG32(mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_L_0, lower_32_bits(msix_db_reg));
- WREG32(mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_H_0, upper_32_bits(msix_db_reg));
- } else {
- WREG32(mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_L_0,
- lower_32_bits(gaudi2->virt_msix_db_dma_addr));
- WREG32(mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_H_0,
- upper_32_bits(gaudi2->virt_msix_db_dma_addr));
- }
+ /* Init CQ0 DB - configure the monitor to trigger MSI-X interrupt */
+ WREG32(mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_L_0, lower_32_bits(gaudi2->virt_msix_db_dma_addr));
+ WREG32(mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_H_0, upper_32_bits(gaudi2->virt_msix_db_dma_addr));
WREG32(mmDCORE0_SYNC_MNGR_GLBL_LBW_DATA_0, GAUDI2_IRQ_NUM_COMPLETION);
for (i = 0 ; i < GAUDI2_RESERVED_CQ_NUMBER ; i++) {
static void gaudi2_init_dcore_mme(struct hl_device *hdev, int dcore_id,
bool config_qman_only)
{
- u32 queue_id_base, reg_base, clk_en_addr = 0;
+ u32 queue_id_base, reg_base;
switch (dcore_id) {
case 0:
break;
case 1:
queue_id_base = GAUDI2_QUEUE_ID_DCORE1_MME_0_0;
- clk_en_addr = mmDCORE1_MME_CTRL_LO_QM_SLV_CLK_EN;
break;
case 2:
queue_id_base = GAUDI2_QUEUE_ID_DCORE2_MME_0_0;
break;
case 3:
queue_id_base = GAUDI2_QUEUE_ID_DCORE3_MME_0_0;
- clk_en_addr = mmDCORE3_MME_CTRL_LO_QM_SLV_CLK_EN;
break;
default:
dev_err(hdev->dev, "Invalid dcore id %u\n", dcore_id);
return;
}
- if (clk_en_addr && !(hdev->fw_components & FW_TYPE_BOOT_CPU))
- WREG32(clk_en_addr, 0x1);
-
if (!config_qman_only) {
reg_base = gaudi2_mme_acc_blocks_bases[dcore_id];
gaudi2_init_mme_acc(hdev, reg_base);
{
u32 sob_id;
- /* TODO:
- * Remove when virtual MSI-X doorbell is supported in simulator (SW-93022) and in F/W
- * (SW-93024).
- */
- if (!hdev->pdev || hdev->asic_prop.fw_security_enabled) {
- u32 interrupt_id = GAUDI2_IRQ_NUM_DCORE0_DEC0_NRM + 2 * decoder_id;
-
- WREG32(base_addr + BRDG_CTRL_NRM_MSIX_LBW_AWADDR, mmPCIE_DBI_MSIX_DOORBELL_OFF);
- WREG32(base_addr + BRDG_CTRL_NRM_MSIX_LBW_WDATA, interrupt_id);
- WREG32(base_addr + BRDG_CTRL_ABNRM_MSIX_LBW_AWADDR, mmPCIE_DBI_MSIX_DOORBELL_OFF);
- WREG32(base_addr + BRDG_CTRL_ABNRM_MSIX_LBW_WDATA, interrupt_id + 1);
- return;
- }
-
/* VCMD normal interrupt */
sob_id = GAUDI2_RESERVED_SOB_DEC_NRM_FIRST + decoder_id;
WREG32(base_addr + BRDG_CTRL_NRM_MSIX_LBW_AWADDR,
}
}
-static void gaudi2_init_msix_gw_table(struct hl_device *hdev)
-{
- u32 first_reg_offset, last_reg_offset, msix_gw_table_base;
- u8 first_bit, last_bit;
- int i;
-
- msix_gw_table_base = mmPCIE_WRAP_MSIX_GW_TABLE_0;
- first_reg_offset = (GAUDI2_IRQ_NUM_USER_FIRST >> 5) << 2;
- first_bit = GAUDI2_IRQ_NUM_USER_FIRST % 32;
- last_reg_offset = (GAUDI2_IRQ_NUM_USER_LAST >> 5) << 2;
- last_bit = GAUDI2_IRQ_NUM_USER_LAST % 32;
-
- if (first_reg_offset == last_reg_offset) {
- WREG32(msix_gw_table_base + first_reg_offset, GENMASK(last_bit, first_bit));
- return;
- }
-
- WREG32(msix_gw_table_base + first_reg_offset, GENMASK(31, first_bit));
- WREG32(msix_gw_table_base + last_reg_offset, GENMASK(last_bit, 0));
-
- for (i = first_reg_offset + 4; i < last_reg_offset ; i += 4)
- WREG32(msix_gw_table_base + i, 0xFFFFFFFF);
-}
-
static int gaudi2_mmu_update_asid_hop0_addr(struct hl_device *hdev,
u32 stlb_base, u32 asid, u64 phys_addr)
{
mmu_base = mmPMMU_HBW_MMU_BASE;
stlb_base = mmPMMU_HBW_STLB_BASE;
- RMWREG32(stlb_base + STLB_HOP_CONFIGURATION_OFFSET,
+ RMWREG32_SHIFTED(stlb_base + STLB_HOP_CONFIGURATION_OFFSET,
(0 << PMMU_HBW_STLB_HOP_CONFIGURATION_FIRST_HOP_SHIFT) |
(5 << PMMU_HBW_STLB_HOP_CONFIGURATION_FIRST_LOOKUP_HOP_SMALL_P_SHIFT) |
(4 << PMMU_HBW_STLB_HOP_CONFIGURATION_FIRST_LOOKUP_HOP_LARGE_P_SHIFT) |
if (PAGE_SIZE == SZ_64K) {
/* Set page sizes to 64K on hop5 and 16M on hop4 + enable 8 bit hops */
- RMWREG32(mmu_base + MMU_STATIC_MULTI_PAGE_SIZE_OFFSET,
+ RMWREG32_SHIFTED(mmu_base + MMU_STATIC_MULTI_PAGE_SIZE_OFFSET,
FIELD_PREP(DCORE0_HMMU0_MMU_STATIC_MULTI_PAGE_SIZE_HOP5_PAGE_SIZE_MASK, 4) |
FIELD_PREP(DCORE0_HMMU0_MMU_STATIC_MULTI_PAGE_SIZE_HOP4_PAGE_SIZE_MASK, 3) |
FIELD_PREP(
RMWREG32(mmu_base + MMU_STATIC_MULTI_PAGE_SIZE_OFFSET, 5 /* 64MB */,
MMU_STATIC_MULTI_PAGE_SIZE_HOP4_PAGE_SIZE_MASK);
- RMWREG32(stlb_base + STLB_HOP_CONFIGURATION_OFFSET,
+ RMWREG32_SHIFTED(stlb_base + STLB_HOP_CONFIGURATION_OFFSET,
FIELD_PREP(DCORE0_HMMU0_STLB_HOP_CONFIGURATION_FIRST_HOP_MASK, 0) |
FIELD_PREP(DCORE0_HMMU0_STLB_HOP_CONFIGURATION_FIRST_LOOKUP_HOP_SMALL_P_MASK, 3) |
FIELD_PREP(DCORE0_HMMU0_STLB_HOP_CONFIGURATION_FIRST_LOOKUP_HOP_LARGE_P_MASK, 3) |
return rc;
}
- gaudi2_init_msix_gw_table(hdev);
-
gaudi2_init_scrambler_hbm(hdev);
gaudi2_init_kdma(hdev);
{
switch (event) {
case GAUDI2_EVENT_CPU_CPLD_SHUTDOWN_CAUSE:
+ case GAUDI2_EVENT_CPU_FIX_POWER_ENV_S ... GAUDI2_EVENT_CPU_FIX_THERMAL_ENV_E:
return true;
default:
return false;
static void gaudi2_razwi_rr_hbw_shared_printf_info(struct hl_device *hdev,
u64 rtr_mstr_if_base_addr, bool is_write, char *name,
- bool read_razwi_regs, struct hl_eq_razwi_info *razwi_info)
+ bool read_razwi_regs, struct hl_eq_razwi_info *razwi_info,
+ enum gaudi2_engine_id id, u64 *event_mask)
{
u32 razwi_hi, razwi_lo, razwi_xy;
+ u16 eng_id = id;
+ u8 rd_wr_flag;
if (is_write) {
if (read_razwi_regs) {
razwi_lo = le32_to_cpu(razwi_info->hbw.rr_aw_razwi_lo_reg);
razwi_xy = le32_to_cpu(razwi_info->hbw.rr_aw_razwi_id_reg);
}
+ rd_wr_flag = HL_RAZWI_WRITE;
} else {
if (read_razwi_regs) {
razwi_hi = RREG32(rtr_mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_HI);
razwi_lo = le32_to_cpu(razwi_info->hbw.rr_ar_razwi_lo_reg);
razwi_xy = le32_to_cpu(razwi_info->hbw.rr_ar_razwi_id_reg);
}
+ rd_wr_flag = HL_RAZWI_READ;
}
+ hl_handle_razwi(hdev, (u64)razwi_hi << 32 | razwi_lo, &eng_id, 1,
+ rd_wr_flag | HL_RAZWI_HBW, event_mask);
+
dev_err_ratelimited(hdev->dev,
"%s-RAZWI SHARED RR HBW %s error, address %#llx, Initiator coordinates 0x%x\n",
name, is_write ? "WR" : "RD", (u64)razwi_hi << 32 | razwi_lo, razwi_xy);
static void gaudi2_razwi_rr_lbw_shared_printf_info(struct hl_device *hdev,
u64 rtr_mstr_if_base_addr, bool is_write, char *name,
- bool read_razwi_regs, struct hl_eq_razwi_info *razwi_info)
+ bool read_razwi_regs, struct hl_eq_razwi_info *razwi_info,
+ enum gaudi2_engine_id id, u64 *event_mask)
{
u32 razwi_addr, razwi_xy;
+ u16 eng_id = id;
+ u8 rd_wr_flag;
if (is_write) {
if (read_razwi_regs) {
razwi_xy = le32_to_cpu(razwi_info->lbw.rr_aw_razwi_id_reg);
}
- dev_err_ratelimited(hdev->dev,
- "%s-RAZWI SHARED RR LBW WR error, mstr_if 0x%llx, captured address 0x%x, Initiator coordinates 0x%x\n",
- name, rtr_mstr_if_base_addr, razwi_addr, razwi_xy);
+ rd_wr_flag = HL_RAZWI_WRITE;
} else {
if (read_razwi_regs) {
razwi_addr = RREG32(rtr_mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI);
razwi_xy = le32_to_cpu(razwi_info->lbw.rr_ar_razwi_id_reg);
}
- dev_err_ratelimited(hdev->dev,
- "%s-RAZWI SHARED RR LBW AR error, mstr_if 0x%llx, captured address 0x%x Initiator coordinates 0x%x\n",
- name, rtr_mstr_if_base_addr, razwi_addr, razwi_xy);
+ rd_wr_flag = HL_RAZWI_READ;
+ }
+
+ hl_handle_razwi(hdev, razwi_addr, &eng_id, 1, rd_wr_flag | HL_RAZWI_LBW, event_mask);
+ dev_err_ratelimited(hdev->dev,
+ "%s-RAZWI SHARED RR LBW %s error, mstr_if 0x%llx, captured address 0x%x Initiator coordinates 0x%x\n",
+ name, is_write ? "WR" : "RD", rtr_mstr_if_base_addr, razwi_addr,
+ razwi_xy);
+}
+
+static enum gaudi2_engine_id gaudi2_razwi_calc_engine_id(struct hl_device *hdev,
+ enum razwi_event_sources module, u8 module_idx)
+{
+ switch (module) {
+ case RAZWI_TPC:
+ if (module_idx == (NUM_OF_TPC_PER_DCORE * NUM_OF_DCORES))
+ return GAUDI2_DCORE0_ENGINE_ID_TPC_6;
+ return (((module_idx / NUM_OF_TPC_PER_DCORE) * ENGINE_ID_DCORE_OFFSET) +
+ (module_idx % NUM_OF_TPC_PER_DCORE) +
+ (GAUDI2_DCORE0_ENGINE_ID_TPC_0 - GAUDI2_DCORE0_ENGINE_ID_EDMA_0));
+
+ case RAZWI_MME:
+ return ((GAUDI2_DCORE0_ENGINE_ID_MME - GAUDI2_DCORE0_ENGINE_ID_EDMA_0) +
+ (module_idx * ENGINE_ID_DCORE_OFFSET));
+
+ case RAZWI_EDMA:
+ return (((module_idx / NUM_OF_EDMA_PER_DCORE) * ENGINE_ID_DCORE_OFFSET) +
+ (module_idx % NUM_OF_EDMA_PER_DCORE));
+
+ case RAZWI_PDMA:
+ return (GAUDI2_ENGINE_ID_PDMA_0 + module_idx);
+
+ case RAZWI_NIC:
+ return (GAUDI2_ENGINE_ID_NIC0_0 + (NIC_NUMBER_OF_QM_PER_MACRO * module_idx));
+
+ case RAZWI_DEC:
+ if (module_idx == 8)
+ return GAUDI2_PCIE_ENGINE_ID_DEC_0;
+
+ if (module_idx == 9)
+ return GAUDI2_PCIE_ENGINE_ID_DEC_1;
+ ;
+ return (((module_idx / NUM_OF_DEC_PER_DCORE) * ENGINE_ID_DCORE_OFFSET) +
+ (module_idx % NUM_OF_DEC_PER_DCORE) +
+ (GAUDI2_DCORE0_ENGINE_ID_DEC_0 - GAUDI2_DCORE0_ENGINE_ID_EDMA_0));
+
+ case RAZWI_ROT:
+ return GAUDI2_ENGINE_ID_ROT_0 + module_idx;
+
+ default:
+ return GAUDI2_ENGINE_ID_SIZE;
}
}
*/
static void gaudi2_ack_module_razwi_event_handler(struct hl_device *hdev,
enum razwi_event_sources module, u8 module_idx,
- u8 module_sub_idx, struct hl_eq_razwi_info *razwi_info)
+ u8 module_sub_idx, struct hl_eq_razwi_info *razwi_info,
+ u64 *event_mask)
{
bool via_sft = false, read_razwi_regs = false;
- u32 rtr_id, dcore_id, dcore_rtr_id, sft_id;
+ u32 rtr_id, dcore_id, dcore_rtr_id, sft_id, eng_id;
u64 rtr_mstr_if_base_addr;
u32 hbw_shrd_aw = 0, hbw_shrd_ar = 0;
u32 lbw_shrd_aw = 0, lbw_shrd_ar = 0;
if (!hbw_shrd_aw && !hbw_shrd_ar && !lbw_shrd_aw && !lbw_shrd_ar)
return;
+ eng_id = gaudi2_razwi_calc_engine_id(hdev, module, module_idx);
if (hbw_shrd_aw) {
gaudi2_razwi_rr_hbw_shared_printf_info(hdev, rtr_mstr_if_base_addr, true,
- initiator_name, read_razwi_regs, razwi_info);
+ initiator_name, read_razwi_regs, razwi_info,
+ eng_id, event_mask);
/* Clear event indication */
if (read_razwi_regs)
if (hbw_shrd_ar) {
gaudi2_razwi_rr_hbw_shared_printf_info(hdev, rtr_mstr_if_base_addr, false,
- initiator_name, read_razwi_regs, razwi_info);
+ initiator_name, read_razwi_regs, razwi_info,
+ eng_id, event_mask);
/* Clear event indication */
if (read_razwi_regs)
if (lbw_shrd_aw) {
gaudi2_razwi_rr_lbw_shared_printf_info(hdev, rtr_mstr_if_base_addr, true,
- initiator_name, read_razwi_regs, razwi_info);
+ initiator_name, read_razwi_regs, razwi_info,
+ eng_id, event_mask);
/* Clear event indication */
if (read_razwi_regs)
if (lbw_shrd_ar) {
gaudi2_razwi_rr_lbw_shared_printf_info(hdev, rtr_mstr_if_base_addr, false,
- initiator_name, read_razwi_regs, razwi_info);
+ initiator_name, read_razwi_regs, razwi_info,
+ eng_id, event_mask);
/* Clear event indication */
if (read_razwi_regs)
/* check all TPCs */
for (mod_idx = 0 ; mod_idx < (NUM_OF_TPC_PER_DCORE * NUM_OF_DCORES + 1) ; mod_idx++) {
if (prop->tpc_enabled_mask & BIT(mod_idx))
- gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_TPC, mod_idx, 0, NULL);
+ gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_TPC, mod_idx, 0, NULL,
+ NULL);
}
/* check all MMEs */
for (mod_idx = 0 ; mod_idx < (NUM_OF_MME_PER_DCORE * NUM_OF_DCORES) ; mod_idx++)
for (sub_mod = MME_WAP0 ; sub_mod < MME_INITIATORS_MAX ; sub_mod++)
gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mod_idx,
- sub_mod, NULL);
+ sub_mod, NULL, NULL);
/* check all EDMAs */
for (mod_idx = 0 ; mod_idx < (NUM_OF_EDMA_PER_DCORE * NUM_OF_DCORES) ; mod_idx++)
if (prop->edma_enabled_mask & BIT(mod_idx))
- gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_EDMA, mod_idx, 0, NULL);
+ gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_EDMA, mod_idx, 0, NULL,
+ NULL);
/* check all PDMAs */
for (mod_idx = 0 ; mod_idx < NUM_OF_PDMA ; mod_idx++)
- gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_PDMA, mod_idx, 0, NULL);
+ gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_PDMA, mod_idx, 0, NULL,
+ NULL);
/* check all NICs */
for (mod_idx = 0 ; mod_idx < NIC_NUMBER_OF_PORTS ; mod_idx++)
if (hdev->nic_ports_mask & BIT(mod_idx))
gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_NIC, mod_idx >> 1, 0,
- NULL);
+ NULL, NULL);
/* check all DECs */
for (mod_idx = 0 ; mod_idx < NUMBER_OF_DEC ; mod_idx++)
if (prop->decoder_enabled_mask & BIT(mod_idx))
- gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_DEC, mod_idx, 0, NULL);
+ gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_DEC, mod_idx, 0, NULL,
+ NULL);
/* check all ROTs */
for (mod_idx = 0 ; mod_idx < NUM_OF_ROT ; mod_idx++)
- gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_ROT, mod_idx, 0, NULL);
+ gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_ROT, mod_idx, 0, NULL, NULL);
}
static const char *gaudi2_get_initiators_name(u32 rtr_id)
}
}
+static u16 gaudi2_get_razwi_initiators(u32 rtr_id, u16 *engines)
+{
+ switch (rtr_id) {
+ case DCORE0_RTR0:
+ engines[0] = GAUDI2_DCORE0_ENGINE_ID_DEC_0;
+ engines[1] = GAUDI2_DCORE0_ENGINE_ID_DEC_1;
+ engines[2] = GAUDI2_PCIE_ENGINE_ID_DEC_0;
+ engines[3] = GAUDI2_PCIE_ENGINE_ID_DEC_1;
+ engines[4] = GAUDI2_DCORE0_ENGINE_ID_TPC_6;
+ engines[5] = GAUDI2_ENGINE_ID_PDMA_0;
+ engines[6] = GAUDI2_ENGINE_ID_PDMA_1;
+ engines[7] = GAUDI2_ENGINE_ID_PCIE;
+ engines[8] = GAUDI2_DCORE0_ENGINE_ID_EDMA_0;
+ engines[9] = GAUDI2_DCORE1_ENGINE_ID_EDMA_0;
+ engines[10] = GAUDI2_ENGINE_ID_PSOC;
+ return 11;
+
+ case DCORE0_RTR1:
+ engines[0] = GAUDI2_DCORE0_ENGINE_ID_TPC_0;
+ engines[1] = GAUDI2_DCORE0_ENGINE_ID_TPC_1;
+ return 2;
+
+ case DCORE0_RTR2:
+ engines[0] = GAUDI2_DCORE0_ENGINE_ID_TPC_2;
+ engines[1] = GAUDI2_DCORE0_ENGINE_ID_TPC_3;
+ return 2;
+
+ case DCORE0_RTR3:
+ engines[0] = GAUDI2_DCORE0_ENGINE_ID_TPC_4;
+ engines[1] = GAUDI2_DCORE0_ENGINE_ID_TPC_5;
+ return 2;
+
+ case DCORE0_RTR4:
+ case DCORE0_RTR5:
+ case DCORE0_RTR6:
+ case DCORE0_RTR7:
+ engines[0] = GAUDI2_DCORE0_ENGINE_ID_MME;
+ return 1;
+
+ case DCORE1_RTR0:
+ case DCORE1_RTR1:
+ case DCORE1_RTR2:
+ case DCORE1_RTR3:
+ engines[0] = GAUDI2_DCORE1_ENGINE_ID_MME;
+ return 1;
+
+ case DCORE1_RTR4:
+ engines[0] = GAUDI2_DCORE1_ENGINE_ID_TPC_4;
+ engines[1] = GAUDI2_DCORE1_ENGINE_ID_TPC_5;
+ return 2;
+
+ case DCORE1_RTR5:
+ engines[0] = GAUDI2_DCORE1_ENGINE_ID_TPC_2;
+ engines[1] = GAUDI2_DCORE1_ENGINE_ID_TPC_3;
+ return 2;
+
+ case DCORE1_RTR6:
+ engines[0] = GAUDI2_DCORE1_ENGINE_ID_TPC_0;
+ engines[1] = GAUDI2_DCORE1_ENGINE_ID_TPC_1;
+ return 2;
+
+ case DCORE1_RTR7:
+ engines[0] = GAUDI2_DCORE1_ENGINE_ID_DEC_0;
+ engines[1] = GAUDI2_DCORE1_ENGINE_ID_DEC_1;
+ engines[2] = GAUDI2_ENGINE_ID_NIC0_0;
+ engines[3] = GAUDI2_ENGINE_ID_NIC1_0;
+ engines[4] = GAUDI2_ENGINE_ID_NIC2_0;
+ engines[5] = GAUDI2_ENGINE_ID_NIC3_0;
+ engines[6] = GAUDI2_ENGINE_ID_NIC4_0;
+ engines[7] = GAUDI2_ENGINE_ID_ARC_FARM;
+ engines[8] = GAUDI2_ENGINE_ID_KDMA;
+ engines[9] = GAUDI2_DCORE0_ENGINE_ID_EDMA_1;
+ engines[10] = GAUDI2_DCORE1_ENGINE_ID_EDMA_1;
+ return 11;
+
+ case DCORE2_RTR0:
+ engines[0] = GAUDI2_DCORE2_ENGINE_ID_DEC_0;
+ engines[1] = GAUDI2_DCORE2_ENGINE_ID_DEC_1;
+ engines[2] = GAUDI2_ENGINE_ID_NIC5_0;
+ engines[3] = GAUDI2_ENGINE_ID_NIC6_0;
+ engines[4] = GAUDI2_ENGINE_ID_NIC7_0;
+ engines[5] = GAUDI2_ENGINE_ID_NIC8_0;
+ engines[6] = GAUDI2_DCORE2_ENGINE_ID_EDMA_0;
+ engines[7] = GAUDI2_DCORE3_ENGINE_ID_EDMA_0;
+ engines[8] = GAUDI2_ENGINE_ID_ROT_0;
+ return 9;
+
+ case DCORE2_RTR1:
+ engines[0] = GAUDI2_DCORE2_ENGINE_ID_TPC_4;
+ engines[1] = GAUDI2_DCORE2_ENGINE_ID_TPC_5;
+ return 2;
+
+ case DCORE2_RTR2:
+ engines[0] = GAUDI2_DCORE2_ENGINE_ID_TPC_2;
+ engines[1] = GAUDI2_DCORE2_ENGINE_ID_TPC_3;
+ return 2;
+
+ case DCORE2_RTR3:
+ engines[0] = GAUDI2_DCORE2_ENGINE_ID_TPC_0;
+ engines[1] = GAUDI2_DCORE2_ENGINE_ID_TPC_1;
+ return 2;
+
+ case DCORE2_RTR4:
+ case DCORE2_RTR5:
+ case DCORE2_RTR6:
+ case DCORE2_RTR7:
+ engines[0] = GAUDI2_DCORE2_ENGINE_ID_MME;
+ return 1;
+ case DCORE3_RTR0:
+ case DCORE3_RTR1:
+ case DCORE3_RTR2:
+ case DCORE3_RTR3:
+ engines[0] = GAUDI2_DCORE3_ENGINE_ID_MME;
+ return 1;
+ case DCORE3_RTR4:
+ engines[0] = GAUDI2_DCORE3_ENGINE_ID_TPC_0;
+ engines[1] = GAUDI2_DCORE3_ENGINE_ID_TPC_1;
+ return 2;
+ case DCORE3_RTR5:
+ engines[0] = GAUDI2_DCORE3_ENGINE_ID_TPC_2;
+ engines[1] = GAUDI2_DCORE3_ENGINE_ID_TPC_3;
+ return 2;
+ case DCORE3_RTR6:
+ engines[0] = GAUDI2_DCORE3_ENGINE_ID_TPC_4;
+ engines[1] = GAUDI2_DCORE3_ENGINE_ID_TPC_5;
+ return 2;
+ case DCORE3_RTR7:
+ engines[0] = GAUDI2_DCORE3_ENGINE_ID_DEC_0;
+ engines[1] = GAUDI2_DCORE3_ENGINE_ID_DEC_1;
+ engines[2] = GAUDI2_ENGINE_ID_NIC9_0;
+ engines[3] = GAUDI2_ENGINE_ID_NIC10_0;
+ engines[4] = GAUDI2_ENGINE_ID_NIC11_0;
+ engines[5] = GAUDI2_DCORE2_ENGINE_ID_EDMA_1;
+ engines[6] = GAUDI2_DCORE3_ENGINE_ID_EDMA_1;
+ engines[7] = GAUDI2_ENGINE_ID_ROT_1;
+ engines[8] = GAUDI2_ENGINE_ID_ROT_0;
+ return 9;
+ default:
+ return 0;
+ }
+}
+
static void gaudi2_razwi_unmapped_addr_hbw_printf_info(struct hl_device *hdev, u32 rtr_id,
- u64 rtr_ctrl_base_addr, bool is_write)
+ u64 rtr_ctrl_base_addr, bool is_write,
+ u64 *event_mask)
{
+ u16 engines[HL_RAZWI_MAX_NUM_OF_ENGINES_PER_RTR], num_of_eng;
u32 razwi_hi, razwi_lo;
+ u8 rd_wr_flag;
+
+ num_of_eng = gaudi2_get_razwi_initiators(rtr_id, &engines[0]);
if (is_write) {
razwi_hi = RREG32(rtr_ctrl_base_addr + DEC_RAZWI_HBW_AW_ADDR_HI);
razwi_lo = RREG32(rtr_ctrl_base_addr + DEC_RAZWI_HBW_AW_ADDR_LO);
+ rd_wr_flag = HL_RAZWI_WRITE;
/* Clear set indication */
WREG32(rtr_ctrl_base_addr + DEC_RAZWI_HBW_AW_SET, 0x1);
} else {
razwi_hi = RREG32(rtr_ctrl_base_addr + DEC_RAZWI_HBW_AR_ADDR_HI);
razwi_lo = RREG32(rtr_ctrl_base_addr + DEC_RAZWI_HBW_AR_ADDR_LO);
+ rd_wr_flag = HL_RAZWI_READ;
/* Clear set indication */
WREG32(rtr_ctrl_base_addr + DEC_RAZWI_HBW_AR_SET, 0x1);
}
+ hl_handle_razwi(hdev, (u64)razwi_hi << 32 | razwi_lo, &engines[0], num_of_eng,
+ rd_wr_flag | HL_RAZWI_HBW, event_mask);
dev_err_ratelimited(hdev->dev,
"RAZWI PSOC unmapped HBW %s error, rtr id %u, address %#llx\n",
is_write ? "WR" : "RD", rtr_id, (u64)razwi_hi << 32 | razwi_lo);
}
static void gaudi2_razwi_unmapped_addr_lbw_printf_info(struct hl_device *hdev, u32 rtr_id,
- u64 rtr_ctrl_base_addr, bool is_write)
+ u64 rtr_ctrl_base_addr, bool is_write,
+ u64 *event_mask)
{
+ u16 engines[HL_RAZWI_MAX_NUM_OF_ENGINES_PER_RTR], num_of_eng;
u32 razwi_addr;
+ u8 rd_wr_flag;
+
+ num_of_eng = gaudi2_get_razwi_initiators(rtr_id, &engines[0]);
if (is_write) {
razwi_addr = RREG32(rtr_ctrl_base_addr + DEC_RAZWI_LBW_AW_ADDR);
+ rd_wr_flag = HL_RAZWI_WRITE;
/* Clear set indication */
WREG32(rtr_ctrl_base_addr + DEC_RAZWI_LBW_AW_SET, 0x1);
} else {
razwi_addr = RREG32(rtr_ctrl_base_addr + DEC_RAZWI_LBW_AR_ADDR);
+ rd_wr_flag = HL_RAZWI_READ;
/* Clear set indication */
WREG32(rtr_ctrl_base_addr + DEC_RAZWI_LBW_AR_SET, 0x1);
}
+ hl_handle_razwi(hdev, razwi_addr, &engines[0], num_of_eng, rd_wr_flag | HL_RAZWI_LBW,
+ event_mask);
dev_err_ratelimited(hdev->dev,
"RAZWI PSOC unmapped LBW %s error, rtr id %u, address %#x\n",
is_write ? "WR" : "RD", rtr_id, razwi_addr);
}
/* PSOC RAZWI interrupt occurs only when trying to access a bad address */
-static void gaudi2_ack_psoc_razwi_event_handler(struct hl_device *hdev)
+static void gaudi2_ack_psoc_razwi_event_handler(struct hl_device *hdev, u64 *event_mask)
{
u32 hbw_aw_set, hbw_ar_set, lbw_aw_set, lbw_ar_set, rtr_id, dcore_id, dcore_rtr_id, xy,
razwi_mask_info, razwi_intr = 0;
if (hbw_aw_set)
gaudi2_razwi_unmapped_addr_hbw_printf_info(hdev, rtr_id,
- rtr_ctrl_base_addr, true);
+ rtr_ctrl_base_addr, true, event_mask);
if (hbw_ar_set)
gaudi2_razwi_unmapped_addr_hbw_printf_info(hdev, rtr_id,
- rtr_ctrl_base_addr, false);
+ rtr_ctrl_base_addr, false, event_mask);
if (lbw_aw_set)
gaudi2_razwi_unmapped_addr_lbw_printf_info(hdev, rtr_id,
- rtr_ctrl_base_addr, true);
+ rtr_ctrl_base_addr, true, event_mask);
if (lbw_ar_set)
gaudi2_razwi_unmapped_addr_lbw_printf_info(hdev, rtr_id,
- rtr_ctrl_base_addr, false);
+ rtr_ctrl_base_addr, false, event_mask);
clear:
/* Clear Interrupts only on pldm or if f/w doesn't handle interrupts */
}
static void gaudi2_handle_qm_sei_err(struct hl_device *hdev, u16 event_type,
- struct hl_eq_razwi_info *razwi_info)
+ struct hl_eq_razwi_info *razwi_info, u64 *event_mask)
{
+ enum razwi_event_sources module;
u64 qman_base;
u8 index;
qman_base = mmDCORE0_TPC0_QM_BASE +
(index / NUM_OF_TPC_PER_DCORE) * DCORE_OFFSET +
(index % NUM_OF_TPC_PER_DCORE) * DCORE_TPC_OFFSET;
+ module = RAZWI_TPC;
break;
case GAUDI2_EVENT_TPC24_AXI_ERR_RSP:
qman_base = mmDCORE0_TPC6_QM_BASE;
+ module = RAZWI_TPC;
break;
case GAUDI2_EVENT_MME0_CTRL_AXI_ERROR_RESPONSE:
case GAUDI2_EVENT_MME1_CTRL_AXI_ERROR_RESPONSE:
(GAUDI2_EVENT_MME1_CTRL_AXI_ERROR_RESPONSE -
GAUDI2_EVENT_MME0_CTRL_AXI_ERROR_RESPONSE);
qman_base = mmDCORE0_MME_QM_BASE + index * DCORE_OFFSET;
+ module = RAZWI_MME;
break;
case GAUDI2_EVENT_PDMA_CH0_AXI_ERR_RSP:
case GAUDI2_EVENT_PDMA_CH1_AXI_ERR_RSP:
index = event_type - GAUDI2_EVENT_PDMA_CH0_AXI_ERR_RSP;
qman_base = mmPDMA0_QM_BASE + index * PDMA_OFFSET;
+ module = RAZWI_PDMA;
break;
case GAUDI2_EVENT_ROTATOR0_AXI_ERROR_RESPONSE:
case GAUDI2_EVENT_ROTATOR1_AXI_ERROR_RESPONSE:
index = event_type - GAUDI2_EVENT_ROTATOR0_AXI_ERROR_RESPONSE;
qman_base = mmROT0_QM_BASE + index * ROT_OFFSET;
+ module = RAZWI_ROT;
break;
default:
return;
/* check if RAZWI happened */
if (razwi_info)
- gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_PDMA, 0, 0, razwi_info);
+ gaudi2_ack_module_razwi_event_handler(hdev, module, 0, 0, razwi_info, event_mask);
}
static void gaudi2_handle_qman_err(struct hl_device *hdev, u16 event_type)
}
static void gaudi2_handle_rot_err(struct hl_device *hdev, u8 rot_index,
- struct hl_eq_razwi_with_intr_cause *razwi_with_intr_cause)
+ struct hl_eq_razwi_with_intr_cause *razwi_with_intr_cause,
+ u64 *event_mask)
{
u64 intr_cause_data = le64_to_cpu(razwi_with_intr_cause->intr_cause.intr_cause_data);
int i;
/* check if RAZWI happened */
gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_ROT, rot_index, 0,
- &razwi_with_intr_cause->razwi_info);
+ &razwi_with_intr_cause->razwi_info, event_mask);
}
static void gaudi2_tpc_ack_interrupts(struct hl_device *hdev, u8 tpc_index, char *interrupt_name,
- struct hl_eq_razwi_with_intr_cause *razwi_with_intr_cause)
+ struct hl_eq_razwi_with_intr_cause *razwi_with_intr_cause,
+ u64 *event_mask)
{
u64 intr_cause_data = le64_to_cpu(razwi_with_intr_cause->intr_cause.intr_cause_data);
int i;
/* check if RAZWI happened */
gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_TPC, tpc_index, 0,
- &razwi_with_intr_cause->razwi_info);
+ &razwi_with_intr_cause->razwi_info, event_mask);
}
static void gaudi2_handle_dec_err(struct hl_device *hdev, u8 dec_index, const char *interrupt_name,
- struct hl_eq_razwi_info *razwi_info)
+ struct hl_eq_razwi_info *razwi_info, u64 *event_mask)
{
u32 sts_addr, sts_val, sts_clr_val = 0;
int i;
}
/* check if RAZWI happened */
- gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_DEC, dec_index, 0, razwi_info);
+ gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_DEC, dec_index, 0, razwi_info,
+ event_mask);
/* Write 1 clear errors */
WREG32(sts_addr, sts_clr_val);
}
static void gaudi2_handle_mme_err(struct hl_device *hdev, u8 mme_index, const char *interrupt_name,
- struct hl_eq_razwi_info *razwi_info)
+ struct hl_eq_razwi_info *razwi_info, u64 *event_mask)
{
u32 sts_addr, sts_val, sts_clr_addr, sts_clr_val = 0;
int i;
/* check if RAZWI happened */
for (i = MME_WRITE ; i < MME_INITIATORS_MAX ; i++)
- gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, i, razwi_info);
+ gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, i, razwi_info,
+ event_mask);
WREG32(sts_clr_addr, sts_clr_val);
}
}
static void gaudi2_handle_mme_wap_err(struct hl_device *hdev, u8 mme_index,
- struct hl_eq_razwi_info *razwi_info)
+ struct hl_eq_razwi_info *razwi_info, u64 *event_mask)
{
u32 sts_addr, sts_val, sts_clr_addr, sts_clr_val = 0;
int i;
}
/* check if RAZWI happened on WAP0/1 */
- gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, MME_WAP0, razwi_info);
- gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, MME_WAP1, razwi_info);
+ gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, MME_WAP0, razwi_info,
+ event_mask);
+ gaudi2_ack_module_razwi_event_handler(hdev, RAZWI_MME, mme_index, MME_WAP1, razwi_info,
+ event_mask);
WREG32(sts_clr_addr, sts_clr_val);
}
gaudi2_dma_core_interrupts_cause[i]);
}
-static void gaudi2_print_pcie_mstr_rr_mstr_if_razwi_info(struct hl_device *hdev)
+static void gaudi2_print_pcie_mstr_rr_mstr_if_razwi_info(struct hl_device *hdev, u64 *event_mask)
{
u32 mstr_if_base_addr = mmPCIE_MSTR_RR_MSTR_IF_RR_SHRD_HBW_BASE, razwi_happened_addr;
razwi_happened_addr = mstr_if_base_addr + RR_SHRD_HBW_AW_RAZWI_HAPPENED;
if (RREG32(razwi_happened_addr)) {
gaudi2_razwi_rr_hbw_shared_printf_info(hdev, mstr_if_base_addr, true, "PCIE", true,
- NULL);
+ NULL, GAUDI2_ENGINE_ID_PCIE, event_mask);
WREG32(razwi_happened_addr, 0x1);
}
razwi_happened_addr = mstr_if_base_addr + RR_SHRD_HBW_AR_RAZWI_HAPPENED;
if (RREG32(razwi_happened_addr)) {
gaudi2_razwi_rr_hbw_shared_printf_info(hdev, mstr_if_base_addr, false, "PCIE", true,
- NULL);
+ NULL, GAUDI2_ENGINE_ID_PCIE, event_mask);
WREG32(razwi_happened_addr, 0x1);
}
razwi_happened_addr = mstr_if_base_addr + RR_SHRD_LBW_AW_RAZWI_HAPPENED;
if (RREG32(razwi_happened_addr)) {
gaudi2_razwi_rr_lbw_shared_printf_info(hdev, mstr_if_base_addr, true, "PCIE", true,
- NULL);
+ NULL, GAUDI2_ENGINE_ID_PCIE, event_mask);
WREG32(razwi_happened_addr, 0x1);
}
razwi_happened_addr = mstr_if_base_addr + RR_SHRD_LBW_AR_RAZWI_HAPPENED;
if (RREG32(razwi_happened_addr)) {
gaudi2_razwi_rr_lbw_shared_printf_info(hdev, mstr_if_base_addr, false, "PCIE", true,
- NULL);
+ NULL, GAUDI2_ENGINE_ID_PCIE, event_mask);
WREG32(razwi_happened_addr, 0x1);
}
}
-static void gaudi2_print_pcie_addr_dec_info(struct hl_device *hdev, u64 intr_cause_data)
+static void gaudi2_print_pcie_addr_dec_info(struct hl_device *hdev, u64 intr_cause_data,
+ u64 *event_mask)
{
int i;
case PCIE_WRAP_PCIE_IC_SEI_INTR_IND_AXI_LBW_ERR_INTR_MASK:
break;
case PCIE_WRAP_PCIE_IC_SEI_INTR_IND_BAD_ACCESS_INTR_MASK:
- gaudi2_print_pcie_mstr_rr_mstr_if_razwi_info(hdev);
+ gaudi2_print_pcie_mstr_rr_mstr_if_razwi_info(hdev, event_mask);
break;
}
}
}
}
-static void gaudi2_handle_page_error(struct hl_device *hdev, u64 mmu_base, bool is_pmmu)
+static void gaudi2_handle_page_error(struct hl_device *hdev, u64 mmu_base, bool is_pmmu,
+ u64 *event_mask)
{
u32 valid, val;
u64 addr;
dev_err_ratelimited(hdev->dev, "%s page fault on va 0x%llx\n",
is_pmmu ? "PMMU" : "HMMU", addr);
+ hl_handle_page_fault(hdev, addr, 0, is_pmmu, event_mask);
WREG32(mmu_base + MMU_OFFSET(mmDCORE0_HMMU0_MMU_PAGE_ERROR_CAPTURE), 0);
}
}
static void gaudi2_handle_mmu_spi_sei_generic(struct hl_device *hdev, const char *mmu_name,
- u64 mmu_base, bool is_pmmu)
+ u64 mmu_base, bool is_pmmu, u64 *event_mask)
{
u32 spi_sei_cause, interrupt_clr = 0x0;
int i;
mmu_name, gaudi2_mmu_spi_sei[i].cause);
if (i == 0)
- gaudi2_handle_page_error(hdev, mmu_base, is_pmmu);
+ gaudi2_handle_page_error(hdev, mmu_base, is_pmmu, event_mask);
else if (i == 1)
gaudi2_handle_access_error(hdev, mmu_base, is_pmmu);
WREG32(mmu_base + MMU_INTERRUPT_CLR_OFFSET, interrupt_clr);
}
-static bool gaudi2_handle_sm_err(struct hl_device *hdev, u8 sm_index)
+static void gaudi2_handle_sm_err(struct hl_device *hdev, u8 sm_index)
{
u32 sei_cause_addr, sei_cause_val, sei_cause_cause, sei_cause_log;
u32 cq_intr_addr, cq_intr_val, cq_intr_queue_index;
- bool reset = true;
int i;
sei_cause_addr = mmDCORE0_SYNC_MNGR_GLBL_SM_SEI_CAUSE + DCORE_OFFSET * sm_index;
gaudi2_sm_sei_cause[i].cause_name,
gaudi2_sm_sei_cause[i].log_name,
sei_cause_log & gaudi2_sm_sei_cause[i].log_mask);
-
- /* Due to a potential H/W issue, do not reset upon BRESP errors */
- if (i == 2)
- reset = false;
break;
}
/* Clear CQ_INTR */
WREG32(cq_intr_addr, 0);
}
-
- return reset;
}
-static void gaudi2_handle_mmu_spi_sei_err(struct hl_device *hdev, u16 event_type)
+static void gaudi2_handle_mmu_spi_sei_err(struct hl_device *hdev, u16 event_type, u64 *event_mask)
{
bool is_pmmu = false;
char desc[32];
return;
}
- gaudi2_handle_mmu_spi_sei_generic(hdev, desc, mmu_base, is_pmmu);
+ gaudi2_handle_mmu_spi_sei_generic(hdev, desc, mmu_base, is_pmmu, event_mask);
}
{
struct hl_hw_queue *q = &hdev->kernel_queues[GAUDI2_QUEUE_ID_CPU_PQ];
- dev_err(hdev->dev, "Out of sync with FW, FW: pi=%u, ci=%u, LKD: pi=%u, ci=%u\n",
- sync_err->pi, sync_err->ci, q->pi, atomic_read(&q->ci));
+ dev_err(hdev->dev, "Out of sync with FW, FW: pi=%u, ci=%u, LKD: pi=%u, ci=%d\n",
+ le32_to_cpu(sync_err->pi), le32_to_cpu(sync_err->ci), q->pi, atomic_read(&q->ci));
}
static void gaudi2_handle_pcie_p2p_msix(struct hl_device *hdev)
struct hl_hw_queue *q = &hdev->kernel_queues[GAUDI2_QUEUE_ID_CPU_PQ];
dev_warn(hdev->dev,
- "FW reported sanity check failure, FW: pi=%u, ci=%u, LKD: pi=%u, ci=%u\n",
- sync_err->pi, sync_err->ci, q->pi, atomic_read(&q->ci));
+ "FW reported sanity check failure, FW: pi=%u, ci=%u, LKD: pi=%u, ci=%d\n",
+ le32_to_cpu(sync_err->pi), le32_to_cpu(sync_err->ci), q->pi, atomic_read(&q->ci));
}
static void hl_arc_event_handle(struct hl_device *hdev,
static void gaudi2_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_entry)
{
- u32 ctl, reset_flags = HL_DRV_RESET_HARD | HL_DRV_RESET_DELAY;
struct gaudi2_device *gaudi2 = hdev->asic_specific;
- bool reset_required = false, skip_reset = false;
+ bool reset_required = false, is_critical = false;
+ u32 ctl, reset_flags = HL_DRV_RESET_HARD;
int index, sbte_index;
u64 event_mask = 0;
u16 event_type;
reset_flags |= HL_DRV_RESET_FW_FATAL_ERR;
event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR;
reset_required = gaudi2_handle_ecc_event(hdev, event_type, &eq_entry->ecc_data);
+ is_critical = eq_entry->ecc_data.is_critical;
break;
case GAUDI2_EVENT_TPC0_QM ... GAUDI2_EVENT_PDMA1_QM:
case GAUDI2_EVENT_PDMA_CH0_AXI_ERR_RSP:
case GAUDI2_EVENT_PDMA_CH1_AXI_ERR_RSP:
reset_flags |= HL_DRV_RESET_FW_FATAL_ERR;
- gaudi2_handle_qm_sei_err(hdev, event_type, &eq_entry->razwi_info);
+ gaudi2_handle_qm_sei_err(hdev, event_type, &eq_entry->razwi_info, &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
case GAUDI2_EVENT_ROTATOR0_AXI_ERROR_RESPONSE:
case GAUDI2_EVENT_ROTATOR1_AXI_ERROR_RESPONSE:
index = event_type - GAUDI2_EVENT_ROTATOR0_AXI_ERROR_RESPONSE;
- gaudi2_handle_rot_err(hdev, index, &eq_entry->razwi_with_intr_cause);
- gaudi2_handle_qm_sei_err(hdev, event_type, NULL);
+ gaudi2_handle_rot_err(hdev, index, &eq_entry->razwi_with_intr_cause, &event_mask);
+ gaudi2_handle_qm_sei_err(hdev, event_type, NULL, &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
case GAUDI2_EVENT_TPC0_AXI_ERR_RSP ... GAUDI2_EVENT_TPC24_AXI_ERR_RSP:
index = event_type - GAUDI2_EVENT_TPC0_AXI_ERR_RSP;
gaudi2_tpc_ack_interrupts(hdev, index, "AXI_ERR_RSP",
- &eq_entry->razwi_with_intr_cause);
- gaudi2_handle_qm_sei_err(hdev, event_type, NULL);
+ &eq_entry->razwi_with_intr_cause, &event_mask);
+ gaudi2_handle_qm_sei_err(hdev, event_type, NULL, &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
case GAUDI2_EVENT_DEC0_AXI_ERR_RSPONSE ... GAUDI2_EVENT_DEC9_AXI_ERR_RSPONSE:
index = event_type - GAUDI2_EVENT_DEC0_AXI_ERR_RSPONSE;
- gaudi2_handle_dec_err(hdev, index, "AXI_ERR_RESPONSE", &eq_entry->razwi_info);
+ gaudi2_handle_dec_err(hdev, index, "AXI_ERR_RESPONSE", &eq_entry->razwi_info,
+ &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
case GAUDI2_EVENT_TPC24_KERNEL_ERR:
index = (event_type - GAUDI2_EVENT_TPC0_KERNEL_ERR) /
(GAUDI2_EVENT_TPC1_KERNEL_ERR - GAUDI2_EVENT_TPC0_KERNEL_ERR);
- gaudi2_tpc_ack_interrupts(hdev, index, "KRN_ERR", &eq_entry->razwi_with_intr_cause);
+ gaudi2_tpc_ack_interrupts(hdev, index, "KRN_ERR", &eq_entry->razwi_with_intr_cause,
+ &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
case GAUDI2_EVENT_DEC9_SPI:
index = (event_type - GAUDI2_EVENT_DEC0_SPI) /
(GAUDI2_EVENT_DEC1_SPI - GAUDI2_EVENT_DEC0_SPI);
- gaudi2_handle_dec_err(hdev, index, "SPI", &eq_entry->razwi_info);
+ gaudi2_handle_dec_err(hdev, index, "SPI", &eq_entry->razwi_info, &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
(GAUDI2_EVENT_MME1_CTRL_AXI_ERROR_RESPONSE -
GAUDI2_EVENT_MME0_CTRL_AXI_ERROR_RESPONSE);
gaudi2_handle_mme_err(hdev, index,
- "CTRL_AXI_ERROR_RESPONSE", &eq_entry->razwi_info);
- gaudi2_handle_qm_sei_err(hdev, event_type, NULL);
+ "CTRL_AXI_ERROR_RESPONSE", &eq_entry->razwi_info, &event_mask);
+ gaudi2_handle_qm_sei_err(hdev, event_type, NULL, &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
index = (event_type - GAUDI2_EVENT_MME0_QMAN_SW_ERROR) /
(GAUDI2_EVENT_MME1_QMAN_SW_ERROR -
GAUDI2_EVENT_MME0_QMAN_SW_ERROR);
- gaudi2_handle_mme_err(hdev, index, "QMAN_SW_ERROR", &eq_entry->razwi_info);
+ gaudi2_handle_mme_err(hdev, index, "QMAN_SW_ERROR", &eq_entry->razwi_info,
+ &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
index = (event_type - GAUDI2_EVENT_MME0_WAP_SOURCE_RESULT_INVALID) /
(GAUDI2_EVENT_MME1_WAP_SOURCE_RESULT_INVALID -
GAUDI2_EVENT_MME0_WAP_SOURCE_RESULT_INVALID);
- gaudi2_handle_mme_wap_err(hdev, index, &eq_entry->razwi_info);
+ gaudi2_handle_mme_wap_err(hdev, index, &eq_entry->razwi_info, &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
case GAUDI2_EVENT_PCIE_ADDR_DEC_ERR:
gaudi2_print_pcie_addr_dec_info(hdev,
- le64_to_cpu(eq_entry->intr_cause.intr_cause_data));
+ le64_to_cpu(eq_entry->intr_cause.intr_cause_data), &event_mask);
reset_flags |= HL_DRV_RESET_FW_FATAL_ERR;
event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR;
break;
case GAUDI2_EVENT_HMMU_0_AXI_ERR_RSP ... GAUDI2_EVENT_HMMU_12_AXI_ERR_RSP:
case GAUDI2_EVENT_PMMU0_PAGE_FAULT_WR_PERM ... GAUDI2_EVENT_PMMU0_SECURITY_ERROR:
case GAUDI2_EVENT_PMMU_AXI_ERR_RSP_0:
- gaudi2_handle_mmu_spi_sei_err(hdev, event_type);
+ gaudi2_handle_mmu_spi_sei_err(hdev, event_type, &event_mask);
reset_flags |= HL_DRV_RESET_FW_FATAL_ERR;
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
break;
case GAUDI2_EVENT_PSOC63_RAZWI_OR_PID_MIN_MAX_INTERRUPT:
- gaudi2_ack_psoc_razwi_event_handler(hdev);
+ gaudi2_ack_psoc_razwi_event_handler(hdev, &event_mask);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
case GAUDI2_EVENT_SM0_AXI_ERROR_RESPONSE ... GAUDI2_EVENT_SM3_AXI_ERROR_RESPONSE:
index = event_type - GAUDI2_EVENT_SM0_AXI_ERROR_RESPONSE;
- skip_reset = !gaudi2_handle_sm_err(hdev, index);
+ gaudi2_handle_sm_err(hdev, index);
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
event_mask |= HL_NOTIFIER_EVENT_USER_ENGINE_ERR;
break;
+ case GAUDI2_EVENT_CPU_FP32_NOT_SUPPORTED:
+ event_mask |= HL_NOTIFIER_EVENT_GENERAL_HW_ERR;
+ is_critical = true;
+ break;
+
default:
if (gaudi2_irq_map_table[event_type].valid)
dev_err_ratelimited(hdev->dev, "Cannot find handler for event %d\n",
event_type);
}
- if ((gaudi2_irq_map_table[event_type].reset || reset_required) && !skip_reset)
+ if ((gaudi2_irq_map_table[event_type].reset || reset_required) &&
+ (hdev->hard_reset_on_fw_events ||
+ (hdev->asic_prop.fw_security_enabled && is_critical)))
goto reset_device;
/* Send unmask irq only for interrupts not classified as MSG */
return;
reset_device:
- if (hdev->hard_reset_on_fw_events) {
- hl_device_reset(hdev, reset_flags);
- event_mask |= HL_NOTIFIER_EVENT_DEVICE_RESET;
+ if (hdev->asic_prop.fw_security_enabled && is_critical) {
+ reset_flags |= HL_DRV_RESET_BYPASS_REQ_TO_FW;
+ event_mask |= HL_NOTIFIER_EVENT_DEVICE_UNAVAILABLE;
} else {
- if (!gaudi2_irq_map_table[event_type].msg)
- hl_fw_unmask_irq(hdev, event_type);
+ reset_flags |= HL_DRV_RESET_DELAY;
}
+ event_mask |= HL_NOTIFIER_EVENT_DEVICE_RESET;
+ hl_device_cond_reset(hdev, reset_flags, event_mask);
+}
- if (event_mask)
- hl_notifier_event_send_all(hdev, event_mask);
+static int gaudi2_memset_memory_chunk_using_edma_qm(struct hl_device *hdev,
+ struct packet_lin_dma *lin_dma_pkt, dma_addr_t pkt_dma_addr,
+ u32 hw_queue_id, u32 size, u64 addr, u32 val)
+{
+ u32 ctl, pkt_size;
+ int rc = 0;
+
+ ctl = FIELD_PREP(GAUDI2_PKT_CTL_OPCODE_MASK, PACKET_LIN_DMA);
+ ctl |= FIELD_PREP(GAUDI2_PKT_LIN_DMA_CTL_MEMSET_MASK, 1);
+ ctl |= FIELD_PREP(GAUDI2_PKT_LIN_DMA_CTL_WRCOMP_MASK, 1);
+ ctl |= FIELD_PREP(GAUDI2_PKT_CTL_EB_MASK, 1);
+
+ lin_dma_pkt->ctl = cpu_to_le32(ctl);
+ lin_dma_pkt->src_addr = cpu_to_le64(val);
+ lin_dma_pkt->dst_addr = cpu_to_le64(addr);
+ lin_dma_pkt->tsize = cpu_to_le32(size);
+
+ pkt_size = sizeof(struct packet_lin_dma);
+
+ rc = hl_hw_queue_send_cb_no_cmpl(hdev, hw_queue_id, pkt_size, pkt_dma_addr);
+ if (rc)
+ dev_err(hdev->dev, "Failed to send lin dma packet to H/W queue %d\n",
+ hw_queue_id);
+
+ return rc;
}
static int gaudi2_memset_device_memory(struct hl_device *hdev, u64 addr, u64 size, u64 val)
{
- struct asic_fixed_properties *prop = &hdev->asic_prop;
+ u32 edma_queues_id[] = {GAUDI2_QUEUE_ID_DCORE0_EDMA_0_0,
+ GAUDI2_QUEUE_ID_DCORE1_EDMA_0_0,
+ GAUDI2_QUEUE_ID_DCORE2_EDMA_0_0,
+ GAUDI2_QUEUE_ID_DCORE3_EDMA_0_0};
+ u32 chunk_size, dcore, edma_idx, sob_offset, sob_addr, comp_val,
+ old_mmubp, mmubp, num_of_pkts, busy, pkt_size;
u64 comp_addr, cur_addr = addr, end_addr = addr + size;
- u32 chunk_size, busy, dcore, edma_idx, sob_offset, sob_addr, comp_val, edma_commit;
- u32 old_mmubp, mmubp;
- int rc = 0;
+ struct asic_fixed_properties *prop = &hdev->asic_prop;
+ void *lin_dma_pkts_arr;
+ dma_addr_t pkt_dma_addr;
+ int rc = 0, dma_num = 0;
+
+ if (prop->edma_enabled_mask == 0) {
+ dev_info(hdev->dev, "non of the EDMA engines is enabled - skip dram scrubbing\n");
+ return -EIO;
+ }
sob_offset = hdev->asic_prop.first_available_user_sob[0] * 4;
sob_addr = mmDCORE0_SYNC_MNGR_OBJS_SOB_OBJ_0 + sob_offset;
comp_addr = CFG_BASE + sob_addr;
comp_val = FIELD_PREP(DCORE0_SYNC_MNGR_OBJS_SOB_OBJ_INC_MASK, 1) |
FIELD_PREP(DCORE0_SYNC_MNGR_OBJS_SOB_OBJ_VAL_MASK, 1);
-
- edma_commit = FIELD_PREP(ARC_FARM_KDMA_CTX_COMMIT_LIN_MASK, 1) |
- FIELD_PREP(ARC_FARM_KDMA_CTX_COMMIT_MEM_SET_MASK, 1) |
- FIELD_PREP(ARC_FARM_KDMA_CTX_COMMIT_WR_COMP_EN_MASK, 1);
mmubp = FIELD_PREP(ARC_FARM_KDMA_CTX_AXUSER_HB_MMU_BP_WR_MASK, 1) |
FIELD_PREP(ARC_FARM_KDMA_CTX_AXUSER_HB_MMU_BP_RD_MASK, 1);
- if (prop->edma_enabled_mask == 0) {
- dev_info(hdev->dev, "non of the EDMA engines is enabled - skip dram scrubbing\n");
- return -EIO;
- }
+ /* Calculate how many lin dma pkts we'll need */
+ num_of_pkts = div64_u64(round_up(size, SZ_2G), SZ_2G);
+ pkt_size = sizeof(struct packet_lin_dma);
+
+ lin_dma_pkts_arr = hl_asic_dma_alloc_coherent(hdev, pkt_size * num_of_pkts,
+ &pkt_dma_addr, GFP_KERNEL);
+ if (!lin_dma_pkts_arr)
+ return -ENOMEM;
/*
* set mmu bypass for the scrubbing - all ddmas are configured the same so save
* only the first one to restore later
+ * also set the sob addr for all edma cores for completion.
+ * set QM as trusted to allow it to access physical address with MMU bp.
*/
old_mmubp = RREG32(mmDCORE0_EDMA0_CORE_CTX_AXUSER_HB_MMU_BP);
for (dcore = 0 ; dcore < NUM_OF_DCORES ; dcore++) {
WREG32(mmDCORE0_EDMA0_CORE_CTX_AXUSER_HB_MMU_BP +
edma_offset, mmubp);
+ WREG32(mmDCORE0_EDMA0_CORE_CTX_WR_COMP_ADDR_LO + edma_offset,
+ lower_32_bits(comp_addr));
+ WREG32(mmDCORE0_EDMA0_CORE_CTX_WR_COMP_ADDR_HI + edma_offset,
+ upper_32_bits(comp_addr));
+ WREG32(mmDCORE0_EDMA0_CORE_CTX_WR_COMP_WDATA + edma_offset,
+ comp_val);
+ gaudi2_qman_set_test_mode(hdev,
+ edma_queues_id[dcore] + 4 * edma_idx, true);
}
}
- while (cur_addr < end_addr) {
- int dma_num = 0;
+ WREG32(sob_addr, 0);
- WREG32(sob_addr, 0);
+ while (cur_addr < end_addr) {
for (dcore = 0 ; dcore < NUM_OF_DCORES ; dcore++) {
for (edma_idx = 0 ; edma_idx < NUM_OF_EDMA_PER_DCORE ; edma_idx++) {
- u32 edma_offset = dcore * DCORE_OFFSET +
- edma_idx * DCORE_EDMA_OFFSET;
u32 edma_bit = dcore * NUM_OF_EDMA_PER_DCORE + edma_idx;
if (!(prop->edma_enabled_mask & BIT(edma_bit)))
chunk_size = min_t(u64, SZ_2G, end_addr - cur_addr);
- WREG32(mmDCORE0_EDMA0_CORE_CTX_SRC_BASE_LO + edma_offset,
- lower_32_bits(val));
- WREG32(mmDCORE0_EDMA0_CORE_CTX_SRC_BASE_HI + edma_offset,
- upper_32_bits(val));
-
- WREG32(mmDCORE0_EDMA0_CORE_CTX_DST_BASE_LO + edma_offset,
- lower_32_bits(cur_addr));
- WREG32(mmDCORE0_EDMA0_CORE_CTX_DST_BASE_HI + edma_offset,
- upper_32_bits(cur_addr));
-
- WREG32(mmDCORE0_EDMA0_CORE_CTX_WR_COMP_ADDR_LO + edma_offset,
- lower_32_bits(comp_addr));
- WREG32(mmDCORE0_EDMA0_CORE_CTX_WR_COMP_ADDR_HI + edma_offset,
- upper_32_bits(comp_addr));
- WREG32(mmDCORE0_EDMA0_CORE_CTX_WR_COMP_WDATA + edma_offset,
- comp_val);
-
- WREG32(mmDCORE0_EDMA0_CORE_CTX_DST_TSIZE_0 + edma_offset,
- chunk_size);
- WREG32(mmDCORE0_EDMA0_CORE_CTX_COMMIT + edma_offset, edma_commit);
+ rc = gaudi2_memset_memory_chunk_using_edma_qm(hdev,
+ (struct packet_lin_dma *)lin_dma_pkts_arr + dma_num,
+ pkt_dma_addr + dma_num * pkt_size,
+ edma_queues_id[dcore] + edma_idx * 4,
+ chunk_size, cur_addr, val);
+ if (rc)
+ goto end;
dma_num++;
-
cur_addr += chunk_size;
-
if (cur_addr == end_addr)
- goto poll;
+ break;
}
}
-poll:
- rc = hl_poll_timeout(hdev, sob_addr, busy, (busy == dma_num), 1000, 1000000);
- if (rc) {
- dev_err(hdev->dev, "DMA Timeout during HBM scrubbing\n");
- goto end;
- }
+ }
+
+ rc = hl_poll_timeout(hdev, sob_addr, busy, (busy == dma_num), 1000, 1000000);
+ if (rc) {
+ dev_err(hdev->dev, "DMA Timeout during HBM scrubbing\n");
+ goto end;
}
end:
for (dcore = 0 ; dcore < NUM_OF_DCORES ; dcore++) {
continue;
WREG32(mmDCORE0_EDMA0_CORE_CTX_AXUSER_HB_MMU_BP + edma_offset, old_mmubp);
+ WREG32(mmDCORE0_EDMA0_CORE_CTX_WR_COMP_ADDR_LO + edma_offset, 0);
+ WREG32(mmDCORE0_EDMA0_CORE_CTX_WR_COMP_ADDR_HI + edma_offset, 0);
+ WREG32(mmDCORE0_EDMA0_CORE_CTX_WR_COMP_WDATA + edma_offset, 0);
+ gaudi2_qman_set_test_mode(hdev,
+ edma_queues_id[dcore] + 4 * edma_idx, false);
}
}
WREG32(sob_addr, 0);
+ hl_asic_dma_free_coherent(hdev, pkt_size * num_of_pkts, lin_dma_pkts_arr, pkt_dma_addr);
+
return rc;
}
gaudi2_memset_device_lbw(hdev, cq_lbw_data_addr, size, 0);
gaudi2_memset_device_lbw(hdev, cq_base_l_addr, size, 0);
gaudi2_memset_device_lbw(hdev, cq_base_h_addr, size, 0);
+ gaudi2_memset_device_lbw(hdev, cq_size_addr, size, 0);
cq_lbw_l_addr = mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_L_0 + DCORE_OFFSET;
cq_lbw_h_addr = mmDCORE0_SYNC_MNGR_GLBL_LBW_ADDR_H_0 + DCORE_OFFSET;
if (gaudi2_get_mmu_base(hdev, mmu_id, &mmu_base))
return;
- gaudi2_handle_page_error(hdev, mmu_base, is_pmmu);
+ gaudi2_handle_page_error(hdev, mmu_base, is_pmmu, NULL);
gaudi2_handle_access_error(hdev, mmu_base, is_pmmu);
}
{
struct gaudi2_device *gaudi2 = hdev->asic_specific;
- if (!(gaudi2->hw_cap_initialized & HW_CAP_CPU_Q) || hdev->fw_major_version < 37)
+ if (!(gaudi2->hw_cap_initialized & HW_CAP_CPU_Q))
return 0;
- /* TODO: add check for FW version using minor ver once it's known */
return hl_fw_send_device_activity(hdev, open);
}
#define GAUDI2_CPU_TIMEOUT_USEC 30000000 /* 30s */
-#define GAUDI2_FPGA_CPU_TIMEOUT 100000000 /* 100s */
-
#define NUMBER_OF_PDMA_QUEUES 2
#define NUMBER_OF_EDMA_QUEUES 8
#define NUMBER_OF_MME_QUEUES 4
{mmNIC0_QM_ARC_AUX0_CLUSTER_NUM, mmNIC0_QM_ARC_AUX0_WAKE_UP_EVENT},
{mmNIC0_QM_ARC_AUX0_ARC_RST_REQ, mmNIC0_QM_ARC_AUX0_CID_OFFSET_7},
{mmNIC0_QM_ARC_AUX0_SCRATCHPAD_0, mmNIC0_QM_ARC_AUX0_INFLIGHT_LBU_RD_CNT},
+ {mmNIC0_QM_ARC_AUX0_CBU_EARLY_BRESP_EN, mmNIC0_QM_ARC_AUX0_CBU_EARLY_BRESP_EN},
{mmNIC0_QM_ARC_AUX0_LBU_EARLY_BRESP_EN, mmNIC0_QM_ARC_AUX0_LBU_EARLY_BRESP_EN},
{mmNIC0_QM_ARC_AUX0_DCCM_QUEUE_BASE_ADDR_0, mmNIC0_QM_ARC_AUX0_DCCM_QUEUE_ALERT_MSG},
{mmNIC0_QM_ARC_AUX0_DCCM_Q_PUSH_FIFO_CNT, mmNIC0_QM_ARC_AUX0_QMAN_ARC_CQ_SHADOW_CI},
{
struct hl_hw_queue *q = &hdev->kernel_queues[GOYA_QUEUE_ID_CPU_PQ];
- dev_err(hdev->dev, "Out of sync with FW, FW: pi=%u, ci=%u, LKD: pi=%u, ci=%u\n",
- sync_err->pi, sync_err->ci, q->pi, atomic_read(&q->ci));
+ dev_err(hdev->dev, "Out of sync with FW, FW: pi=%u, ci=%u, LKD: pi=%u, ci=%d\n",
+ le32_to_cpu(sync_err->pi), le32_to_cpu(sync_err->ci), q->pi, atomic_read(&q->ci));
}
static void goya_print_irq_info(struct hl_device *hdev, u16 event_type,
GAUDI2_EVENT_CPU11_STATUS_NIC11_ENG0 = 1317,
GAUDI2_EVENT_CPU11_STATUS_NIC11_ENG1 = 1318,
GAUDI2_EVENT_ARC_DCCM_FULL = 1319,
+ GAUDI2_EVENT_CPU_FP32_NOT_SUPPORTED = 1320,
GAUDI2_EVENT_SIZE,
};
/* SPDX-License-Identifier: GPL-2.0
*
- * Copyright 2018-2021 HabanaLabs, Ltd.
+ * Copyright 2018-2022 HabanaLabs, Ltd.
* All Rights Reserved.
*
*/
.msg = 1, .reset = 0, .name = "STATUS_NIC11_ENG1" },
{ .fc_id = 1319, .cpu_id = 625, .valid = 1,
.msg = 1, .reset = 0, .name = "ARC_DCCM_FULL" },
+ { .fc_id = 1320, .cpu_id = 626, .valid = 1,
+ .msg = 1, .reset = 1, .name = "FP32_NOT_SUPPORTED" },
};
#endif /* __GAUDI2_ASYNC_IDS_MAP_EVENTS_EXT_H_ */
#define PCI_CONFIG_ELBI_STS_MASK (PCI_CONFIG_ELBI_STS_ERR | \
PCI_CONFIG_ELBI_STS_DONE)
+enum hl_revision_id {
+ /* PCI revision ID 0 is not legal */
+ REV_ID_INVALID = 0x00,
+ REV_ID_A = 0x01,
+ REV_ID_B = 0x02,
+};
+
#endif /* INCLUDE_PCI_GENERAL_H_ */
.attrs = mid_att_compass
};
-static int hmc6352_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int hmc6352_probe(struct i2c_client *client)
{
int res;
.driver = {
.name = "hmc6352",
},
- .probe = hmc6352_probe,
+ .probe_new = hmc6352_probe,
.remove = hmc6352_remove,
.id_table = hmc6352_id,
};
u8 regs[NUM_REGS];
};
-static int ics932s401_probe(struct i2c_client *client,
- const struct i2c_device_id *id);
+static int ics932s401_probe(struct i2c_client *client);
static int ics932s401_detect(struct i2c_client *client,
struct i2c_board_info *info);
static void ics932s401_remove(struct i2c_client *client);
.driver = {
.name = "ics932s401",
},
- .probe = ics932s401_probe,
+ .probe_new = ics932s401_probe,
.remove = ics932s401_remove,
.id_table = ics932s401_id,
.detect = ics932s401_detect,
return 0;
}
-static int ics932s401_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ics932s401_probe(struct i2c_client *client)
{
struct ics932s401_data *data;
int err;
* I2C layer
*/
-static int isl29003_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int isl29003_probe(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
struct isl29003_data *data;
.name = ISL29003_DRV_NAME,
.pm = ISL29003_PM_OPS,
},
- .probe = isl29003_probe,
+ .probe_new = isl29003_probe,
.remove = isl29003_remove,
.id_table = isl29003_id,
};
return 0;
}
-static int isl29020_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int isl29020_probe(struct i2c_client *client)
{
int res;
.name = "isl29020",
.pm = ISL29020_PM_OPS,
},
- .probe = isl29020_probe,
+ .probe_new = isl29020_probe,
.remove = isl29020_remove,
.id_table = isl29020_id,
};
MODULE_DEVICE_TABLE(of, lis3lv02d_i2c_dt_ids);
#endif
-static int lis3lv02d_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int lis3lv02d_i2c_probe(struct i2c_client *client)
{
int ret = 0;
struct lis3lv02d_platform_data *pdata = client->dev.platform_data;
.pm = &lis3_pm_ops,
.of_match_table = of_match_ptr(lis3lv02d_i2c_dt_ids),
},
- .probe = lis3lv02d_i2c_probe,
+ .probe_new = lis3lv02d_i2c_probe,
.remove = lis3lv02d_i2c_remove,
.id_table = lis3lv02d_id,
};
return ret;
}
+#define GFX_MEMORY_READY_TIMEOUT 200 /* timeout in milliseconds */
+
static int mei_gfx_memory_ready(struct mei_cl_device *cldev)
{
struct mkhi_gfx_mem_ready req = {0};
- unsigned int mode = MEI_CL_IO_TX_INTERNAL;
+ unsigned int mode = MEI_CL_IO_TX_INTERNAL | MEI_CL_IO_TX_BLOCKING;
req.hdr.group_id = MKHI_GROUP_ID_GFX;
req.hdr.command = MKHI_GFX_MEMORY_READY_CMD_REQ;
req.flags = MKHI_GFX_MEM_READY_PXP_ALLOWED;
dev_dbg(&cldev->dev, "Sending memory ready command\n");
- return __mei_cl_send(cldev->cl, (u8 *)&req, sizeof(req), 0, mode);
+ return __mei_cl_send_timeout(cldev->cl, (u8 *)&req, sizeof(req), 0,
+ mode, GFX_MEMORY_READY_TIMEOUT);
}
static void mei_mkhi_fix(struct mei_cl_device *cldev)
if (cldev->bus->pxp_mode == MEI_DEV_PXP_INIT) {
ret = mei_gfx_memory_ready(cldev);
- if (ret < 0)
+ if (ret < 0) {
dev_err(&cldev->dev, "memory ready command failed %d\n", ret);
- else
+ } else {
dev_dbg(&cldev->dev, "memory ready command sent\n");
+ cldev->bus->pxp_mode = MEI_DEV_PXP_SETUP;
+ }
/* we go to reset after that */
- cldev->bus->pxp_mode = MEI_DEV_PXP_SETUP;
goto out;
}
ssize_t __mei_cl_send(struct mei_cl *cl, const u8 *buf, size_t length, u8 vtag,
unsigned int mode)
{
+ return __mei_cl_send_timeout(cl, buf, length, vtag, mode, MAX_SCHEDULE_TIMEOUT);
+}
+
+/**
+ * __mei_cl_send_timeout - internal client send (write)
+ *
+ * @cl: host client
+ * @buf: buffer to send
+ * @length: buffer length
+ * @vtag: virtual tag
+ * @mode: sending mode
+ * @timeout: send timeout in milliseconds.
+ * effective only for blocking writes: the MEI_CL_IO_TX_BLOCKING mode bit is set.
+ * set timeout to the MAX_SCHEDULE_TIMEOUT to maixum allowed wait.
+ *
+ * Return: written size bytes or < 0 on error
+ */
+ssize_t __mei_cl_send_timeout(struct mei_cl *cl, const u8 *buf, size_t length, u8 vtag,
+ unsigned int mode, unsigned long timeout)
+{
struct mei_device *bus;
struct mei_cl_cb *cb;
ssize_t rets;
cb->buf.size = 0;
}
- rets = mei_cl_write(cl, cb);
+ rets = mei_cl_write(cl, cb, timeout);
if (mode & MEI_CL_IO_SGL && rets == 0)
rets = length;
*
* @cl: host client
* @cb: write callback with filled data
+ * @timeout: send timeout in milliseconds.
+ * effective only for blocking writes: the cb->blocking is set.
+ * set timeout to the MAX_SCHEDULE_TIMEOUT to maixum allowed wait.
*
* Return: number of bytes sent on success, <0 on failure.
*/
-ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb)
+ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb, unsigned long timeout)
{
struct mei_device *dev;
struct mei_msg_data *buf;
if (blocking && cl->writing_state != MEI_WRITE_COMPLETE) {
mutex_unlock(&dev->device_lock);
- rets = wait_event_interruptible(cl->tx_wait,
- cl->writing_state == MEI_WRITE_COMPLETE ||
- (!mei_cl_is_connected(cl)));
+ rets = wait_event_interruptible_timeout(cl->tx_wait,
+ cl->writing_state == MEI_WRITE_COMPLETE ||
+ (!mei_cl_is_connected(cl)),
+ msecs_to_jiffies(timeout));
mutex_lock(&dev->device_lock);
+ /* clean all queue on timeout as something fatal happened */
+ if (rets == 0) {
+ rets = -ETIME;
+ mei_io_tx_list_free_cl(&dev->write_list, cl, NULL);
+ mei_io_tx_list_free_cl(&dev->write_waiting_list, cl, NULL);
+ }
/* wait_event_interruptible returns -ERESTARTSYS */
+ if (rets > 0)
+ rets = 0;
if (rets) {
if (signal_pending(current))
rets = -EINTR;
int mei_cl_irq_connect(struct mei_cl *cl, struct mei_cl_cb *cb,
struct list_head *cmpl_list);
int mei_cl_read_start(struct mei_cl *cl, size_t length, const struct file *fp);
-ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb);
+ssize_t mei_cl_write(struct mei_cl *cl, struct mei_cl_cb *cb, unsigned long timeout);
int mei_cl_irq_write(struct mei_cl *cl, struct mei_cl_cb *cb,
struct list_head *cmpl_list);
goto out;
}
- rets = mei_cl_write(cl, cb);
+ rets = mei_cl_write(cl, cb, MAX_SCHEDULE_TIMEOUT);
out:
mutex_unlock(&dev->device_lock);
return rets;
void mei_cl_bus_dev_fixup(struct mei_cl_device *dev);
ssize_t __mei_cl_send(struct mei_cl *cl, const u8 *buf, size_t length, u8 vtag,
unsigned int mode);
+ssize_t __mei_cl_send_timeout(struct mei_cl *cl, const u8 *buf, size_t length, u8 vtag,
+ unsigned int mode, unsigned long timeout);
ssize_t __mei_cl_recv(struct mei_cl *cl, u8 *buf, size_t length, u8 *vtag,
unsigned int mode, unsigned long timeout);
bool mei_cl_bus_rx_event(struct mei_cl *cl);
return 0;
}
+/**
+ * get_dvsec_vendor0() - Find a related PCI device (function 0)
+ * @dev: PCI device to match
+ * @dev0: The PCI device (function 0) found
+ * @out_pos: The position of PCI device (function 0)
+ *
+ * Returns 0 on success, negative on failure.
+ *
+ * NOTE: If it's successful, the reference of dev0 is increased,
+ * so after using it, the callers must call pci_dev_put() to give
+ * up the reference.
+ */
static int get_dvsec_vendor0(struct pci_dev *dev, struct pci_dev **dev0,
int *out_pos)
{
dev = get_function_0(dev);
if (!dev)
return -1;
+ } else {
+ dev = pci_dev_get(dev);
}
pos = find_dvsec(dev, OCXL_DVSEC_VENDOR_ID);
- if (!pos)
+ if (!pos) {
+ pci_dev_put(dev);
return -1;
+ }
*dev0 = dev;
*out_pos = pos;
return 0;
pci_read_config_dword(dev0, pos + OCXL_DVSEC_VENDOR_RESET_RELOAD,
&reset_reload);
+ pci_dev_put(dev0);
*val = !!(reset_reload & BIT(0));
return 0;
}
reset_reload &= ~BIT(0);
pci_write_config_dword(dev0, pos + OCXL_DVSEC_VENDOR_RESET_RELOAD,
reset_reload);
+ pci_dev_put(dev0);
return 0;
}
goto err_put;
rc = device_register(&info->dev);
- if (rc)
- goto err_put;
+ if (rc) {
+ free_minor(info);
+ put_device(&info->dev);
+ return rc;
+ }
rc = ocxl_sysfs_register_afu(info);
if (rc)
device_unregister(&info->dev);
}
-static char *ocxl_devnode(struct device *dev, umode_t *mode)
+static char *ocxl_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "ocxl/%s", dev_name(dev));
}
if ((cb & (GRU_HANDLE_STRIDE - 1)) || ucbnum >= GRU_NUM_CB)
return -EINVAL;
+again:
gts = gru_find_lock_gts(cb);
if (!gts)
return -EINVAL;
if (ucbnum >= gts->ts_cbr_au_count * GRU_CBR_AU_SIZE)
goto exit;
- gru_check_context_placement(gts);
+ if (gru_check_context_placement(gts)) {
+ gru_unlock_gts(gts);
+ gru_unload_context(gts, 1);
+ goto again;
+ }
/*
* CCH may contain stale data if ts_force_cch_reload is set.
} else {
gts->ts_user_blade_id = req.val1;
gts->ts_user_chiplet_id = req.val0;
- gru_check_context_placement(gts);
+ if (gru_check_context_placement(gts)) {
+ gru_unlock_gts(gts);
+ gru_unload_context(gts, 1);
+ return ret;
+ }
}
break;
case sco_gseg_owner:
* chiplet. Misassignment can occur if the process migrates to a different
* blade or if the user changes the selected blade/chiplet.
*/
-void gru_check_context_placement(struct gru_thread_state *gts)
+int gru_check_context_placement(struct gru_thread_state *gts)
{
struct gru_state *gru;
+ int ret = 0;
/*
* If the current task is the context owner, verify that the
* references. Pthread apps use non-owner references to the CBRs.
*/
gru = gts->ts_gru;
+ /*
+ * If gru or gts->ts_tgid_owner isn't initialized properly, return
+ * success to indicate that the caller does not need to unload the
+ * gru context.The caller is responsible for their inspection and
+ * reinitialization if needed.
+ */
if (!gru || gts->ts_tgid_owner != current->tgid)
- return;
+ return ret;
if (!gru_check_chiplet_assignment(gru, gts)) {
STAT(check_context_unload);
- gru_unload_context(gts, 1);
+ ret = -EINVAL;
} else if (gru_retarget_intr(gts)) {
STAT(check_context_retarget_intr);
}
+
+ return ret;
}
mutex_lock(>s->ts_ctxlock);
preempt_disable();
- gru_check_context_placement(gts);
+ if (gru_check_context_placement(gts)) {
+ preempt_enable();
+ mutex_unlock(>s->ts_ctxlock);
+ gru_unload_context(gts, 1);
+ return VM_FAULT_NOPAGE;
+ }
if (!gts->ts_gru) {
STAT(load_user_context);
extern int gru_user_unload_context(unsigned long arg);
extern int gru_get_exception_detail(unsigned long arg);
extern int gru_set_context_option(unsigned long address);
-extern void gru_check_context_placement(struct gru_thread_state *gts);
+extern int gru_check_context_placement(struct gru_thread_state *gts);
extern int gru_cpu_fault_map_id(void);
extern struct vm_area_struct *gru_find_vma(unsigned long vaddr);
extern void gru_flush_all_tlb(struct gru_state *gru);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Ampere Computing SoC's SMpro Error Monitoring Driver
+ *
+ * Copyright (c) 2022, Ampere Computing LLC
+ *
+ */
+
+#include <linux/i2c.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+/* GPI RAS Error Registers */
+#define GPI_RAS_ERR 0x7E
+
+/* Core and L2C Error Registers */
+#define CORE_CE_ERR_CNT 0x80
+#define CORE_CE_ERR_LEN 0x81
+#define CORE_CE_ERR_DATA 0x82
+#define CORE_UE_ERR_CNT 0x83
+#define CORE_UE_ERR_LEN 0x84
+#define CORE_UE_ERR_DATA 0x85
+
+/* Memory Error Registers */
+#define MEM_CE_ERR_CNT 0x90
+#define MEM_CE_ERR_LEN 0x91
+#define MEM_CE_ERR_DATA 0x92
+#define MEM_UE_ERR_CNT 0x93
+#define MEM_UE_ERR_LEN 0x94
+#define MEM_UE_ERR_DATA 0x95
+
+/* RAS Error/Warning Registers */
+#define ERR_SMPRO_TYPE 0xA0
+#define ERR_PMPRO_TYPE 0xA1
+#define ERR_SMPRO_INFO_LO 0xA2
+#define ERR_SMPRO_INFO_HI 0xA3
+#define ERR_SMPRO_DATA_LO 0xA4
+#define ERR_SMPRO_DATA_HI 0xA5
+#define WARN_SMPRO_INFO_LO 0xAA
+#define WARN_SMPRO_INFO_HI 0xAB
+#define ERR_PMPRO_INFO_LO 0xA6
+#define ERR_PMPRO_INFO_HI 0xA7
+#define ERR_PMPRO_DATA_LO 0xA8
+#define ERR_PMPRO_DATA_HI 0xA9
+#define WARN_PMPRO_INFO_LO 0xAC
+#define WARN_PMPRO_INFO_HI 0xAD
+
+/* PCIE Error Registers */
+#define PCIE_CE_ERR_CNT 0xC0
+#define PCIE_CE_ERR_LEN 0xC1
+#define PCIE_CE_ERR_DATA 0xC2
+#define PCIE_UE_ERR_CNT 0xC3
+#define PCIE_UE_ERR_LEN 0xC4
+#define PCIE_UE_ERR_DATA 0xC5
+
+/* Other Error Registers */
+#define OTHER_CE_ERR_CNT 0xD0
+#define OTHER_CE_ERR_LEN 0xD1
+#define OTHER_CE_ERR_DATA 0xD2
+#define OTHER_UE_ERR_CNT 0xD8
+#define OTHER_UE_ERR_LEN 0xD9
+#define OTHER_UE_ERR_DATA 0xDA
+
+/* Event Data Registers */
+#define VRD_WARN_FAULT_EVENT_DATA 0x78
+#define VRD_HOT_EVENT_DATA 0x79
+#define DIMM_HOT_EVENT_DATA 0x7A
+
+#define MAX_READ_BLOCK_LENGTH 48
+
+#define RAS_SMPRO_ERR 0
+#define RAS_PMPRO_ERR 1
+
+enum RAS_48BYTES_ERR_TYPES {
+ CORE_CE_ERR,
+ CORE_UE_ERR,
+ MEM_CE_ERR,
+ MEM_UE_ERR,
+ PCIE_CE_ERR,
+ PCIE_UE_ERR,
+ OTHER_CE_ERR,
+ OTHER_UE_ERR,
+ NUM_48BYTES_ERR_TYPE,
+};
+
+struct smpro_error_hdr {
+ u8 count; /* Number of the RAS errors */
+ u8 len; /* Number of data bytes */
+ u8 data; /* Start of 48-byte data */
+ u8 max_cnt; /* Max num of errors */
+};
+
+/*
+ * Included Address of registers to get Count, Length of data and Data
+ * of the 48 bytes error data
+ */
+static struct smpro_error_hdr smpro_error_table[] = {
+ [CORE_CE_ERR] = {
+ .count = CORE_CE_ERR_CNT,
+ .len = CORE_CE_ERR_LEN,
+ .data = CORE_CE_ERR_DATA,
+ .max_cnt = 32
+ },
+ [CORE_UE_ERR] = {
+ .count = CORE_UE_ERR_CNT,
+ .len = CORE_UE_ERR_LEN,
+ .data = CORE_UE_ERR_DATA,
+ .max_cnt = 32
+ },
+ [MEM_CE_ERR] = {
+ .count = MEM_CE_ERR_CNT,
+ .len = MEM_CE_ERR_LEN,
+ .data = MEM_CE_ERR_DATA,
+ .max_cnt = 16
+ },
+ [MEM_UE_ERR] = {
+ .count = MEM_UE_ERR_CNT,
+ .len = MEM_UE_ERR_LEN,
+ .data = MEM_UE_ERR_DATA,
+ .max_cnt = 16
+ },
+ [PCIE_CE_ERR] = {
+ .count = PCIE_CE_ERR_CNT,
+ .len = PCIE_CE_ERR_LEN,
+ .data = PCIE_CE_ERR_DATA,
+ .max_cnt = 96
+ },
+ [PCIE_UE_ERR] = {
+ .count = PCIE_UE_ERR_CNT,
+ .len = PCIE_UE_ERR_LEN,
+ .data = PCIE_UE_ERR_DATA,
+ .max_cnt = 96
+ },
+ [OTHER_CE_ERR] = {
+ .count = OTHER_CE_ERR_CNT,
+ .len = OTHER_CE_ERR_LEN,
+ .data = OTHER_CE_ERR_DATA,
+ .max_cnt = 8
+ },
+ [OTHER_UE_ERR] = {
+ .count = OTHER_UE_ERR_CNT,
+ .len = OTHER_UE_ERR_LEN,
+ .data = OTHER_UE_ERR_DATA,
+ .max_cnt = 8
+ },
+};
+
+/*
+ * List of SCP registers which are used to get
+ * one type of RAS Internal errors.
+ */
+struct smpro_int_error_hdr {
+ u8 type;
+ u8 info_l;
+ u8 info_h;
+ u8 data_l;
+ u8 data_h;
+ u8 warn_l;
+ u8 warn_h;
+};
+
+static struct smpro_int_error_hdr list_smpro_int_error_hdr[] = {
+ [RAS_SMPRO_ERR] = {
+ .type = ERR_SMPRO_TYPE,
+ .info_l = ERR_SMPRO_INFO_LO,
+ .info_h = ERR_SMPRO_INFO_HI,
+ .data_l = ERR_SMPRO_DATA_LO,
+ .data_h = ERR_SMPRO_DATA_HI,
+ .warn_l = WARN_SMPRO_INFO_LO,
+ .warn_h = WARN_SMPRO_INFO_HI,
+ },
+ [RAS_PMPRO_ERR] = {
+ .type = ERR_PMPRO_TYPE,
+ .info_l = ERR_PMPRO_INFO_LO,
+ .info_h = ERR_PMPRO_INFO_HI,
+ .data_l = ERR_PMPRO_DATA_LO,
+ .data_h = ERR_PMPRO_DATA_HI,
+ .warn_l = WARN_PMPRO_INFO_LO,
+ .warn_h = WARN_PMPRO_INFO_HI,
+ },
+};
+
+struct smpro_errmon {
+ struct regmap *regmap;
+};
+
+enum EVENT_TYPES {
+ VRD_WARN_FAULT_EVENT,
+ VRD_HOT_EVENT,
+ DIMM_HOT_EVENT,
+ NUM_EVENTS_TYPE,
+};
+
+/* Included Address of event source and data registers */
+static u8 smpro_event_table[NUM_EVENTS_TYPE] = {
+ VRD_WARN_FAULT_EVENT_DATA,
+ VRD_HOT_EVENT_DATA,
+ DIMM_HOT_EVENT_DATA,
+};
+
+static ssize_t smpro_event_data_read(struct device *dev,
+ struct device_attribute *da, char *buf,
+ int channel)
+{
+ struct smpro_errmon *errmon = dev_get_drvdata(dev);
+ s32 event_data;
+ int ret;
+
+ ret = regmap_read(errmon->regmap, smpro_event_table[channel], &event_data);
+ if (ret)
+ return ret;
+ /* Clear event after read */
+ if (event_data != 0)
+ regmap_write(errmon->regmap, smpro_event_table[channel], event_data);
+
+ return sysfs_emit(buf, "%04x\n", event_data);
+}
+
+static ssize_t smpro_overflow_data_read(struct device *dev, struct device_attribute *da,
+ char *buf, int channel)
+{
+ struct smpro_errmon *errmon = dev_get_drvdata(dev);
+ struct smpro_error_hdr *err_info;
+ s32 err_count;
+ int ret;
+
+ err_info = &smpro_error_table[channel];
+
+ ret = regmap_read(errmon->regmap, err_info->count, &err_count);
+ if (ret)
+ return ret;
+
+ /* Bit 8 indicates the overflow status */
+ return sysfs_emit(buf, "%d\n", (err_count & BIT(8)) ? 1 : 0);
+}
+
+static ssize_t smpro_error_data_read(struct device *dev, struct device_attribute *da,
+ char *buf, int channel)
+{
+ struct smpro_errmon *errmon = dev_get_drvdata(dev);
+ unsigned char err_data[MAX_READ_BLOCK_LENGTH];
+ struct smpro_error_hdr *err_info;
+ s32 err_count, err_length;
+ int ret;
+
+ err_info = &smpro_error_table[channel];
+
+ ret = regmap_read(errmon->regmap, err_info->count, &err_count);
+ /* Error count is the low byte */
+ err_count &= 0xff;
+ if (ret || !err_count || err_count > err_info->max_cnt)
+ return ret;
+
+ ret = regmap_read(errmon->regmap, err_info->len, &err_length);
+ if (ret || err_length <= 0)
+ return ret;
+
+ if (err_length > MAX_READ_BLOCK_LENGTH)
+ err_length = MAX_READ_BLOCK_LENGTH;
+
+ memset(err_data, 0x00, MAX_READ_BLOCK_LENGTH);
+ ret = regmap_noinc_read(errmon->regmap, err_info->data, err_data, err_length);
+ if (ret < 0)
+ return ret;
+
+ /* clear the error */
+ ret = regmap_write(errmon->regmap, err_info->count, 0x100);
+ if (ret)
+ return ret;
+ /*
+ * The output of Core/Memory/PCIe/Others UE/CE errors follows the format
+ * specified in section 5.8.1 CE/UE Error Data record in
+ * Altra SOC BMC Interface specification.
+ */
+ return sysfs_emit(buf, "%*phN\n", MAX_READ_BLOCK_LENGTH, err_data);
+}
+
+/*
+ * Output format:
+ * <4-byte hex value of error info><4-byte hex value of error extensive data>
+ * Where:
+ * + error info : The error information
+ * + error data : Extensive data (32 bits)
+ * Reference to section 5.10 RAS Internal Error Register Definition in
+ * Altra SOC BMC Interface specification
+ */
+static ssize_t smpro_internal_err_read(struct device *dev, struct device_attribute *da,
+ char *buf, int channel)
+{
+ struct smpro_errmon *errmon = dev_get_drvdata(dev);
+ struct smpro_int_error_hdr *err_info;
+ unsigned int err[4] = { 0 };
+ unsigned int err_type;
+ unsigned int val;
+ int ret;
+
+ /* read error status */
+ ret = regmap_read(errmon->regmap, GPI_RAS_ERR, &val);
+ if (ret)
+ return ret;
+
+ if ((channel == RAS_SMPRO_ERR && !(val & BIT(0))) ||
+ (channel == RAS_PMPRO_ERR && !(val & BIT(1))))
+ return 0;
+
+ err_info = &list_smpro_int_error_hdr[channel];
+ ret = regmap_read(errmon->regmap, err_info->type, &val);
+ if (ret)
+ return ret;
+
+ err_type = (val & BIT(1)) ? BIT(1) :
+ (val & BIT(2)) ? BIT(2) : 0;
+
+ if (!err_type)
+ return 0;
+
+ ret = regmap_read(errmon->regmap, err_info->info_l, err + 1);
+ if (ret)
+ return ret;
+
+ ret = regmap_read(errmon->regmap, err_info->info_h, err);
+ if (ret)
+ return ret;
+
+ if (err_type & BIT(2)) {
+ /* Error with data type */
+ ret = regmap_read(errmon->regmap, err_info->data_l, err + 3);
+ if (ret)
+ return ret;
+
+ ret = regmap_read(errmon->regmap, err_info->data_h, err + 2);
+ if (ret)
+ return ret;
+ }
+
+ /* clear the read errors */
+ ret = regmap_write(errmon->regmap, err_info->type, err_type);
+ if (ret)
+ return ret;
+
+ return sysfs_emit(buf, "%*phN\n", (int)sizeof(err), err);
+}
+
+/*
+ * Output format:
+ * <4-byte hex value of warining info>
+ * Reference to section 5.10 RAS Internal Error Register Definition in
+ * Altra SOC BMC Interface specification
+ */
+static ssize_t smpro_internal_warn_read(struct device *dev, struct device_attribute *da,
+ char *buf, int channel)
+{
+ struct smpro_errmon *errmon = dev_get_drvdata(dev);
+ struct smpro_int_error_hdr *err_info;
+ unsigned int warn[2] = { 0 };
+ unsigned int val;
+ int ret;
+
+ /* read error status */
+ ret = regmap_read(errmon->regmap, GPI_RAS_ERR, &val);
+ if (ret)
+ return ret;
+
+ if ((channel == RAS_SMPRO_ERR && !(val & BIT(0))) ||
+ (channel == RAS_PMPRO_ERR && !(val & BIT(1))))
+ return 0;
+
+ err_info = &list_smpro_int_error_hdr[channel];
+ ret = regmap_read(errmon->regmap, err_info->type, &val);
+ if (ret)
+ return ret;
+
+ if (!(val & BIT(0)))
+ return 0;
+
+ ret = regmap_read(errmon->regmap, err_info->warn_l, warn + 1);
+ if (ret)
+ return ret;
+
+ ret = regmap_read(errmon->regmap, err_info->warn_h, warn);
+ if (ret)
+ return ret;
+
+ /* clear the warning */
+ ret = regmap_write(errmon->regmap, err_info->type, BIT(0));
+ if (ret)
+ return ret;
+
+ return sysfs_emit(buf, "%*phN\n", (int)sizeof(warn), warn);
+}
+
+#define ERROR_OVERFLOW_RO(_error, _index) \
+ static ssize_t overflow_##_error##_show(struct device *dev, \
+ struct device_attribute *da, \
+ char *buf) \
+ { \
+ return smpro_overflow_data_read(dev, da, buf, _index); \
+ } \
+ static DEVICE_ATTR_RO(overflow_##_error)
+
+ERROR_OVERFLOW_RO(core_ce, CORE_CE_ERR);
+ERROR_OVERFLOW_RO(core_ue, CORE_UE_ERR);
+ERROR_OVERFLOW_RO(mem_ce, MEM_CE_ERR);
+ERROR_OVERFLOW_RO(mem_ue, MEM_UE_ERR);
+ERROR_OVERFLOW_RO(pcie_ce, PCIE_CE_ERR);
+ERROR_OVERFLOW_RO(pcie_ue, PCIE_UE_ERR);
+ERROR_OVERFLOW_RO(other_ce, OTHER_CE_ERR);
+ERROR_OVERFLOW_RO(other_ue, OTHER_UE_ERR);
+
+#define ERROR_RO(_error, _index) \
+ static ssize_t error_##_error##_show(struct device *dev, \
+ struct device_attribute *da, \
+ char *buf) \
+ { \
+ return smpro_error_data_read(dev, da, buf, _index); \
+ } \
+ static DEVICE_ATTR_RO(error_##_error)
+
+ERROR_RO(core_ce, CORE_CE_ERR);
+ERROR_RO(core_ue, CORE_UE_ERR);
+ERROR_RO(mem_ce, MEM_CE_ERR);
+ERROR_RO(mem_ue, MEM_UE_ERR);
+ERROR_RO(pcie_ce, PCIE_CE_ERR);
+ERROR_RO(pcie_ue, PCIE_UE_ERR);
+ERROR_RO(other_ce, OTHER_CE_ERR);
+ERROR_RO(other_ue, OTHER_UE_ERR);
+
+static ssize_t error_smpro_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+ return smpro_internal_err_read(dev, da, buf, RAS_SMPRO_ERR);
+}
+static DEVICE_ATTR_RO(error_smpro);
+
+static ssize_t error_pmpro_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+ return smpro_internal_err_read(dev, da, buf, RAS_PMPRO_ERR);
+}
+static DEVICE_ATTR_RO(error_pmpro);
+
+static ssize_t warn_smpro_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+ return smpro_internal_warn_read(dev, da, buf, RAS_SMPRO_ERR);
+}
+static DEVICE_ATTR_RO(warn_smpro);
+
+static ssize_t warn_pmpro_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+ return smpro_internal_warn_read(dev, da, buf, RAS_PMPRO_ERR);
+}
+static DEVICE_ATTR_RO(warn_pmpro);
+
+#define EVENT_RO(_event, _index) \
+ static ssize_t event_##_event##_show(struct device *dev, \
+ struct device_attribute *da, \
+ char *buf) \
+ { \
+ return smpro_event_data_read(dev, da, buf, _index); \
+ } \
+ static DEVICE_ATTR_RO(event_##_event)
+
+EVENT_RO(vrd_warn_fault, VRD_WARN_FAULT_EVENT);
+EVENT_RO(vrd_hot, VRD_HOT_EVENT);
+EVENT_RO(dimm_hot, DIMM_HOT_EVENT);
+
+static struct attribute *smpro_errmon_attrs[] = {
+ &dev_attr_overflow_core_ce.attr,
+ &dev_attr_overflow_core_ue.attr,
+ &dev_attr_overflow_mem_ce.attr,
+ &dev_attr_overflow_mem_ue.attr,
+ &dev_attr_overflow_pcie_ce.attr,
+ &dev_attr_overflow_pcie_ue.attr,
+ &dev_attr_overflow_other_ce.attr,
+ &dev_attr_overflow_other_ue.attr,
+ &dev_attr_error_core_ce.attr,
+ &dev_attr_error_core_ue.attr,
+ &dev_attr_error_mem_ce.attr,
+ &dev_attr_error_mem_ue.attr,
+ &dev_attr_error_pcie_ce.attr,
+ &dev_attr_error_pcie_ue.attr,
+ &dev_attr_error_other_ce.attr,
+ &dev_attr_error_other_ue.attr,
+ &dev_attr_error_smpro.attr,
+ &dev_attr_error_pmpro.attr,
+ &dev_attr_warn_smpro.attr,
+ &dev_attr_warn_pmpro.attr,
+ &dev_attr_event_vrd_warn_fault.attr,
+ &dev_attr_event_vrd_hot.attr,
+ &dev_attr_event_dimm_hot.attr,
+ NULL
+};
+
+ATTRIBUTE_GROUPS(smpro_errmon);
+
+static int smpro_errmon_probe(struct platform_device *pdev)
+{
+ struct smpro_errmon *errmon;
+
+ errmon = devm_kzalloc(&pdev->dev, sizeof(struct smpro_errmon), GFP_KERNEL);
+ if (!errmon)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, errmon);
+
+ errmon->regmap = dev_get_regmap(pdev->dev.parent, NULL);
+ if (!errmon->regmap)
+ return -ENODEV;
+
+ return 0;
+}
+
+static struct platform_driver smpro_errmon_driver = {
+ .probe = smpro_errmon_probe,
+ .driver = {
+ .name = "smpro-errmon",
+ .dev_groups = smpro_errmon_groups,
+ },
+};
+
+module_platform_driver(smpro_errmon_driver);
+
+MODULE_AUTHOR("Tung Nguyen <tung.nguyen@amperecomputing.com>");
+MODULE_AUTHOR("Thinh Pham <thinh.pham@amperecomputing.com>");
+MODULE_AUTHOR("Hoang Nguyen <hnguyen@amperecomputing.com>");
+MODULE_AUTHOR("Thu Nguyen <thu@os.amperecomputing.com>");
+MODULE_AUTHOR("Quan Nguyen <quan@os.amperecomputing.com>");
+MODULE_DESCRIPTION("Ampere Altra SMpro driver");
+MODULE_LICENSE("GPL");
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Ampere Computing SoC's SMpro Misc Driver
+ *
+ * Copyright (c) 2022, Ampere Computing LLC
+ */
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+/* Boot Stage/Progress Registers */
+#define BOOTSTAGE 0xB0
+#define BOOTSTAGE_LO 0xB1
+#define CUR_BOOTSTAGE 0xB2
+#define BOOTSTAGE_HI 0xB3
+
+/* SOC State Registers */
+#define SOC_POWER_LIMIT 0xE5
+
+struct smpro_misc {
+ struct regmap *regmap;
+};
+
+static ssize_t boot_progress_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+ struct smpro_misc *misc = dev_get_drvdata(dev);
+ u16 boot_progress[3] = { 0 };
+ u32 bootstage;
+ u8 boot_stage;
+ u8 cur_stage;
+ u32 reg_lo;
+ u32 reg;
+ int ret;
+
+ /* Read current boot stage */
+ ret = regmap_read(misc->regmap, CUR_BOOTSTAGE, ®);
+ if (ret)
+ return ret;
+
+ cur_stage = reg & 0xff;
+
+ ret = regmap_read(misc->regmap, BOOTSTAGE, &bootstage);
+ if (ret)
+ return ret;
+
+ boot_stage = (bootstage >> 8) & 0xff;
+
+ if (boot_stage > cur_stage)
+ return -EINVAL;
+
+ ret = regmap_read(misc->regmap, BOOTSTAGE_LO, ®_lo);
+ if (!ret)
+ ret = regmap_read(misc->regmap, BOOTSTAGE_HI, ®);
+ if (ret)
+ return ret;
+
+ /* Firmware to report new boot stage next time */
+ if (boot_stage < cur_stage) {
+ ret = regmap_write(misc->regmap, BOOTSTAGE, ((bootstage & 0xff00) | 0x1));
+ if (ret)
+ return ret;
+ }
+
+ boot_progress[0] = bootstage;
+ boot_progress[1] = swab16(reg);
+ boot_progress[2] = swab16(reg_lo);
+
+ return sysfs_emit(buf, "%*phN\n", (int)sizeof(boot_progress), boot_progress);
+}
+
+static DEVICE_ATTR_RO(boot_progress);
+
+static ssize_t soc_power_limit_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+ struct smpro_misc *misc = dev_get_drvdata(dev);
+ unsigned int value;
+ int ret;
+
+ ret = regmap_read(misc->regmap, SOC_POWER_LIMIT, &value);
+ if (ret)
+ return ret;
+
+ return sysfs_emit(buf, "%d\n", value);
+}
+
+static ssize_t soc_power_limit_store(struct device *dev, struct device_attribute *da,
+ const char *buf, size_t count)
+{
+ struct smpro_misc *misc = dev_get_drvdata(dev);
+ unsigned long val;
+ s32 ret;
+
+ ret = kstrtoul(buf, 0, &val);
+ if (ret)
+ return ret;
+
+ ret = regmap_write(misc->regmap, SOC_POWER_LIMIT, (unsigned int)val);
+ if (ret)
+ return -EPROTO;
+
+ return count;
+}
+
+static DEVICE_ATTR_RW(soc_power_limit);
+
+static struct attribute *smpro_misc_attrs[] = {
+ &dev_attr_boot_progress.attr,
+ &dev_attr_soc_power_limit.attr,
+ NULL
+};
+
+ATTRIBUTE_GROUPS(smpro_misc);
+
+static int smpro_misc_probe(struct platform_device *pdev)
+{
+ struct smpro_misc *misc;
+
+ misc = devm_kzalloc(&pdev->dev, sizeof(struct smpro_misc), GFP_KERNEL);
+ if (!misc)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, misc);
+
+ misc->regmap = dev_get_regmap(pdev->dev.parent, NULL);
+ if (!misc->regmap)
+ return -ENODEV;
+
+ return 0;
+}
+
+static struct platform_driver smpro_misc_driver = {
+ .probe = smpro_misc_probe,
+ .driver = {
+ .name = "smpro-misc",
+ .dev_groups = smpro_misc_groups,
+ },
+};
+
+module_platform_driver(smpro_misc_driver);
+
+MODULE_AUTHOR("Tung Nguyen <tungnguyen@os.amperecomputing.com>");
+MODULE_AUTHOR("Quan Nguyen <quan@os.amperecomputing.com>");
+MODULE_DESCRIPTION("Ampere Altra SMpro Misc driver");
+MODULE_LICENSE("GPL");
#include <linux/genalloc.h>
#include <linux/mm.h>
#include <linux/sram.h>
+#include <linux/set_memory.h>
#include <asm/fncpy.h>
-#include <asm/set_memory.h>
#include "sram.h"
dst_cpy = fncpy(dst, src, size);
- ret = set_memory_ro((unsigned long)base, pages);
- if (ret)
- goto error_out;
- ret = set_memory_x((unsigned long)base, pages);
+ ret = set_memory_rox((unsigned long)base, pages);
if (ret)
goto error_out;
spin_unlock_irqrestore(&fm->lock, flags);
}
if (sock)
- tifm_free_device(&sock->dev);
+ put_device(&sock->dev);
}
spin_lock_irqsave(&fm->lock, flags);
}
*/
static struct i2c_driver tsl2550_driver;
-static int tsl2550_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tsl2550_probe(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
struct tsl2550_data *data;
.of_match_table = tsl2550_of_match,
.pm = TSL2550_PM_OPS,
},
- .probe = tsl2550_probe,
+ .probe_new = tsl2550_probe,
.remove = tsl2550_remove,
.id_table = tsl2550_id,
};
static void bond_miimon_commit(struct bonding *bond)
{
- struct list_head *iter;
struct slave *slave, *primary;
+ bool do_failover = false;
+ struct list_head *iter;
bond_for_each_slave(bond, slave, iter) {
switch (slave->link_new_state) {
bond_miimon_link_change(bond, slave, BOND_LINK_UP);
- if (!bond->curr_active_slave || slave == primary)
- goto do_failover;
+ if (!rcu_access_pointer(bond->curr_active_slave) || slave == primary ||
+ slave->prio > rcu_dereference(bond->curr_active_slave)->prio)
+ do_failover = true;
continue;
bond_miimon_link_change(bond, slave, BOND_LINK_DOWN);
if (slave == rcu_access_pointer(bond->curr_active_slave))
- goto do_failover;
+ do_failover = true;
continue;
continue;
}
+ }
-do_failover:
+ if (do_failover) {
block_netpoll_tx();
bond_select_active_slave(bond);
unblock_netpoll_tx();
*/
static void bond_ab_arp_commit(struct bonding *bond)
{
+ bool do_failover = false;
struct list_head *iter;
unsigned long last_tx;
struct slave *slave;
slave_info(bond->dev, slave->dev, "link status definitely up\n");
if (!rtnl_dereference(bond->curr_active_slave) ||
- slave == rtnl_dereference(bond->primary_slave))
- goto do_failover;
+ slave == rtnl_dereference(bond->primary_slave) ||
+ slave->prio > rtnl_dereference(bond->curr_active_slave)->prio)
+ do_failover = true;
}
if (slave == rtnl_dereference(bond->curr_active_slave)) {
RCU_INIT_POINTER(bond->current_arp_slave, NULL);
- goto do_failover;
+ do_failover = true;
}
continue;
slave->link_new_state);
continue;
}
+ }
-do_failover:
+ if (do_failover) {
block_netpoll_tx();
bond_select_active_slave(bond);
unblock_netpoll_tx();
if (netif_running(dev)) {
int err;
- err = pm_runtime_force_resume(device);
- if (err)
- return err;
+ /* For the wakeup in auto stop mode, no need to gate on the
+ * clock here, hardware will do this automatically.
+ */
+ if (!(device_may_wakeup(device) &&
+ priv->devtype_data.quirks & FLEXCAN_QUIRK_AUTO_STOP_MODE)) {
+ err = pm_runtime_force_resume(device);
+ if (err)
+ return err;
+ }
if (device_may_wakeup(device))
flexcan_enable_wakeup_irq(priv, false);
u8 cmd_no, int channel)
{
struct kvaser_cmd *cmd;
+ size_t cmd_len;
int err;
cmd = kzalloc(sizeof(*cmd), GFP_KERNEL);
return -ENOMEM;
cmd->header.cmd_no = cmd_no;
+ cmd_len = kvaser_usb_hydra_cmd_size(cmd);
if (channel < 0) {
kvaser_usb_hydra_set_cmd_dest_he
(cmd, KVASER_USB_HYDRA_HE_ADDRESS_ILLEGAL);
kvaser_usb_hydra_set_cmd_transid
(cmd, kvaser_usb_hydra_get_next_transid(dev));
- err = kvaser_usb_send_cmd(dev, cmd, kvaser_usb_hydra_cmd_size(cmd));
+ err = kvaser_usb_send_cmd(dev, cmd, cmd_len);
if (err)
goto end;
{
struct kvaser_cmd *cmd;
struct kvaser_usb *dev = priv->dev;
+ size_t cmd_len;
int err;
cmd = kzalloc(sizeof(*cmd), GFP_ATOMIC);
return -ENOMEM;
cmd->header.cmd_no = cmd_no;
+ cmd_len = kvaser_usb_hydra_cmd_size(cmd);
kvaser_usb_hydra_set_cmd_dest_he
(cmd, dev->card_data.hydra.channel_to_he[priv->channel]);
kvaser_usb_hydra_set_cmd_transid
(cmd, kvaser_usb_hydra_get_next_transid(dev));
- err = kvaser_usb_send_cmd_async(priv, cmd,
- kvaser_usb_hydra_cmd_size(cmd));
+ err = kvaser_usb_send_cmd_async(priv, cmd, cmd_len);
if (err)
kfree(cmd);
{
struct kvaser_usb_dev_card_data *card_data = &dev->card_data;
struct kvaser_cmd *cmd;
+ size_t cmd_len;
u32 value = 0;
u32 mask = 0;
u16 cap_cmd_res;
return -ENOMEM;
cmd->header.cmd_no = CMD_GET_CAPABILITIES_REQ;
+ cmd_len = kvaser_usb_hydra_cmd_size(cmd);
cmd->cap_req.cap_cmd = cpu_to_le16(cap_cmd_req);
kvaser_usb_hydra_set_cmd_dest_he(cmd, card_data->hydra.sysdbg_he);
kvaser_usb_hydra_set_cmd_transid
(cmd, kvaser_usb_hydra_get_next_transid(dev));
- err = kvaser_usb_send_cmd(dev, cmd, kvaser_usb_hydra_cmd_size(cmd));
+ err = kvaser_usb_send_cmd(dev, cmd, cmd_len);
if (err)
goto end;
struct kvaser_usb *dev = priv->dev;
struct kvaser_usb_net_hydra_priv *hydra = priv->sub_priv;
struct kvaser_cmd *cmd;
+ size_t cmd_len;
int err;
if (!hydra)
return -ENOMEM;
cmd->header.cmd_no = CMD_GET_BUSPARAMS_REQ;
+ cmd_len = kvaser_usb_hydra_cmd_size(cmd);
kvaser_usb_hydra_set_cmd_dest_he
(cmd, dev->card_data.hydra.channel_to_he[priv->channel]);
kvaser_usb_hydra_set_cmd_transid
reinit_completion(&priv->get_busparams_comp);
- err = kvaser_usb_send_cmd(dev, cmd, kvaser_usb_hydra_cmd_size(cmd));
+ err = kvaser_usb_send_cmd(dev, cmd, cmd_len);
if (err)
return err;
struct kvaser_cmd *cmd;
struct kvaser_usb_net_priv *priv = netdev_priv(netdev);
struct kvaser_usb *dev = priv->dev;
+ size_t cmd_len;
int err;
cmd = kzalloc(sizeof(*cmd), GFP_KERNEL);
return -ENOMEM;
cmd->header.cmd_no = CMD_SET_BUSPARAMS_REQ;
+ cmd_len = kvaser_usb_hydra_cmd_size(cmd);
memcpy(&cmd->set_busparams_req.busparams_nominal, busparams,
sizeof(cmd->set_busparams_req.busparams_nominal));
kvaser_usb_hydra_set_cmd_transid
(cmd, kvaser_usb_hydra_get_next_transid(dev));
- err = kvaser_usb_send_cmd(dev, cmd, kvaser_usb_hydra_cmd_size(cmd));
+ err = kvaser_usb_send_cmd(dev, cmd, cmd_len);
kfree(cmd);
struct kvaser_cmd *cmd;
struct kvaser_usb_net_priv *priv = netdev_priv(netdev);
struct kvaser_usb *dev = priv->dev;
+ size_t cmd_len;
int err;
cmd = kzalloc(sizeof(*cmd), GFP_KERNEL);
return -ENOMEM;
cmd->header.cmd_no = CMD_SET_BUSPARAMS_FD_REQ;
+ cmd_len = kvaser_usb_hydra_cmd_size(cmd);
memcpy(&cmd->set_busparams_req.busparams_data, busparams,
sizeof(cmd->set_busparams_req.busparams_data));
kvaser_usb_hydra_set_cmd_transid
(cmd, kvaser_usb_hydra_get_next_transid(dev));
- err = kvaser_usb_send_cmd(dev, cmd, kvaser_usb_hydra_cmd_size(cmd));
+ err = kvaser_usb_send_cmd(dev, cmd, cmd_len);
kfree(cmd);
static int kvaser_usb_hydra_get_software_details(struct kvaser_usb *dev)
{
struct kvaser_cmd *cmd;
+ size_t cmd_len;
int err;
u32 flags;
struct kvaser_usb_dev_card_data *card_data = &dev->card_data;
return -ENOMEM;
cmd->header.cmd_no = CMD_GET_SOFTWARE_DETAILS_REQ;
+ cmd_len = kvaser_usb_hydra_cmd_size(cmd);
cmd->sw_detail_req.use_ext_cmd = 1;
kvaser_usb_hydra_set_cmd_dest_he
(cmd, KVASER_USB_HYDRA_HE_ADDRESS_ILLEGAL);
kvaser_usb_hydra_set_cmd_transid
(cmd, kvaser_usb_hydra_get_next_transid(dev));
- err = kvaser_usb_send_cmd(dev, cmd, kvaser_usb_hydra_cmd_size(cmd));
+ err = kvaser_usb_send_cmd(dev, cmd, cmd_len);
if (err)
goto end;
{
struct kvaser_usb *dev = priv->dev;
struct kvaser_cmd *cmd;
+ size_t cmd_len;
int err;
if ((priv->can.ctrlmode &
return -ENOMEM;
cmd->header.cmd_no = CMD_SET_DRIVERMODE_REQ;
+ cmd_len = kvaser_usb_hydra_cmd_size(cmd);
kvaser_usb_hydra_set_cmd_dest_he
(cmd, dev->card_data.hydra.channel_to_he[priv->channel]);
kvaser_usb_hydra_set_cmd_transid
else
cmd->set_ctrlmode.mode = KVASER_USB_HYDRA_CTRLMODE_NORMAL;
- err = kvaser_usb_send_cmd(dev, cmd, kvaser_usb_hydra_cmd_size(cmd));
+ err = kvaser_usb_send_cmd(dev, cmd, cmd_len);
kfree(cmd);
return err;
irq_create_mapping(kirq->domain, n);
ret = request_threaded_irq(kirq->irq_num, NULL, ksz_irq_thread_fn,
- IRQF_ONESHOT | IRQF_TRIGGER_FALLING,
- kirq->name, kirq);
+ IRQF_ONESHOT, kirq->name, kirq);
if (ret)
goto out;
config->mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE |
MAC_10 | MAC_100 | MAC_1000FD;
- if ((priv->id == ID_MT7531) && mt753x_is_mac_port(port))
- config->mac_capabilities |= MAC_2500FD;
-
/* This driver does not make use of the speed, duplex, pause or the
* advertisement in its mac_config, so it is safe to mark this driver
* as non-legacy.
/* Port 4 supports automedia if the serdes is associated with it. */
if (port == 4) {
- mv88e6xxx_reg_lock(chip);
err = mv88e6352_g2_scratch_port_has_serdes(chip, port);
if (err < 0)
dev_err(chip->dev, "p%d: failed to read scratch\n",
port);
if (err <= 0)
- goto unlock;
+ return;
cmode = mv88e6352_get_port4_serdes_cmode(chip);
if (cmode < 0)
port);
else
mv88e6xxx_translate_cmode(cmode, supported);
-unlock:
- mv88e6xxx_reg_unlock(chip);
}
}
{
struct mv88e6xxx_chip *chip = ds->priv;
+ mv88e6xxx_reg_lock(chip);
chip->info->ops->phylink_get_caps(chip, port, config);
+ mv88e6xxx_reg_unlock(chip);
if (mv88e6xxx_phy_is_internal(ds, port)) {
__set_bit(PHY_INTERFACE_MODE_INTERNAL,
struct phylink_config pl_config = {};
unsigned long caps;
- mv88e6xxx_get_caps(ds, port, &pl_config);
+ chip->info->ops->phylink_get_caps(chip, port, &pl_config);
caps = pl_config.mac_capabilities;
rx_ring->stats.xdp_drops++;
}
-static void enetc_xdp_free(struct enetc_bdr *rx_ring, int rx_ring_first,
- int rx_ring_last)
-{
- while (rx_ring_first != rx_ring_last) {
- struct enetc_rx_swbd *rx_swbd = &rx_ring->rx_swbd[rx_ring_first];
-
- if (rx_swbd->page) {
- dma_unmap_page(rx_ring->dev, rx_swbd->dma, PAGE_SIZE,
- rx_swbd->dir);
- __free_page(rx_swbd->page);
- rx_swbd->page = NULL;
- }
- enetc_bdr_idx_inc(rx_ring, &rx_ring_first);
- }
- rx_ring->stats.xdp_redirect_failures++;
-}
-
static int enetc_clean_rx_ring_xdp(struct enetc_bdr *rx_ring,
struct napi_struct *napi, int work_limit,
struct bpf_prog *prog)
int orig_i, orig_cleaned_cnt;
struct xdp_buff xdp_buff;
struct sk_buff *skb;
- int tmp_orig_i, err;
u32 bd_status;
+ int err;
rxbd = enetc_rxbd(rx_ring, i);
bd_status = le32_to_cpu(rxbd->r.lstatus);
break;
}
- tmp_orig_i = orig_i;
-
- while (orig_i != i) {
- enetc_flip_rx_buff(rx_ring,
- &rx_ring->rx_swbd[orig_i]);
- enetc_bdr_idx_inc(rx_ring, &orig_i);
- }
-
err = xdp_do_redirect(rx_ring->ndev, &xdp_buff, prog);
if (unlikely(err)) {
- enetc_xdp_free(rx_ring, tmp_orig_i, i);
+ enetc_xdp_drop(rx_ring, orig_i, i);
+ rx_ring->stats.xdp_redirect_failures++;
} else {
+ while (orig_i != i) {
+ enetc_flip_rx_buff(rx_ring,
+ &rx_ring->rx_swbd[orig_i]);
+ enetc_bdr_idx_inc(rx_ring, &orig_i);
+ }
xdp_redirect_frm_cnt++;
rx_ring->stats.xdp_redirect++;
}
* bridging applications.
*/
skb = build_skb(page_address(page), PAGE_SIZE);
+ if (unlikely(!skb)) {
+ page_pool_recycle_direct(rxq->page_pool, page);
+ ndev->stats.rx_dropped++;
+
+ netdev_err_once(ndev, "build_skb failed!\n");
+ goto rx_processing_done;
+ }
+
skb_reserve(skb, data_start);
skb_put(skb, pkt_len - sub_len);
skb_mark_for_recycle(skb);
{
struct e1000_hw *hw = &adapter->hw;
unsigned char *vf_mac = adapter->vf_data[vf].vf_mac_addresses;
- u32 reg, msgbuf[3];
+ u32 reg, msgbuf[3] = {};
u8 *addr = (u8 *)(&msgbuf[1]);
/* process all the same items cleared in a function level reset */
u8 queue_index; /* logical index of the ring*/
u8 reg_idx; /* physical index of the ring */
bool launchtime_enable; /* true if LaunchTime is enabled */
+ ktime_t last_tx_cycle; /* end of the cycle with a launchtime transmission */
+ ktime_t last_ff_cycle; /* Last cycle with an active first flag */
u32 start_time;
u32 end_time;
ktime_t base_time;
ktime_t cycle_time;
+ bool qbv_enable;
/* OS defined structs */
struct pci_dev *pdev;
#define IGC_ADVTXD_L4LEN_SHIFT 8 /* Adv ctxt L4LEN shift */
#define IGC_ADVTXD_MSS_SHIFT 16 /* Adv ctxt MSS shift */
+#define IGC_ADVTXD_TSN_CNTX_FIRST 0x00000080
+
/* Transmit Control */
#define IGC_TCTL_EN 0x00000002 /* enable Tx */
#define IGC_TCTL_PSP 0x00000008 /* pad short packets */
return netdev_mc_count(netdev);
}
-static __le32 igc_tx_launchtime(struct igc_adapter *adapter, ktime_t txtime)
+static __le32 igc_tx_launchtime(struct igc_ring *ring, ktime_t txtime,
+ bool *first_flag, bool *insert_empty)
{
+ struct igc_adapter *adapter = netdev_priv(ring->netdev);
ktime_t cycle_time = adapter->cycle_time;
ktime_t base_time = adapter->base_time;
+ ktime_t now = ktime_get_clocktai();
+ ktime_t baset_est, end_of_cycle;
u32 launchtime;
+ s64 n;
- /* FIXME: when using ETF together with taprio, we may have a
- * case where 'delta' is larger than the cycle_time, this may
- * cause problems if we don't read the current value of
- * IGC_BASET, as the value writen into the launchtime
- * descriptor field may be misinterpreted.
+ n = div64_s64(ktime_sub_ns(now, base_time), cycle_time);
+
+ baset_est = ktime_add_ns(base_time, cycle_time * (n));
+ end_of_cycle = ktime_add_ns(baset_est, cycle_time);
+
+ if (ktime_compare(txtime, end_of_cycle) >= 0) {
+ if (baset_est != ring->last_ff_cycle) {
+ *first_flag = true;
+ ring->last_ff_cycle = baset_est;
+
+ if (ktime_compare(txtime, ring->last_tx_cycle) > 0)
+ *insert_empty = true;
+ }
+ }
+
+ /* Introducing a window at end of cycle on which packets
+ * potentially not honor launchtime. Window of 5us chosen
+ * considering software update the tail pointer and packets
+ * are dma'ed to packet buffer.
*/
- div_s64_rem(ktime_sub_ns(txtime, base_time), cycle_time, &launchtime);
+ if ((ktime_sub_ns(end_of_cycle, now) < 5 * NSEC_PER_USEC))
+ netdev_warn(ring->netdev, "Packet with txtime=%llu may not be honoured\n",
+ txtime);
+
+ ring->last_tx_cycle = end_of_cycle;
+
+ launchtime = ktime_sub_ns(txtime, baset_est);
+ if (launchtime > 0)
+ div_s64_rem(launchtime, cycle_time, &launchtime);
+ else
+ launchtime = 0;
return cpu_to_le32(launchtime);
}
+static int igc_init_empty_frame(struct igc_ring *ring,
+ struct igc_tx_buffer *buffer,
+ struct sk_buff *skb)
+{
+ unsigned int size;
+ dma_addr_t dma;
+
+ size = skb_headlen(skb);
+
+ dma = dma_map_single(ring->dev, skb->data, size, DMA_TO_DEVICE);
+ if (dma_mapping_error(ring->dev, dma)) {
+ netdev_err_once(ring->netdev, "Failed to map DMA for TX\n");
+ return -ENOMEM;
+ }
+
+ buffer->skb = skb;
+ buffer->protocol = 0;
+ buffer->bytecount = skb->len;
+ buffer->gso_segs = 1;
+ buffer->time_stamp = jiffies;
+ dma_unmap_len_set(buffer, len, skb->len);
+ dma_unmap_addr_set(buffer, dma, dma);
+
+ return 0;
+}
+
+static int igc_init_tx_empty_descriptor(struct igc_ring *ring,
+ struct sk_buff *skb,
+ struct igc_tx_buffer *first)
+{
+ union igc_adv_tx_desc *desc;
+ u32 cmd_type, olinfo_status;
+ int err;
+
+ if (!igc_desc_unused(ring))
+ return -EBUSY;
+
+ err = igc_init_empty_frame(ring, first, skb);
+ if (err)
+ return err;
+
+ cmd_type = IGC_ADVTXD_DTYP_DATA | IGC_ADVTXD_DCMD_DEXT |
+ IGC_ADVTXD_DCMD_IFCS | IGC_TXD_DCMD |
+ first->bytecount;
+ olinfo_status = first->bytecount << IGC_ADVTXD_PAYLEN_SHIFT;
+
+ desc = IGC_TX_DESC(ring, ring->next_to_use);
+ desc->read.cmd_type_len = cpu_to_le32(cmd_type);
+ desc->read.olinfo_status = cpu_to_le32(olinfo_status);
+ desc->read.buffer_addr = cpu_to_le64(dma_unmap_addr(first, dma));
+
+ netdev_tx_sent_queue(txring_txq(ring), skb->len);
+
+ first->next_to_watch = desc;
+
+ ring->next_to_use++;
+ if (ring->next_to_use == ring->count)
+ ring->next_to_use = 0;
+
+ return 0;
+}
+
+#define IGC_EMPTY_FRAME_SIZE 60
+
static void igc_tx_ctxtdesc(struct igc_ring *tx_ring,
- struct igc_tx_buffer *first,
+ __le32 launch_time, bool first_flag,
u32 vlan_macip_lens, u32 type_tucmd,
u32 mss_l4len_idx)
{
if (test_bit(IGC_RING_FLAG_TX_CTX_IDX, &tx_ring->flags))
mss_l4len_idx |= tx_ring->reg_idx << 4;
+ if (first_flag)
+ mss_l4len_idx |= IGC_ADVTXD_TSN_CNTX_FIRST;
+
context_desc->vlan_macip_lens = cpu_to_le32(vlan_macip_lens);
context_desc->type_tucmd_mlhl = cpu_to_le32(type_tucmd);
context_desc->mss_l4len_idx = cpu_to_le32(mss_l4len_idx);
-
- /* We assume there is always a valid Tx time available. Invalid times
- * should have been handled by the upper layers.
- */
- if (tx_ring->launchtime_enable) {
- struct igc_adapter *adapter = netdev_priv(tx_ring->netdev);
- ktime_t txtime = first->skb->tstamp;
-
- skb_txtime_consumed(first->skb);
- context_desc->launch_time = igc_tx_launchtime(adapter,
- txtime);
- } else {
- context_desc->launch_time = 0;
- }
+ context_desc->launch_time = launch_time;
}
-static void igc_tx_csum(struct igc_ring *tx_ring, struct igc_tx_buffer *first)
+static void igc_tx_csum(struct igc_ring *tx_ring, struct igc_tx_buffer *first,
+ __le32 launch_time, bool first_flag)
{
struct sk_buff *skb = first->skb;
u32 vlan_macip_lens = 0;
vlan_macip_lens |= skb_network_offset(skb) << IGC_ADVTXD_MACLEN_SHIFT;
vlan_macip_lens |= first->tx_flags & IGC_TX_FLAGS_VLAN_MASK;
- igc_tx_ctxtdesc(tx_ring, first, vlan_macip_lens, type_tucmd, 0);
+ igc_tx_ctxtdesc(tx_ring, launch_time, first_flag,
+ vlan_macip_lens, type_tucmd, 0);
}
static int __igc_maybe_stop_tx(struct igc_ring *tx_ring, const u16 size)
static int igc_tso(struct igc_ring *tx_ring,
struct igc_tx_buffer *first,
+ __le32 launch_time, bool first_flag,
u8 *hdr_len)
{
u32 vlan_macip_lens, type_tucmd, mss_l4len_idx;
vlan_macip_lens |= (ip.hdr - skb->data) << IGC_ADVTXD_MACLEN_SHIFT;
vlan_macip_lens |= first->tx_flags & IGC_TX_FLAGS_VLAN_MASK;
- igc_tx_ctxtdesc(tx_ring, first, vlan_macip_lens,
- type_tucmd, mss_l4len_idx);
+ igc_tx_ctxtdesc(tx_ring, launch_time, first_flag,
+ vlan_macip_lens, type_tucmd, mss_l4len_idx);
return 1;
}
static netdev_tx_t igc_xmit_frame_ring(struct sk_buff *skb,
struct igc_ring *tx_ring)
{
+ bool first_flag = false, insert_empty = false;
u16 count = TXD_USE_COUNT(skb_headlen(skb));
__be16 protocol = vlan_get_protocol(skb);
struct igc_tx_buffer *first;
+ __le32 launch_time = 0;
u32 tx_flags = 0;
unsigned short f;
+ ktime_t txtime;
u8 hdr_len = 0;
int tso = 0;
count += TXD_USE_COUNT(skb_frag_size(
&skb_shinfo(skb)->frags[f]));
- if (igc_maybe_stop_tx(tx_ring, count + 3)) {
+ if (igc_maybe_stop_tx(tx_ring, count + 5)) {
/* this is a hard error */
return NETDEV_TX_BUSY;
}
+ if (!tx_ring->launchtime_enable)
+ goto done;
+
+ txtime = skb->tstamp;
+ skb->tstamp = ktime_set(0, 0);
+ launch_time = igc_tx_launchtime(tx_ring, txtime, &first_flag, &insert_empty);
+
+ if (insert_empty) {
+ struct igc_tx_buffer *empty_info;
+ struct sk_buff *empty;
+ void *data;
+
+ empty_info = &tx_ring->tx_buffer_info[tx_ring->next_to_use];
+ empty = alloc_skb(IGC_EMPTY_FRAME_SIZE, GFP_ATOMIC);
+ if (!empty)
+ goto done;
+
+ data = skb_put(empty, IGC_EMPTY_FRAME_SIZE);
+ memset(data, 0, IGC_EMPTY_FRAME_SIZE);
+
+ igc_tx_ctxtdesc(tx_ring, 0, false, 0, 0, 0);
+
+ if (igc_init_tx_empty_descriptor(tx_ring,
+ empty,
+ empty_info) < 0)
+ dev_kfree_skb_any(empty);
+ }
+
+done:
/* record the location of the first descriptor for this packet */
first = &tx_ring->tx_buffer_info[tx_ring->next_to_use];
first->type = IGC_TX_BUFFER_TYPE_SKB;
first->tx_flags = tx_flags;
first->protocol = protocol;
- tso = igc_tso(tx_ring, first, &hdr_len);
+ tso = igc_tso(tx_ring, first, launch_time, first_flag, &hdr_len);
if (tso < 0)
goto out_drop;
else if (!tso)
- igc_tx_csum(tx_ring, first);
+ igc_tx_csum(tx_ring, first, launch_time, first_flag);
igc_tx_map(tx_ring, first, hdr_len);
bool queue_configured[IGC_MAX_TX_QUEUES] = { };
u32 start_time = 0, end_time = 0;
size_t n;
+ int i;
+
+ adapter->qbv_enable = qopt->enable;
if (!qopt->enable)
return igc_tsn_clear_schedule(adapter);
+ if (qopt->base_time < 0)
+ return -ERANGE;
+
if (adapter->base_time)
return -EALREADY;
for (n = 0; n < qopt->num_entries; n++) {
struct tc_taprio_sched_entry *e = &qopt->entries[n];
- int i;
end_time += e->interval;
+ /* If any of the conditions below are true, we need to manually
+ * control the end time of the cycle.
+ * 1. Qbv users can specify a cycle time that is not equal
+ * to the total GCL intervals. Hence, recalculation is
+ * necessary here to exclude the time interval that
+ * exceeds the cycle time.
+ * 2. According to IEEE Std. 802.1Q-2018 section 8.6.9.2,
+ * once the end of the list is reached, it will switch
+ * to the END_OF_CYCLE state and leave the gates in the
+ * same state until the next cycle is started.
+ */
+ if (end_time > adapter->cycle_time ||
+ n + 1 == qopt->num_entries)
+ end_time = adapter->cycle_time;
+
for (i = 0; i < adapter->num_tx_queues; i++) {
struct igc_ring *ring = adapter->tx_ring[i];
start_time += e->interval;
}
+ /* Check whether a queue gets configured.
+ * If not, set the start and end time to be end time.
+ */
+ for (i = 0; i < adapter->num_tx_queues; i++) {
+ if (!queue_configured[i]) {
+ struct igc_ring *ring = adapter->tx_ring[i];
+
+ ring->start_time = end_time;
+ ring->end_time = end_time;
+ }
+ }
+
return 0;
}
{
unsigned int new_flags = adapter->flags & ~IGC_FLAG_TSN_ANY_ENABLED;
- if (adapter->base_time)
+ if (adapter->qbv_enable)
new_flags |= IGC_FLAG_TSN_QBV_ENABLED;
if (is_any_launchtime(adapter))
wr32(IGC_STQT(i), ring->start_time);
wr32(IGC_ENDQT(i), ring->end_time);
- if (adapter->base_time) {
- /* If we have a base_time we are in "taprio"
- * mode and we need to be strict about the
- * cycles: only transmit a packet if it can be
- * completed during that cycle.
- */
- txqctl |= IGC_TXQCTL_STRICT_CYCLE |
- IGC_TXQCTL_STRICT_END;
- }
+ txqctl |= IGC_TXQCTL_STRICT_CYCLE |
+ IGC_TXQCTL_STRICT_END;
if (ring->launchtime_enable)
txqctl |= IGC_TXQCTL_QUEUE_MODE_LAUNCHT;
hex = true;
break;
case VCAP_FIELD_U128:
+ value = data->u128.value;
+ mask = data->u128.mask;
if (key == VCAP_KF_L3_IP6_SIP || key == VCAP_KF_L3_IP6_DIP) {
u8 nvalue[16], nmask[16];
myri10ge_free_slices(mgp);
abort_with_firmware:
+ kfree(mgp->msix_vectors);
myri10ge_dummy_rdma(mgp, 0);
abort_with_ioremap:
{
/* Get some of the read-only fields from the BAR */
nn->cap = nn_readl(nn, NFP_NET_CFG_CAP);
- nn->cap_w1 = nn_readq(nn, NFP_NET_CFG_CAP_WORD1);
+ nn->cap_w1 = nn_readl(nn, NFP_NET_CFG_CAP_WORD1);
nn->max_mtu = nn_readl(nn, NFP_NET_CFG_MAX_MTU);
/* ABI 4.x and ctrl vNIC always use chained metadata, in other cases
err = register_netdev(dev);
if (err) {
dev_err(&pdev->dev, "Failed to register net device\n");
- goto err_out_mdio_unregister;
+ goto err_out_phy_disconnect;
}
return 0;
+err_out_phy_disconnect:
+ phy_disconnect(dev->phydev);
err_out_mdio_unregister:
mdiobus_unregister(lp->mii_bus);
err_out_mdio:
struct r6040_private *lp = netdev_priv(dev);
unregister_netdev(dev);
+ phy_disconnect(dev->phydev);
mdiobus_unregister(lp->mii_bus);
mdiobus_free(lp->mii_bus);
netif_napi_del(&lp->napi);
priv->desc_bat_dma);
/* Set reset mode */
ravb_write(ndev, CCC_OPC_RESET, CCC);
- pm_runtime_put_sync(&pdev->dev);
unregister_netdev(ndev);
if (info->nc_queues)
netif_napi_del(&priv->napi[RAVB_NC]);
netif_napi_del(&priv->napi[RAVB_BE]);
ravb_mdio_release(priv);
+ pm_runtime_put_sync(&pdev->dev);
pm_runtime_disable(&pdev->dev);
reset_control_assert(priv->rstc);
free_netdev(ndev);
priv->wq = create_singlethread_workqueue("stmmac_wq");
if (!priv->wq) {
dev_err(priv->device, "failed to create workqueue\n");
+ ret = -ENOMEM;
goto error_wq_init;
}
return 0;
}
-#ifdef CONFIG_PM_SLEEP
static int am65_cpsw_nuss_suspend(struct device *dev)
{
struct am65_cpsw_common *common = dev_get_drvdata(dev);
return 0;
}
-#endif /* CONFIG_PM_SLEEP */
static const struct dev_pm_ops am65_cpsw_nuss_dev_pm_ops = {
- SET_SYSTEM_SLEEP_PM_OPS(am65_cpsw_nuss_suspend, am65_cpsw_nuss_resume)
+ SYSTEM_SLEEP_PM_OPS(am65_cpsw_nuss_suspend, am65_cpsw_nuss_resume)
};
static struct platform_driver am65_cpsw_nuss_driver = {
static dev_t ipvtap_major;
static struct cdev ipvtap_cdev;
-static const void *ipvtap_net_namespace(struct device *d)
+static const void *ipvtap_net_namespace(const struct device *d)
{
- struct net_device *dev = to_net_dev(d->parent);
+ const struct net_device *dev = to_net_dev(d->parent);
return dev_net(dev);
}
const struct macsec_ops *ops;
struct macsec_context ctx;
struct macsec_dev *macsec;
- int ret;
+ int ret = 0;
if (!attrs[MACSEC_ATTR_IFINDEX])
return -EINVAL;
macsec_genl_offload_policy, NULL))
return -EINVAL;
+ rtnl_lock();
+
dev = get_dev_from_nl(genl_info_net(info), attrs);
- if (IS_ERR(dev))
- return PTR_ERR(dev);
+ if (IS_ERR(dev)) {
+ ret = PTR_ERR(dev);
+ goto out;
+ }
macsec = macsec_priv(dev);
- if (!tb_offload[MACSEC_OFFLOAD_ATTR_TYPE])
- return -EINVAL;
+ if (!tb_offload[MACSEC_OFFLOAD_ATTR_TYPE]) {
+ ret = -EINVAL;
+ goto out;
+ }
offload = nla_get_u8(tb_offload[MACSEC_OFFLOAD_ATTR_TYPE]);
if (macsec->offload == offload)
- return 0;
+ goto out;
/* Check if the offloading mode is supported by the underlying layers */
if (offload != MACSEC_OFFLOAD_OFF &&
- !macsec_check_offload(offload, macsec))
- return -EOPNOTSUPP;
+ !macsec_check_offload(offload, macsec)) {
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
/* Check if the net device is busy. */
- if (netif_running(dev))
- return -EBUSY;
-
- rtnl_lock();
+ if (netif_running(dev)) {
+ ret = -EBUSY;
+ goto out;
+ }
prev_offload = macsec->offload;
macsec->offload = offload;
rollback:
macsec->offload = prev_offload;
-
+out:
rtnl_unlock();
return ret;
}
*/
static dev_t macvtap_major;
-static const void *macvtap_net_namespace(struct device *d)
+static const void *macvtap_net_namespace(const struct device *d)
{
- struct net_device *dev = to_net_dev(d->parent);
+ const struct net_device *dev = to_net_dev(d->parent);
return dev_net(dev);
}
#define BYTE_FRAME 0x7e
#define BYTE_ESC 0x7d
+#define FCS_INIT 0xffff
+
static DEFINE_IDA(mctp_serial_ida);
enum mctp_serial_state {
buf[2] = dev->txlen;
if (!dev->txpos)
- dev->txfcs = crc_ccitt(0, buf + 1, 2);
+ dev->txfcs = crc_ccitt(FCS_INIT, buf + 1, 2);
txlen = write_chunk(dev, buf + dev->txpos, 3 - dev->txpos);
if (txlen <= 0) {
case 1:
if (c == MCTP_SERIAL_VERSION) {
dev->rxpos++;
- dev->rxfcs = crc_ccitt_byte(0, c);
+ dev->rxfcs = crc_ccitt_byte(FCS_INIT, c);
} else {
dev->rxstate = STATE_ERR;
}
if (peer->timer_handshake_attempts > MAX_TIMER_HANDSHAKES) {
pr_debug("%s: Handshake for peer %llu (%pISpfsc) did not complete after %d attempts, giving up\n",
peer->device->dev->name, peer->internal_id,
- &peer->endpoint.addr, MAX_TIMER_HANDSHAKES + 2);
+ &peer->endpoint.addr, (int)MAX_TIMER_HANDSHAKES + 2);
del_timer(&peer->timer_send_keepalive);
/* We drop all packets without a keypair and don't try again,
++peer->timer_handshake_attempts;
pr_debug("%s: Handshake for peer %llu (%pISpfsc) did not complete after %d seconds, retrying (try %d)\n",
peer->device->dev->name, peer->internal_id,
- &peer->endpoint.addr, REKEY_TIMEOUT,
+ &peer->endpoint.addr, (int)REKEY_TIMEOUT,
peer->timer_handshake_attempts + 1);
/* We clear the endpoint address src address, in case this is
pr_debug("%s: Retrying handshake with peer %llu (%pISpfsc) because we stopped hearing back after %d seconds\n",
peer->device->dev->name, peer->internal_id,
- &peer->endpoint.addr, KEEPALIVE_TIMEOUT + REKEY_TIMEOUT);
+ &peer->endpoint.addr, (int)(KEEPALIVE_TIMEOUT + REKEY_TIMEOUT));
/* We clear the endpoint address src address, in case this is the cause
* of trouble.
*/
pr_debug("%s: Zeroing out all keys for peer %llu (%pISpfsc), since we haven't received a new one in %d seconds\n",
peer->device->dev->name, peer->internal_id,
- &peer->endpoint.addr, REJECT_AFTER_TIME * 3);
+ &peer->endpoint.addr, (int)REJECT_AFTER_TIME * 3);
wg_noise_handshake_clear(&peer->handshake);
wg_noise_keypairs_clear(&peer->keypairs);
wg_peer_put(peer);
if (IS_ERR(resp))
return PTR_ERR(resp);
+ memset(&nfc_target, 0, sizeof(struct nfc_target));
+
rsp = (struct pn533_cmd_jump_dep_response *)resp->data;
rc = rsp->status & PN533_CMD_RET_MASK;
dev_dbg(dev->dev, "Creating new target\n");
+ memset(&nfc_target, 0, sizeof(struct nfc_target));
+
nfc_target.supported_protocols = NFC_PROTO_NFC_DEP_MASK;
nfc_target.nfcid1_len = 10;
memcpy(nfc_target.nfcid1, rsp->nfcid3t, nfc_target.nfcid1_len);
}
EXPORT_SYMBOL_GPL(nvme_remove_namespaces);
-static int nvme_class_uevent(struct device *dev, struct kobj_uevent_env *env)
+static int nvme_class_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
- struct nvme_ctrl *ctrl =
+ const struct nvme_ctrl *ctrl =
container_of(dev, struct nvme_ctrl, ctrl_device);
struct nvmf_ctrl_options *opts = ctrl->opts;
int ret;
queue->sock->sk->sk_rcvtimeo = 10 * HZ;
queue->sock->sk->sk_allocation = GFP_ATOMIC;
+ queue->sock->sk->sk_use_task_frag = false;
nvme_tcp_set_queue_io_cpu(queue);
queue->request = NULL;
queue->data_remaining = 0;
depends on ARCH_AT91 || COMPILE_TEST
help
This driver enable the OTP controller available on Microchip SAMA7G5
- SoCs. It controlls the access to the OTP memory connected to it.
+ SoCs. It controls the access to the OTP memory connected to it.
config NVMEM_MTK_EFUSE
tristate "Mediatek SoCs EFUSE support"
#define STM32_SMC_WRITE_SHADOW 0x03
#define STM32_SMC_READ_OTP 0x04
-/* shadow registers offest */
+/* shadow registers offset */
#define STM32MP15_BSEC_DATA0 0x200
-/* 32 (x 32-bits) lower shadow registers */
-#define STM32MP15_BSEC_NUM_LOWER 32
-
struct stm32_romem_cfg {
int size;
+ u8 lower;
};
struct stm32_romem_priv {
void __iomem *base;
struct nvmem_config cfg;
+ u8 lower;
};
static int stm32_romem_read(void *context, unsigned int offset, void *buf,
for (i = roffset; (i < roffset + rbytes); i += 4) {
u32 otp = i >> 2;
- if (otp < STM32MP15_BSEC_NUM_LOWER) {
+ if (otp < priv->lower) {
/* read lower data from shadow registers */
val = readl_relaxed(
priv->base + STM32MP15_BSEC_DATA0 + i);
}
}
+ if (offset + bytes >= priv->lower * 4)
+ dev_warn(dev, "Update of upper OTPs with ECC protection (word programming, only once)\n");
+
return 0;
}
priv->cfg.dev = dev;
priv->cfg.priv = priv;
priv->cfg.owner = THIS_MODULE;
+ priv->cfg.type = NVMEM_TYPE_OTP;
+
+ priv->lower = 0;
cfg = (const struct stm32_romem_cfg *)
of_match_device(dev->driver->of_match_table, dev)->data;
priv->cfg.reg_read = stm32_romem_read;
} else {
priv->cfg.size = cfg->size;
+ priv->lower = cfg->lower;
priv->cfg.reg_read = stm32_bsec_read;
priv->cfg.reg_write = stm32_bsec_write;
}
return PTR_ERR_OR_ZERO(devm_nvmem_register(dev, &priv->cfg));
}
+/*
+ * STM32MP15 BSEC OTP regions: 4096 OTP bits (with 3072 effective bits)
+ * => 96 x 32-bits data words
+ * - Lower: 1K bits, 2:1 redundancy, incremental bit programming
+ * => 32 (x 32-bits) lower shadow registers = words 0 to 31
+ * - Upper: 2K bits, ECC protection, word programming only
+ * => 64 (x 32-bits) = words 32 to 95
+ */
static const struct stm32_romem_cfg stm32mp15_bsec_cfg = {
- .size = 384, /* 96 x 32-bits data words */
+ .size = 384,
+ .lower = 32,
};
static const struct of_device_id stm32_romem_of_match[] = {
enum u_boot_env_format {
U_BOOT_FORMAT_SINGLE,
U_BOOT_FORMAT_REDUNDANT,
+ U_BOOT_FORMAT_BROADCOM,
};
struct u_boot_env {
uint8_t data[];
} __packed;
+struct u_boot_env_image_broadcom {
+ __le32 magic;
+ __le32 len;
+ __le32 crc32;
+ uint8_t data[0];
+} __packed;
+
static int u_boot_env_read(void *context, unsigned int offset, void *val,
size_t bytes)
{
crc32_data_offset = offsetof(struct u_boot_env_image_redundant, data);
data_offset = offsetof(struct u_boot_env_image_redundant, data);
break;
+ case U_BOOT_FORMAT_BROADCOM:
+ crc32_offset = offsetof(struct u_boot_env_image_broadcom, crc32);
+ crc32_data_offset = offsetof(struct u_boot_env_image_broadcom, data);
+ data_offset = offsetof(struct u_boot_env_image_broadcom, data);
+ break;
}
crc32 = le32_to_cpu(*(__le32 *)(buf + crc32_offset));
crc32_data_len = priv->mtd->size - crc32_data_offset;
{ .compatible = "u-boot,env", .data = (void *)U_BOOT_FORMAT_SINGLE, },
{ .compatible = "u-boot,env-redundant-bool", .data = (void *)U_BOOT_FORMAT_REDUNDANT, },
{ .compatible = "u-boot,env-redundant-count", .data = (void *)U_BOOT_FORMAT_REDUNDANT, },
+ { .compatible = "brcm,env", .data = (void *)U_BOOT_FORMAT_BROADCOM, },
{},
};
/**
* of_device_uevent - Display OF related uevent information
- * @dev: Device to apply DMA configuration
- * @env: Kernel object's userspace event reference
+ * @dev: Device to display the uevent information for
+ * @env: Kernel object's userspace event reference to fill up
*/
-void of_device_uevent(struct device *dev, struct kobj_uevent_env *env)
+void of_device_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
const char *compat, *type;
struct alias_prop *app;
if (p != NULL && l > 0)
strscpy(cmdline, p, min(l, COMMAND_LINE_SIZE));
- /*
- * CONFIG_CMDLINE is meant to be a default in case nothing else
- * managed to set the command line, unless CONFIG_CMDLINE_FORCE
- * is set in which case we override whatever was found earlier.
- */
-#ifdef CONFIG_CMDLINE
-#if defined(CONFIG_CMDLINE_EXTEND)
- strlcat(cmdline, " ", COMMAND_LINE_SIZE);
- strlcat(cmdline, CONFIG_CMDLINE, COMMAND_LINE_SIZE);
-#elif defined(CONFIG_CMDLINE_FORCE)
- strscpy(cmdline, CONFIG_CMDLINE, COMMAND_LINE_SIZE);
-#else
- /* No arguments from boot loader, use kernel's cmdl*/
- if (!((char *)cmdline)[0])
- strscpy(cmdline, CONFIG_CMDLINE, COMMAND_LINE_SIZE);
-#endif
-#endif /* CONFIG_CMDLINE */
-
- pr_debug("Command line is: %s\n", (char *)cmdline);
-
rng_seed = of_get_flat_dt_prop(node, "rng-seed", &l);
if (rng_seed && l > 0) {
add_bootloader_randomness(rng_seed, l);
if (rc)
pr_warn("No chosen node found, continuing without\n");
+ /*
+ * CONFIG_CMDLINE is meant to be a default in case nothing else
+ * managed to set the command line, unless CONFIG_CMDLINE_FORCE
+ * is set in which case we override whatever was found earlier.
+ */
+#ifdef CONFIG_CMDLINE
+#if defined(CONFIG_CMDLINE_EXTEND)
+ strlcat(boot_command_line, " ", COMMAND_LINE_SIZE);
+ strlcat(boot_command_line, CONFIG_CMDLINE, COMMAND_LINE_SIZE);
+#elif defined(CONFIG_CMDLINE_FORCE)
+ strscpy(boot_command_line, CONFIG_CMDLINE, COMMAND_LINE_SIZE);
+#else
+ /* No arguments from boot loader, use kernel's cmdl */
+ if (!boot_command_line[0])
+ strscpy(boot_command_line, CONFIG_CMDLINE, COMMAND_LINE_SIZE);
+#endif
+#endif /* CONFIG_CMDLINE */
+
+ pr_debug("Command line is: %s\n", boot_command_line);
+
/* Setup memory, calling early_init_dt_add_memory_arch */
early_init_dt_scan_memory();
return NULL;
}
+EXPORT_SYMBOL_GPL(of_msi_get_domain);
/**
* of_msi_configure - Set the msi_domain field of a device
/* Create the work queue and queue the LED task */
led_wq = create_singlethread_workqueue("led_wq");
+ if (!led_wq)
+ return -ENOMEM;
+
queue_delayed_work(led_wq, &led_task, 0);
return 0;
#include <linux/delay.h>
#include <linux/gpio/consumer.h>
#include <linux/mfd/syscon.h>
+#include <linux/phy/pcie.h>
#include <linux/phy/phy.h>
#include <linux/platform_device.h>
#include <linux/pm_domain.h>
if (ret)
goto err_disable_clk;
+ ret = phy_set_mode_ext(pcie_ep->phy, PHY_MODE_PCIE, PHY_MODE_PCIE_EP);
+ if (ret)
+ goto err_phy_exit;
+
ret = phy_power_on(pcie_ep->phy);
if (ret)
goto err_phy_exit;
#include <linux/pci.h>
#include <linux/pm_runtime.h>
#include <linux/platform_device.h>
+#include <linux/phy/pcie.h>
#include <linux/phy/phy.h>
#include <linux/regulator/consumer.h>
#include <linux/reset.h>
if (ret)
return ret;
+ ret = phy_set_mode_ext(pcie->phy, PHY_MODE_PCIE, PHY_MODE_PCIE_RC);
+ if (ret)
+ goto err_deinit;
+
ret = phy_power_on(pcie->phy);
if (ret)
goto err_deinit;
EXPORT_SYMBOL(pcmcia_reset_card);
-static int pcmcia_socket_uevent(struct device *dev,
+static int pcmcia_socket_uevent(const struct device *dev,
struct kobj_uevent_env *env)
{
- struct pcmcia_socket *s = container_of(dev, struct pcmcia_socket, dev);
+ const struct pcmcia_socket *s = container_of(dev, struct pcmcia_socket, dev);
if (add_uevent_var(env, "SOCKET_NO=%u", s->sock))
return -ENOMEM;
u8 phyctl_offset;
bool dedicated_clocks;
bool phy0_dual_route;
+ bool needs_phy2_siddq;
int missing_phys;
};
return ret;
}
+ /* Some PHYs on some SoCs need the help of PHY2 to work. */
+ if (data->cfg->needs_phy2_siddq && phy->index != 2) {
+ struct sun4i_usb_phy *phy2 = &data->phys[2];
+
+ ret = clk_prepare_enable(phy2->clk);
+ if (ret) {
+ reset_control_assert(phy->reset);
+ clk_disable_unprepare(phy->clk2);
+ clk_disable_unprepare(phy->clk);
+ return ret;
+ }
+
+ ret = reset_control_deassert(phy2->reset);
+ if (ret) {
+ clk_disable_unprepare(phy2->clk);
+ reset_control_assert(phy->reset);
+ clk_disable_unprepare(phy->clk2);
+ clk_disable_unprepare(phy->clk);
+ return ret;
+ }
+
+ /*
+ * This extra clock is just needed to access the
+ * REG_HCI_PHY_CTL PMU register for PHY2.
+ */
+ ret = clk_prepare_enable(phy2->clk2);
+ if (ret) {
+ reset_control_assert(phy2->reset);
+ clk_disable_unprepare(phy2->clk);
+ reset_control_assert(phy->reset);
+ clk_disable_unprepare(phy->clk2);
+ clk_disable_unprepare(phy->clk);
+ return ret;
+ }
+
+ if (phy2->pmu && data->cfg->hci_phy_ctl_clear) {
+ val = readl(phy2->pmu + REG_HCI_PHY_CTL);
+ val &= ~data->cfg->hci_phy_ctl_clear;
+ writel(val, phy2->pmu + REG_HCI_PHY_CTL);
+ }
+
+ clk_disable_unprepare(phy->clk2);
+ }
+
if (phy->pmu && data->cfg->hci_phy_ctl_clear) {
val = readl(phy->pmu + REG_HCI_PHY_CTL);
val &= ~data->cfg->hci_phy_ctl_clear;
data->phy0_init = false;
}
+ if (data->cfg->needs_phy2_siddq && phy->index != 2) {
+ struct sun4i_usb_phy *phy2 = &data->phys[2];
+
+ clk_disable_unprepare(phy2->clk);
+ reset_control_assert(phy2->reset);
+ }
+
sun4i_usb_phy_passby(phy, 0);
reset_control_assert(phy->reset);
clk_disable_unprepare(phy->clk2);
dev_err(dev, "failed to get clock %s\n", name);
return PTR_ERR(phy->clk2);
}
+ } else {
+ snprintf(name, sizeof(name), "pmu%d_clk", i);
+ phy->clk2 = devm_clk_get_optional(dev, name);
+ if (IS_ERR(phy->clk2)) {
+ dev_err(dev, "failed to get clock %s\n", name);
+ return PTR_ERR(phy->clk2);
+ }
}
snprintf(name, sizeof(name), "usb%d_reset", i);
.missing_phys = BIT(1) | BIT(2),
};
+static const struct sun4i_usb_phy_cfg sun50i_h616_cfg = {
+ .num_phys = 4,
+ .type = sun50i_h6_phy,
+ .disc_thresh = 3,
+ .phyctl_offset = REG_PHYCTL_A33,
+ .dedicated_clocks = true,
+ .phy0_dual_route = true,
+ .hci_phy_ctl_clear = PHY_CTL_SIDDQ,
+ .needs_phy2_siddq = true,
+};
+
static const struct of_device_id sun4i_usb_phy_of_match[] = {
{ .compatible = "allwinner,sun4i-a10-usb-phy", .data = &sun4i_a10_cfg },
{ .compatible = "allwinner,sun5i-a13-usb-phy", .data = &sun5i_a13_cfg },
{ .compatible = "allwinner,sun50i-a64-usb-phy",
.data = &sun50i_a64_cfg},
{ .compatible = "allwinner,sun50i-h6-usb-phy", .data = &sun50i_h6_cfg },
+ { .compatible = "allwinner,sun50i-h616-usb-phy", .data = &sun50i_h616_cfg },
{ },
};
MODULE_DEVICE_TABLE(of, sun4i_usb_phy_of_match);
#define SUN6I_DPHY_ANA0_REG 0x4c
#define SUN6I_DPHY_ANA0_REG_PWS BIT(31)
+#define SUN6I_DPHY_ANA0_REG_PWEND BIT(30)
+#define SUN6I_DPHY_ANA0_REG_PWENC BIT(29)
#define SUN6I_DPHY_ANA0_REG_DMPC BIT(28)
#define SUN6I_DPHY_ANA0_REG_DMPD(n) (((n) & 0xf) << 24)
+#define SUN6I_DPHY_ANA0_REG_SRXDT(n) (((n) & 0xf) << 20)
+#define SUN6I_DPHY_ANA0_REG_SRXCK(n) (((n) & 0xf) << 16)
+#define SUN6I_DPHY_ANA0_REG_SDIV2 BIT(15)
#define SUN6I_DPHY_ANA0_REG_SLV(n) (((n) & 7) << 12)
#define SUN6I_DPHY_ANA0_REG_DEN(n) (((n) & 0xf) << 8)
+#define SUN6I_DPHY_ANA0_REG_PLR(n) (((n) & 0xf) << 4)
#define SUN6I_DPHY_ANA0_REG_SFB(n) (((n) & 3) << 2)
+#define SUN6I_DPHY_ANA0_REG_RSD BIT(1)
+#define SUN6I_DPHY_ANA0_REG_SELSCK BIT(0)
#define SUN6I_DPHY_ANA1_REG 0x50
#define SUN6I_DPHY_ANA1_REG_VTTMODE BIT(31)
#define SUN6I_DPHY_ANA3_EN_LDOR BIT(18)
#define SUN6I_DPHY_ANA4_REG 0x5c
+#define SUN6I_DPHY_ANA4_REG_EN_MIPI BIT(31)
+#define SUN6I_DPHY_ANA4_REG_EN_COMTEST BIT(30)
+#define SUN6I_DPHY_ANA4_REG_COMTEST(n) (((n) & 3) << 28)
+#define SUN6I_DPHY_ANA4_REG_IB(n) (((n) & 3) << 25)
#define SUN6I_DPHY_ANA4_REG_DMPLVC BIT(24)
#define SUN6I_DPHY_ANA4_REG_DMPLVD(n) (((n) & 0xf) << 20)
+#define SUN6I_DPHY_ANA4_REG_VTT_SET(n) (((n) & 0x7) << 17)
#define SUN6I_DPHY_ANA4_REG_CKDV(n) (((n) & 0x1f) << 12)
#define SUN6I_DPHY_ANA4_REG_TMSC(n) (((n) & 3) << 10)
#define SUN6I_DPHY_ANA4_REG_TMSD(n) (((n) & 3) << 8)
#define SUN6I_DPHY_DBG5_REG 0xf4
+#define SUN50I_DPHY_TX_SLEW_REG0 0xf8
+#define SUN50I_DPHY_TX_SLEW_REG1 0xfc
+#define SUN50I_DPHY_TX_SLEW_REG2 0x100
+
+#define SUN50I_DPHY_PLL_REG0 0x104
+#define SUN50I_DPHY_PLL_REG0_CP36_EN BIT(23)
+#define SUN50I_DPHY_PLL_REG0_LDO_EN BIT(22)
+#define SUN50I_DPHY_PLL_REG0_EN_LVS BIT(21)
+#define SUN50I_DPHY_PLL_REG0_PLL_EN BIT(20)
+#define SUN50I_DPHY_PLL_REG0_P(n) (((n) & 0xf) << 16)
+#define SUN50I_DPHY_PLL_REG0_N(n) (((n) & 0xff) << 8)
+#define SUN50I_DPHY_PLL_REG0_NDET BIT(7)
+#define SUN50I_DPHY_PLL_REG0_TDIV BIT(6)
+#define SUN50I_DPHY_PLL_REG0_M0(n) (((n) & 3) << 4)
+#define SUN50I_DPHY_PLL_REG0_M1(n) ((n) & 0xf)
+
+#define SUN50I_DPHY_PLL_REG1 0x108
+#define SUN50I_DPHY_PLL_REG1_UNLOCK_MDSEL(n) (((n) & 3) << 14)
+#define SUN50I_DPHY_PLL_REG1_LOCKMDSEL BIT(13)
+#define SUN50I_DPHY_PLL_REG1_LOCKDET_EN BIT(12)
+#define SUN50I_DPHY_PLL_REG1_VSETA(n) (((n) & 0x7) << 9)
+#define SUN50I_DPHY_PLL_REG1_VSETD(n) (((n) & 0x7) << 6)
+#define SUN50I_DPHY_PLL_REG1_LPF_SW BIT(5)
+#define SUN50I_DPHY_PLL_REG1_ICP_SEL(n) (((n) & 3) << 3)
+#define SUN50I_DPHY_PLL_REG1_ATEST_SEL(n) (((n) & 3) << 1)
+#define SUN50I_DPHY_PLL_REG1_TEST_EN BIT(0)
+
+#define SUN50I_DPHY_PLL_REG2 0x10c
+#define SUN50I_DPHY_PLL_REG2_SDM_EN BIT(31)
+#define SUN50I_DPHY_PLL_REG2_FF_EN BIT(30)
+#define SUN50I_DPHY_PLL_REG2_SS_EN BIT(29)
+#define SUN50I_DPHY_PLL_REG2_SS_FRAC(n) (((n) & 0x1ff) << 20)
+#define SUN50I_DPHY_PLL_REG2_SS_INT(n) (((n) & 0xff) << 12)
+#define SUN50I_DPHY_PLL_REG2_FRAC(n) ((n) & 0xfff)
+
+#define SUN50I_COMBO_PHY_REG0 0x110
+#define SUN50I_COMBO_PHY_REG0_EN_TEST_COMBOLDO BIT(5)
+#define SUN50I_COMBO_PHY_REG0_EN_TEST_0P8 BIT(4)
+#define SUN50I_COMBO_PHY_REG0_EN_MIPI BIT(3)
+#define SUN50I_COMBO_PHY_REG0_EN_LVDS BIT(2)
+#define SUN50I_COMBO_PHY_REG0_EN_COMBOLDO BIT(1)
+#define SUN50I_COMBO_PHY_REG0_EN_CP BIT(0)
+
+#define SUN50I_COMBO_PHY_REG1 0x114
+#define SUN50I_COMBO_PHY_REG2_REG_VREF1P6(n) (((n) & 0x7) << 4)
+#define SUN50I_COMBO_PHY_REG2_REG_VREF0P8(n) ((n) & 0x7)
+
+#define SUN50I_COMBO_PHY_REG2 0x118
+#define SUN50I_COMBO_PHY_REG2_HS_STOP_DLY(n) ((n) & 0xff)
+
enum sun6i_dphy_direction {
SUN6I_DPHY_DIRECTION_TX,
SUN6I_DPHY_DIRECTION_RX,
};
+struct sun6i_dphy;
+
+struct sun6i_dphy_variant {
+ void (*tx_power_on)(struct sun6i_dphy *dphy);
+ bool rx_supported;
+};
+
struct sun6i_dphy {
struct clk *bus_clk;
struct clk *mod_clk;
struct phy *phy;
struct phy_configure_opts_mipi_dphy config;
+ const struct sun6i_dphy_variant *variant;
enum sun6i_dphy_direction direction;
};
return 0;
}
-static int sun6i_dphy_tx_power_on(struct sun6i_dphy *dphy)
+static void sun6i_a31_mipi_dphy_tx_power_on(struct sun6i_dphy *dphy)
{
u8 lanes_mask = GENMASK(dphy->config.lanes - 1, 0);
- regmap_write(dphy->regs, SUN6I_DPHY_TX_CTL_REG,
- SUN6I_DPHY_TX_CTL_HS_TX_CLK_CONT);
-
- regmap_write(dphy->regs, SUN6I_DPHY_TX_TIME0_REG,
- SUN6I_DPHY_TX_TIME0_LP_CLK_DIV(14) |
- SUN6I_DPHY_TX_TIME0_HS_PREPARE(6) |
- SUN6I_DPHY_TX_TIME0_HS_TRAIL(10));
-
- regmap_write(dphy->regs, SUN6I_DPHY_TX_TIME1_REG,
- SUN6I_DPHY_TX_TIME1_CLK_PREPARE(7) |
- SUN6I_DPHY_TX_TIME1_CLK_ZERO(50) |
- SUN6I_DPHY_TX_TIME1_CLK_PRE(3) |
- SUN6I_DPHY_TX_TIME1_CLK_POST(10));
-
- regmap_write(dphy->regs, SUN6I_DPHY_TX_TIME2_REG,
- SUN6I_DPHY_TX_TIME2_CLK_TRAIL(30));
-
- regmap_write(dphy->regs, SUN6I_DPHY_TX_TIME3_REG, 0);
-
- regmap_write(dphy->regs, SUN6I_DPHY_TX_TIME4_REG,
- SUN6I_DPHY_TX_TIME4_HS_TX_ANA0(3) |
- SUN6I_DPHY_TX_TIME4_HS_TX_ANA1(3));
-
- regmap_write(dphy->regs, SUN6I_DPHY_GCTL_REG,
- SUN6I_DPHY_GCTL_LANE_NUM(dphy->config.lanes) |
- SUN6I_DPHY_GCTL_EN);
-
regmap_write(dphy->regs, SUN6I_DPHY_ANA0_REG,
SUN6I_DPHY_ANA0_REG_PWS |
SUN6I_DPHY_ANA0_REG_DMPC |
SUN6I_DPHY_ANA3_EN_LDOC |
SUN6I_DPHY_ANA3_EN_LDOD);
udelay(1);
+}
+
+static void sun50i_a100_mipi_dphy_tx_power_on(struct sun6i_dphy *dphy)
+{
+ unsigned long mipi_symbol_rate = dphy->config.hs_clk_rate;
+ unsigned int div, n;
+
+ regmap_write(dphy->regs, SUN6I_DPHY_ANA4_REG,
+ SUN6I_DPHY_ANA4_REG_IB(2) |
+ SUN6I_DPHY_ANA4_REG_DMPLVD(4) |
+ SUN6I_DPHY_ANA4_REG_VTT_SET(3) |
+ SUN6I_DPHY_ANA4_REG_CKDV(3) |
+ SUN6I_DPHY_ANA4_REG_TMSD(1) |
+ SUN6I_DPHY_ANA4_REG_TMSC(1) |
+ SUN6I_DPHY_ANA4_REG_TXPUSD(2) |
+ SUN6I_DPHY_ANA4_REG_TXPUSC(3) |
+ SUN6I_DPHY_ANA4_REG_TXDNSD(2) |
+ SUN6I_DPHY_ANA4_REG_TXDNSC(3));
+
+ regmap_update_bits(dphy->regs, SUN6I_DPHY_ANA2_REG,
+ SUN6I_DPHY_ANA2_EN_CK_CPU,
+ SUN6I_DPHY_ANA2_EN_CK_CPU);
+
+ regmap_update_bits(dphy->regs, SUN6I_DPHY_ANA2_REG,
+ SUN6I_DPHY_ANA2_REG_ENIB,
+ SUN6I_DPHY_ANA2_REG_ENIB);
+
+ regmap_write(dphy->regs, SUN6I_DPHY_ANA3_REG,
+ SUN6I_DPHY_ANA3_EN_LDOR |
+ SUN6I_DPHY_ANA3_EN_LDOC |
+ SUN6I_DPHY_ANA3_EN_LDOD);
+
+ regmap_write(dphy->regs, SUN6I_DPHY_ANA0_REG,
+ SUN6I_DPHY_ANA0_REG_PLR(4) |
+ SUN6I_DPHY_ANA0_REG_SFB(1));
+
+ regmap_write(dphy->regs, SUN50I_COMBO_PHY_REG0,
+ SUN50I_COMBO_PHY_REG0_EN_CP);
+
+ /* Choose a divider to limit the VCO frequency to around 2 GHz. */
+ div = 16 >> order_base_2(DIV_ROUND_UP(mipi_symbol_rate, 264000000));
+ n = mipi_symbol_rate * div / 24000000;
+
+ regmap_write(dphy->regs, SUN50I_DPHY_PLL_REG0,
+ SUN50I_DPHY_PLL_REG0_CP36_EN |
+ SUN50I_DPHY_PLL_REG0_LDO_EN |
+ SUN50I_DPHY_PLL_REG0_EN_LVS |
+ SUN50I_DPHY_PLL_REG0_PLL_EN |
+ SUN50I_DPHY_PLL_REG0_NDET |
+ SUN50I_DPHY_PLL_REG0_P((div - 1) % 8) |
+ SUN50I_DPHY_PLL_REG0_N(n) |
+ SUN50I_DPHY_PLL_REG0_M0((div - 1) / 8) |
+ SUN50I_DPHY_PLL_REG0_M1(2));
+
+ /* Disable sigma-delta modulation. */
+ regmap_write(dphy->regs, SUN50I_DPHY_PLL_REG2, 0);
+
+ regmap_update_bits(dphy->regs, SUN6I_DPHY_ANA4_REG,
+ SUN6I_DPHY_ANA4_REG_EN_MIPI,
+ SUN6I_DPHY_ANA4_REG_EN_MIPI);
+
+ regmap_update_bits(dphy->regs, SUN50I_COMBO_PHY_REG0,
+ SUN50I_COMBO_PHY_REG0_EN_MIPI |
+ SUN50I_COMBO_PHY_REG0_EN_COMBOLDO,
+ SUN50I_COMBO_PHY_REG0_EN_MIPI |
+ SUN50I_COMBO_PHY_REG0_EN_COMBOLDO);
+
+ regmap_write(dphy->regs, SUN50I_COMBO_PHY_REG2,
+ SUN50I_COMBO_PHY_REG2_HS_STOP_DLY(20));
+ udelay(1);
+}
+
+static int sun6i_dphy_tx_power_on(struct sun6i_dphy *dphy)
+{
+ u8 lanes_mask = GENMASK(dphy->config.lanes - 1, 0);
+
+ regmap_write(dphy->regs, SUN6I_DPHY_TX_CTL_REG,
+ SUN6I_DPHY_TX_CTL_HS_TX_CLK_CONT);
+
+ regmap_write(dphy->regs, SUN6I_DPHY_TX_TIME0_REG,
+ SUN6I_DPHY_TX_TIME0_LP_CLK_DIV(14) |
+ SUN6I_DPHY_TX_TIME0_HS_PREPARE(6) |
+ SUN6I_DPHY_TX_TIME0_HS_TRAIL(10));
+
+ regmap_write(dphy->regs, SUN6I_DPHY_TX_TIME1_REG,
+ SUN6I_DPHY_TX_TIME1_CLK_PREPARE(7) |
+ SUN6I_DPHY_TX_TIME1_CLK_ZERO(50) |
+ SUN6I_DPHY_TX_TIME1_CLK_PRE(3) |
+ SUN6I_DPHY_TX_TIME1_CLK_POST(10));
+
+ regmap_write(dphy->regs, SUN6I_DPHY_TX_TIME2_REG,
+ SUN6I_DPHY_TX_TIME2_CLK_TRAIL(30));
+
+ regmap_write(dphy->regs, SUN6I_DPHY_TX_TIME3_REG, 0);
+
+ regmap_write(dphy->regs, SUN6I_DPHY_TX_TIME4_REG,
+ SUN6I_DPHY_TX_TIME4_HS_TX_ANA0(3) |
+ SUN6I_DPHY_TX_TIME4_HS_TX_ANA1(3));
+
+ dphy->variant->tx_power_on(dphy);
regmap_update_bits(dphy->regs, SUN6I_DPHY_ANA3_REG,
SUN6I_DPHY_ANA3_EN_VTTC |
SUN6I_DPHY_ANA2_EN_P2S_CPU_MASK,
SUN6I_DPHY_ANA2_EN_P2S_CPU(lanes_mask));
+ regmap_write(dphy->regs, SUN6I_DPHY_GCTL_REG,
+ SUN6I_DPHY_GCTL_LANE_NUM(dphy->config.lanes) |
+ SUN6I_DPHY_GCTL_EN);
+
return 0;
}
.reg_bits = 32,
.val_bits = 32,
.reg_stride = 4,
- .max_register = SUN6I_DPHY_DBG5_REG,
+ .max_register = SUN50I_COMBO_PHY_REG2,
.name = "mipi-dphy",
};
if (!dphy)
return -ENOMEM;
+ dphy->variant = device_get_match_data(&pdev->dev);
+ if (!dphy->variant)
+ return -EINVAL;
+
regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(regs)) {
dev_err(&pdev->dev, "Couldn't map the DPHY encoder registers\n");
ret = of_property_read_string(pdev->dev.of_node, "allwinner,direction",
&direction);
- if (!ret && !strncmp(direction, "rx", 2))
+ if (!ret && !strncmp(direction, "rx", 2)) {
+ if (!dphy->variant->rx_supported) {
+ dev_err(&pdev->dev, "RX not supported on this variant\n");
+ return -EOPNOTSUPP;
+ }
+
dphy->direction = SUN6I_DPHY_DIRECTION_RX;
+ }
phy_set_drvdata(dphy->phy, dphy);
phy_provider = devm_of_phy_provider_register(&pdev->dev, of_phy_simple_xlate);
return PTR_ERR_OR_ZERO(phy_provider);
}
+static const struct sun6i_dphy_variant sun6i_a31_mipi_dphy_variant = {
+ .tx_power_on = sun6i_a31_mipi_dphy_tx_power_on,
+ .rx_supported = true,
+};
+
+static const struct sun6i_dphy_variant sun50i_a100_mipi_dphy_variant = {
+ .tx_power_on = sun50i_a100_mipi_dphy_tx_power_on,
+};
+
static const struct of_device_id sun6i_dphy_of_table[] = {
- { .compatible = "allwinner,sun6i-a31-mipi-dphy" },
+ {
+ .compatible = "allwinner,sun6i-a31-mipi-dphy",
+ .data = &sun6i_a31_mipi_dphy_variant,
+ },
+ {
+ .compatible = "allwinner,sun50i-a100-mipi-dphy",
+ .data = &sun50i_a100_mipi_dphy_variant,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, sun6i_dphy_of_table);
#define PIARBCTL_CAM 0x00
#define PIARBCTL_SPLITTER 0x04
#define PIARBCTL_MISC 0x08
-#define PIARBCTL_MISC_SECURE_MASK 0x80000000
-#define PIARBCTL_MISC_USB_SELECT_MASK 0x40000000
-#define PIARBCTL_MISC_USB_4G_SDRAM_MASK 0x20000000
-#define PIARBCTL_MISC_USB_PRIORITY_MASK 0x000f0000
-#define PIARBCTL_MISC_USB_MEM_PAGE_MASK 0x0000f000
-#define PIARBCTL_MISC_CAM1_MEM_PAGE_MASK 0x00000f00
-#define PIARBCTL_MISC_CAM0_MEM_PAGE_MASK 0x000000f0
-#define PIARBCTL_MISC_SATA_PRIORITY_MASK 0x0000000f
+#define PIARBCTL_MISC_SATA_PRIORITY_MASK GENMASK(3, 0)
+#define PIARBCTL_MISC_CAM0_MEM_PAGE_MASK GENMASK(7, 4)
+#define PIARBCTL_MISC_CAM1_MEM_PAGE_MASK GENMASK(11, 8)
+#define PIARBCTL_MISC_USB_MEM_PAGE_MASK GENMASK(15, 12)
+#define PIARBCTL_MISC_USB_PRIORITY_MASK GENMASK(19, 16)
+#define PIARBCTL_MISC_USB_4G_SDRAM_MASK BIT(29)
+#define PIARBCTL_MISC_USB_SELECT_MASK BIT(30)
+#define PIARBCTL_MISC_SECURE_MASK BIT(31)
#define PIARBCTL_MISC_USB_ONLY_MASK \
(PIARBCTL_MISC_USB_SELECT_MASK | \
/* Register definitions for the USB CTRL block */
#define USB_CTRL_SETUP 0x00
-#define USB_CTRL_SETUP_STRAP_IPP_SEL_MASK 0x02000000
-#define USB_CTRL_SETUP_SCB2_EN_MASK 0x00008000
-#define USB_CTRL_SETUP_tca_drv_sel_MASK 0x01000000
-#define USB_CTRL_SETUP_SCB1_EN_MASK 0x00004000
-#define USB_CTRL_SETUP_SOFT_SHUTDOWN_MASK 0x00000200
-#define USB_CTRL_SETUP_IPP_MASK 0x00000020
-#define USB_CTRL_SETUP_IOC_MASK 0x00000010
+#define USB_CTRL_SETUP_IOC_MASK BIT(4)
+#define USB_CTRL_SETUP_IPP_MASK BIT(5)
+#define USB_CTRL_SETUP_SOFT_SHUTDOWN_MASK BIT(9)
+#define USB_CTRL_SETUP_SCB1_EN_MASK BIT(14)
+#define USB_CTRL_SETUP_SCB2_EN_MASK BIT(15)
+#define USB_CTRL_SETUP_tca_drv_sel_MASK BIT(24)
+#define USB_CTRL_SETUP_STRAP_IPP_SEL_MASK BIT(25)
#define USB_CTRL_USB_PM 0x04
-#define USB_CTRL_USB_PM_USB_PWRDN_MASK 0x80000000
-#define USB_CTRL_USB_PM_SOFT_RESET_MASK 0x40000000
-#define USB_CTRL_USB_PM_BDC_SOFT_RESETB_MASK 0x00800000
-#define USB_CTRL_USB_PM_XHC_SOFT_RESETB_MASK 0x00400000
-#define USB_CTRL_USB_PM_XHC_PME_EN_MASK 0x00000010
-#define USB_CTRL_USB_PM_XHC_S2_CLK_SWITCH_EN_MASK 0x00000008
+#define USB_CTRL_USB_PM_XHC_S2_CLK_SWITCH_EN_MASK BIT(3)
+#define USB_CTRL_USB_PM_XHC_PME_EN_MASK BIT(4)
+#define USB_CTRL_USB_PM_XHC_SOFT_RESETB_MASK BIT(22)
+#define USB_CTRL_USB_PM_BDC_SOFT_RESETB_MASK BIT(23)
+#define USB_CTRL_USB_PM_SOFT_RESET_MASK BIT(30)
+#define USB_CTRL_USB_PM_USB_PWRDN_MASK BIT(31)
#define USB_CTRL_USB_PM_STATUS 0x08
#define USB_CTRL_USB_DEVICE_CTL1 0x10
-#define USB_CTRL_USB_DEVICE_CTL1_PORT_MODE_MASK 0x00000003
+#define USB_CTRL_USB_DEVICE_CTL1_PORT_MODE_MASK GENMASK(1, 0)
#define USB_CTRL_TEST_PORT_CTL 0x30
-#define USB_CTRL_TEST_PORT_CTL_TPOUT_SEL_MASK 0x000000ff
+#define USB_CTRL_TEST_PORT_CTL_TPOUT_SEL_MASK GENMASK(7, 0)
#define USB_CTRL_TEST_PORT_CTL_TPOUT_SEL_PME_GEN_MASK 0x0000002e
#define USB_CTRL_TP_DIAG1 0x34
-#define USB_CTLR_TP_DIAG1_wake_MASK 0x00000002
+#define USB_CTLR_TP_DIAG1_wake_MASK BIT(1)
#define USB_CTRL_CTLR_CSHCR 0x50
-#define USB_CTRL_CTLR_CSHCR_ctl_pme_en_MASK 0x00040000
+#define USB_CTRL_CTLR_CSHCR_ctl_pme_en_MASK BIT(18)
/* Register definitions for the USB_PHY block in 7211b0 */
#define USB_PHY_PLL_CTL 0x00
-#define USB_PHY_PLL_CTL_PLL_RESETB_MASK 0x40000000
+#define USB_PHY_PLL_CTL_PLL_SUSPEND_MASK BIT(27)
+#define USB_PHY_PLL_CTL_PLL_RESETB_MASK BIT(30)
#define USB_PHY_PLL_LDO_CTL 0x08
-#define USB_PHY_PLL_LDO_CTL_AFE_CORERDY_MASK 0x00000004
-#define USB_PHY_PLL_LDO_CTL_AFE_LDO_PWRDWNB_MASK 0x00000002
-#define USB_PHY_PLL_LDO_CTL_AFE_BG_PWRDWNB_MASK 0x00000001
+#define USB_PHY_PLL_LDO_CTL_AFE_BG_PWRDWNB_MASK BIT(0)
+#define USB_PHY_PLL_LDO_CTL_AFE_LDO_PWRDWNB_MASK BIT(1)
+#define USB_PHY_PLL_LDO_CTL_AFE_CORERDY_MASK BIT(2)
#define USB_PHY_UTMI_CTL_1 0x04
-#define USB_PHY_UTMI_CTL_1_POWER_UP_FSM_EN_MASK 0x00000800
-#define USB_PHY_UTMI_CTL_1_PHY_MODE_MASK 0x0000000c
+#define USB_PHY_UTMI_CTL_1_PHY_MODE_MASK GENMASK(3, 2)
#define USB_PHY_UTMI_CTL_1_PHY_MODE_SHIFT 2
+#define USB_PHY_UTMI_CTL_1_POWER_UP_FSM_EN_MASK BIT(11)
#define USB_PHY_IDDQ 0x1c
-#define USB_PHY_IDDQ_phy_iddq_MASK 0x00000001
+#define USB_PHY_IDDQ_phy_iddq_MASK BIT(0)
#define USB_PHY_STATUS 0x20
-#define USB_PHY_STATUS_pll_lock_MASK 0x00000001
+#define USB_PHY_STATUS_pll_lock_MASK BIT(0)
/* Register definitions for the MDIO registers in the DWC2 block of
* the 7211b0.
/* Register definitions for the BDC EC block in 7211b0 */
#define BDC_EC_AXIRDA 0x0c
-#define BDC_EC_AXIRDA_RTS_MASK 0xf0000000
+#define BDC_EC_AXIRDA_RTS_MASK GENMASK(31, 28)
#define BDC_EC_AXIRDA_RTS_SHIFT 28
if (USB_CTRL_MASK(USB_DEVICE_CTL1, PORT_MODE)) {
reg = brcm_usb_readl(USB_CTRL_REG(ctrl, USB_DEVICE_CTL1));
reg &= ~USB_CTRL_MASK(USB_DEVICE_CTL1, PORT_MODE);
- reg |= params->mode;
+ reg |= params->port_mode;
brcm_usb_writel(reg, USB_CTRL_REG(ctrl, USB_DEVICE_CTL1));
}
- switch (params->mode) {
+ switch (params->supported_port_modes) {
case USB_CTLR_MODE_HOST:
USB_CTRL_UNSET(ctrl, USB_PM, BDC_SOFT_RESETB);
break;
brcm_usb_writel(reg, usb_phy + USB_PHY_UTMI_CTL_1);
}
+ /* Disable PLL auto suspend */
+ reg = brcm_usb_readl(usb_phy + USB_PHY_PLL_CTL);
+ reg |= USB_PHY_PLL_CTL_PLL_SUSPEND_MASK;
+ brcm_usb_writel(reg, usb_phy + USB_PHY_PLL_CTL);
+
/* Init the PHY */
reg = USB_PHY_PLL_LDO_CTL_AFE_CORERDY_MASK |
USB_PHY_PLL_LDO_CTL_AFE_LDO_PWRDWNB_MASK |
/* Set the PHY_MODE */
reg = brcm_usb_readl(usb_phy + USB_PHY_UTMI_CTL_1);
reg &= ~USB_PHY_UTMI_CTL_1_PHY_MODE_MASK;
- reg |= params->mode << USB_PHY_UTMI_CTL_1_PHY_MODE_SHIFT;
+ reg |= params->supported_port_modes << USB_PHY_UTMI_CTL_1_PHY_MODE_SHIFT;
brcm_usb_writel(reg, usb_phy + USB_PHY_UTMI_CTL_1);
usb_init_common(params);
* the default "Read Transaction Size" of 6 (1024 bytes).
* Set it to 4 (256 bytes).
*/
- if ((params->mode != USB_CTLR_MODE_HOST) && bdc_ec) {
+ if ((params->supported_port_modes != USB_CTLR_MODE_HOST) && bdc_ec) {
reg = brcm_usb_readl(bdc_ec + BDC_EC_AXIRDA);
reg &= ~BDC_EC_AXIRDA_RTS_MASK;
reg |= (0x4 << BDC_EC_AXIRDA_RTS_SHIFT);
pr_debug("%s\n", __func__);
- if (!params->wake_enabled) {
- USB_CTRL_SET(ctrl, USB_PM, USB_PWRDN);
-
+ if (params->wake_enabled) {
/* Switch to using slower clock during suspend to save power */
USB_CTRL_SET(ctrl, USB_PM, XHC_S2_CLK_SWITCH_EN);
- } else {
usb_wake_enable_7216(params, true);
+ } else {
+ USB_CTRL_SET(ctrl, USB_PM, USB_PWRDN);
}
}
return reg;
}
-static void usb_set_dual_select(struct brcm_usb_init_params *params, int mode)
+static void usb_set_dual_select(struct brcm_usb_init_params *params)
{
void __iomem *ctrl = params->regs[BRCM_REGS_CTRL];
u32 reg;
reg = brcm_usb_readl(USB_CTRL_REG(ctrl, USB_DEVICE_CTL1));
reg &= ~USB_CTRL_MASK(USB_DEVICE_CTL1, PORT_MODE);
- reg |= mode;
+ reg |= params->port_mode;
brcm_usb_writel(reg, USB_CTRL_REG(ctrl, USB_DEVICE_CTL1));
}
params->family_name = "7216";
params->ops = &bcm7216_ops;
- params->suspend_with_clocks = true;
}
void brcm_usb_dvr_init_7211b0(struct brcm_usb_init_params *params)
params->family_name = "7211";
params->ops = &bcm7211b0_ops;
- params->suspend_with_clocks = true;
}
/* Register definitions for the USB CTRL block */
#define USB_CTRL_SETUP 0x00
-#define USB_CTRL_SETUP_IOC_MASK 0x00000010
-#define USB_CTRL_SETUP_IPP_MASK 0x00000020
-#define USB_CTRL_SETUP_BABO_MASK 0x00000001
-#define USB_CTRL_SETUP_FNHW_MASK 0x00000002
-#define USB_CTRL_SETUP_FNBO_MASK 0x00000004
-#define USB_CTRL_SETUP_WABO_MASK 0x00000008
-#define USB_CTRL_SETUP_SCB_CLIENT_SWAP_MASK 0x00002000 /* option */
-#define USB_CTRL_SETUP_SCB1_EN_MASK 0x00004000 /* option */
-#define USB_CTRL_SETUP_SCB2_EN_MASK 0x00008000 /* option */
-#define USB_CTRL_SETUP_SS_EHCI64BIT_EN_MASK 0X00020000 /* option */
-#define USB_CTRL_SETUP_SS_EHCI64BIT_EN_VAR_MASK 0x00010000 /* option */
-#define USB_CTRL_SETUP_STRAP_IPP_SEL_MASK 0x02000000 /* option */
-#define USB_CTRL_SETUP_CC_DRD_MODE_ENABLE_MASK 0x04000000 /* option */
-#define USB_CTRL_SETUP_STRAP_CC_DRD_MODE_ENABLE_SEL_MASK 0x08000000 /* opt */
-#define USB_CTRL_SETUP_OC3_DISABLE_MASK 0xc0000000 /* option */
+#define USB_CTRL_SETUP_BABO_MASK BIT(0)
+#define USB_CTRL_SETUP_FNHW_MASK BIT(1)
+#define USB_CTRL_SETUP_FNBO_MASK BIT(2)
+#define USB_CTRL_SETUP_WABO_MASK BIT(3)
+#define USB_CTRL_SETUP_IOC_MASK BIT(4)
+#define USB_CTRL_SETUP_IPP_MASK BIT(5)
+#define USB_CTRL_SETUP_SCB_CLIENT_SWAP_MASK BIT(13) /* option */
+#define USB_CTRL_SETUP_SCB1_EN_MASK BIT(14) /* option */
+#define USB_CTRL_SETUP_SCB2_EN_MASK BIT(15) /* option */
+#define USB_CTRL_SETUP_SS_EHCI64BIT_EN_MASK BIT(17) /* option */
+#define USB_CTRL_SETUP_SS_EHCI64BIT_EN_VAR_MASK BIT(16) /* option */
+#define USB_CTRL_SETUP_STRAP_IPP_SEL_MASK BIT(25) /* option */
+#define USB_CTRL_SETUP_CC_DRD_MODE_ENABLE_MASK BIT(26) /* option */
+#define USB_CTRL_SETUP_STRAP_CC_DRD_MODE_ENABLE_SEL_MASK BIT(27) /* opt */
+#define USB_CTRL_SETUP_OC3_DISABLE_MASK GENMASK(31, 30) /* option */
#define USB_CTRL_PLL_CTL 0x04
-#define USB_CTRL_PLL_CTL_PLL_SUSPEND_EN_MASK 0x08000000
-#define USB_CTRL_PLL_CTL_PLL_RESETB_MASK 0x40000000
-#define USB_CTRL_PLL_CTL_PLL_IDDQ_PWRDN_MASK 0x80000000 /* option */
+#define USB_CTRL_PLL_CTL_PLL_SUSPEND_EN_MASK BIT(27)
+#define USB_CTRL_PLL_CTL_PLL_RESETB_MASK BIT(30)
+#define USB_CTRL_PLL_CTL_PLL_IDDQ_PWRDN_MASK BIT(31) /* option */
#define USB_CTRL_EBRIDGE 0x0c
-#define USB_CTRL_EBRIDGE_ESTOP_SCB_REQ_MASK 0x00020000 /* option */
-#define USB_CTRL_EBRIDGE_EBR_SCB_SIZE_MASK 0x00000f80 /* option */
+#define USB_CTRL_EBRIDGE_EBR_SCB_SIZE_MASK GENMASK(11, 7) /* option */
+#define USB_CTRL_EBRIDGE_ESTOP_SCB_REQ_MASK BIT(17) /* option */
#define USB_CTRL_OBRIDGE 0x10
-#define USB_CTRL_OBRIDGE_LS_KEEP_ALIVE_MASK 0x08000000
+#define USB_CTRL_OBRIDGE_LS_KEEP_ALIVE_MASK BIT(27)
#define USB_CTRL_MDIO 0x14
#define USB_CTRL_MDIO2 0x18
#define USB_CTRL_UTMI_CTL_1 0x2c
-#define USB_CTRL_UTMI_CTL_1_POWER_UP_FSM_EN_MASK 0x00000800
-#define USB_CTRL_UTMI_CTL_1_POWER_UP_FSM_EN_P1_MASK 0x08000000
+#define USB_CTRL_UTMI_CTL_1_POWER_UP_FSM_EN_MASK BIT(11)
+#define USB_CTRL_UTMI_CTL_1_POWER_UP_FSM_EN_P1_MASK BIT(27)
#define USB_CTRL_USB_PM 0x34
-#define USB_CTRL_USB_PM_BDC_SOFT_RESETB_MASK 0x00800000 /* option */
-#define USB_CTRL_USB_PM_XHC_SOFT_RESETB_MASK 0x00400000 /* option */
-#define USB_CTRL_USB_PM_XHC_SOFT_RESETB_VAR_MASK 0x40000000 /* option */
-#define USB_CTRL_USB_PM_USB_PWRDN_MASK 0x80000000 /* option */
-#define USB_CTRL_USB_PM_SOFT_RESET_MASK 0x40000000 /* option */
-#define USB_CTRL_USB_PM_USB20_HC_RESETB_MASK 0x30000000 /* option */
-#define USB_CTRL_USB_PM_USB20_HC_RESETB_VAR_MASK 0x00300000 /* option */
-#define USB_CTRL_USB_PM_RMTWKUP_EN_MASK 0x00000001
+#define USB_CTRL_USB_PM_RMTWKUP_EN_MASK BIT(0)
+#define USB_CTRL_USB_PM_USB20_HC_RESETB_VAR_MASK GENMASK(21, 20) /* option */
+#define USB_CTRL_USB_PM_XHC_SOFT_RESETB_MASK BIT(22) /* option */
+#define USB_CTRL_USB_PM_BDC_SOFT_RESETB_MASK BIT(23) /* option */
+#define USB_CTRL_USB_PM_USB20_HC_RESETB_MASK GENMASK(29, 28) /* option */
+#define USB_CTRL_USB_PM_XHC_SOFT_RESETB_VAR_MASK BIT(30) /* option */
+#define USB_CTRL_USB_PM_SOFT_RESET_MASK BIT(30) /* option */
+#define USB_CTRL_USB_PM_USB_PWRDN_MASK BIT(31) /* option */
#define USB_CTRL_USB_PM_STATUS 0x38
#define USB_CTRL_USB30_CTL1 0x60
-#define USB_CTRL_USB30_CTL1_PHY3_PLL_SEQ_START_MASK 0x00000010
-#define USB_CTRL_USB30_CTL1_PHY3_RESETB_MASK 0x00010000
-#define USB_CTRL_USB30_CTL1_XHC_SOFT_RESETB_MASK 0x00020000 /* option */
-#define USB_CTRL_USB30_CTL1_USB3_IOC_MASK 0x10000000 /* option */
-#define USB_CTRL_USB30_CTL1_USB3_IPP_MASK 0x20000000 /* option */
+#define USB_CTRL_USB30_CTL1_PHY3_PLL_SEQ_START_MASK BIT(4)
+#define USB_CTRL_USB30_CTL1_PHY3_RESETB_MASK BIT(16)
+#define USB_CTRL_USB30_CTL1_XHC_SOFT_RESETB_MASK BIT(17) /* option */
+#define USB_CTRL_USB30_CTL1_USB3_IOC_MASK BIT(28) /* option */
+#define USB_CTRL_USB30_CTL1_USB3_IPP_MASK BIT(29) /* option */
#define USB_CTRL_USB30_PCTL 0x70
-#define USB_CTRL_USB30_PCTL_PHY3_SOFT_RESETB_MASK 0x00000002
-#define USB_CTRL_USB30_PCTL_PHY3_IDDQ_OVERRIDE_MASK 0x00008000
-#define USB_CTRL_USB30_PCTL_PHY3_SOFT_RESETB_P1_MASK 0x00020000
+#define USB_CTRL_USB30_PCTL_PHY3_SOFT_RESETB_MASK BIT(1)
+#define USB_CTRL_USB30_PCTL_PHY3_IDDQ_OVERRIDE_MASK BIT(15)
+#define USB_CTRL_USB30_PCTL_PHY3_SOFT_RESETB_P1_MASK BIT(17)
#define USB_CTRL_USB_DEVICE_CTL1 0x90
-#define USB_CTRL_USB_DEVICE_CTL1_PORT_MODE_MASK 0x00000003 /* option */
+#define USB_CTRL_USB_DEVICE_CTL1_PORT_MODE_MASK GENMASK(1, 0) /* option */
/* Register definitions for the XHCI EC block */
#define USB_XHCI_EC_IRAADR 0x658
reg = brcm_usb_readl(USB_CTRL_REG(ctrl, USB_DEVICE_CTL1));
reg &= ~USB_CTRL_MASK_FAMILY(params, USB_DEVICE_CTL1,
PORT_MODE);
- reg |= params->mode;
+ reg |= params->port_mode;
brcm_usb_writel(reg, USB_CTRL_REG(ctrl, USB_DEVICE_CTL1));
}
if (USB_CTRL_MASK_FAMILY(params, USB_PM, BDC_SOFT_RESETB)) {
- switch (params->mode) {
+ switch (params->supported_port_modes) {
case USB_CTLR_MODE_HOST:
USB_CTRL_UNSET_FAMILY(params, USB_PM, BDC_SOFT_RESETB);
break;
}
}
if (USB_CTRL_MASK_FAMILY(params, SETUP, CC_DRD_MODE_ENABLE)) {
- if (params->mode == USB_CTLR_MODE_TYPEC_PD)
+ if (params->supported_port_modes == USB_CTLR_MODE_TYPEC_PD)
USB_CTRL_SET_FAMILY(params, SETUP, CC_DRD_MODE_ENABLE);
else
USB_CTRL_UNSET_FAMILY(params, SETUP,
return reg;
}
-static void usb_set_dual_select(struct brcm_usb_init_params *params, int mode)
+static void usb_set_dual_select(struct brcm_usb_init_params *params)
{
void __iomem *ctrl = params->regs[BRCM_REGS_CTRL];
u32 reg;
reg = brcm_usb_readl(USB_CTRL_REG(ctrl, USB_DEVICE_CTL1));
reg &= ~USB_CTRL_MASK_FAMILY(params, USB_DEVICE_CTL1,
PORT_MODE);
- reg |= mode;
+ reg |= params->port_mode;
brcm_usb_writel(reg, USB_CTRL_REG(ctrl, USB_DEVICE_CTL1));
}
}
void (*uninit_eohci)(struct brcm_usb_init_params *params);
void (*uninit_xhci)(struct brcm_usb_init_params *params);
int (*get_dual_select)(struct brcm_usb_init_params *params);
- void (*set_dual_select)(struct brcm_usb_init_params *params, int mode);
+ void (*set_dual_select)(struct brcm_usb_init_params *params);
};
struct brcm_usb_init_params {
void __iomem *regs[BRCM_REGS_MAX];
int ioc;
int ipp;
- int mode;
+ int supported_port_modes;
+ int port_mode;
u32 family_id;
u32 product_id;
int selected_family;
const struct brcm_usb_init_ops *ops;
struct regmap *syscon_piarbctl;
bool wake_enabled;
- bool suspend_with_clocks;
};
void brcm_usb_dvr_init_4908(struct brcm_usb_init_params *params);
return 0;
}
-static inline void brcm_usb_set_dual_select(struct brcm_usb_init_params *ini,
- int mode)
+static inline void brcm_usb_set_dual_select(struct brcm_usb_init_params *ini)
{
if (ini->ops->set_dual_select)
- ini->ops->set_dual_select(ini, mode);
+ ini->ops->set_dual_select(ini);
}
#endif /* _USB_BRCM_COMMON_INIT_H */
static irqreturn_t brcm_usb_phy_wake_isr(int irq, void *dev_id)
{
- struct phy *gphy = dev_id;
+ struct device *dev = dev_id;
- pm_wakeup_event(&gphy->dev, 0);
+ pm_wakeup_event(dev, 0);
return IRQ_HANDLED;
}
return sprintf(buf, "%s\n",
value_to_name(&brcm_dr_mode_to_name[0],
ARRAY_SIZE(brcm_dr_mode_to_name),
- priv->ini.mode));
+ priv->ini.supported_port_modes));
}
static DEVICE_ATTR_RO(dr_mode);
res = name_to_value(&brcm_dual_mode_to_name[0],
ARRAY_SIZE(brcm_dual_mode_to_name), buf, &value);
if (!res) {
- brcm_usb_set_dual_select(&priv->ini, value);
+ priv->ini.port_mode = value;
+ brcm_usb_set_dual_select(&priv->ini);
res = len;
}
mutex_unlock(&sysfs_lock);
priv->suspend_clk = NULL;
}
- priv->wake_irq = platform_get_irq_byname(pdev, "wake");
+ priv->wake_irq = platform_get_irq_byname_optional(pdev, "wake");
if (priv->wake_irq < 0)
- priv->wake_irq = platform_get_irq_byname(pdev, "wakeup");
+ priv->wake_irq = platform_get_irq_byname_optional(pdev, "wakeup");
if (priv->wake_irq >= 0) {
err = devm_request_irq(dev, priv->wake_irq,
brcm_usb_phy_wake_isr, 0,
- dev_name(dev), gphy);
+ dev_name(dev), dev);
if (err < 0)
return err;
device_set_wakeup_capable(dev, 1);
of_property_read_u32(dn, "brcm,ipp", &priv->ini.ipp);
of_property_read_u32(dn, "brcm,ioc", &priv->ini.ioc);
- priv->ini.mode = USB_CTLR_MODE_HOST;
+ priv->ini.supported_port_modes = USB_CTLR_MODE_HOST;
err = of_property_read_string(dn, "dr_mode", &mode);
if (err == 0) {
name_to_value(&brcm_dr_mode_to_name[0],
ARRAY_SIZE(brcm_dr_mode_to_name),
- mode, &priv->ini.mode);
+ mode, &priv->ini.supported_port_modes);
}
+ /* Default port_mode to supported port_modes */
+ priv->ini.port_mode = priv->ini.supported_port_modes;
+
if (of_property_read_bool(dn, "brcm,has-xhci"))
priv->has_xhci = true;
if (of_property_read_bool(dn, "brcm,has-eohci"))
* Create sysfs entries for mode.
* Remove "dual_select" attribute if not in dual mode
*/
- if (priv->ini.mode != USB_CTLR_MODE_DRD)
+ if (priv->ini.supported_port_modes != USB_CTLR_MODE_DRD)
brcm_usb_phy_attrs[1] = NULL;
err = sysfs_create_group(&dev->kobj, &brcm_usb_phy_group);
if (err)
* and newer XHCI->2.0-clks/3.0-clks.
*/
- if (!priv->ini.suspend_with_clocks) {
+ if (!priv->ini.wake_enabled) {
if (priv->phys[BRCM_USB_PHY_3_0].inited)
clk_disable_unprepare(priv->usb_30_clk);
if (priv->phys[BRCM_USB_PHY_2_0].inited ||
{
struct brcm_usb_phy_data *priv = dev_get_drvdata(dev);
- clk_prepare_enable(priv->usb_20_clk);
- clk_prepare_enable(priv->usb_30_clk);
+ if (!priv->ini.wake_enabled) {
+ clk_prepare_enable(priv->usb_20_clk);
+ clk_prepare_enable(priv->usb_30_clk);
+ }
brcm_usb_init_ipp(&priv->ini);
/*
#include <linux/mfd/syscon.h>
#include <linux/mfd/syscon/imx7-iomuxc-gpr.h>
#include <linux/module.h>
+#include <linux/of_device.h>
#include <linux/phy/phy.h>
#include <linux/platform_device.h>
#include <linux/regmap.h>
#define IMX8MM_PCIE_PHY_CMN_REG065 0x194
#define ANA_AUX_RX_TERM (BIT(7) | BIT(4))
#define ANA_AUX_TX_LVL GENMASK(3, 0)
-#define IMX8MM_PCIE_PHY_CMN_REG75 0x1D4
-#define PCIE_PHY_CMN_REG75_PLL_DONE 0x3
+#define IMX8MM_PCIE_PHY_CMN_REG075 0x1D4
+#define ANA_PLL_DONE 0x3
#define PCIE_PHY_TRSV_REG5 0x414
-#define PCIE_PHY_TRSV_REG5_GEN1_DEEMP 0x2D
#define PCIE_PHY_TRSV_REG6 0x418
-#define PCIE_PHY_TRSV_REG6_GEN2_DEEMP 0xF
#define IMX8MM_GPR_PCIE_REF_CLK_SEL GENMASK(25, 24)
#define IMX8MM_GPR_PCIE_REF_CLK_PLL FIELD_PREP(IMX8MM_GPR_PCIE_REF_CLK_SEL, 0x3)
#define IMX8MM_GPR_PCIE_SSC_EN BIT(16)
#define IMX8MM_GPR_PCIE_AUX_EN_OVERRIDE BIT(9)
+enum imx8_pcie_phy_type {
+ IMX8MM,
+ IMX8MP,
+};
+
+struct imx8_pcie_phy_drvdata {
+ const char *gpr;
+ enum imx8_pcie_phy_type variant;
+};
+
struct imx8_pcie_phy {
void __iomem *base;
struct clk *clk;
struct phy *phy;
struct regmap *iomuxc_gpr;
+ struct reset_control *perst;
struct reset_control *reset;
u32 refclk_pad_mode;
u32 tx_deemph_gen1;
u32 tx_deemph_gen2;
bool clkreq_unused;
+ const struct imx8_pcie_phy_drvdata *drvdata;
};
static int imx8_pcie_phy_power_on(struct phy *phy)
u32 val, pad_mode;
struct imx8_pcie_phy *imx8_phy = phy_get_drvdata(phy);
- reset_control_assert(imx8_phy->reset);
-
pad_mode = imx8_phy->refclk_pad_mode;
- /* Set AUX_EN_OVERRIDE 1'b0, when the CLKREQ# isn't hooked */
- regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
- IMX8MM_GPR_PCIE_AUX_EN_OVERRIDE,
- imx8_phy->clkreq_unused ?
- 0 : IMX8MM_GPR_PCIE_AUX_EN_OVERRIDE);
- regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
- IMX8MM_GPR_PCIE_AUX_EN,
- IMX8MM_GPR_PCIE_AUX_EN);
- regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
- IMX8MM_GPR_PCIE_POWER_OFF, 0);
- regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
- IMX8MM_GPR_PCIE_SSC_EN, 0);
-
- regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
- IMX8MM_GPR_PCIE_REF_CLK_SEL,
- pad_mode == IMX8_PCIE_REFCLK_PAD_INPUT ?
- IMX8MM_GPR_PCIE_REF_CLK_EXT :
- IMX8MM_GPR_PCIE_REF_CLK_PLL);
- usleep_range(100, 200);
-
- /* Do the PHY common block reset */
- regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
- IMX8MM_GPR_PCIE_CMN_RST,
- IMX8MM_GPR_PCIE_CMN_RST);
- usleep_range(200, 500);
+ switch (imx8_phy->drvdata->variant) {
+ case IMX8MM:
+ reset_control_assert(imx8_phy->reset);
+
+ /* Tune PHY de-emphasis setting to pass PCIe compliance. */
+ if (imx8_phy->tx_deemph_gen1)
+ writel(imx8_phy->tx_deemph_gen1,
+ imx8_phy->base + PCIE_PHY_TRSV_REG5);
+ if (imx8_phy->tx_deemph_gen2)
+ writel(imx8_phy->tx_deemph_gen2,
+ imx8_phy->base + PCIE_PHY_TRSV_REG6);
+ break;
+ case IMX8MP: /* Do nothing. */
+ break;
+ }
if (pad_mode == IMX8_PCIE_REFCLK_PAD_INPUT ||
pad_mode == IMX8_PCIE_REFCLK_PAD_UNUSED) {
imx8_phy->base + IMX8MM_PCIE_PHY_CMN_REG065);
}
- /* Tune PHY de-emphasis setting to pass PCIe compliance. */
- if (imx8_phy->tx_deemph_gen1)
- writel(imx8_phy->tx_deemph_gen1,
- imx8_phy->base + PCIE_PHY_TRSV_REG5);
- if (imx8_phy->tx_deemph_gen2)
- writel(imx8_phy->tx_deemph_gen2,
- imx8_phy->base + PCIE_PHY_TRSV_REG6);
+ /* Set AUX_EN_OVERRIDE 1'b0, when the CLKREQ# isn't hooked */
+ regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
+ IMX8MM_GPR_PCIE_AUX_EN_OVERRIDE,
+ imx8_phy->clkreq_unused ?
+ 0 : IMX8MM_GPR_PCIE_AUX_EN_OVERRIDE);
+ regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
+ IMX8MM_GPR_PCIE_AUX_EN,
+ IMX8MM_GPR_PCIE_AUX_EN);
+ regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
+ IMX8MM_GPR_PCIE_POWER_OFF, 0);
+ regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
+ IMX8MM_GPR_PCIE_SSC_EN, 0);
+
+ regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
+ IMX8MM_GPR_PCIE_REF_CLK_SEL,
+ pad_mode == IMX8_PCIE_REFCLK_PAD_INPUT ?
+ IMX8MM_GPR_PCIE_REF_CLK_EXT :
+ IMX8MM_GPR_PCIE_REF_CLK_PLL);
+ usleep_range(100, 200);
+
+ /* Do the PHY common block reset */
+ regmap_update_bits(imx8_phy->iomuxc_gpr, IOMUXC_GPR14,
+ IMX8MM_GPR_PCIE_CMN_RST,
+ IMX8MM_GPR_PCIE_CMN_RST);
- reset_control_deassert(imx8_phy->reset);
+ switch (imx8_phy->drvdata->variant) {
+ case IMX8MP:
+ reset_control_deassert(imx8_phy->perst);
+ fallthrough;
+ case IMX8MM:
+ reset_control_deassert(imx8_phy->reset);
+ usleep_range(200, 500);
+ break;
+ }
/* Polling to check the phy is ready or not. */
- ret = readl_poll_timeout(imx8_phy->base + IMX8MM_PCIE_PHY_CMN_REG75,
- val, val == PCIE_PHY_CMN_REG75_PLL_DONE,
- 10, 20000);
+ ret = readl_poll_timeout(imx8_phy->base + IMX8MM_PCIE_PHY_CMN_REG075,
+ val, val == ANA_PLL_DONE, 10, 20000);
return ret;
}
.owner = THIS_MODULE,
};
+static const struct imx8_pcie_phy_drvdata imx8mm_drvdata = {
+ .gpr = "fsl,imx8mm-iomuxc-gpr",
+ .variant = IMX8MM,
+};
+
+static const struct imx8_pcie_phy_drvdata imx8mp_drvdata = {
+ .gpr = "fsl,imx8mp-iomuxc-gpr",
+ .variant = IMX8MP,
+};
+
+static const struct of_device_id imx8_pcie_phy_of_match[] = {
+ {.compatible = "fsl,imx8mm-pcie-phy", .data = &imx8mm_drvdata, },
+ {.compatible = "fsl,imx8mp-pcie-phy", .data = &imx8mp_drvdata, },
+ { },
+};
+MODULE_DEVICE_TABLE(of, imx8_pcie_phy_of_match);
+
static int imx8_pcie_phy_probe(struct platform_device *pdev)
{
struct phy_provider *phy_provider;
if (!imx8_phy)
return -ENOMEM;
+ imx8_phy->drvdata = of_device_get_match_data(dev);
+
/* get PHY refclk pad mode */
of_property_read_u32(np, "fsl,refclk-pad-mode",
&imx8_phy->refclk_pad_mode);
/* Grab GPR config register range */
imx8_phy->iomuxc_gpr =
- syscon_regmap_lookup_by_compatible("fsl,imx6q-iomuxc-gpr");
+ syscon_regmap_lookup_by_compatible(imx8_phy->drvdata->gpr);
if (IS_ERR(imx8_phy->iomuxc_gpr)) {
dev_err(dev, "unable to find iomuxc registers\n");
return PTR_ERR(imx8_phy->iomuxc_gpr);
return PTR_ERR(imx8_phy->reset);
}
+ if (imx8_phy->drvdata->variant == IMX8MP) {
+ imx8_phy->perst =
+ devm_reset_control_get_exclusive(dev, "perst");
+ if (IS_ERR(imx8_phy->perst))
+ dev_err_probe(dev, PTR_ERR(imx8_phy->perst),
+ "Failed to get PCIE PHY PERST control\n");
+ }
+
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
imx8_phy->base = devm_ioremap_resource(dev, res);
if (IS_ERR(imx8_phy->base))
return PTR_ERR_OR_ZERO(phy_provider);
}
-static const struct of_device_id imx8_pcie_phy_of_match[] = {
- {.compatible = "fsl,imx8mm-pcie-phy",},
- { },
-};
-MODULE_DEVICE_TABLE(of, imx8_pcie_phy_of_match);
-
static struct platform_driver imx8_pcie_phy_driver = {
.probe = imx8_pcie_phy_probe,
.driver = {
{
struct device *dev = &pdev->dev;
struct phy_provider *provider;
- struct resource *resource;
void __iomem *base;
struct phy *phy;
- resource = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- base = devm_ioremap_resource(dev, resource);
+ base = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
if (IS_ERR(base))
return PTR_ERR(base);
if (ret)
return ret;
+ /* COMPHY register reset (cleared automatically) */
+ comphy_lane_reg_set(lane, COMPHY_SFT_RESET, SFT_RST, SFT_RST);
+
/*
* 0. Set PHY OTG Control(0x5d034), bit 4, Power up OTG module The
* register belong to UTMI module, so it is set in UTMI phy driver.
tristate "Qualcomm QMP PHY Driver"
depends on OF && COMMON_CLK && (ARCH_QCOM || COMPILE_TEST)
select GENERIC_PHY
+ select MFD_SYSCON
help
Enable this to support the QMP PHY transceiver that is used
with controllers such as PCIe, UFS, and USB on Qualcomm chips.
#include <linux/reset.h>
#include <linux/slab.h>
-#include <dt-bindings/phy/phy.h>
+#include <dt-bindings/phy/phy-qcom-qmp.h>
#include "phy-qcom-qmp.h"
#define CLAMP_EN BIT(0) /* enables i/o clamp_n */
#define PHY_INIT_COMPLETE_TIMEOUT 10000
-#define POWER_DOWN_DELAY_US_MIN 10
-#define POWER_DOWN_DELAY_US_MAX 11
struct qmp_phy_init_tbl {
unsigned int offset;
unsigned int val;
/*
- * register part of layout ?
- * if yes, then offset gives index in the reg-layout
- */
- bool in_layout;
- /*
* mask of lanes for which this register is written
* for cases when second lane needs different values
*/
.lane_mask = 0xff, \
}
-#define QMP_PHY_INIT_CFG_L(o, v) \
- { \
- .offset = o, \
- .val = v, \
- .in_layout = true, \
- .lane_mask = 0xff, \
- }
-
#define QMP_PHY_INIT_CFG_LANE(o, v, l) \
{ \
.offset = o, \
[QPHY_SW_RESET] = 0x00,
[QPHY_START_CTRL] = 0x08,
[QPHY_PCS_STATUS] = 0x174,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = 0x04,
[QPHY_PCS_AUTONOMOUS_MODE_CTRL] = 0x0d8,
[QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR] = 0x0dc,
[QPHY_PCS_LFPS_RXTERM_IRQ_STATUS] = 0x170,
{ 0x3f, 0xff, 0xff, 0xff }
};
-struct qmp_phy;
+struct qmp_combo;
+
+struct qmp_combo_offsets {
+ u16 com;
+ u16 txa;
+ u16 rxa;
+ u16 txb;
+ u16 rxb;
+ u16 usb3_serdes;
+ u16 usb3_pcs_misc;
+ u16 usb3_pcs;
+ u16 usb3_pcs_usb;
+ u16 dp_serdes;
+ u16 dp_dp_phy;
+};
-/* struct qmp_phy_cfg - per-PHY initialization config */
struct qmp_phy_cfg {
- /* phy-type - PCIE/UFS/USB */
- unsigned int type;
- int lanes;
+ const struct qmp_combo_offsets *offsets;
/* Init sequence for PHY blocks - serdes, tx, rx, pcs */
const struct qmp_phy_init_tbl *serdes_tbl;
const struct qmp_phy_init_tbl *pcs_usb_tbl;
int pcs_usb_tbl_num;
+ const struct qmp_phy_init_tbl *dp_serdes_tbl;
+ int dp_serdes_tbl_num;
+ const struct qmp_phy_init_tbl *dp_tx_tbl;
+ int dp_tx_tbl_num;
+
/* Init sequence for DP PHY block link rates */
const struct qmp_phy_init_tbl *serdes_tbl_rbr;
int serdes_tbl_rbr_num;
const u8 (*pre_emphasis_hbr3_hbr2)[4][4];
/* DP PHY callbacks */
- int (*configure_dp_phy)(struct qmp_phy *qphy);
- void (*configure_dp_tx)(struct qmp_phy *qphy);
- int (*calibrate_dp_phy)(struct qmp_phy *qphy);
- void (*dp_aux_init)(struct qmp_phy *qphy);
+ int (*configure_dp_phy)(struct qmp_combo *qmp);
+ void (*configure_dp_tx)(struct qmp_combo *qmp);
+ int (*calibrate_dp_phy)(struct qmp_combo *qmp);
+ void (*dp_aux_init)(struct qmp_combo *qmp);
/* clock ids to be requested */
const char * const *clk_list;
/* array of registers with different offsets */
const unsigned int *regs;
- unsigned int start_ctrl;
- unsigned int pwrdn_ctrl;
- /* bit offset of PHYSTATUS in QPHY_PCS_STATUS register */
- unsigned int phy_status;
-
/* true, if PHY needs delay after POWER_DOWN */
bool has_pwrdn_delay;
- /* power_down delay in usec */
- int pwrdn_delay_min;
- int pwrdn_delay_max;
/* Offset from PCS to PCS_USB region */
unsigned int pcs_usb_offset;
};
-struct qmp_phy_combo_cfg {
- const struct qmp_phy_cfg *usb_cfg;
- const struct qmp_phy_cfg *dp_cfg;
-};
+struct qmp_combo {
+ struct device *dev;
-/**
- * struct qmp_phy - per-lane phy descriptor
- *
- * @phy: generic phy
- * @cfg: phy specific configuration
- * @serdes: iomapped memory space for phy's serdes (i.e. PLL)
- * @tx: iomapped memory space for lane's tx
- * @rx: iomapped memory space for lane's rx
- * @pcs: iomapped memory space for lane's pcs
- * @tx2: iomapped memory space for second lane's tx (in dual lane PHYs)
- * @rx2: iomapped memory space for second lane's rx (in dual lane PHYs)
- * @pcs_misc: iomapped memory space for lane's pcs_misc
- * @pcs_usb: iomapped memory space for lane's pcs_usb
- * @pipe_clk: pipe clock
- * @qmp: QMP phy to which this lane belongs
- * @mode: current PHY mode
- * @dp_aux_cfg: Display port aux config
- * @dp_opts: Display port optional config
- * @dp_clks: Display port clocks
- */
-struct qmp_phy {
- struct phy *phy;
const struct qmp_phy_cfg *cfg;
+
+ void __iomem *com;
+
void __iomem *serdes;
void __iomem *tx;
void __iomem *rx;
void __iomem *rx2;
void __iomem *pcs_misc;
void __iomem *pcs_usb;
- struct clk *pipe_clk;
- struct qcom_qmp *qmp;
- enum phy_mode mode;
- unsigned int dp_aux_cfg;
- struct phy_configure_opts_dp dp_opts;
- struct qmp_phy_dp_clks *dp_clks;
-};
-struct qmp_phy_dp_clks {
- struct qmp_phy *qphy;
- struct clk_hw dp_link_hw;
- struct clk_hw dp_pixel_hw;
-};
-
-/**
- * struct qcom_qmp - structure holding QMP phy block attributes
- *
- * @dev: device
- * @dp_com: iomapped memory space for phy's dp_com control block
- *
- * @clks: array of clocks required by phy
- * @resets: array of resets required by phy
- * @vregs: regulator supplies bulk data
- *
- * @phys: array of per-lane phy descriptors
- * @phy_mutex: mutex lock for PHY common block initialization
- * @init_count: phy common block initialization count
- * @ufs_reset: optional UFS PHY reset handle
- */
-struct qcom_qmp {
- struct device *dev;
- void __iomem *dp_com;
+ void __iomem *dp_serdes;
+ void __iomem *dp_tx;
+ void __iomem *dp_tx2;
+ void __iomem *dp_dp_phy;
+ struct clk *pipe_clk;
struct clk_bulk_data *clks;
struct reset_control_bulk_data *resets;
struct regulator_bulk_data *vregs;
- struct qmp_phy **phys;
-
struct mutex phy_mutex;
int init_count;
- struct reset_control *ufs_reset;
+ struct phy *usb_phy;
+ enum phy_mode mode;
+
+ struct phy *dp_phy;
+ unsigned int dp_aux_cfg;
+ struct phy_configure_opts_dp dp_opts;
+
+ struct clk_fixed_rate pipe_clk_fixed;
+ struct clk_hw dp_link_hw;
+ struct clk_hw dp_pixel_hw;
};
-static void qcom_qmp_v3_phy_dp_aux_init(struct qmp_phy *qphy);
-static void qcom_qmp_v3_phy_configure_dp_tx(struct qmp_phy *qphy);
-static int qcom_qmp_v3_phy_configure_dp_phy(struct qmp_phy *qphy);
-static int qcom_qmp_v3_dp_phy_calibrate(struct qmp_phy *qphy);
+static void qmp_v3_dp_aux_init(struct qmp_combo *qmp);
+static void qmp_v3_configure_dp_tx(struct qmp_combo *qmp);
+static int qmp_v3_configure_dp_phy(struct qmp_combo *qmp);
+static int qmp_v3_calibrate_dp_phy(struct qmp_combo *qmp);
-static void qcom_qmp_v4_phy_dp_aux_init(struct qmp_phy *qphy);
-static void qcom_qmp_v4_phy_configure_dp_tx(struct qmp_phy *qphy);
-static int qcom_qmp_v4_phy_configure_dp_phy(struct qmp_phy *qphy);
-static int qcom_qmp_v4_dp_phy_calibrate(struct qmp_phy *qphy);
+static void qmp_v4_dp_aux_init(struct qmp_combo *qmp);
+static void qmp_v4_configure_dp_tx(struct qmp_combo *qmp);
+static int qmp_v4_configure_dp_phy(struct qmp_combo *qmp);
+static int qmp_v4_calibrate_dp_phy(struct qmp_combo *qmp);
-static int qcom_qmp_v5_phy_configure_dp_phy(struct qmp_phy *qphy);
+static int qmp_v5_configure_dp_phy(struct qmp_combo *qmp);
static inline void qphy_setbits(void __iomem *base, u32 offset, u32 val)
{
};
static const char * const qmp_v4_phy_clk_l[] = {
- "aux", "ref_clk_src", "ref", "com_aux",
+ "aux", "ref", "com_aux",
};
/* the primary usb3 phy on sm8250 doesn't have a ref clock */
"phy",
};
-static const struct qmp_phy_cfg sc7180_usb3phy_cfg = {
- .type = PHY_TYPE_USB3,
- .lanes = 2,
+static const struct qmp_combo_offsets qmp_combo_offsets_v5 = {
+ .com = 0x0000,
+ .txa = 0x0400,
+ .rxa = 0x0600,
+ .txb = 0x0a00,
+ .rxb = 0x0c00,
+ .usb3_serdes = 0x1000,
+ .usb3_pcs_misc = 0x1200,
+ .usb3_pcs = 0x1400,
+ .usb3_pcs_usb = 0x1700,
+ .dp_serdes = 0x2000,
+ .dp_dp_phy = 0x2200,
+};
+static const struct qmp_phy_cfg sc7180_usb3dpphy_cfg = {
.serdes_tbl = qmp_v3_usb3_serdes_tbl,
.serdes_tbl_num = ARRAY_SIZE(qmp_v3_usb3_serdes_tbl),
.tx_tbl = qmp_v3_usb3_tx_tbl,
.rx_tbl_num = ARRAY_SIZE(qmp_v3_usb3_rx_tbl),
.pcs_tbl = qmp_v3_usb3_pcs_tbl,
.pcs_tbl_num = ARRAY_SIZE(qmp_v3_usb3_pcs_tbl),
- .clk_list = qmp_v3_phy_clk_l,
- .num_clks = ARRAY_SIZE(qmp_v3_phy_clk_l),
- .reset_list = sc7180_usb3phy_reset_l,
- .num_resets = ARRAY_SIZE(sc7180_usb3phy_reset_l),
- .vreg_list = qmp_phy_vreg_l,
- .num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
- .regs = qmp_v3_usb3phy_regs_layout,
-
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
-};
-static const struct qmp_phy_cfg sc7180_dpphy_cfg = {
- .type = PHY_TYPE_DP,
- .lanes = 2,
-
- .serdes_tbl = qmp_v3_dp_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(qmp_v3_dp_serdes_tbl),
- .tx_tbl = qmp_v3_dp_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(qmp_v3_dp_tx_tbl),
+ .dp_serdes_tbl = qmp_v3_dp_serdes_tbl,
+ .dp_serdes_tbl_num = ARRAY_SIZE(qmp_v3_dp_serdes_tbl),
+ .dp_tx_tbl = qmp_v3_dp_tx_tbl,
+ .dp_tx_tbl_num = ARRAY_SIZE(qmp_v3_dp_tx_tbl),
.serdes_tbl_rbr = qmp_v3_dp_serdes_tbl_rbr,
.serdes_tbl_rbr_num = ARRAY_SIZE(qmp_v3_dp_serdes_tbl_rbr),
.swing_hbr3_hbr2 = &qmp_dp_v3_voltage_swing_hbr3_hbr2,
.pre_emphasis_hbr3_hbr2 = &qmp_dp_v3_pre_emphasis_hbr3_hbr2,
+ .dp_aux_init = qmp_v3_dp_aux_init,
+ .configure_dp_tx = qmp_v3_configure_dp_tx,
+ .configure_dp_phy = qmp_v3_configure_dp_phy,
+ .calibrate_dp_phy = qmp_v3_calibrate_dp_phy,
+
.clk_list = qmp_v3_phy_clk_l,
.num_clks = ARRAY_SIZE(qmp_v3_phy_clk_l),
.reset_list = sc7180_usb3phy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = qmp_v3_usb3phy_regs_layout,
- .dp_aux_init = qcom_qmp_v3_phy_dp_aux_init,
- .configure_dp_tx = qcom_qmp_v3_phy_configure_dp_tx,
- .configure_dp_phy = qcom_qmp_v3_phy_configure_dp_phy,
- .calibrate_dp_phy = qcom_qmp_v3_dp_phy_calibrate,
-};
-
-static const struct qmp_phy_combo_cfg sc7180_usb3dpphy_cfg = {
- .usb_cfg = &sc7180_usb3phy_cfg,
- .dp_cfg = &sc7180_dpphy_cfg,
+ .has_pwrdn_delay = true,
};
-static const struct qmp_phy_cfg sdm845_usb3phy_cfg = {
- .type = PHY_TYPE_USB3,
- .lanes = 2,
-
+static const struct qmp_phy_cfg sdm845_usb3dpphy_cfg = {
.serdes_tbl = qmp_v3_usb3_serdes_tbl,
.serdes_tbl_num = ARRAY_SIZE(qmp_v3_usb3_serdes_tbl),
.tx_tbl = qmp_v3_usb3_tx_tbl,
.rx_tbl_num = ARRAY_SIZE(qmp_v3_usb3_rx_tbl),
.pcs_tbl = qmp_v3_usb3_pcs_tbl,
.pcs_tbl_num = ARRAY_SIZE(qmp_v3_usb3_pcs_tbl),
+
+ .dp_serdes_tbl = qmp_v3_dp_serdes_tbl,
+ .dp_serdes_tbl_num = ARRAY_SIZE(qmp_v3_dp_serdes_tbl),
+ .dp_tx_tbl = qmp_v3_dp_tx_tbl,
+ .dp_tx_tbl_num = ARRAY_SIZE(qmp_v3_dp_tx_tbl),
+
+ .serdes_tbl_rbr = qmp_v3_dp_serdes_tbl_rbr,
+ .serdes_tbl_rbr_num = ARRAY_SIZE(qmp_v3_dp_serdes_tbl_rbr),
+ .serdes_tbl_hbr = qmp_v3_dp_serdes_tbl_hbr,
+ .serdes_tbl_hbr_num = ARRAY_SIZE(qmp_v3_dp_serdes_tbl_hbr),
+ .serdes_tbl_hbr2 = qmp_v3_dp_serdes_tbl_hbr2,
+ .serdes_tbl_hbr2_num = ARRAY_SIZE(qmp_v3_dp_serdes_tbl_hbr2),
+ .serdes_tbl_hbr3 = qmp_v3_dp_serdes_tbl_hbr3,
+ .serdes_tbl_hbr3_num = ARRAY_SIZE(qmp_v3_dp_serdes_tbl_hbr3),
+
+ .swing_hbr_rbr = &qmp_dp_v3_voltage_swing_hbr_rbr,
+ .pre_emphasis_hbr_rbr = &qmp_dp_v3_pre_emphasis_hbr_rbr,
+ .swing_hbr3_hbr2 = &qmp_dp_v3_voltage_swing_hbr3_hbr2,
+ .pre_emphasis_hbr3_hbr2 = &qmp_dp_v3_pre_emphasis_hbr3_hbr2,
+
+ .dp_aux_init = qmp_v3_dp_aux_init,
+ .configure_dp_tx = qmp_v3_configure_dp_tx,
+ .configure_dp_phy = qmp_v3_configure_dp_phy,
+ .calibrate_dp_phy = qmp_v3_calibrate_dp_phy,
+
.clk_list = qmp_v3_phy_clk_l,
.num_clks = ARRAY_SIZE(qmp_v3_phy_clk_l),
.reset_list = msm8996_usb3phy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = qmp_v3_usb3phy_regs_layout,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
};
-static const struct qmp_phy_combo_cfg sdm845_usb3dpphy_cfg = {
- .usb_cfg = &sdm845_usb3phy_cfg,
- .dp_cfg = &sc7180_dpphy_cfg,
-};
-
-static const struct qmp_phy_cfg sm8150_usb3phy_cfg = {
- .type = PHY_TYPE_USB3,
- .lanes = 2,
-
+static const struct qmp_phy_cfg sc8180x_usb3dpphy_cfg = {
.serdes_tbl = sm8150_usb3_serdes_tbl,
.serdes_tbl_num = ARRAY_SIZE(sm8150_usb3_serdes_tbl),
.tx_tbl = sm8150_usb3_tx_tbl,
.pcs_tbl_num = ARRAY_SIZE(sm8150_usb3_pcs_tbl),
.pcs_usb_tbl = sm8150_usb3_pcs_usb_tbl,
.pcs_usb_tbl_num = ARRAY_SIZE(sm8150_usb3_pcs_usb_tbl),
- .clk_list = qmp_v4_phy_clk_l,
- .num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l),
- .reset_list = msm8996_usb3phy_reset_l,
- .num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
- .vreg_list = qmp_phy_vreg_l,
- .num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
- .regs = qmp_v4_usb3phy_regs_layout,
- .pcs_usb_offset = 0x300,
-
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
-};
-
-static const struct qmp_phy_cfg sc8180x_dpphy_cfg = {
- .type = PHY_TYPE_DP,
- .lanes = 2,
-
- .serdes_tbl = qmp_v4_dp_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(qmp_v4_dp_serdes_tbl),
- .tx_tbl = qmp_v4_dp_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(qmp_v4_dp_tx_tbl),
+ .dp_serdes_tbl = qmp_v4_dp_serdes_tbl,
+ .dp_serdes_tbl_num = ARRAY_SIZE(qmp_v4_dp_serdes_tbl),
+ .dp_tx_tbl = qmp_v4_dp_tx_tbl,
+ .dp_tx_tbl_num = ARRAY_SIZE(qmp_v4_dp_tx_tbl),
.serdes_tbl_rbr = qmp_v4_dp_serdes_tbl_rbr,
.serdes_tbl_rbr_num = ARRAY_SIZE(qmp_v4_dp_serdes_tbl_rbr),
.swing_hbr3_hbr2 = &qmp_dp_v3_voltage_swing_hbr3_hbr2,
.pre_emphasis_hbr3_hbr2 = &qmp_dp_v3_pre_emphasis_hbr3_hbr2,
- .clk_list = qmp_v3_phy_clk_l,
- .num_clks = ARRAY_SIZE(qmp_v3_phy_clk_l),
- .reset_list = sc7180_usb3phy_reset_l,
- .num_resets = ARRAY_SIZE(sc7180_usb3phy_reset_l),
+ .dp_aux_init = qmp_v4_dp_aux_init,
+ .configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_phy = qmp_v4_configure_dp_phy,
+ .calibrate_dp_phy = qmp_v4_calibrate_dp_phy,
+
+ .clk_list = qmp_v4_phy_clk_l,
+ .num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l),
+ .reset_list = msm8996_usb3phy_reset_l,
+ .num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
- .regs = qmp_v3_usb3phy_regs_layout,
-
- .dp_aux_init = qcom_qmp_v4_phy_dp_aux_init,
- .configure_dp_tx = qcom_qmp_v4_phy_configure_dp_tx,
- .configure_dp_phy = qcom_qmp_v4_phy_configure_dp_phy,
- .calibrate_dp_phy = qcom_qmp_v4_dp_phy_calibrate,
-};
+ .regs = qmp_v4_usb3phy_regs_layout,
+ .pcs_usb_offset = 0x300,
-static const struct qmp_phy_combo_cfg sc8180x_usb3dpphy_cfg = {
- .usb_cfg = &sm8150_usb3phy_cfg,
- .dp_cfg = &sc8180x_dpphy_cfg,
+ .has_pwrdn_delay = true,
};
-static const struct qmp_phy_cfg sc8280xp_usb43dp_usb_cfg = {
- .type = PHY_TYPE_USB3,
- .lanes = 2,
+static const struct qmp_phy_cfg sc8280xp_usb43dpphy_cfg = {
+ .offsets = &qmp_combo_offsets_v5,
.serdes_tbl = sc8280xp_usb43dp_serdes_tbl,
.serdes_tbl_num = ARRAY_SIZE(sc8280xp_usb43dp_serdes_tbl),
.rx_tbl_num = ARRAY_SIZE(sc8280xp_usb43dp_rx_tbl),
.pcs_tbl = sc8280xp_usb43dp_pcs_tbl,
.pcs_tbl_num = ARRAY_SIZE(sc8280xp_usb43dp_pcs_tbl),
- .clk_list = qmp_v4_phy_clk_l,
- .num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l),
- .reset_list = msm8996_usb3phy_reset_l,
- .num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
- .vreg_list = qmp_phy_vreg_l,
- .num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
- .regs = qmp_v4_usb3phy_regs_layout,
- .pcs_usb_offset = 0x300,
-
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
-};
-
-static const struct qmp_phy_cfg sc8280xp_usb43dp_dp_cfg = {
- .type = PHY_TYPE_DP,
- .lanes = 2,
- .serdes_tbl = qmp_v5_dp_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(qmp_v5_dp_serdes_tbl),
- .tx_tbl = qmp_v5_5nm_dp_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(qmp_v5_5nm_dp_tx_tbl),
+ .dp_serdes_tbl = qmp_v5_dp_serdes_tbl,
+ .dp_serdes_tbl_num = ARRAY_SIZE(qmp_v5_dp_serdes_tbl),
+ .dp_tx_tbl = qmp_v5_5nm_dp_tx_tbl,
+ .dp_tx_tbl_num = ARRAY_SIZE(qmp_v5_5nm_dp_tx_tbl),
.serdes_tbl_rbr = qmp_v4_dp_serdes_tbl_rbr,
.serdes_tbl_rbr_num = ARRAY_SIZE(qmp_v4_dp_serdes_tbl_rbr),
.swing_hbr3_hbr2 = &qmp_dp_v5_voltage_swing_hbr3_hbr2,
.pre_emphasis_hbr3_hbr2 = &qmp_dp_v5_pre_emphasis_hbr3_hbr2,
+ .dp_aux_init = qmp_v4_dp_aux_init,
+ .configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_phy = qmp_v5_configure_dp_phy,
+ .calibrate_dp_phy = qmp_v4_calibrate_dp_phy,
+
.clk_list = qmp_v4_phy_clk_l,
.num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l),
.reset_list = msm8996_usb3phy_reset_l,
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = qmp_v4_usb3phy_regs_layout,
-
- .dp_aux_init = qcom_qmp_v4_phy_dp_aux_init,
- .configure_dp_tx = qcom_qmp_v4_phy_configure_dp_tx,
- .configure_dp_phy = qcom_qmp_v5_phy_configure_dp_phy,
- .calibrate_dp_phy = qcom_qmp_v4_dp_phy_calibrate,
-};
-
-static const struct qmp_phy_combo_cfg sc8280xp_usb43dpphy_combo_cfg = {
- .usb_cfg = &sc8280xp_usb43dp_usb_cfg,
- .dp_cfg = &sc8280xp_usb43dp_dp_cfg,
};
-static const struct qmp_phy_cfg sm8250_usb3phy_cfg = {
- .type = PHY_TYPE_USB3,
- .lanes = 2,
-
+static const struct qmp_phy_cfg sm8250_usb3dpphy_cfg = {
.serdes_tbl = sm8150_usb3_serdes_tbl,
.serdes_tbl_num = ARRAY_SIZE(sm8150_usb3_serdes_tbl),
.tx_tbl = sm8250_usb3_tx_tbl,
.pcs_tbl_num = ARRAY_SIZE(sm8250_usb3_pcs_tbl),
.pcs_usb_tbl = sm8250_usb3_pcs_usb_tbl,
.pcs_usb_tbl_num = ARRAY_SIZE(sm8250_usb3_pcs_usb_tbl),
- .clk_list = qmp_v4_sm8250_usbphy_clk_l,
- .num_clks = ARRAY_SIZE(qmp_v4_sm8250_usbphy_clk_l),
- .reset_list = msm8996_usb3phy_reset_l,
- .num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
- .vreg_list = qmp_phy_vreg_l,
- .num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
- .regs = qmp_v4_usb3phy_regs_layout,
- .pcs_usb_offset = 0x300,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
-};
-
-static const struct qmp_phy_cfg sm8250_dpphy_cfg = {
- .type = PHY_TYPE_DP,
- .lanes = 2,
-
- .serdes_tbl = qmp_v4_dp_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(qmp_v4_dp_serdes_tbl),
- .tx_tbl = qmp_v4_dp_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(qmp_v4_dp_tx_tbl),
+ .dp_serdes_tbl = qmp_v4_dp_serdes_tbl,
+ .dp_serdes_tbl_num = ARRAY_SIZE(qmp_v4_dp_serdes_tbl),
+ .dp_tx_tbl = qmp_v4_dp_tx_tbl,
+ .dp_tx_tbl_num = ARRAY_SIZE(qmp_v4_dp_tx_tbl),
.serdes_tbl_rbr = qmp_v4_dp_serdes_tbl_rbr,
.serdes_tbl_rbr_num = ARRAY_SIZE(qmp_v4_dp_serdes_tbl_rbr),
.swing_hbr3_hbr2 = &qmp_dp_v3_voltage_swing_hbr3_hbr2,
.pre_emphasis_hbr3_hbr2 = &qmp_dp_v3_pre_emphasis_hbr3_hbr2,
- .clk_list = qmp_v4_phy_clk_l,
- .num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l),
+ .dp_aux_init = qmp_v4_dp_aux_init,
+ .configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_phy = qmp_v4_configure_dp_phy,
+ .calibrate_dp_phy = qmp_v4_calibrate_dp_phy,
+
+ .clk_list = qmp_v4_sm8250_usbphy_clk_l,
+ .num_clks = ARRAY_SIZE(qmp_v4_sm8250_usbphy_clk_l),
.reset_list = msm8996_usb3phy_reset_l,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = qmp_v4_usb3phy_regs_layout,
+ .pcs_usb_offset = 0x300,
- .dp_aux_init = qcom_qmp_v4_phy_dp_aux_init,
- .configure_dp_tx = qcom_qmp_v4_phy_configure_dp_tx,
- .configure_dp_phy = qcom_qmp_v4_phy_configure_dp_phy,
- .calibrate_dp_phy = qcom_qmp_v4_dp_phy_calibrate,
-};
-
-static const struct qmp_phy_combo_cfg sm8250_usb3dpphy_cfg = {
- .usb_cfg = &sm8250_usb3phy_cfg,
- .dp_cfg = &sm8250_dpphy_cfg,
+ .has_pwrdn_delay = true,
};
static void qmp_combo_configure_lane(void __iomem *base,
- const unsigned int *regs,
const struct qmp_phy_init_tbl tbl[],
int num,
u8 lane_mask)
if (!(t->lane_mask & lane_mask))
continue;
- if (t->in_layout)
- writel(t->val, base + regs[t->offset]);
- else
- writel(t->val, base + t->offset);
+ writel(t->val, base + t->offset);
}
}
static void qmp_combo_configure(void __iomem *base,
- const unsigned int *regs,
const struct qmp_phy_init_tbl tbl[],
int num)
{
- qmp_combo_configure_lane(base, regs, tbl, num, 0xff);
+ qmp_combo_configure_lane(base, tbl, num, 0xff);
}
-static int qmp_combo_serdes_init(struct qmp_phy *qphy)
+static int qmp_combo_dp_serdes_init(struct qmp_combo *qmp)
{
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *serdes = qphy->serdes;
- const struct phy_configure_opts_dp *dp_opts = &qphy->dp_opts;
- const struct qmp_phy_init_tbl *serdes_tbl = cfg->serdes_tbl;
- int serdes_tbl_num = cfg->serdes_tbl_num;
-
- qmp_combo_configure(serdes, cfg->regs, serdes_tbl, serdes_tbl_num);
-
- if (cfg->type == PHY_TYPE_DP) {
- switch (dp_opts->link_rate) {
- case 1620:
- qmp_combo_configure(serdes, cfg->regs,
- cfg->serdes_tbl_rbr,
- cfg->serdes_tbl_rbr_num);
- break;
- case 2700:
- qmp_combo_configure(serdes, cfg->regs,
- cfg->serdes_tbl_hbr,
- cfg->serdes_tbl_hbr_num);
- break;
- case 5400:
- qmp_combo_configure(serdes, cfg->regs,
- cfg->serdes_tbl_hbr2,
- cfg->serdes_tbl_hbr2_num);
- break;
- case 8100:
- qmp_combo_configure(serdes, cfg->regs,
- cfg->serdes_tbl_hbr3,
- cfg->serdes_tbl_hbr3_num);
- break;
- default:
- /* Other link rates aren't supported */
- return -EINVAL;
- }
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *serdes = qmp->dp_serdes;
+ const struct phy_configure_opts_dp *dp_opts = &qmp->dp_opts;
+
+ qmp_combo_configure(serdes, cfg->dp_serdes_tbl, cfg->dp_serdes_tbl_num);
+
+ switch (dp_opts->link_rate) {
+ case 1620:
+ qmp_combo_configure(serdes, cfg->serdes_tbl_rbr,
+ cfg->serdes_tbl_rbr_num);
+ break;
+ case 2700:
+ qmp_combo_configure(serdes, cfg->serdes_tbl_hbr,
+ cfg->serdes_tbl_hbr_num);
+ break;
+ case 5400:
+ qmp_combo_configure(serdes, cfg->serdes_tbl_hbr2,
+ cfg->serdes_tbl_hbr2_num);
+ break;
+ case 8100:
+ qmp_combo_configure(serdes, cfg->serdes_tbl_hbr3,
+ cfg->serdes_tbl_hbr3_num);
+ break;
+ default:
+ /* Other link rates aren't supported */
+ return -EINVAL;
}
return 0;
}
-static void qcom_qmp_v3_phy_dp_aux_init(struct qmp_phy *qphy)
+static void qmp_v3_dp_aux_init(struct qmp_combo *qmp)
{
writel(DP_PHY_PD_CTL_PWRDN | DP_PHY_PD_CTL_AUX_PWRDN |
DP_PHY_PD_CTL_PLL_PWRDN | DP_PHY_PD_CTL_DP_CLAMP_EN,
- qphy->pcs + QSERDES_DP_PHY_PD_CTL);
+ qmp->dp_dp_phy + QSERDES_DP_PHY_PD_CTL);
/* Turn on BIAS current for PHY/PLL */
writel(QSERDES_V3_COM_BIAS_EN | QSERDES_V3_COM_BIAS_EN_MUX |
QSERDES_V3_COM_CLKBUF_L_EN | QSERDES_V3_COM_EN_SYSCLK_TX_SEL,
- qphy->serdes + QSERDES_V3_COM_BIAS_EN_CLKBUFLR_EN);
+ qmp->dp_serdes + QSERDES_V3_COM_BIAS_EN_CLKBUFLR_EN);
- writel(DP_PHY_PD_CTL_PSR_PWRDN, qphy->pcs + QSERDES_DP_PHY_PD_CTL);
+ writel(DP_PHY_PD_CTL_PSR_PWRDN, qmp->dp_dp_phy + QSERDES_DP_PHY_PD_CTL);
writel(DP_PHY_PD_CTL_PWRDN | DP_PHY_PD_CTL_AUX_PWRDN |
DP_PHY_PD_CTL_LANE_0_1_PWRDN |
DP_PHY_PD_CTL_LANE_2_3_PWRDN | DP_PHY_PD_CTL_PLL_PWRDN |
DP_PHY_PD_CTL_DP_CLAMP_EN,
- qphy->pcs + QSERDES_DP_PHY_PD_CTL);
+ qmp->dp_dp_phy + QSERDES_DP_PHY_PD_CTL);
writel(QSERDES_V3_COM_BIAS_EN |
QSERDES_V3_COM_BIAS_EN_MUX | QSERDES_V3_COM_CLKBUF_R_EN |
QSERDES_V3_COM_CLKBUF_L_EN | QSERDES_V3_COM_EN_SYSCLK_TX_SEL |
QSERDES_V3_COM_CLKBUF_RX_DRIVE_L,
- qphy->serdes + QSERDES_V3_COM_BIAS_EN_CLKBUFLR_EN);
-
- writel(0x00, qphy->pcs + QSERDES_DP_PHY_AUX_CFG0);
- writel(0x13, qphy->pcs + QSERDES_DP_PHY_AUX_CFG1);
- writel(0x24, qphy->pcs + QSERDES_DP_PHY_AUX_CFG2);
- writel(0x00, qphy->pcs + QSERDES_DP_PHY_AUX_CFG3);
- writel(0x0a, qphy->pcs + QSERDES_DP_PHY_AUX_CFG4);
- writel(0x26, qphy->pcs + QSERDES_DP_PHY_AUX_CFG5);
- writel(0x0a, qphy->pcs + QSERDES_DP_PHY_AUX_CFG6);
- writel(0x03, qphy->pcs + QSERDES_DP_PHY_AUX_CFG7);
- writel(0xbb, qphy->pcs + QSERDES_DP_PHY_AUX_CFG8);
- writel(0x03, qphy->pcs + QSERDES_DP_PHY_AUX_CFG9);
- qphy->dp_aux_cfg = 0;
+ qmp->dp_serdes + QSERDES_V3_COM_BIAS_EN_CLKBUFLR_EN);
+
+ writel(0x00, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG0);
+ writel(0x13, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG1);
+ writel(0x24, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG2);
+ writel(0x00, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG3);
+ writel(0x0a, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG4);
+ writel(0x26, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG5);
+ writel(0x0a, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG6);
+ writel(0x03, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG7);
+ writel(0xbb, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG8);
+ writel(0x03, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG9);
+ qmp->dp_aux_cfg = 0;
writel(PHY_AUX_STOP_ERR_MASK | PHY_AUX_DEC_ERR_MASK |
PHY_AUX_SYNC_ERR_MASK | PHY_AUX_ALIGN_ERR_MASK |
PHY_AUX_REQ_ERR_MASK,
- qphy->pcs + QSERDES_V3_DP_PHY_AUX_INTERRUPT_MASK);
+ qmp->dp_dp_phy + QSERDES_V3_DP_PHY_AUX_INTERRUPT_MASK);
}
-static int qmp_combo_configure_dp_swing(struct qmp_phy *qphy,
+static int qmp_combo_configure_dp_swing(struct qmp_combo *qmp,
unsigned int drv_lvl_reg, unsigned int emp_post_reg)
{
- const struct phy_configure_opts_dp *dp_opts = &qphy->dp_opts;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ const struct phy_configure_opts_dp *dp_opts = &qmp->dp_opts;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
unsigned int v_level = 0, p_level = 0;
u8 voltage_swing_cfg, pre_emphasis_cfg;
int i;
voltage_swing_cfg |= DP_PHY_TXn_TX_DRV_LVL_MUX_EN;
pre_emphasis_cfg |= DP_PHY_TXn_TX_EMP_POST1_LVL_MUX_EN;
- writel(voltage_swing_cfg, qphy->tx + drv_lvl_reg);
- writel(pre_emphasis_cfg, qphy->tx + emp_post_reg);
- writel(voltage_swing_cfg, qphy->tx2 + drv_lvl_reg);
- writel(pre_emphasis_cfg, qphy->tx2 + emp_post_reg);
+ writel(voltage_swing_cfg, qmp->dp_tx + drv_lvl_reg);
+ writel(pre_emphasis_cfg, qmp->dp_tx + emp_post_reg);
+ writel(voltage_swing_cfg, qmp->dp_tx2 + drv_lvl_reg);
+ writel(pre_emphasis_cfg, qmp->dp_tx2 + emp_post_reg);
return 0;
}
-static void qcom_qmp_v3_phy_configure_dp_tx(struct qmp_phy *qphy)
+static void qmp_v3_configure_dp_tx(struct qmp_combo *qmp)
{
- const struct phy_configure_opts_dp *dp_opts = &qphy->dp_opts;
+ const struct phy_configure_opts_dp *dp_opts = &qmp->dp_opts;
u32 bias_en, drvr_en;
- if (qmp_combo_configure_dp_swing(qphy, QSERDES_V3_TX_TX_DRV_LVL,
+ if (qmp_combo_configure_dp_swing(qmp, QSERDES_V3_TX_TX_DRV_LVL,
QSERDES_V3_TX_TX_EMP_POST1_LVL) < 0)
return;
drvr_en = 0x10;
}
- writel(drvr_en, qphy->tx + QSERDES_V3_TX_HIGHZ_DRVR_EN);
- writel(bias_en, qphy->tx + QSERDES_V3_TX_TRANSCEIVER_BIAS_EN);
- writel(drvr_en, qphy->tx2 + QSERDES_V3_TX_HIGHZ_DRVR_EN);
- writel(bias_en, qphy->tx2 + QSERDES_V3_TX_TRANSCEIVER_BIAS_EN);
+ writel(drvr_en, qmp->dp_tx + QSERDES_V3_TX_HIGHZ_DRVR_EN);
+ writel(bias_en, qmp->dp_tx + QSERDES_V3_TX_TRANSCEIVER_BIAS_EN);
+ writel(drvr_en, qmp->dp_tx2 + QSERDES_V3_TX_HIGHZ_DRVR_EN);
+ writel(bias_en, qmp->dp_tx2 + QSERDES_V3_TX_TRANSCEIVER_BIAS_EN);
}
-static bool qmp_combo_configure_dp_mode(struct qmp_phy *qphy)
+static bool qmp_combo_configure_dp_mode(struct qmp_combo *qmp)
{
u32 val;
bool reverse = false;
* if (lane_cnt == 4 || orientation == ORIENTATION_CC1)
* val |= DP_PHY_PD_CTL_LANE_2_3_PWRDN;
* if (orientation == ORIENTATION_CC2)
- * writel(0x4c, qphy->pcs + QSERDES_V3_DP_PHY_MODE);
+ * writel(0x4c, qmp->dp_dp_phy + QSERDES_V3_DP_PHY_MODE);
*/
val |= DP_PHY_PD_CTL_LANE_2_3_PWRDN;
- writel(val, qphy->pcs + QSERDES_DP_PHY_PD_CTL);
+ writel(val, qmp->dp_dp_phy + QSERDES_DP_PHY_PD_CTL);
- writel(0x5c, qphy->pcs + QSERDES_DP_PHY_MODE);
+ writel(0x5c, qmp->dp_dp_phy + QSERDES_DP_PHY_MODE);
return reverse;
}
-static int qcom_qmp_v3_phy_configure_dp_phy(struct qmp_phy *qphy)
+static int qmp_v3_configure_dp_phy(struct qmp_combo *qmp)
{
- const struct qmp_phy_dp_clks *dp_clks = qphy->dp_clks;
- const struct phy_configure_opts_dp *dp_opts = &qphy->dp_opts;
+ const struct phy_configure_opts_dp *dp_opts = &qmp->dp_opts;
u32 phy_vco_div, status;
unsigned long pixel_freq;
- qmp_combo_configure_dp_mode(qphy);
+ qmp_combo_configure_dp_mode(qmp);
- writel(0x05, qphy->pcs + QSERDES_V3_DP_PHY_TX0_TX1_LANE_CTL);
- writel(0x05, qphy->pcs + QSERDES_V3_DP_PHY_TX2_TX3_LANE_CTL);
+ writel(0x05, qmp->dp_dp_phy + QSERDES_V3_DP_PHY_TX0_TX1_LANE_CTL);
+ writel(0x05, qmp->dp_dp_phy + QSERDES_V3_DP_PHY_TX2_TX3_LANE_CTL);
switch (dp_opts->link_rate) {
case 1620:
/* Other link rates aren't supported */
return -EINVAL;
}
- writel(phy_vco_div, qphy->pcs + QSERDES_V3_DP_PHY_VCO_DIV);
+ writel(phy_vco_div, qmp->dp_dp_phy + QSERDES_V3_DP_PHY_VCO_DIV);
- clk_set_rate(dp_clks->dp_link_hw.clk, dp_opts->link_rate * 100000);
- clk_set_rate(dp_clks->dp_pixel_hw.clk, pixel_freq);
+ clk_set_rate(qmp->dp_link_hw.clk, dp_opts->link_rate * 100000);
+ clk_set_rate(qmp->dp_pixel_hw.clk, pixel_freq);
- writel(0x04, qphy->pcs + QSERDES_DP_PHY_AUX_CFG2);
- writel(0x01, qphy->pcs + QSERDES_DP_PHY_CFG);
- writel(0x05, qphy->pcs + QSERDES_DP_PHY_CFG);
- writel(0x01, qphy->pcs + QSERDES_DP_PHY_CFG);
- writel(0x09, qphy->pcs + QSERDES_DP_PHY_CFG);
+ writel(0x04, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG2);
+ writel(0x01, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
+ writel(0x05, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
+ writel(0x01, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
+ writel(0x09, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
- writel(0x20, qphy->serdes + QSERDES_V3_COM_RESETSM_CNTRL);
+ writel(0x20, qmp->dp_serdes + QSERDES_V3_COM_RESETSM_CNTRL);
- if (readl_poll_timeout(qphy->serdes + QSERDES_V3_COM_C_READY_STATUS,
+ if (readl_poll_timeout(qmp->dp_serdes + QSERDES_V3_COM_C_READY_STATUS,
status,
((status & BIT(0)) > 0),
500,
10000))
return -ETIMEDOUT;
- writel(0x19, qphy->pcs + QSERDES_DP_PHY_CFG);
+ writel(0x19, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
- if (readl_poll_timeout(qphy->pcs + QSERDES_V3_DP_PHY_STATUS,
+ if (readl_poll_timeout(qmp->dp_dp_phy + QSERDES_V3_DP_PHY_STATUS,
status,
((status & BIT(1)) > 0),
500,
10000))
return -ETIMEDOUT;
- writel(0x18, qphy->pcs + QSERDES_DP_PHY_CFG);
+ writel(0x18, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
udelay(2000);
- writel(0x19, qphy->pcs + QSERDES_DP_PHY_CFG);
+ writel(0x19, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
- return readl_poll_timeout(qphy->pcs + QSERDES_V3_DP_PHY_STATUS,
+ return readl_poll_timeout(qmp->dp_dp_phy + QSERDES_V3_DP_PHY_STATUS,
status,
((status & BIT(1)) > 0),
500,
* We need to calibrate the aux setting here as many times
* as the caller tries
*/
-static int qcom_qmp_v3_dp_phy_calibrate(struct qmp_phy *qphy)
+static int qmp_v3_calibrate_dp_phy(struct qmp_combo *qmp)
{
static const u8 cfg1_settings[] = { 0x13, 0x23, 0x1d };
u8 val;
- qphy->dp_aux_cfg++;
- qphy->dp_aux_cfg %= ARRAY_SIZE(cfg1_settings);
- val = cfg1_settings[qphy->dp_aux_cfg];
+ qmp->dp_aux_cfg++;
+ qmp->dp_aux_cfg %= ARRAY_SIZE(cfg1_settings);
+ val = cfg1_settings[qmp->dp_aux_cfg];
- writel(val, qphy->pcs + QSERDES_DP_PHY_AUX_CFG1);
+ writel(val, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG1);
return 0;
}
-static void qcom_qmp_v4_phy_dp_aux_init(struct qmp_phy *qphy)
+static void qmp_v4_dp_aux_init(struct qmp_combo *qmp)
{
writel(DP_PHY_PD_CTL_PWRDN | DP_PHY_PD_CTL_PSR_PWRDN | DP_PHY_PD_CTL_AUX_PWRDN |
DP_PHY_PD_CTL_PLL_PWRDN | DP_PHY_PD_CTL_DP_CLAMP_EN,
- qphy->pcs + QSERDES_DP_PHY_PD_CTL);
+ qmp->dp_dp_phy + QSERDES_DP_PHY_PD_CTL);
/* Turn on BIAS current for PHY/PLL */
- writel(0x17, qphy->serdes + QSERDES_V4_COM_BIAS_EN_CLKBUFLR_EN);
-
- writel(0x00, qphy->pcs + QSERDES_DP_PHY_AUX_CFG0);
- writel(0x13, qphy->pcs + QSERDES_DP_PHY_AUX_CFG1);
- writel(0xa4, qphy->pcs + QSERDES_DP_PHY_AUX_CFG2);
- writel(0x00, qphy->pcs + QSERDES_DP_PHY_AUX_CFG3);
- writel(0x0a, qphy->pcs + QSERDES_DP_PHY_AUX_CFG4);
- writel(0x26, qphy->pcs + QSERDES_DP_PHY_AUX_CFG5);
- writel(0x0a, qphy->pcs + QSERDES_DP_PHY_AUX_CFG6);
- writel(0x03, qphy->pcs + QSERDES_DP_PHY_AUX_CFG7);
- writel(0xb7, qphy->pcs + QSERDES_DP_PHY_AUX_CFG8);
- writel(0x03, qphy->pcs + QSERDES_DP_PHY_AUX_CFG9);
- qphy->dp_aux_cfg = 0;
+ writel(0x17, qmp->dp_serdes + QSERDES_V4_COM_BIAS_EN_CLKBUFLR_EN);
+
+ writel(0x00, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG0);
+ writel(0x13, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG1);
+ writel(0xa4, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG2);
+ writel(0x00, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG3);
+ writel(0x0a, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG4);
+ writel(0x26, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG5);
+ writel(0x0a, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG6);
+ writel(0x03, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG7);
+ writel(0xb7, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG8);
+ writel(0x03, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG9);
+ qmp->dp_aux_cfg = 0;
writel(PHY_AUX_STOP_ERR_MASK | PHY_AUX_DEC_ERR_MASK |
PHY_AUX_SYNC_ERR_MASK | PHY_AUX_ALIGN_ERR_MASK |
PHY_AUX_REQ_ERR_MASK,
- qphy->pcs + QSERDES_V4_DP_PHY_AUX_INTERRUPT_MASK);
+ qmp->dp_dp_phy + QSERDES_V4_DP_PHY_AUX_INTERRUPT_MASK);
}
-static void qcom_qmp_v4_phy_configure_dp_tx(struct qmp_phy *qphy)
+static void qmp_v4_configure_dp_tx(struct qmp_combo *qmp)
{
/* Program default values before writing proper values */
- writel(0x27, qphy->tx + QSERDES_V4_TX_TX_DRV_LVL);
- writel(0x27, qphy->tx2 + QSERDES_V4_TX_TX_DRV_LVL);
+ writel(0x27, qmp->dp_tx + QSERDES_V4_TX_TX_DRV_LVL);
+ writel(0x27, qmp->dp_tx2 + QSERDES_V4_TX_TX_DRV_LVL);
- writel(0x20, qphy->tx + QSERDES_V4_TX_TX_EMP_POST1_LVL);
- writel(0x20, qphy->tx2 + QSERDES_V4_TX_TX_EMP_POST1_LVL);
+ writel(0x20, qmp->dp_tx + QSERDES_V4_TX_TX_EMP_POST1_LVL);
+ writel(0x20, qmp->dp_tx2 + QSERDES_V4_TX_TX_EMP_POST1_LVL);
- qmp_combo_configure_dp_swing(qphy, QSERDES_V4_TX_TX_DRV_LVL,
+ qmp_combo_configure_dp_swing(qmp, QSERDES_V4_TX_TX_DRV_LVL,
QSERDES_V4_TX_TX_EMP_POST1_LVL);
}
-static int qcom_qmp_v45_phy_configure_dp_phy(struct qmp_phy *qphy)
+static int qmp_v45_configure_dp_phy(struct qmp_combo *qmp)
{
- const struct qmp_phy_dp_clks *dp_clks = qphy->dp_clks;
- const struct phy_configure_opts_dp *dp_opts = &qphy->dp_opts;
+ const struct phy_configure_opts_dp *dp_opts = &qmp->dp_opts;
u32 phy_vco_div, status;
unsigned long pixel_freq;
- writel(0x0f, qphy->pcs + QSERDES_V4_DP_PHY_CFG_1);
+ writel(0x0f, qmp->dp_dp_phy + QSERDES_V4_DP_PHY_CFG_1);
- qmp_combo_configure_dp_mode(qphy);
+ qmp_combo_configure_dp_mode(qmp);
- writel(0x13, qphy->pcs + QSERDES_DP_PHY_AUX_CFG1);
- writel(0xa4, qphy->pcs + QSERDES_DP_PHY_AUX_CFG2);
+ writel(0x13, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG1);
+ writel(0xa4, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG2);
- writel(0x05, qphy->pcs + QSERDES_V4_DP_PHY_TX0_TX1_LANE_CTL);
- writel(0x05, qphy->pcs + QSERDES_V4_DP_PHY_TX2_TX3_LANE_CTL);
+ writel(0x05, qmp->dp_dp_phy + QSERDES_V4_DP_PHY_TX0_TX1_LANE_CTL);
+ writel(0x05, qmp->dp_dp_phy + QSERDES_V4_DP_PHY_TX2_TX3_LANE_CTL);
switch (dp_opts->link_rate) {
case 1620:
/* Other link rates aren't supported */
return -EINVAL;
}
- writel(phy_vco_div, qphy->pcs + QSERDES_V4_DP_PHY_VCO_DIV);
+ writel(phy_vco_div, qmp->dp_dp_phy + QSERDES_V4_DP_PHY_VCO_DIV);
- clk_set_rate(dp_clks->dp_link_hw.clk, dp_opts->link_rate * 100000);
- clk_set_rate(dp_clks->dp_pixel_hw.clk, pixel_freq);
+ clk_set_rate(qmp->dp_link_hw.clk, dp_opts->link_rate * 100000);
+ clk_set_rate(qmp->dp_pixel_hw.clk, pixel_freq);
- writel(0x01, qphy->pcs + QSERDES_DP_PHY_CFG);
- writel(0x05, qphy->pcs + QSERDES_DP_PHY_CFG);
- writel(0x01, qphy->pcs + QSERDES_DP_PHY_CFG);
- writel(0x09, qphy->pcs + QSERDES_DP_PHY_CFG);
+ writel(0x01, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
+ writel(0x05, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
+ writel(0x01, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
+ writel(0x09, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
- writel(0x20, qphy->serdes + QSERDES_V4_COM_RESETSM_CNTRL);
+ writel(0x20, qmp->dp_serdes + QSERDES_V4_COM_RESETSM_CNTRL);
- if (readl_poll_timeout(qphy->serdes + QSERDES_V4_COM_C_READY_STATUS,
+ if (readl_poll_timeout(qmp->dp_serdes + QSERDES_V4_COM_C_READY_STATUS,
status,
((status & BIT(0)) > 0),
500,
10000))
return -ETIMEDOUT;
- if (readl_poll_timeout(qphy->serdes + QSERDES_V4_COM_CMN_STATUS,
+ if (readl_poll_timeout(qmp->dp_serdes + QSERDES_V4_COM_CMN_STATUS,
status,
((status & BIT(0)) > 0),
500,
10000))
return -ETIMEDOUT;
- if (readl_poll_timeout(qphy->serdes + QSERDES_V4_COM_CMN_STATUS,
+ if (readl_poll_timeout(qmp->dp_serdes + QSERDES_V4_COM_CMN_STATUS,
status,
((status & BIT(1)) > 0),
500,
10000))
return -ETIMEDOUT;
- writel(0x19, qphy->pcs + QSERDES_DP_PHY_CFG);
+ writel(0x19, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
- if (readl_poll_timeout(qphy->pcs + QSERDES_V4_DP_PHY_STATUS,
+ if (readl_poll_timeout(qmp->dp_dp_phy + QSERDES_V4_DP_PHY_STATUS,
status,
((status & BIT(0)) > 0),
500,
10000))
return -ETIMEDOUT;
- if (readl_poll_timeout(qphy->pcs + QSERDES_V4_DP_PHY_STATUS,
+ if (readl_poll_timeout(qmp->dp_dp_phy + QSERDES_V4_DP_PHY_STATUS,
status,
((status & BIT(1)) > 0),
500,
return 0;
}
-static int qcom_qmp_v4_phy_configure_dp_phy(struct qmp_phy *qphy)
+static int qmp_v4_configure_dp_phy(struct qmp_combo *qmp)
{
- const struct phy_configure_opts_dp *dp_opts = &qphy->dp_opts;
+ const struct phy_configure_opts_dp *dp_opts = &qmp->dp_opts;
u32 bias0_en, drvr0_en, bias1_en, drvr1_en;
bool reverse = false;
u32 status;
int ret;
- ret = qcom_qmp_v45_phy_configure_dp_phy(qphy);
+ ret = qmp_v45_configure_dp_phy(qmp);
if (ret < 0)
return ret;
drvr1_en = 0x10;
}
- writel(drvr0_en, qphy->tx + QSERDES_V4_TX_HIGHZ_DRVR_EN);
- writel(bias0_en, qphy->tx + QSERDES_V4_TX_TRANSCEIVER_BIAS_EN);
- writel(drvr1_en, qphy->tx2 + QSERDES_V4_TX_HIGHZ_DRVR_EN);
- writel(bias1_en, qphy->tx2 + QSERDES_V4_TX_TRANSCEIVER_BIAS_EN);
+ writel(drvr0_en, qmp->dp_tx + QSERDES_V4_TX_HIGHZ_DRVR_EN);
+ writel(bias0_en, qmp->dp_tx + QSERDES_V4_TX_TRANSCEIVER_BIAS_EN);
+ writel(drvr1_en, qmp->dp_tx2 + QSERDES_V4_TX_HIGHZ_DRVR_EN);
+ writel(bias1_en, qmp->dp_tx2 + QSERDES_V4_TX_TRANSCEIVER_BIAS_EN);
- writel(0x18, qphy->pcs + QSERDES_DP_PHY_CFG);
+ writel(0x18, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
udelay(2000);
- writel(0x19, qphy->pcs + QSERDES_DP_PHY_CFG);
+ writel(0x19, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
- if (readl_poll_timeout(qphy->pcs + QSERDES_V4_DP_PHY_STATUS,
+ if (readl_poll_timeout(qmp->dp_dp_phy + QSERDES_V4_DP_PHY_STATUS,
status,
((status & BIT(1)) > 0),
500,
10000))
return -ETIMEDOUT;
- writel(0x0a, qphy->tx + QSERDES_V4_TX_TX_POL_INV);
- writel(0x0a, qphy->tx2 + QSERDES_V4_TX_TX_POL_INV);
+ writel(0x0a, qmp->dp_tx + QSERDES_V4_TX_TX_POL_INV);
+ writel(0x0a, qmp->dp_tx2 + QSERDES_V4_TX_TX_POL_INV);
- writel(0x27, qphy->tx + QSERDES_V4_TX_TX_DRV_LVL);
- writel(0x27, qphy->tx2 + QSERDES_V4_TX_TX_DRV_LVL);
+ writel(0x27, qmp->dp_tx + QSERDES_V4_TX_TX_DRV_LVL);
+ writel(0x27, qmp->dp_tx2 + QSERDES_V4_TX_TX_DRV_LVL);
- writel(0x20, qphy->tx + QSERDES_V4_TX_TX_EMP_POST1_LVL);
- writel(0x20, qphy->tx2 + QSERDES_V4_TX_TX_EMP_POST1_LVL);
+ writel(0x20, qmp->dp_tx + QSERDES_V4_TX_TX_EMP_POST1_LVL);
+ writel(0x20, qmp->dp_tx2 + QSERDES_V4_TX_TX_EMP_POST1_LVL);
return 0;
}
-static int qcom_qmp_v5_phy_configure_dp_phy(struct qmp_phy *qphy)
+static int qmp_v5_configure_dp_phy(struct qmp_combo *qmp)
{
- const struct phy_configure_opts_dp *dp_opts = &qphy->dp_opts;
+ const struct phy_configure_opts_dp *dp_opts = &qmp->dp_opts;
u32 bias0_en, drvr0_en, bias1_en, drvr1_en;
bool reverse = false;
u32 status;
int ret;
- ret = qcom_qmp_v45_phy_configure_dp_phy(qphy);
+ ret = qmp_v45_configure_dp_phy(qmp);
if (ret < 0)
return ret;
drvr1_en = 0x10;
}
- writel(drvr0_en, qphy->tx + QSERDES_V5_5NM_TX_HIGHZ_DRVR_EN);
- writel(bias0_en, qphy->tx + QSERDES_V5_5NM_TX_TRANSCEIVER_BIAS_EN);
- writel(drvr1_en, qphy->tx2 + QSERDES_V5_5NM_TX_HIGHZ_DRVR_EN);
- writel(bias1_en, qphy->tx2 + QSERDES_V5_5NM_TX_TRANSCEIVER_BIAS_EN);
+ writel(drvr0_en, qmp->dp_tx + QSERDES_V5_5NM_TX_HIGHZ_DRVR_EN);
+ writel(bias0_en, qmp->dp_tx + QSERDES_V5_5NM_TX_TRANSCEIVER_BIAS_EN);
+ writel(drvr1_en, qmp->dp_tx2 + QSERDES_V5_5NM_TX_HIGHZ_DRVR_EN);
+ writel(bias1_en, qmp->dp_tx2 + QSERDES_V5_5NM_TX_TRANSCEIVER_BIAS_EN);
- writel(0x18, qphy->pcs + QSERDES_DP_PHY_CFG);
+ writel(0x18, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
udelay(2000);
- writel(0x19, qphy->pcs + QSERDES_DP_PHY_CFG);
+ writel(0x19, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
- if (readl_poll_timeout(qphy->pcs + QSERDES_V4_DP_PHY_STATUS,
+ if (readl_poll_timeout(qmp->dp_dp_phy + QSERDES_V4_DP_PHY_STATUS,
status,
((status & BIT(1)) > 0),
500,
10000))
return -ETIMEDOUT;
- writel(0x0a, qphy->tx + QSERDES_V5_5NM_TX_TX_POL_INV);
- writel(0x0a, qphy->tx2 + QSERDES_V5_5NM_TX_TX_POL_INV);
+ writel(0x0a, qmp->dp_tx + QSERDES_V5_5NM_TX_TX_POL_INV);
+ writel(0x0a, qmp->dp_tx2 + QSERDES_V5_5NM_TX_TX_POL_INV);
- writel(0x27, qphy->tx + QSERDES_V5_5NM_TX_TX_DRV_LVL);
- writel(0x27, qphy->tx2 + QSERDES_V5_5NM_TX_TX_DRV_LVL);
+ writel(0x27, qmp->dp_tx + QSERDES_V5_5NM_TX_TX_DRV_LVL);
+ writel(0x27, qmp->dp_tx2 + QSERDES_V5_5NM_TX_TX_DRV_LVL);
- writel(0x20, qphy->tx + QSERDES_V5_5NM_TX_TX_EMP_POST1_LVL);
- writel(0x20, qphy->tx2 + QSERDES_V5_5NM_TX_TX_EMP_POST1_LVL);
+ writel(0x20, qmp->dp_tx + QSERDES_V5_5NM_TX_TX_EMP_POST1_LVL);
+ writel(0x20, qmp->dp_tx2 + QSERDES_V5_5NM_TX_TX_EMP_POST1_LVL);
return 0;
}
* We need to calibrate the aux setting here as many times
* as the caller tries
*/
-static int qcom_qmp_v4_dp_phy_calibrate(struct qmp_phy *qphy)
+static int qmp_v4_calibrate_dp_phy(struct qmp_combo *qmp)
{
static const u8 cfg1_settings[] = { 0x20, 0x13, 0x23, 0x1d };
u8 val;
- qphy->dp_aux_cfg++;
- qphy->dp_aux_cfg %= ARRAY_SIZE(cfg1_settings);
- val = cfg1_settings[qphy->dp_aux_cfg];
+ qmp->dp_aux_cfg++;
+ qmp->dp_aux_cfg %= ARRAY_SIZE(cfg1_settings);
+ val = cfg1_settings[qmp->dp_aux_cfg];
- writel(val, qphy->pcs + QSERDES_DP_PHY_AUX_CFG1);
+ writel(val, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG1);
return 0;
}
-static int qcom_qmp_dp_phy_configure(struct phy *phy, union phy_configure_opts *opts)
+static int qmp_combo_dp_configure(struct phy *phy, union phy_configure_opts *opts)
{
const struct phy_configure_opts_dp *dp_opts = &opts->dp;
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
- memcpy(&qphy->dp_opts, dp_opts, sizeof(*dp_opts));
- if (qphy->dp_opts.set_voltages) {
- cfg->configure_dp_tx(qphy);
- qphy->dp_opts.set_voltages = 0;
+ memcpy(&qmp->dp_opts, dp_opts, sizeof(*dp_opts));
+ if (qmp->dp_opts.set_voltages) {
+ cfg->configure_dp_tx(qmp);
+ qmp->dp_opts.set_voltages = 0;
}
return 0;
}
-static int qcom_qmp_dp_phy_calibrate(struct phy *phy)
+static int qmp_combo_dp_calibrate(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
if (cfg->calibrate_dp_phy)
- return cfg->calibrate_dp_phy(qphy);
+ return cfg->calibrate_dp_phy(qmp);
return 0;
}
-static int qmp_combo_com_init(struct qmp_phy *qphy)
+static int qmp_combo_com_init(struct qmp_combo *qmp)
{
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *pcs = qphy->pcs;
- void __iomem *dp_com = qmp->dp_com;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *com = qmp->com;
int ret;
mutex_lock(&qmp->phy_mutex);
return 0;
}
- /* turn on regulator supplies */
ret = regulator_bulk_enable(cfg->num_vregs, qmp->vregs);
if (ret) {
dev_err(qmp->dev, "failed to enable regulators, err=%d\n", ret);
if (ret)
goto err_assert_reset;
- qphy_setbits(dp_com, QPHY_V3_DP_COM_POWER_DOWN_CTRL, SW_PWRDN);
+ qphy_setbits(com, QPHY_V3_DP_COM_POWER_DOWN_CTRL, SW_PWRDN);
/* override hardware control for reset of qmp phy */
- qphy_setbits(dp_com, QPHY_V3_DP_COM_RESET_OVRD_CTRL,
+ qphy_setbits(com, QPHY_V3_DP_COM_RESET_OVRD_CTRL,
SW_DPPHY_RESET_MUX | SW_DPPHY_RESET |
SW_USB3PHY_RESET_MUX | SW_USB3PHY_RESET);
/* Default type-c orientation, i.e CC1 */
- qphy_setbits(dp_com, QPHY_V3_DP_COM_TYPEC_CTRL, 0x02);
+ qphy_setbits(com, QPHY_V3_DP_COM_TYPEC_CTRL, 0x02);
- qphy_setbits(dp_com, QPHY_V3_DP_COM_PHY_MODE_CTRL, USB3_MODE | DP_MODE);
+ qphy_setbits(com, QPHY_V3_DP_COM_PHY_MODE_CTRL, USB3_MODE | DP_MODE);
/* bring both QMP USB and QMP DP PHYs PCS block out of reset */
- qphy_clrbits(dp_com, QPHY_V3_DP_COM_RESET_OVRD_CTRL,
+ qphy_clrbits(com, QPHY_V3_DP_COM_RESET_OVRD_CTRL,
SW_DPPHY_RESET_MUX | SW_DPPHY_RESET |
SW_USB3PHY_RESET_MUX | SW_USB3PHY_RESET);
- qphy_clrbits(dp_com, QPHY_V3_DP_COM_SWI_CTRL, 0x03);
- qphy_clrbits(dp_com, QPHY_V3_DP_COM_SW_RESET, SW_RESET);
+ qphy_clrbits(com, QPHY_V3_DP_COM_SWI_CTRL, 0x03);
+ qphy_clrbits(com, QPHY_V3_DP_COM_SW_RESET, SW_RESET);
- if (cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL])
- qphy_setbits(pcs,
- cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
- cfg->pwrdn_ctrl);
- else
- qphy_setbits(pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
- cfg->pwrdn_ctrl);
+ qphy_setbits(qmp->pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
+ SW_PWRDN);
mutex_unlock(&qmp->phy_mutex);
return ret;
}
-static int qmp_combo_com_exit(struct qmp_phy *qphy)
+static int qmp_combo_com_exit(struct qmp_combo *qmp)
{
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
mutex_lock(&qmp->phy_mutex);
if (--qmp->init_count) {
return 0;
}
- reset_control_assert(qmp->ufs_reset);
-
reset_control_bulk_assert(cfg->num_resets, qmp->resets);
clk_bulk_disable_unprepare(cfg->num_clks, qmp->clks);
return 0;
}
-static int qmp_combo_init(struct phy *phy)
+static int qmp_combo_dp_init(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
int ret;
- dev_vdbg(qmp->dev, "Initializing QMP phy\n");
- ret = qmp_combo_com_init(qphy);
+ ret = qmp_combo_com_init(qmp);
if (ret)
return ret;
- if (cfg->type == PHY_TYPE_DP)
- cfg->dp_aux_init(qphy);
+ cfg->dp_aux_init(qmp);
+
+ return 0;
+}
+
+static int qmp_combo_dp_exit(struct phy *phy)
+{
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
+
+ qmp_combo_com_exit(qmp);
return 0;
}
-static int qmp_combo_power_on(struct phy *phy)
+static int qmp_combo_dp_power_on(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *tx = qphy->tx;
- void __iomem *rx = qphy->rx;
- void __iomem *pcs = qphy->pcs;
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *tx = qmp->dp_tx;
+ void __iomem *tx2 = qmp->dp_tx2;
+
+ qmp_combo_dp_serdes_init(qmp);
+
+ qmp_combo_configure_lane(tx, cfg->dp_tx_tbl, cfg->dp_tx_tbl_num, 1);
+ qmp_combo_configure_lane(tx2, cfg->dp_tx_tbl, cfg->dp_tx_tbl_num, 2);
+
+ /* Configure special DP tx tunings */
+ cfg->configure_dp_tx(qmp);
+
+ /* Configure link rate, swing, etc. */
+ cfg->configure_dp_phy(qmp);
+
+ return 0;
+}
+
+static int qmp_combo_dp_power_off(struct phy *phy)
+{
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
+
+ /* Assert DP PHY power down */
+ writel(DP_PHY_PD_CTL_PSR_PWRDN, qmp->dp_dp_phy + QSERDES_DP_PHY_PD_CTL);
+
+ return 0;
+}
+
+static int qmp_combo_usb_power_on(struct phy *phy)
+{
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *serdes = qmp->serdes;
+ void __iomem *tx = qmp->tx;
+ void __iomem *rx = qmp->rx;
+ void __iomem *tx2 = qmp->tx2;
+ void __iomem *rx2 = qmp->rx2;
+ void __iomem *pcs = qmp->pcs;
void __iomem *status;
- unsigned int mask, val, ready;
+ unsigned int val;
int ret;
- qmp_combo_serdes_init(qphy);
+ qmp_combo_configure(serdes, cfg->serdes_tbl, cfg->serdes_tbl_num);
- ret = clk_prepare_enable(qphy->pipe_clk);
+ ret = clk_prepare_enable(qmp->pipe_clk);
if (ret) {
dev_err(qmp->dev, "pipe_clk enable failed err=%d\n", ret);
return ret;
}
/* Tx, Rx, and PCS configurations */
- qmp_combo_configure_lane(tx, cfg->regs, cfg->tx_tbl, cfg->tx_tbl_num, 1);
+ qmp_combo_configure_lane(tx, cfg->tx_tbl, cfg->tx_tbl_num, 1);
+ qmp_combo_configure_lane(tx2, cfg->tx_tbl, cfg->tx_tbl_num, 2);
- if (cfg->lanes >= 2) {
- qmp_combo_configure_lane(qphy->tx2, cfg->regs, cfg->tx_tbl,
- cfg->tx_tbl_num, 2);
- }
+ qmp_combo_configure_lane(rx, cfg->rx_tbl, cfg->rx_tbl_num, 1);
+ qmp_combo_configure_lane(rx2, cfg->rx_tbl, cfg->rx_tbl_num, 2);
- /* Configure special DP tx tunings */
- if (cfg->type == PHY_TYPE_DP)
- cfg->configure_dp_tx(qphy);
-
- qmp_combo_configure_lane(rx, cfg->regs, cfg->rx_tbl, cfg->rx_tbl_num, 1);
-
- if (cfg->lanes >= 2) {
- qmp_combo_configure_lane(qphy->rx2, cfg->regs, cfg->rx_tbl,
- cfg->rx_tbl_num, 2);
- }
-
- /* Configure link rate, swing, etc. */
- if (cfg->type == PHY_TYPE_DP)
- cfg->configure_dp_phy(qphy);
- else
- qmp_combo_configure(pcs, cfg->regs, cfg->pcs_tbl, cfg->pcs_tbl_num);
-
- ret = reset_control_deassert(qmp->ufs_reset);
- if (ret)
- goto err_disable_pipe_clk;
+ qmp_combo_configure(pcs, cfg->pcs_tbl, cfg->pcs_tbl_num);
if (cfg->has_pwrdn_delay)
- usleep_range(cfg->pwrdn_delay_min, cfg->pwrdn_delay_max);
+ usleep_range(10, 20);
- if (cfg->type != PHY_TYPE_DP) {
- /* Pull PHY out of reset state */
- qphy_clrbits(pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
- /* start SerDes and Phy-Coding-Sublayer */
- qphy_setbits(pcs, cfg->regs[QPHY_START_CTRL], cfg->start_ctrl);
+ /* Pull PHY out of reset state */
+ qphy_clrbits(pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
- status = pcs + cfg->regs[QPHY_PCS_STATUS];
- mask = cfg->phy_status;
- ready = 0;
+ /* start SerDes and Phy-Coding-Sublayer */
+ qphy_setbits(pcs, cfg->regs[QPHY_START_CTRL], SERDES_START | PCS_START);
- ret = readl_poll_timeout(status, val, (val & mask) == ready, 10,
- PHY_INIT_COMPLETE_TIMEOUT);
- if (ret) {
- dev_err(qmp->dev, "phy initialization timed-out\n");
- goto err_disable_pipe_clk;
- }
+ status = pcs + cfg->regs[QPHY_PCS_STATUS];
+ ret = readl_poll_timeout(status, val, !(val & PHYSTATUS), 200,
+ PHY_INIT_COMPLETE_TIMEOUT);
+ if (ret) {
+ dev_err(qmp->dev, "phy initialization timed-out\n");
+ goto err_disable_pipe_clk;
}
+
return 0;
err_disable_pipe_clk:
- clk_disable_unprepare(qphy->pipe_clk);
+ clk_disable_unprepare(qmp->pipe_clk);
return ret;
}
-static int qmp_combo_power_off(struct phy *phy)
+static int qmp_combo_usb_power_off(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
- clk_disable_unprepare(qphy->pipe_clk);
+ clk_disable_unprepare(qmp->pipe_clk);
- if (cfg->type == PHY_TYPE_DP) {
- /* Assert DP PHY power down */
- writel(DP_PHY_PD_CTL_PSR_PWRDN, qphy->pcs + QSERDES_DP_PHY_PD_CTL);
- } else {
- /* PHY reset */
- qphy_setbits(qphy->pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
-
- /* stop SerDes and Phy-Coding-Sublayer */
- qphy_clrbits(qphy->pcs, cfg->regs[QPHY_START_CTRL], cfg->start_ctrl);
-
- /* Put PHY into POWER DOWN state: active low */
- if (cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL]) {
- qphy_clrbits(qphy->pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
- cfg->pwrdn_ctrl);
- } else {
- qphy_clrbits(qphy->pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
- cfg->pwrdn_ctrl);
- }
- }
+ /* PHY reset */
+ qphy_setbits(qmp->pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
- return 0;
-}
-
-static int qmp_combo_exit(struct phy *phy)
-{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
+ /* stop SerDes and Phy-Coding-Sublayer */
+ qphy_clrbits(qmp->pcs, cfg->regs[QPHY_START_CTRL],
+ SERDES_START | PCS_START);
- qmp_combo_com_exit(qphy);
+ /* Put PHY into POWER DOWN state: active low */
+ qphy_clrbits(qmp->pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
+ SW_PWRDN);
return 0;
}
-static int qmp_combo_enable(struct phy *phy)
+static int qmp_combo_usb_init(struct phy *phy)
{
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
int ret;
- ret = qmp_combo_init(phy);
+ ret = qmp_combo_com_init(qmp);
if (ret)
return ret;
- ret = qmp_combo_power_on(phy);
+ ret = qmp_combo_usb_power_on(phy);
if (ret)
- qmp_combo_exit(phy);
+ qmp_combo_com_exit(qmp);
return ret;
}
-static int qmp_combo_disable(struct phy *phy)
+static int qmp_combo_usb_exit(struct phy *phy)
{
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
int ret;
- ret = qmp_combo_power_off(phy);
+ ret = qmp_combo_usb_power_off(phy);
if (ret)
return ret;
- return qmp_combo_exit(phy);
+
+ return qmp_combo_com_exit(qmp);
}
-static int qmp_combo_set_mode(struct phy *phy, enum phy_mode mode, int submode)
+static int qmp_combo_usb_set_mode(struct phy *phy, enum phy_mode mode, int submode)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
+ struct qmp_combo *qmp = phy_get_drvdata(phy);
- qphy->mode = mode;
+ qmp->mode = mode;
return 0;
}
-static void qmp_combo_enable_autonomous_mode(struct qmp_phy *qphy)
+static const struct phy_ops qmp_combo_usb_phy_ops = {
+ .init = qmp_combo_usb_init,
+ .exit = qmp_combo_usb_exit,
+ .set_mode = qmp_combo_usb_set_mode,
+ .owner = THIS_MODULE,
+};
+
+static const struct phy_ops qmp_combo_dp_phy_ops = {
+ .init = qmp_combo_dp_init,
+ .configure = qmp_combo_dp_configure,
+ .power_on = qmp_combo_dp_power_on,
+ .calibrate = qmp_combo_dp_calibrate,
+ .power_off = qmp_combo_dp_power_off,
+ .exit = qmp_combo_dp_exit,
+ .owner = THIS_MODULE,
+};
+
+static void qmp_combo_enable_autonomous_mode(struct qmp_combo *qmp)
{
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *pcs_usb = qphy->pcs_usb ?: qphy->pcs;
- void __iomem *pcs_misc = qphy->pcs_misc;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *pcs_usb = qmp->pcs_usb ?: qmp->pcs;
+ void __iomem *pcs_misc = qmp->pcs_misc;
u32 intr_mask;
- if (qphy->mode == PHY_MODE_USB_HOST_SS ||
- qphy->mode == PHY_MODE_USB_DEVICE_SS)
+ if (qmp->mode == PHY_MODE_USB_HOST_SS ||
+ qmp->mode == PHY_MODE_USB_DEVICE_SS)
intr_mask = ARCVR_DTCT_EN | ALFPS_DTCT_EN;
else
intr_mask = ARCVR_DTCT_EN | ARCVR_DTCT_EVENT_SEL;
qphy_clrbits(pcs_misc, QPHY_V3_PCS_MISC_CLAMP_ENABLE, CLAMP_EN);
}
-static void qmp_combo_disable_autonomous_mode(struct qmp_phy *qphy)
+static void qmp_combo_disable_autonomous_mode(struct qmp_combo *qmp)
{
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *pcs_usb = qphy->pcs_usb ?: qphy->pcs;
- void __iomem *pcs_misc = qphy->pcs_misc;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *pcs_usb = qmp->pcs_usb ?: qmp->pcs;
+ void __iomem *pcs_misc = qmp->pcs_misc;
/* Disable i/o clamp_n on resume for normal mode */
if (pcs_misc)
static int __maybe_unused qmp_combo_runtime_suspend(struct device *dev)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
- struct qmp_phy *qphy = qmp->phys[0];
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_combo *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
- dev_vdbg(dev, "Suspending QMP phy, mode:%d\n", qphy->mode);
-
- /* Supported only for USB3 PHY and luckily USB3 is the first phy */
- if (cfg->type != PHY_TYPE_USB3)
- return 0;
+ dev_vdbg(dev, "Suspending QMP phy, mode:%d\n", qmp->mode);
if (!qmp->init_count) {
dev_vdbg(dev, "PHY not initialized, bailing out\n");
return 0;
}
- qmp_combo_enable_autonomous_mode(qphy);
+ qmp_combo_enable_autonomous_mode(qmp);
- clk_disable_unprepare(qphy->pipe_clk);
+ clk_disable_unprepare(qmp->pipe_clk);
clk_bulk_disable_unprepare(cfg->num_clks, qmp->clks);
return 0;
static int __maybe_unused qmp_combo_runtime_resume(struct device *dev)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
- struct qmp_phy *qphy = qmp->phys[0];
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_combo *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
int ret = 0;
- dev_vdbg(dev, "Resuming QMP phy, mode:%d\n", qphy->mode);
-
- /* Supported only for USB3 PHY and luckily USB3 is the first phy */
- if (cfg->type != PHY_TYPE_USB3)
- return 0;
+ dev_vdbg(dev, "Resuming QMP phy, mode:%d\n", qmp->mode);
if (!qmp->init_count) {
dev_vdbg(dev, "PHY not initialized, bailing out\n");
if (ret)
return ret;
- ret = clk_prepare_enable(qphy->pipe_clk);
+ ret = clk_prepare_enable(qmp->pipe_clk);
if (ret) {
dev_err(dev, "pipe_clk enable failed, err=%d\n", ret);
clk_bulk_disable_unprepare(cfg->num_clks, qmp->clks);
return ret;
}
- qmp_combo_disable_autonomous_mode(qphy);
+ qmp_combo_disable_autonomous_mode(qmp);
return 0;
}
-static int qmp_combo_vreg_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static const struct dev_pm_ops qmp_combo_pm_ops = {
+ SET_RUNTIME_PM_OPS(qmp_combo_runtime_suspend,
+ qmp_combo_runtime_resume, NULL)
+};
+
+static int qmp_combo_vreg_init(struct qmp_combo *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int num = cfg->num_vregs;
int ret, i;
return 0;
}
-static int qmp_combo_reset_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static int qmp_combo_reset_init(struct qmp_combo *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int i;
int ret;
return 0;
}
-static int qmp_combo_clk_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static int qmp_combo_clk_init(struct qmp_combo *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int num = cfg->num_clks;
int i;
* clk | +-------+ | +-----+
* +---------------+
*/
-static int phy_pipe_clk_register(struct qcom_qmp *qmp, struct device_node *np)
+static int phy_pipe_clk_register(struct qmp_combo *qmp, struct device_node *np)
{
- struct clk_fixed_rate *fixed;
+ struct clk_fixed_rate *fixed = &qmp->pipe_clk_fixed;
struct clk_init_data init = { };
- int ret;
-
- ret = of_property_read_string(np, "clock-output-names", &init.name);
- if (ret) {
- dev_err(qmp->dev, "%pOFn: No clock-output-names\n", np);
- return ret;
- }
-
- fixed = devm_kzalloc(qmp->dev, sizeof(*fixed), GFP_KERNEL);
- if (!fixed)
- return -ENOMEM;
+ char name[64];
+ snprintf(name, sizeof(name), "%s::pipe_clk", dev_name(qmp->dev));
+ init.name = name;
init.ops = &clk_fixed_rate_ops;
/* controllers using QMP phys use 125MHz pipe clock interface */
fixed->fixed_rate = 125000000;
fixed->hw.init = &init;
- ret = devm_clk_hw_register(qmp->dev, &fixed->hw);
- if (ret)
- return ret;
-
- ret = of_clk_add_hw_provider(np, of_clk_hw_simple_get, &fixed->hw);
- if (ret)
- return ret;
-
- /*
- * Roll a devm action because the clock provider is the child node, but
- * the child node is not actually a device.
- */
- return devm_add_action_or_reset(qmp->dev, phy_clk_release_provider, np);
+ return devm_clk_hw_register(qmp->dev, &fixed->hw);
}
/*
* for DP pixel clock
*
*/
-static int qcom_qmp_dp_pixel_clk_determine_rate(struct clk_hw *hw,
- struct clk_rate_request *req)
+static int qmp_dp_pixel_clk_determine_rate(struct clk_hw *hw, struct clk_rate_request *req)
{
switch (req->rate) {
case 1620000000UL / 2:
}
}
-static unsigned long
-qcom_qmp_dp_pixel_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+static unsigned long qmp_dp_pixel_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
{
- const struct qmp_phy_dp_clks *dp_clks;
- const struct qmp_phy *qphy;
+ const struct qmp_combo *qmp;
const struct phy_configure_opts_dp *dp_opts;
- dp_clks = container_of(hw, struct qmp_phy_dp_clks, dp_pixel_hw);
- qphy = dp_clks->qphy;
- dp_opts = &qphy->dp_opts;
+ qmp = container_of(hw, struct qmp_combo, dp_pixel_hw);
+ dp_opts = &qmp->dp_opts;
switch (dp_opts->link_rate) {
case 1620:
}
}
-static const struct clk_ops qcom_qmp_dp_pixel_clk_ops = {
- .determine_rate = qcom_qmp_dp_pixel_clk_determine_rate,
- .recalc_rate = qcom_qmp_dp_pixel_clk_recalc_rate,
+static const struct clk_ops qmp_dp_pixel_clk_ops = {
+ .determine_rate = qmp_dp_pixel_clk_determine_rate,
+ .recalc_rate = qmp_dp_pixel_clk_recalc_rate,
};
-static int qcom_qmp_dp_link_clk_determine_rate(struct clk_hw *hw,
- struct clk_rate_request *req)
+static int qmp_dp_link_clk_determine_rate(struct clk_hw *hw, struct clk_rate_request *req)
{
switch (req->rate) {
case 162000000:
}
}
-static unsigned long
-qcom_qmp_dp_link_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+static unsigned long qmp_dp_link_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
{
- const struct qmp_phy_dp_clks *dp_clks;
- const struct qmp_phy *qphy;
+ const struct qmp_combo *qmp;
const struct phy_configure_opts_dp *dp_opts;
- dp_clks = container_of(hw, struct qmp_phy_dp_clks, dp_link_hw);
- qphy = dp_clks->qphy;
- dp_opts = &qphy->dp_opts;
+ qmp = container_of(hw, struct qmp_combo, dp_link_hw);
+ dp_opts = &qmp->dp_opts;
switch (dp_opts->link_rate) {
case 1620:
}
}
-static const struct clk_ops qcom_qmp_dp_link_clk_ops = {
- .determine_rate = qcom_qmp_dp_link_clk_determine_rate,
- .recalc_rate = qcom_qmp_dp_link_clk_recalc_rate,
+static const struct clk_ops qmp_dp_link_clk_ops = {
+ .determine_rate = qmp_dp_link_clk_determine_rate,
+ .recalc_rate = qmp_dp_link_clk_recalc_rate,
};
-static struct clk_hw *
-qcom_qmp_dp_clks_hw_get(struct of_phandle_args *clkspec, void *data)
+static struct clk_hw *qmp_dp_clks_hw_get(struct of_phandle_args *clkspec, void *data)
{
- struct qmp_phy_dp_clks *dp_clks = data;
+ struct qmp_combo *qmp = data;
unsigned int idx = clkspec->args[0];
if (idx >= 2) {
}
if (idx == 0)
- return &dp_clks->dp_link_hw;
+ return &qmp->dp_link_hw;
- return &dp_clks->dp_pixel_hw;
+ return &qmp->dp_pixel_hw;
}
-static int phy_dp_clks_register(struct qcom_qmp *qmp, struct qmp_phy *qphy,
- struct device_node *np)
+static int phy_dp_clks_register(struct qmp_combo *qmp, struct device_node *np)
{
struct clk_init_data init = { };
- struct qmp_phy_dp_clks *dp_clks;
char name[64];
int ret;
- dp_clks = devm_kzalloc(qmp->dev, sizeof(*dp_clks), GFP_KERNEL);
- if (!dp_clks)
- return -ENOMEM;
-
- dp_clks->qphy = qphy;
- qphy->dp_clks = dp_clks;
-
snprintf(name, sizeof(name), "%s::link_clk", dev_name(qmp->dev));
- init.ops = &qcom_qmp_dp_link_clk_ops;
+ init.ops = &qmp_dp_link_clk_ops;
init.name = name;
- dp_clks->dp_link_hw.init = &init;
- ret = devm_clk_hw_register(qmp->dev, &dp_clks->dp_link_hw);
+ qmp->dp_link_hw.init = &init;
+ ret = devm_clk_hw_register(qmp->dev, &qmp->dp_link_hw);
if (ret)
return ret;
snprintf(name, sizeof(name), "%s::vco_div_clk", dev_name(qmp->dev));
- init.ops = &qcom_qmp_dp_pixel_clk_ops;
+ init.ops = &qmp_dp_pixel_clk_ops;
init.name = name;
- dp_clks->dp_pixel_hw.init = &init;
- ret = devm_clk_hw_register(qmp->dev, &dp_clks->dp_pixel_hw);
+ qmp->dp_pixel_hw.init = &init;
+ ret = devm_clk_hw_register(qmp->dev, &qmp->dp_pixel_hw);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static struct clk_hw *qmp_combo_clk_hw_get(struct of_phandle_args *clkspec, void *data)
+{
+ struct qmp_combo *qmp = data;
+
+ switch (clkspec->args[0]) {
+ case QMP_USB43DP_USB3_PIPE_CLK:
+ return &qmp->pipe_clk_fixed.hw;
+ case QMP_USB43DP_DP_LINK_CLK:
+ return &qmp->dp_link_hw;
+ case QMP_USB43DP_DP_VCO_DIV_CLK:
+ return &qmp->dp_pixel_hw;
+ }
+
+ return ERR_PTR(-EINVAL);
+}
+
+static int qmp_combo_register_clocks(struct qmp_combo *qmp, struct device_node *usb_np,
+ struct device_node *dp_np)
+{
+ int ret;
+
+ ret = phy_pipe_clk_register(qmp, usb_np);
+ if (ret)
+ return ret;
+
+ ret = phy_dp_clks_register(qmp, dp_np);
if (ret)
return ret;
- ret = of_clk_add_hw_provider(np, qcom_qmp_dp_clks_hw_get, dp_clks);
+ /*
+ * Register a single provider for bindings without child nodes.
+ */
+ if (usb_np == qmp->dev->of_node)
+ return devm_of_clk_add_hw_provider(qmp->dev, qmp_combo_clk_hw_get, qmp);
+
+ /*
+ * Register multiple providers for legacy bindings with child nodes.
+ */
+ ret = of_clk_add_hw_provider(usb_np, of_clk_hw_simple_get,
+ &qmp->pipe_clk_fixed.hw);
if (ret)
return ret;
* Roll a devm action because the clock provider is the child node, but
* the child node is not actually a device.
*/
- return devm_add_action_or_reset(qmp->dev, phy_clk_release_provider, np);
-}
+ ret = devm_add_action_or_reset(qmp->dev, phy_clk_release_provider, usb_np);
+ if (ret)
+ return ret;
-static const struct phy_ops qmp_combo_usb_ops = {
- .init = qmp_combo_enable,
- .exit = qmp_combo_disable,
- .set_mode = qmp_combo_set_mode,
- .owner = THIS_MODULE,
-};
+ ret = of_clk_add_hw_provider(dp_np, qmp_dp_clks_hw_get, qmp);
+ if (ret)
+ return ret;
-static const struct phy_ops qmp_combo_dp_ops = {
- .init = qmp_combo_init,
- .configure = qcom_qmp_dp_phy_configure,
- .power_on = qmp_combo_power_on,
- .calibrate = qcom_qmp_dp_phy_calibrate,
- .power_off = qmp_combo_power_off,
- .exit = qmp_combo_exit,
- .set_mode = qmp_combo_set_mode,
- .owner = THIS_MODULE,
-};
+ return devm_add_action_or_reset(qmp->dev, phy_clk_release_provider, dp_np);
+}
-static int qmp_combo_create(struct device *dev, struct device_node *np, int id,
- void __iomem *serdes, const struct qmp_phy_cfg *cfg)
+static int qmp_combo_parse_dt_lecacy_dp(struct qmp_combo *qmp, struct device_node *np)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
- struct phy *generic_phy;
- struct qmp_phy *qphy;
- const struct phy_ops *ops;
- int ret;
+ struct device *dev = qmp->dev;
- qphy = devm_kzalloc(dev, sizeof(*qphy), GFP_KERNEL);
- if (!qphy)
- return -ENOMEM;
+ /*
+ * Get memory resources from the DP child node:
+ * Resources are indexed as: tx -> 0; rx -> 1; pcs -> 2;
+ * tx2 -> 3; rx2 -> 4
+ *
+ * Note that only tx/tx2 and pcs (dp_phy) are used by the DP
+ * implementation.
+ */
+ qmp->dp_tx = devm_of_iomap(dev, np, 0, NULL);
+ if (IS_ERR(qmp->dp_tx))
+ return PTR_ERR(qmp->dp_tx);
+
+ qmp->dp_dp_phy = devm_of_iomap(dev, np, 2, NULL);
+ if (IS_ERR(qmp->dp_dp_phy))
+ return PTR_ERR(qmp->dp_dp_phy);
+
+ qmp->dp_tx2 = devm_of_iomap(dev, np, 3, NULL);
+ if (IS_ERR(qmp->dp_tx2))
+ return PTR_ERR(qmp->dp_tx2);
+
+ return 0;
+}
+
+static int qmp_combo_parse_dt_lecacy_usb(struct qmp_combo *qmp, struct device_node *np)
+{
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
- qphy->cfg = cfg;
- qphy->serdes = serdes;
/*
- * Get memory resources for each phy lane:
- * Resources are indexed as: tx -> 0; rx -> 1; pcs -> 2.
- * For dual lane PHYs: tx2 -> 3, rx2 -> 4, pcs_misc (optional) -> 5
- * For single lane PHYs: pcs_misc (optional) -> 3.
+ * Get memory resources from the USB child node:
+ * Resources are indexed as: tx -> 0; rx -> 1; pcs -> 2;
+ * tx2 -> 3; rx2 -> 4; pcs_misc (optional) -> 5
*/
- qphy->tx = devm_of_iomap(dev, np, 0, NULL);
- if (IS_ERR(qphy->tx))
- return PTR_ERR(qphy->tx);
+ qmp->tx = devm_of_iomap(dev, np, 0, NULL);
+ if (IS_ERR(qmp->tx))
+ return PTR_ERR(qmp->tx);
- qphy->rx = devm_of_iomap(dev, np, 1, NULL);
- if (IS_ERR(qphy->rx))
- return PTR_ERR(qphy->rx);
+ qmp->rx = devm_of_iomap(dev, np, 1, NULL);
+ if (IS_ERR(qmp->rx))
+ return PTR_ERR(qmp->rx);
- qphy->pcs = devm_of_iomap(dev, np, 2, NULL);
- if (IS_ERR(qphy->pcs))
- return PTR_ERR(qphy->pcs);
+ qmp->pcs = devm_of_iomap(dev, np, 2, NULL);
+ if (IS_ERR(qmp->pcs))
+ return PTR_ERR(qmp->pcs);
if (cfg->pcs_usb_offset)
- qphy->pcs_usb = qphy->pcs + cfg->pcs_usb_offset;
-
- if (cfg->lanes >= 2) {
- qphy->tx2 = devm_of_iomap(dev, np, 3, NULL);
- if (IS_ERR(qphy->tx2))
- return PTR_ERR(qphy->tx2);
+ qmp->pcs_usb = qmp->pcs + cfg->pcs_usb_offset;
- qphy->rx2 = devm_of_iomap(dev, np, 4, NULL);
- if (IS_ERR(qphy->rx2))
- return PTR_ERR(qphy->rx2);
+ qmp->tx2 = devm_of_iomap(dev, np, 3, NULL);
+ if (IS_ERR(qmp->tx2))
+ return PTR_ERR(qmp->tx2);
- qphy->pcs_misc = devm_of_iomap(dev, np, 5, NULL);
- } else {
- qphy->pcs_misc = devm_of_iomap(dev, np, 3, NULL);
- }
+ qmp->rx2 = devm_of_iomap(dev, np, 4, NULL);
+ if (IS_ERR(qmp->rx2))
+ return PTR_ERR(qmp->rx2);
- if (IS_ERR(qphy->pcs_misc)) {
+ qmp->pcs_misc = devm_of_iomap(dev, np, 5, NULL);
+ if (IS_ERR(qmp->pcs_misc)) {
dev_vdbg(dev, "PHY pcs_misc-reg not used\n");
- qphy->pcs_misc = NULL;
+ qmp->pcs_misc = NULL;
}
- /*
- * Get PHY's Pipe clock, if any. USB3 and PCIe are PIPE3
- * based phys, so they essentially have pipe clock. So,
- * we return error in case phy is USB3 or PIPE type.
- * Otherwise, we initialize pipe clock to NULL for
- * all phys that don't need this.
- */
- qphy->pipe_clk = devm_get_clk_from_child(dev, np, NULL);
- if (IS_ERR(qphy->pipe_clk)) {
- if (cfg->type == PHY_TYPE_USB3)
- return dev_err_probe(dev, PTR_ERR(qphy->pipe_clk),
- "failed to get lane%d pipe_clk\n",
- id);
- qphy->pipe_clk = NULL;
+ qmp->pipe_clk = devm_get_clk_from_child(dev, np, NULL);
+ if (IS_ERR(qmp->pipe_clk)) {
+ return dev_err_probe(dev, PTR_ERR(qmp->pipe_clk),
+ "failed to get pipe clock\n");
}
- if (cfg->type == PHY_TYPE_DP)
- ops = &qmp_combo_dp_ops;
- else
- ops = &qmp_combo_usb_ops;
+ return 0;
+}
- generic_phy = devm_phy_create(dev, np, ops);
- if (IS_ERR(generic_phy)) {
- ret = PTR_ERR(generic_phy);
- dev_err(dev, "failed to create qphy %d\n", ret);
+static int qmp_combo_parse_dt_legacy(struct qmp_combo *qmp, struct device_node *usb_np,
+ struct device_node *dp_np)
+{
+ struct platform_device *pdev = to_platform_device(qmp->dev);
+ int ret;
+
+ qmp->serdes = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(qmp->serdes))
+ return PTR_ERR(qmp->serdes);
+
+ qmp->com = devm_platform_ioremap_resource(pdev, 1);
+ if (IS_ERR(qmp->com))
+ return PTR_ERR(qmp->com);
+
+ qmp->dp_serdes = devm_platform_ioremap_resource(pdev, 2);
+ if (IS_ERR(qmp->dp_serdes))
+ return PTR_ERR(qmp->dp_serdes);
+
+ ret = qmp_combo_parse_dt_lecacy_usb(qmp, usb_np);
+ if (ret)
return ret;
- }
- qphy->phy = generic_phy;
- qphy->qmp = qmp;
- qmp->phys[id] = qphy;
- phy_set_drvdata(generic_phy, qphy);
+ ret = qmp_combo_parse_dt_lecacy_dp(qmp, dp_np);
+ if (ret)
+ return ret;
return 0;
}
-static const struct of_device_id qmp_combo_of_match_table[] = {
- {
- .compatible = "qcom,sc7180-qmp-usb3-dp-phy",
- .data = &sc7180_usb3dpphy_cfg,
- },
- {
- .compatible = "qcom,sdm845-qmp-usb3-dp-phy",
- .data = &sdm845_usb3dpphy_cfg,
- },
- {
- .compatible = "qcom,sm8250-qmp-usb3-dp-phy",
- .data = &sm8250_usb3dpphy_cfg,
- },
- {
- .compatible = "qcom,sc8180x-qmp-usb3-dp-phy",
- .data = &sc8180x_usb3dpphy_cfg,
- },
- {
- .compatible = "qcom,sc8280xp-qmp-usb43dp-phy",
- .data = &sc8280xp_usb43dpphy_combo_cfg,
- },
- { }
-};
-MODULE_DEVICE_TABLE(of, qmp_combo_of_match_table);
+static int qmp_combo_parse_dt(struct qmp_combo *qmp)
+{
+ struct platform_device *pdev = to_platform_device(qmp->dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ const struct qmp_combo_offsets *offs = cfg->offsets;
+ struct device *dev = qmp->dev;
+ void __iomem *base;
-static const struct dev_pm_ops qmp_combo_pm_ops = {
- SET_RUNTIME_PM_OPS(qmp_combo_runtime_suspend,
- qmp_combo_runtime_resume, NULL)
-};
+ if (!offs)
+ return -EINVAL;
+
+ base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+
+ qmp->com = base + offs->com;
+ qmp->tx = base + offs->txa;
+ qmp->rx = base + offs->rxa;
+ qmp->tx2 = base + offs->txb;
+ qmp->rx2 = base + offs->rxb;
+
+ qmp->serdes = base + offs->usb3_serdes;
+ qmp->pcs_misc = base + offs->usb3_pcs_misc;
+ qmp->pcs = base + offs->usb3_pcs;
+ qmp->pcs_usb = base + offs->usb3_pcs_usb;
+
+ qmp->dp_serdes = base + offs->dp_serdes;
+ qmp->dp_tx = base + offs->txa;
+ qmp->dp_tx2 = base + offs->txb;
+ qmp->dp_dp_phy = base + offs->dp_dp_phy;
+
+ qmp->pipe_clk = devm_clk_get(dev, "usb3_pipe");
+ if (IS_ERR(qmp->pipe_clk)) {
+ return dev_err_probe(dev, PTR_ERR(qmp->pipe_clk),
+ "failed to get usb3_pipe clock\n");
+ }
+
+ return 0;
+}
+
+static struct phy *qmp_combo_phy_xlate(struct device *dev, struct of_phandle_args *args)
+{
+ struct qmp_combo *qmp = dev_get_drvdata(dev);
+
+ if (args->args_count == 0)
+ return ERR_PTR(-EINVAL);
+
+ switch (args->args[0]) {
+ case QMP_USB43DP_USB3_PHY:
+ return qmp->usb_phy;
+ case QMP_USB43DP_DP_PHY:
+ return qmp->dp_phy;
+ }
+
+ return ERR_PTR(-EINVAL);
+}
static int qmp_combo_probe(struct platform_device *pdev)
{
- struct qcom_qmp *qmp;
+ struct qmp_combo *qmp;
struct device *dev = &pdev->dev;
- struct device_node *child;
+ struct device_node *dp_np, *usb_np;
struct phy_provider *phy_provider;
- void __iomem *serdes;
- void __iomem *usb_serdes;
- void __iomem *dp_serdes = NULL;
- const struct qmp_phy_combo_cfg *combo_cfg = NULL;
- const struct qmp_phy_cfg *cfg = NULL;
- const struct qmp_phy_cfg *usb_cfg = NULL;
- const struct qmp_phy_cfg *dp_cfg = NULL;
- int num, id, expected_phys;
int ret;
qmp = devm_kzalloc(dev, sizeof(*qmp), GFP_KERNEL);
return -ENOMEM;
qmp->dev = dev;
- dev_set_drvdata(dev, qmp);
- /* Get the specific init parameters of QMP phy */
- combo_cfg = of_device_get_match_data(dev);
- if (!combo_cfg)
+ qmp->cfg = of_device_get_match_data(dev);
+ if (!qmp->cfg)
return -EINVAL;
- usb_cfg = combo_cfg->usb_cfg;
- cfg = usb_cfg; /* Setup clks and regulators */
-
- /* per PHY serdes; usually located at base address */
- usb_serdes = serdes = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(serdes))
- return PTR_ERR(serdes);
-
- qmp->dp_com = devm_platform_ioremap_resource(pdev, 1);
- if (IS_ERR(qmp->dp_com))
- return PTR_ERR(qmp->dp_com);
-
- /* Only two serdes for combo PHY */
- dp_serdes = devm_platform_ioremap_resource(pdev, 2);
- if (IS_ERR(dp_serdes))
- return PTR_ERR(dp_serdes);
-
- dp_cfg = combo_cfg->dp_cfg;
- expected_phys = 2;
-
mutex_init(&qmp->phy_mutex);
- ret = qmp_combo_clk_init(dev, cfg);
+ ret = qmp_combo_clk_init(qmp);
if (ret)
return ret;
- ret = qmp_combo_reset_init(dev, cfg);
+ ret = qmp_combo_reset_init(qmp);
if (ret)
return ret;
- ret = qmp_combo_vreg_init(dev, cfg);
+ ret = qmp_combo_vreg_init(qmp);
if (ret)
- return dev_err_probe(dev, ret,
- "failed to get regulator supplies\n");
+ return ret;
- num = of_get_available_child_count(dev->of_node);
- /* do we have a rogue child node ? */
- if (num > expected_phys)
- return -EINVAL;
+ /* Check for legacy binding with child nodes. */
+ usb_np = of_get_child_by_name(dev->of_node, "usb3-phy");
+ if (usb_np) {
+ dp_np = of_get_child_by_name(dev->of_node, "dp-phy");
+ if (!dp_np) {
+ of_node_put(usb_np);
+ return -EINVAL;
+ }
- qmp->phys = devm_kcalloc(dev, num, sizeof(*qmp->phys), GFP_KERNEL);
- if (!qmp->phys)
- return -ENOMEM;
+ ret = qmp_combo_parse_dt_legacy(qmp, usb_np, dp_np);
+ } else {
+ usb_np = of_node_get(dev->of_node);
+ dp_np = of_node_get(dev->of_node);
+
+ ret = qmp_combo_parse_dt(qmp);
+ }
+ if (ret)
+ goto err_node_put;
pm_runtime_set_active(dev);
ret = devm_pm_runtime_enable(dev);
if (ret)
- return ret;
+ goto err_node_put;
/*
* Prevent runtime pm from being ON by default. Users can enable
* it using power/control in sysfs.
*/
pm_runtime_forbid(dev);
- id = 0;
- for_each_available_child_of_node(dev->of_node, child) {
- if (of_node_name_eq(child, "dp-phy")) {
- cfg = dp_cfg;
- serdes = dp_serdes;
-
- /* Create per-lane phy */
- ret = qmp_combo_create(dev, child, id, serdes, cfg);
- if (ret) {
- dev_err(dev, "failed to create lane%d phy, %d\n",
- id, ret);
- goto err_node_put;
- }
-
- ret = phy_dp_clks_register(qmp, qmp->phys[id], child);
- if (ret) {
- dev_err(qmp->dev,
- "failed to register DP clock source\n");
- goto err_node_put;
- }
- } else if (of_node_name_eq(child, "usb3-phy")) {
- cfg = usb_cfg;
- serdes = usb_serdes;
-
- /* Create per-lane phy */
- ret = qmp_combo_create(dev, child, id, serdes, cfg);
- if (ret) {
- dev_err(dev, "failed to create lane%d phy, %d\n",
- id, ret);
- goto err_node_put;
- }
-
- /*
- * Register the pipe clock provided by phy.
- * See function description to see details of this pipe clock.
- */
- ret = phy_pipe_clk_register(qmp, child);
- if (ret) {
- dev_err(qmp->dev,
- "failed to register pipe clock source\n");
- goto err_node_put;
- }
- }
+ ret = qmp_combo_register_clocks(qmp, usb_np, dp_np);
+ if (ret)
+ goto err_node_put;
+
+ qmp->usb_phy = devm_phy_create(dev, usb_np, &qmp_combo_usb_phy_ops);
+ if (IS_ERR(qmp->usb_phy)) {
+ ret = PTR_ERR(qmp->usb_phy);
+ dev_err(dev, "failed to create USB PHY: %d\n", ret);
+ goto err_node_put;
+ }
+
+ phy_set_drvdata(qmp->usb_phy, qmp);
- id++;
+ qmp->dp_phy = devm_phy_create(dev, dp_np, &qmp_combo_dp_phy_ops);
+ if (IS_ERR(qmp->dp_phy)) {
+ ret = PTR_ERR(qmp->dp_phy);
+ dev_err(dev, "failed to create DP PHY: %d\n", ret);
+ goto err_node_put;
}
- phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
+ phy_set_drvdata(qmp->dp_phy, qmp);
+
+ dev_set_drvdata(dev, qmp);
+
+ if (usb_np == dev->of_node)
+ phy_provider = devm_of_phy_provider_register(dev, qmp_combo_phy_xlate);
+ else
+ phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
+
+ of_node_put(usb_np);
+ of_node_put(dp_np);
return PTR_ERR_OR_ZERO(phy_provider);
err_node_put:
- of_node_put(child);
+ of_node_put(usb_np);
+ of_node_put(dp_np);
return ret;
}
+static const struct of_device_id qmp_combo_of_match_table[] = {
+ {
+ .compatible = "qcom,sc7180-qmp-usb3-dp-phy",
+ .data = &sc7180_usb3dpphy_cfg,
+ },
+ {
+ .compatible = "qcom,sc8180x-qmp-usb3-dp-phy",
+ .data = &sc8180x_usb3dpphy_cfg,
+ },
+ {
+ .compatible = "qcom,sc8280xp-qmp-usb43dp-phy",
+ .data = &sc8280xp_usb43dpphy_cfg,
+ },
+ {
+ .compatible = "qcom,sdm845-qmp-usb3-dp-phy",
+ .data = &sdm845_usb3dpphy_cfg,
+ },
+ {
+ .compatible = "qcom,sm8250-qmp-usb3-dp-phy",
+ .data = &sm8250_usb3dpphy_cfg,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, qmp_combo_of_match_table);
+
static struct platform_driver qmp_combo_driver = {
.probe = qmp_combo_probe,
.driver = {
#include <linux/reset.h>
#include <linux/slab.h>
-#include <dt-bindings/phy/phy.h>
-
#include "phy-qcom-qmp.h"
/* QPHY_SW_RESET bit */
#define PLL_READY_GATE_EN BIT(3)
/* QPHY_PCS_STATUS bit */
#define PHYSTATUS BIT(6)
-#define PHYSTATUS_4_20 BIT(7)
/* QPHY_COM_PCS_READY_STATUS bit */
#define PCS_READY BIT(0)
#define PHY_INIT_COMPLETE_TIMEOUT 10000
#define POWER_DOWN_DELAY_US_MIN 10
-#define POWER_DOWN_DELAY_US_MAX 11
+#define POWER_DOWN_DELAY_US_MAX 20
struct qmp_phy_init_tbl {
unsigned int offset;
unsigned int val;
/*
- * register part of layout ?
- * if yes, then offset gives index in the reg-layout
- */
- bool in_layout;
- /*
* mask of lanes for which this register is written
* for cases when second lane needs different values
*/
.lane_mask = 0xff, \
}
-#define QMP_PHY_INIT_CFG_L(o, v) \
- { \
- .offset = o, \
- .val = v, \
- .in_layout = true, \
- .lane_mask = 0xff, \
- }
-
#define QMP_PHY_INIT_CFG_LANE(o, v, l) \
{ \
.offset = o, \
QPHY_SW_RESET,
QPHY_START_CTRL,
QPHY_PCS_STATUS,
- QPHY_PCS_POWER_DOWN_CONTROL,
/* Keep last to ensure regs_layout arrays are properly initialized */
QPHY_LAYOUT_SIZE
};
/* array of registers with different offsets */
const unsigned int *regs;
-
- unsigned int start_ctrl;
- unsigned int pwrdn_ctrl;
- unsigned int mask_com_pcs_ready;
- /* bit offset of PHYSTATUS in QPHY_PCS_STATUS register */
- unsigned int phy_status;
-
- /* true, if PHY needs delay after POWER_DOWN */
- bool has_pwrdn_delay;
- /* power_down delay in usec */
- int pwrdn_delay_min;
- int pwrdn_delay_max;
};
/**
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = pciephy_regs_layout,
-
- .start_ctrl = PCS_START | PLL_READY_GATE_EN,
- .pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
- .mask_com_pcs_ready = PCS_READY,
- .phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
};
static void qmp_pcie_msm8996_configure_lane(void __iomem *base,
- const unsigned int *regs,
const struct qmp_phy_init_tbl tbl[],
int num,
u8 lane_mask)
if (!(t->lane_mask & lane_mask))
continue;
- if (t->in_layout)
- writel(t->val, base + regs[t->offset]);
- else
- writel(t->val, base + t->offset);
+ writel(t->val, base + t->offset);
}
}
static void qmp_pcie_msm8996_configure(void __iomem *base,
- const unsigned int *regs,
const struct qmp_phy_init_tbl tbl[],
int num)
{
- qmp_pcie_msm8996_configure_lane(base, regs, tbl, num, 0xff);
+ qmp_pcie_msm8996_configure_lane(base, tbl, num, 0xff);
}
static int qmp_pcie_msm8996_serdes_init(struct qmp_phy *qphy)
const struct qmp_phy_init_tbl *serdes_tbl = cfg->serdes_tbl;
int serdes_tbl_num = cfg->serdes_tbl_num;
void __iomem *status;
- unsigned int mask, val;
+ unsigned int val;
int ret;
- qmp_pcie_msm8996_configure(serdes, cfg->regs, serdes_tbl, serdes_tbl_num);
+ qmp_pcie_msm8996_configure(serdes, serdes_tbl, serdes_tbl_num);
qphy_clrbits(serdes, cfg->regs[QPHY_COM_SW_RESET], SW_RESET);
qphy_setbits(serdes, cfg->regs[QPHY_COM_START_CONTROL],
SERDES_START | PCS_START);
status = serdes + cfg->regs[QPHY_COM_PCS_READY_STATUS];
- mask = cfg->mask_com_pcs_ready;
-
- ret = readl_poll_timeout(status, val, (val & mask), 10,
+ ret = readl_poll_timeout(status, val, (val & PCS_READY), 200,
PHY_INIT_COMPLETE_TIMEOUT);
if (ret) {
dev_err(qmp->dev,
return 0;
}
- /* turn on regulator supplies */
ret = regulator_bulk_enable(cfg->num_vregs, qmp->vregs);
if (ret) {
dev_err(qmp->dev, "failed to enable regulators, err=%d\n", ret);
void __iomem *rx = qphy->rx;
void __iomem *pcs = qphy->pcs;
void __iomem *status;
- unsigned int mask, val, ready;
+ unsigned int val;
int ret;
qmp_pcie_msm8996_serdes_init(qphy);
}
/* Tx, Rx, and PCS configurations */
- qmp_pcie_msm8996_configure_lane(tx, cfg->regs, cfg->tx_tbl,
- cfg->tx_tbl_num, 1);
-
- qmp_pcie_msm8996_configure_lane(rx, cfg->regs, cfg->rx_tbl,
- cfg->rx_tbl_num, 1);
-
- qmp_pcie_msm8996_configure(pcs, cfg->regs, cfg->pcs_tbl, cfg->pcs_tbl_num);
+ qmp_pcie_msm8996_configure_lane(tx, cfg->tx_tbl, cfg->tx_tbl_num, 1);
+ qmp_pcie_msm8996_configure_lane(rx, cfg->rx_tbl, cfg->rx_tbl_num, 1);
+ qmp_pcie_msm8996_configure(pcs, cfg->pcs_tbl, cfg->pcs_tbl_num);
/*
* Pull out PHY from POWER DOWN state.
* This is active low enable signal to power-down PHY.
*/
- qphy_setbits(pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL, cfg->pwrdn_ctrl);
+ qphy_setbits(pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
+ SW_PWRDN | REFCLK_DRV_DSBL);
- if (cfg->has_pwrdn_delay)
- usleep_range(cfg->pwrdn_delay_min, cfg->pwrdn_delay_max);
+ usleep_range(POWER_DOWN_DELAY_US_MIN, POWER_DOWN_DELAY_US_MAX);
/* Pull PHY out of reset state */
qphy_clrbits(pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
/* start SerDes and Phy-Coding-Sublayer */
- qphy_setbits(pcs, cfg->regs[QPHY_START_CTRL], cfg->start_ctrl);
+ qphy_setbits(pcs, cfg->regs[QPHY_START_CTRL],
+ PCS_START | PLL_READY_GATE_EN);
status = pcs + cfg->regs[QPHY_PCS_STATUS];
- mask = cfg->phy_status;
- ready = 0;
-
- ret = readl_poll_timeout(status, val, (val & mask) == ready, 10,
+ ret = readl_poll_timeout(status, val, !(val & PHYSTATUS), 200,
PHY_INIT_COMPLETE_TIMEOUT);
if (ret) {
dev_err(qmp->dev, "phy initialization timed-out\n");
qphy_setbits(qphy->pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
/* stop SerDes and Phy-Coding-Sublayer */
- qphy_clrbits(qphy->pcs, cfg->regs[QPHY_START_CTRL], cfg->start_ctrl);
+ qphy_clrbits(qphy->pcs, cfg->regs[QPHY_START_CTRL],
+ SERDES_START | PCS_START);
/* Put PHY into POWER DOWN state: active low */
- if (cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL]) {
- qphy_clrbits(qphy->pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
- cfg->pwrdn_ctrl);
- } else {
- qphy_clrbits(qphy->pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
- cfg->pwrdn_ctrl);
- }
+ qphy_clrbits(qphy->pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
+ SW_PWRDN | REFCLK_DRV_DSBL);
return 0;
}
qphy->cfg = cfg;
qphy->serdes = serdes;
/*
- * Get memory resources for each phy lane:
+ * Get memory resources for each PHY:
* Resources are indexed as: tx -> 0; rx -> 1; pcs -> 2.
*/
qphy->tx = devm_of_iomap(dev, np, 0, NULL);
qmp->dev = dev;
dev_set_drvdata(dev, qmp);
- /* Get the specific init parameters of QMP phy */
cfg = of_device_get_match_data(dev);
if (!cfg)
return -EINVAL;
- /* per PHY serdes; usually located at base address */
serdes = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(serdes))
return PTR_ERR(serdes);
ret = qmp_pcie_msm8996_vreg_init(dev, cfg);
if (ret)
- return dev_err_probe(dev, ret,
- "failed to get regulator supplies\n");
+ return ret;
num = of_get_available_child_count(dev->of_node);
/* do we have a rogue child node ? */
#include <linux/io.h>
#include <linux/iopoll.h>
#include <linux/kernel.h>
+#include <linux/mfd/syscon.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/of_address.h>
+#include <linux/phy/pcie.h>
#include <linux/phy/phy.h>
#include <linux/platform_device.h>
+#include <linux/regmap.h>
#include <linux/regulator/consumer.h>
#include <linux/reset.h>
#include <linux/slab.h>
-#include <dt-bindings/phy/phy.h>
-
#include "phy-qcom-qmp.h"
/* QPHY_SW_RESET bit */
unsigned int offset;
unsigned int val;
/*
- * register part of layout ?
- * if yes, then offset gives index in the reg-layout
- */
- bool in_layout;
- /*
* mask of lanes for which this register is written
* for cases when second lane needs different values
*/
.lane_mask = 0xff, \
}
-#define QMP_PHY_INIT_CFG_L(o, v) \
- { \
- .offset = o, \
- .val = v, \
- .in_layout = true, \
- .lane_mask = 0xff, \
- }
-
#define QMP_PHY_INIT_CFG_LANE(o, v, l) \
{ \
.offset = o, \
/* set of registers with offsets different per-PHY */
enum qphy_reg_layout {
- /* Common block control registers */
- QPHY_COM_SW_RESET,
- QPHY_COM_POWER_DOWN_CONTROL,
- QPHY_COM_START_CONTROL,
- QPHY_COM_PCS_READY_STATUS,
/* PCS registers */
QPHY_SW_RESET,
QPHY_START_CTRL,
};
static const unsigned int pciephy_regs_layout[QPHY_LAYOUT_SIZE] = {
- [QPHY_COM_SW_RESET] = 0x400,
- [QPHY_COM_POWER_DOWN_CONTROL] = 0x404,
- [QPHY_COM_START_CONTROL] = 0x408,
- [QPHY_COM_PCS_READY_STATUS] = 0x448,
[QPHY_SW_RESET] = 0x00,
[QPHY_START_CTRL] = 0x08,
[QPHY_PCS_STATUS] = 0x174,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = 0x04,
};
static const unsigned int sdm845_qmp_pciephy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_SW_RESET] = 0x00,
[QPHY_START_CTRL] = 0x08,
[QPHY_PCS_STATUS] = 0x174,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = 0x04,
};
static const unsigned int sdm845_qhp_pciephy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_SW_RESET] = 0x00,
[QPHY_START_CTRL] = 0x08,
[QPHY_PCS_STATUS] = 0x2ac,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = 0x04,
};
static const unsigned int sm8250_pcie_regs_layout[QPHY_LAYOUT_SIZE] = {
QMP_PHY_INIT_CFG(QPHY_V2_PCS_RX_SIGDET_LVL, 0x99),
QMP_PHY_INIT_CFG(QPHY_V2_PCS_TXDEEMPH_M6DB_V0, 0x15),
QMP_PHY_INIT_CFG(QPHY_V2_PCS_TXDEEMPH_M3P5DB_V0, 0xe),
- QMP_PHY_INIT_CFG_L(QPHY_SW_RESET, 0x0),
- QMP_PHY_INIT_CFG_L(QPHY_START_CTRL, 0x3),
};
static const struct qmp_phy_init_tbl ipq8074_pcie_gen3_serdes_tbl[] = {
QMP_PHY_INIT_CFG(QPHY_V4_PCS_FLL_CNTRL1, 0x01),
QMP_PHY_INIT_CFG(QPHY_V4_PCS_P2U3_WAKEUP_DLY_TIME_AUXCLK_H, 0x0),
QMP_PHY_INIT_CFG(QPHY_V4_PCS_P2U3_WAKEUP_DLY_TIME_AUXCLK_L, 0x1),
+ QMP_PHY_INIT_CFG(QPHY_V4_PCS_G12S1_TXDEEMPH_M3P5DB, 0x10),
+ QMP_PHY_INIT_CFG(QPHY_V4_PCS_RX_DCC_CAL_CONFIG, 0x01),
+ QMP_PHY_INIT_CFG(QPHY_V4_PCS_RX_SIGDET_LVL, 0xaa),
+ QMP_PHY_INIT_CFG(QPHY_V4_PCS_REFGEN_REQ_CONFIG1, 0x0d),
+};
+
+static const struct qmp_phy_init_tbl ipq8074_pcie_gen3_pcs_misc_tbl[] = {
QMP_PHY_INIT_CFG(QPHY_V4_PCS_PCIE_OSC_DTCT_ACTIONS, 0x0),
QMP_PHY_INIT_CFG(QPHY_V4_PCS_PCIE_L1P1_WAKEUP_DLY_TIME_AUXCLK_H, 0x00),
QMP_PHY_INIT_CFG(QPHY_V4_PCS_PCIE_L1P1_WAKEUP_DLY_TIME_AUXCLK_L, 0x01),
QMP_PHY_INIT_CFG(QPHY_V4_PCS_PCIE_OSC_DTCT_MODE2_CONFIG2, 0x50),
QMP_PHY_INIT_CFG(QPHY_V4_PCS_PCIE_OSC_DTCT_MODE2_CONFIG4, 0x1a),
QMP_PHY_INIT_CFG(QPHY_V4_PCS_PCIE_OSC_DTCT_MODE2_CONFIG5, 0x6),
- QMP_PHY_INIT_CFG(QPHY_V4_PCS_G12S1_TXDEEMPH_M3P5DB, 0x10),
QMP_PHY_INIT_CFG(QPHY_V4_PCS_PCIE_ENDPOINT_REFCLK_DRIVE, 0xc1),
- QMP_PHY_INIT_CFG(QPHY_V4_PCS_RX_DCC_CAL_CONFIG, 0x01),
- QMP_PHY_INIT_CFG(QPHY_V4_PCS_RX_SIGDET_LVL, 0xaa),
- QMP_PHY_INIT_CFG(QPHY_V4_PCS_REFGEN_REQ_CONFIG1, 0x0d),
};
static const struct qmp_phy_init_tbl sdm845_qmp_pcie_serdes_tbl[] = {
QMP_PHY_INIT_CFG(QPHY_V4_PCS_PCIE_ENDPOINT_REFCLK_DRIVE, 0xc1),
};
+static const struct qmp_phy_init_tbl sc8280xp_qmp_pcie_serdes_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_EN_CENTER, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_PER1, 0x31),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_PER2, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_STEP_SIZE1_MODE0, 0xde),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_STEP_SIZE2_MODE0, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_STEP_SIZE1_MODE1, 0x4c),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_STEP_SIZE2_MODE1, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CLK_ENABLE1, 0x90),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_IVCO, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CP_CTRL_MODE0, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CP_CTRL_MODE1, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_RCTRL_MODE0, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_RCTRL_MODE1, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_CCTRL_MODE0, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_CCTRL_MODE1, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SYSCLK_EN_SEL, 0x08),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP_EN, 0x42),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP1_MODE0, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP2_MODE0, 0x1a),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP1_MODE1, 0x14),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP2_MODE1, 0x34),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_DEC_START_MODE0, 0x82),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_DEC_START_MODE1, 0x68),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_DIV_FRAC_START1_MODE0, 0x55),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_DIV_FRAC_START2_MODE0, 0x55),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_DIV_FRAC_START3_MODE0, 0x03),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_DIV_FRAC_START1_MODE1, 0xab),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_DIV_FRAC_START2_MODE1, 0xaa),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_DIV_FRAC_START3_MODE1, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_VCO_TUNE_MAP, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_VCO_TUNE1_MODE0, 0x24),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_VCO_TUNE1_MODE1, 0xb4),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_VCO_TUNE2_MODE1, 0x03),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CLK_SELECT, 0x34),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_HSCLK_SEL, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CORECLK_DIV_MODE1, 0x08),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_BIN_VCOCAL_CMP_CODE1_MODE0, 0xb9),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_BIN_VCOCAL_CMP_CODE2_MODE0, 0x1e),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_BIN_VCOCAL_CMP_CODE1_MODE1, 0x94),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_BIN_VCOCAL_CMP_CODE2_MODE1, 0x18),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_BIN_VCOCAL_HSCLK_SEL, 0x11),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x1_pcie_rc_serdes_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SYSCLK_BUF_ENABLE, 0x07),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x2_pcie_rc_serdes_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_BIAS_EN_CLKBUFLR_EN, 0x14),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x4_pcie_serdes_4ln_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_BIAS_EN_CLKBUFLR_EN, 0x1c),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x1_pcie_tx_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V5_TX_PI_QEC_CTRL, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V5_TX_LANE_MODE_1, 0x75),
+ QMP_PHY_INIT_CFG(QSERDES_V5_TX_LANE_MODE_4, 0x3f),
+ QMP_PHY_INIT_CFG(QSERDES_V5_TX_RES_CODE_LANE_OFFSET_TX, 0x1d),
+ QMP_PHY_INIT_CFG(QSERDES_V5_TX_RES_CODE_LANE_OFFSET_RX, 0x0c),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x1_pcie_rx_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_00_LOW, 0x7f),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_00_HIGH, 0xff),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_00_HIGH2, 0xbf),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_00_HIGH3, 0x3f),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_00_HIGH4, 0xd8),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_01_LOW, 0xdc),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_01_HIGH, 0xdc),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_01_HIGH2, 0x5c),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_01_HIGH3, 0x34),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_01_HIGH4, 0xa6),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_TX_ADAPT_POST_THRESH, 0xf0),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_10_HIGH3, 0x34),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_VGA_CAL_CNTRL2, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_GM_CAL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_UCDR_SB2_THRESH1, 0x08),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_UCDR_SB2_THRESH2, 0x08),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_UCDR_PI_CONTROLS, 0xf0),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_DFE_CTLE_POST_CAL_OFFSET, 0x38),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x1_pcie_pcs_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_REFGEN_REQ_CONFIG1, 0x05),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_RX_SIGDET_LVL, 0x77),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_RATE_SLEW_CNTRL1, 0x0b),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x1_pcie_pcs_misc_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_PCIE_OSC_DTCT_ACTIONS, 0x00),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_PCIE_INT_AUX_CLK_CONFIG1, 0x00),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_PCIE_EQ_CONFIG2, 0x0f),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_PCIE_ENDPOINT_REFCLK_DRIVE, 0xc1),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x2_pcie_tx_tbl[] = {
+ QMP_PHY_INIT_CFG_LANE(QSERDES_V5_TX_PI_QEC_CTRL, 0x02, 1),
+ QMP_PHY_INIT_CFG_LANE(QSERDES_V5_TX_PI_QEC_CTRL, 0x04, 2),
+ QMP_PHY_INIT_CFG(QSERDES_V5_TX_LANE_MODE_1, 0xd5),
+ QMP_PHY_INIT_CFG(QSERDES_V5_TX_LANE_MODE_4, 0x3f),
+ QMP_PHY_INIT_CFG(QSERDES_V5_TX_RES_CODE_LANE_OFFSET_TX, 0x11),
+ QMP_PHY_INIT_CFG(QSERDES_V5_TX_RES_CODE_LANE_OFFSET_RX, 0x0c),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x2_pcie_rx_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_00_LOW, 0x7f),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_00_HIGH, 0xff),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_00_HIGH2, 0x7f),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_00_HIGH3, 0x34),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_00_HIGH4, 0xd8),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_01_LOW, 0xdc),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_01_HIGH, 0xdc),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_01_HIGH2, 0x5c),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_01_HIGH3, 0x34),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_01_HIGH4, 0xa6),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_RX_MODE_10_HIGH3, 0x34),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_VGA_CAL_CNTRL2, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_GM_CAL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_UCDR_SB2_THRESH1, 0x08),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_UCDR_SB2_THRESH2, 0x08),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_UCDR_PI_CONTROLS, 0xf0),
+ QMP_PHY_INIT_CFG(QSERDES_V5_RX_DFE_CTLE_POST_CAL_OFFSET, 0x38),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x2_pcie_pcs_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_REFGEN_REQ_CONFIG1, 0x05),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_RX_SIGDET_LVL, 0x88),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_RATE_SLEW_CNTRL1, 0x0b),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_EQ_CONFIG3, 0x0f),
+};
+
+static const struct qmp_phy_init_tbl sc8280xp_qmp_gen3x2_pcie_pcs_misc_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_PCIE_POWER_STATE_CONFIG2, 0x1d),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_PCIE_POWER_STATE_CONFIG4, 0x07),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_PCIE_ENDPOINT_REFCLK_DRIVE, 0xc1),
+ QMP_PHY_INIT_CFG(QPHY_V5_PCS_PCIE_OSC_DTCT_ACTIONS, 0x00),
+};
+
static const struct qmp_phy_init_tbl sm8250_qmp_pcie_serdes_tbl[] = {
QMP_PHY_INIT_CFG(QSERDES_V4_COM_SYSCLK_EN_SEL, 0x08),
QMP_PHY_INIT_CFG(QSERDES_V4_COM_CLK_SELECT, 0x34),
};
static const struct qmp_phy_init_tbl sm8450_qmp_gen4x2_pcie_serdes_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_BIAS_EN_CLKBUFLR_EN, 0x14),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_IVCO, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP_EN, 0x46),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP_CFG, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_VCO_TUNE_MAP, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_HSCLK_SEL, 0x12),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_HSCLK_HS_SWITCH_SEL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CORECLK_DIV_MODE0, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CORECLK_DIV_MODE1, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CMN_MISC1, 0x88),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CMN_CONFIG, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CMN_MODE, 0x14),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_VCO_DC_LEVEL_CTRL, 0x0f),
+};
+
+static const struct qmp_phy_init_tbl sm8450_qmp_gen4x2_pcie_rc_serdes_tbl[] = {
QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_PER1, 0x31),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_PER2, 0x01),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_STEP_SIZE1_MODE0, 0xde),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_STEP_SIZE2_MODE0, 0x07),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_STEP_SIZE1_MODE1, 0x97),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_STEP_SIZE2_MODE1, 0x0c),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_BIAS_EN_CLKBUFLR_EN, 0x14),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_CLK_ENABLE1, 0x90),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_IVCO, 0x0f),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_CP_CTRL_MODE0, 0x06),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_CP_CTRL_MODE1, 0x06),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_RCTRL_MODE0, 0x16),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_CCTRL_MODE0, 0x36),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_CCTRL_MODE1, 0x36),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_SYSCLK_EN_SEL, 0x08),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP_EN, 0x46),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP_CFG, 0x04),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP1_MODE0, 0x0a),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP2_MODE0, 0x1a),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP1_MODE1, 0x14),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_DIV_FRAC_START1_MODE1, 0x55),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_DIV_FRAC_START2_MODE1, 0x55),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_DIV_FRAC_START3_MODE1, 0x05),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_VCO_TUNE_MAP, 0x02),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_CLK_SELECT, 0x34),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_HSCLK_SEL, 0x12),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_HSCLK_HS_SWITCH_SEL, 0x00),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_CORECLK_DIV_MODE0, 0x0a),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_CORECLK_DIV_MODE1, 0x04),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_CMN_MISC1, 0x88),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_CORE_CLK_EN, 0x20),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_CMN_CONFIG, 0x06),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_CMN_MODE, 0x14),
- QMP_PHY_INIT_CFG(QSERDES_V5_COM_VCO_DC_LEVEL_CTRL, 0x0f),
};
static const struct qmp_phy_init_tbl sm8450_qmp_gen4x2_pcie_tx_tbl[] = {
};
static const struct qmp_phy_init_tbl sm8450_qmp_gen4x2_pcie_pcs_tbl[] = {
- QMP_PHY_INIT_CFG(QPHY_V5_PCS_EQ_CONFIG2, 0x16),
- QMP_PHY_INIT_CFG(QPHY_V5_PCS_EQ_CONFIG3, 0x22),
- QMP_PHY_INIT_CFG(QPHY_V5_PCS_G3S2_PRE_GAIN, 0x2e),
- QMP_PHY_INIT_CFG(QPHY_V5_PCS_RX_SIGDET_LVL, 0x99),
+ QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_EQ_CONFIG4, 0x16),
+ QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_EQ_CONFIG5, 0x22),
+ QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_G3S2_PRE_GAIN, 0x2e),
+ QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_RX_SIGDET_LVL, 0x99),
};
static const struct qmp_phy_init_tbl sm8450_qmp_gen4x2_pcie_pcs_misc_tbl[] = {
- QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_PCIE_ENDPOINT_REFCLK_DRIVE, 0xc1),
- QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_PCIE_OSC_DTCT_ACTIONS, 0x00),
QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_PCIE_G4_EQ_CONFIG5, 0x02),
QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_PCIE_EQ_CONFIG1, 0x16),
QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_PCIE_RX_MARGINING_CONFIG3, 0x28),
QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_PCIE_G4_PRE_GAIN, 0x2e),
};
+static const struct qmp_phy_init_tbl sm8450_qmp_gen4x2_pcie_rc_pcs_misc_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_PCIE_ENDPOINT_REFCLK_DRIVE, 0xc1),
+ QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_PCIE_OSC_DTCT_ACTIONS, 0x00),
+ QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_PCIE_PRESET_P10_POST, 0x00),
+};
+
+static const struct qmp_phy_init_tbl sm8450_qmp_gen4x2_pcie_ep_serdes_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_BG_TIMER, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SYS_CLK_CTRL, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CP_CTRL_MODE0, 0x27),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CP_CTRL_MODE1, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_RCTRL_MODE0, 0x17),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_RCTRL_MODE1, 0x19),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_CCTRL_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_PLL_CCTRL_MODE1, 0x03),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_SYSCLK_EN_SEL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP1_MODE0, 0xff),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP2_MODE0, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP1_MODE1, 0xff),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_LOCK_CMP2_MODE1, 0x09),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_DEC_START_MODE0, 0x19),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_DEC_START_MODE1, 0x28),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_INTEGLOOP_GAIN0_MODE0, 0xfb),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_INTEGLOOP_GAIN1_MODE0, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_INTEGLOOP_GAIN0_MODE1, 0xfb),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_INTEGLOOP_GAIN1_MODE1, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V5_COM_CORE_CLK_EN, 0x60),
+};
+
+static const struct qmp_phy_init_tbl sm8450_qmp_gen4x2_pcie_ep_pcs_misc_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V5_20_PCS_PCIE_OSC_DTCT_MODE2_CONFIG5, 0x08),
+};
+
+struct qmp_pcie_offsets {
+ u16 serdes;
+ u16 pcs;
+ u16 pcs_misc;
+ u16 tx;
+ u16 rx;
+ u16 tx2;
+ u16 rx2;
+};
+
+struct qmp_phy_cfg_tbls {
+ const struct qmp_phy_init_tbl *serdes;
+ int serdes_num;
+ const struct qmp_phy_init_tbl *tx;
+ int tx_num;
+ const struct qmp_phy_init_tbl *rx;
+ int rx_num;
+ const struct qmp_phy_init_tbl *pcs;
+ int pcs_num;
+ const struct qmp_phy_init_tbl *pcs_misc;
+ int pcs_misc_num;
+};
+
/* struct qmp_phy_cfg - per-PHY initialization config */
struct qmp_phy_cfg {
int lanes;
- /* Init sequence for PHY blocks - serdes, tx, rx, pcs */
- const struct qmp_phy_init_tbl *serdes_tbl;
- int serdes_tbl_num;
- const struct qmp_phy_init_tbl *serdes_tbl_sec;
- int serdes_tbl_num_sec;
- const struct qmp_phy_init_tbl *tx_tbl;
- int tx_tbl_num;
- const struct qmp_phy_init_tbl *tx_tbl_sec;
- int tx_tbl_num_sec;
- const struct qmp_phy_init_tbl *rx_tbl;
- int rx_tbl_num;
- const struct qmp_phy_init_tbl *rx_tbl_sec;
- int rx_tbl_num_sec;
- const struct qmp_phy_init_tbl *pcs_tbl;
- int pcs_tbl_num;
- const struct qmp_phy_init_tbl *pcs_tbl_sec;
- int pcs_tbl_num_sec;
- const struct qmp_phy_init_tbl *pcs_misc_tbl;
- int pcs_misc_tbl_num;
- const struct qmp_phy_init_tbl *pcs_misc_tbl_sec;
- int pcs_misc_tbl_num_sec;
+ const struct qmp_pcie_offsets *offsets;
+
+ /* Main init sequence for PHY blocks - serdes, tx, rx, pcs */
+ const struct qmp_phy_cfg_tbls tbls;
+ /*
+ * Additional init sequences for PHY blocks, providing additional
+ * register programming. They are used for providing separate sequences
+ * for the Root Complex and End Point use cases.
+ *
+ * If EP mode is not supported, both tables can be left unset.
+ */
+ const struct qmp_phy_cfg_tbls *tbls_rc;
+ const struct qmp_phy_cfg_tbls *tbls_ep;
+
+ const struct qmp_phy_init_tbl *serdes_4ln_tbl;
+ int serdes_4ln_num;
/* clock ids to be requested */
const char * const *clk_list;
/* array of registers with different offsets */
const unsigned int *regs;
- unsigned int start_ctrl;
unsigned int pwrdn_ctrl;
/* bit offset of PHYSTATUS in QPHY_PCS_STATUS register */
unsigned int phy_status;
- /* true, if PHY needs delay after POWER_DOWN */
- bool has_pwrdn_delay;
- /* power_down delay in usec */
- int pwrdn_delay_min;
- int pwrdn_delay_max;
+ bool skip_start_delay;
/* QMP PHY pipe clock interface rate */
unsigned long pipe_clock_rate;
};
-/**
- * struct qmp_phy - per-lane phy descriptor
- *
- * @phy: generic phy
- * @cfg: phy specific configuration
- * @serdes: iomapped memory space for phy's serdes (i.e. PLL)
- * @tx: iomapped memory space for lane's tx
- * @rx: iomapped memory space for lane's rx
- * @pcs: iomapped memory space for lane's pcs
- * @tx2: iomapped memory space for second lane's tx (in dual lane PHYs)
- * @rx2: iomapped memory space for second lane's rx (in dual lane PHYs)
- * @pcs_misc: iomapped memory space for lane's pcs_misc
- * @pipe_clk: pipe clock
- * @qmp: QMP phy to which this lane belongs
- */
-struct qmp_phy {
- struct phy *phy;
+struct qmp_pcie {
+ struct device *dev;
+
const struct qmp_phy_cfg *cfg;
+ bool tcsr_4ln_config;
+
void __iomem *serdes;
+ void __iomem *pcs;
+ void __iomem *pcs_misc;
void __iomem *tx;
void __iomem *rx;
- void __iomem *pcs;
void __iomem *tx2;
void __iomem *rx2;
- void __iomem *pcs_misc;
- struct clk *pipe_clk;
- struct qcom_qmp *qmp;
-};
-/**
- * struct qcom_qmp - structure holding QMP phy block attributes
- *
- * @dev: device
- *
- * @clks: array of clocks required by phy
- * @resets: array of resets required by phy
- * @vregs: regulator supplies bulk data
- *
- * @phys: array of per-lane phy descriptors
- */
-struct qcom_qmp {
- struct device *dev;
+ void __iomem *port_b;
struct clk_bulk_data *clks;
+ struct clk_bulk_data pipe_clks[2];
+ int num_pipe_clks;
+
struct reset_control_bulk_data *resets;
struct regulator_bulk_data *vregs;
- struct qmp_phy **phys;
+ struct phy *phy;
+ int mode;
+
+ struct clk_fixed_rate pipe_clk_fixed;
};
static inline void qphy_setbits(void __iomem *base, u32 offset, u32 val)
}
/* list of clocks required by phy */
+static const char * const ipq8074_pciephy_clk_l[] = {
+ "aux", "cfg_ahb",
+};
+
static const char * const msm8996_phy_clk_l[] = {
"aux", "cfg_ahb", "ref",
};
+static const char * const sc8280xp_pciephy_clk_l[] = {
+ "aux", "cfg_ahb", "ref", "rchng",
+};
static const char * const sdm845_pciephy_clk_l[] = {
"aux", "cfg_ahb", "ref", "refgen",
"vdda-phy", "vdda-pll",
};
-static const char * const ipq8074_pciephy_clk_l[] = {
- "aux", "cfg_ahb",
-};
-
/* list of resets */
static const char * const ipq8074_pciephy_reset_l[] = {
"phy", "common",
"phy",
};
+static const struct qmp_pcie_offsets qmp_pcie_offsets_v5 = {
+ .serdes = 0,
+ .pcs = 0x0200,
+ .pcs_misc = 0x0600,
+ .tx = 0x0e00,
+ .rx = 0x1000,
+ .tx2 = 0x1600,
+ .rx2 = 0x1800,
+};
+
static const struct qmp_phy_cfg ipq8074_pciephy_cfg = {
.lanes = 1,
- .serdes_tbl = ipq8074_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(ipq8074_pcie_serdes_tbl),
- .tx_tbl = ipq8074_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(ipq8074_pcie_tx_tbl),
- .rx_tbl = ipq8074_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(ipq8074_pcie_rx_tbl),
- .pcs_tbl = ipq8074_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(ipq8074_pcie_pcs_tbl),
+ .tbls = {
+ .serdes = ipq8074_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(ipq8074_pcie_serdes_tbl),
+ .tx = ipq8074_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(ipq8074_pcie_tx_tbl),
+ .rx = ipq8074_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(ipq8074_pcie_rx_tbl),
+ .pcs = ipq8074_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(ipq8074_pcie_pcs_tbl),
+ },
.clk_list = ipq8074_pciephy_clk_l,
.num_clks = ARRAY_SIZE(ipq8074_pciephy_clk_l),
.reset_list = ipq8074_pciephy_reset_l,
.num_vregs = 0,
.regs = pciephy_regs_layout,
- .start_ctrl = SERDES_START | PCS_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
.phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
};
static const struct qmp_phy_cfg ipq8074_pciephy_gen3_cfg = {
.lanes = 1,
- .serdes_tbl = ipq8074_pcie_gen3_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(ipq8074_pcie_gen3_serdes_tbl),
- .tx_tbl = ipq8074_pcie_gen3_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(ipq8074_pcie_gen3_tx_tbl),
- .rx_tbl = ipq8074_pcie_gen3_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(ipq8074_pcie_gen3_rx_tbl),
- .pcs_tbl = ipq8074_pcie_gen3_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(ipq8074_pcie_gen3_pcs_tbl),
+ .tbls = {
+ .serdes = ipq8074_pcie_gen3_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(ipq8074_pcie_gen3_serdes_tbl),
+ .tx = ipq8074_pcie_gen3_tx_tbl,
+ .tx_num = ARRAY_SIZE(ipq8074_pcie_gen3_tx_tbl),
+ .rx = ipq8074_pcie_gen3_rx_tbl,
+ .rx_num = ARRAY_SIZE(ipq8074_pcie_gen3_rx_tbl),
+ .pcs = ipq8074_pcie_gen3_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(ipq8074_pcie_gen3_pcs_tbl),
+ .pcs_misc = ipq8074_pcie_gen3_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(ipq8074_pcie_gen3_pcs_misc_tbl),
+ },
.clk_list = ipq8074_pciephy_clk_l,
.num_clks = ARRAY_SIZE(ipq8074_pciephy_clk_l),
.reset_list = ipq8074_pciephy_reset_l,
.num_vregs = 0,
.regs = ipq_pciephy_gen3_regs_layout,
- .start_ctrl = SERDES_START | PCS_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
+ .phy_status = PHYSTATUS,
.pipe_clock_rate = 250000000,
};
static const struct qmp_phy_cfg ipq6018_pciephy_cfg = {
.lanes = 1,
- .serdes_tbl = ipq6018_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(ipq6018_pcie_serdes_tbl),
- .tx_tbl = ipq6018_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(ipq6018_pcie_tx_tbl),
- .rx_tbl = ipq6018_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(ipq6018_pcie_rx_tbl),
- .pcs_tbl = ipq6018_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(ipq6018_pcie_pcs_tbl),
- .pcs_misc_tbl = ipq6018_pcie_pcs_misc_tbl,
- .pcs_misc_tbl_num = ARRAY_SIZE(ipq6018_pcie_pcs_misc_tbl),
+ .tbls = {
+ .serdes = ipq6018_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(ipq6018_pcie_serdes_tbl),
+ .tx = ipq6018_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(ipq6018_pcie_tx_tbl),
+ .rx = ipq6018_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(ipq6018_pcie_rx_tbl),
+ .pcs = ipq6018_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(ipq6018_pcie_pcs_tbl),
+ .pcs_misc = ipq6018_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(ipq6018_pcie_pcs_misc_tbl),
+ },
.clk_list = ipq8074_pciephy_clk_l,
.num_clks = ARRAY_SIZE(ipq8074_pciephy_clk_l),
.reset_list = ipq8074_pciephy_reset_l,
.num_vregs = 0,
.regs = ipq_pciephy_gen3_regs_layout,
- .start_ctrl = SERDES_START | PCS_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
+ .phy_status = PHYSTATUS,
};
static const struct qmp_phy_cfg sdm845_qmp_pciephy_cfg = {
.lanes = 1,
- .serdes_tbl = sdm845_qmp_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(sdm845_qmp_pcie_serdes_tbl),
- .tx_tbl = sdm845_qmp_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(sdm845_qmp_pcie_tx_tbl),
- .rx_tbl = sdm845_qmp_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(sdm845_qmp_pcie_rx_tbl),
- .pcs_tbl = sdm845_qmp_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(sdm845_qmp_pcie_pcs_tbl),
- .pcs_misc_tbl = sdm845_qmp_pcie_pcs_misc_tbl,
- .pcs_misc_tbl_num = ARRAY_SIZE(sdm845_qmp_pcie_pcs_misc_tbl),
+ .tbls = {
+ .serdes = sdm845_qmp_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sdm845_qmp_pcie_serdes_tbl),
+ .tx = sdm845_qmp_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sdm845_qmp_pcie_tx_tbl),
+ .rx = sdm845_qmp_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sdm845_qmp_pcie_rx_tbl),
+ .pcs = sdm845_qmp_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sdm845_qmp_pcie_pcs_tbl),
+ .pcs_misc = sdm845_qmp_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sdm845_qmp_pcie_pcs_misc_tbl),
+ },
.clk_list = sdm845_pciephy_clk_l,
.num_clks = ARRAY_SIZE(sdm845_pciephy_clk_l),
.reset_list = sdm845_pciephy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sdm845_qmp_pciephy_regs_layout,
- .start_ctrl = PCS_START | SERDES_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
.phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
};
static const struct qmp_phy_cfg sdm845_qhp_pciephy_cfg = {
.lanes = 1,
- .serdes_tbl = sdm845_qhp_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(sdm845_qhp_pcie_serdes_tbl),
- .tx_tbl = sdm845_qhp_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(sdm845_qhp_pcie_tx_tbl),
- .rx_tbl = sdm845_qhp_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(sdm845_qhp_pcie_rx_tbl),
- .pcs_tbl = sdm845_qhp_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(sdm845_qhp_pcie_pcs_tbl),
+ .tbls = {
+ .serdes = sdm845_qhp_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sdm845_qhp_pcie_serdes_tbl),
+ .tx = sdm845_qhp_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sdm845_qhp_pcie_tx_tbl),
+ .rx = sdm845_qhp_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sdm845_qhp_pcie_rx_tbl),
+ .pcs = sdm845_qhp_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sdm845_qhp_pcie_pcs_tbl),
+ },
.clk_list = sdm845_pciephy_clk_l,
.num_clks = ARRAY_SIZE(sdm845_pciephy_clk_l),
.reset_list = sdm845_pciephy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sdm845_qhp_pciephy_regs_layout,
- .start_ctrl = PCS_START | SERDES_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
.phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
};
static const struct qmp_phy_cfg sm8250_qmp_gen3x1_pciephy_cfg = {
.lanes = 1,
- .serdes_tbl = sm8250_qmp_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(sm8250_qmp_pcie_serdes_tbl),
- .serdes_tbl_sec = sm8250_qmp_gen3x1_pcie_serdes_tbl,
- .serdes_tbl_num_sec = ARRAY_SIZE(sm8250_qmp_gen3x1_pcie_serdes_tbl),
- .tx_tbl = sm8250_qmp_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(sm8250_qmp_pcie_tx_tbl),
- .rx_tbl = sm8250_qmp_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(sm8250_qmp_pcie_rx_tbl),
- .rx_tbl_sec = sm8250_qmp_gen3x1_pcie_rx_tbl,
- .rx_tbl_num_sec = ARRAY_SIZE(sm8250_qmp_gen3x1_pcie_rx_tbl),
- .pcs_tbl = sm8250_qmp_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(sm8250_qmp_pcie_pcs_tbl),
- .pcs_tbl_sec = sm8250_qmp_gen3x1_pcie_pcs_tbl,
- .pcs_tbl_num_sec = ARRAY_SIZE(sm8250_qmp_gen3x1_pcie_pcs_tbl),
- .pcs_misc_tbl = sm8250_qmp_pcie_pcs_misc_tbl,
- .pcs_misc_tbl_num = ARRAY_SIZE(sm8250_qmp_pcie_pcs_misc_tbl),
- .pcs_misc_tbl_sec = sm8250_qmp_gen3x1_pcie_pcs_misc_tbl,
- .pcs_misc_tbl_num_sec = ARRAY_SIZE(sm8250_qmp_gen3x1_pcie_pcs_misc_tbl),
+ .tbls = {
+ .serdes = sm8250_qmp_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sm8250_qmp_pcie_serdes_tbl),
+ .tx = sm8250_qmp_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sm8250_qmp_pcie_tx_tbl),
+ .rx = sm8250_qmp_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sm8250_qmp_pcie_rx_tbl),
+ .pcs = sm8250_qmp_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sm8250_qmp_pcie_pcs_tbl),
+ .pcs_misc = sm8250_qmp_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sm8250_qmp_pcie_pcs_misc_tbl),
+ },
+ .tbls_rc = &(const struct qmp_phy_cfg_tbls) {
+ .serdes = sm8250_qmp_gen3x1_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sm8250_qmp_gen3x1_pcie_serdes_tbl),
+ .rx = sm8250_qmp_gen3x1_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sm8250_qmp_gen3x1_pcie_rx_tbl),
+ .pcs = sm8250_qmp_gen3x1_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sm8250_qmp_gen3x1_pcie_pcs_tbl),
+ .pcs_misc = sm8250_qmp_gen3x1_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sm8250_qmp_gen3x1_pcie_pcs_misc_tbl),
+ },
.clk_list = sdm845_pciephy_clk_l,
.num_clks = ARRAY_SIZE(sdm845_pciephy_clk_l),
.reset_list = sdm845_pciephy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sm8250_pcie_regs_layout,
- .start_ctrl = PCS_START | SERDES_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
.phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
};
static const struct qmp_phy_cfg sm8250_qmp_gen3x2_pciephy_cfg = {
.lanes = 2,
- .serdes_tbl = sm8250_qmp_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(sm8250_qmp_pcie_serdes_tbl),
- .tx_tbl = sm8250_qmp_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(sm8250_qmp_pcie_tx_tbl),
- .tx_tbl_sec = sm8250_qmp_gen3x2_pcie_tx_tbl,
- .tx_tbl_num_sec = ARRAY_SIZE(sm8250_qmp_gen3x2_pcie_tx_tbl),
- .rx_tbl = sm8250_qmp_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(sm8250_qmp_pcie_rx_tbl),
- .rx_tbl_sec = sm8250_qmp_gen3x2_pcie_rx_tbl,
- .rx_tbl_num_sec = ARRAY_SIZE(sm8250_qmp_gen3x2_pcie_rx_tbl),
- .pcs_tbl = sm8250_qmp_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(sm8250_qmp_pcie_pcs_tbl),
- .pcs_tbl_sec = sm8250_qmp_gen3x2_pcie_pcs_tbl,
- .pcs_tbl_num_sec = ARRAY_SIZE(sm8250_qmp_gen3x2_pcie_pcs_tbl),
- .pcs_misc_tbl = sm8250_qmp_pcie_pcs_misc_tbl,
- .pcs_misc_tbl_num = ARRAY_SIZE(sm8250_qmp_pcie_pcs_misc_tbl),
- .pcs_misc_tbl_sec = sm8250_qmp_gen3x2_pcie_pcs_misc_tbl,
- .pcs_misc_tbl_num_sec = ARRAY_SIZE(sm8250_qmp_gen3x2_pcie_pcs_misc_tbl),
+ .tbls = {
+ .serdes = sm8250_qmp_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sm8250_qmp_pcie_serdes_tbl),
+ .tx = sm8250_qmp_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sm8250_qmp_pcie_tx_tbl),
+ .rx = sm8250_qmp_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sm8250_qmp_pcie_rx_tbl),
+ .pcs = sm8250_qmp_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sm8250_qmp_pcie_pcs_tbl),
+ .pcs_misc = sm8250_qmp_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sm8250_qmp_pcie_pcs_misc_tbl),
+ },
+ .tbls_rc = &(const struct qmp_phy_cfg_tbls) {
+ .tx = sm8250_qmp_gen3x2_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sm8250_qmp_gen3x2_pcie_tx_tbl),
+ .rx = sm8250_qmp_gen3x2_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sm8250_qmp_gen3x2_pcie_rx_tbl),
+ .pcs = sm8250_qmp_gen3x2_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sm8250_qmp_gen3x2_pcie_pcs_tbl),
+ .pcs_misc = sm8250_qmp_gen3x2_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sm8250_qmp_gen3x2_pcie_pcs_misc_tbl),
+ },
.clk_list = sdm845_pciephy_clk_l,
.num_clks = ARRAY_SIZE(sdm845_pciephy_clk_l),
.reset_list = sdm845_pciephy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sm8250_pcie_regs_layout,
- .start_ctrl = PCS_START | SERDES_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
.phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
};
static const struct qmp_phy_cfg msm8998_pciephy_cfg = {
.lanes = 1,
- .serdes_tbl = msm8998_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(msm8998_pcie_serdes_tbl),
- .tx_tbl = msm8998_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(msm8998_pcie_tx_tbl),
- .rx_tbl = msm8998_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(msm8998_pcie_rx_tbl),
- .pcs_tbl = msm8998_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(msm8998_pcie_pcs_tbl),
+ .tbls = {
+ .serdes = msm8998_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(msm8998_pcie_serdes_tbl),
+ .tx = msm8998_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(msm8998_pcie_tx_tbl),
+ .rx = msm8998_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(msm8998_pcie_rx_tbl),
+ .pcs = msm8998_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(msm8998_pcie_pcs_tbl),
+ },
.clk_list = msm8996_phy_clk_l,
.num_clks = ARRAY_SIZE(msm8996_phy_clk_l),
.reset_list = ipq8074_pciephy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = pciephy_regs_layout,
- .start_ctrl = SERDES_START | PCS_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
.phy_status = PHYSTATUS,
+
+ .skip_start_delay = true,
};
static const struct qmp_phy_cfg sc8180x_pciephy_cfg = {
.lanes = 1,
- .serdes_tbl = sc8180x_qmp_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(sc8180x_qmp_pcie_serdes_tbl),
- .tx_tbl = sc8180x_qmp_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(sc8180x_qmp_pcie_tx_tbl),
- .rx_tbl = sc8180x_qmp_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(sc8180x_qmp_pcie_rx_tbl),
- .pcs_tbl = sc8180x_qmp_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(sc8180x_qmp_pcie_pcs_tbl),
- .pcs_misc_tbl = sc8180x_qmp_pcie_pcs_misc_tbl,
- .pcs_misc_tbl_num = ARRAY_SIZE(sc8180x_qmp_pcie_pcs_misc_tbl),
+ .tbls = {
+ .serdes = sc8180x_qmp_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sc8180x_qmp_pcie_serdes_tbl),
+ .tx = sc8180x_qmp_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sc8180x_qmp_pcie_tx_tbl),
+ .rx = sc8180x_qmp_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sc8180x_qmp_pcie_rx_tbl),
+ .pcs = sc8180x_qmp_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sc8180x_qmp_pcie_pcs_tbl),
+ .pcs_misc = sc8180x_qmp_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sc8180x_qmp_pcie_pcs_misc_tbl),
+ },
.clk_list = sdm845_pciephy_clk_l,
.num_clks = ARRAY_SIZE(sdm845_pciephy_clk_l),
.reset_list = sdm845_pciephy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sm8250_pcie_regs_layout,
- .start_ctrl = PCS_START | SERDES_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
+ .phy_status = PHYSTATUS,
+};
+
+static const struct qmp_phy_cfg sc8280xp_qmp_gen3x1_pciephy_cfg = {
+ .lanes = 1,
+
+ .offsets = &qmp_pcie_offsets_v5,
+
+ .tbls = {
+ .serdes = sc8280xp_qmp_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sc8280xp_qmp_pcie_serdes_tbl),
+ .tx = sc8280xp_qmp_gen3x1_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sc8280xp_qmp_gen3x1_pcie_tx_tbl),
+ .rx = sc8280xp_qmp_gen3x1_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sc8280xp_qmp_gen3x1_pcie_rx_tbl),
+ .pcs = sc8280xp_qmp_gen3x1_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sc8280xp_qmp_gen3x1_pcie_pcs_tbl),
+ .pcs_misc = sc8280xp_qmp_gen3x1_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sc8280xp_qmp_gen3x1_pcie_pcs_misc_tbl),
+ },
+
+ .tbls_rc = &(const struct qmp_phy_cfg_tbls) {
+ .serdes = sc8280xp_qmp_gen3x1_pcie_rc_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sc8280xp_qmp_gen3x1_pcie_rc_serdes_tbl),
+ },
+
+ .clk_list = sc8280xp_pciephy_clk_l,
+ .num_clks = ARRAY_SIZE(sc8280xp_pciephy_clk_l),
+ .reset_list = sdm845_pciephy_reset_l,
+ .num_resets = ARRAY_SIZE(sdm845_pciephy_reset_l),
+ .vreg_list = qmp_phy_vreg_l,
+ .num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
+ .regs = sm8250_pcie_regs_layout,
+
+ .pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
+ .phy_status = PHYSTATUS,
+};
+
+static const struct qmp_phy_cfg sc8280xp_qmp_gen3x2_pciephy_cfg = {
+ .lanes = 2,
+
+ .offsets = &qmp_pcie_offsets_v5,
+
+ .tbls = {
+ .serdes = sc8280xp_qmp_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sc8280xp_qmp_pcie_serdes_tbl),
+ .tx = sc8280xp_qmp_gen3x2_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sc8280xp_qmp_gen3x2_pcie_tx_tbl),
+ .rx = sc8280xp_qmp_gen3x2_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sc8280xp_qmp_gen3x2_pcie_rx_tbl),
+ .pcs = sc8280xp_qmp_gen3x2_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sc8280xp_qmp_gen3x2_pcie_pcs_tbl),
+ .pcs_misc = sc8280xp_qmp_gen3x2_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sc8280xp_qmp_gen3x2_pcie_pcs_misc_tbl),
+ },
+
+ .tbls_rc = &(const struct qmp_phy_cfg_tbls) {
+ .serdes = sc8280xp_qmp_gen3x2_pcie_rc_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sc8280xp_qmp_gen3x2_pcie_rc_serdes_tbl),
+ },
+
+ .clk_list = sc8280xp_pciephy_clk_l,
+ .num_clks = ARRAY_SIZE(sc8280xp_pciephy_clk_l),
+ .reset_list = sdm845_pciephy_reset_l,
+ .num_resets = ARRAY_SIZE(sdm845_pciephy_reset_l),
+ .vreg_list = qmp_phy_vreg_l,
+ .num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
+ .regs = sm8250_pcie_regs_layout,
+
+ .pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
+ .phy_status = PHYSTATUS,
+};
+
+static const struct qmp_phy_cfg sc8280xp_qmp_gen3x4_pciephy_cfg = {
+ .lanes = 4,
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
+ .offsets = &qmp_pcie_offsets_v5,
+
+ .tbls = {
+ .serdes = sc8280xp_qmp_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sc8280xp_qmp_pcie_serdes_tbl),
+ .tx = sc8280xp_qmp_gen3x2_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sc8280xp_qmp_gen3x2_pcie_tx_tbl),
+ .rx = sc8280xp_qmp_gen3x2_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sc8280xp_qmp_gen3x2_pcie_rx_tbl),
+ .pcs = sc8280xp_qmp_gen3x2_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sc8280xp_qmp_gen3x2_pcie_pcs_tbl),
+ .pcs_misc = sc8280xp_qmp_gen3x2_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sc8280xp_qmp_gen3x2_pcie_pcs_misc_tbl),
+ },
+
+ .tbls_rc = &(const struct qmp_phy_cfg_tbls) {
+ .serdes = sc8280xp_qmp_gen3x2_pcie_rc_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sc8280xp_qmp_gen3x2_pcie_rc_serdes_tbl),
+ },
+
+ .serdes_4ln_tbl = sc8280xp_qmp_gen3x4_pcie_serdes_4ln_tbl,
+ .serdes_4ln_num = ARRAY_SIZE(sc8280xp_qmp_gen3x4_pcie_serdes_4ln_tbl),
+
+ .clk_list = sc8280xp_pciephy_clk_l,
+ .num_clks = ARRAY_SIZE(sc8280xp_pciephy_clk_l),
+ .reset_list = sdm845_pciephy_reset_l,
+ .num_resets = ARRAY_SIZE(sdm845_pciephy_reset_l),
+ .vreg_list = qmp_phy_vreg_l,
+ .num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
+ .regs = sm8250_pcie_regs_layout,
+
+ .pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
+ .phy_status = PHYSTATUS,
};
static const struct qmp_phy_cfg sdx55_qmp_pciephy_cfg = {
.lanes = 2,
- .serdes_tbl = sdx55_qmp_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(sdx55_qmp_pcie_serdes_tbl),
- .tx_tbl = sdx55_qmp_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(sdx55_qmp_pcie_tx_tbl),
- .rx_tbl = sdx55_qmp_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(sdx55_qmp_pcie_rx_tbl),
- .pcs_tbl = sdx55_qmp_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(sdx55_qmp_pcie_pcs_tbl),
- .pcs_misc_tbl = sdx55_qmp_pcie_pcs_misc_tbl,
- .pcs_misc_tbl_num = ARRAY_SIZE(sdx55_qmp_pcie_pcs_misc_tbl),
+ .tbls = {
+ .serdes = sdx55_qmp_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sdx55_qmp_pcie_serdes_tbl),
+ .tx = sdx55_qmp_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sdx55_qmp_pcie_tx_tbl),
+ .rx = sdx55_qmp_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sdx55_qmp_pcie_rx_tbl),
+ .pcs = sdx55_qmp_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sdx55_qmp_pcie_pcs_tbl),
+ .pcs_misc = sdx55_qmp_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sdx55_qmp_pcie_pcs_misc_tbl),
+ },
.clk_list = sdm845_pciephy_clk_l,
.num_clks = ARRAY_SIZE(sdm845_pciephy_clk_l),
.reset_list = sdm845_pciephy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sm8250_pcie_regs_layout,
- .start_ctrl = PCS_START | SERDES_START,
.pwrdn_ctrl = SW_PWRDN,
.phy_status = PHYSTATUS_4_20,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
};
static const struct qmp_phy_cfg sm8450_qmp_gen3x1_pciephy_cfg = {
.lanes = 1,
- .serdes_tbl = sm8450_qmp_gen3x1_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(sm8450_qmp_gen3x1_pcie_serdes_tbl),
- .tx_tbl = sm8450_qmp_gen3x1_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(sm8450_qmp_gen3x1_pcie_tx_tbl),
- .rx_tbl = sm8450_qmp_gen3x1_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(sm8450_qmp_gen3x1_pcie_rx_tbl),
- .pcs_tbl = sm8450_qmp_gen3x1_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(sm8450_qmp_gen3x1_pcie_pcs_tbl),
- .pcs_misc_tbl = sm8450_qmp_gen3x1_pcie_pcs_misc_tbl,
- .pcs_misc_tbl_num = ARRAY_SIZE(sm8450_qmp_gen3x1_pcie_pcs_misc_tbl),
+ .tbls = {
+ .serdes = sm8450_qmp_gen3x1_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sm8450_qmp_gen3x1_pcie_serdes_tbl),
+ .tx = sm8450_qmp_gen3x1_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sm8450_qmp_gen3x1_pcie_tx_tbl),
+ .rx = sm8450_qmp_gen3x1_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sm8450_qmp_gen3x1_pcie_rx_tbl),
+ .pcs = sm8450_qmp_gen3x1_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sm8450_qmp_gen3x1_pcie_pcs_tbl),
+ .pcs_misc = sm8450_qmp_gen3x1_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sm8450_qmp_gen3x1_pcie_pcs_misc_tbl),
+ },
.clk_list = sdm845_pciephy_clk_l,
.num_clks = ARRAY_SIZE(sdm845_pciephy_clk_l),
.reset_list = sdm845_pciephy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sm8250_pcie_regs_layout,
- .start_ctrl = SERDES_START | PCS_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
.phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
};
static const struct qmp_phy_cfg sm8450_qmp_gen4x2_pciephy_cfg = {
.lanes = 2,
- .serdes_tbl = sm8450_qmp_gen4x2_pcie_serdes_tbl,
- .serdes_tbl_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_serdes_tbl),
- .tx_tbl = sm8450_qmp_gen4x2_pcie_tx_tbl,
- .tx_tbl_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_tx_tbl),
- .rx_tbl = sm8450_qmp_gen4x2_pcie_rx_tbl,
- .rx_tbl_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_rx_tbl),
- .pcs_tbl = sm8450_qmp_gen4x2_pcie_pcs_tbl,
- .pcs_tbl_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_pcs_tbl),
- .pcs_misc_tbl = sm8450_qmp_gen4x2_pcie_pcs_misc_tbl,
- .pcs_misc_tbl_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_pcs_misc_tbl),
+ .tbls = {
+ .serdes = sm8450_qmp_gen4x2_pcie_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_serdes_tbl),
+ .tx = sm8450_qmp_gen4x2_pcie_tx_tbl,
+ .tx_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_tx_tbl),
+ .rx = sm8450_qmp_gen4x2_pcie_rx_tbl,
+ .rx_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_rx_tbl),
+ .pcs = sm8450_qmp_gen4x2_pcie_pcs_tbl,
+ .pcs_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_pcs_tbl),
+ .pcs_misc = sm8450_qmp_gen4x2_pcie_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_pcs_misc_tbl),
+ },
+
+ .tbls_rc = &(const struct qmp_phy_cfg_tbls) {
+ .serdes = sm8450_qmp_gen4x2_pcie_rc_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_rc_serdes_tbl),
+ .pcs_misc = sm8450_qmp_gen4x2_pcie_rc_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_rc_pcs_misc_tbl),
+ },
+
+ .tbls_ep = &(const struct qmp_phy_cfg_tbls) {
+ .serdes = sm8450_qmp_gen4x2_pcie_ep_serdes_tbl,
+ .serdes_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_ep_serdes_tbl),
+ .pcs_misc = sm8450_qmp_gen4x2_pcie_ep_pcs_misc_tbl,
+ .pcs_misc_num = ARRAY_SIZE(sm8450_qmp_gen4x2_pcie_ep_pcs_misc_tbl),
+ },
+
.clk_list = sdm845_pciephy_clk_l,
.num_clks = ARRAY_SIZE(sdm845_pciephy_clk_l),
.reset_list = sdm845_pciephy_reset_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sm8250_pcie_regs_layout,
- .start_ctrl = SERDES_START | PCS_START,
.pwrdn_ctrl = SW_PWRDN | REFCLK_DRV_DSBL,
.phy_status = PHYSTATUS_4_20,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = 995, /* us */
- .pwrdn_delay_max = 1005, /* us */
};
static void qmp_pcie_configure_lane(void __iomem *base,
- const unsigned int *regs,
const struct qmp_phy_init_tbl tbl[],
int num,
u8 lane_mask)
if (!(t->lane_mask & lane_mask))
continue;
- if (t->in_layout)
- writel(t->val, base + regs[t->offset]);
- else
- writel(t->val, base + t->offset);
+ writel(t->val, base + t->offset);
}
}
static void qmp_pcie_configure(void __iomem *base,
- const unsigned int *regs,
const struct qmp_phy_init_tbl tbl[],
int num)
{
- qmp_pcie_configure_lane(base, regs, tbl, num, 0xff);
+ qmp_pcie_configure_lane(base, tbl, num, 0xff);
}
-static int qmp_pcie_serdes_init(struct qmp_phy *qphy)
+static void qmp_pcie_init_port_b(struct qmp_pcie *qmp, const struct qmp_phy_cfg_tbls *tbls)
{
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *serdes = qphy->serdes;
- const struct qmp_phy_init_tbl *serdes_tbl = cfg->serdes_tbl;
- int serdes_tbl_num = cfg->serdes_tbl_num;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ const struct qmp_pcie_offsets *offs = cfg->offsets;
+ void __iomem *tx3, *rx3, *tx4, *rx4;
- qmp_pcie_configure(serdes, cfg->regs, serdes_tbl, serdes_tbl_num);
- qmp_pcie_configure(serdes, cfg->regs, cfg->serdes_tbl_sec, cfg->serdes_tbl_num_sec);
+ tx3 = qmp->port_b + offs->tx;
+ rx3 = qmp->port_b + offs->rx;
+ tx4 = qmp->port_b + offs->tx2;
+ rx4 = qmp->port_b + offs->rx2;
- return 0;
+ qmp_pcie_configure_lane(tx3, tbls->tx, tbls->tx_num, 1);
+ qmp_pcie_configure_lane(rx3, tbls->rx, tbls->rx_num, 1);
+
+ qmp_pcie_configure_lane(tx4, tbls->tx, tbls->tx_num, 2);
+ qmp_pcie_configure_lane(rx4, tbls->rx, tbls->rx_num, 2);
+}
+
+static void qmp_pcie_init_registers(struct qmp_pcie *qmp, const struct qmp_phy_cfg_tbls *tbls)
+{
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *serdes = qmp->serdes;
+ void __iomem *tx = qmp->tx;
+ void __iomem *rx = qmp->rx;
+ void __iomem *tx2 = qmp->tx2;
+ void __iomem *rx2 = qmp->rx2;
+ void __iomem *pcs = qmp->pcs;
+ void __iomem *pcs_misc = qmp->pcs_misc;
+
+ if (!tbls)
+ return;
+
+ qmp_pcie_configure(serdes, tbls->serdes, tbls->serdes_num);
+
+ qmp_pcie_configure_lane(tx, tbls->tx, tbls->tx_num, 1);
+ qmp_pcie_configure_lane(rx, tbls->rx, tbls->rx_num, 1);
+
+ if (cfg->lanes >= 2) {
+ qmp_pcie_configure_lane(tx2, tbls->tx, tbls->tx_num, 2);
+ qmp_pcie_configure_lane(rx2, tbls->rx, tbls->rx_num, 2);
+ }
+
+ qmp_pcie_configure(pcs, tbls->pcs, tbls->pcs_num);
+ qmp_pcie_configure(pcs_misc, tbls->pcs_misc, tbls->pcs_misc_num);
+
+ if (cfg->lanes >= 4 && qmp->tcsr_4ln_config) {
+ qmp_pcie_configure(serdes, cfg->serdes_4ln_tbl, cfg->serdes_4ln_num);
+ qmp_pcie_init_port_b(qmp, tbls);
+ }
}
static int qmp_pcie_init(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *pcs = qphy->pcs;
+ struct qmp_pcie *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
int ret;
- /* turn on regulator supplies */
ret = regulator_bulk_enable(cfg->num_vregs, qmp->vregs);
if (ret) {
dev_err(qmp->dev, "failed to enable regulators, err=%d\n", ret);
goto err_disable_regulators;
}
+ usleep_range(200, 300);
+
ret = reset_control_bulk_deassert(cfg->num_resets, qmp->resets);
if (ret) {
dev_err(qmp->dev, "reset deassert failed\n");
if (ret)
goto err_assert_reset;
- if (cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL])
- qphy_setbits(pcs,
- cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
- cfg->pwrdn_ctrl);
- else
- qphy_setbits(pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
- cfg->pwrdn_ctrl);
-
return 0;
err_assert_reset:
static int qmp_pcie_exit(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_pcie *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
reset_control_bulk_assert(cfg->num_resets, qmp->resets);
static int qmp_pcie_power_on(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *tx = qphy->tx;
- void __iomem *rx = qphy->rx;
- void __iomem *pcs = qphy->pcs;
- void __iomem *pcs_misc = qphy->pcs_misc;
+ struct qmp_pcie *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ const struct qmp_phy_cfg_tbls *mode_tbls;
+ void __iomem *pcs = qmp->pcs;
void __iomem *status;
- unsigned int mask, val, ready;
+ unsigned int mask, val;
int ret;
- qmp_pcie_serdes_init(qphy);
-
- ret = clk_prepare_enable(qphy->pipe_clk);
- if (ret) {
- dev_err(qmp->dev, "pipe_clk enable failed err=%d\n", ret);
- return ret;
- }
-
- /* Tx, Rx, and PCS configurations */
- qmp_pcie_configure_lane(tx, cfg->regs, cfg->tx_tbl, cfg->tx_tbl_num, 1);
- qmp_pcie_configure_lane(tx, cfg->regs, cfg->tx_tbl_sec, cfg->tx_tbl_num_sec, 1);
-
- if (cfg->lanes >= 2) {
- qmp_pcie_configure_lane(qphy->tx2, cfg->regs, cfg->tx_tbl,
- cfg->tx_tbl_num, 2);
- qmp_pcie_configure_lane(qphy->tx2, cfg->regs, cfg->tx_tbl_sec,
- cfg->tx_tbl_num_sec, 2);
- }
-
- qmp_pcie_configure_lane(rx, cfg->regs, cfg->rx_tbl, cfg->rx_tbl_num, 1);
- qmp_pcie_configure_lane(rx, cfg->regs, cfg->rx_tbl_sec, cfg->rx_tbl_num_sec, 1);
+ qphy_setbits(pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
+ cfg->pwrdn_ctrl);
- if (cfg->lanes >= 2) {
- qmp_pcie_configure_lane(qphy->rx2, cfg->regs, cfg->rx_tbl,
- cfg->rx_tbl_num, 2);
- qmp_pcie_configure_lane(qphy->rx2, cfg->regs, cfg->rx_tbl_sec,
- cfg->rx_tbl_num_sec, 2);
- }
-
- qmp_pcie_configure(pcs, cfg->regs, cfg->pcs_tbl, cfg->pcs_tbl_num);
- qmp_pcie_configure(pcs, cfg->regs, cfg->pcs_tbl_sec, cfg->pcs_tbl_num_sec);
+ if (qmp->mode == PHY_MODE_PCIE_RC)
+ mode_tbls = cfg->tbls_rc;
+ else
+ mode_tbls = cfg->tbls_ep;
- qmp_pcie_configure(pcs_misc, cfg->regs, cfg->pcs_misc_tbl, cfg->pcs_misc_tbl_num);
- qmp_pcie_configure(pcs_misc, cfg->regs, cfg->pcs_misc_tbl_sec, cfg->pcs_misc_tbl_num_sec);
+ qmp_pcie_init_registers(qmp, &cfg->tbls);
+ qmp_pcie_init_registers(qmp, mode_tbls);
- /*
- * Pull out PHY from POWER DOWN state.
- * This is active low enable signal to power-down PHY.
- */
- qphy_setbits(pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL, cfg->pwrdn_ctrl);
-
- if (cfg->has_pwrdn_delay)
- usleep_range(cfg->pwrdn_delay_min, cfg->pwrdn_delay_max);
+ ret = clk_bulk_prepare_enable(qmp->num_pipe_clks, qmp->pipe_clks);
+ if (ret)
+ return ret;
/* Pull PHY out of reset state */
qphy_clrbits(pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
/* start SerDes and Phy-Coding-Sublayer */
- qphy_setbits(pcs, cfg->regs[QPHY_START_CTRL], cfg->start_ctrl);
+ qphy_setbits(pcs, cfg->regs[QPHY_START_CTRL], SERDES_START | PCS_START);
+
+ if (!cfg->skip_start_delay)
+ usleep_range(1000, 1200);
status = pcs + cfg->regs[QPHY_PCS_STATUS];
mask = cfg->phy_status;
- ready = 0;
-
- ret = readl_poll_timeout(status, val, (val & mask) == ready, 10,
+ ret = readl_poll_timeout(status, val, !(val & mask), 200,
PHY_INIT_COMPLETE_TIMEOUT);
if (ret) {
dev_err(qmp->dev, "phy initialization timed-out\n");
return 0;
err_disable_pipe_clk:
- clk_disable_unprepare(qphy->pipe_clk);
+ clk_bulk_disable_unprepare(qmp->num_pipe_clks, qmp->pipe_clks);
return ret;
}
static int qmp_pcie_power_off(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_pcie *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
- clk_disable_unprepare(qphy->pipe_clk);
+ clk_bulk_disable_unprepare(qmp->num_pipe_clks, qmp->pipe_clks);
/* PHY reset */
- qphy_setbits(qphy->pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
+ qphy_setbits(qmp->pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
/* stop SerDes and Phy-Coding-Sublayer */
- qphy_clrbits(qphy->pcs, cfg->regs[QPHY_START_CTRL], cfg->start_ctrl);
+ qphy_clrbits(qmp->pcs, cfg->regs[QPHY_START_CTRL],
+ SERDES_START | PCS_START);
/* Put PHY into POWER DOWN state: active low */
- if (cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL]) {
- qphy_clrbits(qphy->pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
- cfg->pwrdn_ctrl);
- } else {
- qphy_clrbits(qphy->pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
- cfg->pwrdn_ctrl);
- }
+ qphy_clrbits(qmp->pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
+ cfg->pwrdn_ctrl);
return 0;
}
return qmp_pcie_exit(phy);
}
-static int qmp_pcie_vreg_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static int qmp_pcie_set_mode(struct phy *phy, enum phy_mode mode, int submode)
+{
+ struct qmp_pcie *qmp = phy_get_drvdata(phy);
+
+ switch (submode) {
+ case PHY_MODE_PCIE_RC:
+ case PHY_MODE_PCIE_EP:
+ qmp->mode = submode;
+ break;
+ default:
+ dev_err(&phy->dev, "Unsupported submode %d\n", submode);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct phy_ops qmp_pcie_phy_ops = {
+ .power_on = qmp_pcie_enable,
+ .power_off = qmp_pcie_disable,
+ .set_mode = qmp_pcie_set_mode,
+ .owner = THIS_MODULE,
+};
+
+static int qmp_pcie_vreg_init(struct qmp_pcie *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int num = cfg->num_vregs;
int i;
return devm_regulator_bulk_get(dev, num, qmp->vregs);
}
-static int qmp_pcie_reset_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static int qmp_pcie_reset_init(struct qmp_pcie *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int i;
int ret;
return 0;
}
-static int qmp_pcie_clk_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static int qmp_pcie_clk_init(struct qmp_pcie *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int num = cfg->num_clks;
int i;
* clk | +-------+ | +-----+
* +---------------+
*/
-static int phy_pipe_clk_register(struct qcom_qmp *qmp, struct device_node *np)
+static int phy_pipe_clk_register(struct qmp_pcie *qmp, struct device_node *np)
{
- struct clk_fixed_rate *fixed;
+ struct clk_fixed_rate *fixed = &qmp->pipe_clk_fixed;
struct clk_init_data init = { };
int ret;
return ret;
}
- fixed = devm_kzalloc(qmp->dev, sizeof(*fixed), GFP_KERNEL);
- if (!fixed)
- return -ENOMEM;
-
init.ops = &clk_fixed_rate_ops;
/*
* Controllers using QMP PHY-s use 125MHz pipe clock interface
* unless other frequency is specified in the PHY config.
*/
- if (qmp->phys[0]->cfg->pipe_clock_rate)
- fixed->fixed_rate = qmp->phys[0]->cfg->pipe_clock_rate;
+ if (qmp->cfg->pipe_clock_rate)
+ fixed->fixed_rate = qmp->cfg->pipe_clock_rate;
else
fixed->fixed_rate = 125000000;
return devm_add_action_or_reset(qmp->dev, phy_clk_release_provider, np);
}
-static const struct phy_ops qmp_pcie_ops = {
- .power_on = qmp_pcie_enable,
- .power_off = qmp_pcie_disable,
- .owner = THIS_MODULE,
-};
-
-static int qmp_pcie_create(struct device *dev, struct device_node *np, int id,
- void __iomem *serdes, const struct qmp_phy_cfg *cfg)
+static int qmp_pcie_parse_dt_legacy(struct qmp_pcie *qmp, struct device_node *np)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
- struct phy *generic_phy;
- struct qmp_phy *qphy;
- int ret;
+ struct platform_device *pdev = to_platform_device(qmp->dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
+ struct clk *clk;
- qphy = devm_kzalloc(dev, sizeof(*qphy), GFP_KERNEL);
- if (!qphy)
- return -ENOMEM;
+ qmp->serdes = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(qmp->serdes))
+ return PTR_ERR(qmp->serdes);
- qphy->cfg = cfg;
- qphy->serdes = serdes;
/*
- * Get memory resources for each phy lane:
+ * Get memory resources for the PHY:
* Resources are indexed as: tx -> 0; rx -> 1; pcs -> 2.
* For dual lane PHYs: tx2 -> 3, rx2 -> 4, pcs_misc (optional) -> 5
* For single lane PHYs: pcs_misc (optional) -> 3.
*/
- qphy->tx = devm_of_iomap(dev, np, 0, NULL);
- if (IS_ERR(qphy->tx))
- return PTR_ERR(qphy->tx);
+ qmp->tx = devm_of_iomap(dev, np, 0, NULL);
+ if (IS_ERR(qmp->tx))
+ return PTR_ERR(qmp->tx);
if (of_device_is_compatible(dev->of_node, "qcom,sdm845-qhp-pcie-phy"))
- qphy->rx = qphy->tx;
+ qmp->rx = qmp->tx;
else
- qphy->rx = devm_of_iomap(dev, np, 1, NULL);
- if (IS_ERR(qphy->rx))
- return PTR_ERR(qphy->rx);
+ qmp->rx = devm_of_iomap(dev, np, 1, NULL);
+ if (IS_ERR(qmp->rx))
+ return PTR_ERR(qmp->rx);
- qphy->pcs = devm_of_iomap(dev, np, 2, NULL);
- if (IS_ERR(qphy->pcs))
- return PTR_ERR(qphy->pcs);
+ qmp->pcs = devm_of_iomap(dev, np, 2, NULL);
+ if (IS_ERR(qmp->pcs))
+ return PTR_ERR(qmp->pcs);
if (cfg->lanes >= 2) {
- qphy->tx2 = devm_of_iomap(dev, np, 3, NULL);
- if (IS_ERR(qphy->tx2))
- return PTR_ERR(qphy->tx2);
+ qmp->tx2 = devm_of_iomap(dev, np, 3, NULL);
+ if (IS_ERR(qmp->tx2))
+ return PTR_ERR(qmp->tx2);
- qphy->rx2 = devm_of_iomap(dev, np, 4, NULL);
- if (IS_ERR(qphy->rx2))
- return PTR_ERR(qphy->rx2);
+ qmp->rx2 = devm_of_iomap(dev, np, 4, NULL);
+ if (IS_ERR(qmp->rx2))
+ return PTR_ERR(qmp->rx2);
- qphy->pcs_misc = devm_of_iomap(dev, np, 5, NULL);
+ qmp->pcs_misc = devm_of_iomap(dev, np, 5, NULL);
} else {
- qphy->pcs_misc = devm_of_iomap(dev, np, 3, NULL);
+ qmp->pcs_misc = devm_of_iomap(dev, np, 3, NULL);
}
- if (IS_ERR(qphy->pcs_misc) &&
+ if (IS_ERR(qmp->pcs_misc) &&
of_device_is_compatible(dev->of_node, "qcom,ipq6018-qmp-pcie-phy"))
- qphy->pcs_misc = qphy->pcs + 0x400;
+ qmp->pcs_misc = qmp->pcs + 0x400;
- if (IS_ERR(qphy->pcs_misc)) {
- if (cfg->pcs_misc_tbl || cfg->pcs_misc_tbl_sec)
- return PTR_ERR(qphy->pcs_misc);
+ if (IS_ERR(qmp->pcs_misc)) {
+ if (cfg->tbls.pcs_misc ||
+ (cfg->tbls_rc && cfg->tbls_rc->pcs_misc) ||
+ (cfg->tbls_ep && cfg->tbls_ep->pcs_misc)) {
+ return PTR_ERR(qmp->pcs_misc);
+ }
}
- qphy->pipe_clk = devm_get_clk_from_child(dev, np, NULL);
- if (IS_ERR(qphy->pipe_clk)) {
- return dev_err_probe(dev, PTR_ERR(qphy->pipe_clk),
- "failed to get lane%d pipe clock\n", id);
+ clk = devm_get_clk_from_child(dev, np, NULL);
+ if (IS_ERR(clk)) {
+ return dev_err_probe(dev, PTR_ERR(clk),
+ "failed to get pipe clock\n");
+ }
+
+ qmp->num_pipe_clks = 1;
+ qmp->pipe_clks[0].id = "pipe";
+ qmp->pipe_clks[0].clk = clk;
+
+ return 0;
+}
+
+static int qmp_pcie_get_4ln_config(struct qmp_pcie *qmp)
+{
+ struct regmap *tcsr;
+ unsigned int args[2];
+ int ret;
+
+ tcsr = syscon_regmap_lookup_by_phandle_args(qmp->dev->of_node,
+ "qcom,4ln-config-sel",
+ ARRAY_SIZE(args), args);
+ if (IS_ERR(tcsr)) {
+ ret = PTR_ERR(tcsr);
+ if (ret == -ENOENT)
+ return 0;
+
+ dev_err(qmp->dev, "failed to lookup syscon: %d\n", ret);
+ return ret;
}
- generic_phy = devm_phy_create(dev, np, &qmp_pcie_ops);
- if (IS_ERR(generic_phy)) {
- ret = PTR_ERR(generic_phy);
- dev_err(dev, "failed to create qphy %d\n", ret);
+ ret = regmap_test_bits(tcsr, args[0], BIT(args[1]));
+ if (ret < 0) {
+ dev_err(qmp->dev, "failed to read tcsr: %d\n", ret);
return ret;
}
- qphy->phy = generic_phy;
- qphy->qmp = qmp;
- qmp->phys[id] = qphy;
- phy_set_drvdata(generic_phy, qphy);
+ qmp->tcsr_4ln_config = ret;
+
+ dev_dbg(qmp->dev, "4ln_config_sel = %d\n", qmp->tcsr_4ln_config);
return 0;
}
-static const struct of_device_id qmp_pcie_of_match_table[] = {
- {
- .compatible = "qcom,msm8998-qmp-pcie-phy",
- .data = &msm8998_pciephy_cfg,
- }, {
- .compatible = "qcom,ipq8074-qmp-pcie-phy",
- .data = &ipq8074_pciephy_cfg,
- }, {
- .compatible = "qcom,ipq8074-qmp-gen3-pcie-phy",
- .data = &ipq8074_pciephy_gen3_cfg,
- }, {
- .compatible = "qcom,ipq6018-qmp-pcie-phy",
- .data = &ipq6018_pciephy_cfg,
- }, {
- .compatible = "qcom,sc8180x-qmp-pcie-phy",
- .data = &sc8180x_pciephy_cfg,
- }, {
- .compatible = "qcom,sdm845-qhp-pcie-phy",
- .data = &sdm845_qhp_pciephy_cfg,
- }, {
- .compatible = "qcom,sdm845-qmp-pcie-phy",
- .data = &sdm845_qmp_pciephy_cfg,
- }, {
- .compatible = "qcom,sm8250-qmp-gen3x1-pcie-phy",
- .data = &sm8250_qmp_gen3x1_pciephy_cfg,
- }, {
- .compatible = "qcom,sm8250-qmp-gen3x2-pcie-phy",
- .data = &sm8250_qmp_gen3x2_pciephy_cfg,
- }, {
- .compatible = "qcom,sm8250-qmp-modem-pcie-phy",
- .data = &sm8250_qmp_gen3x2_pciephy_cfg,
- }, {
- .compatible = "qcom,sdx55-qmp-pcie-phy",
- .data = &sdx55_qmp_pciephy_cfg,
- }, {
- .compatible = "qcom,sm8450-qmp-gen3x1-pcie-phy",
- .data = &sm8450_qmp_gen3x1_pciephy_cfg,
- }, {
- .compatible = "qcom,sm8450-qmp-gen4x2-pcie-phy",
- .data = &sm8450_qmp_gen4x2_pciephy_cfg,
- },
- { },
-};
-MODULE_DEVICE_TABLE(of, qmp_pcie_of_match_table);
+static int qmp_pcie_parse_dt(struct qmp_pcie *qmp)
+{
+ struct platform_device *pdev = to_platform_device(qmp->dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ const struct qmp_pcie_offsets *offs = cfg->offsets;
+ struct device *dev = qmp->dev;
+ void __iomem *base;
+ int ret;
+
+ if (!offs)
+ return -EINVAL;
+
+ ret = qmp_pcie_get_4ln_config(qmp);
+ if (ret)
+ return ret;
+
+ base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+
+ qmp->serdes = base + offs->serdes;
+ qmp->pcs = base + offs->pcs;
+ qmp->pcs_misc = base + offs->pcs_misc;
+ qmp->tx = base + offs->tx;
+ qmp->rx = base + offs->rx;
+
+ if (cfg->lanes >= 2) {
+ qmp->tx2 = base + offs->tx2;
+ qmp->rx2 = base + offs->rx2;
+ }
+
+ if (qmp->cfg->lanes >= 4 && qmp->tcsr_4ln_config) {
+ qmp->port_b = devm_platform_ioremap_resource(pdev, 1);
+ if (IS_ERR(qmp->port_b))
+ return PTR_ERR(qmp->port_b);
+ }
+
+ qmp->num_pipe_clks = 2;
+ qmp->pipe_clks[0].id = "pipe";
+ qmp->pipe_clks[1].id = "pipediv2";
+
+ ret = devm_clk_bulk_get(dev, qmp->num_pipe_clks, qmp->pipe_clks);
+ if (ret)
+ return ret;
+
+ return 0;
+}
static int qmp_pcie_probe(struct platform_device *pdev)
{
- struct qcom_qmp *qmp;
struct device *dev = &pdev->dev;
- struct device_node *child;
struct phy_provider *phy_provider;
- void __iomem *serdes;
- const struct qmp_phy_cfg *cfg = NULL;
- int num, id;
+ struct device_node *np;
+ struct qmp_pcie *qmp;
int ret;
qmp = devm_kzalloc(dev, sizeof(*qmp), GFP_KERNEL);
return -ENOMEM;
qmp->dev = dev;
- dev_set_drvdata(dev, qmp);
- /* Get the specific init parameters of QMP phy */
- cfg = of_device_get_match_data(dev);
- if (!cfg)
+ qmp->cfg = of_device_get_match_data(dev);
+ if (!qmp->cfg)
return -EINVAL;
- /* per PHY serdes; usually located at base address */
- serdes = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(serdes))
- return PTR_ERR(serdes);
+ WARN_ON_ONCE(!qmp->cfg->pwrdn_ctrl);
+ WARN_ON_ONCE(!qmp->cfg->phy_status);
- ret = qmp_pcie_clk_init(dev, cfg);
+ ret = qmp_pcie_clk_init(qmp);
if (ret)
return ret;
- ret = qmp_pcie_reset_init(dev, cfg);
+ ret = qmp_pcie_reset_init(qmp);
if (ret)
return ret;
- ret = qmp_pcie_vreg_init(dev, cfg);
+ ret = qmp_pcie_vreg_init(qmp);
if (ret)
- return dev_err_probe(dev, ret,
- "failed to get regulator supplies\n");
-
- num = of_get_available_child_count(dev->of_node);
- /* do we have a rogue child node ? */
- if (num > 1)
- return -EINVAL;
+ return ret;
- qmp->phys = devm_kcalloc(dev, num, sizeof(*qmp->phys), GFP_KERNEL);
- if (!qmp->phys)
- return -ENOMEM;
+ /* Check for legacy binding with child node. */
+ np = of_get_next_available_child(dev->of_node, NULL);
+ if (np) {
+ ret = qmp_pcie_parse_dt_legacy(qmp, np);
+ } else {
+ np = of_node_get(dev->of_node);
+ ret = qmp_pcie_parse_dt(qmp);
+ }
+ if (ret)
+ goto err_node_put;
- id = 0;
- for_each_available_child_of_node(dev->of_node, child) {
- /* Create per-lane phy */
- ret = qmp_pcie_create(dev, child, id, serdes, cfg);
- if (ret) {
- dev_err(dev, "failed to create lane%d phy, %d\n",
- id, ret);
- goto err_node_put;
- }
+ ret = phy_pipe_clk_register(qmp, np);
+ if (ret)
+ goto err_node_put;
- /*
- * Register the pipe clock provided by phy.
- * See function description to see details of this pipe clock.
- */
- ret = phy_pipe_clk_register(qmp, child);
- if (ret) {
- dev_err(qmp->dev,
- "failed to register pipe clock source\n");
- goto err_node_put;
- }
+ qmp->mode = PHY_MODE_PCIE_RC;
- id++;
+ qmp->phy = devm_phy_create(dev, np, &qmp_pcie_phy_ops);
+ if (IS_ERR(qmp->phy)) {
+ ret = PTR_ERR(qmp->phy);
+ dev_err(dev, "failed to create PHY: %d\n", ret);
+ goto err_node_put;
}
+ phy_set_drvdata(qmp->phy, qmp);
+
+ of_node_put(np);
+
phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
return PTR_ERR_OR_ZERO(phy_provider);
err_node_put:
- of_node_put(child);
+ of_node_put(np);
return ret;
}
+static const struct of_device_id qmp_pcie_of_match_table[] = {
+ {
+ .compatible = "qcom,ipq6018-qmp-pcie-phy",
+ .data = &ipq6018_pciephy_cfg,
+ }, {
+ .compatible = "qcom,ipq8074-qmp-gen3-pcie-phy",
+ .data = &ipq8074_pciephy_gen3_cfg,
+ }, {
+ .compatible = "qcom,ipq8074-qmp-pcie-phy",
+ .data = &ipq8074_pciephy_cfg,
+ }, {
+ .compatible = "qcom,msm8998-qmp-pcie-phy",
+ .data = &msm8998_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sc8180x-qmp-pcie-phy",
+ .data = &sc8180x_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sc8280xp-qmp-gen3x1-pcie-phy",
+ .data = &sc8280xp_qmp_gen3x1_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sc8280xp-qmp-gen3x2-pcie-phy",
+ .data = &sc8280xp_qmp_gen3x2_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sc8280xp-qmp-gen3x4-pcie-phy",
+ .data = &sc8280xp_qmp_gen3x4_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sdm845-qhp-pcie-phy",
+ .data = &sdm845_qhp_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sdm845-qmp-pcie-phy",
+ .data = &sdm845_qmp_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sdx55-qmp-pcie-phy",
+ .data = &sdx55_qmp_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sm8250-qmp-gen3x1-pcie-phy",
+ .data = &sm8250_qmp_gen3x1_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sm8250-qmp-gen3x2-pcie-phy",
+ .data = &sm8250_qmp_gen3x2_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sm8250-qmp-modem-pcie-phy",
+ .data = &sm8250_qmp_gen3x2_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sm8450-qmp-gen3x1-pcie-phy",
+ .data = &sm8450_qmp_gen3x1_pciephy_cfg,
+ }, {
+ .compatible = "qcom,sm8450-qmp-gen4x2-pcie-phy",
+ .data = &sm8450_qmp_gen4x2_pciephy_cfg,
+ },
+ { },
+};
+MODULE_DEVICE_TABLE(of, qmp_pcie_of_match_table);
+
static struct platform_driver qmp_pcie_driver = {
.probe = qmp_pcie_probe,
.driver = {
#define QCOM_PHY_QMP_PCS_PCIE_V5_H_
/* Only for QMP V5 PHY - PCS_PCIE registers */
+#define QPHY_V5_PCS_PCIE_POWER_STATE_CONFIG2 0x0c
+#define QPHY_V5_PCS_PCIE_POWER_STATE_CONFIG4 0x14
#define QPHY_V5_PCS_PCIE_ENDPOINT_REFCLK_DRIVE 0x20
#define QPHY_V5_PCS_PCIE_INT_AUX_CLK_CONFIG1 0x54
#define QPHY_V5_PCS_PCIE_OSC_DTCT_ACTIONS 0x94
/* Only for QMP V5_20 PHY - PCIe PCS registers */
#define QPHY_V5_20_PCS_PCIE_ENDPOINT_REFCLK_DRIVE 0x01c
+#define QPHY_V5_20_PCS_PCIE_OSC_DTCT_MODE2_CONFIG5 0x084
#define QPHY_V5_20_PCS_PCIE_OSC_DTCT_ACTIONS 0x090
#define QPHY_V5_20_PCS_PCIE_EQ_CONFIG1 0x0a0
+#define QPHY_V5_20_PCS_PCIE_PRESET_P10_POST 0x0e0
#define QPHY_V5_20_PCS_PCIE_G4_EQ_CONFIG5 0x108
#define QPHY_V5_20_PCS_PCIE_G4_PRE_GAIN 0x15c
#define QPHY_V5_20_PCS_PCIE_RX_MARGINING_CONFIG3 0x184
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2022, Linaro Ltd.
+ */
+
+#ifndef QCOM_PHY_QMP_PCS_V5_20_H_
+#define QCOM_PHY_QMP_PCS_V5_20_H_
+
+#define QPHY_V5_20_PCS_G3S2_PRE_GAIN 0x170
+#define QPHY_V5_20_PCS_RX_SIGDET_LVL 0x188
+#define QPHY_V5_20_PCS_EQ_CONFIG4 0x1e0
+#define QPHY_V5_20_PCS_EQ_CONFIG5 0x1e4
+
+#endif
#include <linux/reset.h>
#include <linux/slab.h>
-#include <dt-bindings/phy/phy.h>
-
#include "phy-qcom-qmp.h"
/* QPHY_SW_RESET bit */
/* QPHY_START_CONTROL bits */
#define SERDES_START BIT(0)
#define PCS_START BIT(1)
-/* QPHY_PCS_STATUS bit */
-#define PHYSTATUS BIT(6)
/* QPHY_PCS_READY_STATUS bit */
#define PCS_READY BIT(0)
unsigned int offset;
unsigned int val;
/*
- * register part of layout ?
- * if yes, then offset gives index in the reg-layout
- */
- bool in_layout;
- /*
* mask of lanes for which this register is written
* for cases when second lane needs different values
*/
.lane_mask = 0xff, \
}
-#define QMP_PHY_INIT_CFG_L(o, v) \
- { \
- .offset = o, \
- .val = v, \
- .in_layout = true, \
- .lane_mask = 0xff, \
- }
-
#define QMP_PHY_INIT_CFG_LANE(o, v, l) \
{ \
.offset = o, \
static const unsigned int msm8996_ufsphy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_START_CTRL] = 0x00,
[QPHY_PCS_READY_STATUS] = 0x168,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = 0x04,
};
static const unsigned int sdm845_ufsphy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_START_CTRL] = 0x00,
[QPHY_PCS_READY_STATUS] = 0x160,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = 0x04,
};
static const unsigned int sm6115_ufsphy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_START_CTRL] = 0x00,
[QPHY_PCS_READY_STATUS] = 0x168,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = 0x04,
};
static const unsigned int sm8150_ufsphy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_START_CTRL] = QPHY_V4_PCS_UFS_PHY_START,
[QPHY_PCS_READY_STATUS] = QPHY_V4_PCS_UFS_READY_STATUS,
[QPHY_SW_RESET] = QPHY_V4_PCS_UFS_SW_RESET,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = QPHY_V4_PCS_UFS_POWER_DOWN_CONTROL,
};
static const struct qmp_phy_init_tbl msm8996_ufs_serdes_tbl[] = {
QMP_PHY_INIT_CFG(QPHY_V5_PCS_UFS_MULTI_LANE_CTRL1, 0x02),
};
+struct qmp_ufs_offsets {
+ u16 serdes;
+ u16 pcs;
+ u16 tx;
+ u16 rx;
+ u16 tx2;
+ u16 rx2;
+};
+
/* struct qmp_phy_cfg - per-PHY initialization config */
struct qmp_phy_cfg {
int lanes;
+ const struct qmp_ufs_offsets *offsets;
+
/* Init sequence for PHY blocks - serdes, tx, rx, pcs */
const struct qmp_phy_init_tbl *serdes_tbl;
int serdes_tbl_num;
/* array of registers with different offsets */
const unsigned int *regs;
- unsigned int start_ctrl;
- unsigned int pwrdn_ctrl;
- /* bit offset of PHYSTATUS in QPHY_PCS_STATUS register */
- unsigned int phy_status;
-
/* true, if PCS block has no separate SW_RESET register */
bool no_pcs_sw_reset;
};
-/**
- * struct qmp_phy - per-lane phy descriptor
- *
- * @phy: generic phy
- * @cfg: phy specific configuration
- * @serdes: iomapped memory space for phy's serdes (i.e. PLL)
- * @tx: iomapped memory space for lane's tx
- * @rx: iomapped memory space for lane's rx
- * @pcs: iomapped memory space for lane's pcs
- * @tx2: iomapped memory space for second lane's tx (in dual lane PHYs)
- * @rx2: iomapped memory space for second lane's rx (in dual lane PHYs)
- * @pcs_misc: iomapped memory space for lane's pcs_misc
- * @qmp: QMP phy to which this lane belongs
- */
-struct qmp_phy {
- struct phy *phy;
+struct qmp_ufs {
+ struct device *dev;
+
const struct qmp_phy_cfg *cfg;
+
void __iomem *serdes;
+ void __iomem *pcs;
+ void __iomem *pcs_misc;
void __iomem *tx;
void __iomem *rx;
- void __iomem *pcs;
void __iomem *tx2;
void __iomem *rx2;
- void __iomem *pcs_misc;
- struct qcom_qmp *qmp;
-};
-
-/**
- * struct qcom_qmp - structure holding QMP phy block attributes
- *
- * @dev: device
- *
- * @clks: array of clocks required by phy
- * @resets: array of resets required by phy
- * @vregs: regulator supplies bulk data
- *
- * @phys: array of per-lane phy descriptors
- * @ufs_reset: optional UFS PHY reset handle
- */
-struct qcom_qmp {
- struct device *dev;
struct clk_bulk_data *clks;
struct regulator_bulk_data *vregs;
-
- struct qmp_phy **phys;
-
struct reset_control *ufs_reset;
+
+ struct phy *phy;
};
static inline void qphy_setbits(void __iomem *base, u32 offset, u32 val)
"vdda-phy", "vdda-pll",
};
+static const struct qmp_ufs_offsets qmp_ufs_offsets_v5 = {
+ .serdes = 0,
+ .pcs = 0xc00,
+ .tx = 0x400,
+ .rx = 0x600,
+ .tx2 = 0x800,
+ .rx2 = 0xa00,
+};
+
static const struct qmp_phy_cfg msm8996_ufs_cfg = {
.lanes = 1,
.regs = msm8996_ufsphy_regs_layout,
- .start_ctrl = SERDES_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.no_pcs_sw_reset = true,
};
+static const struct qmp_phy_cfg sc8280xp_ufsphy_cfg = {
+ .lanes = 2,
+
+ .offsets = &qmp_ufs_offsets_v5,
+
+ .serdes_tbl = sm8350_ufsphy_serdes_tbl,
+ .serdes_tbl_num = ARRAY_SIZE(sm8350_ufsphy_serdes_tbl),
+ .tx_tbl = sm8350_ufsphy_tx_tbl,
+ .tx_tbl_num = ARRAY_SIZE(sm8350_ufsphy_tx_tbl),
+ .rx_tbl = sm8350_ufsphy_rx_tbl,
+ .rx_tbl_num = ARRAY_SIZE(sm8350_ufsphy_rx_tbl),
+ .pcs_tbl = sm8350_ufsphy_pcs_tbl,
+ .pcs_tbl_num = ARRAY_SIZE(sm8350_ufsphy_pcs_tbl),
+ .clk_list = sdm845_ufs_phy_clk_l,
+ .num_clks = ARRAY_SIZE(sdm845_ufs_phy_clk_l),
+ .vreg_list = qmp_phy_vreg_l,
+ .num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
+ .regs = sm8150_ufsphy_regs_layout,
+};
+
static const struct qmp_phy_cfg sdm845_ufsphy_cfg = {
.lanes = 2,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sdm845_ufsphy_regs_layout,
- .start_ctrl = SERDES_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.no_pcs_sw_reset = true,
};
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sm6115_ufsphy_regs_layout,
- .start_ctrl = SERDES_START,
- .pwrdn_ctrl = SW_PWRDN,
-
.no_pcs_sw_reset = true,
};
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sm8150_ufsphy_regs_layout,
-
- .start_ctrl = SERDES_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
};
static const struct qmp_phy_cfg sm8350_ufsphy_cfg = {
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sm8150_ufsphy_regs_layout,
-
- .start_ctrl = SERDES_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
};
static const struct qmp_phy_cfg sm8450_ufsphy_cfg = {
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = sm8150_ufsphy_regs_layout,
-
- .start_ctrl = SERDES_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
};
static void qmp_ufs_configure_lane(void __iomem *base,
- const unsigned int *regs,
const struct qmp_phy_init_tbl tbl[],
int num,
u8 lane_mask)
if (!(t->lane_mask & lane_mask))
continue;
- if (t->in_layout)
- writel(t->val, base + regs[t->offset]);
- else
- writel(t->val, base + t->offset);
+ writel(t->val, base + t->offset);
}
}
static void qmp_ufs_configure(void __iomem *base,
- const unsigned int *regs,
const struct qmp_phy_init_tbl tbl[],
int num)
{
- qmp_ufs_configure_lane(base, regs, tbl, num, 0xff);
+ qmp_ufs_configure_lane(base, tbl, num, 0xff);
}
-static int qmp_ufs_serdes_init(struct qmp_phy *qphy)
+static int qmp_ufs_serdes_init(struct qmp_ufs *qmp)
{
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *serdes = qphy->serdes;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *serdes = qmp->serdes;
const struct qmp_phy_init_tbl *serdes_tbl = cfg->serdes_tbl;
int serdes_tbl_num = cfg->serdes_tbl_num;
- qmp_ufs_configure(serdes, cfg->regs, serdes_tbl, serdes_tbl_num);
+ qmp_ufs_configure(serdes, serdes_tbl, serdes_tbl_num);
return 0;
}
-static int qmp_ufs_com_init(struct qmp_phy *qphy)
+static int qmp_ufs_com_init(struct qmp_ufs *qmp)
{
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *pcs = qphy->pcs;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *pcs = qmp->pcs;
int ret;
- /* turn on regulator supplies */
ret = regulator_bulk_enable(cfg->num_vregs, qmp->vregs);
if (ret) {
dev_err(qmp->dev, "failed to enable regulators, err=%d\n", ret);
if (ret)
goto err_disable_regulators;
- if (cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL])
- qphy_setbits(pcs,
- cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
- cfg->pwrdn_ctrl);
- else
- qphy_setbits(pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
- cfg->pwrdn_ctrl);
+ qphy_setbits(pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL], SW_PWRDN);
return 0;
return ret;
}
-static int qmp_ufs_com_exit(struct qmp_phy *qphy)
+static int qmp_ufs_com_exit(struct qmp_ufs *qmp)
{
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
reset_control_assert(qmp->ufs_reset);
static int qmp_ufs_init(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_ufs *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
int ret;
dev_vdbg(qmp->dev, "Initializing QMP phy\n");
return ret;
}
- ret = qmp_ufs_com_init(qphy);
+ ret = qmp_ufs_com_init(qmp);
if (ret)
return ret;
static int qmp_ufs_power_on(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *tx = qphy->tx;
- void __iomem *rx = qphy->rx;
- void __iomem *pcs = qphy->pcs;
+ struct qmp_ufs *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *tx = qmp->tx;
+ void __iomem *rx = qmp->rx;
+ void __iomem *pcs = qmp->pcs;
void __iomem *status;
- unsigned int mask, val, ready;
+ unsigned int val;
int ret;
- qmp_ufs_serdes_init(qphy);
+ qmp_ufs_serdes_init(qmp);
/* Tx, Rx, and PCS configurations */
- qmp_ufs_configure_lane(tx, cfg->regs, cfg->tx_tbl, cfg->tx_tbl_num, 1);
-
- if (cfg->lanes >= 2) {
- qmp_ufs_configure_lane(qphy->tx2, cfg->regs,
- cfg->tx_tbl, cfg->tx_tbl_num, 2);
- }
-
- qmp_ufs_configure_lane(rx, cfg->regs, cfg->rx_tbl, cfg->rx_tbl_num, 1);
+ qmp_ufs_configure_lane(tx, cfg->tx_tbl, cfg->tx_tbl_num, 1);
+ qmp_ufs_configure_lane(rx, cfg->rx_tbl, cfg->rx_tbl_num, 1);
if (cfg->lanes >= 2) {
- qmp_ufs_configure_lane(qphy->rx2, cfg->regs,
- cfg->rx_tbl, cfg->rx_tbl_num, 2);
+ qmp_ufs_configure_lane(qmp->tx2, cfg->tx_tbl, cfg->tx_tbl_num, 2);
+ qmp_ufs_configure_lane(qmp->rx2, cfg->rx_tbl, cfg->rx_tbl_num, 2);
}
- qmp_ufs_configure(pcs, cfg->regs, cfg->pcs_tbl, cfg->pcs_tbl_num);
+ qmp_ufs_configure(pcs, cfg->pcs_tbl, cfg->pcs_tbl_num);
ret = reset_control_deassert(qmp->ufs_reset);
if (ret)
/* Pull PHY out of reset state */
if (!cfg->no_pcs_sw_reset)
qphy_clrbits(pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
- /* start SerDes and Phy-Coding-Sublayer */
- qphy_setbits(pcs, cfg->regs[QPHY_START_CTRL], cfg->start_ctrl);
- status = pcs + cfg->regs[QPHY_PCS_READY_STATUS];
- mask = PCS_READY;
- ready = PCS_READY;
+ /* start SerDes */
+ qphy_setbits(pcs, cfg->regs[QPHY_START_CTRL], SERDES_START);
- ret = readl_poll_timeout(status, val, (val & mask) == ready, 10,
+ status = pcs + cfg->regs[QPHY_PCS_READY_STATUS];
+ ret = readl_poll_timeout(status, val, (val & PCS_READY), 200,
PHY_INIT_COMPLETE_TIMEOUT);
if (ret) {
dev_err(qmp->dev, "phy initialization timed-out\n");
static int qmp_ufs_power_off(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_ufs *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
/* PHY reset */
if (!cfg->no_pcs_sw_reset)
- qphy_setbits(qphy->pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
+ qphy_setbits(qmp->pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
- /* stop SerDes and Phy-Coding-Sublayer */
- qphy_clrbits(qphy->pcs, cfg->regs[QPHY_START_CTRL], cfg->start_ctrl);
+ /* stop SerDes */
+ qphy_clrbits(qmp->pcs, cfg->regs[QPHY_START_CTRL], SERDES_START);
/* Put PHY into POWER DOWN state: active low */
- if (cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL]) {
- qphy_clrbits(qphy->pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
- cfg->pwrdn_ctrl);
- } else {
- qphy_clrbits(qphy->pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
- cfg->pwrdn_ctrl);
- }
+ qphy_clrbits(qmp->pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
+ SW_PWRDN);
return 0;
}
static int qmp_ufs_exit(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
+ struct qmp_ufs *qmp = phy_get_drvdata(phy);
- qmp_ufs_com_exit(qphy);
+ qmp_ufs_com_exit(qmp);
return 0;
}
return qmp_ufs_exit(phy);
}
-static int qmp_ufs_vreg_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static const struct phy_ops qcom_qmp_ufs_phy_ops = {
+ .power_on = qmp_ufs_enable,
+ .power_off = qmp_ufs_disable,
+ .owner = THIS_MODULE,
+};
+
+static int qmp_ufs_vreg_init(struct qmp_ufs *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int num = cfg->num_vregs;
int i;
return devm_regulator_bulk_get(dev, num, qmp->vregs);
}
-static int qmp_ufs_clk_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static int qmp_ufs_clk_init(struct qmp_ufs *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int num = cfg->num_clks;
int i;
return devm_clk_bulk_get(dev, num, qmp->clks);
}
-static const struct phy_ops qcom_qmp_ufs_ops = {
- .power_on = qmp_ufs_enable,
- .power_off = qmp_ufs_disable,
- .owner = THIS_MODULE,
-};
-
-static int qmp_ufs_create(struct device *dev, struct device_node *np, int id,
- void __iomem *serdes, const struct qmp_phy_cfg *cfg)
+static int qmp_ufs_parse_dt_legacy(struct qmp_ufs *qmp, struct device_node *np)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
- struct phy *generic_phy;
- struct qmp_phy *qphy;
- int ret;
+ struct platform_device *pdev = to_platform_device(qmp->dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
- qphy = devm_kzalloc(dev, sizeof(*qphy), GFP_KERNEL);
- if (!qphy)
- return -ENOMEM;
+ qmp->serdes = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(qmp->serdes))
+ return PTR_ERR(qmp->serdes);
- qphy->cfg = cfg;
- qphy->serdes = serdes;
/*
- * Get memory resources for each phy lane:
+ * Get memory resources for the PHY:
* Resources are indexed as: tx -> 0; rx -> 1; pcs -> 2.
* For dual lane PHYs: tx2 -> 3, rx2 -> 4, pcs_misc (optional) -> 5
* For single lane PHYs: pcs_misc (optional) -> 3.
*/
- qphy->tx = devm_of_iomap(dev, np, 0, NULL);
- if (IS_ERR(qphy->tx))
- return PTR_ERR(qphy->tx);
+ qmp->tx = devm_of_iomap(dev, np, 0, NULL);
+ if (IS_ERR(qmp->tx))
+ return PTR_ERR(qmp->tx);
- qphy->rx = devm_of_iomap(dev, np, 1, NULL);
- if (IS_ERR(qphy->rx))
- return PTR_ERR(qphy->rx);
+ qmp->rx = devm_of_iomap(dev, np, 1, NULL);
+ if (IS_ERR(qmp->rx))
+ return PTR_ERR(qmp->rx);
- qphy->pcs = devm_of_iomap(dev, np, 2, NULL);
- if (IS_ERR(qphy->pcs))
- return PTR_ERR(qphy->pcs);
+ qmp->pcs = devm_of_iomap(dev, np, 2, NULL);
+ if (IS_ERR(qmp->pcs))
+ return PTR_ERR(qmp->pcs);
if (cfg->lanes >= 2) {
- qphy->tx2 = devm_of_iomap(dev, np, 3, NULL);
- if (IS_ERR(qphy->tx2))
- return PTR_ERR(qphy->tx2);
+ qmp->tx2 = devm_of_iomap(dev, np, 3, NULL);
+ if (IS_ERR(qmp->tx2))
+ return PTR_ERR(qmp->tx2);
- qphy->rx2 = devm_of_iomap(dev, np, 4, NULL);
- if (IS_ERR(qphy->rx2))
- return PTR_ERR(qphy->rx2);
+ qmp->rx2 = devm_of_iomap(dev, np, 4, NULL);
+ if (IS_ERR(qmp->rx2))
+ return PTR_ERR(qmp->rx2);
- qphy->pcs_misc = devm_of_iomap(dev, np, 5, NULL);
+ qmp->pcs_misc = devm_of_iomap(dev, np, 5, NULL);
} else {
- qphy->pcs_misc = devm_of_iomap(dev, np, 3, NULL);
+ qmp->pcs_misc = devm_of_iomap(dev, np, 3, NULL);
}
- if (IS_ERR(qphy->pcs_misc))
+ if (IS_ERR(qmp->pcs_misc))
dev_vdbg(dev, "PHY pcs_misc-reg not used\n");
- generic_phy = devm_phy_create(dev, np, &qcom_qmp_ufs_ops);
- if (IS_ERR(generic_phy)) {
- ret = PTR_ERR(generic_phy);
- dev_err(dev, "failed to create qphy %d\n", ret);
+ return 0;
+}
+
+static int qmp_ufs_parse_dt(struct qmp_ufs *qmp)
+{
+ struct platform_device *pdev = to_platform_device(qmp->dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ const struct qmp_ufs_offsets *offs = cfg->offsets;
+ void __iomem *base;
+
+ if (!offs)
+ return -EINVAL;
+
+ base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+
+ qmp->serdes = base + offs->serdes;
+ qmp->pcs = base + offs->pcs;
+ qmp->tx = base + offs->tx;
+ qmp->rx = base + offs->rx;
+
+ if (cfg->lanes >= 2) {
+ qmp->tx2 = base + offs->tx2;
+ qmp->rx2 = base + offs->rx2;
+ }
+
+ return 0;
+}
+
+static int qmp_ufs_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct phy_provider *phy_provider;
+ struct device_node *np;
+ struct qmp_ufs *qmp;
+ int ret;
+
+ qmp = devm_kzalloc(dev, sizeof(*qmp), GFP_KERNEL);
+ if (!qmp)
+ return -ENOMEM;
+
+ qmp->dev = dev;
+
+ qmp->cfg = of_device_get_match_data(dev);
+ if (!qmp->cfg)
+ return -EINVAL;
+
+ ret = qmp_ufs_clk_init(qmp);
+ if (ret)
return ret;
+
+ ret = qmp_ufs_vreg_init(qmp);
+ if (ret)
+ return ret;
+
+ /* Check for legacy binding with child node. */
+ np = of_get_next_available_child(dev->of_node, NULL);
+ if (np) {
+ ret = qmp_ufs_parse_dt_legacy(qmp, np);
+ } else {
+ np = of_node_get(dev->of_node);
+ ret = qmp_ufs_parse_dt(qmp);
}
+ if (ret)
+ goto err_node_put;
- qphy->phy = generic_phy;
- qphy->qmp = qmp;
- qmp->phys[id] = qphy;
- phy_set_drvdata(generic_phy, qphy);
+ qmp->phy = devm_phy_create(dev, np, &qcom_qmp_ufs_phy_ops);
+ if (IS_ERR(qmp->phy)) {
+ ret = PTR_ERR(qmp->phy);
+ dev_err(dev, "failed to create PHY: %d\n", ret);
+ goto err_node_put;
+ }
- return 0;
+ phy_set_drvdata(qmp->phy, qmp);
+
+ of_node_put(np);
+
+ phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
+
+ return PTR_ERR_OR_ZERO(phy_provider);
+
+err_node_put:
+ of_node_put(np);
+ return ret;
}
static const struct of_device_id qmp_ufs_of_match_table[] = {
.data = &sm8150_ufsphy_cfg,
}, {
.compatible = "qcom,sc8280xp-qmp-ufs-phy",
- .data = &sm8350_ufsphy_cfg,
+ .data = &sc8280xp_ufsphy_cfg,
}, {
.compatible = "qcom,sdm845-qmp-ufs-phy",
.data = &sdm845_ufsphy_cfg,
};
MODULE_DEVICE_TABLE(of, qmp_ufs_of_match_table);
-static int qmp_ufs_probe(struct platform_device *pdev)
-{
- struct qcom_qmp *qmp;
- struct device *dev = &pdev->dev;
- struct device_node *child;
- struct phy_provider *phy_provider;
- void __iomem *serdes;
- const struct qmp_phy_cfg *cfg = NULL;
- int num, id;
- int ret;
-
- qmp = devm_kzalloc(dev, sizeof(*qmp), GFP_KERNEL);
- if (!qmp)
- return -ENOMEM;
-
- qmp->dev = dev;
- dev_set_drvdata(dev, qmp);
-
- /* Get the specific init parameters of QMP phy */
- cfg = of_device_get_match_data(dev);
- if (!cfg)
- return -EINVAL;
-
- /* per PHY serdes; usually located at base address */
- serdes = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(serdes))
- return PTR_ERR(serdes);
-
- ret = qmp_ufs_clk_init(dev, cfg);
- if (ret)
- return ret;
-
- ret = qmp_ufs_vreg_init(dev, cfg);
- if (ret)
- return dev_err_probe(dev, ret,
- "failed to get regulator supplies\n");
-
- num = of_get_available_child_count(dev->of_node);
- /* do we have a rogue child node ? */
- if (num > 1)
- return -EINVAL;
-
- qmp->phys = devm_kcalloc(dev, num, sizeof(*qmp->phys), GFP_KERNEL);
- if (!qmp->phys)
- return -ENOMEM;
-
- id = 0;
- for_each_available_child_of_node(dev->of_node, child) {
- /* Create per-lane phy */
- ret = qmp_ufs_create(dev, child, id, serdes, cfg);
- if (ret) {
- dev_err(dev, "failed to create lane%d phy, %d\n",
- id, ret);
- goto err_node_put;
- }
-
- id++;
- }
-
- phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
-
- return PTR_ERR_OR_ZERO(phy_provider);
-
-err_node_put:
- of_node_put(child);
- return ret;
-}
-
static struct platform_driver qmp_ufs_driver = {
.probe = qmp_ufs_probe,
.driver = {
#include <linux/reset.h>
#include <linux/slab.h>
-#include <dt-bindings/phy/phy.h>
-
#include "phy-qcom-qmp.h"
/* QPHY_SW_RESET bit */
#define CLAMP_EN BIT(0) /* enables i/o clamp_n */
#define PHY_INIT_COMPLETE_TIMEOUT 10000
-#define POWER_DOWN_DELAY_US_MIN 10
-#define POWER_DOWN_DELAY_US_MAX 11
struct qmp_phy_init_tbl {
unsigned int offset;
unsigned int val;
/*
- * register part of layout ?
- * if yes, then offset gives index in the reg-layout
- */
- bool in_layout;
- /*
* mask of lanes for which this register is written
* for cases when second lane needs different values
*/
.lane_mask = 0xff, \
}
-#define QMP_PHY_INIT_CFG_L(o, v) \
- { \
- .offset = o, \
- .val = v, \
- .in_layout = true, \
- .lane_mask = 0xff, \
- }
-
#define QMP_PHY_INIT_CFG_LANE(o, v, l) \
{ \
.offset = o, \
[QPHY_PCS_AUTONOMOUS_MODE_CTRL] = 0x0d4,
[QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR] = 0x0d8,
[QPHY_PCS_LFPS_RXTERM_IRQ_STATUS] = 0x178,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = 0x04,
};
static const unsigned int qmp_v3_usb3phy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_PCS_AUTONOMOUS_MODE_CTRL] = 0x0d8,
[QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR] = 0x0dc,
[QPHY_PCS_LFPS_RXTERM_IRQ_STATUS] = 0x170,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = 0x04,
};
static const unsigned int qmp_v4_usb3phy_regs_layout[QPHY_LAYOUT_SIZE] = {
QMP_PHY_INIT_CFG(QPHY_V5_PCS_REFGEN_REQ_CONFIG1, 0x21),
};
+struct qmp_usb_offsets {
+ u16 serdes;
+ u16 pcs;
+ u16 pcs_usb;
+ u16 tx;
+ u16 rx;
+};
+
/* struct qmp_phy_cfg - per-PHY initialization config */
struct qmp_phy_cfg {
int lanes;
+ const struct qmp_usb_offsets *offsets;
+
/* Init sequence for PHY blocks - serdes, tx, rx, pcs */
const struct qmp_phy_init_tbl *serdes_tbl;
int serdes_tbl_num;
/* array of registers with different offsets */
const unsigned int *regs;
- unsigned int start_ctrl;
- unsigned int pwrdn_ctrl;
- /* bit offset of PHYSTATUS in QPHY_PCS_STATUS register */
- unsigned int phy_status;
-
/* true, if PHY needs delay after POWER_DOWN */
bool has_pwrdn_delay;
- /* power_down delay in usec */
- int pwrdn_delay_min;
- int pwrdn_delay_max;
/* true, if PHY has a separate DP_COM control block */
bool has_phy_dp_com_ctrl;
unsigned int pcs_usb_offset;
};
-/**
- * struct qmp_phy - per-lane phy descriptor
- *
- * @phy: generic phy
- * @cfg: phy specific configuration
- * @serdes: iomapped memory space for phy's serdes (i.e. PLL)
- * @tx: iomapped memory space for lane's tx
- * @rx: iomapped memory space for lane's rx
- * @pcs: iomapped memory space for lane's pcs
- * @tx2: iomapped memory space for second lane's tx (in dual lane PHYs)
- * @rx2: iomapped memory space for second lane's rx (in dual lane PHYs)
- * @pcs_misc: iomapped memory space for lane's pcs_misc
- * @pcs_usb: iomapped memory space for lane's pcs_usb
- * @pipe_clk: pipe clock
- * @qmp: QMP phy to which this lane belongs
- * @mode: current PHY mode
- */
-struct qmp_phy {
- struct phy *phy;
+struct qmp_usb {
+ struct device *dev;
+
const struct qmp_phy_cfg *cfg;
+
void __iomem *serdes;
+ void __iomem *pcs;
+ void __iomem *pcs_misc;
+ void __iomem *pcs_usb;
void __iomem *tx;
void __iomem *rx;
- void __iomem *pcs;
void __iomem *tx2;
void __iomem *rx2;
- void __iomem *pcs_misc;
- void __iomem *pcs_usb;
- struct clk *pipe_clk;
- struct qcom_qmp *qmp;
- enum phy_mode mode;
-};
-/**
- * struct qcom_qmp - structure holding QMP phy block attributes
- *
- * @dev: device
- * @dp_com: iomapped memory space for phy's dp_com control block
- *
- * @clks: array of clocks required by phy
- * @resets: array of resets required by phy
- * @vregs: regulator supplies bulk data
- *
- * @phys: array of per-lane phy descriptors
- */
-struct qcom_qmp {
- struct device *dev;
void __iomem *dp_com;
+ struct clk *pipe_clk;
struct clk_bulk_data *clks;
struct reset_control_bulk_data *resets;
struct regulator_bulk_data *vregs;
- struct qmp_phy **phys;
+ enum phy_mode mode;
+
+ struct phy *phy;
+
+ struct clk_fixed_rate pipe_clk_fixed;
};
static inline void qphy_setbits(void __iomem *base, u32 offset, u32 val)
};
static const char * const qmp_v4_phy_clk_l[] = {
+ "aux", "ref", "com_aux",
+};
+
+static const char * const qmp_v4_ref_phy_clk_l[] = {
"aux", "ref_clk_src", "ref", "com_aux",
};
"vdda-phy", "vdda-pll",
};
+static const struct qmp_usb_offsets qmp_usb_offsets_v5 = {
+ .serdes = 0,
+ .pcs = 0x0200,
+ .pcs_usb = 0x1200,
+ .tx = 0x0e00,
+ .rx = 0x1000,
+};
+
static const struct qmp_phy_cfg ipq8074_usb3phy_cfg = {
.lanes = 1,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
- .regs = usb3phy_regs_layout,
-
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
+ .regs = qmp_v3_usb3phy_regs_layout,
};
static const struct qmp_phy_cfg msm8996_usb3phy_cfg = {
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = usb3phy_regs_layout,
-
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
};
static const struct qmp_phy_cfg qmp_v3_usb3phy_cfg = {
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = qmp_v3_usb3phy_regs_layout,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
-
.has_phy_dp_com_ctrl = true,
};
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = qmp_v3_usb3phy_regs_layout,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
-
.has_phy_dp_com_ctrl = true,
};
static const struct qmp_phy_cfg sc8280xp_usb3_uniphy_cfg = {
.lanes = 1,
+ .offsets = &qmp_usb_offsets_v5,
+
.serdes_tbl = sc8280xp_usb3_uniphy_serdes_tbl,
.serdes_tbl_num = ARRAY_SIZE(sc8280xp_usb3_uniphy_serdes_tbl),
.tx_tbl = sc8280xp_usb3_uniphy_tx_tbl,
.pcs_tbl_num = ARRAY_SIZE(sc8280xp_usb3_uniphy_pcs_tbl),
.clk_list = qmp_v4_phy_clk_l,
.num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l),
- .reset_list = msm8996_usb3phy_reset_l,
- .num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
+ .reset_list = qcm2290_usb3phy_reset_l,
+ .num_resets = ARRAY_SIZE(qcm2290_usb3phy_reset_l),
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = qmp_v4_usb3phy_regs_layout,
-
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
- .has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
};
static const struct qmp_phy_cfg qmp_v3_usb3_uniphy_cfg = {
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = qmp_v3_usb3phy_regs_layout,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
};
static const struct qmp_phy_cfg msm8998_usb3phy_cfg = {
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = qmp_v3_usb3phy_regs_layout,
-
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
};
static const struct qmp_phy_cfg sm8150_usb3phy_cfg = {
.pcs_tbl_num = ARRAY_SIZE(sm8150_usb3_pcs_tbl),
.pcs_usb_tbl = sm8150_usb3_pcs_usb_tbl,
.pcs_usb_tbl_num = ARRAY_SIZE(sm8150_usb3_pcs_usb_tbl),
- .clk_list = qmp_v4_phy_clk_l,
- .num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l),
+ .clk_list = qmp_v4_ref_phy_clk_l,
+ .num_clks = ARRAY_SIZE(qmp_v4_ref_phy_clk_l),
.reset_list = msm8996_usb3phy_reset_l,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
.vreg_list = qmp_phy_vreg_l,
.regs = qmp_v4_usb3phy_regs_layout,
.pcs_usb_offset = 0x300,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
-
.has_phy_dp_com_ctrl = true,
};
.pcs_tbl_num = ARRAY_SIZE(sm8150_usb3_uniphy_pcs_tbl),
.pcs_usb_tbl = sm8150_usb3_uniphy_pcs_usb_tbl,
.pcs_usb_tbl_num = ARRAY_SIZE(sm8150_usb3_uniphy_pcs_usb_tbl),
- .clk_list = qmp_v4_phy_clk_l,
- .num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l),
+ .clk_list = qmp_v4_ref_phy_clk_l,
+ .num_clks = ARRAY_SIZE(qmp_v4_ref_phy_clk_l),
.reset_list = msm8996_usb3phy_reset_l,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
.vreg_list = qmp_phy_vreg_l,
.regs = qmp_v4_usb3phy_regs_layout,
.pcs_usb_offset = 0x600,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
};
static const struct qmp_phy_cfg sm8250_usb3phy_cfg = {
.regs = qmp_v4_usb3phy_regs_layout,
.pcs_usb_offset = 0x300,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
-
.has_phy_dp_com_ctrl = true,
};
.pcs_tbl_num = ARRAY_SIZE(sm8250_usb3_uniphy_pcs_tbl),
.pcs_usb_tbl = sm8250_usb3_uniphy_pcs_usb_tbl,
.pcs_usb_tbl_num = ARRAY_SIZE(sm8250_usb3_uniphy_pcs_usb_tbl),
- .clk_list = qmp_v4_phy_clk_l,
- .num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l),
+ .clk_list = qmp_v4_ref_phy_clk_l,
+ .num_clks = ARRAY_SIZE(qmp_v4_ref_phy_clk_l),
.reset_list = msm8996_usb3phy_reset_l,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
.vreg_list = qmp_phy_vreg_l,
.regs = qmp_v4_usb3phy_regs_layout,
.pcs_usb_offset = 0x600,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
};
static const struct qmp_phy_cfg sdx55_usb3_uniphy_cfg = {
.regs = qmp_v4_usb3phy_regs_layout,
.pcs_usb_offset = 0x600,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
};
static const struct qmp_phy_cfg sdx65_usb3_uniphy_cfg = {
.regs = qmp_v4_usb3phy_regs_layout,
.pcs_usb_offset = 0x1000,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
};
static const struct qmp_phy_cfg sm8350_usb3phy_cfg = {
.regs = qmp_v4_usb3phy_regs_layout,
.pcs_usb_offset = 0x300,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
-
.has_phy_dp_com_ctrl = true,
};
.pcs_tbl_num = ARRAY_SIZE(sm8350_usb3_uniphy_pcs_tbl),
.pcs_usb_tbl = sm8350_usb3_uniphy_pcs_usb_tbl,
.pcs_usb_tbl_num = ARRAY_SIZE(sm8350_usb3_uniphy_pcs_usb_tbl),
- .clk_list = qmp_v4_phy_clk_l,
- .num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l),
+ .clk_list = qmp_v4_ref_phy_clk_l,
+ .num_clks = ARRAY_SIZE(qmp_v4_ref_phy_clk_l),
.reset_list = msm8996_usb3phy_reset_l,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
.vreg_list = qmp_phy_vreg_l,
.regs = qmp_v4_usb3phy_regs_layout,
.pcs_usb_offset = 0x1000,
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
-
.has_pwrdn_delay = true,
- .pwrdn_delay_min = POWER_DOWN_DELAY_US_MIN,
- .pwrdn_delay_max = POWER_DOWN_DELAY_US_MAX,
};
static const struct qmp_phy_cfg qcm2290_usb3phy_cfg = {
.vreg_list = qmp_phy_vreg_l,
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
.regs = qcm2290_usb3phy_regs_layout,
-
- .start_ctrl = SERDES_START | PCS_START,
- .pwrdn_ctrl = SW_PWRDN,
- .phy_status = PHYSTATUS,
};
static void qmp_usb_configure_lane(void __iomem *base,
- const unsigned int *regs,
const struct qmp_phy_init_tbl tbl[],
int num,
u8 lane_mask)
if (!(t->lane_mask & lane_mask))
continue;
- if (t->in_layout)
- writel(t->val, base + regs[t->offset]);
- else
- writel(t->val, base + t->offset);
+ writel(t->val, base + t->offset);
}
}
static void qmp_usb_configure(void __iomem *base,
- const unsigned int *regs,
const struct qmp_phy_init_tbl tbl[],
int num)
{
- qmp_usb_configure_lane(base, regs, tbl, num, 0xff);
+ qmp_usb_configure_lane(base, tbl, num, 0xff);
}
-static int qmp_usb_serdes_init(struct qmp_phy *qphy)
+static int qmp_usb_serdes_init(struct qmp_usb *qmp)
{
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *serdes = qphy->serdes;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *serdes = qmp->serdes;
const struct qmp_phy_init_tbl *serdes_tbl = cfg->serdes_tbl;
int serdes_tbl_num = cfg->serdes_tbl_num;
- qmp_usb_configure(serdes, cfg->regs, serdes_tbl, serdes_tbl_num);
+ qmp_usb_configure(serdes, serdes_tbl, serdes_tbl_num);
return 0;
}
static int qmp_usb_init(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *pcs = qphy->pcs;
+ struct qmp_usb *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *pcs = qmp->pcs;
void __iomem *dp_com = qmp->dp_com;
int ret;
- /* turn on regulator supplies */
ret = regulator_bulk_enable(cfg->num_vregs, qmp->vregs);
if (ret) {
dev_err(qmp->dev, "failed to enable regulators, err=%d\n", ret);
qphy_clrbits(dp_com, QPHY_V3_DP_COM_SW_RESET, SW_RESET);
}
- if (cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL])
- qphy_setbits(pcs,
- cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
- cfg->pwrdn_ctrl);
- else
- qphy_setbits(pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
- cfg->pwrdn_ctrl);
+ qphy_setbits(pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL], SW_PWRDN);
return 0;
static int qmp_usb_exit(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_usb *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
reset_control_bulk_assert(cfg->num_resets, qmp->resets);
static int qmp_usb_power_on(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- struct qcom_qmp *qmp = qphy->qmp;
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *tx = qphy->tx;
- void __iomem *rx = qphy->rx;
- void __iomem *pcs = qphy->pcs;
+ struct qmp_usb *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *tx = qmp->tx;
+ void __iomem *rx = qmp->rx;
+ void __iomem *pcs = qmp->pcs;
void __iomem *status;
- unsigned int mask, val, ready;
+ unsigned int val;
int ret;
- qmp_usb_serdes_init(qphy);
+ qmp_usb_serdes_init(qmp);
- ret = clk_prepare_enable(qphy->pipe_clk);
+ ret = clk_prepare_enable(qmp->pipe_clk);
if (ret) {
dev_err(qmp->dev, "pipe_clk enable failed err=%d\n", ret);
return ret;
}
/* Tx, Rx, and PCS configurations */
- qmp_usb_configure_lane(tx, cfg->regs, cfg->tx_tbl, cfg->tx_tbl_num, 1);
-
- if (cfg->lanes >= 2) {
- qmp_usb_configure_lane(qphy->tx2, cfg->regs,
- cfg->tx_tbl, cfg->tx_tbl_num, 2);
- }
-
- qmp_usb_configure_lane(rx, cfg->regs, cfg->rx_tbl, cfg->rx_tbl_num, 1);
+ qmp_usb_configure_lane(tx, cfg->tx_tbl, cfg->tx_tbl_num, 1);
+ qmp_usb_configure_lane(rx, cfg->rx_tbl, cfg->rx_tbl_num, 1);
if (cfg->lanes >= 2) {
- qmp_usb_configure_lane(qphy->rx2, cfg->regs,
- cfg->rx_tbl, cfg->rx_tbl_num, 2);
+ qmp_usb_configure_lane(qmp->tx2, cfg->tx_tbl, cfg->tx_tbl_num, 2);
+ qmp_usb_configure_lane(qmp->rx2, cfg->rx_tbl, cfg->rx_tbl_num, 2);
}
- /* Configure link rate, swing, etc. */
- qmp_usb_configure(pcs, cfg->regs, cfg->pcs_tbl, cfg->pcs_tbl_num);
+ qmp_usb_configure(pcs, cfg->pcs_tbl, cfg->pcs_tbl_num);
if (cfg->has_pwrdn_delay)
- usleep_range(cfg->pwrdn_delay_min, cfg->pwrdn_delay_max);
+ usleep_range(10, 20);
/* Pull PHY out of reset state */
qphy_clrbits(pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
/* start SerDes and Phy-Coding-Sublayer */
- qphy_setbits(pcs, cfg->regs[QPHY_START_CTRL], cfg->start_ctrl);
+ qphy_setbits(pcs, cfg->regs[QPHY_START_CTRL], SERDES_START | PCS_START);
status = pcs + cfg->regs[QPHY_PCS_STATUS];
- mask = cfg->phy_status;
- ready = 0;
-
- ret = readl_poll_timeout(status, val, (val & mask) == ready, 10,
+ ret = readl_poll_timeout(status, val, !(val & PHYSTATUS), 200,
PHY_INIT_COMPLETE_TIMEOUT);
if (ret) {
dev_err(qmp->dev, "phy initialization timed-out\n");
return 0;
err_disable_pipe_clk:
- clk_disable_unprepare(qphy->pipe_clk);
+ clk_disable_unprepare(qmp->pipe_clk);
return ret;
}
static int qmp_usb_power_off(struct phy *phy)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_usb *qmp = phy_get_drvdata(phy);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
- clk_disable_unprepare(qphy->pipe_clk);
+ clk_disable_unprepare(qmp->pipe_clk);
/* PHY reset */
- qphy_setbits(qphy->pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
+ qphy_setbits(qmp->pcs, cfg->regs[QPHY_SW_RESET], SW_RESET);
/* stop SerDes and Phy-Coding-Sublayer */
- qphy_clrbits(qphy->pcs, cfg->regs[QPHY_START_CTRL], cfg->start_ctrl);
+ qphy_clrbits(qmp->pcs, cfg->regs[QPHY_START_CTRL],
+ SERDES_START | PCS_START);
/* Put PHY into POWER DOWN state: active low */
- if (cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL]) {
- qphy_clrbits(qphy->pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
- cfg->pwrdn_ctrl);
- } else {
- qphy_clrbits(qphy->pcs, QPHY_V2_PCS_POWER_DOWN_CONTROL,
- cfg->pwrdn_ctrl);
- }
+ qphy_clrbits(qmp->pcs, cfg->regs[QPHY_PCS_POWER_DOWN_CONTROL],
+ SW_PWRDN);
return 0;
}
static int qmp_usb_set_mode(struct phy *phy, enum phy_mode mode, int submode)
{
- struct qmp_phy *qphy = phy_get_drvdata(phy);
+ struct qmp_usb *qmp = phy_get_drvdata(phy);
- qphy->mode = mode;
+ qmp->mode = mode;
return 0;
}
-static void qmp_usb_enable_autonomous_mode(struct qmp_phy *qphy)
+static const struct phy_ops qmp_usb_phy_ops = {
+ .init = qmp_usb_enable,
+ .exit = qmp_usb_disable,
+ .set_mode = qmp_usb_set_mode,
+ .owner = THIS_MODULE,
+};
+
+static void qmp_usb_enable_autonomous_mode(struct qmp_usb *qmp)
{
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *pcs_usb = qphy->pcs_usb ?: qphy->pcs;
- void __iomem *pcs_misc = qphy->pcs_misc;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *pcs_usb = qmp->pcs_usb ?: qmp->pcs;
+ void __iomem *pcs_misc = qmp->pcs_misc;
u32 intr_mask;
- if (qphy->mode == PHY_MODE_USB_HOST_SS ||
- qphy->mode == PHY_MODE_USB_DEVICE_SS)
+ if (qmp->mode == PHY_MODE_USB_HOST_SS ||
+ qmp->mode == PHY_MODE_USB_DEVICE_SS)
intr_mask = ARCVR_DTCT_EN | ALFPS_DTCT_EN;
else
intr_mask = ARCVR_DTCT_EN | ARCVR_DTCT_EVENT_SEL;
qphy_clrbits(pcs_misc, QPHY_V3_PCS_MISC_CLAMP_ENABLE, CLAMP_EN);
}
-static void qmp_usb_disable_autonomous_mode(struct qmp_phy *qphy)
+static void qmp_usb_disable_autonomous_mode(struct qmp_usb *qmp)
{
- const struct qmp_phy_cfg *cfg = qphy->cfg;
- void __iomem *pcs_usb = qphy->pcs_usb ?: qphy->pcs;
- void __iomem *pcs_misc = qphy->pcs_misc;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ void __iomem *pcs_usb = qmp->pcs_usb ?: qmp->pcs;
+ void __iomem *pcs_misc = qmp->pcs_misc;
/* Disable i/o clamp_n on resume for normal mode */
if (pcs_misc)
static int __maybe_unused qmp_usb_runtime_suspend(struct device *dev)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
- struct qmp_phy *qphy = qmp->phys[0];
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_usb *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
- dev_vdbg(dev, "Suspending QMP phy, mode:%d\n", qphy->mode);
+ dev_vdbg(dev, "Suspending QMP phy, mode:%d\n", qmp->mode);
- if (!qphy->phy->init_count) {
+ if (!qmp->phy->init_count) {
dev_vdbg(dev, "PHY not initialized, bailing out\n");
return 0;
}
- qmp_usb_enable_autonomous_mode(qphy);
+ qmp_usb_enable_autonomous_mode(qmp);
- clk_disable_unprepare(qphy->pipe_clk);
+ clk_disable_unprepare(qmp->pipe_clk);
clk_bulk_disable_unprepare(cfg->num_clks, qmp->clks);
return 0;
static int __maybe_unused qmp_usb_runtime_resume(struct device *dev)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
- struct qmp_phy *qphy = qmp->phys[0];
- const struct qmp_phy_cfg *cfg = qphy->cfg;
+ struct qmp_usb *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
int ret = 0;
- dev_vdbg(dev, "Resuming QMP phy, mode:%d\n", qphy->mode);
+ dev_vdbg(dev, "Resuming QMP phy, mode:%d\n", qmp->mode);
- if (!qphy->phy->init_count) {
+ if (!qmp->phy->init_count) {
dev_vdbg(dev, "PHY not initialized, bailing out\n");
return 0;
}
if (ret)
return ret;
- ret = clk_prepare_enable(qphy->pipe_clk);
+ ret = clk_prepare_enable(qmp->pipe_clk);
if (ret) {
dev_err(dev, "pipe_clk enable failed, err=%d\n", ret);
clk_bulk_disable_unprepare(cfg->num_clks, qmp->clks);
return ret;
}
- qmp_usb_disable_autonomous_mode(qphy);
+ qmp_usb_disable_autonomous_mode(qmp);
return 0;
}
-static int qmp_usb_vreg_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static const struct dev_pm_ops qmp_usb_pm_ops = {
+ SET_RUNTIME_PM_OPS(qmp_usb_runtime_suspend,
+ qmp_usb_runtime_resume, NULL)
+};
+
+static int qmp_usb_vreg_init(struct qmp_usb *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int num = cfg->num_vregs;
int i;
return devm_regulator_bulk_get(dev, num, qmp->vregs);
}
-static int qmp_usb_reset_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static int qmp_usb_reset_init(struct qmp_usb *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int i;
int ret;
return 0;
}
-static int qmp_usb_clk_init(struct device *dev, const struct qmp_phy_cfg *cfg)
+static int qmp_usb_clk_init(struct qmp_usb *qmp)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
int num = cfg->num_clks;
int i;
* clk | +-------+ | +-----+
* +---------------+
*/
-static int phy_pipe_clk_register(struct qcom_qmp *qmp, struct device_node *np)
+static int phy_pipe_clk_register(struct qmp_usb *qmp, struct device_node *np)
{
- struct clk_fixed_rate *fixed;
+ struct clk_fixed_rate *fixed = &qmp->pipe_clk_fixed;
struct clk_init_data init = { };
int ret;
return ret;
}
- fixed = devm_kzalloc(qmp->dev, sizeof(*fixed), GFP_KERNEL);
- if (!fixed)
- return -ENOMEM;
-
init.ops = &clk_fixed_rate_ops;
/* controllers using QMP phys use 125MHz pipe clock interface */
return devm_add_action_or_reset(qmp->dev, phy_clk_release_provider, np);
}
-static const struct phy_ops qmp_usb_ops = {
- .init = qmp_usb_enable,
- .exit = qmp_usb_disable,
- .set_mode = qmp_usb_set_mode,
- .owner = THIS_MODULE,
-};
-
static void __iomem *qmp_usb_iomap(struct device *dev, struct device_node *np,
int index, bool exclusive)
{
return devm_of_iomap(dev, np, index, NULL);
}
-static
-int qmp_usb_create(struct device *dev, struct device_node *np, int id,
- void __iomem *serdes, const struct qmp_phy_cfg *cfg)
+static int qmp_usb_parse_dt_legacy(struct qmp_usb *qmp, struct device_node *np)
{
- struct qcom_qmp *qmp = dev_get_drvdata(dev);
- struct phy *generic_phy;
- struct qmp_phy *qphy;
+ struct platform_device *pdev = to_platform_device(qmp->dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ struct device *dev = qmp->dev;
bool exclusive = true;
- int ret;
+
+ qmp->serdes = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(qmp->serdes))
+ return PTR_ERR(qmp->serdes);
+
+ if (cfg->has_phy_dp_com_ctrl) {
+ qmp->dp_com = devm_platform_ioremap_resource(pdev, 1);
+ if (IS_ERR(qmp->dp_com))
+ return PTR_ERR(qmp->dp_com);
+ }
/*
* FIXME: These bindings should be fixed to not rely on overlapping
if (of_device_is_compatible(dev->of_node, "qcom,sm8350-qmp-usb3-uni-phy"))
exclusive = false;
- qphy = devm_kzalloc(dev, sizeof(*qphy), GFP_KERNEL);
- if (!qphy)
- return -ENOMEM;
-
- qphy->cfg = cfg;
- qphy->serdes = serdes;
/*
- * Get memory resources for each phy lane:
+ * Get memory resources for the PHY:
* Resources are indexed as: tx -> 0; rx -> 1; pcs -> 2.
* For dual lane PHYs: tx2 -> 3, rx2 -> 4, pcs_misc (optional) -> 5
* For single lane PHYs: pcs_misc (optional) -> 3.
*/
- qphy->tx = devm_of_iomap(dev, np, 0, NULL);
- if (IS_ERR(qphy->tx))
- return PTR_ERR(qphy->tx);
+ qmp->tx = devm_of_iomap(dev, np, 0, NULL);
+ if (IS_ERR(qmp->tx))
+ return PTR_ERR(qmp->tx);
- qphy->rx = devm_of_iomap(dev, np, 1, NULL);
- if (IS_ERR(qphy->rx))
- return PTR_ERR(qphy->rx);
+ qmp->rx = devm_of_iomap(dev, np, 1, NULL);
+ if (IS_ERR(qmp->rx))
+ return PTR_ERR(qmp->rx);
- qphy->pcs = qmp_usb_iomap(dev, np, 2, exclusive);
- if (IS_ERR(qphy->pcs))
- return PTR_ERR(qphy->pcs);
+ qmp->pcs = qmp_usb_iomap(dev, np, 2, exclusive);
+ if (IS_ERR(qmp->pcs))
+ return PTR_ERR(qmp->pcs);
if (cfg->pcs_usb_offset)
- qphy->pcs_usb = qphy->pcs + cfg->pcs_usb_offset;
+ qmp->pcs_usb = qmp->pcs + cfg->pcs_usb_offset;
if (cfg->lanes >= 2) {
- qphy->tx2 = devm_of_iomap(dev, np, 3, NULL);
- if (IS_ERR(qphy->tx2))
- return PTR_ERR(qphy->tx2);
+ qmp->tx2 = devm_of_iomap(dev, np, 3, NULL);
+ if (IS_ERR(qmp->tx2))
+ return PTR_ERR(qmp->tx2);
- qphy->rx2 = devm_of_iomap(dev, np, 4, NULL);
- if (IS_ERR(qphy->rx2))
- return PTR_ERR(qphy->rx2);
+ qmp->rx2 = devm_of_iomap(dev, np, 4, NULL);
+ if (IS_ERR(qmp->rx2))
+ return PTR_ERR(qmp->rx2);
- qphy->pcs_misc = devm_of_iomap(dev, np, 5, NULL);
+ qmp->pcs_misc = devm_of_iomap(dev, np, 5, NULL);
} else {
- qphy->pcs_misc = devm_of_iomap(dev, np, 3, NULL);
+ qmp->pcs_misc = devm_of_iomap(dev, np, 3, NULL);
}
- if (IS_ERR(qphy->pcs_misc)) {
+ if (IS_ERR(qmp->pcs_misc)) {
dev_vdbg(dev, "PHY pcs_misc-reg not used\n");
- qphy->pcs_misc = NULL;
+ qmp->pcs_misc = NULL;
}
- qphy->pipe_clk = devm_get_clk_from_child(dev, np, NULL);
- if (IS_ERR(qphy->pipe_clk)) {
- return dev_err_probe(dev, PTR_ERR(qphy->pipe_clk),
- "failed to get lane%d pipe clock\n", id);
+ qmp->pipe_clk = devm_get_clk_from_child(dev, np, NULL);
+ if (IS_ERR(qmp->pipe_clk)) {
+ return dev_err_probe(dev, PTR_ERR(qmp->pipe_clk),
+ "failed to get pipe clock\n");
+ }
+
+ return 0;
+}
+
+static int qmp_usb_parse_dt(struct qmp_usb *qmp)
+{
+ struct platform_device *pdev = to_platform_device(qmp->dev);
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ const struct qmp_usb_offsets *offs = cfg->offsets;
+ struct device *dev = qmp->dev;
+ void __iomem *base;
+
+ if (!offs)
+ return -EINVAL;
+
+ base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+
+ qmp->serdes = base + offs->serdes;
+ qmp->pcs = base + offs->pcs;
+ qmp->pcs_usb = base + offs->pcs_usb;
+ qmp->tx = base + offs->tx;
+ qmp->rx = base + offs->rx;
+
+ qmp->pipe_clk = devm_clk_get(dev, "pipe");
+ if (IS_ERR(qmp->pipe_clk)) {
+ return dev_err_probe(dev, PTR_ERR(qmp->pipe_clk),
+ "failed to get pipe clock\n");
}
- generic_phy = devm_phy_create(dev, np, &qmp_usb_ops);
- if (IS_ERR(generic_phy)) {
- ret = PTR_ERR(generic_phy);
- dev_err(dev, "failed to create qphy %d\n", ret);
+ return 0;
+}
+
+static int qmp_usb_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct phy_provider *phy_provider;
+ struct device_node *np;
+ struct qmp_usb *qmp;
+ int ret;
+
+ qmp = devm_kzalloc(dev, sizeof(*qmp), GFP_KERNEL);
+ if (!qmp)
+ return -ENOMEM;
+
+ qmp->dev = dev;
+
+ qmp->cfg = of_device_get_match_data(dev);
+ if (!qmp->cfg)
+ return -EINVAL;
+
+ ret = qmp_usb_clk_init(qmp);
+ if (ret)
return ret;
+
+ ret = qmp_usb_reset_init(qmp);
+ if (ret)
+ return ret;
+
+ ret = qmp_usb_vreg_init(qmp);
+ if (ret)
+ return ret;
+
+ /* Check for legacy binding with child node. */
+ np = of_get_next_available_child(dev->of_node, NULL);
+ if (np) {
+ ret = qmp_usb_parse_dt_legacy(qmp, np);
+ } else {
+ np = of_node_get(dev->of_node);
+ ret = qmp_usb_parse_dt(qmp);
+ }
+ if (ret)
+ goto err_node_put;
+
+ pm_runtime_set_active(dev);
+ ret = devm_pm_runtime_enable(dev);
+ if (ret)
+ goto err_node_put;
+ /*
+ * Prevent runtime pm from being ON by default. Users can enable
+ * it using power/control in sysfs.
+ */
+ pm_runtime_forbid(dev);
+
+ ret = phy_pipe_clk_register(qmp, np);
+ if (ret)
+ goto err_node_put;
+
+ qmp->phy = devm_phy_create(dev, np, &qmp_usb_phy_ops);
+ if (IS_ERR(qmp->phy)) {
+ ret = PTR_ERR(qmp->phy);
+ dev_err(dev, "failed to create PHY: %d\n", ret);
+ goto err_node_put;
}
- qphy->phy = generic_phy;
- qphy->qmp = qmp;
- qmp->phys[id] = qphy;
- phy_set_drvdata(generic_phy, qphy);
+ phy_set_drvdata(qmp->phy, qmp);
- return 0;
+ of_node_put(np);
+
+ phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
+
+ return PTR_ERR_OR_ZERO(phy_provider);
+
+err_node_put:
+ of_node_put(np);
+ return ret;
}
static const struct of_device_id qmp_usb_of_match_table[] = {
{
+ .compatible = "qcom,ipq6018-qmp-usb3-phy",
+ .data = &ipq8074_usb3phy_cfg,
+ }, {
.compatible = "qcom,ipq8074-qmp-usb3-phy",
.data = &ipq8074_usb3phy_cfg,
}, {
.compatible = "qcom,msm8996-qmp-usb3-phy",
.data = &msm8996_usb3phy_cfg,
}, {
- .compatible = "qcom,ipq6018-qmp-usb3-phy",
- .data = &ipq8074_usb3phy_cfg,
+ .compatible = "qcom,msm8998-qmp-usb3-phy",
+ .data = &msm8998_usb3phy_cfg,
+ }, {
+ .compatible = "qcom,qcm2290-qmp-usb3-phy",
+ .data = &qcm2290_usb3phy_cfg,
}, {
.compatible = "qcom,sc7180-qmp-usb3-phy",
.data = &sc7180_usb3phy_cfg,
.compatible = "qcom,sdm845-qmp-usb3-uni-phy",
.data = &qmp_v3_usb3_uniphy_cfg,
}, {
- .compatible = "qcom,msm8998-qmp-usb3-phy",
- .data = &msm8998_usb3phy_cfg,
+ .compatible = "qcom,sdx55-qmp-usb3-uni-phy",
+ .data = &sdx55_usb3_uniphy_cfg,
+ }, {
+ .compatible = "qcom,sdx65-qmp-usb3-uni-phy",
+ .data = &sdx65_usb3_uniphy_cfg,
}, {
.compatible = "qcom,sm8150-qmp-usb3-phy",
.data = &sm8150_usb3phy_cfg,
.compatible = "qcom,sm8250-qmp-usb3-uni-phy",
.data = &sm8250_usb3_uniphy_cfg,
}, {
- .compatible = "qcom,sdx55-qmp-usb3-uni-phy",
- .data = &sdx55_usb3_uniphy_cfg,
- }, {
- .compatible = "qcom,sdx65-qmp-usb3-uni-phy",
- .data = &sdx65_usb3_uniphy_cfg,
- }, {
.compatible = "qcom,sm8350-qmp-usb3-phy",
.data = &sm8350_usb3phy_cfg,
}, {
}, {
.compatible = "qcom,sm8450-qmp-usb3-phy",
.data = &sm8350_usb3phy_cfg,
- }, {
- .compatible = "qcom,qcm2290-qmp-usb3-phy",
- .data = &qcm2290_usb3phy_cfg,
},
{ },
};
MODULE_DEVICE_TABLE(of, qmp_usb_of_match_table);
-static const struct dev_pm_ops qmp_usb_pm_ops = {
- SET_RUNTIME_PM_OPS(qmp_usb_runtime_suspend,
- qmp_usb_runtime_resume, NULL)
-};
-
-static int qmp_usb_probe(struct platform_device *pdev)
-{
- struct qcom_qmp *qmp;
- struct device *dev = &pdev->dev;
- struct device_node *child;
- struct phy_provider *phy_provider;
- void __iomem *serdes;
- const struct qmp_phy_cfg *cfg = NULL;
- int num, id;
- int ret;
-
- qmp = devm_kzalloc(dev, sizeof(*qmp), GFP_KERNEL);
- if (!qmp)
- return -ENOMEM;
-
- qmp->dev = dev;
- dev_set_drvdata(dev, qmp);
-
- /* Get the specific init parameters of QMP phy */
- cfg = of_device_get_match_data(dev);
- if (!cfg)
- return -EINVAL;
-
- /* per PHY serdes; usually located at base address */
- serdes = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(serdes))
- return PTR_ERR(serdes);
-
- /* per PHY dp_com; if PHY has dp_com control block */
- if (cfg->has_phy_dp_com_ctrl) {
- qmp->dp_com = devm_platform_ioremap_resource(pdev, 1);
- if (IS_ERR(qmp->dp_com))
- return PTR_ERR(qmp->dp_com);
- }
-
- ret = qmp_usb_clk_init(dev, cfg);
- if (ret)
- return ret;
-
- ret = qmp_usb_reset_init(dev, cfg);
- if (ret)
- return ret;
-
- ret = qmp_usb_vreg_init(dev, cfg);
- if (ret)
- return dev_err_probe(dev, ret,
- "failed to get regulator supplies\n");
-
- num = of_get_available_child_count(dev->of_node);
- /* do we have a rogue child node ? */
- if (num > 1)
- return -EINVAL;
-
- qmp->phys = devm_kcalloc(dev, num, sizeof(*qmp->phys), GFP_KERNEL);
- if (!qmp->phys)
- return -ENOMEM;
-
- pm_runtime_set_active(dev);
- ret = devm_pm_runtime_enable(dev);
- if (ret)
- return ret;
- /*
- * Prevent runtime pm from being ON by default. Users can enable
- * it using power/control in sysfs.
- */
- pm_runtime_forbid(dev);
-
- id = 0;
- for_each_available_child_of_node(dev->of_node, child) {
- /* Create per-lane phy */
- ret = qmp_usb_create(dev, child, id, serdes, cfg);
- if (ret) {
- dev_err(dev, "failed to create lane%d phy, %d\n",
- id, ret);
- goto err_node_put;
- }
-
- /*
- * Register the pipe clock provided by phy.
- * See function description to see details of this pipe clock.
- */
- ret = phy_pipe_clk_register(qmp, child);
- if (ret) {
- dev_err(qmp->dev,
- "failed to register pipe clock source\n");
- goto err_node_put;
- }
-
- id++;
- }
-
- phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
-
- return PTR_ERR_OR_ZERO(phy_provider);
-
-err_node_put:
- of_node_put(child);
- return ret;
-}
-
static struct platform_driver qmp_usb_driver = {
.probe = qmp_usb_probe,
.driver = {
#include "phy-qcom-qmp-pcs-pcie-v4_20.h"
#include "phy-qcom-qmp-pcs-v5.h"
+#include "phy-qcom-qmp-pcs-v5_20.h"
#include "phy-qcom-qmp-pcs-pcie-v5.h"
#include "phy-qcom-qmp-pcs-usb-v5.h"
#include "phy-qcom-qmp-pcs-ufs-v5.h"
#
# Phy drivers for Renesas platforms
#
+# NOTE: Please sorted config names alphabetically.
+config PHY_R8A779F0_ETHERNET_SERDES
+ tristate "Renesas R-Car S4-8 Ethernet SERDES driver"
+ depends on ARCH_RENESAS || COMPILE_TEST
+ select GENERIC_PHY
+ help
+ Support for Ethernet SERDES found on Renesas R-Car S4-8 SoCs.
+
config PHY_RCAR_GEN2
tristate "Renesas R-Car generation 2 USB PHY driver"
depends on ARCH_RENESAS
# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_PHY_R8A779F0_ETHERNET_SERDES) += r8a779f0-ether-serdes.o
obj-$(CONFIG_PHY_RCAR_GEN2) += phy-rcar-gen2.o
obj-$(CONFIG_PHY_RCAR_GEN3_PCIE) += phy-rcar-gen3-pcie.o
obj-$(CONFIG_PHY_RCAR_GEN3_USB2) += phy-rcar-gen3-usb2.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Renesas Ethernet SERDES device driver
+ *
+ * Copyright (C) 2022 Renesas Electronics Corporation
+ */
+
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/iopoll.h>
+#include <linux/kernel.h>
+#include <linux/phy.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+
+#define R8A779F0_ETH_SERDES_NUM 3
+#define R8A779F0_ETH_SERDES_OFFSET 0x0400
+#define R8A779F0_ETH_SERDES_BANK_SELECT 0x03fc
+#define R8A779F0_ETH_SERDES_TIMEOUT_US 100000
+#define R8A779F0_ETH_SERDES_NUM_RETRY_LINKUP 3
+#define R8A779F0_ETH_SERDES_NUM_RETRY_INIT 3
+
+struct r8a779f0_eth_serdes_drv_data;
+struct r8a779f0_eth_serdes_channel {
+ struct r8a779f0_eth_serdes_drv_data *dd;
+ struct phy *phy;
+ void __iomem *addr;
+ phy_interface_t phy_interface;
+ int speed;
+ int index;
+};
+
+struct r8a779f0_eth_serdes_drv_data {
+ void __iomem *addr;
+ struct platform_device *pdev;
+ struct reset_control *reset;
+ struct r8a779f0_eth_serdes_channel channel[R8A779F0_ETH_SERDES_NUM];
+ bool initialized;
+};
+
+/*
+ * The datasheet describes initialization procedure without any information
+ * about registers' name/bits. So, this is all black magic to initialize
+ * the hardware.
+ */
+static void r8a779f0_eth_serdes_write32(void __iomem *addr, u32 offs, u32 bank, u32 data)
+{
+ iowrite32(bank, addr + R8A779F0_ETH_SERDES_BANK_SELECT);
+ iowrite32(data, addr + offs);
+}
+
+static int
+r8a779f0_eth_serdes_reg_wait(struct r8a779f0_eth_serdes_channel *channel,
+ u32 offs, u32 bank, u32 mask, u32 expected)
+{
+ int ret;
+ u32 val;
+
+ iowrite32(bank, channel->addr + R8A779F0_ETH_SERDES_BANK_SELECT);
+
+ ret = readl_poll_timeout_atomic(channel->addr + offs, val,
+ (val & mask) == expected,
+ 1, R8A779F0_ETH_SERDES_TIMEOUT_US);
+ if (ret)
+ dev_dbg(&channel->phy->dev,
+ "%s: index %d, offs %x, bank %x, mask %x, expected %x\n",
+ __func__, channel->index, offs, bank, mask, expected);
+
+ return ret;
+}
+
+static int
+r8a779f0_eth_serdes_common_init_ram(struct r8a779f0_eth_serdes_drv_data *dd)
+{
+ struct r8a779f0_eth_serdes_channel *channel;
+ int i, ret;
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM; i++) {
+ channel = &dd->channel[i];
+ ret = r8a779f0_eth_serdes_reg_wait(channel, 0x026c, 0x180, BIT(0), 0x01);
+ if (ret)
+ return ret;
+ }
+
+ r8a779f0_eth_serdes_write32(dd->addr, 0x026c, 0x180, 0x03);
+
+ return ret;
+}
+
+static int
+r8a779f0_eth_serdes_common_setting(struct r8a779f0_eth_serdes_channel *channel)
+{
+ struct r8a779f0_eth_serdes_drv_data *dd = channel->dd;
+
+ switch (channel->phy_interface) {
+ case PHY_INTERFACE_MODE_SGMII:
+ r8a779f0_eth_serdes_write32(dd->addr, 0x0244, 0x180, 0x0097);
+ r8a779f0_eth_serdes_write32(dd->addr, 0x01d0, 0x180, 0x0060);
+ r8a779f0_eth_serdes_write32(dd->addr, 0x01d8, 0x180, 0x2200);
+ r8a779f0_eth_serdes_write32(dd->addr, 0x01d4, 0x180, 0x0000);
+ r8a779f0_eth_serdes_write32(dd->addr, 0x01e0, 0x180, 0x003d);
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static int
+r8a779f0_eth_serdes_chan_setting(struct r8a779f0_eth_serdes_channel *channel)
+{
+ int ret;
+
+ switch (channel->phy_interface) {
+ case PHY_INTERFACE_MODE_SGMII:
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0000, 0x380, 0x2000);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x01c0, 0x180, 0x0011);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0248, 0x180, 0x0540);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0258, 0x180, 0x0015);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0144, 0x180, 0x0100);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x01a0, 0x180, 0x0000);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x00d0, 0x180, 0x0002);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0150, 0x180, 0x0003);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x00c8, 0x180, 0x0100);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0148, 0x180, 0x0100);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0174, 0x180, 0x0000);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0160, 0x180, 0x0007);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x01ac, 0x180, 0x0000);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x00c4, 0x180, 0x0310);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x00c8, 0x380, 0x0101);
+ ret = r8a779f0_eth_serdes_reg_wait(channel, 0x00c8, 0x0180, BIT(0), 0);
+ if (ret)
+ return ret;
+
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0148, 0x180, 0x0101);
+ ret = r8a779f0_eth_serdes_reg_wait(channel, 0x0148, 0x0180, BIT(0), 0);
+ if (ret)
+ return ret;
+
+ r8a779f0_eth_serdes_write32(channel->addr, 0x00c4, 0x180, 0x1310);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x00d8, 0x180, 0x1800);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x00dc, 0x180, 0x0000);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x001c, 0x300, 0x0001);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0000, 0x380, 0x2100);
+ ret = r8a779f0_eth_serdes_reg_wait(channel, 0x0000, 0x0380, BIT(8), 0);
+ if (ret)
+ return ret;
+
+ if (channel->speed == 1000)
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0000, 0x1f00, 0x0140);
+ else if (channel->speed == 100)
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0000, 0x1f00, 0x2100);
+
+ /* For AN_ON */
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0004, 0x1f80, 0x0005);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0028, 0x1f80, 0x07a1);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0000, 0x1f80, 0x0208);
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static int
+r8a779f0_eth_serdes_chan_speed(struct r8a779f0_eth_serdes_channel *channel)
+{
+ int ret;
+
+ switch (channel->phy_interface) {
+ case PHY_INTERFACE_MODE_SGMII:
+ /* For AN_ON */
+ if (channel->speed == 1000)
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0000, 0x1f00, 0x1140);
+ else if (channel->speed == 100)
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0000, 0x1f00, 0x3100);
+ ret = r8a779f0_eth_serdes_reg_wait(channel, 0x0008, 0x1f80, BIT(0), 1);
+ if (ret)
+ return ret;
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0008, 0x1f80, 0x0000);
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+
+static int r8a779f0_eth_serdes_monitor_linkup(struct r8a779f0_eth_serdes_channel *channel)
+{
+ int i, ret;
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM_RETRY_LINKUP; i++) {
+ ret = r8a779f0_eth_serdes_reg_wait(channel, 0x0004, 0x300,
+ BIT(2), BIT(2));
+ if (!ret)
+ break;
+
+ /* restart */
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0144, 0x180, 0x0100);
+ udelay(1);
+ r8a779f0_eth_serdes_write32(channel->addr, 0x0144, 0x180, 0x0000);
+ }
+
+ return ret;
+}
+
+static int r8a779f0_eth_serdes_hw_init(struct r8a779f0_eth_serdes_channel *channel)
+{
+ struct r8a779f0_eth_serdes_drv_data *dd = channel->dd;
+ int i, ret;
+
+ if (dd->initialized)
+ return 0;
+
+ ret = r8a779f0_eth_serdes_common_init_ram(dd);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM; i++) {
+ ret = r8a779f0_eth_serdes_reg_wait(&dd->channel[i], 0x0000,
+ 0x300, BIT(15), 0);
+ if (ret)
+ return ret;
+ }
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM; i++)
+ r8a779f0_eth_serdes_write32(dd->channel[i].addr, 0x03d4, 0x380, 0x0443);
+
+ ret = r8a779f0_eth_serdes_common_setting(channel);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM; i++)
+ r8a779f0_eth_serdes_write32(dd->channel[i].addr, 0x03d0, 0x380, 0x0001);
+
+
+ r8a779f0_eth_serdes_write32(dd->addr, 0x0000, 0x380, 0x8000);
+
+ ret = r8a779f0_eth_serdes_common_init_ram(dd);
+ if (ret)
+ return ret;
+
+ ret = r8a779f0_eth_serdes_reg_wait(&dd->channel[0], 0x0000, 0x380, BIT(15), 0);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM; i++) {
+ ret = r8a779f0_eth_serdes_chan_setting(&dd->channel[i]);
+ if (ret)
+ return ret;
+ }
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM; i++) {
+ ret = r8a779f0_eth_serdes_chan_speed(&dd->channel[i]);
+ if (ret)
+ return ret;
+ }
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM; i++)
+ r8a779f0_eth_serdes_write32(dd->channel[i].addr, 0x03c0, 0x380, 0x0000);
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM; i++)
+ r8a779f0_eth_serdes_write32(dd->channel[i].addr, 0x03d0, 0x380, 0x0000);
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM; i++) {
+ ret = r8a779f0_eth_serdes_monitor_linkup(&dd->channel[i]);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int r8a779f0_eth_serdes_init(struct phy *p)
+{
+ struct r8a779f0_eth_serdes_channel *channel = phy_get_drvdata(p);
+ int i, ret;
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM_RETRY_INIT; i++) {
+ ret = r8a779f0_eth_serdes_hw_init(channel);
+ if (!ret) {
+ channel->dd->initialized = true;
+ break;
+ }
+ usleep_range(1000, 2000);
+ }
+
+ return ret;
+}
+
+static int r8a779f0_eth_serdes_set_mode(struct phy *p, enum phy_mode mode,
+ int submode)
+{
+ struct r8a779f0_eth_serdes_channel *channel = phy_get_drvdata(p);
+
+ if (mode != PHY_MODE_ETHERNET)
+ return -EOPNOTSUPP;
+
+ switch (submode) {
+ case PHY_INTERFACE_MODE_GMII:
+ case PHY_INTERFACE_MODE_SGMII:
+ case PHY_INTERFACE_MODE_USXGMII:
+ channel->phy_interface = submode;
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static int r8a779f0_eth_serdes_set_speed(struct phy *p, int speed)
+{
+ struct r8a779f0_eth_serdes_channel *channel = phy_get_drvdata(p);
+
+ channel->speed = speed;
+
+ return 0;
+}
+
+static const struct phy_ops r8a779f0_eth_serdes_ops = {
+ .init = r8a779f0_eth_serdes_init,
+ .set_mode = r8a779f0_eth_serdes_set_mode,
+ .set_speed = r8a779f0_eth_serdes_set_speed,
+};
+
+static struct phy *r8a779f0_eth_serdes_xlate(struct device *dev,
+ struct of_phandle_args *args)
+{
+ struct r8a779f0_eth_serdes_drv_data *dd = dev_get_drvdata(dev);
+
+ if (args->args[0] >= R8A779F0_ETH_SERDES_NUM)
+ return ERR_PTR(-ENODEV);
+
+ return dd->channel[args->args[0]].phy;
+}
+
+static const struct of_device_id r8a779f0_eth_serdes_of_table[] = {
+ { .compatible = "renesas,r8a779f0-ether-serdes", },
+ { }
+};
+MODULE_DEVICE_TABLE(of, r8a779f0_eth_serdes_of_table);
+
+static int r8a779f0_eth_serdes_probe(struct platform_device *pdev)
+{
+ struct r8a779f0_eth_serdes_drv_data *dd;
+ struct phy_provider *provider;
+ struct resource *res;
+ int i;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res) {
+ dev_err(&pdev->dev, "invalid resource\n");
+ return -EINVAL;
+ }
+
+ dd = devm_kzalloc(&pdev->dev, sizeof(*dd), GFP_KERNEL);
+ if (!dd)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, dd);
+ dd->pdev = pdev;
+ dd->addr = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(dd->addr))
+ return PTR_ERR(dd->addr);
+
+ dd->reset = devm_reset_control_get(&pdev->dev, NULL);
+ if (IS_ERR(dd->reset))
+ return PTR_ERR(dd->reset);
+
+ reset_control_reset(dd->reset);
+
+ for (i = 0; i < R8A779F0_ETH_SERDES_NUM; i++) {
+ struct r8a779f0_eth_serdes_channel *channel = &dd->channel[i];
+
+ channel->phy = devm_phy_create(&pdev->dev, NULL,
+ &r8a779f0_eth_serdes_ops);
+ if (IS_ERR(channel->phy))
+ return PTR_ERR(channel->phy);
+ channel->addr = dd->addr + R8A779F0_ETH_SERDES_OFFSET * i;
+ channel->dd = dd;
+ channel->index = i;
+ phy_set_drvdata(channel->phy, channel);
+ }
+
+ provider = devm_of_phy_provider_register(&pdev->dev,
+ r8a779f0_eth_serdes_xlate);
+ if (IS_ERR(provider))
+ return PTR_ERR(provider);
+
+ pm_runtime_enable(&pdev->dev);
+ pm_runtime_get_sync(&pdev->dev);
+
+ return 0;
+}
+
+static int r8a779f0_eth_serdes_remove(struct platform_device *pdev)
+{
+ pm_runtime_put(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
+
+ platform_set_drvdata(pdev, NULL);
+
+ return 0;
+}
+
+static struct platform_driver r8a779f0_eth_serdes_driver_platform = {
+ .probe = r8a779f0_eth_serdes_probe,
+ .remove = r8a779f0_eth_serdes_remove,
+ .driver = {
+ .name = "r8a779f0_eth_serdes",
+ .of_match_table = r8a779f0_eth_serdes_of_table,
+ }
+};
+module_platform_driver(r8a779f0_eth_serdes_driver_platform);
+MODULE_AUTHOR("Yoshihiro Shimoda");
+MODULE_DESCRIPTION("Renesas Ethernet SERDES device driver");
+MODULE_LICENSE("GPL");
#include <linux/phy/phy.h>
#define P2U_CONTROL_CMN 0x74
+#define P2U_CONTROL_CMN_ENABLE_L2_EXIT_RATE_CHANGE BIT(13)
#define P2U_CONTROL_CMN_SKP_SIZE_PROTECTION_EN BIT(20)
#define P2U_PERIODIC_EQ_CTRL_GEN3 0xc0
return 0;
}
+static int tegra_p2u_calibrate(struct phy *x)
+{
+ struct tegra_p2u *phy = phy_get_drvdata(x);
+ u32 val;
+
+ val = p2u_readl(phy, P2U_CONTROL_CMN);
+ val |= P2U_CONTROL_CMN_ENABLE_L2_EXIT_RATE_CHANGE;
+ p2u_writel(phy, val, P2U_CONTROL_CMN);
+
+ return 0;
+}
+
static const struct phy_ops ops = {
.power_on = tegra_p2u_power_on,
+ .calibrate = tegra_p2u_calibrate,
.owner = THIS_MODULE,
};
static const struct tegra_xusb_port_ops tegra124_usb3_port_ops = {
.release = tegra_xusb_usb3_port_release,
- .remove = tegra_xusb_usb3_port_remove,
.enable = tegra124_usb3_port_enable,
.disable = tegra124_usb3_port_disable,
.map = tegra124_usb3_port_map,
static const struct tegra_xusb_port_ops tegra186_usb3_port_ops = {
.release = tegra_xusb_usb3_port_release,
- .remove = tegra_xusb_usb3_port_remove,
.enable = tegra186_usb3_port_enable,
.disable = tegra186_usb3_port_disable,
.map = tegra186_usb3_port_map,
static const struct tegra_xusb_port_ops tegra210_usb3_port_ops = {
.release = tegra_xusb_usb3_port_release,
- .remove = tegra_xusb_usb3_port_remove,
.enable = tegra210_usb3_port_enable,
.disable = tegra210_usb3_port_disable,
.map = tegra210_usb3_port_map,
return -EINVAL;
}
- usb3->supply = regulator_get(&port->dev, "vbus");
- return PTR_ERR_OR_ZERO(usb3->supply);
+ return 0;
}
static int tegra_xusb_add_usb3_port(struct tegra_xusb_padctl *padctl,
kfree(usb3);
}
-void tegra_xusb_usb3_port_remove(struct tegra_xusb_port *port)
-{
- struct tegra_xusb_usb3_port *usb3 = to_usb3_port(port);
-
- regulator_put(usb3->supply);
-}
-
static void __tegra_xusb_remove_ports(struct tegra_xusb_padctl *padctl)
{
struct tegra_xusb_port *port, *tmp;
struct tegra_xusb_usb3_port {
struct tegra_xusb_port base;
- struct regulator *supply;
bool context_saved;
unsigned int port;
bool internal;
tegra_xusb_find_usb3_port(struct tegra_xusb_padctl *padctl,
unsigned int index);
void tegra_xusb_usb3_port_release(struct tegra_xusb_port *port);
-void tegra_xusb_usb3_port_remove(struct tegra_xusb_port *port);
struct tegra_xusb_port_ops {
void (*release)(struct tegra_xusb_port *port);
const struct reg_field (*regfields)[PHY_GMII_SEL_LAST];
bool use_of_data;
u64 extra_modes;
+ u32 num_qsgmii_main_ports;
};
struct phy_gmii_sel_priv {
.use_of_data = true,
.regfields = phy_gmii_sel_fields_am654,
.extra_modes = BIT(PHY_INTERFACE_MODE_QSGMII),
+ .num_ports = 4,
+ .num_qsgmii_main_ports = 1,
+};
+
+static const
+struct phy_gmii_sel_soc_data phy_gmii_sel_cpsw9g_soc_j721e = {
+ .use_of_data = true,
+ .regfields = phy_gmii_sel_fields_am654,
+ .extra_modes = BIT(PHY_INTERFACE_MODE_QSGMII),
+ .num_ports = 8,
+ .num_qsgmii_main_ports = 2,
};
static const struct of_device_id phy_gmii_sel_id_table[] = {
.compatible = "ti,j7200-cpsw5g-phy-gmii-sel",
.data = &phy_gmii_sel_cpsw5g_soc_j7200,
},
+ {
+ .compatible = "ti,j721e-cpsw9g-phy-gmii-sel",
+ .data = &phy_gmii_sel_cpsw9g_soc_j721e,
+ },
{}
};
MODULE_DEVICE_TABLE(of, phy_gmii_sel_id_table);
static int phy_gmii_sel_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
+ const struct phy_gmii_sel_soc_data *soc_data;
struct device_node *node = dev->of_node;
const struct of_device_id *of_id;
struct phy_gmii_sel_priv *priv;
u32 main_ports = 1;
int ret;
+ u32 i;
of_id = of_match_node(phy_gmii_sel_id_table, pdev->dev.of_node);
if (!of_id)
priv->dev = &pdev->dev;
priv->soc_data = of_id->data;
+ soc_data = priv->soc_data;
priv->num_ports = priv->soc_data->num_ports;
- of_property_read_u32(node, "ti,qsgmii-main-ports", &main_ports);
+ priv->qsgmii_main_ports = 0;
+
/*
- * Ensure that main_ports is within bounds. If the property
- * ti,qsgmii-main-ports is not mentioned, or the value mentioned
- * is out of bounds, default to 1.
+ * Based on the compatible, try to read the appropriate number of
+ * QSGMII main ports from the "ti,qsgmii-main-ports" property from
+ * the device-tree node.
*/
- if (main_ports < 1 || main_ports > 4)
- main_ports = 1;
- priv->qsgmii_main_ports = PHY_GMII_PORT(main_ports);
+ for (i = 0; i < soc_data->num_qsgmii_main_ports; i++) {
+ of_property_read_u32_index(node, "ti,qsgmii-main-ports", i, &main_ports);
+ /*
+ * Ensure that main_ports is within bounds.
+ */
+ if (main_ports < 1 || main_ports > soc_data->num_ports) {
+ dev_err(dev, "Invalid qsgmii main port provided\n");
+ return -EINVAL;
+ }
+ priv->qsgmii_main_ports |= PHY_GMII_PORT(main_ports);
+ }
priv->regmap = syscon_node_to_regmap(node->parent);
if (IS_ERR(priv->regmap)) {
static const struct reg_field phy_en_refclk = REG_FIELD(WIZ_SERDES_RST, 30, 30);
static const struct reg_field pll1_refclk_mux_sel =
REG_FIELD(WIZ_SERDES_RST, 29, 29);
+static const struct reg_field pll1_refclk_mux_sel_2 =
+ REG_FIELD(WIZ_SERDES_RST, 22, 23);
static const struct reg_field pll0_refclk_mux_sel =
REG_FIELD(WIZ_SERDES_RST, 28, 28);
+static const struct reg_field pll0_refclk_mux_sel_2 =
+ REG_FIELD(WIZ_SERDES_RST, 28, 29);
static const struct reg_field refclk_dig_sel_16g =
REG_FIELD(WIZ_SERDES_RST, 24, 25);
static const struct reg_field refclk_dig_sel_10g =
REG_FIELD(WIZ_SERDES_RST, 24, 24);
static const struct reg_field pma_cmn_refclk_int_mode =
REG_FIELD(WIZ_SERDES_TOP_CTRL, 28, 29);
+static const struct reg_field pma_cmn_refclk1_int_mode =
+ REG_FIELD(WIZ_SERDES_TOP_CTRL, 20, 21);
static const struct reg_field pma_cmn_refclk_mode =
REG_FIELD(WIZ_SERDES_TOP_CTRL, 30, 31);
static const struct reg_field pma_cmn_refclk_dig_div =
J721E_WIZ_10G, /* Also for J7200 SR1.0 */
AM64_WIZ_10G,
J7200_WIZ_10G, /* J7200 SR2.0 */
+ J784S4_WIZ_10G,
+ J721S2_WIZ_10G,
};
struct wiz_data {
switch (wiz->type) {
case AM64_WIZ_10G:
case J7200_WIZ_10G:
+ case J784S4_WIZ_10G:
+ case J721S2_WIZ_10G:
of_clk_del_provider(dev->of_node);
return;
default:
switch (wiz->type) {
case AM64_WIZ_10G:
case J7200_WIZ_10G:
+ case J784S4_WIZ_10G:
+ case J721S2_WIZ_10G:
ret = wiz_clock_register(wiz);
if (ret)
dev_err(dev, "Failed to register wiz clocks\n");
break;
case J721E_WIZ_10G:
case J7200_WIZ_10G:
+ case J721S2_WIZ_10G:
if (wiz->lane_phy_type[lane] == PHY_TYPE_SGMII)
return regmap_field_write(wiz->p0_fullrt_div[lane], 0x2);
break;
.clk_div_sel_num = WIZ_DIV_NUM_CLOCKS_10G,
};
+static struct wiz_data j784s4_10g_data = {
+ .type = J784S4_WIZ_10G,
+ .pll0_refclk_mux_sel = &pll0_refclk_mux_sel_2,
+ .pll1_refclk_mux_sel = &pll1_refclk_mux_sel_2,
+ .refclk_dig_sel = &refclk_dig_sel_16g,
+ .pma_cmn_refclk1_int_mode = &pma_cmn_refclk1_int_mode,
+ .clk_mux_sel = clk_mux_sel_10g_2_refclk,
+ .clk_div_sel_num = WIZ_DIV_NUM_CLOCKS_10G,
+};
+
+static struct wiz_data j721s2_10g_data = {
+ .type = J721S2_WIZ_10G,
+ .pll0_refclk_mux_sel = &pll0_refclk_mux_sel,
+ .pll1_refclk_mux_sel = &pll1_refclk_mux_sel,
+ .refclk_dig_sel = &refclk_dig_sel_10g,
+ .clk_mux_sel = clk_mux_sel_10g,
+ .clk_div_sel_num = WIZ_DIV_NUM_CLOCKS_10G,
+};
+
static const struct of_device_id wiz_id_table[] = {
{
.compatible = "ti,j721e-wiz-16g", .data = &j721e_16g_data,
{
.compatible = "ti,j7200-wiz-10g", .data = &j7200_pg2_10g_data,
},
+ {
+ .compatible = "ti,j784s4-wiz-10g", .data = &j784s4_10g_data,
+ },
+ {
+ .compatible = "ti,j721s2-wiz-10g", .data = &j721s2_10g_data,
+ },
{}
};
MODULE_DEVICE_TABLE(of, wiz_id_table);
if (!resp->source_cap_count && !resp->sink_cap_count)
return;
- port->partner_pd = usb_power_delivery_register(NULL, &desc);
+ port->partner_pd = typec_partner_usb_power_delivery_register(port->partner, &desc);
if (IS_ERR(port->partner_pd)) {
dev_warn(typec->dev, "Failed to register partner PD device, port: %d\n", port_num);
return;
(chip->id == CHIP_PM8607) ? chip->companion : chip->client;
if (!info->i2c_8606) {
dev_err(&pdev->dev, "Missed I2C address of 88PM8606!\n");
- ret = -EINVAL;
- goto out;
+ return -EINVAL;
}
info->dev = &pdev->dev;
psy_cfg.drv_data = info;
psy_cfg.supplied_to = pm860x_supplied_to;
psy_cfg.num_supplicants = ARRAY_SIZE(pm860x_supplied_to);
- info->usb = power_supply_register(&pdev->dev, &pm860x_charger_desc,
- &psy_cfg);
+ info->usb = devm_power_supply_register(&pdev->dev, &pm860x_charger_desc,
+ &psy_cfg);
if (IS_ERR(info->usb)) {
- ret = PTR_ERR(info->usb);
- goto out;
+ return PTR_ERR(info->usb);
}
pm860x_init_charger(info);
for (i = 0; i < ARRAY_SIZE(info->irq); i++) {
- ret = request_threaded_irq(info->irq[i], NULL,
- pm860x_irq_descs[i].handler,
- IRQF_ONESHOT, pm860x_irq_descs[i].name, info);
+ ret = devm_request_threaded_irq(&pdev->dev, info->irq[i], NULL,
+ pm860x_irq_descs[i].handler,
+ IRQF_ONESHOT,
+ pm860x_irq_descs[i].name, info);
if (ret < 0) {
dev_err(chip->dev, "Failed to request IRQ: #%d: %d\n",
info->irq[i], ret);
- goto out_irq;
+ return ret;
}
}
return 0;
-
-out_irq:
- power_supply_unregister(info->usb);
- while (--i >= 0)
- free_irq(info->irq[i], info);
-out:
- return ret;
-}
-
-static int pm860x_charger_remove(struct platform_device *pdev)
-{
- struct pm860x_charger_info *info = platform_get_drvdata(pdev);
- int i;
-
- power_supply_unregister(info->usb);
- for (i = 0; i < info->irq_nums; i++)
- free_irq(info->irq[i], info);
- return 0;
}
static struct platform_driver pm860x_charger_driver = {
.name = "88pm860x-charger",
},
.probe = pm860x_charger_probe,
- .remove = pm860x_charger_remove,
};
module_platform_driver(pm860x_charger_driver);
*
* Due to a asic bug it is necessary to lower the input current to the vbus
* charger when charging with at some specific levels. This issue is only valid
- * for below a certain battery voltage. This function makes sure that the
+ * for below a certain battery voltage. This function makes sure that
* the allowed current limit isn't exceeded.
*/
static void ab8500_charger_check_vbat_work(struct work_struct *work)
if (ret)
return ret;
- return platform_driver_register(&ab8500_charger_driver);
+ ret = platform_driver_register(&ab8500_charger_driver);
+ if (ret) {
+ platform_unregister_drivers(ab8500_charger_component_drivers,
+ ARRAY_SIZE(ab8500_charger_component_drivers));
+ return ret;
+ }
+
+ return 0;
}
static void __exit ab8500_charger_exit(void)
.num_properties = ARRAY_SIZE(adp5061_props),
};
-static int adp5061_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int adp5061_probe(struct i2c_client *client)
{
struct power_supply_config psy_cfg = {};
struct adp5061_state *st;
.driver = {
.name = KBUILD_MODNAME,
},
- .probe = adp5061_probe,
+ .probe_new = adp5061_probe,
.id_table = adp5061_id,
};
module_i2c_driver(adp5061_driver);
* Describe the setting in linear_range table.
*/
static const struct linear_range input_current_limit_ranges[] = {
- {
- .min = 0,
- .step = 32000,
- .min_sel = 0x0,
- .max_sel = 0x1ff,
- },
+ LINEAR_RANGE(0, 0x0, 0x1ff, 32000),
};
/* Possible trickle, pre-charging and termination current values */
static const struct linear_range charging_current_ranges[] = {
- {
- .min = 0,
- .step = 64000,
- .min_sel = 0x0,
- .max_sel = 0x10,
- }, {
- .min = 1024000,
- .step = 0,
- .min_sel = 0x11,
- .max_sel = 0x1f,
- },
+ LINEAR_RANGE(0, 0x0, 0x10, 64000),
+ LINEAR_RANGE(1024000, 0x11, 0x1f, 0),
};
/*
* and battery over voltage protection have same possible values
*/
static const struct linear_range charge_voltage_regulation_ranges[] = {
- {
- .min = 2560000,
- .step = 0,
- .min_sel = 0,
- .max_sel = 0xA0,
- }, {
- .min = 2560000,
- .step = 16000,
- .min_sel = 0xA0,
- .max_sel = 0x4B0,
- }, {
- .min = 19200000,
- .step = 0,
- .min_sel = 0x4B0,
- .max_sel = 0x7FF,
- },
+ LINEAR_RANGE(2560000, 0, 0xA0, 0),
+ LINEAR_RANGE(2560000, 0xA0, 0x4B0, 16000),
+ LINEAR_RANGE(19200000, 0x4B0, 0x7FF, 0),
};
/* Possible VSYS voltage regulation values */
static const struct linear_range vsys_voltage_regulation_ranges[] = {
- {
- .min = 2560000,
- .step = 0,
- .min_sel = 0,
- .max_sel = 0x28,
- }, {
- .min = 2560000,
- .step = 64000,
- .min_sel = 0x28,
- .max_sel = 0x12C,
- }, {
- .min = 19200000,
- .step = 0,
- .min_sel = 0x12C,
- .max_sel = 0x1FF,
- },
+ LINEAR_RANGE(2560000, 0, 0x28, 0),
+ LINEAR_RANGE(2560000, 0x28, 0x12C, 64000),
+ LINEAR_RANGE(19200000, 0x12C, 0x1FF, 0),
};
/* Possible settings for switching from trickle to pre-charging limits */
static const struct linear_range trickle_to_pre_threshold_ranges[] = {
- {
- .min = 2048000,
- .step = 0,
- .min_sel = 0,
- .max_sel = 0x20,
- }, {
- .min = 2048000,
- .step = 64000,
- .min_sel = 0x20,
- .max_sel = 0x12C,
- }, {
- .min = 19200000,
- .step = 0,
- .min_sel = 0x12C,
- .max_sel = 0x1FF
- }
+ LINEAR_RANGE(2048000, 0, 0x20, 0),
+ LINEAR_RANGE(2048000, 0x20, 0x12C, 64000),
+ LINEAR_RANGE(19200000, 0x12C, 0x1FF, 0),
};
/* Possible current values for fast-charging constant current phase */
static const struct linear_range fast_charge_current_ranges[] = {
- {
- .min = 0,
- .step = 64000,
- .min_sel = 0,
- .max_sel = 0xFF,
- }
+ LINEAR_RANGE(0, 0, 0xFF, 64000),
};
struct battery_init {
}
/* main bq2415x probe function */
-static int bq2415x_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bq2415x_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
int ret;
int num;
char *name = NULL;
.of_match_table = of_match_ptr(bq2415x_of_match_table),
.acpi_match_table = ACPI_PTR(bq2415x_i2c_acpi_match),
},
- .probe = bq2415x_probe,
+ .probe_new = bq2415x_probe,
.remove = bq2415x_remove,
.id_table = bq2415x_i2c_id_table,
};
return 0;
}
-static int bq24190_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bq24190_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct i2c_adapter *adapter = client->adapter;
struct device *dev = &client->dev;
struct power_supply_config charger_cfg = {}, battery_cfg = {};
MODULE_DEVICE_TABLE(of, bq24190_of_match);
static struct i2c_driver bq24190_driver = {
- .probe = bq24190_probe,
+ .probe_new = bq24190_probe,
.remove = bq24190_remove,
.shutdown = bq24190_shutdown,
.id_table = bq24190_i2c_ids,
return 0;
}
-static int bq24257_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bq24257_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct i2c_adapter *adapter = client->adapter;
struct device *dev = &client->dev;
const struct acpi_device_id *acpi_id;
.acpi_match_table = ACPI_PTR(bq24257_acpi_match),
.pm = &bq24257_pm,
},
- .probe = bq24257_probe,
+ .probe_new = bq24257_probe,
.remove = bq24257_remove,
.id_table = bq24257_i2c_ids,
};
return pdata;
}
-static int bq24735_charger_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bq24735_charger_probe(struct i2c_client *client)
{
int ret;
struct bq24735 *charger;
.name = "bq24735-charger",
.of_match_table = bq24735_match_ids,
},
- .probe = bq24735_charger_probe,
+ .probe_new = bq24735_charger_probe,
.id_table = bq24735_charger_id,
};
.volatile_reg = bq2515x_volatile_register,
};
-static int bq2515x_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bq2515x_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct bq2515x_device *bq2515x;
struct power_supply_config charger_cfg = {};
.name = "bq2515x-charger",
.of_match_table = bq2515x_of_match,
},
- .probe = bq2515x_probe,
+ .probe_new = bq2515x_probe,
.id_table = bq2515x_i2c_ids,
};
module_i2c_driver(bq2515x_driver);
return 0;
}
-static int bq256xx_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bq256xx_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct bq256xx_device *bq;
struct power_supply_config psy_cfg = { };
.of_match_table = bq256xx_of_match,
.acpi_match_table = bq256xx_acpi_match,
},
- .probe = bq256xx_probe,
+ .probe_new = bq256xx_probe,
.id_table = bq256xx_i2c_ids,
};
module_i2c_driver(bq256xx_driver);
val->intval = POWER_SUPPLY_HEALTH_UNSPEC_FAILURE;
break;
- case POWER_SUPPLY_PROP_CONSTANT_CHARGE_CURRENT_MAX:
- val->intval = bq25890_find_val(bq->init_data.ichg, TBL_ICHG);
+ case POWER_SUPPLY_PROP_PRECHARGE_CURRENT:
+ val->intval = bq25890_find_val(bq->init_data.iprechg, TBL_ITERM);
+ break;
+
+ case POWER_SUPPLY_PROP_CHARGE_TERM_CURRENT:
+ val->intval = bq25890_find_val(bq->init_data.iterm, TBL_ITERM);
+ break;
+
+ case POWER_SUPPLY_PROP_INPUT_CURRENT_LIMIT:
+ ret = bq25890_field_read(bq, F_IINLIM);
+ if (ret < 0)
+ return ret;
+
+ val->intval = bq25890_find_val(ret, TBL_IINLIM);
+ break;
+
+ case POWER_SUPPLY_PROP_CURRENT_NOW: /* I_BAT now */
+ /*
+ * This is ADC-sampled immediate charge current supplied
+ * from charger to battery. The property name is confusing,
+ * for clarification refer to:
+ * Documentation/ABI/testing/sysfs-class-power
+ * /sys/class/power_supply/<supply_name>/current_now
+ */
+ ret = bq25890_field_read(bq, F_ICHGR); /* read measured value */
+ if (ret < 0)
+ return ret;
+
+ /* converted_val = ADC_val * 50mA (table 10.3.19) */
+ val->intval = ret * -50000;
+ break;
+
+ case POWER_SUPPLY_PROP_CONSTANT_CHARGE_CURRENT: /* I_BAT user limit */
+ /*
+ * This is user-configured constant charge current supplied
+ * from charger to battery in first phase of charging, when
+ * battery voltage is below constant charge voltage.
+ *
+ * This value reflects the current hardware setting.
+ *
+ * The POWER_SUPPLY_PROP_CONSTANT_CHARGE_CURRENT_MAX is the
+ * maximum value of this property.
+ */
+ ret = bq25890_field_read(bq, F_ICHG);
+ if (ret < 0)
+ return ret;
+ val->intval = bq25890_find_val(ret, TBL_ICHG);
/* When temperature is too low, charge current is decreased */
if (bq->state.ntc_fault == NTC_FAULT_COOL) {
}
break;
- case POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE:
- if (!state.online) {
- val->intval = 0;
- break;
- }
+ case POWER_SUPPLY_PROP_CONSTANT_CHARGE_CURRENT_MAX: /* I_BAT max */
+ /*
+ * This is maximum allowed constant charge current supplied
+ * from charger to battery in first phase of charging, when
+ * battery voltage is below constant charge voltage.
+ *
+ * This value is constant for each battery and set from DT.
+ */
+ val->intval = bq25890_find_val(bq->init_data.ichg, TBL_ICHG);
+ break;
+ case POWER_SUPPLY_PROP_VOLTAGE_NOW: /* V_BAT now */
+ /*
+ * This is ADC-sampled immediate charge voltage supplied
+ * from charger to battery. The property name is confusing,
+ * for clarification refer to:
+ * Documentation/ABI/testing/sysfs-class-power
+ * /sys/class/power_supply/<supply_name>/voltage_now
+ */
ret = bq25890_field_read(bq, F_BATV); /* read measured value */
if (ret < 0)
return ret;
val->intval = 2304000 + ret * 20000;
break;
- case POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE_MAX:
- val->intval = bq25890_find_val(bq->init_data.vreg, TBL_VREG);
- break;
-
- case POWER_SUPPLY_PROP_PRECHARGE_CURRENT:
- val->intval = bq25890_find_val(bq->init_data.iprechg, TBL_ITERM);
- break;
-
- case POWER_SUPPLY_PROP_CHARGE_TERM_CURRENT:
- val->intval = bq25890_find_val(bq->init_data.iterm, TBL_ITERM);
- break;
-
- case POWER_SUPPLY_PROP_INPUT_CURRENT_LIMIT:
- ret = bq25890_field_read(bq, F_IINLIM);
- if (ret < 0)
- return ret;
-
- val->intval = bq25890_find_val(ret, TBL_IINLIM);
- break;
-
- case POWER_SUPPLY_PROP_VOLTAGE_NOW:
- ret = bq25890_field_read(bq, F_SYSV); /* read measured value */
+ case POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE: /* V_BAT user limit */
+ /*
+ * This is user-configured constant charge voltage supplied
+ * from charger to battery in second phase of charging, when
+ * battery voltage reached constant charge voltage.
+ *
+ * This value reflects the current hardware setting.
+ *
+ * The POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE_MAX is the
+ * maximum value of this property.
+ */
+ ret = bq25890_field_read(bq, F_VREG);
if (ret < 0)
return ret;
- /* converted_val = 2.304V + ADC_val * 20mV (table 10.3.15) */
- val->intval = 2304000 + ret * 20000;
+ val->intval = bq25890_find_val(ret, TBL_VREG);
break;
- case POWER_SUPPLY_PROP_CURRENT_NOW:
- ret = bq25890_field_read(bq, F_ICHGR); /* read measured value */
- if (ret < 0)
- return ret;
-
- /* converted_val = ADC_val * 50mA (table 10.3.19) */
- val->intval = ret * -50000;
+ case POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE_MAX: /* V_BAT max */
+ /*
+ * This is maximum allowed constant charge voltage supplied
+ * from charger to battery in second phase of charging, when
+ * battery voltage reached constant charge voltage.
+ *
+ * This value is constant for each battery and set from DT.
+ */
+ val->intval = bq25890_find_val(bq->init_data.vreg, TBL_VREG);
break;
case POWER_SUPPLY_PROP_TEMP:
const union power_supply_propval *val)
{
struct bq25890_device *bq = power_supply_get_drvdata(psy);
+ int maxval;
u8 lval;
switch (psp) {
+ case POWER_SUPPLY_PROP_CONSTANT_CHARGE_CURRENT:
+ maxval = bq25890_find_val(bq->init_data.ichg, TBL_ICHG);
+ lval = bq25890_find_idx(min(val->intval, maxval), TBL_ICHG);
+ return bq25890_field_write(bq, F_ICHG, lval);
+ case POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE:
+ maxval = bq25890_find_val(bq->init_data.vreg, TBL_VREG);
+ lval = bq25890_find_idx(min(val->intval, maxval), TBL_VREG);
+ return bq25890_field_write(bq, F_VREG, lval);
case POWER_SUPPLY_PROP_INPUT_CURRENT_LIMIT:
lval = bq25890_find_idx(val->intval, TBL_IINLIM);
return bq25890_field_write(bq, F_IINLIM, lval);
enum power_supply_property psp)
{
switch (psp) {
+ case POWER_SUPPLY_PROP_CONSTANT_CHARGE_CURRENT:
+ case POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE:
case POWER_SUPPLY_PROP_INPUT_CURRENT_LIMIT:
return true;
default:
POWER_SUPPLY_PROP_CHARGE_TYPE,
POWER_SUPPLY_PROP_ONLINE,
POWER_SUPPLY_PROP_HEALTH,
+ POWER_SUPPLY_PROP_CONSTANT_CHARGE_CURRENT,
POWER_SUPPLY_PROP_CONSTANT_CHARGE_CURRENT_MAX,
POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE,
POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE_MAX,
return bq25890_field_read(bq, F_OTG_CFG);
}
+static int bq25890_vbus_get_voltage(struct regulator_dev *rdev)
+{
+ struct bq25890_device *bq = rdev_get_drvdata(rdev);
+
+ return bq25890_get_vbus_voltage(bq);
+}
+
+static int bq25890_vsys_get_voltage(struct regulator_dev *rdev)
+{
+ struct bq25890_device *bq = rdev_get_drvdata(rdev);
+ int ret;
+
+ /* Should be some output voltage ? */
+ ret = bq25890_field_read(bq, F_SYSV); /* read measured value */
+ if (ret < 0)
+ return ret;
+
+ /* converted_val = 2.304V + ADC_val * 20mV (table 10.3.15) */
+ return 2304000 + ret * 20000;
+}
+
static const struct regulator_ops bq25890_vbus_ops = {
.enable = bq25890_vbus_enable,
.disable = bq25890_vbus_disable,
.is_enabled = bq25890_vbus_is_enabled,
+ .get_voltage = bq25890_vbus_get_voltage,
};
static const struct regulator_desc bq25890_vbus_desc = {
.type = REGULATOR_VOLTAGE,
.owner = THIS_MODULE,
.ops = &bq25890_vbus_ops,
- .fixed_uV = 5000000,
- .n_voltages = 1,
};
+
+static const struct regulator_ops bq25890_vsys_ops = {
+ .get_voltage = bq25890_vsys_get_voltage,
+};
+
+static const struct regulator_desc bq25890_vsys_desc = {
+ .name = "vsys",
+ .of_match = "vsys",
+ .type = REGULATOR_VOLTAGE,
+ .owner = THIS_MODULE,
+ .ops = &bq25890_vsys_ops,
+};
+
+static int bq25890_register_regulator(struct bq25890_device *bq)
+{
+ struct bq25890_platform_data *pdata = dev_get_platdata(bq->dev);
+ struct regulator_config cfg = {
+ .dev = bq->dev,
+ .driver_data = bq,
+ };
+ struct regulator_dev *reg;
+
+ if (pdata)
+ cfg.init_data = pdata->regulator_init_data;
+
+ reg = devm_regulator_register(bq->dev, &bq25890_vbus_desc, &cfg);
+ if (IS_ERR(reg)) {
+ return dev_err_probe(bq->dev, PTR_ERR(reg),
+ "registering vbus regulator");
+ }
+
+ /* pdata->regulator_init_data is for vbus only */
+ cfg.init_data = NULL;
+ reg = devm_regulator_register(bq->dev, &bq25890_vsys_desc, &cfg);
+ if (IS_ERR(reg)) {
+ return dev_err_probe(bq->dev, PTR_ERR(reg),
+ "registering vsys regulator");
+ }
+
+ return 0;
+}
+#else
+static inline int
+bq25890_register_regulator(struct bq25890_device *bq)
+{
+ return 0;
+}
#endif
static int bq25890_get_chip_version(struct bq25890_device *bq)
return 0;
}
-static int bq25890_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static void bq25890_non_devm_cleanup(void *data)
+{
+ struct bq25890_device *bq = data;
+
+ cancel_delayed_work_sync(&bq->pump_express_work);
+}
+
+static int bq25890_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
struct bq25890_device *bq;
/* OTG reporting */
bq->usb_phy = devm_usb_get_phy(dev, USB_PHY_TYPE_USB2);
- if (!IS_ERR_OR_NULL(bq->usb_phy)) {
- INIT_WORK(&bq->usb_work, bq25890_usb_work);
- bq->usb_nb.notifier_call = bq25890_usb_notifier;
- usb_register_notifier(bq->usb_phy, &bq->usb_nb);
- }
-#ifdef CONFIG_REGULATOR
- else {
- struct bq25890_platform_data *pdata = dev_get_platdata(dev);
- struct regulator_config cfg = { };
- struct regulator_dev *reg;
-
- cfg.dev = dev;
- cfg.driver_data = bq;
- if (pdata)
- cfg.init_data = pdata->regulator_init_data;
-
- reg = devm_regulator_register(dev, &bq25890_vbus_desc, &cfg);
- if (IS_ERR(reg))
- return dev_err_probe(dev, PTR_ERR(reg), "registering regulator");
- }
-#endif
+
+ /*
+ * This must be before bq25890_power_supply_init(), so that it runs
+ * after devm unregisters the power_supply.
+ */
+ ret = devm_add_action_or_reset(dev, bq25890_non_devm_cleanup, bq);
+ if (ret)
+ return ret;
+
+ ret = bq25890_register_regulator(bq);
+ if (ret)
+ return ret;
ret = bq25890_power_supply_init(bq);
- if (ret < 0) {
- dev_err(dev, "Failed to register power supply\n");
- goto err_unregister_usb_notifier;
- }
+ if (ret < 0)
+ return dev_err_probe(dev, ret, "registering power supply\n");
ret = devm_request_threaded_irq(dev, client->irq, NULL,
bq25890_irq_handler_thread,
IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
BQ25890_IRQ_PIN, bq);
if (ret)
- goto err_unregister_usb_notifier;
-
- return 0;
+ return ret;
-err_unregister_usb_notifier:
- if (!IS_ERR_OR_NULL(bq->usb_phy))
- usb_unregister_notifier(bq->usb_phy, &bq->usb_nb);
+ if (!IS_ERR_OR_NULL(bq->usb_phy)) {
+ INIT_WORK(&bq->usb_work, bq25890_usb_work);
+ bq->usb_nb.notifier_call = bq25890_usb_notifier;
+ usb_register_notifier(bq->usb_phy, &bq->usb_nb);
+ }
- return ret;
+ return 0;
}
static void bq25890_remove(struct i2c_client *client)
{
struct bq25890_device *bq = i2c_get_clientdata(client);
- if (!IS_ERR_OR_NULL(bq->usb_phy))
+ if (!IS_ERR_OR_NULL(bq->usb_phy)) {
usb_unregister_notifier(bq->usb_phy, &bq->usb_nb);
+ cancel_work_sync(&bq->usb_work);
+ }
if (!bq->skip_reset) {
/* reset all registers to default values */
.acpi_match_table = ACPI_PTR(bq25890_acpi_match),
.pm = &bq25890_pm,
},
- .probe = bq25890_probe,
+ .probe_new = bq25890_probe,
.remove = bq25890_remove,
.shutdown = bq25890_shutdown,
.id_table = bq25890_i2c_ids,
return 0;
}
-static int bq25980_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bq25980_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct device *dev = &client->dev;
struct bq25980_device *bq;
int ret;
.name = "bq25980-charger",
.of_match_table = bq25980_of_match,
},
- .probe = bq25980_probe,
+ .probe_new = bq25980_probe,
.id_table = bq25980_i2c_ids,
};
module_i2c_driver(bq25980_driver);
return 0;
}
-static int bq27xxx_battery_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bq27xxx_battery_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct bq27xxx_device_info *di;
int ret;
char *name;
.name = "bq27xxx-battery",
.of_match_table = of_match_ptr(bq27xxx_battery_i2c_of_match_table),
},
- .probe = bq27xxx_battery_i2c_probe,
+ .probe_new = bq27xxx_battery_i2c_probe,
.remove = bq27xxx_battery_i2c_remove,
.id_table = bq27xxx_i2c_id_table,
};
}
cw_bat->battery_workqueue = create_singlethread_workqueue("rk_battery");
+ if (!cw_bat->battery_workqueue)
+ return -ENOMEM;
+
devm_delayed_work_autocancel(&client->dev,
&cw_bat->battery_delay_work, cw_bat_work);
queue_delayed_work(cw_bat->battery_workqueue,
}
};
-static int ds278x_battery_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ds278x_battery_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct ds278x_platform_data *pdata = client->dev.platform_data;
struct power_supply_config psy_cfg = {};
struct ds278x_info *info;
.name = "ds2782-battery",
.pm = &ds278x_battery_pm_ops,
},
- .probe = ds278x_battery_probe,
+ .probe_new = ds278x_battery_probe,
.remove = ds278x_battery_remove,
.id_table = ds278x_id,
};
}
#endif
-static int lp8727_probe(struct i2c_client *cl, const struct i2c_device_id *id)
+static int lp8727_probe(struct i2c_client *cl)
{
struct lp8727_chg *pchg;
struct lp8727_platform_data *pdata;
.name = "lp8727",
.of_match_table = of_match_ptr(lp8727_dt_ids),
},
- .probe = lp8727_probe,
+ .probe_new = lp8727_probe,
.remove = lp8727_remove,
.id_table = lp8727_ids,
};
static int lp8788_irq_register(struct platform_device *pdev,
struct lp8788_charger *pchg)
{
- const char *name[] = {
+ static const char * const name[] = {
LP8788_CHG_IRQ, LP8788_PRSW_IRQ, LP8788_BATT_IRQ
};
int i;
POWER_SUPPLY_PROP_CURRENT_NOW,
};
-static int ltc294x_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ltc294x_i2c_probe(struct i2c_client *client)
{
struct power_supply_config psy_cfg = {};
struct ltc294x_info *info;
.of_match_table = ltc294x_i2c_of_match,
.pm = LTC294X_PM_OPS,
},
- .probe = ltc294x_i2c_probe,
+ .probe_new = ltc294x_i2c_probe,
.shutdown = ltc294x_i2c_shutdown,
.id_table = ltc294x_i2c_id,
};
regmap_write(info->regmap, LTC4162L_CHARGE_STATUS_ALERTS_REG, 0);
}
-static int ltc4162l_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ltc4162l_probe(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
struct device *dev = &client->dev;
MODULE_DEVICE_TABLE(of, ltc4162l_of_match);
static struct i2c_driver ltc4162l_driver = {
- .probe = ltc4162l_probe,
+ .probe_new = ltc4162l_probe,
.alert = ltc4162l_alert,
.id_table = ltc4162l_i2c_id_table,
.driver = {
POWER_SUPPLY_PROP_MANUFACTURER,
};
-static int max14656_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max14656_probe(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
struct device *dev = &client->dev;
.name = "max14656",
.of_match_table = max14656_match_table,
},
- .probe = max14656_probe,
+ .probe_new = max14656_probe,
.id_table = max14656_id,
};
module_i2c_driver(max14656_i2c_driver);
.num_properties = ARRAY_SIZE(max17040_battery_props),
};
-static int max17040_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max17040_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct i2c_adapter *adapter = client->adapter;
struct power_supply_config psy_cfg = {};
struct max17040_chip *chip;
.of_match_table = max17040_of_match,
.pm = MAX17040_PM_OPS,
},
- .probe = max17040_probe,
+ .probe_new = max17040_probe,
.id_table = max17040_id,
};
module_i2c_driver(max17040_i2c_driver);
.num_properties = ARRAY_SIZE(max17042_battery_props) - 2,
};
-static int max17042_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int max17042_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct i2c_adapter *adapter = client->adapter;
const struct power_supply_desc *max17042_desc = &max17042_psy_desc;
struct power_supply_config psy_cfg = {};
.of_match_table = of_match_ptr(max17042_dt_match),
.pm = &max17042_pm_ops,
},
- .probe = max17042_probe,
+ .probe_new = max17042_probe,
.id_table = max17042_id,
};
module_i2c_driver(max17042_i2c_driver);
MT6360_RANGE_MAX,
};
-#define MT6360_LINEAR_RANGE(idx, _min, _min_sel, _max_sel, _step) \
- [idx] = REGULATOR_LINEAR_RANGE(_min, _min_sel, _max_sel, _step)
-
static const struct linear_range mt6360_chg_range[MT6360_RANGE_MAX] = {
- MT6360_LINEAR_RANGE(MT6360_RANGE_VMIVR, 3900000, 0, 0x5F, 100000),
- MT6360_LINEAR_RANGE(MT6360_RANGE_ICHG, 100000, 0, 0x31, 100000),
- MT6360_LINEAR_RANGE(MT6360_RANGE_VOREG, 3900000, 0, 0x51, 10000),
- MT6360_LINEAR_RANGE(MT6360_RANGE_AICR, 100000, 0, 0x3F, 50000),
- MT6360_LINEAR_RANGE(MT6360_RANGE_IPREC, 100000, 0, 0x0F, 50000),
- MT6360_LINEAR_RANGE(MT6360_RANGE_IEOC, 100000, 0, 0x0F, 50000),
+ LINEAR_RANGE_IDX(MT6360_RANGE_VMIVR, 3900000, 0, 0x5F, 100000),
+ LINEAR_RANGE_IDX(MT6360_RANGE_ICHG, 100000, 0, 0x31, 100000),
+ LINEAR_RANGE_IDX(MT6360_RANGE_VOREG, 3900000, 0, 0x51, 10000),
+ LINEAR_RANGE_IDX(MT6360_RANGE_AICR, 100000, 0, 0x3F, 50000),
+ LINEAR_RANGE_IDX(MT6360_RANGE_IPREC, 100000, 0, 0x0F, 50000),
+ LINEAR_RANGE_IDX(MT6360_RANGE_IEOC, 100000, 0, 0x0F, 50000),
};
struct mt6360_chg_info {
#ifdef CONFIG_SYSFS
extern void power_supply_init_attrs(struct device_type *dev_type);
-extern int power_supply_uevent(struct device *dev, struct kobj_uevent_env *env);
+extern int power_supply_uevent(const struct device *dev, struct kobj_uevent_env *env);
#else
int i, tab_len, size;
propname = kasprintf(GFP_KERNEL, "ocv-capacity-table-%d", index);
+ if (!propname) {
+ power_supply_put_battery_info(psy, info);
+ err = -ENOMEM;
+ goto out_put_node;
+ }
list = of_get_property(battery_np, propname, &size);
if (!list || !size) {
dev_err(&psy->dev, "failed to get %s\n", propname);
* power_supply_vbat2ri() - find the battery internal resistance
* from the battery voltage
* @info: The battery information container
- * @table: Pointer to battery resistance temperature table
* @vbat_uv: The battery voltage in microvolt
* @charging: If we are charging (true) or not (false)
*
register_cooler_failed:
psy_unregister_thermal(psy);
register_thermal_failed:
- device_del(dev);
wakeup_init_failed:
+ device_del(dev);
device_add_failed:
check_supplies_failed:
dev_set_name_failed:
}
}
-static int add_prop_uevent(struct device *dev, struct kobj_uevent_env *env,
+static int add_prop_uevent(const struct device *dev, struct kobj_uevent_env *env,
enum power_supply_property prop, char *prop_buf)
{
int ret = 0;
pwr_attr = &power_supply_attrs[prop];
dev_attr = &pwr_attr->dev_attr;
- ret = power_supply_show_property(dev, dev_attr, prop_buf);
+ ret = power_supply_show_property((struct device *)dev, dev_attr, prop_buf);
if (ret == -ENODEV || ret == -ENODATA) {
/*
* When a battery is absent, we expect -ENODEV. Don't abort;
pwr_attr->prop_name, prop_buf);
}
-int power_supply_uevent(struct device *dev, struct kobj_uevent_env *env)
+int power_supply_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
- struct power_supply *psy = dev_get_drvdata(dev);
+ const struct power_supply *psy = dev_get_drvdata(dev);
int ret = 0, j;
char *prop_buf;
return -ENODEV;
charger = devm_kzalloc(&pdev->dev, sizeof(*charger), GFP_KERNEL);
- if (!charger)
+ if (!charger) {
+ of_node_put(node);
return -ENOMEM;
+ }
charger->rk808 = rk808;
.num_properties = ARRAY_SIZE(rt5033_battery_props),
};
-static int rt5033_battery_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int rt5033_battery_probe(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
struct power_supply_config psy_cfg = {};
.name = "rt5033-battery",
.of_match_table = rt5033_battery_of_match,
},
- .probe = rt5033_battery_probe,
+ .probe_new = rt5033_battery_probe,
.remove = rt5033_battery_remove,
.id_table = rt5033_battery_id,
};
.cache_type = REGCACHE_RBTREE,
};
-static int rt9455_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int rt9455_probe(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
struct device *dev = &client->dev;
#endif
static struct i2c_driver rt9455_driver = {
- .probe = rt9455_probe,
+ .probe_new = rt9455_probe,
.remove = rt9455_remove,
.id_table = rt9455_i2c_id_table,
.driver = {
.get_property = sbs_get_property,
};
-static int sbs_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int sbs_probe(struct i2c_client *client)
{
struct power_supply_config psy_cfg = {};
struct sbs_info *chip;
MODULE_DEVICE_TABLE(i2c, sbs_id);
static struct i2c_driver sbs_driver = {
- .probe = sbs_probe,
+ .probe_new = sbs_probe,
.id_table = sbs_id,
.driver = {
.name = "sbs-charger",
i2c_mux_del_adapters(sbsm->muxc);
}
-static int sbsm_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int sbsm_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct i2c_adapter *adapter = client->adapter;
struct sbsm_data *data;
struct device *dev = &client->dev;
.name = "sbsm",
.of_match_table = of_match_ptr(sbsm_dt_ids),
},
- .probe = sbsm_probe,
+ .probe_new = sbsm_probe,
.alert = sbsm_alert,
.id_table = sbsm_ids
};
.n_voltages = 1,
};
-static int smb347_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int smb347_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct power_supply_config mains_usb_cfg = {};
struct regulator_config usb_rdev_cfg = {};
struct device *dev = &client->dev;
.name = "smb347",
.of_match_table = smb3xx_of_match,
},
- .probe = smb347_probe,
+ .probe_new = smb347_probe,
.remove = smb347_remove,
.shutdown = smb347_shutdown,
.id_table = smb347_id,
.n_voltages = 1,
};
-static int ucs1002_probe(struct i2c_client *client,
- const struct i2c_device_id *dev_id)
+static int ucs1002_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
struct power_supply_config charger_config = {};
.name = "ucs1002",
.of_match_table = ucs1002_of_match,
},
- .probe = ucs1002_probe,
+ .probe_new = ucs1002_probe,
};
module_i2c_driver(ucs1002_driver);
return 0;
}
-static int z2_batt_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int z2_batt_probe(struct i2c_client *client)
{
int ret = 0;
int props = 1; /* POWER_SUPPLY_PROP_PRESENT */
charger->charge_gpiod = devm_gpiod_get_optional(&client->dev,
NULL, GPIOD_IN);
- if (IS_ERR(charger->charge_gpiod))
- return dev_err_probe(&client->dev,
+ if (IS_ERR(charger->charge_gpiod)) {
+ ret = dev_err_probe(&client->dev,
PTR_ERR(charger->charge_gpiod),
"failed to get charge GPIO\n");
+ goto err;
+ }
if (charger->charge_gpiod) {
gpiod_set_consumer_name(charger->charge_gpiod, "BATT CHRG");
.name = "z2-battery",
.pm = Z2_BATTERY_PM_OPS
},
- .probe = z2_batt_probe,
+ .probe_new = z2_batt_probe,
.remove = z2_batt_remove,
.id_table = z2_batt_id,
};
* instance, specified by one of enum ps3_lpm_tb_type.
* @tb_cache: Optional user supplied buffer to use as the trace buffer cache.
* If NULL, the driver will allocate and manage an internal buffer.
- * Unused when when @tb_type is PS3_LPM_TB_TYPE_NONE.
+ * Unused when @tb_type is PS3_LPM_TB_TYPE_NONE.
* @tb_cache_size: The size in bytes of the user supplied @tb_cache buffer.
* Unused when @tb_cache is NULL or @tb_type is PS3_LPM_TB_TYPE_NONE.
*/
config PWM_JZ4740
tristate "Ingenic JZ47xx PWM support"
- depends on MIPS || COMPILE_TEST
- depends on COMMON_CLK
+ depends on MACH_INGENIC || COMPILE_TEST
+ depends on COMMON_CLK && OF
select MFD_SYSCON
help
Generic PWM framework driver for Ingenic JZ47xx based
config PWM_PXA
tristate "PXA PWM support"
- depends on ARCH_PXA || COMPILE_TEST
+ depends on ARCH_PXA || ARCH_MMP || COMPILE_TEST
depends on HAS_IOMEM
help
Generic PWM framework driver for PXA.
static DEFINE_MUTEX(pwm_lookup_lock);
static LIST_HEAD(pwm_lookup_list);
+
+/* protects access to pwm_chips, allocated_pwms, and pwm_tree */
static DEFINE_MUTEX(pwm_lock);
+
static LIST_HEAD(pwm_chips);
static DECLARE_BITMAP(allocated_pwms, MAX_PWMS);
static RADIX_TREE(pwm_tree, GFP_KERNEL);
return radix_tree_lookup(&pwm_tree, pwm);
}
+/* Called with pwm_lock held */
static int alloc_pwms(unsigned int count)
{
unsigned int start;
if (start + count > MAX_PWMS)
return -ENOSPC;
+ bitmap_set(allocated_pwms, start, count);
+
return start;
}
+/* Called with pwm_lock held */
static void free_pwms(struct pwm_chip *chip)
{
unsigned int i;
}
if (pwm->chip->ops->get_state) {
- pwm->chip->ops->get_state(pwm->chip, pwm, &pwm->state);
- trace_pwm_get(pwm, &pwm->state);
+ struct pwm_state state;
+
+ err = pwm->chip->ops->get_state(pwm->chip, pwm, &state);
+ trace_pwm_get(pwm, &state, err);
+
+ if (!err)
+ pwm->state = state;
if (IS_ENABLED(CONFIG_PWM_DEBUG))
pwm->last = pwm->state;
if (!pwm_ops_check(chip))
return -EINVAL;
+ chip->pwms = kcalloc(chip->npwm, sizeof(*pwm), GFP_KERNEL);
+ if (!chip->pwms)
+ return -ENOMEM;
+
mutex_lock(&pwm_lock);
ret = alloc_pwms(chip->npwm);
- if (ret < 0)
- goto out;
+ if (ret < 0) {
+ mutex_unlock(&pwm_lock);
+ kfree(chip->pwms);
+ return ret;
+ }
chip->base = ret;
- chip->pwms = kcalloc(chip->npwm, sizeof(*pwm), GFP_KERNEL);
- if (!chip->pwms) {
- ret = -ENOMEM;
- goto out;
- }
-
for (i = 0; i < chip->npwm; i++) {
pwm = &chip->pwms[i];
radix_tree_insert(&pwm_tree, pwm->pwm, pwm);
}
- bitmap_set(allocated_pwms, chip->base, chip->npwm);
-
- INIT_LIST_HEAD(&chip->list);
list_add(&chip->list, &pwm_chips);
- ret = 0;
+ mutex_unlock(&pwm_lock);
if (IS_ENABLED(CONFIG_OF))
of_pwmchip_add(chip);
-out:
- mutex_unlock(&pwm_lock);
-
- if (!ret)
- pwmchip_sysfs_export(chip);
+ pwmchip_sysfs_export(chip);
- return ret;
+ return 0;
}
EXPORT_SYMBOL_GPL(pwmchip_add);
* checks.
*/
- chip->ops->get_state(chip, pwm, &s1);
- trace_pwm_get(pwm, &s1);
+ err = chip->ops->get_state(chip, pwm, &s1);
+ trace_pwm_get(pwm, &s1, err);
+ if (err)
+ /* If that failed there isn't much to debug */
+ return;
/*
* The lowlevel driver either ignored .polarity (which is a bug) or as
/* reapply the state that the driver reported being configured. */
err = chip->ops->apply(chip, pwm, &s1);
+ trace_pwm_apply(pwm, &s1, err);
if (err) {
*last = s1;
dev_err(chip->dev, "failed to reapply current setting\n");
return;
}
- trace_pwm_apply(pwm, &s1);
-
- chip->ops->get_state(chip, pwm, last);
- trace_pwm_get(pwm, last);
+ err = chip->ops->get_state(chip, pwm, last);
+ trace_pwm_get(pwm, last, err);
+ if (err)
+ return;
/* reapplication of the current state should give an exact match */
if (s1.enabled != last->enabled ||
return 0;
err = chip->ops->apply(chip, pwm, state);
+ trace_pwm_apply(pwm, state, err);
if (err)
return err;
- trace_pwm_apply(pwm, state);
-
pwm->state = *state;
/*
static int __init pwm_debugfs_init(void)
{
- debugfs_create_file("pwm", S_IFREG | 0444, NULL, NULL,
- &pwm_debugfs_fops);
+ debugfs_create_file("pwm", 0444, NULL, NULL, &pwm_debugfs_fops);
return 0;
}
return 0;
}
-static void atmel_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int atmel_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct atmel_pwm_chip *atmel_pwm = to_atmel_pwm_chip(chip);
u32 sr, cmr;
state->polarity = PWM_POLARITY_INVERSED;
else
state->polarity = PWM_POLARITY_NORMAL;
+
+ return 0;
}
static const struct pwm_ops atmel_pwm_ops = {
ndelay(400);
}
-static void iproc_pwmc_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int iproc_pwmc_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct iproc_pwmc *ip = to_iproc_pwmc(chip);
u64 tmp, multi, rate;
if (rate == 0) {
state->period = 0;
state->duty_cycle = 0;
- return;
+ return 0;
}
value = readl(ip->base + IPROC_PWM_PRESCALE_OFFSET);
value = readl(ip->base + IPROC_PWM_DUTY_CYCLE_OFFSET(pwm->hwpwm));
tmp = (value & IPROC_PWM_PERIOD_MAX) * multi;
state->duty_cycle = div64_u64(tmp, rate);
+
+ return 0;
}
static int iproc_pwmc_apply(struct pwm_chip *chip, struct pwm_device *pwm,
return 0;
}
-static void crc_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int crc_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct crystalcove_pwm *crc_pwm = to_crc_pwm(chip);
struct device *dev = crc_pwm->chip.dev;
error = regmap_read(crc_pwm->regmap, PWM0_CLK_DIV, &clk_div_reg);
if (error) {
dev_err(dev, "Error reading PWM0_CLK_DIV %d\n", error);
- return;
+ return error;
}
error = regmap_read(crc_pwm->regmap, PWM0_DUTY_CYCLE, &duty_cycle_reg);
if (error) {
dev_err(dev, "Error reading PWM0_DUTY_CYCLE %d\n", error);
- return;
+ return error;
}
clk_div = (clk_div_reg & ~PWM_OUTPUT_ENABLE) + 1;
DIV_ROUND_UP_ULL(duty_cycle_reg * state->period, PWM_MAX_LEVEL);
state->polarity = PWM_POLARITY_NORMAL;
state->enabled = !!(clk_div_reg & PWM_OUTPUT_ENABLE);
+
+ return 0;
}
static const struct pwm_ops crc_pwm_ops = {
return 0;
}
-static void cros_ec_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int cros_ec_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct cros_ec_pwm_device *ec_pwm = pwm_to_cros_ec_pwm(chip);
struct cros_ec_pwm *channel = pwm_get_chip_data(pwm);
ret = cros_ec_pwm_get_duty(ec_pwm, pwm->hwpwm);
if (ret < 0) {
dev_err(chip->dev, "error getting initial duty: %d\n", ret);
- return;
+ return ret;
}
state->enabled = (ret > 0);
state->duty_cycle = channel->duty_cycle;
else
state->duty_cycle = ret;
+
+ return 0;
}
static struct pwm_device *
return 0;
}
-static void dwc_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int dwc_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct dwc_pwm *dwc = to_dwc_pwm(chip);
u64 duty, period;
state->polarity = PWM_POLARITY_INVERSED;
pm_runtime_put_sync(chip->dev);
+
+ return 0;
}
static const struct pwm_ops dwc_pwm_ops = {
regmap_read(fpc->regmap, FTM_FMS, &val);
if (val & FTM_FMS_WPEN)
- regmap_update_bits(fpc->regmap, FTM_MODE, FTM_MODE_WPDIS,
- FTM_MODE_WPDIS);
+ regmap_set_bits(fpc->regmap, FTM_MODE, FTM_MODE_WPDIS);
}
static void ftm_set_write_protection(struct fsl_pwm_chip *fpc)
{
- regmap_update_bits(fpc->regmap, FTM_FMS, FTM_FMS_WPEN, FTM_FMS_WPEN);
+ regmap_set_bits(fpc->regmap, FTM_FMS, FTM_FMS_WPEN);
}
static bool fsl_pwm_periodcfg_are_equal(const struct fsl_pwm_periodcfg *a,
ret = clk_prepare_enable(fpc->ipg_clk);
if (!ret && fpc->soc->has_enable_bits) {
mutex_lock(&fpc->lock);
- regmap_update_bits(fpc->regmap, FTM_SC, BIT(pwm->hwpwm + 16),
- BIT(pwm->hwpwm + 16));
+ regmap_set_bits(fpc->regmap, FTM_SC, BIT(pwm->hwpwm + 16));
mutex_unlock(&fpc->lock);
}
if (fpc->soc->has_enable_bits) {
mutex_lock(&fpc->lock);
- regmap_update_bits(fpc->regmap, FTM_SC, BIT(pwm->hwpwm + 16),
- 0);
+ regmap_clear_bits(fpc->regmap, FTM_SC, BIT(pwm->hwpwm + 16));
mutex_unlock(&fpc->lock);
}
if (!newstate->enabled) {
if (oldstate->enabled) {
- regmap_update_bits(fpc->regmap, FTM_OUTMASK,
- BIT(pwm->hwpwm), BIT(pwm->hwpwm));
+ regmap_set_bits(fpc->regmap, FTM_OUTMASK,
+ BIT(pwm->hwpwm));
clk_disable_unprepare(fpc->clk[FSL_PWM_CLK_CNTEN]);
clk_disable_unprepare(fpc->clk[fpc->period.clk_select]);
}
goto end_mutex;
}
- regmap_update_bits(fpc->regmap, FTM_OUTMASK, BIT(pwm->hwpwm),
- 0);
+ regmap_clear_bits(fpc->regmap, FTM_OUTMASK, BIT(pwm->hwpwm));
}
end_mutex:
PWM_POLARITY_MASK, (0x0 << PWM_POLARITY_SHIFT));
}
-static void hibvt_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int hibvt_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct hibvt_pwm_chip *hi_pwm_chip = to_hibvt_pwm_chip(chip);
void __iomem *base;
value = readl(base + PWM_CTRL_ADDR(pwm->hwpwm));
state->enabled = (PWM_ENABLE_MASK & value);
+
+ return 0;
}
static int hibvt_pwm_apply(struct pwm_chip *chip, struct pwm_device *pwm,
val |= BIT(pwm->hwpwm);
img_pwm_writel(imgchip, PWM_CTRL_CFG, val);
- regmap_update_bits(imgchip->periph_regs, PERIP_PWM_PDM_CONTROL,
- PERIP_PWM_PDM_CONTROL_CH_MASK <<
- PERIP_PWM_PDM_CONTROL_CH_SHIFT(pwm->hwpwm), 0);
+ regmap_clear_bits(imgchip->periph_regs, PERIP_PWM_PDM_CONTROL,
+ PERIP_PWM_PDM_CONTROL_CH_MASK <<
+ PERIP_PWM_PDM_CONTROL_CH_SHIFT(pwm->hwpwm));
return 0;
}
for (i = 0; i < imgchip->chip.npwm; i++)
if (imgchip->suspend_ctrl_cfg & BIT(i))
- regmap_update_bits(imgchip->periph_regs,
- PERIP_PWM_PDM_CONTROL,
- PERIP_PWM_PDM_CONTROL_CH_MASK <<
- PERIP_PWM_PDM_CONTROL_CH_SHIFT(i),
- 0);
+ regmap_clear_bits(imgchip->periph_regs,
+ PERIP_PWM_PDM_CONTROL,
+ PERIP_PWM_PDM_CONTROL_CH_MASK <<
+ PERIP_PWM_PDM_CONTROL_CH_SHIFT(i));
if (pm_runtime_status_suspended(dev))
img_pwm_runtime_suspend(dev);
return 0;
}
-static void pwm_imx_tpm_get_state(struct pwm_chip *chip,
- struct pwm_device *pwm,
- struct pwm_state *state)
+static int pwm_imx_tpm_get_state(struct pwm_chip *chip,
+ struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct imx_tpm_pwm_chip *tpm = to_imx_tpm_pwm_chip(chip);
u32 rate, val, prescale;
/* get channel status */
state->enabled = FIELD_GET(PWM_IMX_TPM_CnSC_ELS, val) ? true : false;
+
+ return 0;
}
/* this function is supposed to be called with mutex hold */
clk_disable_unprepare(imx->clk_ipg);
}
-static void pwm_imx27_get_state(struct pwm_chip *chip,
- struct pwm_device *pwm, struct pwm_state *state)
+static int pwm_imx27_get_state(struct pwm_chip *chip,
+ struct pwm_device *pwm, struct pwm_state *state)
{
struct pwm_imx27_chip *imx = to_pwm_imx27_chip(chip);
u32 period, prescaler, pwm_clk, val;
ret = pwm_imx27_clk_prepare_enable(imx);
if (ret < 0)
- return;
+ return ret;
val = readl(imx->mmio_base + MX3_PWMCR);
state->duty_cycle = DIV_ROUND_UP_ULL(tmp, pwm_clk);
pwm_imx27_clk_disable_unprepare(imx);
+
+ return 0;
}
static void pwm_imx27_sw_reset(struct pwm_chip *chip)
return lgm_pwm_enable(chip, 1);
}
-static void lgm_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int lgm_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct lgm_pwm_chip *pc = to_lgm_pwm_chip(chip);
u32 duty, val;
regmap_read(pc->regmap, LGM_PWM_FAN_CON0, &val);
duty = FIELD_GET(LGM_PWM_FAN_DC_MSK, val);
state->duty_cycle = DIV_ROUND_UP(duty * pc->period, LGM_PWM_MAX_DUTY_CYCLE);
+
+ return 0;
}
static const struct pwm_ops lgm_pwm_ops = {
int ret;
if (!duty_scale)
- return regmap_update_bits(iqs62x->regmap, IQS620_PWR_SETTINGS,
- IQS620_PWR_SETTINGS_PWM_OUT, 0);
+ return regmap_clear_bits(iqs62x->regmap, IQS620_PWR_SETTINGS,
+ IQS620_PWR_SETTINGS_PWM_OUT);
ret = regmap_write(iqs62x->regmap, IQS620_PWM_DUTY_CYCLE,
duty_scale - 1);
return ret;
}
-static void iqs620_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int iqs620_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct iqs620_pwm_private *iqs620_pwm;
mutex_unlock(&iqs620_pwm->lock);
state->period = IQS620_PWM_PERIOD_NS;
+
+ return 0;
}
static int iqs620_pwm_notifier(struct notifier_block *notifier,
struct jz4740_pwm_chip *jz = to_jz4740(chip);
/* Enable PWM output */
- regmap_update_bits(jz->map, TCU_REG_TCSRc(pwm->hwpwm),
- TCU_TCSR_PWM_EN, TCU_TCSR_PWM_EN);
+ regmap_set_bits(jz->map, TCU_REG_TCSRc(pwm->hwpwm), TCU_TCSR_PWM_EN);
/* Start counter */
regmap_write(jz->map, TCU_REG_TESR, BIT(pwm->hwpwm));
* In TCU2 mode (channel 1/2 on JZ4750+), this must be done before the
* counter is stopped, while in TCU1 mode the order does not matter.
*/
- regmap_update_bits(jz->map, TCU_REG_TCSRc(pwm->hwpwm),
- TCU_TCSR_PWM_EN, 0);
+ regmap_clear_bits(jz->map, TCU_REG_TCSRc(pwm->hwpwm), TCU_TCSR_PWM_EN);
/* Stop counter */
regmap_write(jz->map, TCU_REG_TECR, BIT(pwm->hwpwm));
regmap_write(jz4740->map, TCU_REG_TDFRc(pwm->hwpwm), period);
/* Set abrupt shutdown */
- regmap_update_bits(jz4740->map, TCU_REG_TCSRc(pwm->hwpwm),
- TCU_TCSR_PWM_SD, TCU_TCSR_PWM_SD);
+ regmap_set_bits(jz4740->map, TCU_REG_TCSRc(pwm->hwpwm),
+ TCU_TCSR_PWM_SD);
/*
* Set polarity.
return devm_pwmchip_add(dev, &jz4740->chip);
}
-static const struct soc_info __maybe_unused jz4740_soc_info = {
+static const struct soc_info jz4740_soc_info = {
.num_pwms = 8,
};
-static const struct soc_info __maybe_unused jz4725b_soc_info = {
+static const struct soc_info jz4725b_soc_info = {
.num_pwms = 6,
};
-static const struct soc_info __maybe_unused x1000_soc_info = {
+static const struct soc_info x1000_soc_info = {
.num_pwms = 5,
};
-#ifdef CONFIG_OF
static const struct of_device_id jz4740_pwm_dt_ids[] = {
{ .compatible = "ingenic,jz4740-pwm", .data = &jz4740_soc_info },
{ .compatible = "ingenic,jz4725b-pwm", .data = &jz4725b_soc_info },
{},
};
MODULE_DEVICE_TABLE(of, jz4740_pwm_dt_ids);
-#endif
static struct platform_driver jz4740_pwm_driver = {
.driver = {
.name = "jz4740-pwm",
- .of_match_table = of_match_ptr(jz4740_pwm_dt_ids),
+ .of_match_table = jz4740_pwm_dt_ids,
},
.probe = jz4740_pwm_probe,
};
KMB_PWM_LEADIN_OFFSET(ch));
}
-static void keembay_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int keembay_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct keembay_pwm *priv = to_keembay_pwm_dev(chip);
unsigned long long high, low;
state->duty_cycle = DIV_ROUND_UP_ULL(high, clk_rate);
state->period = DIV_ROUND_UP_ULL(high + low, clk_rate);
state->polarity = PWM_POLARITY_NORMAL;
+
+ return 0;
}
static int keembay_pwm_apply(struct pwm_chip *chip, struct pwm_device *pwm,
u32 val;
/*
- * With clk_rate < NSEC_PER_SEC this cannot overflow.
+ * With clk_rate <= NSEC_PER_SEC this cannot overflow.
* With duty_ns <= period_ns < max_period_ns this also fits into an u32.
*/
val = mul_u64_u64_div_u64(duty_ns, lpc18xx_pwm->clk_rate, NSEC_PER_SEC);
return ret;
}
-static void pwm_lpss_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int pwm_lpss_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct pwm_lpss_chip *lpwm = to_lpwm(chip);
unsigned long base_unit_range;
state->enabled = !!(ctrl & PWM_ENABLE);
pm_runtime_put(chip->dev);
+
+ return 0;
}
static const struct pwm_ops pwm_lpss_ops = {
static const struct pwm_mediatek_of_data mt7622_pwm_data = {
.num_pwms = 6,
.pwm45_fixup = false,
- .has_ck_26m_sel = false,
+ .has_ck_26m_sel = true,
};
static const struct pwm_mediatek_of_data mt7623_pwm_data = {
.has_ck_26m_sel = true,
};
+static const struct pwm_mediatek_of_data mt7986_pwm_data = {
+ .num_pwms = 2,
+ .pwm45_fixup = false,
+ .has_ck_26m_sel = true,
+};
+
static const struct pwm_mediatek_of_data mt8516_pwm_data = {
.num_pwms = 5,
.pwm45_fixup = false,
{ .compatible = "mediatek,mt7623-pwm", .data = &mt7623_pwm_data },
{ .compatible = "mediatek,mt7628-pwm", .data = &mt7628_pwm_data },
{ .compatible = "mediatek,mt7629-pwm", .data = &mt7629_pwm_data },
+ { .compatible = "mediatek,mt7986-pwm", .data = &mt7986_pwm_data },
{ .compatible = "mediatek,mt8183-pwm", .data = &mt8183_pwm_data },
{ .compatible = "mediatek,mt8365-pwm", .data = &mt8365_pwm_data },
{ .compatible = "mediatek,mt8516-pwm", .data = &mt8516_pwm_data },
return cnt * fin_ns * (channel->pre_div + 1);
}
-static void meson_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int meson_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct meson_pwm *meson = to_meson_pwm(chip);
struct meson_pwm_channel_data *channel_data;
u32 value, tmp;
if (!state)
- return;
+ return 0;
channel = &meson->channels[pwm->hwpwm];
channel_data = &meson_pwm_per_channel_data[pwm->hwpwm];
state->period = 0;
state->duty_cycle = 0;
}
+
+ return 0;
}
static const struct pwm_ops meson_pwm_ops = {
return 0;
}
-static void mtk_disp_pwm_get_state(struct pwm_chip *chip,
- struct pwm_device *pwm,
- struct pwm_state *state)
+static int mtk_disp_pwm_get_state(struct pwm_chip *chip,
+ struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct mtk_disp_pwm *mdp = to_mtk_disp_pwm(chip);
u64 rate, period, high_width;
- u32 clk_div, con0, con1;
+ u32 clk_div, pwm_en, con0, con1;
int err;
err = clk_prepare_enable(mdp->clk_main);
if (err < 0) {
dev_err(chip->dev, "Can't enable mdp->clk_main: %pe\n", ERR_PTR(err));
- return;
+ return err;
}
err = clk_prepare_enable(mdp->clk_mm);
if (err < 0) {
dev_err(chip->dev, "Can't enable mdp->clk_mm: %pe\n", ERR_PTR(err));
clk_disable_unprepare(mdp->clk_main);
- return;
+ return err;
}
rate = clk_get_rate(mdp->clk_main);
con0 = readl(mdp->base + mdp->data->con0);
con1 = readl(mdp->base + mdp->data->con1);
- state->enabled = !!(con0 & BIT(0));
+ pwm_en = readl(mdp->base + DISP_PWM_EN);
+ state->enabled = !!(pwm_en & mdp->data->enable_mask);
clk_div = FIELD_GET(PWM_CLKDIV_MASK, con0);
period = FIELD_GET(PWM_PERIOD_MASK, con1);
/*
state->polarity = PWM_POLARITY_NORMAL;
clk_disable_unprepare(mdp->clk_mm);
clk_disable_unprepare(mdp->clk_main);
+
+ return 0;
}
static const struct pwm_ops mtk_disp_pwm_ops = {
return ret;
}
-static void pca9685_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int pca9685_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct pca9685 *pca = to_pca(chip);
unsigned long long duty;
*/
state->duty_cycle = 0;
state->enabled = false;
- return;
+ return 0;
}
state->enabled = true;
duty = pca9685_pwm_get_duty(pca, pwm->hwpwm);
state->duty_cycle = DIV_ROUND_DOWN_ULL(duty * state->period, PCA9685_COUNTER_RANGE);
+
+ return 0;
}
static int pca9685_pwm_request(struct pwm_chip *chip, struct pwm_device *pwm)
.cache_type = REGCACHE_NONE,
};
-static int pca9685_pwm_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int pca9685_pwm_probe(struct i2c_client *client)
{
struct pca9685 *pca;
unsigned int reg;
.of_match_table = of_match_ptr(pca9685_dt_ids),
.pm = &pca9685_pwm_pm,
},
- .probe = pca9685_pwm_probe,
+ .probe_new = pca9685_pwm_probe,
.remove = pca9685_pwm_remove,
.id_table = pca9685_id,
};
*
* 2008-02-13 initial version
* eric miao <eric.miao@marvell.com>
+ *
+ * Links to reference manuals for some of the supported PWM chips can be found
+ * in Documentation/arm/marvell.rst.
+ *
+ * Limitations:
+ * - When PWM is stopped, the current PWM period stops abruptly at the next
+ * input clock (PWMCR_SD is set) and the output is driven to inactive.
*/
#include <linux/module.h>
unsigned long long c;
unsigned long period_cycles, prescale, pv, dc;
unsigned long offset;
- int rc;
offset = pwm->hwpwm ? 0x10 : 0;
else
dc = mul_u64_u64_div_u64(pv + 1, duty_ns, period_ns);
- /* NOTE: the clock to PWM has to be enabled first
- * before writing to the registers
- */
- rc = clk_prepare_enable(pc->clk);
- if (rc < 0)
- return rc;
-
- writel(prescale, pc->mmio_base + offset + PWMCR);
+ writel(prescale | PWMCR_SD, pc->mmio_base + offset + PWMCR);
writel(dc, pc->mmio_base + offset + PWMDCR);
writel(pv, pc->mmio_base + offset + PWMPCR);
- clk_disable_unprepare(pc->clk);
return 0;
}
-static int pxa_pwm_enable(struct pwm_chip *chip, struct pwm_device *pwm)
-{
- struct pxa_pwm_chip *pc = to_pxa_pwm_chip(chip);
-
- return clk_prepare_enable(pc->clk);
-}
-
-static void pxa_pwm_disable(struct pwm_chip *chip, struct pwm_device *pwm)
-{
- struct pxa_pwm_chip *pc = to_pxa_pwm_chip(chip);
-
- clk_disable_unprepare(pc->clk);
-}
-
static int pxa_pwm_apply(struct pwm_chip *chip, struct pwm_device *pwm,
const struct pwm_state *state)
{
+ struct pxa_pwm_chip *pc = to_pxa_pwm_chip(chip);
+ u64 duty_cycle;
int err;
if (state->polarity != PWM_POLARITY_NORMAL)
return -EINVAL;
- if (!state->enabled) {
- if (pwm->state.enabled)
- pxa_pwm_disable(chip, pwm);
+ err = clk_prepare_enable(pc->clk);
+ if (err)
+ return err;
- return 0;
- }
+ duty_cycle = state->enabled ? state->duty_cycle : 0;
- err = pxa_pwm_config(chip, pwm, state->duty_cycle, state->period);
- if (err)
+ err = pxa_pwm_config(chip, pwm, duty_cycle, state->period);
+ if (err) {
+ clk_disable_unprepare(pc->clk);
return err;
+ }
+
+ if (state->enabled && !pwm->state.enabled)
+ return 0;
+
+ clk_disable_unprepare(pc->clk);
- if (!pwm->state.enabled)
- return pxa_pwm_enable(chip, pwm);
+ if (!state->enabled && pwm->state.enabled)
+ clk_disable_unprepare(pc->clk);
return 0;
}
return 0;
}
-static void raspberrypi_pwm_get_state(struct pwm_chip *chip,
- struct pwm_device *pwm,
- struct pwm_state *state)
+static int raspberrypi_pwm_get_state(struct pwm_chip *chip,
+ struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct raspberrypi_pwm *rpipwm = raspberrypi_pwm_from_chip(chip);
RPI_PWM_MAX_DUTY);
state->enabled = !!(rpipwm->duty_cycle);
state->polarity = PWM_POLARITY_NORMAL;
+
+ return 0;
}
static int raspberrypi_pwm_apply(struct pwm_chip *chip, struct pwm_device *pwm,
return container_of(c, struct rockchip_pwm_chip, chip);
}
-static void rockchip_pwm_get_state(struct pwm_chip *chip,
- struct pwm_device *pwm,
- struct pwm_state *state)
+static int rockchip_pwm_get_state(struct pwm_chip *chip,
+ struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct rockchip_pwm_chip *pc = to_rockchip_pwm_chip(chip);
u32 enable_conf = pc->data->enable_conf;
ret = clk_enable(pc->pclk);
if (ret)
- return;
+ return ret;
ret = clk_enable(pc->clk);
if (ret)
- return;
+ return ret;
clk_rate = clk_get_rate(pc->clk);
clk_disable(pc->clk);
clk_disable(pc->pclk);
+
+ return 0;
}
static void rockchip_pwm_config(struct pwm_chip *chip, struct pwm_device *pwm,
"New real_period = %u ns\n", ddata->real_period);
}
-static void pwm_sifive_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int pwm_sifive_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct pwm_sifive_ddata *ddata = pwm_sifive_chip_to_ddata(chip);
u32 duty, val;
state->duty_cycle =
(u64)duty * ddata->real_period >> PWM_SIFIVE_CMPWIDTH;
state->polarity = PWM_POLARITY_INVERSED;
+
+ return 0;
}
static int pwm_sifive_apply(struct pwm_chip *chip, struct pwm_device *pwm,
struct pwm_sifive_ddata *ddata =
container_of(nb, struct pwm_sifive_ddata, notifier);
- if (event == POST_RATE_CHANGE)
+ if (event == POST_RATE_CHANGE) {
+ mutex_lock(&ddata->lock);
pwm_sifive_update_clock(ddata, ndata->new_rate);
+ mutex_unlock(&ddata->lock);
+ }
return NOTIFY_OK;
}
#define sl28cpld_pwm_from_chip(_chip) \
container_of(_chip, struct sl28cpld_pwm, pwm_chip)
-static void sl28cpld_pwm_get_state(struct pwm_chip *chip,
- struct pwm_device *pwm,
- struct pwm_state *state)
+static int sl28cpld_pwm_get_state(struct pwm_chip *chip,
+ struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct sl28cpld_pwm *priv = sl28cpld_pwm_from_chip(chip);
unsigned int reg;
* the PWM core.
*/
state->duty_cycle = min(state->duty_cycle, state->period);
+
+ return 0;
}
static int sl28cpld_pwm_apply(struct pwm_chip *chip, struct pwm_device *pwm,
writel_relaxed(val, spc->base + offset);
}
-static void sprd_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int sprd_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct sprd_pwm_chip *spc =
container_of(chip, struct sprd_pwm_chip, chip);
if (ret) {
dev_err(spc->dev, "failed to enable pwm%u clocks\n",
pwm->hwpwm);
- return;
+ return ret;
}
val = sprd_pwm_read(spc, pwm->hwpwm, SPRD_PWM_ENABLE);
/* Disable PWM clocks if the PWM channel is not in enable state. */
if (!state->enabled)
clk_bulk_disable_unprepare(SPRD_PWM_CHN_CLKS_NUM, chn->clks);
+
+ return 0;
}
static int sprd_pwm_config(struct sprd_pwm_chip *spc, struct pwm_device *pwm,
if (reenable) {
/* Start LP timer in continuous mode */
- ret = regmap_update_bits(priv->regmap, STM32_LPTIM_CR,
- STM32_LPTIM_CNTSTRT,
- STM32_LPTIM_CNTSTRT);
+ ret = regmap_set_bits(priv->regmap, STM32_LPTIM_CR,
+ STM32_LPTIM_CNTSTRT);
if (ret) {
regmap_write(priv->regmap, STM32_LPTIM_CR, 0);
goto err;
return ret;
}
-static void stm32_pwm_lp_get_state(struct pwm_chip *chip,
- struct pwm_device *pwm,
- struct pwm_state *state)
+static int stm32_pwm_lp_get_state(struct pwm_chip *chip,
+ struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct stm32_pwm_lp *priv = to_stm32_pwm_lp(chip);
unsigned long rate = clk_get_rate(priv->clk);
tmp = prd - val;
tmp = (tmp << presc) * NSEC_PER_SEC;
state->duty_cycle = DIV_ROUND_CLOSEST_ULL(tmp, rate);
+
+ return 0;
}
static const struct pwm_ops stm32_pwm_lp_ops = {
int ret;
/* Ensure registers have been updated, enable counter and capture */
- regmap_update_bits(priv->regmap, TIM_EGR, TIM_EGR_UG, TIM_EGR_UG);
- regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_CEN, TIM_CR1_CEN);
+ regmap_set_bits(priv->regmap, TIM_EGR, TIM_EGR_UG);
+ regmap_set_bits(priv->regmap, TIM_CR1, TIM_CR1_CEN);
/* Use cc1 or cc3 DMA resp for PWM input channels 1 & 2 or 3 & 4 */
dma_id = pwm->hwpwm < 2 ? STM32_TIMERS_DMA_CH1 : STM32_TIMERS_DMA_CH3;
ccen = pwm->hwpwm < 2 ? TIM_CCER_CC12E : TIM_CCER_CC34E;
ccr = pwm->hwpwm < 2 ? TIM_CCR1 : TIM_CCR3;
- regmap_update_bits(priv->regmap, TIM_CCER, ccen, ccen);
+ regmap_set_bits(priv->regmap, TIM_CCER, ccen);
/*
* Timer DMA burst mode. Request 2 registers, 2 bursts, to get both
}
stop:
- regmap_update_bits(priv->regmap, TIM_CCER, ccen, 0);
- regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_CEN, 0);
+ regmap_clear_bits(priv->regmap, TIM_CCER, ccen);
+ regmap_clear_bits(priv->regmap, TIM_CR1, TIM_CR1_CEN);
return ret;
}
regmap_write(priv->regmap, TIM_PSC, prescaler);
regmap_write(priv->regmap, TIM_ARR, prd - 1);
- regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_ARPE, TIM_CR1_ARPE);
+ regmap_set_bits(priv->regmap, TIM_CR1, TIM_CR1_ARPE);
/* Calculate the duty cycles */
dty = prd * duty_ns;
else
regmap_update_bits(priv->regmap, TIM_CCMR2, mask, ccmr);
- regmap_update_bits(priv->regmap, TIM_BDTR, TIM_BDTR_MOE, TIM_BDTR_MOE);
+ regmap_set_bits(priv->regmap, TIM_BDTR, TIM_BDTR_MOE);
return 0;
}
if (priv->have_complementary_output)
mask |= TIM_CCER_CC1NE << (ch * 4);
- regmap_update_bits(priv->regmap, TIM_CCER, mask, mask);
+ regmap_set_bits(priv->regmap, TIM_CCER, mask);
/* Make sure that registers are updated */
- regmap_update_bits(priv->regmap, TIM_EGR, TIM_EGR_UG, TIM_EGR_UG);
+ regmap_set_bits(priv->regmap, TIM_EGR, TIM_EGR_UG);
/* Enable controller */
- regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_CEN, TIM_CR1_CEN);
+ regmap_set_bits(priv->regmap, TIM_CR1, TIM_CR1_CEN);
return 0;
}
if (priv->have_complementary_output)
mask |= TIM_CCER_CC1NE << (ch * 4);
- regmap_update_bits(priv->regmap, TIM_CCER, mask, 0);
+ regmap_clear_bits(priv->regmap, TIM_CCER, mask);
/* When all channels are disabled, we can disable the controller */
if (!active_channels(priv))
- regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_CEN, 0);
+ regmap_clear_bits(priv->regmap, TIM_CR1, TIM_CR1_CEN);
clk_disable(priv->clk);
}
* If complementary bit doesn't exist writing 1 will have no
* effect so we can detect it.
*/
- regmap_update_bits(priv->regmap,
- TIM_CCER, TIM_CCER_CC1NE, TIM_CCER_CC1NE);
+ regmap_set_bits(priv->regmap, TIM_CCER, TIM_CCER_CC1NE);
regmap_read(priv->regmap, TIM_CCER, &ccer);
- regmap_update_bits(priv->regmap, TIM_CCER, TIM_CCER_CC1NE, 0);
+ regmap_clear_bits(priv->regmap, TIM_CCER, TIM_CCER_CC1NE);
priv->have_complementary_output = (ccer != 0);
}
* If channels enable bits don't exist writing 1 will have no
* effect so we can detect and count them.
*/
- regmap_update_bits(priv->regmap,
- TIM_CCER, TIM_CCER_CCXE, TIM_CCER_CCXE);
+ regmap_set_bits(priv->regmap, TIM_CCER, TIM_CCER_CCXE);
regmap_read(priv->regmap, TIM_CCER, &ccer);
- regmap_update_bits(priv->regmap, TIM_CCER, TIM_CCER_CCXE, 0);
+ regmap_clear_bits(priv->regmap, TIM_CCER, TIM_CCER_CCXE);
if (ccer & TIM_CCER_CC1E)
npwm++;
writel(val, chip->base + offset);
}
-static void sun4i_pwm_get_state(struct pwm_chip *chip,
- struct pwm_device *pwm,
- struct pwm_state *state)
+static int sun4i_pwm_get_state(struct pwm_chip *chip,
+ struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct sun4i_pwm_chip *sun4i_pwm = to_sun4i_pwm_chip(chip);
u64 clk_rate, tmp;
unsigned int prescaler;
clk_rate = clk_get_rate(sun4i_pwm->clk);
+ if (!clk_rate)
+ return -EINVAL;
val = sun4i_pwm_readl(sun4i_pwm, PWM_CTRL_REG);
state->duty_cycle = DIV_ROUND_UP_ULL(state->period, 2);
state->polarity = PWM_POLARITY_NORMAL;
state->enabled = true;
- return;
+ return 0;
}
if ((PWM_REG_PRESCAL(val, pwm->hwpwm) == PWM_PRESCAL_MASK) &&
prescaler = prescaler_table[PWM_REG_PRESCAL(val, pwm->hwpwm)];
if (prescaler == 0)
- return;
+ return -EINVAL;
if (val & BIT_CH(PWM_ACT_STATE, pwm->hwpwm))
state->polarity = PWM_POLARITY_NORMAL;
tmp = (u64)prescaler * NSEC_PER_SEC * PWM_REG_PRD(val);
state->period = DIV_ROUND_CLOSEST_ULL(tmp, clk_rate);
+
+ return 0;
}
static int sun4i_pwm_calculate(struct sun4i_pwm_chip *sun4i_pwm,
return 0;
}
-static void sunplus_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int sunplus_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct sunplus_pwm *priv = to_sunplus_pwm(chip);
u32 mode0, dd_freq, duty;
}
state->polarity = PWM_POLARITY_NORMAL;
+
+ return 0;
}
static const struct pwm_ops sunplus_pwm_ops = {
* source clock rate as required_clk_rate, PWM controller will
* be able to configure the requested period.
*/
- required_clk_rate =
- (NSEC_PER_SEC / period_ns) << PWM_DUTY_WIDTH;
+ required_clk_rate = DIV_ROUND_UP_ULL((u64)NSEC_PER_SEC << PWM_DUTY_WIDTH,
+ period_ns);
+
+ if (required_clk_rate > clk_round_rate(pc->clk, required_clk_rate))
+ /*
+ * required_clk_rate is a lower bound for the input
+ * rate; for lower rates there is no value for PWM_SCALE
+ * that yields a period less than or equal to the
+ * requested period. Hence, for lower rates, double the
+ * required_clk_rate to get a clock rate that can meet
+ * the requested period.
+ */
+ required_clk_rate *= 2;
err = dev_pm_opp_set_rate(pc->dev, required_clk_rate);
if (err < 0)
return 0;
}
-static void visconti_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state)
+static int visconti_pwm_get_state(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state)
{
struct visconti_pwm_chip *priv = visconti_pwm_from_chip(chip);
u32 period, duty, pwmc0, pwmc0_clk;
state->polarity = PWM_POLARITY_NORMAL;
state->enabled = true;
+
+ return 0;
}
static const struct pwm_ops visconti_pwm_ops = {
return 0;
}
-static void xilinx_pwm_get_state(struct pwm_chip *chip,
- struct pwm_device *unused,
- struct pwm_state *state)
+static int xilinx_pwm_get_state(struct pwm_chip *chip,
+ struct pwm_device *unused,
+ struct pwm_state *state)
{
struct xilinx_timer_priv *priv = xilinx_pwm_chip_to_priv(chip);
u32 tlr0, tlr1, tcsr0, tcsr1;
*/
if (state->period == state->duty_cycle)
state->duty_cycle = 0;
+
+ return 0;
}
static const struct pwm_ops xilinx_pwm_ops = {
It's safe to say N here if you're not interested in utilizing
a slave processor.
+config XLNX_R5_REMOTEPROC
+ tristate "Xilinx R5 remoteproc support"
+ depends on PM && ARCH_ZYNQMP
+ select ZYNQMP_FIRMWARE
+ select RPMSG_VIRTIO
+ select MAILBOX
+ select ZYNQMP_IPI_MBOX
+ help
+ Say y or m here to support Xilinx R5 remote processors via the remote
+ processor framework.
+
+ It's safe to say N if not interested in using RPU r5f cores.
+
endif # REMOTEPROC
endmenu
obj-$(CONFIG_STM32_RPROC) += stm32_rproc.o
obj-$(CONFIG_TI_K3_DSP_REMOTEPROC) += ti_k3_dsp_remoteproc.o
obj-$(CONFIG_TI_K3_R5_REMOTEPROC) += ti_k3_r5_remoteproc.o
+obj-$(CONFIG_XLNX_R5_REMOTEPROC) += xlnx_r5_remoteproc.o
struct device *dev = rproc->dev.parent;
int ret = 0;
- /* Make sure work is finished */
- flush_work(&priv->rproc_work);
-
if (rproc->state == RPROC_CRASHED) {
priv->flags &= ~REMOTE_IS_READY;
return 0;
{
struct imx_dsp_rproc *priv = container_of(work, struct imx_dsp_rproc,
rproc_work);
+ struct rproc *rproc = priv->rproc;
+
+ mutex_lock(&rproc->lock);
+
+ if (rproc->state != RPROC_RUNNING)
+ goto unlock_mutex;
rproc_vq_interrupt(priv->rproc, 0);
rproc_vq_interrupt(priv->rproc, 1);
+
+unlock_mutex:
+ mutex_unlock(&rproc->lock);
}
/**
* Copyright (c) 2017 Pengutronix, Oleksij Rempel <kernel@pengutronix.de>
*/
+#include <dt-bindings/firmware/imx/rsrc.h>
#include <linux/arm-smccc.h>
#include <linux/clk.h>
#include <linux/err.h>
+#include <linux/firmware/imx/sci.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
#include <linux/mailbox_client.h>
#include <linux/of_reserved_mem.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
+#include <linux/pm_domain.h>
#include <linux/regmap.h>
#include <linux/remoteproc.h>
#include <linux/workqueue.h>
#define IMX_SIP_RPROC_STARTED 0x01
#define IMX_SIP_RPROC_STOP 0x02
+#define IMX_SC_IRQ_GROUP_REBOOTED 5
+
/**
* struct imx_rproc_mem - slim internal memory structure
* @cpu_addr: MPU virtual address of the memory region
size_t size;
};
-/* att flags */
+/* att flags: lower 16 bits specifying core, higher 16 bits for flags */
/* M4 own area. Can be mapped at probe */
-#define ATT_OWN BIT(1)
-#define ATT_IOMEM BIT(2)
+#define ATT_OWN BIT(31)
+#define ATT_IOMEM BIT(30)
+
+#define ATT_CORE_MASK 0xffff
+#define ATT_CORE(I) BIT((I))
+
+static int imx_rproc_xtr_mbox_init(struct rproc *rproc);
+static void imx_rproc_free_mbox(struct rproc *rproc);
+static int imx_rproc_detach_pd(struct rproc *rproc);
struct imx_rproc {
struct device *dev;
struct work_struct rproc_work;
struct workqueue_struct *workqueue;
void __iomem *rsc_table;
+ struct imx_sc_ipc *ipc_handle;
+ struct notifier_block rproc_nb;
+ u32 rproc_pt; /* partition id */
+ u32 rsrc_id; /* resource id */
+ u32 entry; /* cpu start address */
+ int num_pd;
+ u32 core_index;
+ struct device **pd_dev;
+ struct device_link **pd_dev_link;
};
static const struct imx_rproc_att imx_rproc_att_imx93[] = {
{ 0x80000000, 0x80000000, 0x10000000, 0 },
{ 0x90000000, 0x80000000, 0x10000000, 0 },
- { 0xC0000000, 0xa0000000, 0x10000000, 0 },
- { 0xD0000000, 0xa0000000, 0x10000000, 0 },
+ { 0xC0000000, 0xC0000000, 0x10000000, 0 },
+ { 0xD0000000, 0xC0000000, 0x10000000, 0 },
+};
+
+static const struct imx_rproc_att imx_rproc_att_imx8qm[] = {
+ /* dev addr , sys addr , size , flags */
+ { 0x08000000, 0x08000000, 0x10000000, 0},
+ /* TCML */
+ { 0x1FFE0000, 0x34FE0000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(0)},
+ { 0x1FFE0000, 0x38FE0000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(1)},
+ /* TCMU */
+ { 0x20000000, 0x35000000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(0)},
+ { 0x20000000, 0x39000000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(1)},
+ /* DDR (Data) */
+ { 0x80000000, 0x80000000, 0x60000000, 0 },
+};
+
+static const struct imx_rproc_att imx_rproc_att_imx8qxp[] = {
+ { 0x08000000, 0x08000000, 0x10000000, 0 },
+ /* TCML/U */
+ { 0x1FFE0000, 0x34FE0000, 0x00040000, ATT_OWN | ATT_IOMEM },
+ /* OCRAM(Low 96KB) */
+ { 0x21000000, 0x00100000, 0x00018000, 0 },
+ /* OCRAM */
+ { 0x21100000, 0x00100000, 0x00040000, 0 },
+ /* DDR (Data) */
+ { 0x80000000, 0x80000000, 0x60000000, 0 },
};
static const struct imx_rproc_att imx_rproc_att_imx8mn[] = {
.method = IMX_RPROC_MMIO,
};
+static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qm = {
+ .att = imx_rproc_att_imx8qm,
+ .att_size = ARRAY_SIZE(imx_rproc_att_imx8qm),
+ .method = IMX_RPROC_SCU_API,
+};
+
+static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qxp = {
+ .att = imx_rproc_att_imx8qxp,
+ .att_size = ARRAY_SIZE(imx_rproc_att_imx8qxp),
+ .method = IMX_RPROC_SCU_API,
+};
+
static const struct imx_rproc_dcfg imx_rproc_cfg_imx8ulp = {
.att = imx_rproc_att_imx8ulp,
.att_size = ARRAY_SIZE(imx_rproc_att_imx8ulp),
struct arm_smccc_res res;
int ret;
+ ret = imx_rproc_xtr_mbox_init(rproc);
+ if (ret)
+ return ret;
+
switch (dcfg->method) {
case IMX_RPROC_MMIO:
ret = regmap_update_bits(priv->regmap, dcfg->src_reg, dcfg->src_mask,
arm_smccc_smc(IMX_SIP_RPROC, IMX_SIP_RPROC_START, 0, 0, 0, 0, 0, 0, &res);
ret = res.a0;
break;
+ case IMX_RPROC_SCU_API:
+ ret = imx_sc_pm_cpu_start(priv->ipc_handle, priv->rsrc_id, true, priv->entry);
+ break;
default:
return -EOPNOTSUPP;
}
if (res.a1)
dev_info(dev, "Not in wfi, force stopped\n");
break;
+ case IMX_RPROC_SCU_API:
+ ret = imx_sc_pm_cpu_start(priv->ipc_handle, priv->rsrc_id, false, priv->entry);
+ break;
default:
return -EOPNOTSUPP;
}
if (ret)
dev_err(dev, "Failed to stop remote core\n");
+ else
+ imx_rproc_free_mbox(rproc);
return ret;
}
for (i = 0; i < dcfg->att_size; i++) {
const struct imx_rproc_att *att = &dcfg->att[i];
+ /*
+ * Ignore entries not belong to current core:
+ * i.MX8QM has dual general M4_[0,1] cores, M4_0's own entries
+ * has "ATT_CORE(0) & BIT(0)" true, M4_1's own entries has
+ * "ATT_CORE(1) & BIT(1)" true.
+ */
+ if (att->flags & ATT_CORE_MASK) {
+ if (!((BIT(priv->core_index)) & (att->flags & ATT_CORE_MASK)))
+ continue;
+ }
+
if (da >= att->da && da + len < att->da + att->size) {
unsigned int offset = da - att->da;
static int imx_rproc_attach(struct rproc *rproc)
{
+ return imx_rproc_xtr_mbox_init(rproc);
+}
+
+static int imx_rproc_detach(struct rproc *rproc)
+{
+ struct imx_rproc *priv = rproc->priv;
+ const struct imx_rproc_dcfg *dcfg = priv->dcfg;
+
+ if (dcfg->method != IMX_RPROC_SCU_API)
+ return -EOPNOTSUPP;
+
+ if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id))
+ return -EOPNOTSUPP;
+
+ imx_rproc_free_mbox(rproc);
+
return 0;
}
static const struct rproc_ops imx_rproc_ops = {
.prepare = imx_rproc_prepare,
.attach = imx_rproc_attach,
+ .detach = imx_rproc_detach,
.start = imx_rproc_start,
.stop = imx_rproc_stop,
.kick = imx_rproc_kick,
struct device *dev = priv->dev;
struct mbox_client *cl;
+ /*
+ * stop() and detach() will free the mbox channels, so need
+ * to request mbox channels in start() and attach().
+ *
+ * Because start() and attach() not able to handle mbox defer
+ * probe, imx_rproc_xtr_mbox_init is also called in probe().
+ * The check is to avoid request mbox again when start() or
+ * attach() after probe() returns success.
+ */
+ if (priv->tx_ch && priv->rx_ch)
+ return 0;
+
if (!of_get_property(dev->of_node, "mbox-names", NULL))
return 0;
{
struct imx_rproc *priv = rproc->priv;
- mbox_free_channel(priv->tx_ch);
- mbox_free_channel(priv->rx_ch);
+ if (priv->tx_ch) {
+ mbox_free_channel(priv->tx_ch);
+ priv->tx_ch = NULL;
+ }
+
+ if (priv->rx_ch) {
+ mbox_free_channel(priv->rx_ch);
+ priv->rx_ch = NULL;
+ }
+}
+
+static void imx_rproc_put_scu(struct rproc *rproc)
+{
+ struct imx_rproc *priv = rproc->priv;
+ const struct imx_rproc_dcfg *dcfg = priv->dcfg;
+
+ if (dcfg->method != IMX_RPROC_SCU_API)
+ return;
+
+ if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id)) {
+ imx_rproc_detach_pd(rproc);
+ return;
+ }
+
+ imx_scu_irq_group_enable(IMX_SC_IRQ_GROUP_REBOOTED, BIT(priv->rproc_pt), false);
+ imx_scu_irq_unregister_notifier(&priv->rproc_nb);
+}
+
+static int imx_rproc_partition_notify(struct notifier_block *nb,
+ unsigned long event, void *group)
+{
+ struct imx_rproc *priv = container_of(nb, struct imx_rproc, rproc_nb);
+
+ /* Ignore other irqs */
+ if (!((event & BIT(priv->rproc_pt)) && (*(u8 *)group == IMX_SC_IRQ_GROUP_REBOOTED)))
+ return 0;
+
+ rproc_report_crash(priv->rproc, RPROC_WATCHDOG);
+
+ pr_info("Partition%d reset!\n", priv->rproc_pt);
+
+ return 0;
+}
+
+static int imx_rproc_attach_pd(struct imx_rproc *priv)
+{
+ struct device *dev = priv->dev;
+ int ret, i;
+
+ /*
+ * If there is only one power-domain entry, the platform driver framework
+ * will handle it, no need handle it in this driver.
+ */
+ priv->num_pd = of_count_phandle_with_args(dev->of_node, "power-domains",
+ "#power-domain-cells");
+ if (priv->num_pd <= 1)
+ return 0;
+
+ priv->pd_dev = devm_kmalloc_array(dev, priv->num_pd, sizeof(*priv->pd_dev), GFP_KERNEL);
+ if (!priv->pd_dev)
+ return -ENOMEM;
+
+ priv->pd_dev_link = devm_kmalloc_array(dev, priv->num_pd, sizeof(*priv->pd_dev_link),
+ GFP_KERNEL);
+
+ if (!priv->pd_dev_link)
+ return -ENOMEM;
+
+ for (i = 0; i < priv->num_pd; i++) {
+ priv->pd_dev[i] = dev_pm_domain_attach_by_id(dev, i);
+ if (IS_ERR(priv->pd_dev[i])) {
+ ret = PTR_ERR(priv->pd_dev[i]);
+ goto detach_pd;
+ }
+
+ priv->pd_dev_link[i] = device_link_add(dev, priv->pd_dev[i], DL_FLAG_STATELESS |
+ DL_FLAG_PM_RUNTIME | DL_FLAG_RPM_ACTIVE);
+ if (!priv->pd_dev_link[i]) {
+ dev_pm_domain_detach(priv->pd_dev[i], false);
+ ret = -EINVAL;
+ goto detach_pd;
+ }
+ }
+
+ return 0;
+
+detach_pd:
+ while (--i >= 0) {
+ device_link_del(priv->pd_dev_link[i]);
+ dev_pm_domain_detach(priv->pd_dev[i], false);
+ }
+
+ return ret;
+}
+
+static int imx_rproc_detach_pd(struct rproc *rproc)
+{
+ struct imx_rproc *priv = rproc->priv;
+ int i;
+
+ /*
+ * If there is only one power-domain entry, the platform driver framework
+ * will handle it, no need handle it in this driver.
+ */
+ if (priv->num_pd <= 1)
+ return 0;
+
+ for (i = 0; i < priv->num_pd; i++) {
+ device_link_del(priv->pd_dev_link[i]);
+ dev_pm_domain_detach(priv->pd_dev[i], false);
+ }
+
+ return 0;
}
static int imx_rproc_detect_mode(struct imx_rproc *priv)
struct arm_smccc_res res;
int ret;
u32 val;
+ u8 pt;
switch (dcfg->method) {
case IMX_RPROC_NONE:
if (res.a0)
priv->rproc->state = RPROC_DETACHED;
return 0;
+ case IMX_RPROC_SCU_API:
+ ret = imx_scu_get_handle(&priv->ipc_handle);
+ if (ret)
+ return ret;
+ ret = of_property_read_u32(dev->of_node, "fsl,resource-id", &priv->rsrc_id);
+ if (ret) {
+ dev_err(dev, "No fsl,resource-id property\n");
+ return ret;
+ }
+
+ if (priv->rsrc_id == IMX_SC_R_M4_1_PID0)
+ priv->core_index = 1;
+ else
+ priv->core_index = 0;
+
+ /*
+ * If Mcore resource is not owned by Acore partition, It is kicked by ROM,
+ * and Linux could only do IPC with Mcore and nothing else.
+ */
+ if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id)) {
+ if (of_property_read_u32(dev->of_node, "fsl,entry-address", &priv->entry))
+ return -EINVAL;
+
+ return imx_rproc_attach_pd(priv);
+ }
+
+ priv->rproc->state = RPROC_DETACHED;
+ priv->rproc->recovery_disabled = false;
+ rproc_set_feature(priv->rproc, RPROC_FEAT_ATTACH_ON_RECOVERY);
+
+ /* Get partition id and enable irq in SCFW */
+ ret = imx_sc_rm_get_resource_owner(priv->ipc_handle, priv->rsrc_id, &pt);
+ if (ret) {
+ dev_err(dev, "not able to get resource owner\n");
+ return ret;
+ }
+
+ priv->rproc_pt = pt;
+ priv->rproc_nb.notifier_call = imx_rproc_partition_notify;
+
+ ret = imx_scu_irq_register_notifier(&priv->rproc_nb);
+ if (ret) {
+ dev_err(dev, "register scu notifier failed, %d\n", ret);
+ return ret;
+ }
+
+ ret = imx_scu_irq_group_enable(IMX_SC_IRQ_GROUP_REBOOTED, BIT(priv->rproc_pt),
+ true);
+ if (ret) {
+ imx_scu_irq_unregister_notifier(&priv->rproc_nb);
+ dev_err(dev, "Enable irq failed, %d\n", ret);
+ return ret;
+ }
+
+ return 0;
default:
break;
}
ret = imx_rproc_clk_enable(priv);
if (ret)
- goto err_put_mbox;
+ goto err_put_scu;
INIT_WORK(&priv->rproc_work, imx_rproc_vq_work);
err_put_clk:
clk_disable_unprepare(priv->clk);
+err_put_scu:
+ imx_rproc_put_scu(rproc);
err_put_mbox:
imx_rproc_free_mbox(rproc);
err_put_wkq:
clk_disable_unprepare(priv->clk);
rproc_del(rproc);
+ imx_rproc_put_scu(rproc);
imx_rproc_free_mbox(rproc);
destroy_workqueue(priv->workqueue);
rproc_free(rproc);
{ .compatible = "fsl,imx8mm-cm4", .data = &imx_rproc_cfg_imx8mq },
{ .compatible = "fsl,imx8mn-cm7", .data = &imx_rproc_cfg_imx8mn },
{ .compatible = "fsl,imx8mp-cm7", .data = &imx_rproc_cfg_imx8mn },
+ { .compatible = "fsl,imx8qxp-cm4", .data = &imx_rproc_cfg_imx8qxp },
+ { .compatible = "fsl,imx8qm-cm4", .data = &imx_rproc_cfg_imx8qm },
{ .compatible = "fsl,imx8ulp-cm33", .data = &imx_rproc_cfg_imx8ulp },
{ .compatible = "fsl,imx93-cm33", .data = &imx_rproc_cfg_imx93 },
{},
}
ret = of_address_to_resource(node, 0, &r);
+ of_node_put(node);
if (ret)
return ret;
detach_proxy_pds:
adsp_pds_detach(adsp, adsp->proxy_pds, adsp->proxy_pd_count);
free_rproc:
+ device_init_wakeup(adsp->dev, false);
rproc_free(rproc);
return ret;
qcom_remove_sysmon_subdev(adsp->sysmon);
qcom_remove_smd_subdev(adsp->rproc, &adsp->smd_subdev);
qcom_remove_ssr_subdev(adsp->rproc, &adsp->ssr_subdev);
+ adsp_pds_detach(adsp, adsp->proxy_pds, adsp->proxy_pd_count);
+ device_init_wakeup(adsp->dev, false);
rproc_free(adsp->rproc);
return 0;
if (ret) {
dev_err(wcss->dev,
"xo cbcr enabling timed out (rc:%d)\n", ret);
- return ret;
+ goto disable_xo_cbcr_clk;
}
writel(0, wcss->reg_base + Q6SS_CGC_OVERRIDE);
val = readl(wcss->reg_base + Q6SS_SLEEP_CBCR);
val &= ~Q6SS_CLK_ENABLE;
writel(val, wcss->reg_base + Q6SS_SLEEP_CBCR);
+disable_xo_cbcr_clk:
val = readl(wcss->reg_base + Q6SS_XO_CBCR);
val &= ~Q6SS_CLK_ENABLE;
writel(val, wcss->reg_base + Q6SS_XO_CBCR);
int ret;
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "qdsp6");
+ if (!res)
+ return -EINVAL;
+
wcss->reg_base = devm_ioremap(&pdev->dev, res->start,
resource_size(res));
if (!wcss->reg_base)
struct qmi_response_type_v01 resp;
};
-static struct qmi_elem_info ssctl_shutdown_resp_ei[] = {
+static const struct qmi_elem_info ssctl_shutdown_resp_ei[] = {
{
.data_type = QMI_STRUCT,
.elem_len = 1,
u32 evt_driven;
};
-static struct qmi_elem_info ssctl_subsys_event_req_ei[] = {
+static const struct qmi_elem_info ssctl_subsys_event_req_ei[] = {
{
.data_type = QMI_DATA_LEN,
.elem_len = 1,
struct qmi_response_type_v01 resp;
};
-static struct qmi_elem_info ssctl_subsys_event_resp_ei[] = {
+static const struct qmi_elem_info ssctl_subsys_event_resp_ei[] = {
{
.data_type = QMI_STRUCT,
.elem_len = 1,
{}
};
-static struct qmi_elem_info ssctl_shutdown_ind_ei[] = {
+static const struct qmi_elem_info ssctl_shutdown_ind_ei[] = {
{}
};
if (sysmon->shutdown_irq != -ENODATA) {
dev_err(sysmon->dev,
"failed to retrieve shutdown-ack IRQ\n");
- return ERR_PTR(sysmon->shutdown_irq);
+ ret = sysmon->shutdown_irq;
+ kfree(sysmon);
+ return ERR_PTR(ret);
}
} else {
ret = devm_request_threaded_irq(sysmon->dev,
if (ret) {
dev_err(sysmon->dev,
"failed to acquire shutdown-ack IRQ\n");
+ kfree(sysmon);
return ERR_PTR(ret);
}
}
rvdev_data.rsc_offset = offset;
rvdev_data.rsc = rsc;
- pdev = platform_device_register_data(dev, "rproc-virtio", rvdev_data.index, &rvdev_data,
+ /*
+ * When there is more than one remote processor, rproc->nb_vdev number is
+ * same for each separate instances of "rproc". If rvdev_data.index is used
+ * as device id, then we get duplication in sysfs, so need to use
+ * PLATFORM_DEVID_AUTO to auto select device id.
+ */
+ pdev = platform_device_register_data(dev, "rproc-virtio", PLATFORM_DEVID_AUTO, &rvdev_data,
sizeof(rvdev_data));
if (IS_ERR(pdev)) {
dev_err(dev, "failed to create rproc-virtio device\n");
mutex_lock(&rproc->lock);
- if (rproc->state == RPROC_CRASHED || rproc->state == RPROC_OFFLINE) {
+ if (rproc->state == RPROC_CRASHED) {
/* handle only the first crash detected */
mutex_unlock(&rproc->lock);
return;
}
+ if (rproc->state == RPROC_OFFLINE) {
+ /* Don't recover if the remote processor was stopped */
+ mutex_unlock(&rproc->lock);
+ goto out;
+ }
+
rproc->state = RPROC_CRASHED;
dev_err(dev, "handling crash #%u in %s\n", ++rproc->crash_cnt,
rproc->name);
if (!rproc->recovery_disabled)
rproc_trigger_recovery(rproc);
+out:
pm_relax(rproc->dev.parent);
}
rcu_read_lock();
list_for_each_entry_rcu(r, &rproc_list, node) {
- if (r->dev.parent && r->dev.parent->of_node == np) {
+ if (r->dev.parent && device_match_of_node(r->dev.parent, np)) {
/* prevent underlying implementation from being removed */
if (!try_module_get(r->dev.parent->driver->owner)) {
dev_err(&r->dev, "can't get owner\n");
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ZynqMP R5 Remote Processor driver
+ *
+ */
+
+#include <dt-bindings/power/xlnx-zynqmp-power.h>
+#include <linux/dma-mapping.h>
+#include <linux/firmware/xlnx-zynqmp.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/remoteproc.h>
+#include <linux/slab.h>
+
+#include "remoteproc_internal.h"
+
+/*
+ * settings for RPU cluster mode which
+ * reflects possible values of xlnx,cluster-mode dt-property
+ */
+enum zynqmp_r5_cluster_mode {
+ SPLIT_MODE = 0, /* When cores run as separate processor */
+ LOCKSTEP_MODE = 1, /* cores execute same code in lockstep,clk-for-clk */
+ SINGLE_CPU_MODE = 2, /* core0 is held in reset and only core1 runs */
+};
+
+/**
+ * struct mem_bank_data - Memory Bank description
+ *
+ * @addr: Start address of memory bank
+ * @size: Size of Memory bank
+ * @pm_domain_id: Power-domains id of memory bank for firmware to turn on/off
+ * @bank_name: name of the bank for remoteproc framework
+ */
+struct mem_bank_data {
+ phys_addr_t addr;
+ size_t size;
+ u32 pm_domain_id;
+ char *bank_name;
+};
+
+/*
+ * Hardcoded TCM bank values. This will be removed once TCM bindings are
+ * accepted for system-dt specifications and upstreamed in linux kernel
+ */
+static const struct mem_bank_data zynqmp_tcm_banks[] = {
+ {0xffe00000UL, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
+ {0xffe20000UL, 0x10000UL, PD_R5_0_BTCM, "btcm0"},
+ {0xffe90000UL, 0x10000UL, PD_R5_1_ATCM, "atcm1"},
+ {0xffeb0000UL, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
+};
+
+/**
+ * struct zynqmp_r5_core
+ *
+ * @dev: device of RPU instance
+ * @np: device node of RPU instance
+ * @tcm_bank_count: number TCM banks accessible to this RPU
+ * @tcm_banks: array of each TCM bank data
+ * @rmem_count: Number of reserved mem regions
+ * @rmem: reserved memory region nodes from device tree
+ * @rproc: rproc handle
+ * @pm_domain_id: RPU CPU power domain id
+ */
+struct zynqmp_r5_core {
+ struct device *dev;
+ struct device_node *np;
+ int tcm_bank_count;
+ struct mem_bank_data **tcm_banks;
+ int rmem_count;
+ struct reserved_mem **rmem;
+ struct rproc *rproc;
+ u32 pm_domain_id;
+};
+
+/**
+ * struct zynqmp_r5_cluster
+ *
+ * @dev: r5f subsystem cluster device node
+ * @mode: cluster mode of type zynqmp_r5_cluster_mode
+ * @core_count: number of r5 cores used for this cluster mode
+ * @r5_cores: Array of pointers pointing to r5 core
+ */
+struct zynqmp_r5_cluster {
+ struct device *dev;
+ enum zynqmp_r5_cluster_mode mode;
+ int core_count;
+ struct zynqmp_r5_core **r5_cores;
+};
+
+/*
+ * zynqmp_r5_set_mode()
+ *
+ * set RPU cluster and TCM operation mode
+ *
+ * @r5_core: pointer to zynqmp_r5_core type object
+ * @fw_reg_val: value expected by firmware to configure RPU cluster mode
+ * @tcm_mode: value expected by fw to configure TCM mode (lockstep or split)
+ *
+ * Return: 0 for success and < 0 for failure
+ */
+static int zynqmp_r5_set_mode(struct zynqmp_r5_core *r5_core,
+ enum rpu_oper_mode fw_reg_val,
+ enum rpu_tcm_comb tcm_mode)
+{
+ int ret;
+
+ ret = zynqmp_pm_set_rpu_mode(r5_core->pm_domain_id, fw_reg_val);
+ if (ret < 0) {
+ dev_err(r5_core->dev, "failed to set RPU mode\n");
+ return ret;
+ }
+
+ ret = zynqmp_pm_set_tcm_config(r5_core->pm_domain_id, tcm_mode);
+ if (ret < 0)
+ dev_err(r5_core->dev, "failed to configure TCM\n");
+
+ return ret;
+}
+
+/*
+ * zynqmp_r5_rproc_start()
+ * @rproc: single R5 core's corresponding rproc instance
+ *
+ * Start R5 Core from designated boot address.
+ *
+ * return 0 on success, otherwise non-zero value on failure
+ */
+static int zynqmp_r5_rproc_start(struct rproc *rproc)
+{
+ struct zynqmp_r5_core *r5_core = rproc->priv;
+ enum rpu_boot_mem bootmem;
+ int ret;
+
+ /*
+ * The exception vector pointers (EVP) refer to the base-address of
+ * exception vectors (for reset, IRQ, FIQ, etc). The reset-vector
+ * starts at the base-address and subsequent vectors are on 4-byte
+ * boundaries.
+ *
+ * Exception vectors can start either from 0x0000_0000 (LOVEC) or
+ * from 0xFFFF_0000 (HIVEC) which is mapped in the OCM (On-Chip Memory)
+ *
+ * Usually firmware will put Exception vectors at LOVEC.
+ *
+ * It is not recommend that you change the exception vector.
+ * Changing the EVP to HIVEC will result in increased interrupt latency
+ * and jitter. Also, if the OCM is secured and the Cortex-R5F processor
+ * is non-secured, then the Cortex-R5F processor cannot access the
+ * HIVEC exception vectors in the OCM.
+ */
+ bootmem = (rproc->bootaddr >= 0xFFFC0000) ?
+ PM_RPU_BOOTMEM_HIVEC : PM_RPU_BOOTMEM_LOVEC;
+
+ dev_dbg(r5_core->dev, "RPU boot addr 0x%llx from %s.", rproc->bootaddr,
+ bootmem == PM_RPU_BOOTMEM_HIVEC ? "OCM" : "TCM");
+
+ ret = zynqmp_pm_request_wake(r5_core->pm_domain_id, 1,
+ bootmem, ZYNQMP_PM_REQUEST_ACK_NO);
+ if (ret)
+ dev_err(r5_core->dev,
+ "failed to start RPU = 0x%x\n", r5_core->pm_domain_id);
+ return ret;
+}
+
+/*
+ * zynqmp_r5_rproc_stop()
+ * @rproc: single R5 core's corresponding rproc instance
+ *
+ * Power down R5 Core.
+ *
+ * return 0 on success, otherwise non-zero value on failure
+ */
+static int zynqmp_r5_rproc_stop(struct rproc *rproc)
+{
+ struct zynqmp_r5_core *r5_core = rproc->priv;
+ int ret;
+
+ ret = zynqmp_pm_force_pwrdwn(r5_core->pm_domain_id,
+ ZYNQMP_PM_REQUEST_ACK_BLOCKING);
+ if (ret)
+ dev_err(r5_core->dev, "failed to stop remoteproc RPU %d\n", ret);
+
+ return ret;
+}
+
+/*
+ * zynqmp_r5_mem_region_map()
+ * @rproc: single R5 core's corresponding rproc instance
+ * @mem: mem descriptor to map reserved memory-regions
+ *
+ * Callback to map va for memory-region's carveout.
+ *
+ * return 0 on success, otherwise non-zero value on failure
+ */
+static int zynqmp_r5_mem_region_map(struct rproc *rproc,
+ struct rproc_mem_entry *mem)
+{
+ void __iomem *va;
+
+ va = ioremap_wc(mem->dma, mem->len);
+ if (IS_ERR_OR_NULL(va))
+ return -ENOMEM;
+
+ mem->va = (void *)va;
+
+ return 0;
+}
+
+/*
+ * zynqmp_r5_rproc_mem_unmap
+ * @rproc: single R5 core's corresponding rproc instance
+ * @mem: mem entry to unmap
+ *
+ * Unmap memory-region carveout
+ *
+ * return: always returns 0
+ */
+static int zynqmp_r5_mem_region_unmap(struct rproc *rproc,
+ struct rproc_mem_entry *mem)
+{
+ iounmap((void __iomem *)mem->va);
+ return 0;
+}
+
+/*
+ * add_mem_regions_carveout()
+ * @rproc: single R5 core's corresponding rproc instance
+ *
+ * Construct rproc mem carveouts from memory-region property nodes
+ *
+ * return 0 on success, otherwise non-zero value on failure
+ */
+static int add_mem_regions_carveout(struct rproc *rproc)
+{
+ struct rproc_mem_entry *rproc_mem;
+ struct zynqmp_r5_core *r5_core;
+ struct reserved_mem *rmem;
+ int i, num_mem_regions;
+
+ r5_core = (struct zynqmp_r5_core *)rproc->priv;
+ num_mem_regions = r5_core->rmem_count;
+
+ for (i = 0; i < num_mem_regions; i++) {
+ rmem = r5_core->rmem[i];
+
+ if (!strncmp(rmem->name, "vdev0buffer", strlen("vdev0buffer"))) {
+ /* Init reserved memory for vdev buffer */
+ rproc_mem = rproc_of_resm_mem_entry_init(&rproc->dev, i,
+ rmem->size,
+ rmem->base,
+ rmem->name);
+ } else {
+ /* Register associated reserved memory regions */
+ rproc_mem = rproc_mem_entry_init(&rproc->dev, NULL,
+ (dma_addr_t)rmem->base,
+ rmem->size, rmem->base,
+ zynqmp_r5_mem_region_map,
+ zynqmp_r5_mem_region_unmap,
+ rmem->name);
+ }
+
+ if (!rproc_mem)
+ return -ENOMEM;
+
+ rproc_add_carveout(rproc, rproc_mem);
+
+ dev_dbg(&rproc->dev, "reserved mem carveout %s addr=%llx, size=0x%llx",
+ rmem->name, rmem->base, rmem->size);
+ }
+
+ return 0;
+}
+
+/*
+ * tcm_mem_unmap()
+ * @rproc: single R5 core's corresponding rproc instance
+ * @mem: tcm mem entry to unmap
+ *
+ * Unmap TCM banks when powering down R5 core.
+ *
+ * return always 0
+ */
+static int tcm_mem_unmap(struct rproc *rproc, struct rproc_mem_entry *mem)
+{
+ iounmap((void __iomem *)mem->va);
+
+ return 0;
+}
+
+/*
+ * tcm_mem_map()
+ * @rproc: single R5 core's corresponding rproc instance
+ * @mem: tcm memory entry descriptor
+ *
+ * Given TCM bank entry, this func setup virtual address for TCM bank
+ * remoteproc carveout. It also takes care of va to da address translation
+ *
+ * return 0 on success, otherwise non-zero value on failure
+ */
+static int tcm_mem_map(struct rproc *rproc,
+ struct rproc_mem_entry *mem)
+{
+ void __iomem *va;
+
+ va = ioremap_wc(mem->dma, mem->len);
+ if (IS_ERR_OR_NULL(va))
+ return -ENOMEM;
+
+ /* Update memory entry va */
+ mem->va = (void *)va;
+
+ /* clear TCMs */
+ memset_io(va, 0, mem->len);
+
+ /*
+ * The R5s expect their TCM banks to be at address 0x0 and 0x2000,
+ * while on the Linux side they are at 0xffexxxxx.
+ *
+ * Zero out the high 12 bits of the address. This will give
+ * expected values for TCM Banks 0A and 0B (0x0 and 0x20000).
+ */
+ mem->da &= 0x000fffff;
+
+ /*
+ * TCM Banks 1A and 1B still have to be translated.
+ *
+ * Below handle these two banks' absolute addresses (0xffe90000 and
+ * 0xffeb0000) and convert to the expected relative addresses
+ * (0x0 and 0x20000).
+ */
+ if (mem->da == 0x90000 || mem->da == 0xB0000)
+ mem->da -= 0x90000;
+
+ /* if translated TCM bank address is not valid report error */
+ if (mem->da != 0x0 && mem->da != 0x20000) {
+ dev_err(&rproc->dev, "invalid TCM address: %x\n", mem->da);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+/*
+ * add_tcm_carveout_split_mode()
+ * @rproc: single R5 core's corresponding rproc instance
+ *
+ * allocate and add remoteproc carveout for TCM memory in split mode
+ *
+ * return 0 on success, otherwise non-zero value on failure
+ */
+static int add_tcm_carveout_split_mode(struct rproc *rproc)
+{
+ struct rproc_mem_entry *rproc_mem;
+ struct zynqmp_r5_core *r5_core;
+ int i, num_banks, ret;
+ phys_addr_t bank_addr;
+ struct device *dev;
+ u32 pm_domain_id;
+ size_t bank_size;
+ char *bank_name;
+
+ r5_core = (struct zynqmp_r5_core *)rproc->priv;
+ dev = r5_core->dev;
+ num_banks = r5_core->tcm_bank_count;
+
+ /*
+ * Power-on Each 64KB TCM,
+ * register its address space, map and unmap functions
+ * and add carveouts accordingly
+ */
+ for (i = 0; i < num_banks; i++) {
+ bank_addr = r5_core->tcm_banks[i]->addr;
+ bank_name = r5_core->tcm_banks[i]->bank_name;
+ bank_size = r5_core->tcm_banks[i]->size;
+ pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
+
+ ret = zynqmp_pm_request_node(pm_domain_id,
+ ZYNQMP_PM_CAPABILITY_ACCESS, 0,
+ ZYNQMP_PM_REQUEST_ACK_BLOCKING);
+ if (ret < 0) {
+ dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
+ goto release_tcm_split;
+ }
+
+ dev_dbg(dev, "TCM carveout split mode %s addr=%llx, size=0x%lx",
+ bank_name, bank_addr, bank_size);
+
+ rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
+ bank_size, bank_addr,
+ tcm_mem_map, tcm_mem_unmap,
+ bank_name);
+ if (!rproc_mem) {
+ ret = -ENOMEM;
+ zynqmp_pm_release_node(pm_domain_id);
+ goto release_tcm_split;
+ }
+
+ rproc_add_carveout(rproc, rproc_mem);
+ }
+
+ return 0;
+
+release_tcm_split:
+ /* If failed, Turn off all TCM banks turned on before */
+ for (i--; i >= 0; i--) {
+ pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
+ zynqmp_pm_release_node(pm_domain_id);
+ }
+ return ret;
+}
+
+/*
+ * add_tcm_carveout_lockstep_mode()
+ * @rproc: single R5 core's corresponding rproc instance
+ *
+ * allocate and add remoteproc carveout for TCM memory in lockstep mode
+ *
+ * return 0 on success, otherwise non-zero value on failure
+ */
+static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
+{
+ struct rproc_mem_entry *rproc_mem;
+ struct zynqmp_r5_core *r5_core;
+ int i, num_banks, ret;
+ phys_addr_t bank_addr;
+ size_t bank_size = 0;
+ struct device *dev;
+ u32 pm_domain_id;
+ char *bank_name;
+
+ r5_core = (struct zynqmp_r5_core *)rproc->priv;
+ dev = r5_core->dev;
+
+ /* Go through zynqmp banks for r5 node */
+ num_banks = r5_core->tcm_bank_count;
+
+ /*
+ * In lockstep mode, TCM is contiguous memory block
+ * However, each TCM block still needs to be enabled individually.
+ * So, Enable each TCM block individually, but add their size
+ * to create contiguous memory region.
+ */
+ bank_addr = r5_core->tcm_banks[0]->addr;
+ bank_name = r5_core->tcm_banks[0]->bank_name;
+
+ for (i = 0; i < num_banks; i++) {
+ bank_size += r5_core->tcm_banks[i]->size;
+ pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
+
+ /* Turn on each TCM bank individually */
+ ret = zynqmp_pm_request_node(pm_domain_id,
+ ZYNQMP_PM_CAPABILITY_ACCESS, 0,
+ ZYNQMP_PM_REQUEST_ACK_BLOCKING);
+ if (ret < 0) {
+ dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
+ goto release_tcm_lockstep;
+ }
+ }
+
+ dev_dbg(dev, "TCM add carveout lockstep mode %s addr=0x%llx, size=0x%lx",
+ bank_name, bank_addr, bank_size);
+
+ /* Register TCM address range, TCM map and unmap functions */
+ rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
+ bank_size, bank_addr,
+ tcm_mem_map, tcm_mem_unmap,
+ bank_name);
+ if (!rproc_mem) {
+ ret = -ENOMEM;
+ goto release_tcm_lockstep;
+ }
+
+ /* If registration is success, add carveouts */
+ rproc_add_carveout(rproc, rproc_mem);
+
+ return 0;
+
+release_tcm_lockstep:
+ /* If failed, Turn off all TCM banks turned on before */
+ for (i--; i >= 0; i--) {
+ pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
+ zynqmp_pm_release_node(pm_domain_id);
+ }
+ return ret;
+}
+
+/*
+ * add_tcm_banks()
+ * @rproc: single R5 core's corresponding rproc instance
+ *
+ * allocate and add remoteproc carveouts for TCM memory based on cluster mode
+ *
+ * return 0 on success, otherwise non-zero value on failure
+ */
+static int add_tcm_banks(struct rproc *rproc)
+{
+ struct zynqmp_r5_cluster *cluster;
+ struct zynqmp_r5_core *r5_core;
+ struct device *dev;
+
+ r5_core = (struct zynqmp_r5_core *)rproc->priv;
+ if (!r5_core)
+ return -EINVAL;
+
+ dev = r5_core->dev;
+
+ cluster = dev_get_drvdata(dev->parent);
+ if (!cluster) {
+ dev_err(dev->parent, "Invalid driver data\n");
+ return -EINVAL;
+ }
+
+ /*
+ * In lockstep mode TCM banks are one contiguous memory region of 256Kb
+ * In split mode, each TCM bank is 64Kb and not contiguous.
+ * We add memory carveouts accordingly.
+ */
+ if (cluster->mode == SPLIT_MODE)
+ return add_tcm_carveout_split_mode(rproc);
+ else if (cluster->mode == LOCKSTEP_MODE)
+ return add_tcm_carveout_lockstep_mode(rproc);
+
+ return -EINVAL;
+}
+
+/*
+ * zynqmp_r5_parse_fw()
+ * @rproc: single R5 core's corresponding rproc instance
+ * @fw: ptr to firmware to be loaded onto r5 core
+ *
+ * get resource table if available
+ *
+ * return 0 on success, otherwise non-zero value on failure
+ */
+static int zynqmp_r5_parse_fw(struct rproc *rproc, const struct firmware *fw)
+{
+ int ret;
+
+ ret = rproc_elf_load_rsc_table(rproc, fw);
+ if (ret == -EINVAL) {
+ /*
+ * resource table only required for IPC.
+ * if not present, this is not necessarily an error;
+ * for example, loading r5 hello world application
+ * so simply inform user and keep going.
+ */
+ dev_info(&rproc->dev, "no resource table found.\n");
+ ret = 0;
+ }
+ return ret;
+}
+
+/**
+ * zynqmp_r5_rproc_prepare()
+ * adds carveouts for TCM bank and reserved memory regions
+ *
+ * @rproc: Device node of each rproc
+ *
+ * Return: 0 for success else < 0 error code
+ */
+static int zynqmp_r5_rproc_prepare(struct rproc *rproc)
+{
+ int ret;
+
+ ret = add_tcm_banks(rproc);
+ if (ret) {
+ dev_err(&rproc->dev, "failed to get TCM banks, err %d\n", ret);
+ return ret;
+ }
+
+ ret = add_mem_regions_carveout(rproc);
+ if (ret) {
+ dev_err(&rproc->dev, "failed to get reserve mem regions %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+/**
+ * zynqmp_r5_rproc_unprepare()
+ * Turns off TCM banks using power-domain id
+ *
+ * @rproc: Device node of each rproc
+ *
+ * Return: always 0
+ */
+static int zynqmp_r5_rproc_unprepare(struct rproc *rproc)
+{
+ struct zynqmp_r5_core *r5_core;
+ u32 pm_domain_id;
+ int i;
+
+ r5_core = (struct zynqmp_r5_core *)rproc->priv;
+
+ for (i = 0; i < r5_core->tcm_bank_count; i++) {
+ pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
+ if (zynqmp_pm_release_node(pm_domain_id))
+ dev_warn(r5_core->dev,
+ "can't turn off TCM bank 0x%x", pm_domain_id);
+ }
+
+ return 0;
+}
+
+static const struct rproc_ops zynqmp_r5_rproc_ops = {
+ .prepare = zynqmp_r5_rproc_prepare,
+ .unprepare = zynqmp_r5_rproc_unprepare,
+ .start = zynqmp_r5_rproc_start,
+ .stop = zynqmp_r5_rproc_stop,
+ .load = rproc_elf_load_segments,
+ .parse_fw = zynqmp_r5_parse_fw,
+ .find_loaded_rsc_table = rproc_elf_find_loaded_rsc_table,
+ .sanity_check = rproc_elf_sanity_check,
+ .get_boot_addr = rproc_elf_get_boot_addr,
+};
+
+/**
+ * zynqmp_r5_add_rproc_core()
+ * Allocate and add struct rproc object for each r5f core
+ * This is called for each individual r5f core
+ *
+ * @cdev: Device node of each r5 core
+ *
+ * Return: zynqmp_r5_core object for success else error code pointer
+ */
+static struct zynqmp_r5_core *zynqmp_r5_add_rproc_core(struct device *cdev)
+{
+ struct zynqmp_r5_core *r5_core;
+ struct rproc *r5_rproc;
+ int ret;
+
+ /* Set up DMA mask */
+ ret = dma_set_coherent_mask(cdev, DMA_BIT_MASK(32));
+ if (ret)
+ return ERR_PTR(ret);
+
+ /* Allocate remoteproc instance */
+ r5_rproc = rproc_alloc(cdev, dev_name(cdev),
+ &zynqmp_r5_rproc_ops,
+ NULL, sizeof(struct zynqmp_r5_core));
+ if (!r5_rproc) {
+ dev_err(cdev, "failed to allocate memory for rproc instance\n");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ r5_rproc->auto_boot = false;
+ r5_core = (struct zynqmp_r5_core *)r5_rproc->priv;
+ r5_core->dev = cdev;
+ r5_core->np = dev_of_node(cdev);
+ if (!r5_core->np) {
+ dev_err(cdev, "can't get device node for r5 core\n");
+ ret = -EINVAL;
+ goto free_rproc;
+ }
+
+ /* Add R5 remoteproc core */
+ ret = rproc_add(r5_rproc);
+ if (ret) {
+ dev_err(cdev, "failed to add r5 remoteproc\n");
+ goto free_rproc;
+ }
+
+ r5_core->rproc = r5_rproc;
+ return r5_core;
+
+free_rproc:
+ rproc_free(r5_rproc);
+ return ERR_PTR(ret);
+}
+
+/**
+ * zynqmp_r5_get_tcm_node()
+ * Ideally this function should parse tcm node and store information
+ * in r5_core instance. For now, Hardcoded TCM information is used.
+ * This approach is used as TCM bindings for system-dt is being developed
+ *
+ * @cluster: pointer to zynqmp_r5_cluster type object
+ *
+ * Return: 0 for success and < 0 error code for failure.
+ */
+static int zynqmp_r5_get_tcm_node(struct zynqmp_r5_cluster *cluster)
+{
+ struct device *dev = cluster->dev;
+ struct zynqmp_r5_core *r5_core;
+ int tcm_bank_count, tcm_node;
+ int i, j;
+
+ tcm_bank_count = ARRAY_SIZE(zynqmp_tcm_banks);
+
+ /* count per core tcm banks */
+ tcm_bank_count = tcm_bank_count / cluster->core_count;
+
+ /*
+ * r5 core 0 will use all of TCM banks in lockstep mode.
+ * In split mode, r5 core0 will use 128k and r5 core1 will use another
+ * 128k. Assign TCM banks to each core accordingly
+ */
+ tcm_node = 0;
+ for (i = 0; i < cluster->core_count; i++) {
+ r5_core = cluster->r5_cores[i];
+ r5_core->tcm_banks = devm_kcalloc(dev, tcm_bank_count,
+ sizeof(struct mem_bank_data *),
+ GFP_KERNEL);
+ if (!r5_core->tcm_banks)
+ return -ENOMEM;
+
+ for (j = 0; j < tcm_bank_count; j++) {
+ /*
+ * Use pre-defined TCM reg values.
+ * Eventually this should be replaced by values
+ * parsed from dts.
+ */
+ r5_core->tcm_banks[j] =
+ (struct mem_bank_data *)&zynqmp_tcm_banks[tcm_node];
+ tcm_node++;
+ }
+
+ r5_core->tcm_bank_count = tcm_bank_count;
+ }
+
+ return 0;
+}
+
+/**
+ * zynqmp_r5_get_mem_region_node()
+ * parse memory-region property and get reserved mem regions
+ *
+ * @r5_core: pointer to zynqmp_r5_core type object
+ *
+ * Return: 0 for success and error code for failure.
+ */
+static int zynqmp_r5_get_mem_region_node(struct zynqmp_r5_core *r5_core)
+{
+ struct device_node *np, *rmem_np;
+ struct reserved_mem **rmem;
+ int res_mem_count, i;
+ struct device *dev;
+
+ dev = r5_core->dev;
+ np = r5_core->np;
+
+ res_mem_count = of_property_count_elems_of_size(np, "memory-region",
+ sizeof(phandle));
+ if (res_mem_count <= 0) {
+ dev_warn(dev, "failed to get memory-region property %d\n",
+ res_mem_count);
+ return 0;
+ }
+
+ rmem = devm_kcalloc(dev, res_mem_count,
+ sizeof(struct reserved_mem *), GFP_KERNEL);
+ if (!rmem)
+ return -ENOMEM;
+
+ for (i = 0; i < res_mem_count; i++) {
+ rmem_np = of_parse_phandle(np, "memory-region", i);
+ if (!rmem_np)
+ goto release_rmem;
+
+ rmem[i] = of_reserved_mem_lookup(rmem_np);
+ if (!rmem[i]) {
+ of_node_put(rmem_np);
+ goto release_rmem;
+ }
+
+ of_node_put(rmem_np);
+ }
+
+ r5_core->rmem_count = res_mem_count;
+ r5_core->rmem = rmem;
+ return 0;
+
+release_rmem:
+ return -EINVAL;
+}
+
+/*
+ * zynqmp_r5_core_init()
+ * Create and initialize zynqmp_r5_core type object
+ *
+ * @cluster: pointer to zynqmp_r5_cluster type object
+ * @fw_reg_val: value expected by firmware to configure RPU cluster mode
+ * @tcm_mode: value expected by fw to configure TCM mode (lockstep or split)
+ *
+ * Return: 0 for success and error code for failure.
+ */
+static int zynqmp_r5_core_init(struct zynqmp_r5_cluster *cluster,
+ enum rpu_oper_mode fw_reg_val,
+ enum rpu_tcm_comb tcm_mode)
+{
+ struct device *dev = cluster->dev;
+ struct zynqmp_r5_core *r5_core;
+ int ret, i;
+
+ ret = zynqmp_r5_get_tcm_node(cluster);
+ if (ret < 0) {
+ dev_err(dev, "can't get tcm node, err %d\n", ret);
+ return ret;
+ }
+
+ for (i = 0; i < cluster->core_count; i++) {
+ r5_core = cluster->r5_cores[i];
+
+ ret = zynqmp_r5_get_mem_region_node(r5_core);
+ if (ret)
+ dev_warn(dev, "memory-region prop failed %d\n", ret);
+
+ /* Initialize r5 cores with power-domains parsed from dts */
+ ret = of_property_read_u32_index(r5_core->np, "power-domains",
+ 1, &r5_core->pm_domain_id);
+ if (ret) {
+ dev_err(dev, "failed to get power-domains property\n");
+ return ret;
+ }
+
+ ret = zynqmp_r5_set_mode(r5_core, fw_reg_val, tcm_mode);
+ if (ret) {
+ dev_err(dev, "failed to set r5 cluster mode %d, err %d\n",
+ cluster->mode, ret);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * zynqmp_r5_cluster_init()
+ * Create and initialize zynqmp_r5_cluster type object
+ *
+ * @cluster: pointer to zynqmp_r5_cluster type object
+ *
+ * Return: 0 for success and error code for failure.
+ */
+static int zynqmp_r5_cluster_init(struct zynqmp_r5_cluster *cluster)
+{
+ enum zynqmp_r5_cluster_mode cluster_mode = LOCKSTEP_MODE;
+ struct device *dev = cluster->dev;
+ struct device_node *dev_node = dev_of_node(dev);
+ struct platform_device *child_pdev;
+ struct zynqmp_r5_core **r5_cores;
+ enum rpu_oper_mode fw_reg_val;
+ struct device **child_devs;
+ struct device_node *child;
+ enum rpu_tcm_comb tcm_mode;
+ int core_count, ret, i;
+
+ ret = of_property_read_u32(dev_node, "xlnx,cluster-mode", &cluster_mode);
+
+ /*
+ * on success returns 0, if not defined then returns -EINVAL,
+ * In that case, default is LOCKSTEP mode. Other than that
+ * returns relative error code < 0.
+ */
+ if (ret != -EINVAL && ret != 0) {
+ dev_err(dev, "Invalid xlnx,cluster-mode property\n");
+ return ret;
+ }
+
+ /*
+ * For now driver only supports split mode and lockstep mode.
+ * fail driver probe if either of that is not set in dts.
+ */
+ if (cluster_mode == LOCKSTEP_MODE) {
+ tcm_mode = PM_RPU_TCM_COMB;
+ fw_reg_val = PM_RPU_MODE_LOCKSTEP;
+ } else if (cluster_mode == SPLIT_MODE) {
+ tcm_mode = PM_RPU_TCM_SPLIT;
+ fw_reg_val = PM_RPU_MODE_SPLIT;
+ } else {
+ dev_err(dev, "driver does not support cluster mode %d\n", cluster_mode);
+ return -EINVAL;
+ }
+
+ /*
+ * Number of cores is decided by number of child nodes of
+ * r5f subsystem node in dts. If Split mode is used in dts
+ * 2 child nodes are expected.
+ * In lockstep mode if two child nodes are available,
+ * only use first child node and consider it as core0
+ * and ignore core1 dt node.
+ */
+ core_count = of_get_available_child_count(dev_node);
+ if (core_count == 0) {
+ dev_err(dev, "Invalid number of r5 cores %d", core_count);
+ return -EINVAL;
+ } else if (cluster_mode == SPLIT_MODE && core_count != 2) {
+ dev_err(dev, "Invalid number of r5 cores for split mode\n");
+ return -EINVAL;
+ } else if (cluster_mode == LOCKSTEP_MODE && core_count == 2) {
+ dev_warn(dev, "Only r5 core0 will be used\n");
+ core_count = 1;
+ }
+
+ child_devs = kcalloc(core_count, sizeof(struct device *), GFP_KERNEL);
+ if (!child_devs)
+ return -ENOMEM;
+
+ r5_cores = kcalloc(core_count,
+ sizeof(struct zynqmp_r5_core *), GFP_KERNEL);
+ if (!r5_cores) {
+ kfree(child_devs);
+ return -ENOMEM;
+ }
+
+ i = 0;
+ for_each_available_child_of_node(dev_node, child) {
+ child_pdev = of_find_device_by_node(child);
+ if (!child_pdev) {
+ of_node_put(child);
+ ret = -ENODEV;
+ goto release_r5_cores;
+ }
+
+ child_devs[i] = &child_pdev->dev;
+
+ /* create and add remoteproc instance of type struct rproc */
+ r5_cores[i] = zynqmp_r5_add_rproc_core(&child_pdev->dev);
+ if (IS_ERR(r5_cores[i])) {
+ of_node_put(child);
+ ret = PTR_ERR(r5_cores[i]);
+ r5_cores[i] = NULL;
+ goto release_r5_cores;
+ }
+
+ /*
+ * If two child nodes are available in dts in lockstep mode,
+ * then ignore second child node.
+ */
+ if (cluster_mode == LOCKSTEP_MODE) {
+ of_node_put(child);
+ break;
+ }
+
+ i++;
+ }
+
+ cluster->mode = cluster_mode;
+ cluster->core_count = core_count;
+ cluster->r5_cores = r5_cores;
+
+ ret = zynqmp_r5_core_init(cluster, fw_reg_val, tcm_mode);
+ if (ret < 0) {
+ dev_err(dev, "failed to init r5 core err %d\n", ret);
+ cluster->core_count = 0;
+ cluster->r5_cores = NULL;
+
+ /*
+ * at this point rproc resources for each core are allocated.
+ * adjust index to free resources in reverse order
+ */
+ i = core_count - 1;
+ goto release_r5_cores;
+ }
+
+ kfree(child_devs);
+ return 0;
+
+release_r5_cores:
+ while (i >= 0) {
+ put_device(child_devs[i]);
+ if (r5_cores[i]) {
+ of_reserved_mem_device_release(r5_cores[i]->dev);
+ rproc_del(r5_cores[i]->rproc);
+ rproc_free(r5_cores[i]->rproc);
+ }
+ i--;
+ }
+ kfree(r5_cores);
+ kfree(child_devs);
+ return ret;
+}
+
+static void zynqmp_r5_cluster_exit(void *data)
+{
+ struct platform_device *pdev = (struct platform_device *)data;
+ struct zynqmp_r5_cluster *cluster;
+ struct zynqmp_r5_core *r5_core;
+ int i;
+
+ cluster = (struct zynqmp_r5_cluster *)platform_get_drvdata(pdev);
+ if (!cluster)
+ return;
+
+ for (i = 0; i < cluster->core_count; i++) {
+ r5_core = cluster->r5_cores[i];
+ of_reserved_mem_device_release(r5_core->dev);
+ put_device(r5_core->dev);
+ rproc_del(r5_core->rproc);
+ rproc_free(r5_core->rproc);
+ }
+
+ kfree(cluster->r5_cores);
+ kfree(cluster);
+ platform_set_drvdata(pdev, NULL);
+}
+
+/*
+ * zynqmp_r5_remoteproc_probe()
+ * parse device-tree, initialize hardware and allocate required resources
+ * and remoteproc ops
+ *
+ * @pdev: domain platform device for R5 cluster
+ *
+ * Return: 0 for success and < 0 for failure.
+ */
+static int zynqmp_r5_remoteproc_probe(struct platform_device *pdev)
+{
+ struct zynqmp_r5_cluster *cluster;
+ struct device *dev = &pdev->dev;
+ int ret;
+
+ cluster = kzalloc(sizeof(*cluster), GFP_KERNEL);
+ if (!cluster)
+ return -ENOMEM;
+
+ cluster->dev = dev;
+
+ ret = devm_of_platform_populate(dev);
+ if (ret) {
+ dev_err_probe(dev, ret, "failed to populate platform dev\n");
+ kfree(cluster);
+ return ret;
+ }
+
+ /* wire in so each core can be cleaned up at driver remove */
+ platform_set_drvdata(pdev, cluster);
+
+ ret = zynqmp_r5_cluster_init(cluster);
+ if (ret) {
+ kfree(cluster);
+ platform_set_drvdata(pdev, NULL);
+ dev_err_probe(dev, ret, "Invalid r5f subsystem device tree\n");
+ return ret;
+ }
+
+ ret = devm_add_action_or_reset(dev, zynqmp_r5_cluster_exit, pdev);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+/* Match table for OF platform binding */
+static const struct of_device_id zynqmp_r5_remoteproc_match[] = {
+ { .compatible = "xlnx,zynqmp-r5fss", },
+ { /* end of list */ },
+};
+MODULE_DEVICE_TABLE(of, zynqmp_r5_remoteproc_match);
+
+static struct platform_driver zynqmp_r5_remoteproc_driver = {
+ .probe = zynqmp_r5_remoteproc_probe,
+ .driver = {
+ .name = "zynqmp_r5_remoteproc",
+ .of_match_table = zynqmp_r5_remoteproc_match,
+ },
+};
+module_platform_driver(zynqmp_r5_remoteproc_driver);
+
+MODULE_DESCRIPTION("Xilinx R5F remote processor driver");
+MODULE_AUTHOR("Xilinx Inc.");
+MODULE_LICENSE("GPL");
config RTC_DRV_ISL12026
tristate "Intersil ISL12026"
- depends on OF || COMPILE_TEST
+ depends on OF
help
If you say yes here you get support for the
Intersil ISL12026 RTC chip.
This driver can also be built as a module. If so, the module
will be called rtc-bq32k.
-config RTC_DRV_DM355EVM
- tristate "TI DaVinci DM355 EVM RTC"
- depends on MFD_DM355EVM_MSP
- help
- Supports the RTC firmware in the MSP430 on the DM355 EVM.
-
config RTC_DRV_TWL92330
bool "TI TWL92330/Menelaus"
depends on MENELAUS
This driver can also be built as a module. If so, the module
will be called rtc-asm9260.
-config RTC_DRV_DAVINCI
- tristate "TI DaVinci RTC"
- depends on ARCH_DAVINCI_DM365 || COMPILE_TEST
- help
- If you say yes here you get support for the RTC on the
- DaVinci platforms (DM365).
-
- This driver can also be built as a module. If so, the module
- will be called rtc-davinci.
-
config RTC_DRV_DIGICOLOR
tristate "Conexant Digicolor RTC"
depends on ARCH_DIGICOLOR || COMPILE_TEST
obj-$(CONFIG_RTC_DRV_DA9052) += rtc-da9052.o
obj-$(CONFIG_RTC_DRV_DA9055) += rtc-da9055.o
obj-$(CONFIG_RTC_DRV_DA9063) += rtc-da9063.o
-obj-$(CONFIG_RTC_DRV_DAVINCI) += rtc-davinci.o
obj-$(CONFIG_RTC_DRV_DIGICOLOR) += rtc-digicolor.o
-obj-$(CONFIG_RTC_DRV_DM355EVM) += rtc-dm355evm.o
obj-$(CONFIG_RTC_DRV_DS1216) += rtc-ds1216.o
obj-$(CONFIG_RTC_DRV_DS1286) += rtc-ds1286.o
obj-$(CONFIG_RTC_DRV_DS1302) += rtc-ds1302.o
rtc->id = id;
rtc->dev.parent = dev;
- err = dev_set_name(&rtc->dev, "rtc%d", id);
+ err = devm_add_action_or_reset(dev, devm_rtc_release_device, rtc);
if (err)
return ERR_PTR(err);
- err = devm_add_action_or_reset(dev, devm_rtc_release_device, rtc);
+ err = dev_set_name(&rtc->dev, "rtc%d", id);
if (err)
return ERR_PTR(err);
*
* This could all instead be done in the lower level driver,
* but since more than one lower level RTC implementation needs it,
- * then it's probably best best to do it here instead of there..
+ * then it's probably best to do it here instead of there..
*/
/* Get the "before" timestamp */
#include <linux/bcd.h>
#include <linux/i2c.h>
+#include <linux/kstrtox.h>
#include <linux/module.h>
#include <linux/of_device.h>
#include <linux/rtc.h>
}
#endif
-static int abx80x_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static const struct i2c_device_id abx80x_id[] = {
+ { "abx80x", ABX80X },
+ { "ab0801", AB0801 },
+ { "ab0803", AB0803 },
+ { "ab0804", AB0804 },
+ { "ab0805", AB0805 },
+ { "ab1801", AB1801 },
+ { "ab1803", AB1803 },
+ { "ab1804", AB1804 },
+ { "ab1805", AB1805 },
+ { "rv1805", RV1805 },
+ { }
+};
+MODULE_DEVICE_TABLE(i2c, abx80x_id);
+
+static int abx80x_probe(struct i2c_client *client)
{
struct device_node *np = client->dev.of_node;
struct abx80x_priv *priv;
int i, data, err, trickle_cfg = -EINVAL;
char buf[7];
+ const struct i2c_device_id *id = i2c_match_id(abx80x_id, client);
unsigned int part = id->driver_data;
unsigned int partnumber;
unsigned int majrev, minrev;
return devm_rtc_register_device(priv->rtc);
}
-static const struct i2c_device_id abx80x_id[] = {
- { "abx80x", ABX80X },
- { "ab0801", AB0801 },
- { "ab0803", AB0803 },
- { "ab0804", AB0804 },
- { "ab0805", AB0805 },
- { "ab1801", AB1801 },
- { "ab1803", AB1803 },
- { "ab1804", AB1804 },
- { "ab1805", AB1805 },
- { "rv1805", RV1805 },
- { }
-};
-MODULE_DEVICE_TABLE(i2c, abx80x_id);
-
#ifdef CONFIG_OF
static const struct of_device_id abx80x_of_match[] = {
{
.name = "rtc-abx80x",
.of_match_table = of_match_ptr(abx80x_of_match),
},
- .probe = abx80x_probe,
+ .probe_new = abx80x_probe,
.id_table = abx80x_id,
};
*
* Note that there is still a possibility that the mask is updated
* before interrupts have actually been disabled in hardware. The only
- * way to be certain would be to poll the IMR-register, which is is
+ * way to be certain would be to poll the IMR-register, which is
* the very register we are trying to emulate. The register read back
* is a reasonable heuristic.
*/
#include <linux/i2c.h>
#include <linux/rtc.h>
#include <linux/init.h>
+#include <linux/kstrtox.h>
#include <linux/errno.h>
#include <linux/bcd.h>
return IRQ_NONE;
}
+#ifdef CONFIG_ACPI
+
+#include <linux/acpi.h>
+
+static u32 rtc_handler(void *context)
+{
+ struct device *dev = context;
+ struct cmos_rtc *cmos = dev_get_drvdata(dev);
+ unsigned char rtc_control = 0;
+ unsigned char rtc_intr;
+ unsigned long flags;
+
+
+ /*
+ * Always update rtc irq when ACPI is used as RTC Alarm.
+ * Or else, ACPI SCI is enabled during suspend/resume only,
+ * update rtc irq in that case.
+ */
+ if (cmos_use_acpi_alarm())
+ cmos_interrupt(0, (void *)cmos->rtc);
+ else {
+ /* Fix me: can we use cmos_interrupt() here as well? */
+ spin_lock_irqsave(&rtc_lock, flags);
+ if (cmos_rtc.suspend_ctrl)
+ rtc_control = CMOS_READ(RTC_CONTROL);
+ if (rtc_control & RTC_AIE) {
+ cmos_rtc.suspend_ctrl &= ~RTC_AIE;
+ CMOS_WRITE(rtc_control, RTC_CONTROL);
+ rtc_intr = CMOS_READ(RTC_INTR_FLAGS);
+ rtc_update_irq(cmos->rtc, 1, rtc_intr);
+ }
+ spin_unlock_irqrestore(&rtc_lock, flags);
+ }
+
+ pm_wakeup_hard_event(dev);
+ acpi_clear_event(ACPI_EVENT_RTC);
+ acpi_disable_event(ACPI_EVENT_RTC, 0);
+ return ACPI_INTERRUPT_HANDLED;
+}
+
+static void acpi_rtc_event_setup(struct device *dev)
+{
+ if (acpi_disabled)
+ return;
+
+ acpi_install_fixed_event_handler(ACPI_EVENT_RTC, rtc_handler, dev);
+ /*
+ * After the RTC handler is installed, the Fixed_RTC event should
+ * be disabled. Only when the RTC alarm is set will it be enabled.
+ */
+ acpi_clear_event(ACPI_EVENT_RTC);
+ acpi_disable_event(ACPI_EVENT_RTC, 0);
+}
+
+static void acpi_rtc_event_cleanup(void)
+{
+ if (acpi_disabled)
+ return;
+
+ acpi_remove_fixed_event_handler(ACPI_EVENT_RTC, rtc_handler);
+}
+
+static void rtc_wake_on(struct device *dev)
+{
+ acpi_clear_event(ACPI_EVENT_RTC);
+ acpi_enable_event(ACPI_EVENT_RTC, 0);
+}
+
+static void rtc_wake_off(struct device *dev)
+{
+ acpi_disable_event(ACPI_EVENT_RTC, 0);
+}
+
+#ifdef CONFIG_X86
+/* Enable use_acpi_alarm mode for Intel platforms no earlier than 2015 */
+static void use_acpi_alarm_quirks(void)
+{
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL)
+ return;
+
+ if (!is_hpet_enabled())
+ return;
+
+ if (dmi_get_bios_year() < 2015)
+ return;
+
+ use_acpi_alarm = true;
+}
+#else
+static inline void use_acpi_alarm_quirks(void) { }
+#endif
+
+static void acpi_cmos_wake_setup(struct device *dev)
+{
+ if (acpi_disabled)
+ return;
+
+ use_acpi_alarm_quirks();
+
+ cmos_rtc.wake_on = rtc_wake_on;
+ cmos_rtc.wake_off = rtc_wake_off;
+
+ /* ACPI tables bug workaround. */
+ if (acpi_gbl_FADT.month_alarm && !acpi_gbl_FADT.day_alarm) {
+ dev_dbg(dev, "bogus FADT month_alarm (%d)\n",
+ acpi_gbl_FADT.month_alarm);
+ acpi_gbl_FADT.month_alarm = 0;
+ }
+
+ cmos_rtc.day_alrm = acpi_gbl_FADT.day_alarm;
+ cmos_rtc.mon_alrm = acpi_gbl_FADT.month_alarm;
+ cmos_rtc.century = acpi_gbl_FADT.century;
+
+ if (acpi_gbl_FADT.flags & ACPI_FADT_S4_RTC_WAKE)
+ dev_info(dev, "RTC can wake from S4\n");
+
+ /* RTC always wakes from S1/S2/S3, and often S4/STD */
+ device_init_wakeup(dev, 1);
+}
+
+static void cmos_check_acpi_rtc_status(struct device *dev,
+ unsigned char *rtc_control)
+{
+ struct cmos_rtc *cmos = dev_get_drvdata(dev);
+ acpi_event_status rtc_status;
+ acpi_status status;
+
+ if (acpi_gbl_FADT.flags & ACPI_FADT_FIXED_RTC)
+ return;
+
+ status = acpi_get_event_status(ACPI_EVENT_RTC, &rtc_status);
+ if (ACPI_FAILURE(status)) {
+ dev_err(dev, "Could not get RTC status\n");
+ } else if (rtc_status & ACPI_EVENT_FLAG_SET) {
+ unsigned char mask;
+ *rtc_control &= ~RTC_AIE;
+ CMOS_WRITE(*rtc_control, RTC_CONTROL);
+ mask = CMOS_READ(RTC_INTR_FLAGS);
+ rtc_update_irq(cmos->rtc, 1, mask);
+ }
+}
+
+#else /* !CONFIG_ACPI */
+
+static inline void acpi_rtc_event_setup(struct device *dev)
+{
+}
+
+static inline void acpi_rtc_event_cleanup(void)
+{
+}
+
+static inline void acpi_cmos_wake_setup(struct device *dev)
+{
+}
+
+static inline void cmos_check_acpi_rtc_status(struct device *dev,
+ unsigned char *rtc_control)
+{
+}
+#endif /* CONFIG_ACPI */
+
#ifdef CONFIG_PNP
#define INITSECTION
if (info->address_space)
address_space = info->address_space;
- if (info->rtc_day_alarm && info->rtc_day_alarm < 128)
- cmos_rtc.day_alrm = info->rtc_day_alarm;
- if (info->rtc_mon_alarm && info->rtc_mon_alarm < 128)
- cmos_rtc.mon_alrm = info->rtc_mon_alarm;
- if (info->rtc_century && info->rtc_century < 128)
- cmos_rtc.century = info->rtc_century;
+ cmos_rtc.day_alrm = info->rtc_day_alarm;
+ cmos_rtc.mon_alrm = info->rtc_mon_alarm;
+ cmos_rtc.century = info->rtc_century;
if (info->wake_on && info->wake_off) {
cmos_rtc.wake_on = info->wake_on;
cmos_rtc.wake_off = info->wake_off;
}
+ } else {
+ acpi_cmos_wake_setup(dev);
}
+ if (cmos_rtc.day_alrm >= 128)
+ cmos_rtc.day_alrm = 0;
+
+ if (cmos_rtc.mon_alrm >= 128)
+ cmos_rtc.mon_alrm = 0;
+
+ if (cmos_rtc.century >= 128)
+ cmos_rtc.century = 0;
+
cmos_rtc.dev = dev;
dev_set_drvdata(dev, &cmos_rtc);
nvmem_cfg.size = address_space - NVRAM_OFFSET;
devm_rtc_nvmem_register(cmos_rtc.rtc, &nvmem_cfg);
+ /*
+ * Everything has gone well so far, so by default register a handler for
+ * the ACPI RTC fixed event.
+ */
+ if (!info)
+ acpi_rtc_event_setup(dev);
+
dev_info(dev, "%s%s, %d bytes nvram%s\n",
!is_valid_irq(rtc_irq) ? "no alarms" :
cmos_rtc.mon_alrm ? "alarms up to one year" :
hpet_unregister_irq_handler(cmos_interrupt);
}
+ if (!dev_get_platdata(dev))
+ acpi_rtc_event_cleanup();
+
cmos->rtc = NULL;
ports = cmos->iomem;
}
}
-static void cmos_check_acpi_rtc_status(struct device *dev,
- unsigned char *rtc_control);
-
static int __maybe_unused cmos_resume(struct device *dev)
{
struct cmos_rtc *cmos = dev_get_drvdata(dev);
* predate even PNPBIOS should set up platform_bus devices.
*/
-#ifdef CONFIG_ACPI
-
-#include <linux/acpi.h>
-
-static u32 rtc_handler(void *context)
-{
- struct device *dev = context;
- struct cmos_rtc *cmos = dev_get_drvdata(dev);
- unsigned char rtc_control = 0;
- unsigned char rtc_intr;
- unsigned long flags;
-
-
- /*
- * Always update rtc irq when ACPI is used as RTC Alarm.
- * Or else, ACPI SCI is enabled during suspend/resume only,
- * update rtc irq in that case.
- */
- if (cmos_use_acpi_alarm())
- cmos_interrupt(0, (void *)cmos->rtc);
- else {
- /* Fix me: can we use cmos_interrupt() here as well? */
- spin_lock_irqsave(&rtc_lock, flags);
- if (cmos_rtc.suspend_ctrl)
- rtc_control = CMOS_READ(RTC_CONTROL);
- if (rtc_control & RTC_AIE) {
- cmos_rtc.suspend_ctrl &= ~RTC_AIE;
- CMOS_WRITE(rtc_control, RTC_CONTROL);
- rtc_intr = CMOS_READ(RTC_INTR_FLAGS);
- rtc_update_irq(cmos->rtc, 1, rtc_intr);
- }
- spin_unlock_irqrestore(&rtc_lock, flags);
- }
-
- pm_wakeup_hard_event(dev);
- acpi_clear_event(ACPI_EVENT_RTC);
- acpi_disable_event(ACPI_EVENT_RTC, 0);
- return ACPI_INTERRUPT_HANDLED;
-}
-
-static inline void rtc_wake_setup(struct device *dev)
-{
- if (acpi_disabled)
- return;
-
- acpi_install_fixed_event_handler(ACPI_EVENT_RTC, rtc_handler, dev);
- /*
- * After the RTC handler is installed, the Fixed_RTC event should
- * be disabled. Only when the RTC alarm is set will it be enabled.
- */
- acpi_clear_event(ACPI_EVENT_RTC);
- acpi_disable_event(ACPI_EVENT_RTC, 0);
-}
-
-static void rtc_wake_on(struct device *dev)
-{
- acpi_clear_event(ACPI_EVENT_RTC);
- acpi_enable_event(ACPI_EVENT_RTC, 0);
-}
-
-static void rtc_wake_off(struct device *dev)
-{
- acpi_disable_event(ACPI_EVENT_RTC, 0);
-}
-
-#ifdef CONFIG_X86
-/* Enable use_acpi_alarm mode for Intel platforms no earlier than 2015 */
-static void use_acpi_alarm_quirks(void)
-{
- if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL)
- return;
-
- if (!is_hpet_enabled())
- return;
-
- if (dmi_get_bios_year() < 2015)
- return;
-
- use_acpi_alarm = true;
-}
-#else
-static inline void use_acpi_alarm_quirks(void) { }
-#endif
-
-/* Every ACPI platform has a mc146818 compatible "cmos rtc". Here we find
- * its device node and pass extra config data. This helps its driver use
- * capabilities that the now-obsolete mc146818 didn't have, and informs it
- * that this board's RTC is wakeup-capable (per ACPI spec).
- */
-static struct cmos_rtc_board_info acpi_rtc_info;
-
-static void cmos_wake_setup(struct device *dev)
-{
- if (acpi_disabled)
- return;
-
- use_acpi_alarm_quirks();
-
- acpi_rtc_info.wake_on = rtc_wake_on;
- acpi_rtc_info.wake_off = rtc_wake_off;
-
- /* workaround bug in some ACPI tables */
- if (acpi_gbl_FADT.month_alarm && !acpi_gbl_FADT.day_alarm) {
- dev_dbg(dev, "bogus FADT month_alarm (%d)\n",
- acpi_gbl_FADT.month_alarm);
- acpi_gbl_FADT.month_alarm = 0;
- }
-
- acpi_rtc_info.rtc_day_alarm = acpi_gbl_FADT.day_alarm;
- acpi_rtc_info.rtc_mon_alarm = acpi_gbl_FADT.month_alarm;
- acpi_rtc_info.rtc_century = acpi_gbl_FADT.century;
-
- /* NOTE: S4_RTC_WAKE is NOT currently useful to Linux */
- if (acpi_gbl_FADT.flags & ACPI_FADT_S4_RTC_WAKE)
- dev_info(dev, "RTC can wake from S4\n");
-
- dev->platform_data = &acpi_rtc_info;
-
- /* RTC always wakes from S1/S2/S3, and often S4/STD */
- device_init_wakeup(dev, 1);
-}
-
-static void cmos_check_acpi_rtc_status(struct device *dev,
- unsigned char *rtc_control)
-{
- struct cmos_rtc *cmos = dev_get_drvdata(dev);
- acpi_event_status rtc_status;
- acpi_status status;
-
- if (acpi_gbl_FADT.flags & ACPI_FADT_FIXED_RTC)
- return;
-
- status = acpi_get_event_status(ACPI_EVENT_RTC, &rtc_status);
- if (ACPI_FAILURE(status)) {
- dev_err(dev, "Could not get RTC status\n");
- } else if (rtc_status & ACPI_EVENT_FLAG_SET) {
- unsigned char mask;
- *rtc_control &= ~RTC_AIE;
- CMOS_WRITE(*rtc_control, RTC_CONTROL);
- mask = CMOS_READ(RTC_INTR_FLAGS);
- rtc_update_irq(cmos->rtc, 1, mask);
- }
-}
-
-#else
-
-static void cmos_wake_setup(struct device *dev)
-{
-}
-
-static void cmos_check_acpi_rtc_status(struct device *dev,
- unsigned char *rtc_control)
-{
-}
-
-static void rtc_wake_setup(struct device *dev)
-{
-}
-#endif
-
#ifdef CONFIG_PNP
#include <linux/pnp.h>
static int cmos_pnp_probe(struct pnp_dev *pnp, const struct pnp_device_id *id)
{
- int irq, ret;
-
- cmos_wake_setup(&pnp->dev);
+ int irq;
if (pnp_port_start(pnp, 0) == 0x70 && !pnp_irq_valid(pnp, 0)) {
irq = 0;
irq = pnp_irq(pnp, 0);
}
- ret = cmos_do_probe(&pnp->dev, pnp_get_resource(pnp, IORESOURCE_IO, 0), irq);
- if (ret)
- return ret;
-
- rtc_wake_setup(&pnp->dev);
-
- return 0;
+ return cmos_do_probe(&pnp->dev, pnp_get_resource(pnp, IORESOURCE_IO, 0), irq);
}
static void cmos_pnp_remove(struct pnp_dev *pnp)
static int __init cmos_platform_probe(struct platform_device *pdev)
{
struct resource *resource;
- int irq, ret;
+ int irq;
cmos_of_init(pdev);
- cmos_wake_setup(&pdev->dev);
if (RTC_IOMAPPED)
resource = platform_get_resource(pdev, IORESOURCE_IO, 0);
if (irq < 0)
irq = -1;
- ret = cmos_do_probe(&pdev->dev, resource, irq);
- if (ret)
- return ret;
-
- rtc_wake_setup(&pdev->dev);
-
- return 0;
+ return cmos_do_probe(&pdev->dev, resource, irq);
}
static int cmos_platform_remove(struct platform_device *pdev)
#define DRV_NAME "cros-ec-rtc"
+#define SECS_PER_DAY (24 * 60 * 60)
+
/**
* struct cros_ec_rtc - Driver data for EC RTC
*
msg.msg.insize = sizeof(msg.data);
ret = cros_ec_cmd_xfer_status(cros_ec, &msg.msg);
- if (ret < 0) {
- dev_err(cros_ec->dev,
- "error getting %s from EC: %d\n",
- command == EC_CMD_RTC_GET_VALUE ? "time" : "alarm",
- ret);
+ if (ret < 0)
return ret;
- }
*response = msg.data.time;
static int cros_ec_rtc_set(struct cros_ec_device *cros_ec, u32 command,
u32 param)
{
- int ret = 0;
+ int ret;
struct {
struct cros_ec_command msg;
struct ec_response_rtc data;
msg.data.time = param;
ret = cros_ec_cmd_xfer_status(cros_ec, &msg.msg);
- if (ret < 0) {
- dev_err(cros_ec->dev, "error setting %s on EC: %d\n",
- command == EC_CMD_RTC_SET_VALUE ? "time" : "alarm",
- ret);
+ if (ret < 0)
return ret;
- }
-
return 0;
}
ret = cros_ec_rtc_set(cros_ec, EC_CMD_RTC_SET_ALARM, alarm_offset);
if (ret < 0) {
- dev_err(dev, "error setting alarm: %d\n", ret);
- return ret;
+ if (ret == -EINVAL && alarm_offset >= SECS_PER_DAY) {
+ /*
+ * RTC chips on some older Chromebooks can only handle
+ * alarms up to 24h in the future. Try to set an alarm
+ * below that limit to avoid suspend failures.
+ */
+ ret = cros_ec_rtc_set(cros_ec, EC_CMD_RTC_SET_ALARM,
+ SECS_PER_DAY - 1);
+ }
+
+ if (ret < 0) {
+ dev_err(dev, "error setting alarm in %u seconds: %d\n",
+ alarm_offset, ret);
+ return ret;
+ }
}
return 0;
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * DaVinci Power Management and Real Time Clock Driver for TI platforms
- *
- * Copyright (C) 2009 Texas Instruments, Inc
- *
- * Author: Miguel Aguilar <miguel.aguilar@ridgerun.com>
- */
-#include <linux/kernel.h>
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/ioport.h>
-#include <linux/delay.h>
-#include <linux/spinlock.h>
-#include <linux/rtc.h>
-#include <linux/bcd.h>
-#include <linux/platform_device.h>
-#include <linux/io.h>
-#include <linux/slab.h>
-
-/*
- * The DaVinci RTC is a simple RTC with the following
- * Sec: 0 - 59 : BCD count
- * Min: 0 - 59 : BCD count
- * Hour: 0 - 23 : BCD count
- * Day: 0 - 0x7FFF(32767) : Binary count ( Over 89 years )
- */
-
-/* PRTC interface registers */
-#define DAVINCI_PRTCIF_PID 0x00
-#define PRTCIF_CTLR 0x04
-#define PRTCIF_LDATA 0x08
-#define PRTCIF_UDATA 0x0C
-#define PRTCIF_INTEN 0x10
-#define PRTCIF_INTFLG 0x14
-
-/* PRTCIF_CTLR bit fields */
-#define PRTCIF_CTLR_BUSY BIT(31)
-#define PRTCIF_CTLR_SIZE BIT(25)
-#define PRTCIF_CTLR_DIR BIT(24)
-#define PRTCIF_CTLR_BENU_MSB BIT(23)
-#define PRTCIF_CTLR_BENU_3RD_BYTE BIT(22)
-#define PRTCIF_CTLR_BENU_2ND_BYTE BIT(21)
-#define PRTCIF_CTLR_BENU_LSB BIT(20)
-#define PRTCIF_CTLR_BENU_MASK (0x00F00000)
-#define PRTCIF_CTLR_BENL_MSB BIT(19)
-#define PRTCIF_CTLR_BENL_3RD_BYTE BIT(18)
-#define PRTCIF_CTLR_BENL_2ND_BYTE BIT(17)
-#define PRTCIF_CTLR_BENL_LSB BIT(16)
-#define PRTCIF_CTLR_BENL_MASK (0x000F0000)
-
-/* PRTCIF_INTEN bit fields */
-#define PRTCIF_INTEN_RTCSS BIT(1)
-#define PRTCIF_INTEN_RTCIF BIT(0)
-#define PRTCIF_INTEN_MASK (PRTCIF_INTEN_RTCSS \
- | PRTCIF_INTEN_RTCIF)
-
-/* PRTCIF_INTFLG bit fields */
-#define PRTCIF_INTFLG_RTCSS BIT(1)
-#define PRTCIF_INTFLG_RTCIF BIT(0)
-#define PRTCIF_INTFLG_MASK (PRTCIF_INTFLG_RTCSS \
- | PRTCIF_INTFLG_RTCIF)
-
-/* PRTC subsystem registers */
-#define PRTCSS_RTC_INTC_EXTENA1 (0x0C)
-#define PRTCSS_RTC_CTRL (0x10)
-#define PRTCSS_RTC_WDT (0x11)
-#define PRTCSS_RTC_TMR0 (0x12)
-#define PRTCSS_RTC_TMR1 (0x13)
-#define PRTCSS_RTC_CCTRL (0x14)
-#define PRTCSS_RTC_SEC (0x15)
-#define PRTCSS_RTC_MIN (0x16)
-#define PRTCSS_RTC_HOUR (0x17)
-#define PRTCSS_RTC_DAY0 (0x18)
-#define PRTCSS_RTC_DAY1 (0x19)
-#define PRTCSS_RTC_AMIN (0x1A)
-#define PRTCSS_RTC_AHOUR (0x1B)
-#define PRTCSS_RTC_ADAY0 (0x1C)
-#define PRTCSS_RTC_ADAY1 (0x1D)
-#define PRTCSS_RTC_CLKC_CNT (0x20)
-
-/* PRTCSS_RTC_INTC_EXTENA1 */
-#define PRTCSS_RTC_INTC_EXTENA1_MASK (0x07)
-
-/* PRTCSS_RTC_CTRL bit fields */
-#define PRTCSS_RTC_CTRL_WDTBUS BIT(7)
-#define PRTCSS_RTC_CTRL_WEN BIT(6)
-#define PRTCSS_RTC_CTRL_WDRT BIT(5)
-#define PRTCSS_RTC_CTRL_WDTFLG BIT(4)
-#define PRTCSS_RTC_CTRL_TE BIT(3)
-#define PRTCSS_RTC_CTRL_TIEN BIT(2)
-#define PRTCSS_RTC_CTRL_TMRFLG BIT(1)
-#define PRTCSS_RTC_CTRL_TMMD BIT(0)
-
-/* PRTCSS_RTC_CCTRL bit fields */
-#define PRTCSS_RTC_CCTRL_CALBUSY BIT(7)
-#define PRTCSS_RTC_CCTRL_DAEN BIT(5)
-#define PRTCSS_RTC_CCTRL_HAEN BIT(4)
-#define PRTCSS_RTC_CCTRL_MAEN BIT(3)
-#define PRTCSS_RTC_CCTRL_ALMFLG BIT(2)
-#define PRTCSS_RTC_CCTRL_AIEN BIT(1)
-#define PRTCSS_RTC_CCTRL_CAEN BIT(0)
-
-static DEFINE_SPINLOCK(davinci_rtc_lock);
-
-struct davinci_rtc {
- struct rtc_device *rtc;
- void __iomem *base;
- int irq;
-};
-
-static inline void rtcif_write(struct davinci_rtc *davinci_rtc,
- u32 val, u32 addr)
-{
- writel(val, davinci_rtc->base + addr);
-}
-
-static inline u32 rtcif_read(struct davinci_rtc *davinci_rtc, u32 addr)
-{
- return readl(davinci_rtc->base + addr);
-}
-
-static inline void rtcif_wait(struct davinci_rtc *davinci_rtc)
-{
- while (rtcif_read(davinci_rtc, PRTCIF_CTLR) & PRTCIF_CTLR_BUSY)
- cpu_relax();
-}
-
-static inline void rtcss_write(struct davinci_rtc *davinci_rtc,
- unsigned long val, u8 addr)
-{
- rtcif_wait(davinci_rtc);
-
- rtcif_write(davinci_rtc, PRTCIF_CTLR_BENL_LSB | addr, PRTCIF_CTLR);
- rtcif_write(davinci_rtc, val, PRTCIF_LDATA);
-
- rtcif_wait(davinci_rtc);
-}
-
-static inline u8 rtcss_read(struct davinci_rtc *davinci_rtc, u8 addr)
-{
- rtcif_wait(davinci_rtc);
-
- rtcif_write(davinci_rtc, PRTCIF_CTLR_DIR | PRTCIF_CTLR_BENL_LSB | addr,
- PRTCIF_CTLR);
-
- rtcif_wait(davinci_rtc);
-
- return rtcif_read(davinci_rtc, PRTCIF_LDATA);
-}
-
-static inline void davinci_rtcss_calendar_wait(struct davinci_rtc *davinci_rtc)
-{
- while (rtcss_read(davinci_rtc, PRTCSS_RTC_CCTRL) &
- PRTCSS_RTC_CCTRL_CALBUSY)
- cpu_relax();
-}
-
-static irqreturn_t davinci_rtc_interrupt(int irq, void *class_dev)
-{
- struct davinci_rtc *davinci_rtc = class_dev;
- unsigned long events = 0;
- u32 irq_flg;
- u8 alm_irq, tmr_irq;
- u8 rtc_ctrl, rtc_cctrl;
- int ret = IRQ_NONE;
-
- irq_flg = rtcif_read(davinci_rtc, PRTCIF_INTFLG) &
- PRTCIF_INTFLG_RTCSS;
-
- alm_irq = rtcss_read(davinci_rtc, PRTCSS_RTC_CCTRL) &
- PRTCSS_RTC_CCTRL_ALMFLG;
-
- tmr_irq = rtcss_read(davinci_rtc, PRTCSS_RTC_CTRL) &
- PRTCSS_RTC_CTRL_TMRFLG;
-
- if (irq_flg) {
- if (alm_irq) {
- events |= RTC_IRQF | RTC_AF;
- rtc_cctrl = rtcss_read(davinci_rtc, PRTCSS_RTC_CCTRL);
- rtc_cctrl |= PRTCSS_RTC_CCTRL_ALMFLG;
- rtcss_write(davinci_rtc, rtc_cctrl, PRTCSS_RTC_CCTRL);
- } else if (tmr_irq) {
- events |= RTC_IRQF | RTC_PF;
- rtc_ctrl = rtcss_read(davinci_rtc, PRTCSS_RTC_CTRL);
- rtc_ctrl |= PRTCSS_RTC_CTRL_TMRFLG;
- rtcss_write(davinci_rtc, rtc_ctrl, PRTCSS_RTC_CTRL);
- }
-
- rtcif_write(davinci_rtc, PRTCIF_INTFLG_RTCSS,
- PRTCIF_INTFLG);
- rtc_update_irq(davinci_rtc->rtc, 1, events);
-
- ret = IRQ_HANDLED;
- }
-
- return ret;
-}
-
-static int
-davinci_rtc_ioctl(struct device *dev, unsigned int cmd, unsigned long arg)
-{
- struct davinci_rtc *davinci_rtc = dev_get_drvdata(dev);
- u8 rtc_ctrl;
- unsigned long flags;
- int ret = 0;
-
- spin_lock_irqsave(&davinci_rtc_lock, flags);
-
- rtc_ctrl = rtcss_read(davinci_rtc, PRTCSS_RTC_CTRL);
-
- switch (cmd) {
- case RTC_WIE_ON:
- rtc_ctrl |= PRTCSS_RTC_CTRL_WEN | PRTCSS_RTC_CTRL_WDTFLG;
- break;
- case RTC_WIE_OFF:
- rtc_ctrl &= ~PRTCSS_RTC_CTRL_WEN;
- break;
- default:
- ret = -ENOIOCTLCMD;
- }
-
- rtcss_write(davinci_rtc, rtc_ctrl, PRTCSS_RTC_CTRL);
-
- spin_unlock_irqrestore(&davinci_rtc_lock, flags);
-
- return ret;
-}
-
-static void convertfromdays(u16 days, struct rtc_time *tm)
-{
- int tmp_days, year, mon;
-
- for (year = 2000;; year++) {
- tmp_days = rtc_year_days(1, 12, year);
- if (days >= tmp_days)
- days -= tmp_days;
- else {
- for (mon = 0;; mon++) {
- tmp_days = rtc_month_days(mon, year);
- if (days >= tmp_days) {
- days -= tmp_days;
- } else {
- tm->tm_year = year - 1900;
- tm->tm_mon = mon;
- tm->tm_mday = days + 1;
- break;
- }
- }
- break;
- }
- }
-}
-
-static void convert2days(u16 *days, struct rtc_time *tm)
-{
- int i;
- *days = 0;
-
- for (i = 2000; i < 1900 + tm->tm_year; i++)
- *days += rtc_year_days(1, 12, i);
-
- *days += rtc_year_days(tm->tm_mday, tm->tm_mon, 1900 + tm->tm_year);
-}
-
-static int davinci_rtc_read_time(struct device *dev, struct rtc_time *tm)
-{
- struct davinci_rtc *davinci_rtc = dev_get_drvdata(dev);
- u16 days = 0;
- u8 day0, day1;
- unsigned long flags;
-
- spin_lock_irqsave(&davinci_rtc_lock, flags);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- tm->tm_sec = bcd2bin(rtcss_read(davinci_rtc, PRTCSS_RTC_SEC));
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- tm->tm_min = bcd2bin(rtcss_read(davinci_rtc, PRTCSS_RTC_MIN));
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- tm->tm_hour = bcd2bin(rtcss_read(davinci_rtc, PRTCSS_RTC_HOUR));
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- day0 = rtcss_read(davinci_rtc, PRTCSS_RTC_DAY0);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- day1 = rtcss_read(davinci_rtc, PRTCSS_RTC_DAY1);
-
- spin_unlock_irqrestore(&davinci_rtc_lock, flags);
-
- days |= day1;
- days <<= 8;
- days |= day0;
-
- convertfromdays(days, tm);
-
- return 0;
-}
-
-static int davinci_rtc_set_time(struct device *dev, struct rtc_time *tm)
-{
- struct davinci_rtc *davinci_rtc = dev_get_drvdata(dev);
- u16 days;
- u8 rtc_cctrl;
- unsigned long flags;
-
- convert2days(&days, tm);
-
- spin_lock_irqsave(&davinci_rtc_lock, flags);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- rtcss_write(davinci_rtc, bin2bcd(tm->tm_sec), PRTCSS_RTC_SEC);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- rtcss_write(davinci_rtc, bin2bcd(tm->tm_min), PRTCSS_RTC_MIN);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- rtcss_write(davinci_rtc, bin2bcd(tm->tm_hour), PRTCSS_RTC_HOUR);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- rtcss_write(davinci_rtc, days & 0xFF, PRTCSS_RTC_DAY0);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- rtcss_write(davinci_rtc, (days & 0xFF00) >> 8, PRTCSS_RTC_DAY1);
-
- rtc_cctrl = rtcss_read(davinci_rtc, PRTCSS_RTC_CCTRL);
- rtc_cctrl |= PRTCSS_RTC_CCTRL_CAEN;
- rtcss_write(davinci_rtc, rtc_cctrl, PRTCSS_RTC_CCTRL);
-
- spin_unlock_irqrestore(&davinci_rtc_lock, flags);
-
- return 0;
-}
-
-static int davinci_rtc_alarm_irq_enable(struct device *dev,
- unsigned int enabled)
-{
- struct davinci_rtc *davinci_rtc = dev_get_drvdata(dev);
- unsigned long flags;
- u8 rtc_cctrl = rtcss_read(davinci_rtc, PRTCSS_RTC_CCTRL);
-
- spin_lock_irqsave(&davinci_rtc_lock, flags);
-
- if (enabled)
- rtc_cctrl |= PRTCSS_RTC_CCTRL_DAEN |
- PRTCSS_RTC_CCTRL_HAEN |
- PRTCSS_RTC_CCTRL_MAEN |
- PRTCSS_RTC_CCTRL_ALMFLG |
- PRTCSS_RTC_CCTRL_AIEN;
- else
- rtc_cctrl &= ~PRTCSS_RTC_CCTRL_AIEN;
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- rtcss_write(davinci_rtc, rtc_cctrl, PRTCSS_RTC_CCTRL);
-
- spin_unlock_irqrestore(&davinci_rtc_lock, flags);
-
- return 0;
-}
-
-static int davinci_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *alm)
-{
- struct davinci_rtc *davinci_rtc = dev_get_drvdata(dev);
- u16 days = 0;
- u8 day0, day1;
- unsigned long flags;
-
- alm->time.tm_sec = 0;
-
- spin_lock_irqsave(&davinci_rtc_lock, flags);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- alm->time.tm_min = bcd2bin(rtcss_read(davinci_rtc, PRTCSS_RTC_AMIN));
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- alm->time.tm_hour = bcd2bin(rtcss_read(davinci_rtc, PRTCSS_RTC_AHOUR));
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- day0 = rtcss_read(davinci_rtc, PRTCSS_RTC_ADAY0);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- day1 = rtcss_read(davinci_rtc, PRTCSS_RTC_ADAY1);
-
- spin_unlock_irqrestore(&davinci_rtc_lock, flags);
- days |= day1;
- days <<= 8;
- days |= day0;
-
- convertfromdays(days, &alm->time);
-
- alm->pending = !!(rtcss_read(davinci_rtc,
- PRTCSS_RTC_CCTRL) &
- PRTCSS_RTC_CCTRL_AIEN);
- alm->enabled = alm->pending && device_may_wakeup(dev);
-
- return 0;
-}
-
-static int davinci_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alm)
-{
- struct davinci_rtc *davinci_rtc = dev_get_drvdata(dev);
- unsigned long flags;
- u16 days;
-
- convert2days(&days, &alm->time);
-
- spin_lock_irqsave(&davinci_rtc_lock, flags);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- rtcss_write(davinci_rtc, bin2bcd(alm->time.tm_min), PRTCSS_RTC_AMIN);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- rtcss_write(davinci_rtc, bin2bcd(alm->time.tm_hour), PRTCSS_RTC_AHOUR);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- rtcss_write(davinci_rtc, days & 0xFF, PRTCSS_RTC_ADAY0);
-
- davinci_rtcss_calendar_wait(davinci_rtc);
- rtcss_write(davinci_rtc, (days & 0xFF00) >> 8, PRTCSS_RTC_ADAY1);
-
- spin_unlock_irqrestore(&davinci_rtc_lock, flags);
-
- return 0;
-}
-
-static const struct rtc_class_ops davinci_rtc_ops = {
- .ioctl = davinci_rtc_ioctl,
- .read_time = davinci_rtc_read_time,
- .set_time = davinci_rtc_set_time,
- .alarm_irq_enable = davinci_rtc_alarm_irq_enable,
- .read_alarm = davinci_rtc_read_alarm,
- .set_alarm = davinci_rtc_set_alarm,
-};
-
-static int __init davinci_rtc_probe(struct platform_device *pdev)
-{
- struct device *dev = &pdev->dev;
- struct davinci_rtc *davinci_rtc;
- int ret = 0;
-
- davinci_rtc = devm_kzalloc(&pdev->dev, sizeof(struct davinci_rtc), GFP_KERNEL);
- if (!davinci_rtc)
- return -ENOMEM;
-
- davinci_rtc->irq = platform_get_irq(pdev, 0);
- if (davinci_rtc->irq < 0)
- return davinci_rtc->irq;
-
- davinci_rtc->base = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(davinci_rtc->base))
- return PTR_ERR(davinci_rtc->base);
-
- platform_set_drvdata(pdev, davinci_rtc);
-
- davinci_rtc->rtc = devm_rtc_allocate_device(&pdev->dev);
- if (IS_ERR(davinci_rtc->rtc))
- return PTR_ERR(davinci_rtc->rtc);
-
- davinci_rtc->rtc->ops = &davinci_rtc_ops;
- davinci_rtc->rtc->range_min = RTC_TIMESTAMP_BEGIN_2000;
- davinci_rtc->rtc->range_max = RTC_TIMESTAMP_BEGIN_2000 + (1 << 16) * 86400ULL - 1;
-
- rtcif_write(davinci_rtc, PRTCIF_INTFLG_RTCSS, PRTCIF_INTFLG);
- rtcif_write(davinci_rtc, 0, PRTCIF_INTEN);
- rtcss_write(davinci_rtc, 0, PRTCSS_RTC_INTC_EXTENA1);
-
- rtcss_write(davinci_rtc, 0, PRTCSS_RTC_CTRL);
- rtcss_write(davinci_rtc, 0, PRTCSS_RTC_CCTRL);
-
- ret = devm_request_irq(dev, davinci_rtc->irq, davinci_rtc_interrupt,
- 0, "davinci_rtc", davinci_rtc);
- if (ret < 0) {
- dev_err(dev, "unable to register davinci RTC interrupt\n");
- return ret;
- }
-
- /* Enable interrupts */
- rtcif_write(davinci_rtc, PRTCIF_INTEN_RTCSS, PRTCIF_INTEN);
- rtcss_write(davinci_rtc, PRTCSS_RTC_INTC_EXTENA1_MASK,
- PRTCSS_RTC_INTC_EXTENA1);
-
- rtcss_write(davinci_rtc, PRTCSS_RTC_CCTRL_CAEN, PRTCSS_RTC_CCTRL);
-
- device_init_wakeup(&pdev->dev, 0);
-
- return devm_rtc_register_device(davinci_rtc->rtc);
-}
-
-static int __exit davinci_rtc_remove(struct platform_device *pdev)
-{
- struct davinci_rtc *davinci_rtc = platform_get_drvdata(pdev);
-
- device_init_wakeup(&pdev->dev, 0);
-
- rtcif_write(davinci_rtc, 0, PRTCIF_INTEN);
-
- return 0;
-}
-
-static struct platform_driver davinci_rtc_driver = {
- .remove = __exit_p(davinci_rtc_remove),
- .driver = {
- .name = "rtc_davinci",
- },
-};
-
-module_platform_driver_probe(davinci_rtc_driver, davinci_rtc_probe);
-
-MODULE_AUTHOR("Miguel Aguilar <miguel.aguilar@ridgerun.com>");
-MODULE_DESCRIPTION("Texas Instruments DaVinci PRTC Driver");
-MODULE_LICENSE("GPL");
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0+
-/*
- * rtc-dm355evm.c - access battery-backed counter in MSP430 firmware
- *
- * Copyright (c) 2008 by David Brownell
- */
-#include <linux/kernel.h>
-#include <linux/init.h>
-#include <linux/rtc.h>
-#include <linux/platform_device.h>
-
-#include <linux/mfd/dm355evm_msp.h>
-#include <linux/module.h>
-
-
-/*
- * The MSP430 firmware on the DM355 EVM uses a watch crystal to feed
- * a 1 Hz counter. When a backup battery is supplied, that makes a
- * reasonable RTC for applications where alarms and non-NTP drift
- * compensation aren't important.
- *
- * The only real glitch is the inability to read or write all four
- * counter bytes atomically: the count may increment in the middle
- * of an operation, causing trouble when the LSB rolls over.
- *
- * This driver was tested with firmware revision A4.
- */
-union evm_time {
- u8 bytes[4];
- u32 value;
-};
-
-static int dm355evm_rtc_read_time(struct device *dev, struct rtc_time *tm)
-{
- union evm_time time;
- int status;
- int tries = 0;
-
- do {
- /*
- * Read LSB(0) to MSB(3) bytes. Defend against the counter
- * rolling over by re-reading until the value is stable,
- * and assuming the four reads take at most a few seconds.
- */
- status = dm355evm_msp_read(DM355EVM_MSP_RTC_0);
- if (status < 0)
- return status;
- if (tries && time.bytes[0] == status)
- break;
- time.bytes[0] = status;
-
- status = dm355evm_msp_read(DM355EVM_MSP_RTC_1);
- if (status < 0)
- return status;
- if (tries && time.bytes[1] == status)
- break;
- time.bytes[1] = status;
-
- status = dm355evm_msp_read(DM355EVM_MSP_RTC_2);
- if (status < 0)
- return status;
- if (tries && time.bytes[2] == status)
- break;
- time.bytes[2] = status;
-
- status = dm355evm_msp_read(DM355EVM_MSP_RTC_3);
- if (status < 0)
- return status;
- if (tries && time.bytes[3] == status)
- break;
- time.bytes[3] = status;
-
- } while (++tries < 5);
-
- dev_dbg(dev, "read timestamp %08x\n", time.value);
-
- rtc_time64_to_tm(le32_to_cpu(time.value), tm);
- return 0;
-}
-
-static int dm355evm_rtc_set_time(struct device *dev, struct rtc_time *tm)
-{
- union evm_time time;
- unsigned long value;
- int status;
-
- value = rtc_tm_to_time64(tm);
- time.value = cpu_to_le32(value);
-
- dev_dbg(dev, "write timestamp %08x\n", time.value);
-
- /*
- * REVISIT handle non-atomic writes ... maybe just retry until
- * byte[1] sticks (no rollover)?
- */
- status = dm355evm_msp_write(time.bytes[0], DM355EVM_MSP_RTC_0);
- if (status < 0)
- return status;
-
- status = dm355evm_msp_write(time.bytes[1], DM355EVM_MSP_RTC_1);
- if (status < 0)
- return status;
-
- status = dm355evm_msp_write(time.bytes[2], DM355EVM_MSP_RTC_2);
- if (status < 0)
- return status;
-
- status = dm355evm_msp_write(time.bytes[3], DM355EVM_MSP_RTC_3);
- if (status < 0)
- return status;
-
- return 0;
-}
-
-static const struct rtc_class_ops dm355evm_rtc_ops = {
- .read_time = dm355evm_rtc_read_time,
- .set_time = dm355evm_rtc_set_time,
-};
-
-/*----------------------------------------------------------------------*/
-
-static int dm355evm_rtc_probe(struct platform_device *pdev)
-{
- struct rtc_device *rtc;
-
- rtc = devm_rtc_allocate_device(&pdev->dev);
- if (IS_ERR(rtc))
- return PTR_ERR(rtc);
-
- platform_set_drvdata(pdev, rtc);
-
- rtc->ops = &dm355evm_rtc_ops;
- rtc->range_max = U32_MAX;
-
- return devm_rtc_register_device(rtc);
-}
-
-/*
- * I2C is used to talk to the MSP430, but this platform device is
- * exposed by an MFD driver that manages I2C communications.
- */
-static struct platform_driver rtc_dm355evm_driver = {
- .probe = dm355evm_rtc_probe,
- .driver = {
- .name = "rtc-dm355evm",
- },
-};
-
-module_platform_driver(rtc_dm355evm_driver);
-
-MODULE_LICENSE("GPL");
return 0;
}
-static void ds1302_remove(struct spi_device *spi)
-{
- spi_set_drvdata(spi, NULL);
-}
-
#ifdef CONFIG_OF
static const struct of_device_id ds1302_dt_ids[] = {
{ .compatible = "maxim,ds1302", },
.driver.name = "rtc-ds1302",
.driver.of_match_table = of_match_ptr(ds1302_dt_ids),
.probe = ds1302_probe,
- .remove = ds1302_remove,
.id_table = ds1302_spi_ids,
};
#include <linux/bcd.h>
#include <linux/i2c.h>
#include <linux/init.h>
+#include <linux/kstrtox.h>
#include <linux/mod_devicetable.h>
#include <linux/module.h>
#include <linux/property.h>
regmap_read(ds1307->regmap, M41TXX_REG_CONTROL, &ctrl_reg);
- return scnprintf(buf, PAGE_SIZE, (ctrl_reg & M41TXX_BIT_FT) ? "on\n" :
- "off\n");
+ return sysfs_emit(buf, (ctrl_reg & M41TXX_BIT_FT) ? "on\n" : "off\n");
}
static DEVICE_ATTR_RW(frequency_test);
return err;
century = (dt->tm_year / 100) + 19;
- err = regmap_write(map, DS1347_CENTURY_REG, century);
+ err = regmap_write(map, DS1347_CENTURY_REG, bin2bcd(century));
if (err)
return err;
if (!pdata)
return -ENOMEM;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- ioaddr = devm_ioremap_resource(&pdev->dev, res);
+ ioaddr = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
if (IS_ERR(ioaddr))
return PTR_ERR(ioaddr);
clear_bit(RTC_FEATURE_UPDATE_INTERRUPT, rtc->features);
set_bit(RTC_FEATURE_ALARM_WAKEUP_ONLY, rtc->features);
+ device_init_wakeup(&dev->dev, true);
+
return devm_rtc_register_device(rtc);
}
},
};
-static int __init ftm_alarm_init(void)
-{
- return platform_driver_register(&ftm_rtc_driver);
-}
-
-device_initcall(ftm_alarm_init);
+module_platform_driver(ftm_rtc_driver);
MODULE_DESCRIPTION("NXP/Freescale FlexTimer alarm driver");
MODULE_AUTHOR("Biwen Li <biwen.li@nxp.com>");
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/regmap.h>
+#include <linux/hwmon.h>
/* ISL register offsets */
#define ISL12022_REG_SC 0x00
#define ISL12022_REG_SR 0x07
#define ISL12022_REG_INT 0x08
+#define ISL12022_REG_BETA 0x0d
+#define ISL12022_REG_TEMP_L 0x28
+
/* ISL register bits */
#define ISL12022_HR_MIL (1 << 7) /* military or 24 hour time */
#define ISL12022_INT_WRTC (1 << 6)
+#define ISL12022_BETA_TSE (1 << 7)
static struct i2c_driver isl12022_driver;
struct regmap *regmap;
};
+static umode_t isl12022_hwmon_is_visible(const void *data,
+ enum hwmon_sensor_types type,
+ u32 attr, int channel)
+{
+ if (type == hwmon_temp && attr == hwmon_temp_input)
+ return 0444;
+
+ return 0;
+}
+
+/*
+ * A user-initiated temperature conversion is not started by this function,
+ * so the temperature is updated once every ~60 seconds.
+ */
+static int isl12022_hwmon_read_temp(struct device *dev, long *mC)
+{
+ struct isl12022 *isl12022 = dev_get_drvdata(dev);
+ struct regmap *regmap = isl12022->regmap;
+ u8 temp_buf[2];
+ int temp, ret;
+
+ ret = regmap_bulk_read(regmap, ISL12022_REG_TEMP_L,
+ temp_buf, sizeof(temp_buf));
+ if (ret)
+ return ret;
+ /*
+ * Temperature is represented as a 10-bit number, unit half-Kelvins.
+ */
+ temp = (temp_buf[1] << 8) | temp_buf[0];
+ temp *= 500;
+ temp -= 273000;
+
+ *mC = temp;
+
+ return 0;
+}
+
+static int isl12022_hwmon_read(struct device *dev,
+ enum hwmon_sensor_types type,
+ u32 attr, int channel, long *val)
+{
+ if (type == hwmon_temp && attr == hwmon_temp_input)
+ return isl12022_hwmon_read_temp(dev, val);
+
+ return -EOPNOTSUPP;
+}
+
+static const struct hwmon_channel_info *isl12022_hwmon_info[] = {
+ HWMON_CHANNEL_INFO(temp, HWMON_T_INPUT),
+ NULL
+};
+
+static const struct hwmon_ops isl12022_hwmon_ops = {
+ .is_visible = isl12022_hwmon_is_visible,
+ .read = isl12022_hwmon_read,
+};
+
+static const struct hwmon_chip_info isl12022_hwmon_chip_info = {
+ .ops = &isl12022_hwmon_ops,
+ .info = isl12022_hwmon_info,
+};
+
+static void isl12022_hwmon_register(struct device *dev)
+{
+ struct isl12022 *isl12022;
+ struct device *hwmon;
+ int ret;
+
+ if (!IS_REACHABLE(CONFIG_HWMON))
+ return;
+
+ isl12022 = dev_get_drvdata(dev);
+
+ ret = regmap_update_bits(isl12022->regmap, ISL12022_REG_BETA,
+ ISL12022_BETA_TSE, ISL12022_BETA_TSE);
+ if (ret) {
+ dev_warn(dev, "unable to enable temperature sensor\n");
+ return;
+ }
+
+ hwmon = devm_hwmon_device_register_with_info(dev, "isl12022", isl12022,
+ &isl12022_hwmon_chip_info,
+ NULL);
+ if (IS_ERR(hwmon))
+ dev_warn(dev, "unable to register hwmon device: %pe\n", hwmon);
+}
+
/*
* In the routines that deal directly with the isl12022 hardware, we use
* rtc_time -- month 0-11, hour 0-23, yr = calendar year-epoch.
return PTR_ERR(isl12022->regmap);
}
+ isl12022_hwmon_register(&client->dev);
+
isl12022->rtc = devm_rtc_allocate_device(&client->dev);
if (IS_ERR(isl12022->rtc))
return PTR_ERR(isl12022->rtc);
}
static int
-isl1208_probe(struct i2c_client *client, const struct i2c_device_id *id)
+isl1208_probe(struct i2c_client *client)
{
int rc = 0;
struct isl1208_state *isl1208;
if (!isl1208->config)
return -ENODEV;
} else {
+ const struct i2c_device_id *id = i2c_match_id(isl1208_id, client);
+
if (id->driver_data >= ISL_LAST_ID)
return -ENODEV;
isl1208->config = &isl1208_configs[id->driver_data];
.name = "rtc-isl1208",
.of_match_table = of_match_ptr(isl1208_of_match),
},
- .probe = isl1208_probe,
+ .probe_new = isl1208_probe,
.id_table = isl1208_id,
};
* @ppos: pointer to the position to write. No seeks allowed
*
* A write to a watchdog device is defined as a keepalive signal. Any
- * write of data will do, as we we don't define content meaning.
+ * write of data will do, as we don't define content meaning.
*/
static ssize_t wdt_write(struct file *file, const char __user *buf,
size_t count, loff_t *ppos)
*****************************************************************************
*/
-static int m41t80_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int m41t80_probe(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
int rc = 0;
return -ENOMEM;
m41t80_data->client = client;
- if (client->dev.of_node)
+ if (client->dev.of_node) {
m41t80_data->features = (unsigned long)
of_device_get_match_data(&client->dev);
- else
+ } else {
+ const struct i2c_device_id *id = i2c_match_id(m41t80_id, client);
m41t80_data->features = id->driver_data;
+ }
i2c_set_clientdata(client, m41t80_data);
m41t80_data->rtc = devm_rtc_allocate_device(&client->dev);
.of_match_table = of_match_ptr(m41t80_of_match),
.pm = &m41t80_pm,
},
- .probe = m41t80_probe,
+ .probe_new = m41t80_probe,
.remove = m41t80_remove,
.id_table = m41t80_id,
};
return ret;
}
- clk = devm_clk_get(dev, NULL);
+ clk = devm_clk_get_enabled(dev, NULL);
if (IS_ERR(clk)) {
dev_err(dev, "No input reference clock\n");
return PTR_ERR(clk);
}
- ret = clk_prepare_enable(clk);
- if (ret) {
- dev_err(dev, "Failed to enable the reference clock, %d\n", ret);
- return ret;
- }
-
- ret = devm_add_action_or_reset(dev, (void (*) (void *))clk_disable_unprepare, clk);
- if (ret)
- return ret;
-
rate = clk_get_rate(clk);
writew(rate & 0xFFFF, priv->rtc_base + REG_RTC_FREQ_CW_L);
writew((rate >> 16) & 0xFFFF, priv->rtc_base + REG_RTC_FREQ_CW_H);
}
pdata->rtc = devm_rtc_allocate_device(&pdev->dev);
- if (IS_ERR(pdata->rtc))
+ if (IS_ERR(pdata->rtc)) {
+ clk_disable_unprepare(pdata->clk);
return PTR_ERR(pdata->rtc);
+ }
pdata->rtc->ops = &mxc_rtc_ops;
pdata->rtc->range_max = U32_MAX;
.ioctl = nct3018y_ioctl,
};
-static int nct3018y_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int nct3018y_probe(struct i2c_client *client)
{
struct nct3018y *nct3018y;
int err, flags;
.name = "rtc-nct3018y",
.of_match_table = of_match_ptr(nct3018y_of_match),
},
- .probe = nct3018y_probe,
+ .probe_new = nct3018y_probe,
.id_table = nct3018y_id,
};
static struct i2c_driver pcf2127_i2c_driver;
-static int pcf2127_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static const struct i2c_device_id pcf2127_i2c_id[] = {
+ { "pcf2127", 1 },
+ { "pcf2129", 0 },
+ { "pca2129", 0 },
+ { }
+};
+MODULE_DEVICE_TABLE(i2c, pcf2127_i2c_id);
+
+static int pcf2127_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_match_id(pcf2127_i2c_id, client);
struct regmap *regmap;
static const struct regmap_config config = {
.reg_bits = 8,
pcf2127_i2c_driver.driver.name, id->driver_data);
}
-static const struct i2c_device_id pcf2127_i2c_id[] = {
- { "pcf2127", 1 },
- { "pcf2129", 0 },
- { "pca2129", 0 },
- { }
-};
-MODULE_DEVICE_TABLE(i2c, pcf2127_i2c_id);
-
static struct i2c_driver pcf2127_i2c_driver = {
.driver = {
.name = "rtc-pcf2127-i2c",
.of_match_table = of_match_ptr(pcf2127_of_match),
},
- .probe = pcf2127_i2c_probe,
+ .probe_new = pcf2127_i2c_probe,
.id_table = pcf2127_i2c_id,
};
if (ret)
return ret;
- alrm->time.tm_sec = bcd2bin(buf[0]);
- alrm->time.tm_min = bcd2bin(buf[1]);
- alrm->time.tm_hour = bcd2bin(buf[2]);
- alrm->time.tm_mday = bcd2bin(buf[3]);
+ alrm->time.tm_sec = bcd2bin(buf[0] & 0x7f);
+ alrm->time.tm_min = bcd2bin(buf[1] & 0x7f);
+ alrm->time.tm_hour = bcd2bin(buf[2] & 0x3f);
+ alrm->time.tm_mday = bcd2bin(buf[3] & 0x3f);
ret = regmap_read(pcf85063->regmap, PCF85063_REG_CTRL2, &val);
if (ret)
unsigned int buf;
int ret;
- ret = regmap_read(pcf85063->regmap, PCF85063_REG_OFFSET, &buf);
+ ret = regmap_read(pcf85063->regmap, PCF85063_REG_CTRL2, &buf);
if (ret < 0)
return ret;
buf &= PCF85063_REG_CLKO_F_MASK;
static int pcf8523_rtc_read_time(struct device *dev, struct rtc_time *tm)
{
struct pcf8523 *pcf8523 = dev_get_drvdata(dev);
- u8 regs[7];
+ u8 regs[10];
int err;
- err = regmap_bulk_read(pcf8523->regmap, PCF8523_REG_SECONDS, regs,
+ err = regmap_bulk_read(pcf8523->regmap, PCF8523_REG_CONTROL1, regs,
sizeof(regs));
if (err < 0)
return err;
- if (regs[0] & PCF8523_SECONDS_OS)
+ if ((regs[0] & PCF8523_CONTROL1_STOP) || (regs[3] & PCF8523_SECONDS_OS))
return -EINVAL;
- tm->tm_sec = bcd2bin(regs[0] & 0x7f);
- tm->tm_min = bcd2bin(regs[1] & 0x7f);
- tm->tm_hour = bcd2bin(regs[2] & 0x3f);
- tm->tm_mday = bcd2bin(regs[3] & 0x3f);
- tm->tm_wday = regs[4] & 0x7;
- tm->tm_mon = bcd2bin(regs[5] & 0x1f) - 1;
- tm->tm_year = bcd2bin(regs[6]) + 100;
+ tm->tm_sec = bcd2bin(regs[3] & 0x7f);
+ tm->tm_min = bcd2bin(regs[4] & 0x7f);
+ tm->tm_hour = bcd2bin(regs[5] & 0x3f);
+ tm->tm_mday = bcd2bin(regs[6] & 0x3f);
+ tm->tm_wday = regs[7] & 0x7;
+ tm->tm_mon = bcd2bin(regs[8] & 0x1f) - 1;
+ tm->tm_year = bcd2bin(regs[9]) + 100;
return 0;
}
client->irq);
return err;
}
+ } else {
+ clear_bit(RTC_FEATURE_ALARM, pcf8563->rtc->features);
}
err = devm_rtc_register_device(pcf8563->rtc);
spin_lock_init(&pdata->alarm_lock);
+ pdata->rtc = devm_rtc_allocate_device(&pdev->dev);
+ if (IS_ERR(pdata->rtc))
+ return PTR_ERR(pdata->rtc);
+
clk_prepare_enable(pdata->clk);
pic32_rtc_enable(pdata, 1);
device_init_wakeup(&pdev->dev, 1);
- pdata->rtc = devm_rtc_allocate_device(&pdev->dev);
- if (IS_ERR(pdata->rtc))
- return PTR_ERR(pdata->rtc);
-
pdata->rtc->ops = &pic32_rtcops;
pdata->rtc->range_min = RTC_TIMESTAMP_BEGIN_2000;
pdata->rtc->range_max = RTC_TIMESTAMP_END_2099;
*/
static const struct of_device_id pm8xxx_id_table[] = {
{ .compatible = "qcom,pm8921-rtc", .data = &pm8921_regs },
- { .compatible = "qcom,pm8018-rtc", .data = &pm8921_regs },
{ .compatible = "qcom,pm8058-rtc", .data = &pm8058_regs },
{ .compatible = "qcom,pm8941-rtc", .data = &pm8941_regs },
{ .compatible = "qcom,pmk8350-rtc", .data = &pmk8350_regs },
#include <linux/bcd.h>
#include <linux/mfd/rk808.h>
#include <linux/platform_device.h>
-#include <linux/i2c.h>
/* RTC_CTRL_REG bitfields */
#define BIT_RTC_CTRL_REG_STOP_RTC_M BIT(0)
};
struct rk808_rtc {
- struct rk808 *rk808;
+ struct regmap *regmap;
struct rtc_device *rtc;
struct rk_rtc_compat_reg *creg;
int irq;
static int rk808_rtc_readtime(struct device *dev, struct rtc_time *tm)
{
struct rk808_rtc *rk808_rtc = dev_get_drvdata(dev);
- struct rk808 *rk808 = rk808_rtc->rk808;
u8 rtc_data[NUM_TIME_REGS];
int ret;
/* Force an update of the shadowed registers right now */
- ret = regmap_update_bits(rk808->regmap, rk808_rtc->creg->ctrl_reg,
+ ret = regmap_update_bits(rk808_rtc->regmap, rk808_rtc->creg->ctrl_reg,
BIT_RTC_CTRL_REG_RTC_GET_TIME,
BIT_RTC_CTRL_REG_RTC_GET_TIME);
if (ret) {
* 32khz. If we clear the GET_TIME bit here, the time of i2c transfer
* certainly more than 31.25us: 16 * 2.5us at 400kHz bus frequency.
*/
- ret = regmap_update_bits(rk808->regmap, rk808_rtc->creg->ctrl_reg,
+ ret = regmap_update_bits(rk808_rtc->regmap, rk808_rtc->creg->ctrl_reg,
BIT_RTC_CTRL_REG_RTC_GET_TIME,
0);
if (ret) {
return ret;
}
- ret = regmap_bulk_read(rk808->regmap, rk808_rtc->creg->seconds_reg,
+ ret = regmap_bulk_read(rk808_rtc->regmap, rk808_rtc->creg->seconds_reg,
rtc_data, NUM_TIME_REGS);
if (ret) {
dev_err(dev, "Failed to bulk read rtc_data: %d\n", ret);
static int rk808_rtc_set_time(struct device *dev, struct rtc_time *tm)
{
struct rk808_rtc *rk808_rtc = dev_get_drvdata(dev);
- struct rk808 *rk808 = rk808_rtc->rk808;
u8 rtc_data[NUM_TIME_REGS];
int ret;
rtc_data[6] = bin2bcd(tm->tm_wday);
/* Stop RTC while updating the RTC registers */
- ret = regmap_update_bits(rk808->regmap, rk808_rtc->creg->ctrl_reg,
+ ret = regmap_update_bits(rk808_rtc->regmap, rk808_rtc->creg->ctrl_reg,
BIT_RTC_CTRL_REG_STOP_RTC_M,
BIT_RTC_CTRL_REG_STOP_RTC_M);
if (ret) {
return ret;
}
- ret = regmap_bulk_write(rk808->regmap, rk808_rtc->creg->seconds_reg,
+ ret = regmap_bulk_write(rk808_rtc->regmap, rk808_rtc->creg->seconds_reg,
rtc_data, NUM_TIME_REGS);
if (ret) {
dev_err(dev, "Failed to bull write rtc_data: %d\n", ret);
return ret;
}
/* Start RTC again */
- ret = regmap_update_bits(rk808->regmap, rk808_rtc->creg->ctrl_reg,
+ ret = regmap_update_bits(rk808_rtc->regmap, rk808_rtc->creg->ctrl_reg,
BIT_RTC_CTRL_REG_STOP_RTC_M, 0);
if (ret) {
dev_err(dev, "Failed to update RTC control: %d\n", ret);
static int rk808_rtc_readalarm(struct device *dev, struct rtc_wkalrm *alrm)
{
struct rk808_rtc *rk808_rtc = dev_get_drvdata(dev);
- struct rk808 *rk808 = rk808_rtc->rk808;
u8 alrm_data[NUM_ALARM_REGS];
uint32_t int_reg;
int ret;
- ret = regmap_bulk_read(rk808->regmap,
+ ret = regmap_bulk_read(rk808_rtc->regmap,
rk808_rtc->creg->alarm_seconds_reg,
alrm_data, NUM_ALARM_REGS);
if (ret) {
alrm->time.tm_year = (bcd2bin(alrm_data[5] & YEARS_REG_MSK)) + 100;
rockchip_to_gregorian(&alrm->time);
- ret = regmap_read(rk808->regmap, rk808_rtc->creg->int_reg, &int_reg);
+ ret = regmap_read(rk808_rtc->regmap, rk808_rtc->creg->int_reg, &int_reg);
if (ret) {
dev_err(dev, "Failed to read RTC INT REG: %d\n", ret);
return ret;
static int rk808_rtc_stop_alarm(struct rk808_rtc *rk808_rtc)
{
- struct rk808 *rk808 = rk808_rtc->rk808;
int ret;
- ret = regmap_update_bits(rk808->regmap, rk808_rtc->creg->int_reg,
+ ret = regmap_update_bits(rk808_rtc->regmap, rk808_rtc->creg->int_reg,
BIT_RTC_INTERRUPTS_REG_IT_ALARM_M, 0);
return ret;
static int rk808_rtc_start_alarm(struct rk808_rtc *rk808_rtc)
{
- struct rk808 *rk808 = rk808_rtc->rk808;
int ret;
- ret = regmap_update_bits(rk808->regmap, rk808_rtc->creg->int_reg,
+ ret = regmap_update_bits(rk808_rtc->regmap, rk808_rtc->creg->int_reg,
BIT_RTC_INTERRUPTS_REG_IT_ALARM_M,
BIT_RTC_INTERRUPTS_REG_IT_ALARM_M);
static int rk808_rtc_setalarm(struct device *dev, struct rtc_wkalrm *alrm)
{
struct rk808_rtc *rk808_rtc = dev_get_drvdata(dev);
- struct rk808 *rk808 = rk808_rtc->rk808;
u8 alrm_data[NUM_ALARM_REGS];
int ret;
alrm_data[4] = bin2bcd(alrm->time.tm_mon + 1);
alrm_data[5] = bin2bcd(alrm->time.tm_year - 100);
- ret = regmap_bulk_write(rk808->regmap,
+ ret = regmap_bulk_write(rk808_rtc->regmap,
rk808_rtc->creg->alarm_seconds_reg,
alrm_data, NUM_ALARM_REGS);
if (ret) {
static irqreturn_t rk808_alarm_irq(int irq, void *data)
{
struct rk808_rtc *rk808_rtc = data;
- struct rk808 *rk808 = rk808_rtc->rk808;
- struct i2c_client *client = rk808->i2c;
int ret;
- ret = regmap_write(rk808->regmap, rk808_rtc->creg->status_reg,
+ ret = regmap_write(rk808_rtc->regmap, rk808_rtc->creg->status_reg,
RTC_STATUS_MASK);
if (ret) {
- dev_err(&client->dev,
+ dev_err(&rk808_rtc->rtc->dev,
"%s:Failed to update RTC status: %d\n", __func__, ret);
return ret;
}
rtc_update_irq(rk808_rtc->rtc, 1, RTC_IRQF | RTC_AF);
- dev_dbg(&client->dev,
+ dev_dbg(&rk808_rtc->rtc->dev,
"%s:irq=%d\n", __func__, irq);
return IRQ_HANDLED;
}
break;
}
platform_set_drvdata(pdev, rk808_rtc);
- rk808_rtc->rk808 = rk808;
+ rk808_rtc->regmap = dev_get_regmap(pdev->dev.parent, NULL);
+ if (!rk808_rtc->regmap)
+ return -ENODEV;
/* start rtc running by default, and use shadowed timer. */
- ret = regmap_update_bits(rk808->regmap, rk808_rtc->creg->ctrl_reg,
+ ret = regmap_update_bits(rk808_rtc->regmap, rk808_rtc->creg->ctrl_reg,
BIT_RTC_CTRL_REG_STOP_RTC_M |
BIT_RTC_CTRL_REG_RTC_READSEL_M,
BIT_RTC_CTRL_REG_RTC_READSEL_M);
return ret;
}
- ret = regmap_write(rk808->regmap, rk808_rtc->creg->status_reg,
+ ret = regmap_write(rk808_rtc->regmap, rk808_rtc->creg->status_reg,
RTC_STATUS_MASK);
if (ret) {
dev_err(&pdev->dev,
* Ricoh RS5C313 RTC device/driver
* Copyright (C) 2007 Nobuhiro Iwamatsu
*
- * 2005-09-19 modifed by kogiidena
+ * 2005-09-19 modified by kogiidena
*
* Based on the old drivers/char/rs5c313_rtc.c by:
* Copyright (C) 2000 Philipp Rumpf <prumpf@tux.org>
* 1.11a Daniele Bellucci: Audit create_proc_read_entry in rtc_init
* 1.12 Venkatesh Pallipadi: Hooks for emulating rtc on HPET base-timer
* CONFIG_HPET_EMULATE_RTC
- * 1.13 Nobuhiro Iwamatsu: Updata driver.
+ * 1.13 Nobuhiro Iwamatsu: Update driver.
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
while (1) {
RS5C313_CEENABLE; /* CE:H */
- /* Initiatlize control reg. 24 hour */
+ /* Initialize control reg. 24 hour */
rs5c313_write_cntreg(0x04);
if (!(rs5c313_read_cntreg() & RS5C313_CNTREG_ADJ_BSY))
* least 80219 chips; this works around that bug.
*
* The third method on the other hand doesn't work for the SMBus-only
- * configurations, so we use the the first method there, stripping off
+ * configurations, so we use the first method there, stripping off
* the extra register in the process.
*/
if (rs5c->smbus) {
return 0;
}
-static int rs5c372_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int rs5c372_probe(struct i2c_client *client)
{
int err = 0;
int smbus_mode = 0;
rs5c372->client = client;
i2c_set_clientdata(client, rs5c372);
- if (client->dev.of_node)
+ if (client->dev.of_node) {
rs5c372->type = (enum rtc_type)
of_device_get_match_data(&client->dev);
- else
+ } else {
+ const struct i2c_device_id *id = i2c_match_id(rs5c372_id, client);
rs5c372->type = id->driver_data;
+ }
/* we read registers 0x0f then 0x00-0x0f; skip the first one */
rs5c372->regs = &rs5c372->buf[1];
.name = "rtc-rs5c372",
.of_match_table = of_match_ptr(rs5c372_of_match),
},
- .probe = rs5c372_probe,
+ .probe_new = rs5c372_probe,
.remove = rs5c372_remove,
.id_table = rs5c372_id,
};
return PTR_ERR(rv3028->rtc);
if (client->irq > 0) {
+ unsigned long flags;
+
+ /*
+ * If flags = 0, devm_request_threaded_irq() will use IRQ flags
+ * obtained from device tree.
+ */
+ if (dev_fwnode(&client->dev))
+ flags = 0;
+ else
+ flags = IRQF_TRIGGER_LOW;
+
ret = devm_request_threaded_irq(&client->dev, client->irq,
NULL, rv3028_handle_irq,
- IRQF_TRIGGER_LOW | IRQF_ONESHOT,
+ flags | IRQF_ONESHOT,
"rv3028", rv3028);
if (ret) {
dev_warn(&client->dev, "unable to request IRQ, alarms disabled\n");
#include <linux/of.h>
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
+#include <linux/kstrtox.h>
#include <linux/regmap.h>
/* Register map */
return 0;
}
-static int rv8803_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static const struct i2c_device_id rv8803_id[] = {
+ { "rv8803", rv_8803 },
+ { "rv8804", rx_8804 },
+ { "rx8803", rx_8803 },
+ { "rx8900", rx_8900 },
+ { }
+};
+MODULE_DEVICE_TABLE(i2c, rv8803_id);
+
+static int rv8803_probe(struct i2c_client *client)
{
struct i2c_adapter *adapter = client->adapter;
struct rv8803_data *rv8803;
mutex_init(&rv8803->flags_lock);
rv8803->client = client;
- if (client->dev.of_node)
+ if (client->dev.of_node) {
rv8803->type = (enum rv8803_type)
of_device_get_match_data(&client->dev);
- else
+ } else {
+ const struct i2c_device_id *id = i2c_match_id(rv8803_id, client);
+
rv8803->type = id->driver_data;
+ }
i2c_set_clientdata(client, rv8803);
flags = rv8803_read_reg(client, RV8803_FLAG);
return 0;
}
-static const struct i2c_device_id rv8803_id[] = {
- { "rv8803", rv_8803 },
- { "rv8804", rx_8804 },
- { "rx8803", rx_8803 },
- { "rx8900", rx_8900 },
- { }
-};
-MODULE_DEVICE_TABLE(i2c, rv8803_id);
-
static const __maybe_unused struct of_device_id rv8803_of_match[] = {
{
.compatible = "microcrystal,rv8803",
.name = "rtc-rv8803",
.of_match_table = of_match_ptr(rv8803_of_match),
},
- .probe = rv8803_probe,
+ .probe_new = rv8803_probe,
.id_table = rv8803_id,
};
module_i2c_driver(rv8803_driver);
};
MODULE_DEVICE_TABLE(spi, rx6110_spi_id);
-static const struct of_device_id rx6110_spi_of_match[] = {
+static const __maybe_unused struct of_device_id rx6110_spi_of_match[] = {
{ .compatible = "epson,rx6110" },
{ },
};
#include <linux/bitops.h>
#include <linux/i2c.h>
#include <linux/kernel.h>
+#include <linux/kstrtox.h>
#include <linux/module.h>
#include <linux/rtc.h>
.attrs = rx8025_attrs,
};
-static int rx8025_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int rx8025_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_match_id(rx8025_id, client);
struct i2c_adapter *adapter = client->adapter;
struct rx8025_data *rx8025;
int err = 0;
.driver = {
.name = "rtc-rx8025",
},
- .probe = rx8025_probe,
+ .probe_new = rx8025_probe,
.id_table = rx8025_id,
};
set_bit(RTC_FEATURE_ALARM_RES_MINUTE, rtc->rtcdev->features);
clear_bit(RTC_FEATURE_UPDATE_INTERRUPT, rtc->rtcdev->features);
- devm_pm_runtime_enable(&pdev->dev);
+ ret = devm_pm_runtime_enable(&pdev->dev);
+ if (ret < 0)
+ return ret;
ret = pm_runtime_resume_and_get(&pdev->dev);
if (ret < 0)
return ret;
{
struct i2c_client *client = to_i2c_client(dev);
struct s35390a *s35390a = i2c_get_clientdata(client);
- int i, err;
+ int i;
char buf[7], status;
dev_dbg(&client->dev, "%s: tm is secs=%d, mins=%d, hours=%d mday=%d, "
for (i = 0; i < 7; ++i)
buf[i] = bitrev8(buf[i]);
- err = s35390a_set_reg(s35390a, S35390A_CMD_TIME1, buf, sizeof(buf));
-
- return err;
+ return s35390a_set_reg(s35390a, S35390A_CMD_TIME1, buf, sizeof(buf));
}
static int s35390a_rtc_read_time(struct device *dev, struct rtc_time *tm)
return PTR_ERR(info->base);
info->rtc_clk = devm_clk_get(&pdev->dev, "rtc");
- if (IS_ERR(info->rtc_clk)) {
- ret = PTR_ERR(info->rtc_clk);
- if (ret != -EPROBE_DEFER)
- dev_err(&pdev->dev, "failed to find rtc clock\n");
- else
- dev_dbg(&pdev->dev, "probe deferred due to missing rtc clk\n");
- return ret;
- }
+ if (IS_ERR(info->rtc_clk))
+ return dev_err_probe(&pdev->dev, PTR_ERR(info->rtc_clk),
+ "failed to find rtc clock\n");
ret = clk_prepare_enable(info->rtc_clk);
if (ret)
return ret;
#define SNVS_LPPGDR_INIT 0x41736166
#define CNTR_TO_SECS_SH 15
+/* The maximum RTC clock cycles that are allowed to pass between two
+ * consecutive clock counter register reads. If the values are corrupted a
+ * bigger difference is expected. The RTC frequency is 32kHz. With 320 cycles
+ * we end at 10ms which should be enough for most cases. If it once takes
+ * longer than expected we do a retry.
+ */
+#define MAX_RTC_READ_DIFF_CYCLES 320
+
struct snvs_rtc_data {
struct rtc_device *rtc;
struct regmap *regmap;
static u32 rtc_read_lp_counter(struct snvs_rtc_data *data)
{
u64 read1, read2;
+ s64 diff;
unsigned int timeout = 100;
/* As expected, the registers might update between the read of the LSB
do {
read2 = read1;
read1 = rtc_read_lpsrt(data);
- } while (read1 != read2 && --timeout);
+ diff = read1 - read2;
+ } while (((diff < 0) || (diff > MAX_RTC_READ_DIFF_CYCLES)) && --timeout);
if (!timeout)
dev_err(&data->rtc->dev, "Timeout trying to get valid LPSRT Counter read\n");
static int rtc_read_lp_counter_lsb(struct snvs_rtc_data *data, u32 *lsb)
{
u32 count1, count2;
+ s32 diff;
unsigned int timeout = 100;
regmap_read(data->regmap, data->offset + SNVS_LPSRTCLR, &count1);
do {
count2 = count1;
regmap_read(data->regmap, data->offset + SNVS_LPSRTCLR, &count1);
- } while (count1 != count2 && --timeout);
+ diff = count1 - count2;
+ } while (((diff < 0) || (diff > MAX_RTC_READ_DIFF_CYCLES)) && --timeout);
if (!timeout) {
dev_err(&data->rtc->dev, "Timeout trying to get valid LPSRT Counter read\n");
return -ETIMEDOUT;
rtc->clkrate = clk_get_rate(rtc->clk);
if (!rtc->clkrate) {
+ clk_disable_unprepare(rtc->clk);
dev_err(&pdev->dev, "Unable to fetch clock rate\n");
return -EINVAL;
}
* Author: Alessandro Zummo <a.zummo@towertech.it>
*/
+#include <linux/kstrtox.h>
#include <linux/module.h>
#include <linux/rtc.h>
*
* Return: recommended device file name in /dev
*/
-static char *hmcdrv_dev_name(struct device *dev, umode_t *mode)
+static char *hmcdrv_dev_name(const struct device *dev, umode_t *mode)
{
char *nodename = NULL;
const char *devname = dev_name(dev); /* kernel device name */
*
* Return: Allocated string describing the devtmpfs structure.
*/
-static char *cxlflash_devnode(struct device *dev, umode_t *mode)
+static char *cxlflash_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "cxlflash/%s", dev_name(dev));
}
sk->sk_reuse = SK_CAN_REUSE;
sk->sk_sndtimeo = 15 * HZ; /* FIXME: make it configurable */
sk->sk_allocation = GFP_ATOMIC;
+ sk->sk_use_task_frag = false;
sk_set_memalloc(sk);
sock_no_linger(sk);
{
struct qcom_slim_ctrl *ctrl;
struct slim_controller *sctrl;
- struct resource *slim_mem;
int ret, ver;
ctrl = devm_kzalloc(&pdev->dev, sizeof(*ctrl), GFP_KERNEL);
platform_set_drvdata(pdev, ctrl);
dev_set_drvdata(ctrl->dev, ctrl);
- slim_mem = platform_get_resource_byname(pdev, IORESOURCE_MEM, "ctrl");
- ctrl->base = devm_ioremap_resource(ctrl->dev, slim_mem);
+ ctrl->base = devm_platform_ioremap_resource_byname(pdev, "ctrl");
if (IS_ERR(ctrl->base))
return PTR_ERR(ctrl->base);
static const struct of_device_id qcom_slim_dt_match[] = {
{ .compatible = "qcom,slim", },
- { .compatible = "qcom,apq8064-slim", },
{}
};
{
struct qcom_slim_ngd_ctrl *ctrl = d;
void __iomem *base = ctrl->ngd->base;
- u32 stat = readl(base + NGD_INT_STAT);
+ u32 stat;
+
+ if (pm_runtime_suspended(ctrl->ctrl.dev)) {
+ dev_warn_once(ctrl->dev, "Interrupt received while suspended\n");
+ return IRQ_NONE;
+ }
+
+ stat = readl(base + NGD_INT_STAT);
if ((stat & NGD_INT_MSG_BUF_CONTE) ||
(stat & NGD_INT_MSG_TX_INVAL) || (stat & NGD_INT_DEV_ERR) ||
DECLARE_COMPLETION_ONSTACK(done);
int ret, timeout;
- pm_runtime_get_sync(ctrl->dev);
+ ret = pm_runtime_get_sync(ctrl->dev);
+ if (ret < 0)
+ goto pm_put;
txn->comp = &done;
ret = qcom_slim_ngd_xfer_msg(ctrl, txn);
if (ret)
- return ret;
+ goto pm_put;
timeout = wait_for_completion_timeout(&done, HZ);
if (!timeout) {
dev_err(ctrl->dev, "TX timed out:MC:0x%x,mt:0x%x", txn->mc,
txn->mt);
- return -ETIMEDOUT;
+ ret = -ETIMEDOUT;
+ goto pm_put;
}
return 0;
+
+pm_put:
+ pm_runtime_put(ctrl->dev);
+
+ return ret;
+}
+
+static int qcom_slim_calc_coef(struct slim_stream_runtime *rt, int *exp)
+{
+ struct slim_controller *ctrl = rt->dev->ctrl;
+ int coef;
+
+ if (rt->ratem * ctrl->a_framer->superfreq < rt->rate)
+ rt->ratem++;
+
+ coef = rt->ratem;
+ *exp = 0;
+
+ /*
+ * CRM = Cx(2^E) is the formula we are using.
+ * Here C is the coffecient and E is the exponent.
+ * CRM is the Channel Rate Multiplier.
+ * Coefficeint should be either 1 or 3 and exponenet
+ * should be an integer between 0 to 9, inclusive.
+ */
+ while (1) {
+ while ((coef & 0x1) != 0x1) {
+ coef >>= 1;
+ *exp = *exp + 1;
+ }
+
+ if (coef <= 3)
+ break;
+
+ coef++;
+ }
+
+ /*
+ * we rely on the coef value (1 or 3) to set a bit
+ * in the slimbus message packet. This bit is
+ * BIT(5) which is the segment rate coefficient.
+ */
+ if (coef == 1) {
+ if (*exp > 9)
+ return -EIO;
+ coef = 0;
+ } else {
+ if (*exp > 8)
+ return -EIO;
+ coef = 1;
+ }
+
+ return coef;
}
static int qcom_slim_ngd_enable_stream(struct slim_stream_runtime *rt)
struct slim_port *port = &rt->ports[i];
if (txn.msg->num_bytes == 0) {
- int seg_interval = SLIM_SLOTS_PER_SUPERFRAME/rt->ratem;
- int exp;
+ int exp = 0, coef = 0;
wbuf[txn.msg->num_bytes++] = sdev->laddr;
wbuf[txn.msg->num_bytes] = rt->bps >> 2 |
(port->ch.aux_fmt << 6);
- /* Data channel segment interval not multiple of 3 */
- exp = seg_interval % 3;
- if (exp)
+ /* calculate coef dynamically */
+ coef = qcom_slim_calc_coef(rt, &exp);
+ if (coef < 0) {
+ dev_err(&sdev->dev,
+ "%s: error calculating coef %d\n", __func__,
+ coef);
+ return -EIO;
+ }
+
+ if (coef)
wbuf[txn.msg->num_bytes] |= BIT(5);
txn.msg->num_bytes++;
return 0;
}
+ /*
+ * Reinitialize only when registers are not retained or when enumeration
+ * is lost for ngd.
+ */
+ reinit_completion(&ctrl->reconf);
+
writel_relaxed(DEF_NGD_INT_MASK, ngd->base + NGD_INT_EN);
rx_msgq = readl_relaxed(ngd->base + NGD_RX_MSGQ_CFG);
{
struct device *dev = &pdev->dev;
struct qcom_slim_ngd_ctrl *ctrl;
- struct resource *res;
int ret;
struct pdr_service *pds;
dev_set_drvdata(dev, ctrl);
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- ctrl->base = devm_ioremap_resource(dev, res);
+ ctrl->base = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
if (IS_ERR(ctrl->base))
return PTR_ERR(ctrl->base);
{
struct slim_controller *ctrl = rt->dev->ctrl;
struct slim_port *port;
- int num_ports, i, port_id;
+ int num_ports, i, port_id, prrate;
if (rt->ports) {
dev_err(&rt->dev->dev, "Stream already Prepared\n");
rt->bps = cfg->bps;
rt->direction = cfg->direction;
+ prrate = slim_get_prate_code(cfg->rate);
+ if (prrate < 0) {
+ dev_err(&rt->dev->dev, "Cannot get presence rate for rate %d Hz\n",
+ cfg->rate);
+ return prrate;
+ }
+
if (cfg->rate % ctrl->a_framer->superfreq) {
/*
* data rate not exactly multiple of super frame,
port = &rt->ports[i];
port->state = SLIM_PORT_DISCONNECTED;
port->id = port_id;
- port->ch.prrate = slim_get_prate_code(cfg->rate);
+ port->ch.prrate = prrate;
port->ch.id = cfg->chs[i];
port->ch.data_fmt = SLIM_CH_DATA_FMT_NOT_DEFINED;
port->ch.aux_fmt = SLIM_CH_AUX_FMT_NOT_APPLICABLE;
struct slim_controller *ctrl = stream->dev->ctrl;
int ret, i;
+ if (!stream->ports || !stream->num_ports)
+ return -EINVAL;
+
if (ctrl->disable_stream)
ctrl->disable_stream(stream);
{
int i;
+ if (!stream->ports || !stream->num_ports)
+ return -EINVAL;
+
for (i = 0; i < stream->num_ports; i++)
slim_disconnect_port(stream, &stream->ports[i]);
clk_bulk_disable_unprepare(pd->num_subsys_clks, pd->subsys_clks);
/* subsys power off */
- regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_RST_B_BIT);
regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_ISO_BIT);
regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_CLK_DIS_BIT);
+ regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_RST_B_BIT);
regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_ON_2ND_BIT);
regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_ON_BIT);
config ARCH_TEGRA_186_SOC
bool "NVIDIA Tegra186 SoC"
+ depends on !CPU_BIG_ENDIAN
select MAILBOX
select TEGRA_BPMP
select TEGRA_HSP_MBOX
config ARCH_TEGRA_194_SOC
bool "NVIDIA Tegra194 SoC"
+ depends on !CPU_BIG_ENDIAN
select MAILBOX
select PINCTRL_TEGRA194
select TEGRA_BPMP
config ARCH_TEGRA_234_SOC
bool "NVIDIA Tegra234 SoC"
+ depends on !CPU_BIG_ENDIAN
select MAILBOX
select TEGRA_BPMP
select TEGRA_HSP_MBOX
obj-$(CONFIG_SOUNDWIRE_CADENCE) += soundwire-cadence.o
#Intel driver
-soundwire-intel-y := intel.o intel_init.o dmi-quirks.o
+soundwire-intel-y := intel.o intel_auxdevice.o intel_init.o dmi-quirks.o
obj-$(CONFIG_SOUNDWIRE_INTEL) += soundwire-intel.o
#Qualcomm driver
void *stream, int direction)
{
struct sdw_cdns *cdns = snd_soc_dai_get_drvdata(dai);
- struct sdw_cdns_dma_data *dma;
+ struct sdw_cdns_dai_runtime *dai_runtime;
+
+ dai_runtime = cdns->dai_runtime_array[dai->id];
if (stream) {
/* first paranoia check */
- if (direction == SNDRV_PCM_STREAM_PLAYBACK)
- dma = dai->playback_dma_data;
- else
- dma = dai->capture_dma_data;
-
- if (dma) {
+ if (dai_runtime) {
dev_err(dai->dev,
- "dma_data already allocated for dai %s\n",
+ "dai_runtime already allocated for dai %s\n",
dai->name);
return -EINVAL;
}
- /* allocate and set dma info */
- dma = kzalloc(sizeof(*dma), GFP_KERNEL);
- if (!dma)
+ /* allocate and set dai_runtime info */
+ dai_runtime = kzalloc(sizeof(*dai_runtime), GFP_KERNEL);
+ if (!dai_runtime)
return -ENOMEM;
- dma->stream_type = SDW_STREAM_PCM;
+ dai_runtime->stream_type = SDW_STREAM_PCM;
- dma->bus = &cdns->bus;
- dma->link_id = cdns->instance;
+ dai_runtime->bus = &cdns->bus;
+ dai_runtime->link_id = cdns->instance;
- dma->stream = stream;
+ dai_runtime->stream = stream;
+ dai_runtime->direction = direction;
- if (direction == SNDRV_PCM_STREAM_PLAYBACK)
- dai->playback_dma_data = dma;
- else
- dai->capture_dma_data = dma;
+ cdns->dai_runtime_array[dai->id] = dai_runtime;
} else {
- /* for NULL stream we release allocated dma_data */
- if (direction == SNDRV_PCM_STREAM_PLAYBACK) {
- kfree(dai->playback_dma_data);
- dai->playback_dma_data = NULL;
- } else {
- kfree(dai->capture_dma_data);
- dai->capture_dma_data = NULL;
+ /* second paranoia check */
+ if (!dai_runtime) {
+ dev_err(dai->dev,
+ "dai_runtime not allocated for dai %s\n",
+ dai->name);
+ return -EINVAL;
}
+
+ /* for NULL stream we release allocated dai_runtime */
+ kfree(dai_runtime);
+ cdns->dai_runtime_array[dai->id] = NULL;
}
return 0;
}
};
/**
- * struct sdw_cdns_dma_data: Cadence DMA data
+ * struct sdw_cdns_dai_runtime: Cadence DAI runtime data
*
* @name: SoundWire stream name
* @stream: stream runtime
* @hw_params: hw_params to be applied in .prepare step
* @suspended: status set when suspended, to be used in .prepare
* @paused: status set in .trigger, to be used in suspend
+ * @direction: stream direction
*/
-struct sdw_cdns_dma_data {
+struct sdw_cdns_dai_runtime {
char *name;
struct sdw_stream_runtime *stream;
struct sdw_cdns_pdi *pdi;
struct snd_pcm_hw_params *hw_params;
bool suspended;
bool paused;
+ int direction;
};
/**
* @registers: Cadence registers
* @link_up: Link status
* @msg_count: Messages sent on bus
+ * @dai_runtime_array: runtime context for each allocated DAI.
*/
struct sdw_cdns {
struct device *dev;
struct work_struct work;
struct list_head list;
+
+ struct sdw_cdns_dai_runtime **dai_runtime_array;
};
#define bus_to_cdns(_bus) container_of(_bus, struct sdw_cdns, bus)
.driver_data = (void *)intel_tgl_bios,
},
{
+ /* quirk used for NUC15 LAPBC710 skew */
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "Intel Corporation"),
+ DMI_MATCH(DMI_BOARD_NAME, "LAPBC710"),
+ },
+ .driver_data = (void *)intel_tgl_bios,
+ },
+ {
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc"),
DMI_EXACT_MATCH(DMI_PRODUCT_SKU, "0A3E")
#include <linux/acpi.h>
#include <linux/debugfs.h>
#include <linux/delay.h>
-#include <linux/module.h>
-#include <linux/interrupt.h>
#include <linux/io.h>
-#include <linux/auxiliary_bus.h>
#include <sound/pcm_params.h>
#include <linux/pm_runtime.h>
#include <sound/soc.h>
#include "bus.h"
#include "intel.h"
-/* IDA min selected to avoid conflicts with HDaudio/iDISP SDI values */
-#define INTEL_DEV_NUM_IDA_MIN 4
-
-#define INTEL_MASTER_SUSPEND_DELAY_MS 3000
-#define INTEL_MASTER_RESET_ITERATIONS 10
-
-/*
- * debug/config flags for the Intel SoundWire Master.
- *
- * Since we may have multiple masters active, we can have up to 8
- * flags reused in each byte, with master0 using the ls-byte, etc.
- */
-
-#define SDW_INTEL_MASTER_DISABLE_PM_RUNTIME BIT(0)
-#define SDW_INTEL_MASTER_DISABLE_CLOCK_STOP BIT(1)
-#define SDW_INTEL_MASTER_DISABLE_PM_RUNTIME_IDLE BIT(2)
-#define SDW_INTEL_MASTER_DISABLE_MULTI_LINK BIT(3)
-
-static int md_flags;
-module_param_named(sdw_md_flags, md_flags, int, 0444);
-MODULE_PARM_DESC(sdw_md_flags, "SoundWire Intel Master device flags (0x0 all off)");
enum intel_pdi_type {
INTEL_PDI_IN = 0,
* bank switch routines
*/
-static int intel_pre_bank_switch(struct sdw_bus *bus)
+static int intel_pre_bank_switch(struct sdw_intel *sdw)
{
- struct sdw_cdns *cdns = bus_to_cdns(bus);
- struct sdw_intel *sdw = cdns_to_intel(cdns);
+ struct sdw_cdns *cdns = &sdw->cdns;
+ struct sdw_bus *bus = &cdns->bus;
/* Write to register only for multi-link */
if (!bus->multi_link)
return 0;
}
-static int intel_post_bank_switch(struct sdw_bus *bus)
+static int intel_post_bank_switch(struct sdw_intel *sdw)
{
- struct sdw_cdns *cdns = bus_to_cdns(bus);
- struct sdw_intel *sdw = cdns_to_intel(cdns);
+ struct sdw_cdns *cdns = &sdw->cdns;
+ struct sdw_bus *bus = &cdns->bus;
void __iomem *shim = sdw->link_res->shim;
int sync_reg, ret;
{
struct sdw_cdns *cdns = snd_soc_dai_get_drvdata(dai);
struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_cdns_dma_data *dma;
+ struct sdw_cdns_dai_runtime *dai_runtime;
struct sdw_cdns_pdi *pdi;
struct sdw_stream_config sconfig;
struct sdw_port_config *pconfig;
int ch, dir;
int ret;
- dma = snd_soc_dai_get_dma_data(dai, substream);
- if (!dma)
+ dai_runtime = cdns->dai_runtime_array[dai->id];
+ if (!dai_runtime)
return -EIO;
ch = params_channels(params);
sdw_cdns_config_stream(cdns, ch, dir, pdi);
/* store pdi and hw_params, may be needed in prepare step */
- dma->paused = false;
- dma->suspended = false;
- dma->pdi = pdi;
- dma->hw_params = params;
+ dai_runtime->paused = false;
+ dai_runtime->suspended = false;
+ dai_runtime->pdi = pdi;
+ dai_runtime->hw_params = params;
/* Inform DSP about PDI stream number */
ret = intel_params_stream(sdw, substream->stream, dai, params,
sconfig.direction = dir;
sconfig.ch_count = ch;
sconfig.frame_rate = params_rate(params);
- sconfig.type = dma->stream_type;
+ sconfig.type = dai_runtime->stream_type;
sconfig.bps = snd_pcm_format_width(params_format(params));
pconfig->ch_mask = (1 << ch) - 1;
ret = sdw_stream_add_master(&cdns->bus, &sconfig,
- pconfig, 1, dma->stream);
+ pconfig, 1, dai_runtime->stream);
if (ret)
dev_err(cdns->dev, "add master to stream failed:%d\n", ret);
{
struct sdw_cdns *cdns = snd_soc_dai_get_drvdata(dai);
struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_cdns_dma_data *dma;
+ struct sdw_cdns_dai_runtime *dai_runtime;
int ch, dir;
int ret = 0;
- dma = snd_soc_dai_get_dma_data(dai, substream);
- if (!dma) {
- dev_err(dai->dev, "failed to get dma data in %s\n",
+ dai_runtime = cdns->dai_runtime_array[dai->id];
+ if (!dai_runtime) {
+ dev_err(dai->dev, "failed to get dai runtime in %s\n",
__func__);
return -EIO;
}
- if (dma->suspended) {
- dma->suspended = false;
+ if (dai_runtime->suspended) {
+ dai_runtime->suspended = false;
/*
* .prepare() is called after system resume, where we
*/
/* configure stream */
- ch = params_channels(dma->hw_params);
+ ch = params_channels(dai_runtime->hw_params);
if (substream->stream == SNDRV_PCM_STREAM_CAPTURE)
dir = SDW_DATA_DIR_RX;
else
dir = SDW_DATA_DIR_TX;
- intel_pdi_shim_configure(sdw, dma->pdi);
- intel_pdi_alh_configure(sdw, dma->pdi);
- sdw_cdns_config_stream(cdns, ch, dir, dma->pdi);
+ intel_pdi_shim_configure(sdw, dai_runtime->pdi);
+ intel_pdi_alh_configure(sdw, dai_runtime->pdi);
+ sdw_cdns_config_stream(cdns, ch, dir, dai_runtime->pdi);
/* Inform DSP about PDI stream number */
ret = intel_params_stream(sdw, substream->stream, dai,
- dma->hw_params,
+ dai_runtime->hw_params,
sdw->instance,
- dma->pdi->intel_alh_id);
+ dai_runtime->pdi->intel_alh_id);
}
return ret;
{
struct sdw_cdns *cdns = snd_soc_dai_get_drvdata(dai);
struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_cdns_dma_data *dma;
+ struct sdw_cdns_dai_runtime *dai_runtime;
int ret;
- dma = snd_soc_dai_get_dma_data(dai, substream);
- if (!dma)
+ dai_runtime = cdns->dai_runtime_array[dai->id];
+ if (!dai_runtime)
return -EIO;
/*
* DEPREPARED for the first cpu-dai and to RELEASED for the last
* cpu-dai.
*/
- ret = sdw_stream_remove_master(&cdns->bus, dma->stream);
+ ret = sdw_stream_remove_master(&cdns->bus, dai_runtime->stream);
if (ret < 0) {
dev_err(dai->dev, "remove master from stream %s failed: %d\n",
- dma->stream->name, ret);
+ dai_runtime->stream->name, ret);
return ret;
}
return ret;
}
- dma->hw_params = NULL;
- dma->pdi = NULL;
+ dai_runtime->hw_params = NULL;
+ dai_runtime->pdi = NULL;
return 0;
}
static void *intel_get_sdw_stream(struct snd_soc_dai *dai,
int direction)
{
- struct sdw_cdns_dma_data *dma;
-
- if (direction == SNDRV_PCM_STREAM_PLAYBACK)
- dma = dai->playback_dma_data;
- else
- dma = dai->capture_dma_data;
+ struct sdw_cdns *cdns = snd_soc_dai_get_drvdata(dai);
+ struct sdw_cdns_dai_runtime *dai_runtime;
- if (!dma)
+ dai_runtime = cdns->dai_runtime_array[dai->id];
+ if (!dai_runtime)
return ERR_PTR(-EINVAL);
- return dma->stream;
+ return dai_runtime->stream;
}
static int intel_trigger(struct snd_pcm_substream *substream, int cmd, struct snd_soc_dai *dai)
struct sdw_cdns *cdns = snd_soc_dai_get_drvdata(dai);
struct sdw_intel *sdw = cdns_to_intel(cdns);
struct sdw_intel_link_res *res = sdw->link_res;
- struct sdw_cdns_dma_data *dma;
+ struct sdw_cdns_dai_runtime *dai_runtime;
int ret = 0;
/*
if (res->ops && res->ops->trigger)
res->ops->trigger(dai, cmd, substream->stream);
- dma = snd_soc_dai_get_dma_data(dai, substream);
- if (!dma) {
- dev_err(dai->dev, "failed to get dma data in %s\n",
+ dai_runtime = cdns->dai_runtime_array[dai->id];
+ if (!dai_runtime) {
+ dev_err(dai->dev, "failed to get dai runtime in %s\n",
__func__);
return -EIO;
}
* the .trigger callback is used to track the suspend case only.
*/
- dma->suspended = true;
+ dai_runtime->suspended = true;
ret = intel_free_stream(sdw, substream->stream, dai, sdw->instance);
break;
case SNDRV_PCM_TRIGGER_PAUSE_PUSH:
- dma->paused = true;
+ dai_runtime->paused = true;
break;
case SNDRV_PCM_TRIGGER_STOP:
case SNDRV_PCM_TRIGGER_PAUSE_RELEASE:
- dma->paused = false;
+ dai_runtime->paused = false;
break;
default:
break;
for_each_component_dais(component, dai) {
struct sdw_cdns *cdns = snd_soc_dai_get_drvdata(dai);
struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_cdns_dma_data *dma;
- int stream;
+ struct sdw_cdns_dai_runtime *dai_runtime;
int ret;
- dma = dai->playback_dma_data;
- stream = SNDRV_PCM_STREAM_PLAYBACK;
- if (!dma) {
- dma = dai->capture_dma_data;
- stream = SNDRV_PCM_STREAM_CAPTURE;
- }
+ dai_runtime = cdns->dai_runtime_array[dai->id];
- if (!dma)
+ if (!dai_runtime)
continue;
- if (dma->suspended)
+ if (dai_runtime->suspended)
continue;
- if (dma->paused) {
- dma->suspended = true;
+ if (dai_runtime->paused) {
+ dai_runtime->suspended = true;
- ret = intel_free_stream(sdw, stream, dai, sdw->instance);
+ ret = intel_free_stream(sdw, dai_runtime->direction, dai, sdw->instance);
if (ret < 0)
return ret;
}
static int intel_register_dai(struct sdw_intel *sdw)
{
+ struct sdw_cdns_dai_runtime **dai_runtime_array;
struct sdw_cdns_stream_config config;
struct sdw_cdns *cdns = &sdw->cdns;
struct sdw_cdns_streams *stream;
/* DAIs are created based on total number of PDIs supported */
num_dai = cdns->pcm.num_pdi;
+ dai_runtime_array = devm_kcalloc(cdns->dev, num_dai,
+ sizeof(struct sdw_cdns_dai_runtime *),
+ GFP_KERNEL);
+ if (!dai_runtime_array)
+ return -ENOMEM;
+ cdns->dai_runtime_array = dai_runtime_array;
+
dais = devm_kcalloc(cdns->dev, num_dai, sizeof(*dais), GFP_KERNEL);
if (!dais)
return -ENOMEM;
return 0;
}
-static int sdw_master_read_intel_prop(struct sdw_bus *bus)
-{
- struct sdw_master_prop *prop = &bus->prop;
- struct fwnode_handle *link;
- char name[32];
- u32 quirk_mask;
-
- /* Find master handle */
- snprintf(name, sizeof(name),
- "mipi-sdw-link-%d-subproperties", bus->link_id);
-
- link = device_get_named_child_node(bus->dev, name);
- if (!link) {
- dev_err(bus->dev, "Master node %s not found\n", name);
- return -EIO;
- }
-
- fwnode_property_read_u32(link,
- "intel-sdw-ip-clock",
- &prop->mclk_freq);
-
- /* the values reported by BIOS are the 2x clock, not the bus clock */
- prop->mclk_freq /= 2;
-
- fwnode_property_read_u32(link,
- "intel-quirk-mask",
- &quirk_mask);
-
- if (quirk_mask & SDW_INTEL_QUIRK_MASK_BUS_DISABLE)
- prop->hw_disabled = true;
+const struct sdw_intel_hw_ops sdw_intel_cnl_hw_ops = {
+ .debugfs_init = intel_debugfs_init,
+ .debugfs_exit = intel_debugfs_exit,
- prop->quirks = SDW_MASTER_QUIRKS_CLEAR_INITIAL_CLASH |
- SDW_MASTER_QUIRKS_CLEAR_INITIAL_PARITY;
+ .register_dai = intel_register_dai,
- return 0;
-}
+ .check_clock_stop = intel_check_clock_stop,
+ .start_bus = intel_start_bus,
+ .start_bus_after_reset = intel_start_bus_after_reset,
+ .start_bus_after_clock_stop = intel_start_bus_after_clock_stop,
+ .stop_bus = intel_stop_bus,
-static int intel_prop_read(struct sdw_bus *bus)
-{
- /* Initialize with default handler to read all DisCo properties */
- sdw_master_read_prop(bus);
+ .link_power_up = intel_link_power_up,
+ .link_power_down = intel_link_power_down,
- /* read Intel-specific properties */
- sdw_master_read_intel_prop(bus);
+ .shim_check_wake = intel_shim_check_wake,
+ .shim_wake = intel_shim_wake,
- return 0;
-}
-
-static struct sdw_master_ops sdw_intel_ops = {
- .read_prop = intel_prop_read,
- .override_adr = sdw_dmi_override_adr,
- .xfer_msg = cdns_xfer_msg,
- .xfer_msg_defer = cdns_xfer_msg_defer,
- .reset_page_addr = cdns_reset_page_addr,
- .set_bus_conf = cdns_bus_conf,
.pre_bank_switch = intel_pre_bank_switch,
.post_bank_switch = intel_post_bank_switch,
- .read_ping_status = cdns_read_ping_status,
-};
-
-/*
- * probe and init (aux_dev_id argument is required by function prototype but not used)
- */
-static int intel_link_probe(struct auxiliary_device *auxdev,
- const struct auxiliary_device_id *aux_dev_id)
-
-{
- struct device *dev = &auxdev->dev;
- struct sdw_intel_link_dev *ldev = auxiliary_dev_to_sdw_intel_link_dev(auxdev);
- struct sdw_intel *sdw;
- struct sdw_cdns *cdns;
- struct sdw_bus *bus;
- int ret;
-
- sdw = devm_kzalloc(dev, sizeof(*sdw), GFP_KERNEL);
- if (!sdw)
- return -ENOMEM;
-
- cdns = &sdw->cdns;
- bus = &cdns->bus;
-
- sdw->instance = auxdev->id;
- sdw->link_res = &ldev->link_res;
- cdns->dev = dev;
- cdns->registers = sdw->link_res->registers;
- cdns->instance = sdw->instance;
- cdns->msg_count = 0;
-
- bus->link_id = auxdev->id;
- bus->dev_num_ida_min = INTEL_DEV_NUM_IDA_MIN;
- bus->clk_stop_timeout = 1;
-
- sdw_cdns_probe(cdns);
-
- /* Set ops */
- bus->ops = &sdw_intel_ops;
-
- /* set driver data, accessed by snd_soc_dai_get_drvdata() */
- auxiliary_set_drvdata(auxdev, cdns);
-
- /* use generic bandwidth allocation algorithm */
- sdw->cdns.bus.compute_params = sdw_compute_params;
-
- /* avoid resuming from pm_runtime suspend if it's not required */
- dev_pm_set_driver_flags(dev, DPM_FLAG_SMART_SUSPEND);
-
- ret = sdw_bus_master_add(bus, dev, dev->fwnode);
- if (ret) {
- dev_err(dev, "sdw_bus_master_add fail: %d\n", ret);
- return ret;
- }
-
- if (bus->prop.hw_disabled)
- dev_info(dev,
- "SoundWire master %d is disabled, will be ignored\n",
- bus->link_id);
- /*
- * Ignore BIOS err_threshold, it's a really bad idea when dealing
- * with multiple hardware synchronized links
- */
- bus->prop.err_threshold = 0;
-
- return 0;
-}
-
-int intel_link_startup(struct auxiliary_device *auxdev)
-{
- struct device *dev = &auxdev->dev;
- struct sdw_cdns *cdns = auxiliary_get_drvdata(auxdev);
- struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_bus *bus = &cdns->bus;
- int link_flags;
- bool multi_link;
- u32 clock_stop_quirks;
- int ret;
-
- if (bus->prop.hw_disabled) {
- dev_info(dev,
- "SoundWire master %d is disabled, ignoring\n",
- sdw->instance);
- return 0;
- }
-
- link_flags = md_flags >> (bus->link_id * 8);
- multi_link = !(link_flags & SDW_INTEL_MASTER_DISABLE_MULTI_LINK);
- if (!multi_link) {
- dev_dbg(dev, "Multi-link is disabled\n");
- } else {
- /*
- * hardware-based synchronization is required regardless
- * of the number of segments used by a stream: SSP-based
- * synchronization is gated by gsync when the multi-master
- * mode is set.
- */
- bus->hw_sync_min_links = 1;
- }
- bus->multi_link = multi_link;
-
- /* Initialize shim, controller */
- ret = intel_link_power_up(sdw);
- if (ret)
- goto err_init;
-
- /* Register DAIs */
- ret = intel_register_dai(sdw);
- if (ret) {
- dev_err(dev, "DAI registration failed: %d\n", ret);
- goto err_power_up;
- }
-
- intel_debugfs_init(sdw);
-
- /* start bus */
- ret = intel_start_bus(sdw);
- if (ret) {
- dev_err(dev, "bus start failed: %d\n", ret);
- goto err_power_up;
- }
-
- /* Enable runtime PM */
- if (!(link_flags & SDW_INTEL_MASTER_DISABLE_PM_RUNTIME)) {
- pm_runtime_set_autosuspend_delay(dev,
- INTEL_MASTER_SUSPEND_DELAY_MS);
- pm_runtime_use_autosuspend(dev);
- pm_runtime_mark_last_busy(dev);
-
- pm_runtime_set_active(dev);
- pm_runtime_enable(dev);
- }
-
- clock_stop_quirks = sdw->link_res->clock_stop_quirks;
- if (clock_stop_quirks & SDW_INTEL_CLK_STOP_NOT_ALLOWED) {
- /*
- * To keep the clock running we need to prevent
- * pm_runtime suspend from happening by increasing the
- * reference count.
- * This quirk is specified by the parent PCI device in
- * case of specific latency requirements. It will have
- * no effect if pm_runtime is disabled by the user via
- * a module parameter for testing purposes.
- */
- pm_runtime_get_noresume(dev);
- }
-
- /*
- * The runtime PM status of Slave devices is "Unsupported"
- * until they report as ATTACHED. If they don't, e.g. because
- * there are no Slave devices populated or if the power-on is
- * delayed or dependent on a power switch, the Master will
- * remain active and prevent its parent from suspending.
- *
- * Conditionally force the pm_runtime core to re-evaluate the
- * Master status in the absence of any Slave activity. A quirk
- * is provided to e.g. deal with Slaves that may be powered on
- * with a delay. A more complete solution would require the
- * definition of Master properties.
- */
- if (!(link_flags & SDW_INTEL_MASTER_DISABLE_PM_RUNTIME_IDLE))
- pm_runtime_idle(dev);
-
- sdw->startup_done = true;
- return 0;
-
-err_power_up:
- intel_link_power_down(sdw);
-err_init:
- return ret;
-}
-
-static void intel_link_remove(struct auxiliary_device *auxdev)
-{
- struct sdw_cdns *cdns = auxiliary_get_drvdata(auxdev);
- struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_bus *bus = &cdns->bus;
-
- /*
- * Since pm_runtime is already disabled, we don't decrease
- * the refcount when the clock_stop_quirk is
- * SDW_INTEL_CLK_STOP_NOT_ALLOWED
- */
- if (!bus->prop.hw_disabled) {
- intel_debugfs_exit(sdw);
- sdw_cdns_enable_interrupt(cdns, false);
- }
- sdw_bus_master_delete(bus);
-}
-
-int intel_link_process_wakeen_event(struct auxiliary_device *auxdev)
-{
- struct device *dev = &auxdev->dev;
- struct sdw_intel *sdw;
- struct sdw_bus *bus;
-
- sdw = auxiliary_get_drvdata(auxdev);
- bus = &sdw->cdns.bus;
-
- if (bus->prop.hw_disabled || !sdw->startup_done) {
- dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
- bus->link_id);
- return 0;
- }
-
- if (!intel_shim_check_wake(sdw))
- return 0;
-
- /* disable WAKEEN interrupt ASAP to prevent interrupt flood */
- intel_shim_wake(sdw, false);
-
- /*
- * resume the Master, which will generate a bus reset and result in
- * Slaves re-attaching and be re-enumerated. The SoundWire physical
- * device which generated the wake will trigger an interrupt, which
- * will in turn cause the corresponding Linux Slave device to be
- * resumed and the Slave codec driver to check the status.
- */
- pm_request_resume(dev);
-
- return 0;
-}
-
-/*
- * PM calls
- */
-
-static int intel_resume_child_device(struct device *dev, void *data)
-{
- int ret;
- struct sdw_slave *slave = dev_to_sdw_dev(dev);
-
- if (!slave->probed) {
- dev_dbg(dev, "skipping device, no probed driver\n");
- return 0;
- }
- if (!slave->dev_num_sticky) {
- dev_dbg(dev, "skipping device, never detected on bus\n");
- return 0;
- }
-
- ret = pm_request_resume(dev);
- if (ret < 0)
- dev_err(dev, "%s: pm_request_resume failed: %d\n", __func__, ret);
-
- return ret;
-}
-
-static int __maybe_unused intel_pm_prepare(struct device *dev)
-{
- struct sdw_cdns *cdns = dev_get_drvdata(dev);
- struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_bus *bus = &cdns->bus;
- u32 clock_stop_quirks;
- int ret;
-
- if (bus->prop.hw_disabled || !sdw->startup_done) {
- dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
- bus->link_id);
- return 0;
- }
-
- clock_stop_quirks = sdw->link_res->clock_stop_quirks;
-
- if (pm_runtime_suspended(dev) &&
- pm_runtime_suspended(dev->parent) &&
- ((clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET) ||
- !clock_stop_quirks)) {
- /*
- * if we've enabled clock stop, and the parent is suspended, the SHIM registers
- * are not accessible and the shim wake cannot be disabled.
- * The only solution is to resume the entire bus to full power
- */
-
- /*
- * If any operation in this block fails, we keep going since we don't want
- * to prevent system suspend from happening and errors should be recoverable
- * on resume.
- */
-
- /*
- * first resume the device for this link. This will also by construction
- * resume the PCI parent device.
- */
- ret = pm_request_resume(dev);
- if (ret < 0) {
- dev_err(dev, "%s: pm_request_resume failed: %d\n", __func__, ret);
- return 0;
- }
-
- /*
- * Continue resuming the entire bus (parent + child devices) to exit
- * the clock stop mode. If there are no devices connected on this link
- * this is a no-op.
- * The resume to full power could have been implemented with a .prepare
- * step in SoundWire codec drivers. This would however require a lot
- * of code to handle an Intel-specific corner case. It is simpler in
- * practice to add a loop at the link level.
- */
- ret = device_for_each_child(bus->dev, NULL, intel_resume_child_device);
-
- if (ret < 0)
- dev_err(dev, "%s: intel_resume_child_device failed: %d\n", __func__, ret);
- }
-
- return 0;
-}
-
-static int __maybe_unused intel_suspend(struct device *dev)
-{
- struct sdw_cdns *cdns = dev_get_drvdata(dev);
- struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_bus *bus = &cdns->bus;
- u32 clock_stop_quirks;
- int ret;
-
- if (bus->prop.hw_disabled || !sdw->startup_done) {
- dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
- bus->link_id);
- return 0;
- }
-
- if (pm_runtime_suspended(dev)) {
- dev_dbg(dev, "pm_runtime status: suspended\n");
-
- clock_stop_quirks = sdw->link_res->clock_stop_quirks;
-
- if ((clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET) ||
- !clock_stop_quirks) {
-
- if (pm_runtime_suspended(dev->parent)) {
- /*
- * paranoia check: this should not happen with the .prepare
- * resume to full power
- */
- dev_err(dev, "%s: invalid config: parent is suspended\n", __func__);
- } else {
- intel_shim_wake(sdw, false);
- }
- }
-
- return 0;
- }
-
- ret = intel_stop_bus(sdw, false);
- if (ret < 0) {
- dev_err(dev, "%s: cannot stop bus: %d\n", __func__, ret);
- return ret;
- }
-
- return 0;
-}
-
-static int __maybe_unused intel_suspend_runtime(struct device *dev)
-{
- struct sdw_cdns *cdns = dev_get_drvdata(dev);
- struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_bus *bus = &cdns->bus;
- u32 clock_stop_quirks;
- int ret;
-
- if (bus->prop.hw_disabled || !sdw->startup_done) {
- dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
- bus->link_id);
- return 0;
- }
-
- clock_stop_quirks = sdw->link_res->clock_stop_quirks;
-
- if (clock_stop_quirks & SDW_INTEL_CLK_STOP_TEARDOWN) {
- ret = intel_stop_bus(sdw, false);
- if (ret < 0) {
- dev_err(dev, "%s: cannot stop bus during teardown: %d\n",
- __func__, ret);
- return ret;
- }
- } else if (clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET || !clock_stop_quirks) {
- ret = intel_stop_bus(sdw, true);
- if (ret < 0) {
- dev_err(dev, "%s: cannot stop bus during clock_stop: %d\n",
- __func__, ret);
- return ret;
- }
- } else {
- dev_err(dev, "%s clock_stop_quirks %x unsupported\n",
- __func__, clock_stop_quirks);
- ret = -EINVAL;
- }
-
- return ret;
-}
-
-static int __maybe_unused intel_resume(struct device *dev)
-{
- struct sdw_cdns *cdns = dev_get_drvdata(dev);
- struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_bus *bus = &cdns->bus;
- int link_flags;
- int ret;
-
- if (bus->prop.hw_disabled || !sdw->startup_done) {
- dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
- bus->link_id);
- return 0;
- }
-
- link_flags = md_flags >> (bus->link_id * 8);
-
- if (pm_runtime_suspended(dev)) {
- dev_dbg(dev, "pm_runtime status was suspended, forcing active\n");
-
- /* follow required sequence from runtime_pm.rst */
- pm_runtime_disable(dev);
- pm_runtime_set_active(dev);
- pm_runtime_mark_last_busy(dev);
- pm_runtime_enable(dev);
-
- link_flags = md_flags >> (bus->link_id * 8);
-
- if (!(link_flags & SDW_INTEL_MASTER_DISABLE_PM_RUNTIME_IDLE))
- pm_runtime_idle(dev);
- }
-
- ret = intel_link_power_up(sdw);
- if (ret) {
- dev_err(dev, "%s failed: %d\n", __func__, ret);
- return ret;
- }
-
- /*
- * make sure all Slaves are tagged as UNATTACHED and provide
- * reason for reinitialization
- */
- sdw_clear_slave_status(bus, SDW_UNATTACH_REQUEST_MASTER_RESET);
-
- ret = intel_start_bus(sdw);
- if (ret < 0) {
- dev_err(dev, "cannot start bus during resume\n");
- intel_link_power_down(sdw);
- return ret;
- }
-
- /*
- * after system resume, the pm_runtime suspend() may kick in
- * during the enumeration, before any children device force the
- * master device to remain active. Using pm_runtime_get()
- * routines is not really possible, since it'd prevent the
- * master from suspending.
- * A reasonable compromise is to update the pm_runtime
- * counters and delay the pm_runtime suspend by several
- * seconds, by when all enumeration should be complete.
- */
- pm_runtime_mark_last_busy(dev);
-
- return 0;
-}
-
-static int __maybe_unused intel_resume_runtime(struct device *dev)
-{
- struct sdw_cdns *cdns = dev_get_drvdata(dev);
- struct sdw_intel *sdw = cdns_to_intel(cdns);
- struct sdw_bus *bus = &cdns->bus;
- u32 clock_stop_quirks;
- int ret;
-
- if (bus->prop.hw_disabled || !sdw->startup_done) {
- dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
- bus->link_id);
- return 0;
- }
-
- /* unconditionally disable WAKEEN interrupt */
- intel_shim_wake(sdw, false);
-
- clock_stop_quirks = sdw->link_res->clock_stop_quirks;
-
- if (clock_stop_quirks & SDW_INTEL_CLK_STOP_TEARDOWN) {
- ret = intel_link_power_up(sdw);
- if (ret) {
- dev_err(dev, "%s: power_up failed after teardown: %d\n", __func__, ret);
- return ret;
- }
-
- /*
- * make sure all Slaves are tagged as UNATTACHED and provide
- * reason for reinitialization
- */
- sdw_clear_slave_status(bus, SDW_UNATTACH_REQUEST_MASTER_RESET);
-
- ret = intel_start_bus(sdw);
- if (ret < 0) {
- dev_err(dev, "%s: cannot start bus after teardown: %d\n", __func__, ret);
- intel_link_power_down(sdw);
- return ret;
- }
-
-
- } else if (clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET) {
- ret = intel_link_power_up(sdw);
- if (ret) {
- dev_err(dev, "%s: power_up failed after bus reset: %d\n", __func__, ret);
- return ret;
- }
-
- ret = intel_start_bus_after_reset(sdw);
- if (ret < 0) {
- dev_err(dev, "%s: cannot start bus after reset: %d\n", __func__, ret);
- intel_link_power_down(sdw);
- return ret;
- }
- } else if (!clock_stop_quirks) {
-
- intel_check_clock_stop(sdw);
-
- ret = intel_link_power_up(sdw);
- if (ret) {
- dev_err(dev, "%s: power_up failed: %d\n", __func__, ret);
- return ret;
- }
-
- ret = intel_start_bus_after_clock_stop(sdw);
- if (ret < 0) {
- dev_err(dev, "%s: cannot start bus after clock stop: %d\n", __func__, ret);
- intel_link_power_down(sdw);
- return ret;
- }
- } else {
- dev_err(dev, "%s: clock_stop_quirks %x unsupported\n",
- __func__, clock_stop_quirks);
- ret = -EINVAL;
- }
-
- return ret;
-}
-
-static const struct dev_pm_ops intel_pm = {
- .prepare = intel_pm_prepare,
- SET_SYSTEM_SLEEP_PM_OPS(intel_suspend, intel_resume)
- SET_RUNTIME_PM_OPS(intel_suspend_runtime, intel_resume_runtime, NULL)
-};
-
-static const struct auxiliary_device_id intel_link_id_table[] = {
- { .name = "soundwire_intel.link" },
- {},
-};
-MODULE_DEVICE_TABLE(auxiliary, intel_link_id_table);
-
-static struct auxiliary_driver sdw_intel_drv = {
- .probe = intel_link_probe,
- .remove = intel_link_remove,
- .driver = {
- /* auxiliary_driver_register() sets .name to be the modname */
- .pm = &intel_pm,
- },
- .id_table = intel_link_id_table
};
-module_auxiliary_driver(sdw_intel_drv);
+EXPORT_SYMBOL_NS(sdw_intel_cnl_hw_ops, SOUNDWIRE_INTEL);
-MODULE_LICENSE("Dual BSD/GPL");
-MODULE_DESCRIPTION("Intel Soundwire Link Driver");
/**
* struct sdw_intel_link_res - Soundwire Intel link resource structure,
* typically populated by the controller driver.
+ * @hw_ops: platform-specific ops
* @mmio_base: mmio base of SoundWire registers
* @registers: Link IO registers base
* @shim: Audio shim pointer
* @list: used to walk-through all masters exposed by the same controller
*/
struct sdw_intel_link_res {
+ const struct sdw_intel_hw_ops *hw_ops;
+
void __iomem *mmio_base; /* not strictly needed, useful for debug */
void __iomem *registers;
void __iomem *shim;
#endif
};
-int intel_link_startup(struct auxiliary_device *auxdev);
-int intel_link_process_wakeen_event(struct auxiliary_device *auxdev);
+#define cdns_to_intel(_cdns) container_of(_cdns, struct sdw_intel, cdns)
-struct sdw_intel_link_dev {
- struct auxiliary_device auxdev;
- struct sdw_intel_link_res link_res;
-};
+#define INTEL_MASTER_RESET_ITERATIONS 10
+
+#define SDW_INTEL_CHECK_OPS(sdw, cb) ((sdw) && (sdw)->link_res && (sdw)->link_res->hw_ops && \
+ (sdw)->link_res->hw_ops->cb)
+#define SDW_INTEL_OPS(sdw, cb) ((sdw)->link_res->hw_ops->cb)
+
+static inline void sdw_intel_debugfs_init(struct sdw_intel *sdw)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, debugfs_init))
+ SDW_INTEL_OPS(sdw, debugfs_init)(sdw);
+}
+
+static inline void sdw_intel_debugfs_exit(struct sdw_intel *sdw)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, debugfs_exit))
+ SDW_INTEL_OPS(sdw, debugfs_exit)(sdw);
+}
+
+static inline int sdw_intel_register_dai(struct sdw_intel *sdw)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, register_dai))
+ return SDW_INTEL_OPS(sdw, register_dai)(sdw);
+ return -ENOTSUPP;
+}
+
+static inline void sdw_intel_check_clock_stop(struct sdw_intel *sdw)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, check_clock_stop))
+ SDW_INTEL_OPS(sdw, check_clock_stop)(sdw);
+}
+
+static inline int sdw_intel_start_bus(struct sdw_intel *sdw)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, start_bus))
+ return SDW_INTEL_OPS(sdw, start_bus)(sdw);
+ return -ENOTSUPP;
+}
+
+static inline int sdw_intel_start_bus_after_reset(struct sdw_intel *sdw)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, start_bus_after_reset))
+ return SDW_INTEL_OPS(sdw, start_bus_after_reset)(sdw);
+ return -ENOTSUPP;
+}
+
+static inline int sdw_intel_start_bus_after_clock_stop(struct sdw_intel *sdw)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, start_bus_after_clock_stop))
+ return SDW_INTEL_OPS(sdw, start_bus_after_clock_stop)(sdw);
+ return -ENOTSUPP;
+}
+
+static inline int sdw_intel_stop_bus(struct sdw_intel *sdw, bool clock_stop)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, stop_bus))
+ return SDW_INTEL_OPS(sdw, stop_bus)(sdw, clock_stop);
+ return -ENOTSUPP;
+}
+
+static inline int sdw_intel_link_power_up(struct sdw_intel *sdw)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, link_power_up))
+ return SDW_INTEL_OPS(sdw, link_power_up)(sdw);
+ return -ENOTSUPP;
+}
+
+static inline int sdw_intel_link_power_down(struct sdw_intel *sdw)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, link_power_down))
+ return SDW_INTEL_OPS(sdw, link_power_down)(sdw);
+ return -ENOTSUPP;
+}
+
+static inline int sdw_intel_shim_check_wake(struct sdw_intel *sdw)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, shim_check_wake))
+ return SDW_INTEL_OPS(sdw, shim_check_wake)(sdw);
+ return -ENOTSUPP;
+}
-#define auxiliary_dev_to_sdw_intel_link_dev(auxiliary_dev) \
- container_of(auxiliary_dev, struct sdw_intel_link_dev, auxdev)
+static inline void sdw_intel_shim_wake(struct sdw_intel *sdw, bool wake_enable)
+{
+ if (SDW_INTEL_CHECK_OPS(sdw, shim_wake))
+ SDW_INTEL_OPS(sdw, shim_wake)(sdw, wake_enable);
+}
#endif /* __SDW_INTEL_LOCAL_H */
--- /dev/null
+// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
+// Copyright(c) 2015-22 Intel Corporation.
+
+/*
+ * Soundwire Intel Manager Driver
+ */
+
+#include <linux/acpi.h>
+#include <linux/debugfs.h>
+#include <linux/delay.h>
+#include <linux/module.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/auxiliary_bus.h>
+#include <sound/pcm_params.h>
+#include <linux/pm_runtime.h>
+#include <sound/soc.h>
+#include <linux/soundwire/sdw_registers.h>
+#include <linux/soundwire/sdw.h>
+#include <linux/soundwire/sdw_intel.h>
+#include "cadence_master.h"
+#include "bus.h"
+#include "intel.h"
+#include "intel_auxdevice.h"
+
+/* IDA min selected to avoid conflicts with HDaudio/iDISP SDI values */
+#define INTEL_DEV_NUM_IDA_MIN 4
+
+#define INTEL_MASTER_SUSPEND_DELAY_MS 3000
+
+/*
+ * debug/config flags for the Intel SoundWire Master.
+ *
+ * Since we may have multiple masters active, we can have up to 8
+ * flags reused in each byte, with master0 using the ls-byte, etc.
+ */
+
+#define SDW_INTEL_MASTER_DISABLE_PM_RUNTIME BIT(0)
+#define SDW_INTEL_MASTER_DISABLE_CLOCK_STOP BIT(1)
+#define SDW_INTEL_MASTER_DISABLE_PM_RUNTIME_IDLE BIT(2)
+#define SDW_INTEL_MASTER_DISABLE_MULTI_LINK BIT(3)
+
+static int md_flags;
+module_param_named(sdw_md_flags, md_flags, int, 0444);
+MODULE_PARM_DESC(sdw_md_flags, "SoundWire Intel Master device flags (0x0 all off)");
+
+static int generic_pre_bank_switch(struct sdw_bus *bus)
+{
+ struct sdw_cdns *cdns = bus_to_cdns(bus);
+ struct sdw_intel *sdw = cdns_to_intel(cdns);
+
+ return sdw->link_res->hw_ops->pre_bank_switch(sdw);
+}
+
+static int generic_post_bank_switch(struct sdw_bus *bus)
+{
+ struct sdw_cdns *cdns = bus_to_cdns(bus);
+ struct sdw_intel *sdw = cdns_to_intel(cdns);
+
+ return sdw->link_res->hw_ops->post_bank_switch(sdw);
+}
+
+static int sdw_master_read_intel_prop(struct sdw_bus *bus)
+{
+ struct sdw_master_prop *prop = &bus->prop;
+ struct fwnode_handle *link;
+ char name[32];
+ u32 quirk_mask;
+
+ /* Find master handle */
+ snprintf(name, sizeof(name),
+ "mipi-sdw-link-%d-subproperties", bus->link_id);
+
+ link = device_get_named_child_node(bus->dev, name);
+ if (!link) {
+ dev_err(bus->dev, "Master node %s not found\n", name);
+ return -EIO;
+ }
+
+ fwnode_property_read_u32(link,
+ "intel-sdw-ip-clock",
+ &prop->mclk_freq);
+
+ /* the values reported by BIOS are the 2x clock, not the bus clock */
+ prop->mclk_freq /= 2;
+
+ fwnode_property_read_u32(link,
+ "intel-quirk-mask",
+ &quirk_mask);
+
+ if (quirk_mask & SDW_INTEL_QUIRK_MASK_BUS_DISABLE)
+ prop->hw_disabled = true;
+
+ prop->quirks = SDW_MASTER_QUIRKS_CLEAR_INITIAL_CLASH |
+ SDW_MASTER_QUIRKS_CLEAR_INITIAL_PARITY;
+
+ return 0;
+}
+
+static int intel_prop_read(struct sdw_bus *bus)
+{
+ /* Initialize with default handler to read all DisCo properties */
+ sdw_master_read_prop(bus);
+
+ /* read Intel-specific properties */
+ sdw_master_read_intel_prop(bus);
+
+ return 0;
+}
+
+static struct sdw_master_ops sdw_intel_ops = {
+ .read_prop = intel_prop_read,
+ .override_adr = sdw_dmi_override_adr,
+ .xfer_msg = cdns_xfer_msg,
+ .xfer_msg_defer = cdns_xfer_msg_defer,
+ .reset_page_addr = cdns_reset_page_addr,
+ .set_bus_conf = cdns_bus_conf,
+ .pre_bank_switch = generic_pre_bank_switch,
+ .post_bank_switch = generic_post_bank_switch,
+ .read_ping_status = cdns_read_ping_status,
+};
+
+/*
+ * probe and init (aux_dev_id argument is required by function prototype but not used)
+ */
+static int intel_link_probe(struct auxiliary_device *auxdev,
+ const struct auxiliary_device_id *aux_dev_id)
+
+{
+ struct device *dev = &auxdev->dev;
+ struct sdw_intel_link_dev *ldev = auxiliary_dev_to_sdw_intel_link_dev(auxdev);
+ struct sdw_intel *sdw;
+ struct sdw_cdns *cdns;
+ struct sdw_bus *bus;
+ int ret;
+
+ sdw = devm_kzalloc(dev, sizeof(*sdw), GFP_KERNEL);
+ if (!sdw)
+ return -ENOMEM;
+
+ cdns = &sdw->cdns;
+ bus = &cdns->bus;
+
+ sdw->instance = auxdev->id;
+ sdw->link_res = &ldev->link_res;
+ cdns->dev = dev;
+ cdns->registers = sdw->link_res->registers;
+ cdns->instance = sdw->instance;
+ cdns->msg_count = 0;
+
+ bus->link_id = auxdev->id;
+ bus->dev_num_ida_min = INTEL_DEV_NUM_IDA_MIN;
+ bus->clk_stop_timeout = 1;
+
+ sdw_cdns_probe(cdns);
+
+ /* Set ops */
+ bus->ops = &sdw_intel_ops;
+
+ /* set driver data, accessed by snd_soc_dai_get_drvdata() */
+ auxiliary_set_drvdata(auxdev, cdns);
+
+ /* use generic bandwidth allocation algorithm */
+ sdw->cdns.bus.compute_params = sdw_compute_params;
+
+ /* avoid resuming from pm_runtime suspend if it's not required */
+ dev_pm_set_driver_flags(dev, DPM_FLAG_SMART_SUSPEND);
+
+ ret = sdw_bus_master_add(bus, dev, dev->fwnode);
+ if (ret) {
+ dev_err(dev, "sdw_bus_master_add fail: %d\n", ret);
+ return ret;
+ }
+
+ if (bus->prop.hw_disabled)
+ dev_info(dev,
+ "SoundWire master %d is disabled, will be ignored\n",
+ bus->link_id);
+ /*
+ * Ignore BIOS err_threshold, it's a really bad idea when dealing
+ * with multiple hardware synchronized links
+ */
+ bus->prop.err_threshold = 0;
+
+ return 0;
+}
+
+int intel_link_startup(struct auxiliary_device *auxdev)
+{
+ struct device *dev = &auxdev->dev;
+ struct sdw_cdns *cdns = auxiliary_get_drvdata(auxdev);
+ struct sdw_intel *sdw = cdns_to_intel(cdns);
+ struct sdw_bus *bus = &cdns->bus;
+ int link_flags;
+ bool multi_link;
+ u32 clock_stop_quirks;
+ int ret;
+
+ if (bus->prop.hw_disabled) {
+ dev_info(dev,
+ "SoundWire master %d is disabled, ignoring\n",
+ sdw->instance);
+ return 0;
+ }
+
+ link_flags = md_flags >> (bus->link_id * 8);
+ multi_link = !(link_flags & SDW_INTEL_MASTER_DISABLE_MULTI_LINK);
+ if (!multi_link) {
+ dev_dbg(dev, "Multi-link is disabled\n");
+ } else {
+ /*
+ * hardware-based synchronization is required regardless
+ * of the number of segments used by a stream: SSP-based
+ * synchronization is gated by gsync when the multi-master
+ * mode is set.
+ */
+ bus->hw_sync_min_links = 1;
+ }
+ bus->multi_link = multi_link;
+
+ /* Initialize shim, controller */
+ ret = sdw_intel_link_power_up(sdw);
+ if (ret)
+ goto err_init;
+
+ /* Register DAIs */
+ ret = sdw_intel_register_dai(sdw);
+ if (ret) {
+ dev_err(dev, "DAI registration failed: %d\n", ret);
+ goto err_power_up;
+ }
+
+ sdw_intel_debugfs_init(sdw);
+
+ /* start bus */
+ ret = sdw_intel_start_bus(sdw);
+ if (ret) {
+ dev_err(dev, "bus start failed: %d\n", ret);
+ goto err_power_up;
+ }
+
+ /* Enable runtime PM */
+ if (!(link_flags & SDW_INTEL_MASTER_DISABLE_PM_RUNTIME)) {
+ pm_runtime_set_autosuspend_delay(dev,
+ INTEL_MASTER_SUSPEND_DELAY_MS);
+ pm_runtime_use_autosuspend(dev);
+ pm_runtime_mark_last_busy(dev);
+
+ pm_runtime_set_active(dev);
+ pm_runtime_enable(dev);
+ }
+
+ clock_stop_quirks = sdw->link_res->clock_stop_quirks;
+ if (clock_stop_quirks & SDW_INTEL_CLK_STOP_NOT_ALLOWED) {
+ /*
+ * To keep the clock running we need to prevent
+ * pm_runtime suspend from happening by increasing the
+ * reference count.
+ * This quirk is specified by the parent PCI device in
+ * case of specific latency requirements. It will have
+ * no effect if pm_runtime is disabled by the user via
+ * a module parameter for testing purposes.
+ */
+ pm_runtime_get_noresume(dev);
+ }
+
+ /*
+ * The runtime PM status of Slave devices is "Unsupported"
+ * until they report as ATTACHED. If they don't, e.g. because
+ * there are no Slave devices populated or if the power-on is
+ * delayed or dependent on a power switch, the Master will
+ * remain active and prevent its parent from suspending.
+ *
+ * Conditionally force the pm_runtime core to re-evaluate the
+ * Master status in the absence of any Slave activity. A quirk
+ * is provided to e.g. deal with Slaves that may be powered on
+ * with a delay. A more complete solution would require the
+ * definition of Master properties.
+ */
+ if (!(link_flags & SDW_INTEL_MASTER_DISABLE_PM_RUNTIME_IDLE))
+ pm_runtime_idle(dev);
+
+ sdw->startup_done = true;
+ return 0;
+
+err_power_up:
+ sdw_intel_link_power_down(sdw);
+err_init:
+ return ret;
+}
+
+static void intel_link_remove(struct auxiliary_device *auxdev)
+{
+ struct sdw_cdns *cdns = auxiliary_get_drvdata(auxdev);
+ struct sdw_intel *sdw = cdns_to_intel(cdns);
+ struct sdw_bus *bus = &cdns->bus;
+
+ /*
+ * Since pm_runtime is already disabled, we don't decrease
+ * the refcount when the clock_stop_quirk is
+ * SDW_INTEL_CLK_STOP_NOT_ALLOWED
+ */
+ if (!bus->prop.hw_disabled) {
+ sdw_intel_debugfs_exit(sdw);
+ sdw_cdns_enable_interrupt(cdns, false);
+ }
+ sdw_bus_master_delete(bus);
+}
+
+int intel_link_process_wakeen_event(struct auxiliary_device *auxdev)
+{
+ struct device *dev = &auxdev->dev;
+ struct sdw_intel *sdw;
+ struct sdw_bus *bus;
+
+ sdw = auxiliary_get_drvdata(auxdev);
+ bus = &sdw->cdns.bus;
+
+ if (bus->prop.hw_disabled || !sdw->startup_done) {
+ dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
+ bus->link_id);
+ return 0;
+ }
+
+ if (!sdw_intel_shim_check_wake(sdw))
+ return 0;
+
+ /* disable WAKEEN interrupt ASAP to prevent interrupt flood */
+ sdw_intel_shim_wake(sdw, false);
+
+ /*
+ * resume the Master, which will generate a bus reset and result in
+ * Slaves re-attaching and be re-enumerated. The SoundWire physical
+ * device which generated the wake will trigger an interrupt, which
+ * will in turn cause the corresponding Linux Slave device to be
+ * resumed and the Slave codec driver to check the status.
+ */
+ pm_request_resume(dev);
+
+ return 0;
+}
+
+/*
+ * PM calls
+ */
+
+static int intel_resume_child_device(struct device *dev, void *data)
+{
+ int ret;
+ struct sdw_slave *slave = dev_to_sdw_dev(dev);
+
+ if (!slave->probed) {
+ dev_dbg(dev, "skipping device, no probed driver\n");
+ return 0;
+ }
+ if (!slave->dev_num_sticky) {
+ dev_dbg(dev, "skipping device, never detected on bus\n");
+ return 0;
+ }
+
+ ret = pm_request_resume(dev);
+ if (ret < 0)
+ dev_err(dev, "%s: pm_request_resume failed: %d\n", __func__, ret);
+
+ return ret;
+}
+
+static int __maybe_unused intel_pm_prepare(struct device *dev)
+{
+ struct sdw_cdns *cdns = dev_get_drvdata(dev);
+ struct sdw_intel *sdw = cdns_to_intel(cdns);
+ struct sdw_bus *bus = &cdns->bus;
+ u32 clock_stop_quirks;
+ int ret;
+
+ if (bus->prop.hw_disabled || !sdw->startup_done) {
+ dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
+ bus->link_id);
+ return 0;
+ }
+
+ clock_stop_quirks = sdw->link_res->clock_stop_quirks;
+
+ if (pm_runtime_suspended(dev) &&
+ pm_runtime_suspended(dev->parent) &&
+ ((clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET) ||
+ !clock_stop_quirks)) {
+ /*
+ * if we've enabled clock stop, and the parent is suspended, the SHIM registers
+ * are not accessible and the shim wake cannot be disabled.
+ * The only solution is to resume the entire bus to full power
+ */
+
+ /*
+ * If any operation in this block fails, we keep going since we don't want
+ * to prevent system suspend from happening and errors should be recoverable
+ * on resume.
+ */
+
+ /*
+ * first resume the device for this link. This will also by construction
+ * resume the PCI parent device.
+ */
+ ret = pm_request_resume(dev);
+ if (ret < 0) {
+ dev_err(dev, "%s: pm_request_resume failed: %d\n", __func__, ret);
+ return 0;
+ }
+
+ /*
+ * Continue resuming the entire bus (parent + child devices) to exit
+ * the clock stop mode. If there are no devices connected on this link
+ * this is a no-op.
+ * The resume to full power could have been implemented with a .prepare
+ * step in SoundWire codec drivers. This would however require a lot
+ * of code to handle an Intel-specific corner case. It is simpler in
+ * practice to add a loop at the link level.
+ */
+ ret = device_for_each_child(bus->dev, NULL, intel_resume_child_device);
+
+ if (ret < 0)
+ dev_err(dev, "%s: intel_resume_child_device failed: %d\n", __func__, ret);
+ }
+
+ return 0;
+}
+
+static int __maybe_unused intel_suspend(struct device *dev)
+{
+ struct sdw_cdns *cdns = dev_get_drvdata(dev);
+ struct sdw_intel *sdw = cdns_to_intel(cdns);
+ struct sdw_bus *bus = &cdns->bus;
+ u32 clock_stop_quirks;
+ int ret;
+
+ if (bus->prop.hw_disabled || !sdw->startup_done) {
+ dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
+ bus->link_id);
+ return 0;
+ }
+
+ if (pm_runtime_suspended(dev)) {
+ dev_dbg(dev, "pm_runtime status: suspended\n");
+
+ clock_stop_quirks = sdw->link_res->clock_stop_quirks;
+
+ if ((clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET) ||
+ !clock_stop_quirks) {
+
+ if (pm_runtime_suspended(dev->parent)) {
+ /*
+ * paranoia check: this should not happen with the .prepare
+ * resume to full power
+ */
+ dev_err(dev, "%s: invalid config: parent is suspended\n", __func__);
+ } else {
+ sdw_intel_shim_wake(sdw, false);
+ }
+ }
+
+ return 0;
+ }
+
+ ret = sdw_intel_stop_bus(sdw, false);
+ if (ret < 0) {
+ dev_err(dev, "%s: cannot stop bus: %d\n", __func__, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int __maybe_unused intel_suspend_runtime(struct device *dev)
+{
+ struct sdw_cdns *cdns = dev_get_drvdata(dev);
+ struct sdw_intel *sdw = cdns_to_intel(cdns);
+ struct sdw_bus *bus = &cdns->bus;
+ u32 clock_stop_quirks;
+ int ret;
+
+ if (bus->prop.hw_disabled || !sdw->startup_done) {
+ dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
+ bus->link_id);
+ return 0;
+ }
+
+ clock_stop_quirks = sdw->link_res->clock_stop_quirks;
+
+ if (clock_stop_quirks & SDW_INTEL_CLK_STOP_TEARDOWN) {
+ ret = sdw_intel_stop_bus(sdw, false);
+ if (ret < 0) {
+ dev_err(dev, "%s: cannot stop bus during teardown: %d\n",
+ __func__, ret);
+ return ret;
+ }
+ } else if (clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET || !clock_stop_quirks) {
+ ret = sdw_intel_stop_bus(sdw, true);
+ if (ret < 0) {
+ dev_err(dev, "%s: cannot stop bus during clock_stop: %d\n",
+ __func__, ret);
+ return ret;
+ }
+ } else {
+ dev_err(dev, "%s clock_stop_quirks %x unsupported\n",
+ __func__, clock_stop_quirks);
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int __maybe_unused intel_resume(struct device *dev)
+{
+ struct sdw_cdns *cdns = dev_get_drvdata(dev);
+ struct sdw_intel *sdw = cdns_to_intel(cdns);
+ struct sdw_bus *bus = &cdns->bus;
+ int link_flags;
+ int ret;
+
+ if (bus->prop.hw_disabled || !sdw->startup_done) {
+ dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
+ bus->link_id);
+ return 0;
+ }
+
+ link_flags = md_flags >> (bus->link_id * 8);
+
+ if (pm_runtime_suspended(dev)) {
+ dev_dbg(dev, "pm_runtime status was suspended, forcing active\n");
+
+ /* follow required sequence from runtime_pm.rst */
+ pm_runtime_disable(dev);
+ pm_runtime_set_active(dev);
+ pm_runtime_mark_last_busy(dev);
+ pm_runtime_enable(dev);
+
+ link_flags = md_flags >> (bus->link_id * 8);
+
+ if (!(link_flags & SDW_INTEL_MASTER_DISABLE_PM_RUNTIME_IDLE))
+ pm_runtime_idle(dev);
+ }
+
+ ret = sdw_intel_link_power_up(sdw);
+ if (ret) {
+ dev_err(dev, "%s failed: %d\n", __func__, ret);
+ return ret;
+ }
+
+ /*
+ * make sure all Slaves are tagged as UNATTACHED and provide
+ * reason for reinitialization
+ */
+ sdw_clear_slave_status(bus, SDW_UNATTACH_REQUEST_MASTER_RESET);
+
+ ret = sdw_intel_start_bus(sdw);
+ if (ret < 0) {
+ dev_err(dev, "cannot start bus during resume\n");
+ sdw_intel_link_power_down(sdw);
+ return ret;
+ }
+
+ /*
+ * after system resume, the pm_runtime suspend() may kick in
+ * during the enumeration, before any children device force the
+ * master device to remain active. Using pm_runtime_get()
+ * routines is not really possible, since it'd prevent the
+ * master from suspending.
+ * A reasonable compromise is to update the pm_runtime
+ * counters and delay the pm_runtime suspend by several
+ * seconds, by when all enumeration should be complete.
+ */
+ pm_runtime_mark_last_busy(dev);
+
+ return 0;
+}
+
+static int __maybe_unused intel_resume_runtime(struct device *dev)
+{
+ struct sdw_cdns *cdns = dev_get_drvdata(dev);
+ struct sdw_intel *sdw = cdns_to_intel(cdns);
+ struct sdw_bus *bus = &cdns->bus;
+ u32 clock_stop_quirks;
+ int ret;
+
+ if (bus->prop.hw_disabled || !sdw->startup_done) {
+ dev_dbg(dev, "SoundWire master %d is disabled or not-started, ignoring\n",
+ bus->link_id);
+ return 0;
+ }
+
+ /* unconditionally disable WAKEEN interrupt */
+ sdw_intel_shim_wake(sdw, false);
+
+ clock_stop_quirks = sdw->link_res->clock_stop_quirks;
+
+ if (clock_stop_quirks & SDW_INTEL_CLK_STOP_TEARDOWN) {
+ ret = sdw_intel_link_power_up(sdw);
+ if (ret) {
+ dev_err(dev, "%s: power_up failed after teardown: %d\n", __func__, ret);
+ return ret;
+ }
+
+ /*
+ * make sure all Slaves are tagged as UNATTACHED and provide
+ * reason for reinitialization
+ */
+ sdw_clear_slave_status(bus, SDW_UNATTACH_REQUEST_MASTER_RESET);
+
+ ret = sdw_intel_start_bus(sdw);
+ if (ret < 0) {
+ dev_err(dev, "%s: cannot start bus after teardown: %d\n", __func__, ret);
+ sdw_intel_link_power_down(sdw);
+ return ret;
+ }
+
+ } else if (clock_stop_quirks & SDW_INTEL_CLK_STOP_BUS_RESET) {
+ ret = sdw_intel_link_power_up(sdw);
+ if (ret) {
+ dev_err(dev, "%s: power_up failed after bus reset: %d\n", __func__, ret);
+ return ret;
+ }
+
+ ret = sdw_intel_start_bus_after_reset(sdw);
+ if (ret < 0) {
+ dev_err(dev, "%s: cannot start bus after reset: %d\n", __func__, ret);
+ sdw_intel_link_power_down(sdw);
+ return ret;
+ }
+ } else if (!clock_stop_quirks) {
+
+ sdw_intel_check_clock_stop(sdw);
+
+ ret = sdw_intel_link_power_up(sdw);
+ if (ret) {
+ dev_err(dev, "%s: power_up failed: %d\n", __func__, ret);
+ return ret;
+ }
+
+ ret = sdw_intel_start_bus_after_clock_stop(sdw);
+ if (ret < 0) {
+ dev_err(dev, "%s: cannot start bus after clock stop: %d\n", __func__, ret);
+ sdw_intel_link_power_down(sdw);
+ return ret;
+ }
+ } else {
+ dev_err(dev, "%s: clock_stop_quirks %x unsupported\n",
+ __func__, clock_stop_quirks);
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static const struct dev_pm_ops intel_pm = {
+ .prepare = intel_pm_prepare,
+ SET_SYSTEM_SLEEP_PM_OPS(intel_suspend, intel_resume)
+ SET_RUNTIME_PM_OPS(intel_suspend_runtime, intel_resume_runtime, NULL)
+};
+
+static const struct auxiliary_device_id intel_link_id_table[] = {
+ { .name = "soundwire_intel.link" },
+ {},
+};
+MODULE_DEVICE_TABLE(auxiliary, intel_link_id_table);
+
+static struct auxiliary_driver sdw_intel_drv = {
+ .probe = intel_link_probe,
+ .remove = intel_link_remove,
+ .driver = {
+ /* auxiliary_driver_register() sets .name to be the modname */
+ .pm = &intel_pm,
+ },
+ .id_table = intel_link_id_table
+};
+module_auxiliary_driver(sdw_intel_drv);
+
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_DESCRIPTION("Intel Soundwire Link Driver");
--- /dev/null
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/* Copyright(c) 2015-2022 Intel Corporation. */
+
+#ifndef __SDW_INTEL_AUXDEVICE_H
+#define __SDW_INTEL_AUXDEVICE_H
+
+int intel_link_startup(struct auxiliary_device *auxdev);
+int intel_link_process_wakeen_event(struct auxiliary_device *auxdev);
+
+struct sdw_intel_link_dev {
+ struct auxiliary_device auxdev;
+ struct sdw_intel_link_res link_res;
+};
+
+#define auxiliary_dev_to_sdw_intel_link_dev(auxiliary_dev) \
+ container_of(auxiliary_dev, struct sdw_intel_link_dev, auxdev)
+
+#endif /* __SDW_INTEL_AUXDEVICE_H */
#include <linux/soundwire/sdw_intel.h>
#include "cadence_master.h"
#include "intel.h"
+#include "intel_auxdevice.h"
static void intel_link_dev_release(struct device *dev)
{
/* Add link information used in the driver probe */
link = &ldev->link_res;
+ link->hw_ops = res->hw_ops;
link->mmio_base = res->mmio_base;
link->registers = res->mmio_base + SDW_LINK_BASE
+ (SDW_LINK_SIZE * link_id);
#define SWRM_COMP_SW_RESET 0x008
#define SWRM_COMP_STATUS 0x014
+#define SWRM_LINK_MANAGER_EE 0x018
+#define SWRM_EE_CPU 1
#define SWRM_FRM_GEN_ENABLED BIT(0)
#define SWRM_COMP_HW_VERSION 0x00
#define SWRM_COMP_CFG_ADDR 0x04
#define SWRM_REG_VAL_PACK(data, dev, id, reg) \
((reg) | ((id) << 16) | ((dev) << 20) | ((data) << 24))
-#define SWRM_SPECIAL_CMD_ID 0xF
#define MAX_FREQ_NUM 1
#define TIMEOUT_MS 100
#define QCOM_SWRM_MAX_RD_LEN 0x1
u32p_replace_bits(&val, SWRM_DEF_CMD_NO_PINGS, SWRM_MCP_CFG_MAX_NUM_OF_CMD_NO_PINGS_BMSK);
ctrl->reg_write(ctrl, SWRM_MCP_CFG_ADDR, val);
- ctrl->reg_write(ctrl, SWRM_MCP_BUS_CTRL, SWRM_MCP_BUS_CLK_START);
+ if (ctrl->version >= 0x01070000) {
+ ctrl->reg_write(ctrl, SWRM_LINK_MANAGER_EE, SWRM_EE_CPU);
+ ctrl->reg_write(ctrl, SWRM_MCP_BUS_CTRL,
+ SWRM_MCP_BUS_CLK_START << SWRM_EE_CPU);
+ } else {
+ ctrl->reg_write(ctrl, SWRM_MCP_BUS_CTRL, SWRM_MCP_BUS_CLK_START);
+ }
+
/* Configure number of retries of a read/write cmd */
if (ctrl->version > 0x01050001) {
/* Only for versions >= 1.5.1 */
}
if (data->sw_clk_gate_required) {
- ctrl->audio_cgcr = devm_reset_control_get_exclusive(dev, "swr_audio_cgcr");
- if (IS_ERR_OR_NULL(ctrl->audio_cgcr)) {
+ ctrl->audio_cgcr = devm_reset_control_get_optional_exclusive(dev, "swr_audio_cgcr");
+ if (IS_ERR(ctrl->audio_cgcr)) {
dev_err(dev, "Failed to get cgcr reset ctrl required for SW gating\n");
ret = PTR_ERR(ctrl->audio_cgcr);
goto err_init;
} else {
reset_control_reset(ctrl->audio_cgcr);
- ctrl->reg_write(ctrl, SWRM_MCP_BUS_CTRL, SWRM_MCP_BUS_CLK_START);
+ if (ctrl->version >= 0x01070000) {
+ ctrl->reg_write(ctrl, SWRM_LINK_MANAGER_EE, SWRM_EE_CPU);
+ ctrl->reg_write(ctrl, SWRM_MCP_BUS_CTRL,
+ SWRM_MCP_BUS_CLK_START << SWRM_EE_CPU);
+ } else {
+ ctrl->reg_write(ctrl, SWRM_MCP_BUS_CTRL, SWRM_MCP_BUS_CLK_START);
+ }
ctrl->reg_write(ctrl, SWRM_INTERRUPT_CLEAR,
SWRM_INTERRUPT_STATUS_MASTER_CLASH_DET);
{ .compatible = "qcom,soundwire-v1.3.0", .data = &swrm_v1_3_data },
{ .compatible = "qcom,soundwire-v1.5.1", .data = &swrm_v1_5_data },
{ .compatible = "qcom,soundwire-v1.6.0", .data = &swrm_v1_6_data },
+ { .compatible = "qcom,soundwire-v1.7.0", .data = &swrm_v1_5_data },
{/* sentinel */},
};
#define PMIC_ARB_VERSION_V2_MIN 0x20010000
#define PMIC_ARB_VERSION_V3_MIN 0x30000000
#define PMIC_ARB_VERSION_V5_MIN 0x50000000
+#define PMIC_ARB_VERSION_V7_MIN 0x70000000
#define PMIC_ARB_INT_EN 0x0004
+#define PMIC_ARB_FEATURES 0x0004
+#define PMIC_ARB_FEATURES_PERIPH_MASK GENMASK(10, 0)
+
+#define PMIC_ARB_FEATURES1 0x0008
+
/* PMIC Arbiter channel registers offsets */
#define PMIC_ARB_CMD 0x00
#define PMIC_ARB_CONFIG 0x04
#define INVALID_EE 0xFF
/* Ownership Table */
-#define SPMI_OWNERSHIP_TABLE_REG(N) (0x0700 + (4 * (N)))
#define SPMI_OWNERSHIP_PERIPH2OWNER(X) ((X) & 0x7)
/* Channel Status fields */
/* Maximum number of support PMIC peripherals */
#define PMIC_ARB_MAX_PERIPHS 512
+#define PMIC_ARB_MAX_PERIPHS_V7 1024
#define PMIC_ARB_TIMEOUT_US 1000
#define PMIC_ARB_MAX_TRANS_BYTES (8)
((((slave_id) & 0xF) << 28) | \
(((periph_id) & 0xFF) << 20) | \
(((irq_id) & 0x7) << 16) | \
- (((apid) & 0x1FF) << 0))
+ (((apid) & 0x3FF) << 0))
#define hwirq_to_sid(hwirq) (((hwirq) >> 28) & 0xF)
#define hwirq_to_per(hwirq) (((hwirq) >> 20) & 0xFF)
#define hwirq_to_irq(hwirq) (((hwirq) >> 16) & 0x7)
-#define hwirq_to_apid(hwirq) (((hwirq) >> 0) & 0x1FF)
+#define hwirq_to_apid(hwirq) (((hwirq) >> 0) & 0x3FF)
struct pmic_arb_ver_ops;
* @channel: execution environment channel to use for accesses.
* @irq: PMIC ARB interrupt.
* @ee: the current Execution Environment
+ * @bus_instance: on v7: 0 = primary SPMI bus, 1 = secondary SPMI bus
* @min_apid: minimum APID (used for bounding IRQ search)
* @max_apid: maximum APID
+ * @base_apid: on v7: minimum APID associated with the particular SPMI
+ * bus instance
+ * @apid_count: on v5 and v7: number of APIDs associated with the
+ * particular SPMI bus instance
* @mapping_table: in-memory copy of PPID -> APID mapping table.
* @domain: irq domain object for PMIC IRQ domain
* @spmic: SPMI controller object
* @ver_ops: version dependent operations.
- * @ppid_to_apid in-memory copy of PPID -> APID mapping table.
+ * @ppid_to_apid: in-memory copy of PPID -> APID mapping table.
+ * @last_apid: Highest value APID in use
+ * @apid_data: Table of data for all APIDs
+ * @max_periphs: Number of elements in apid_data[]
*/
struct spmi_pmic_arb {
void __iomem *rd_base;
u8 channel;
int irq;
u8 ee;
+ u32 bus_instance;
u16 min_apid;
u16 max_apid;
+ u16 base_apid;
+ int apid_count;
u32 *mapping_table;
DECLARE_BITMAP(mapping_table_valid, PMIC_ARB_MAX_PERIPHS);
struct irq_domain *domain;
const struct pmic_arb_ver_ops *ver_ops;
u16 *ppid_to_apid;
u16 last_apid;
- struct apid_data apid_data[PMIC_ARB_MAX_PERIPHS];
+ struct apid_data *apid_data;
+ int max_periphs;
};
/**
* @irq_clear: on v1 address of PMIC_ARB_SPMI_PIC_IRQ_CLEARn
* on v2 address of SPMI_PIC_IRQ_CLEARn.
* @apid_map_offset: offset of PMIC_ARB_REG_CHNLn
+ * @apid_owner: on v2 and later address of SPMI_PERIPHn_2OWNER_TABLE_REG
*/
struct pmic_arb_ver_ops {
const char *ver_str;
void __iomem *(*irq_status)(struct spmi_pmic_arb *pmic_arb, u16 n);
void __iomem *(*irq_clear)(struct spmi_pmic_arb *pmic_arb, u16 n);
u32 (*apid_map_offset)(u16 n);
+ void __iomem *(*apid_owner)(struct spmi_pmic_arb *pmic_arb, u16 n);
};
static inline void pmic_arb_base_write(struct spmi_pmic_arb *pmic_arb,
struct irq_chip *chip = irq_desc_get_chip(desc);
int first = pmic_arb->min_apid;
int last = pmic_arb->max_apid;
+ /*
+ * acc_offset will be non-zero for the secondary SPMI bus instance on
+ * v7 controllers.
+ */
+ int acc_offset = pmic_arb->base_apid >> 5;
u8 ee = pmic_arb->ee;
u32 status, enable, handled = 0;
int i, id, apid;
chained_irq_enter(chip, desc);
for (i = first >> 5; i <= last >> 5; ++i) {
- status = readl_relaxed(
- ver_ops->owner_acc_status(pmic_arb, ee, i));
+ status = readl_relaxed(ver_ops->owner_acc_status(pmic_arb, ee, i - acc_offset));
if (status)
acc_valid = true;
if (offset >= pmic_arb->core_size)
break;
- regval = readl_relaxed(pmic_arb->cnfg +
- SPMI_OWNERSHIP_TABLE_REG(apid));
+ regval = readl_relaxed(pmic_arb->ver_ops->apid_owner(pmic_arb,
+ apid));
apidd->irq_ee = SPMI_OWNERSHIP_PERIPH2OWNER(regval);
apidd->write_ee = apidd->irq_ee;
static int pmic_arb_read_apid_map_v5(struct spmi_pmic_arb *pmic_arb)
{
- struct apid_data *apidd = pmic_arb->apid_data;
+ struct apid_data *apidd;
struct apid_data *prev_apidd;
- u16 i, apid, ppid;
+ u16 i, apid, ppid, apid_max;
bool valid, is_irq_ee;
u32 regval, offset;
/*
* In order to allow multiple EEs to write to a single PPID in arbiter
- * version 5, there is more than one APID mapped to each PPID.
+ * version 5 and 7, there is more than one APID mapped to each PPID.
* The owner field for each of these mappings specifies the EE which is
* allowed to write to the APID. The owner of the last (highest) APID
* which has the IRQ owner bit set for a given PPID will receive
* interrupts from the PPID.
+ *
+ * In arbiter version 7, the APID numbering space is divided between
+ * the primary bus (0) and secondary bus (1) such that:
+ * APID = 0 to N-1 are assigned to the primary bus
+ * APID = N to N+M-1 are assigned to the secondary bus
+ * where N = number of APIDs supported by the primary bus and
+ * M = number of APIDs supported by the secondary bus
*/
- for (i = 0; ; i++, apidd++) {
+ apidd = &pmic_arb->apid_data[pmic_arb->base_apid];
+ apid_max = pmic_arb->base_apid + pmic_arb->apid_count;
+ for (i = pmic_arb->base_apid; i < apid_max; i++, apidd++) {
offset = pmic_arb->ver_ops->apid_map_offset(i);
if (offset >= pmic_arb->core_size)
break;
ppid = (regval >> 8) & PMIC_ARB_PPID_MASK;
is_irq_ee = PMIC_ARB_CHAN_IS_IRQ_OWNER(regval);
- regval = readl_relaxed(pmic_arb->cnfg +
- SPMI_OWNERSHIP_TABLE_REG(i));
+ regval = readl_relaxed(pmic_arb->ver_ops->apid_owner(pmic_arb,
+ i));
apidd->write_ee = SPMI_OWNERSHIP_PERIPH2OWNER(regval);
apidd->irq_ee = is_irq_ee ? apidd->write_ee : INVALID_EE;
return offset;
}
+/*
+ * v7 offset per ee and per apid for observer channels and per apid for
+ * read/write channels.
+ */
+static int pmic_arb_offset_v7(struct spmi_pmic_arb *pmic_arb, u8 sid, u16 addr,
+ enum pmic_arb_channel ch_type)
+{
+ u16 apid;
+ int rc;
+ u32 offset = 0;
+ u16 ppid = (sid << 8) | (addr >> 8);
+
+ rc = pmic_arb->ver_ops->ppid_to_apid(pmic_arb, ppid);
+ if (rc < 0)
+ return rc;
+
+ apid = rc;
+ switch (ch_type) {
+ case PMIC_ARB_CHANNEL_OBS:
+ offset = 0x8000 * pmic_arb->ee + 0x20 * apid;
+ break;
+ case PMIC_ARB_CHANNEL_RW:
+ if (pmic_arb->apid_data[apid].write_ee != pmic_arb->ee) {
+ dev_err(&pmic_arb->spmic->dev, "disallowed SPMI write to sid=%u, addr=0x%04X\n",
+ sid, addr);
+ return -EPERM;
+ }
+ offset = 0x1000 * apid;
+ break;
+ }
+
+ return offset;
+}
+
static u32 pmic_arb_fmt_cmd_v1(u8 opc, u8 sid, u16 addr, u8 bc)
{
return (opc << 27) | ((sid & 0xf) << 20) | (addr << 4) | (bc & 0x7);
}
static void __iomem *
+pmic_arb_owner_acc_status_v7(struct spmi_pmic_arb *pmic_arb, u8 m, u16 n)
+{
+ return pmic_arb->intr + 0x1000 * m + 0x4 * n;
+}
+
+static void __iomem *
pmic_arb_acc_enable_v1(struct spmi_pmic_arb *pmic_arb, u16 n)
{
return pmic_arb->intr + 0x200 + 0x4 * n;
}
static void __iomem *
+pmic_arb_acc_enable_v7(struct spmi_pmic_arb *pmic_arb, u16 n)
+{
+ return pmic_arb->wr_base + 0x100 + 0x1000 * n;
+}
+
+static void __iomem *
pmic_arb_irq_status_v1(struct spmi_pmic_arb *pmic_arb, u16 n)
{
return pmic_arb->intr + 0x600 + 0x4 * n;
}
static void __iomem *
+pmic_arb_irq_status_v7(struct spmi_pmic_arb *pmic_arb, u16 n)
+{
+ return pmic_arb->wr_base + 0x104 + 0x1000 * n;
+}
+
+static void __iomem *
pmic_arb_irq_clear_v1(struct spmi_pmic_arb *pmic_arb, u16 n)
{
return pmic_arb->intr + 0xA00 + 0x4 * n;
return pmic_arb->wr_base + 0x108 + 0x10000 * n;
}
+static void __iomem *
+pmic_arb_irq_clear_v7(struct spmi_pmic_arb *pmic_arb, u16 n)
+{
+ return pmic_arb->wr_base + 0x108 + 0x1000 * n;
+}
+
static u32 pmic_arb_apid_map_offset_v2(u16 n)
{
return 0x800 + 0x4 * n;
return 0x900 + 0x4 * n;
}
+static u32 pmic_arb_apid_map_offset_v7(u16 n)
+{
+ return 0x2000 + 0x4 * n;
+}
+
+static void __iomem *
+pmic_arb_apid_owner_v2(struct spmi_pmic_arb *pmic_arb, u16 n)
+{
+ return pmic_arb->cnfg + 0x700 + 0x4 * n;
+}
+
+/*
+ * For arbiter version 7, APID ownership table registers have independent
+ * numbering space for each SPMI bus instance, so each is indexed starting from
+ * 0.
+ */
+static void __iomem *
+pmic_arb_apid_owner_v7(struct spmi_pmic_arb *pmic_arb, u16 n)
+{
+ return pmic_arb->cnfg + 0x4 * (n - pmic_arb->base_apid);
+}
+
static const struct pmic_arb_ver_ops pmic_arb_v1 = {
.ver_str = "v1",
.ppid_to_apid = pmic_arb_ppid_to_apid_v1,
.irq_status = pmic_arb_irq_status_v1,
.irq_clear = pmic_arb_irq_clear_v1,
.apid_map_offset = pmic_arb_apid_map_offset_v2,
+ .apid_owner = pmic_arb_apid_owner_v2,
};
static const struct pmic_arb_ver_ops pmic_arb_v2 = {
.irq_status = pmic_arb_irq_status_v2,
.irq_clear = pmic_arb_irq_clear_v2,
.apid_map_offset = pmic_arb_apid_map_offset_v2,
+ .apid_owner = pmic_arb_apid_owner_v2,
};
static const struct pmic_arb_ver_ops pmic_arb_v3 = {
.irq_status = pmic_arb_irq_status_v2,
.irq_clear = pmic_arb_irq_clear_v2,
.apid_map_offset = pmic_arb_apid_map_offset_v2,
+ .apid_owner = pmic_arb_apid_owner_v2,
};
static const struct pmic_arb_ver_ops pmic_arb_v5 = {
.irq_status = pmic_arb_irq_status_v5,
.irq_clear = pmic_arb_irq_clear_v5,
.apid_map_offset = pmic_arb_apid_map_offset_v5,
+ .apid_owner = pmic_arb_apid_owner_v2,
+};
+
+static const struct pmic_arb_ver_ops pmic_arb_v7 = {
+ .ver_str = "v7",
+ .ppid_to_apid = pmic_arb_ppid_to_apid_v5,
+ .non_data_cmd = pmic_arb_non_data_cmd_v2,
+ .offset = pmic_arb_offset_v7,
+ .fmt_cmd = pmic_arb_fmt_cmd_v2,
+ .owner_acc_status = pmic_arb_owner_acc_status_v7,
+ .acc_enable = pmic_arb_acc_enable_v7,
+ .irq_status = pmic_arb_irq_status_v7,
+ .irq_clear = pmic_arb_irq_clear_v7,
+ .apid_map_offset = pmic_arb_apid_map_offset_v7,
+ .apid_owner = pmic_arb_apid_owner_v7,
};
static const struct irq_domain_ops pmic_arb_irq_domain_ops = {
pmic_arb = spmi_controller_get_drvdata(ctrl);
pmic_arb->spmic = ctrl;
+ /*
+ * Please don't replace this with devm_platform_ioremap_resource() or
+ * devm_ioremap_resource(). These both result in a call to
+ * devm_request_mem_region() which prevents multiple mappings of this
+ * register address range. SoCs with PMIC arbiter v7 may define two
+ * arbiter devices, for the two physical SPMI interfaces, which share
+ * some register address ranges (i.e. "core", "obsrvr", and "chnls").
+ * Ensure that both devices probe successfully by calling devm_ioremap()
+ * which does not result in a devm_request_mem_region() call.
+ */
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "core");
- core = devm_ioremap_resource(&ctrl->dev, res);
+ core = devm_ioremap(&ctrl->dev, res->start, resource_size(res));
if (IS_ERR(core)) {
err = PTR_ERR(core);
goto err_put_ctrl;
pmic_arb->ver_ops = &pmic_arb_v2;
else if (hw_ver < PMIC_ARB_VERSION_V5_MIN)
pmic_arb->ver_ops = &pmic_arb_v3;
- else
+ else if (hw_ver < PMIC_ARB_VERSION_V7_MIN)
pmic_arb->ver_ops = &pmic_arb_v5;
+ else
+ pmic_arb->ver_ops = &pmic_arb_v7;
res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
"obsrvr");
- pmic_arb->rd_base = devm_ioremap_resource(&ctrl->dev, res);
+ pmic_arb->rd_base = devm_ioremap(&ctrl->dev, res->start,
+ resource_size(res));
if (IS_ERR(pmic_arb->rd_base)) {
err = PTR_ERR(pmic_arb->rd_base);
goto err_put_ctrl;
res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
"chnls");
- pmic_arb->wr_base = devm_ioremap_resource(&ctrl->dev, res);
+ pmic_arb->wr_base = devm_ioremap(&ctrl->dev, res->start,
+ resource_size(res));
if (IS_ERR(pmic_arb->wr_base)) {
err = PTR_ERR(pmic_arb->wr_base);
goto err_put_ctrl;
}
}
+ pmic_arb->max_periphs = PMIC_ARB_MAX_PERIPHS;
+
+ if (hw_ver >= PMIC_ARB_VERSION_V7_MIN) {
+ pmic_arb->max_periphs = PMIC_ARB_MAX_PERIPHS_V7;
+ /* Optional property for v7: */
+ of_property_read_u32(pdev->dev.of_node, "qcom,bus-id",
+ &pmic_arb->bus_instance);
+ if (pmic_arb->bus_instance > 1) {
+ err = -EINVAL;
+ dev_err(&pdev->dev, "invalid bus instance (%u) specified\n",
+ pmic_arb->bus_instance);
+ goto err_put_ctrl;
+ }
+
+ if (pmic_arb->bus_instance == 0) {
+ pmic_arb->base_apid = 0;
+ pmic_arb->apid_count =
+ readl_relaxed(core + PMIC_ARB_FEATURES) &
+ PMIC_ARB_FEATURES_PERIPH_MASK;
+ } else {
+ pmic_arb->base_apid =
+ readl_relaxed(core + PMIC_ARB_FEATURES) &
+ PMIC_ARB_FEATURES_PERIPH_MASK;
+ pmic_arb->apid_count =
+ readl_relaxed(core + PMIC_ARB_FEATURES1) &
+ PMIC_ARB_FEATURES_PERIPH_MASK;
+ }
+
+ if (pmic_arb->base_apid + pmic_arb->apid_count > pmic_arb->max_periphs) {
+ err = -EINVAL;
+ dev_err(&pdev->dev, "Unsupported APID count %d detected\n",
+ pmic_arb->base_apid + pmic_arb->apid_count);
+ goto err_put_ctrl;
+ }
+ } else if (hw_ver >= PMIC_ARB_VERSION_V5_MIN) {
+ pmic_arb->base_apid = 0;
+ pmic_arb->apid_count = readl_relaxed(core + PMIC_ARB_FEATURES) &
+ PMIC_ARB_FEATURES_PERIPH_MASK;
+
+ if (pmic_arb->apid_count > pmic_arb->max_periphs) {
+ err = -EINVAL;
+ dev_err(&pdev->dev, "Unsupported APID count %d detected\n",
+ pmic_arb->apid_count);
+ goto err_put_ctrl;
+ }
+ }
+
+ pmic_arb->apid_data = devm_kcalloc(&ctrl->dev, pmic_arb->max_periphs,
+ sizeof(*pmic_arb->apid_data),
+ GFP_KERNEL);
+ if (!pmic_arb->apid_data) {
+ err = -ENOMEM;
+ goto err_put_ctrl;
+ }
+
dev_info(&ctrl->dev, "PMIC arbiter version %s (0x%x)\n",
pmic_arb->ver_ops->ver_str, hw_ver);
}
pmic_arb->ee = ee;
- mapping_table = devm_kcalloc(&ctrl->dev, PMIC_ARB_MAX_PERIPHS,
+ mapping_table = devm_kcalloc(&ctrl->dev, pmic_arb->max_periphs,
sizeof(*mapping_table), GFP_KERNEL);
if (!mapping_table) {
err = -ENOMEM;
/* Initialize max_apid/min_apid to the opposite bounds, during
* the irq domain translation, we are sure to update these */
pmic_arb->max_apid = 0;
- pmic_arb->min_apid = PMIC_ARB_MAX_PERIPHS - 1;
+ pmic_arb->min_apid = pmic_arb->max_periphs - 1;
platform_set_drvdata(pdev, ctrl);
raw_spin_lock_init(&pmic_arb->lock);
/* MAX Packet Size */
mpkt = _nbu2ss_readl(&preg->EP_REGS[num].EP_PCKT_ADRS) & EPN_MPKT;
- if ((DMA_MAX_COUNT * mpkt) < length)
- i_write_length = DMA_MAX_COUNT * mpkt;
- else
- i_write_length = length;
+ i_write_length = min(DMA_MAX_COUNT * mpkt, length);
/*------------------------------------------------------------*/
/* Number of transmission packets */
{
struct fieldbus_dev *fb = dev_get_drvdata(dev);
- return sprintf(buf, "%d\n", !!fb->online);
+ return sysfs_emit(buf, "%d\n", !!fb->online);
}
static DEVICE_ATTR_RO(online);
if (!fb->enable_get)
return -EINVAL;
- return sprintf(buf, "%d\n", !!fb->enable_get(fb));
+ return sysfs_emit(buf, "%d\n", !!fb->enable_get(fb));
}
static ssize_t enabled_store(struct device *dev, struct device_attribute *attr,
{
struct fieldbus_dev *fb = dev_get_drvdata(dev);
- /*
- * card_name was provided by child driver, could potentially be long.
- * protect against buffer overrun.
- */
- return snprintf(buf, PAGE_SIZE, "%s\n", fb->card_name);
+ /* card_name was provided by child driver. */
+ return sysfs_emit(buf, "%s\n", fb->card_name);
}
static DEVICE_ATTR_RO(card_name);
{
struct fieldbus_dev *fb = dev_get_drvdata(dev);
- return sprintf(buf, "%zu\n", fb->read_area_sz);
+ return sysfs_emit(buf, "%zu\n", fb->read_area_sz);
}
static DEVICE_ATTR_RO(read_area_size);
{
struct fieldbus_dev *fb = dev_get_drvdata(dev);
- return sprintf(buf, "%zu\n", fb->write_area_sz);
+ return sysfs_emit(buf, "%zu\n", fb->write_area_sz);
}
static DEVICE_ATTR_RO(write_area_size);
break;
}
- return sprintf(buf, "%s\n", t);
+ return sysfs_emit(buf, "%s\n", t);
}
static DEVICE_ATTR_RO(fieldbus_type);
#define MUX_TX_MAX_SIZE 2048
-#define GDM_TTY_READY(gdm) (gdm && gdm->tty_dev && gdm->port.count)
+static inline bool gdm_tty_ready(struct gdm *gdm)
+{
+ return gdm && gdm->tty_dev && gdm->port.count;
+}
static struct tty_driver *gdm_driver[TTY_MAX_COUNT];
static struct gdm *gdm_table[TTY_MAX_COUNT][GDM_TTY_MINOR];
{
struct gdm *gdm = tty_dev->gdm[index];
- if (!GDM_TTY_READY(gdm)) {
+ if (!gdm_tty_ready(gdm)) {
if (complete == RECV_PACKET_PROCESS_COMPLETE)
gdm->tty_dev->recv_func(gdm->tty_dev->priv_dev,
gdm_tty_recv_complete);
{
struct gdm *gdm = arg;
- if (!GDM_TTY_READY(gdm))
+ if (!gdm_tty_ready(gdm))
return;
tty_port_tty_wakeup(&gdm->port);
int sent_len = 0;
int sending_len = 0;
- if (!GDM_TTY_READY(gdm))
+ if (!gdm_tty_ready(gdm))
return -ENODEV;
if (!len)
{
struct gdm *gdm = tty->driver_data;
- if (!GDM_TTY_READY(gdm))
+ if (!gdm_tty_ready(gdm))
return 0;
return WRITE_SIZE;
for (i = 0; i < t->device_count; i++)
printf("device[%d] = %s\n", i, t->devices[i].name);
-
}
int open_sysfs(const char *sys_pfx, const char *node, int flags)
char buf[SYSFS_MAX_INT];
if (read(fd, buf, sizeof(buf)) < 0) {
-
fprintf(stderr, "unable to read from %s%s %s\n", sys_pfx, node,
strerror(errno));
close(fd);
r->apbridge_unipro_latency_max - r->apbridge_unipro_latency_min;
r->gbphy_firmware_latency_jitter =
r->gbphy_firmware_latency_max - r->gbphy_firmware_latency_min;
-
}
/*calculate the aggregate results of all enabled devices */
r->apbridge_unipro_latency_max - r->apbridge_unipro_latency_min;
r->gbphy_firmware_latency_jitter =
r->gbphy_firmware_latency_max - r->gbphy_firmware_latency_min;
-
}
return 0;
fprintf(stderr, "unable to open %s for appending\n", file_name);
abort();
}
-
}
for (i = 0; i < t->device_count; i++) {
if (!device_enabled(t, i))
if (ret == -1)
fprintf(stderr, "unable to write %d bytes to csv.\n", len);
}
-
}
-
if (t->aggregate_output) {
len = format_output(t, &t->aggregate_results, "aggregate",
data, sizeof(data), &tm);
static int close_poll_files(struct loopback_test *t)
{
int i;
+
for (i = 0; i < t->poll_count; i++)
close(t->fds[i].fd);
ts = &t->poll_timeout;
while (1) {
-
ret = ppoll(t->fds, t->poll_count, ts, &mask_old);
if (ret <= 0) {
stop_tests(t);
if (t->stop_all || device_enabled(t, i))
write_sysfs_val(t->devices[i].sysfs_entry, "type", 0);
-
for (i = 0; i < t->device_count; i++) {
if (!device_enabled(t, i))
continue;
return 0;
}
-
void loopback_run(struct loopback_test *t)
{
int i;
if (ret)
goto err;
-
get_results(t);
log_results(t);
err:
printf("Error running test\n");
- return;
}
static int sanity_check(struct loopback_test *t)
fprintf(stderr, "Bad device mask %x\n", (1 << i));
return -1;
}
-
}
-
return 0;
}
return ret;
/* Get the device into a sane initial state */
- ret = adis_initial_startup(st);
+ ret = __adis_initial_startup(st);
if (ret)
return ret;
return ret;
/* Get the device into a sane initial state */
- ret = adis_initial_startup(st);
+ ret = __adis_initial_startup(st);
if (ret)
return ret;
* device probe and remove
*/
-static int adt7316_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int adt7316_i2c_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct adt7316_bus bus = {
.client = client,
.irq = client->irq,
.of_match_table = adt7316_of_match,
.pm = ADT7316_PM_OPS,
},
- .probe = adt7316_i2c_probe,
+ .probe_new = adt7316_i2c_probe,
.id_table = adt7316_i2c_id,
};
module_i2c_driver(adt7316_driver);
static IIO_DEV_ATTR_PHASESYMBOL(0, 0200, NULL, ad9834_write, AD9834_PSEL);
static IIO_CONST_ATTR_PHASE_SCALE(0, "0.0015339808"); /* 2PI/2^12 rad*/
-static IIO_DEV_ATTR_PINCONTROL_EN(0, 0200, NULL,
- ad9834_write, AD9834_PIN_SW);
+static IIO_DEV_ATTR_PINCONTROL_EN(0, 0200, NULL, ad9834_write, AD9834_PIN_SW);
static IIO_DEV_ATTR_OUT_ENABLE(0, 0200, NULL, ad9834_write, AD9834_RESET);
-static IIO_DEV_ATTR_OUTY_ENABLE(0, 1, 0200, NULL,
- ad9834_write, AD9834_OPBITEN);
+static IIO_DEV_ATTR_OUTY_ENABLE(0, 1, 0200, NULL, ad9834_write, AD9834_OPBITEN);
static IIO_DEV_ATTR_OUT_WAVETYPE(0, 0, ad9834_store_wavetype, 0);
static IIO_DEV_ATTR_OUT_WAVETYPE(0, 1, ad9834_store_wavetype, 1);
clk_disable_unprepare(st->mclk);
}
-static int ad5933_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ad5933_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
int ret;
struct ad5933_state *st;
struct iio_dev *indio_dev;
.name = "ad5933",
.of_match_table = ad5933_of_match,
},
- .probe = ad5933_probe,
+ .probe_new = ad5933_probe,
.id_table = ad5933_id,
};
module_i2c_driver(ad5933_driver);
unlock:
mutex_unlock(&st->buf_lock);
- return ret < 0 ? ret : 0;
+ if (ret < 0)
+ return ret;
+
+ return 0;
}
static int ade7854_i2c_read_reg(struct device *dev,
return ret;
}
-static int ade7854_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ade7854_i2c_probe(struct i2c_client *client)
{
struct ade7854_state *st;
struct iio_dev *indio_dev;
.driver = {
.name = "ade7854",
},
- .probe = ade7854_i2c_probe,
+ .probe_new = ade7854_i2c_probe,
.id_table = ade7854_id,
};
module_i2c_driver(ade7854_i2c_driver);
- fix the 'card removal' event when card is inserted when booting
- check what other upstream wireless mechanisms can be used instead of the
custom ones here
+- Switch to use LIB80211.
+- Switch to use MAC80211.
+- Switch to use CFG80211.
Please send any patches to:
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
}
static int ks_wlan_set_stop_request(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
return -EPERM;
/* for SLEEP MODE */
- if (!(*uwrq))
+ if (!(uwrq->mode))
return -EINVAL;
hostif_sme_enqueue(priv, SME_STOP_REQUEST);
{
struct ks_wlan_private *priv = netdev_priv(dev);
struct iw_mlme *mlme = (struct iw_mlme *)extra;
- __u32 mode = 1;
+ union iwreq_data uwrq;
+
+ uwrq.mode = 1;
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
mlme->reason_code == WLAN_REASON_MIC_FAILURE)
return 0;
- return ks_wlan_set_stop_request(dev, NULL, &mode, NULL);
+ return ks_wlan_set_stop_request(dev, NULL, &uwrq, NULL);
}
static int ks_wlan_get_firmware_version(struct net_device *dev,
struct iw_request_info *info,
- struct iw_point *dwrq, char *extra)
+ union iwreq_data *uwrq, char *extra)
{
+ struct iw_point *dwrq = &uwrq->data;
struct ks_wlan_private *priv = netdev_priv(dev);
dwrq->length = priv->version_size + 1;
}
static int ks_wlan_set_preamble(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
return -EPERM;
/* for SLEEP MODE */
- if (*uwrq != LONG_PREAMBLE && *uwrq != SHORT_PREAMBLE)
+ if (uwrq->mode != LONG_PREAMBLE && uwrq->mode != SHORT_PREAMBLE)
return -EINVAL;
- priv->reg.preamble = *uwrq;
+ priv->reg.preamble = uwrq->mode;
priv->need_commit |= SME_MODE_SET;
return -EINPROGRESS; /* Call commit handler */
}
static int ks_wlan_get_preamble(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
return -EPERM;
/* for SLEEP MODE */
- *uwrq = priv->reg.preamble;
+ uwrq->mode = priv->reg.preamble;
return 0;
}
static int ks_wlan_set_power_mgmt(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
- if (*uwrq != POWER_MGMT_ACTIVE &&
- *uwrq != POWER_MGMT_SAVE1 &&
- *uwrq != POWER_MGMT_SAVE2)
+ if (uwrq->mode != POWER_MGMT_ACTIVE &&
+ uwrq->mode != POWER_MGMT_SAVE1 &&
+ uwrq->mode != POWER_MGMT_SAVE2)
return -EINVAL;
- if ((*uwrq == POWER_MGMT_SAVE1 || *uwrq == POWER_MGMT_SAVE2) &&
+ if ((uwrq->mode == POWER_MGMT_SAVE1 || uwrq->mode == POWER_MGMT_SAVE2) &&
(priv->reg.operation_mode != MODE_INFRASTRUCTURE))
return -EINVAL;
- priv->reg.power_mgmt = *uwrq;
+ priv->reg.power_mgmt = uwrq->mode;
hostif_sme_enqueue(priv, SME_POW_MNGMT_REQUEST);
return 0;
}
static int ks_wlan_get_power_mgmt(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
return -EPERM;
/* for SLEEP MODE */
- *uwrq = priv->reg.power_mgmt;
+ uwrq->mode = priv->reg.power_mgmt;
return 0;
}
static int ks_wlan_set_scan_type(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
return -EPERM;
/* for SLEEP MODE */
- if (*uwrq != ACTIVE_SCAN && *uwrq != PASSIVE_SCAN)
+ if (uwrq->mode != ACTIVE_SCAN && uwrq->mode != PASSIVE_SCAN)
return -EINVAL;
- priv->reg.scan_type = *uwrq;
+ priv->reg.scan_type = uwrq->mode;
return 0;
}
static int ks_wlan_get_scan_type(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- *uwrq = priv->reg.scan_type;
+ uwrq->mode = priv->reg.scan_type;
return 0;
}
static int ks_wlan_set_beacon_lost(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- if (*uwrq > BEACON_LOST_COUNT_MAX)
+ if (uwrq->mode > BEACON_LOST_COUNT_MAX)
return -EINVAL;
- priv->reg.beacon_lost_count = *uwrq;
+ priv->reg.beacon_lost_count = uwrq->mode;
if (priv->reg.operation_mode == MODE_INFRASTRUCTURE) {
priv->need_commit |= SME_MODE_SET;
}
static int ks_wlan_get_beacon_lost(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- *uwrq = priv->reg.beacon_lost_count;
+ uwrq->mode = priv->reg.beacon_lost_count;
return 0;
}
static int ks_wlan_set_phy_type(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
- if (*uwrq != D_11B_ONLY_MODE &&
- *uwrq != D_11G_ONLY_MODE &&
- *uwrq != D_11BG_COMPATIBLE_MODE)
+ if (uwrq->mode != D_11B_ONLY_MODE &&
+ uwrq->mode != D_11G_ONLY_MODE &&
+ uwrq->mode != D_11BG_COMPATIBLE_MODE)
return -EINVAL;
/* for SLEEP MODE */
- priv->reg.phy_type = *uwrq;
+ priv->reg.phy_type = uwrq->mode;
priv->need_commit |= SME_MODE_SET;
return -EINPROGRESS; /* Call commit handler */
}
static int ks_wlan_get_phy_type(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- *uwrq = priv->reg.phy_type;
+ uwrq->mode = priv->reg.phy_type;
return 0;
}
static int ks_wlan_set_cts_mode(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- if (*uwrq != CTS_MODE_FALSE && *uwrq != CTS_MODE_TRUE)
+ if (uwrq->mode != CTS_MODE_FALSE && uwrq->mode != CTS_MODE_TRUE)
return -EINVAL;
- priv->reg.cts_mode = (*uwrq == CTS_MODE_FALSE) ? *uwrq :
+ priv->reg.cts_mode = (uwrq->mode == CTS_MODE_FALSE) ? uwrq->mode :
(priv->reg.phy_type == D_11G_ONLY_MODE ||
priv->reg.phy_type == D_11BG_COMPATIBLE_MODE) ?
- *uwrq : !*uwrq;
+ uwrq->mode : !uwrq->mode;
priv->need_commit |= SME_MODE_SET;
return -EINPROGRESS; /* Call commit handler */
}
static int ks_wlan_get_cts_mode(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- *uwrq = priv->reg.cts_mode;
+ uwrq->mode = priv->reg.cts_mode;
return 0;
}
static int ks_wlan_set_sleep_mode(struct net_device *dev,
struct iw_request_info *info,
- __u32 *uwrq, char *extra)
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
- if (*uwrq != SLP_SLEEP &&
- *uwrq != SLP_ACTIVE) {
- netdev_err(dev, "SET_SLEEP_MODE %d error\n", *uwrq);
+ if (uwrq->mode != SLP_SLEEP &&
+ uwrq->mode != SLP_ACTIVE) {
+ netdev_err(dev, "SET_SLEEP_MODE %d error\n", uwrq->mode);
return -EINVAL;
}
- priv->sleep_mode = *uwrq;
+ priv->sleep_mode = uwrq->mode;
netdev_info(dev, "SET_SLEEP_MODE %d\n", priv->sleep_mode);
- if (*uwrq == SLP_SLEEP)
+ if (uwrq->mode == SLP_SLEEP)
hostif_sme_enqueue(priv, SME_STOP_REQUEST);
hostif_sme_enqueue(priv, SME_SLEEP_REQUEST);
static int ks_wlan_get_sleep_mode(struct net_device *dev,
struct iw_request_info *info,
- __u32 *uwrq, char *extra)
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
- *uwrq = priv->sleep_mode;
+ uwrq->mode = priv->sleep_mode;
return 0;
}
static int ks_wlan_set_wps_enable(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- if (*uwrq != 0 && *uwrq != 1)
+ if (uwrq->mode != 0 && uwrq->mode != 1)
return -EINVAL;
- priv->wps.wps_enabled = *uwrq;
+ priv->wps.wps_enabled = uwrq->mode;
hostif_sme_enqueue(priv, SME_WPS_ENABLE_REQUEST);
return 0;
}
static int ks_wlan_get_wps_enable(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- *uwrq = priv->wps.wps_enabled;
- netdev_info(dev, "return=%d\n", *uwrq);
+ uwrq->mode = priv->wps.wps_enabled;
+ netdev_info(dev, "return=%d\n", uwrq->mode);
return 0;
}
static int ks_wlan_set_wps_probe_req(struct net_device *dev,
struct iw_request_info *info,
- struct iw_point *dwrq, char *extra)
+ union iwreq_data *uwrq, char *extra)
{
+ struct iw_point *dwrq = &uwrq->data;
u8 *p = extra;
unsigned char len;
struct ks_wlan_private *priv = netdev_priv(dev);
}
static int ks_wlan_set_tx_gain(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- if (*uwrq > 0xFF)
+ if (uwrq->mode > 0xFF)
return -EINVAL;
- priv->gain.tx_gain = (u8)*uwrq;
+ priv->gain.tx_gain = (u8)uwrq->mode;
priv->gain.tx_mode = (priv->gain.tx_gain < 0xFF) ? 1 : 0;
hostif_sme_enqueue(priv, SME_SET_GAIN);
return 0;
}
static int ks_wlan_get_tx_gain(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- *uwrq = priv->gain.tx_gain;
+ uwrq->mode = priv->gain.tx_gain;
hostif_sme_enqueue(priv, SME_GET_GAIN);
return 0;
}
static int ks_wlan_set_rx_gain(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- if (*uwrq > 0xFF)
+ if (uwrq->mode > 0xFF)
return -EINVAL;
- priv->gain.rx_gain = (u8)*uwrq;
+ priv->gain.rx_gain = (u8)uwrq->mode;
priv->gain.rx_mode = (priv->gain.rx_gain < 0xFF) ? 1 : 0;
hostif_sme_enqueue(priv, SME_SET_GAIN);
return 0;
}
static int ks_wlan_get_rx_gain(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
if (priv->sleep_mode == SLP_SLEEP)
return -EPERM;
/* for SLEEP MODE */
- *uwrq = priv->gain.rx_gain;
+ uwrq->mode = priv->gain.rx_gain;
hostif_sme_enqueue(priv, SME_GET_GAIN);
return 0;
}
static int ks_wlan_get_eeprom_cksum(struct net_device *dev,
- struct iw_request_info *info, __u32 *uwrq,
- char *extra)
+ struct iw_request_info *info,
+ union iwreq_data *uwrq, char *extra)
{
struct ks_wlan_private *priv = netdev_priv(dev);
- *uwrq = priv->eeprom_checksum;
+ uwrq->mode = priv->eeprom_checksum;
return 0;
}
/* get host command history */
static int ks_wlan_hostt(struct net_device *dev, struct iw_request_info *info,
- __u32 *uwrq, char *extra)
+ union iwreq_data *uwrq, char *extra)
{
int i, event;
struct ks_wlan_private *priv = netdev_priv(dev);
/* private_handler */
static const iw_handler ks_wlan_private_handler[] = {
- (iw_handler)NULL, /* 0 */
- (iw_handler)NULL, /* 1, KS_WLAN_GET_DRIVER_VERSION */
- (iw_handler)NULL, /* 2 */
- (iw_handler)ks_wlan_get_firmware_version,/* 3 KS_WLAN_GET_FIRM_VERSION */
- (iw_handler)ks_wlan_set_wps_enable, /* 4 KS_WLAN_SET_WPS_ENABLE */
- (iw_handler)ks_wlan_get_wps_enable, /* 5 KS_WLAN_GET_WPS_ENABLE */
- (iw_handler)ks_wlan_set_wps_probe_req, /* 6 KS_WLAN_SET_WPS_PROBE_REQ */
- (iw_handler)ks_wlan_get_eeprom_cksum, /* 7 KS_WLAN_GET_CONNECT */
- (iw_handler)ks_wlan_set_preamble, /* 8 KS_WLAN_SET_PREAMBLE */
- (iw_handler)ks_wlan_get_preamble, /* 9 KS_WLAN_GET_PREAMBLE */
- (iw_handler)ks_wlan_set_power_mgmt, /* 10 KS_WLAN_SET_POWER_SAVE */
- (iw_handler)ks_wlan_get_power_mgmt, /* 11 KS_WLAN_GET_POWER_SAVE */
- (iw_handler)ks_wlan_set_scan_type, /* 12 KS_WLAN_SET_SCAN_TYPE */
- (iw_handler)ks_wlan_get_scan_type, /* 13 KS_WLAN_GET_SCAN_TYPE */
- (iw_handler)ks_wlan_set_rx_gain, /* 14 KS_WLAN_SET_RX_GAIN */
- (iw_handler)ks_wlan_get_rx_gain, /* 15 KS_WLAN_GET_RX_GAIN */
- (iw_handler)ks_wlan_hostt, /* 16 KS_WLAN_HOSTT */
- (iw_handler)NULL, /* 17 */
- (iw_handler)ks_wlan_set_beacon_lost, /* 18 KS_WLAN_SET_BECAN_LOST */
- (iw_handler)ks_wlan_get_beacon_lost, /* 19 KS_WLAN_GET_BECAN_LOST */
- (iw_handler)ks_wlan_set_tx_gain, /* 20 KS_WLAN_SET_TX_GAIN */
- (iw_handler)ks_wlan_get_tx_gain, /* 21 KS_WLAN_GET_TX_GAIN */
- (iw_handler)ks_wlan_set_phy_type, /* 22 KS_WLAN_SET_PHY_TYPE */
- (iw_handler)ks_wlan_get_phy_type, /* 23 KS_WLAN_GET_PHY_TYPE */
- (iw_handler)ks_wlan_set_cts_mode, /* 24 KS_WLAN_SET_CTS_MODE */
- (iw_handler)ks_wlan_get_cts_mode, /* 25 KS_WLAN_GET_CTS_MODE */
- (iw_handler)NULL, /* 26 */
- (iw_handler)NULL, /* 27 */
- (iw_handler)ks_wlan_set_sleep_mode, /* 28 KS_WLAN_SET_SLEEP_MODE */
- (iw_handler)ks_wlan_get_sleep_mode, /* 29 KS_WLAN_GET_SLEEP_MODE */
- (iw_handler)NULL, /* 30 */
- (iw_handler)NULL, /* 31 */
+ NULL, /* 0 */
+ NULL, /* 1, KS_WLAN_GET_DRIVER_VERSION */
+ NULL, /* 2 */
+ ks_wlan_get_firmware_version, /* 3 KS_WLAN_GET_FIRM_VERSION */
+ ks_wlan_set_wps_enable, /* 4 KS_WLAN_SET_WPS_ENABLE */
+ ks_wlan_get_wps_enable, /* 5 KS_WLAN_GET_WPS_ENABLE */
+ ks_wlan_set_wps_probe_req, /* 6 KS_WLAN_SET_WPS_PROBE_REQ */
+ ks_wlan_get_eeprom_cksum, /* 7 KS_WLAN_GET_CONNECT */
+ ks_wlan_set_preamble, /* 8 KS_WLAN_SET_PREAMBLE */
+ ks_wlan_get_preamble, /* 9 KS_WLAN_GET_PREAMBLE */
+ ks_wlan_set_power_mgmt, /* 10 KS_WLAN_SET_POWER_SAVE */
+ ks_wlan_get_power_mgmt, /* 11 KS_WLAN_GET_POWER_SAVE */
+ ks_wlan_set_scan_type, /* 12 KS_WLAN_SET_SCAN_TYPE */
+ ks_wlan_get_scan_type, /* 13 KS_WLAN_GET_SCAN_TYPE */
+ ks_wlan_set_rx_gain, /* 14 KS_WLAN_SET_RX_GAIN */
+ ks_wlan_get_rx_gain, /* 15 KS_WLAN_GET_RX_GAIN */
+ ks_wlan_hostt, /* 16 KS_WLAN_HOSTT */
+ NULL, /* 17 */
+ ks_wlan_set_beacon_lost, /* 18 KS_WLAN_SET_BECAN_LOST */
+ ks_wlan_get_beacon_lost, /* 19 KS_WLAN_GET_BECAN_LOST */
+ ks_wlan_set_tx_gain, /* 20 KS_WLAN_SET_TX_GAIN */
+ ks_wlan_get_tx_gain, /* 21 KS_WLAN_GET_TX_GAIN */
+ ks_wlan_set_phy_type, /* 22 KS_WLAN_SET_PHY_TYPE */
+ ks_wlan_get_phy_type, /* 23 KS_WLAN_GET_PHY_TYPE */
+ ks_wlan_set_cts_mode, /* 24 KS_WLAN_SET_CTS_MODE */
+ ks_wlan_get_cts_mode, /* 25 KS_WLAN_GET_CTS_MODE */
+ NULL, /* 26 */
+ NULL, /* 27 */
+ ks_wlan_set_sleep_mode, /* 28 KS_WLAN_SET_SLEEP_MODE */
+ ks_wlan_get_sleep_mode, /* 29 KS_WLAN_GET_SLEEP_MODE */
+ NULL, /* 30 */
+ NULL, /* 31 */
};
static const struct iw_handler_def ks_wlan_handler_def = {
switch (cmd) {
case SIOCIWFIRSTPRIV + 20: /* KS_WLAN_SET_STOP_REQ */
- ret = ks_wlan_set_stop_request(dev, NULL, &wrq->u.mode, NULL);
+ ret = ks_wlan_set_stop_request(dev, NULL, &wrq->u, NULL);
break;
// All other calls are currently unsupported
default:
void sun6i_isp_capture_configure(struct sun6i_isp_device *isp_dev)
{
unsigned int width, height;
- unsigned int stride_luma, stride_chroma = 0;
- unsigned int stride_luma_div4, stride_chroma_div4;
+ unsigned int stride_luma, stride_chroma;
+ unsigned int stride_luma_div4, stride_chroma_div4 = 0;
const struct sun6i_isp_capture_format *format;
const struct v4l2_format_info *info;
u32 pixelformat;
if (state->configured)
goto complete;
- sun6i_isp_params_configure_modules(isp_dev,
- &sun6i_isp_params_config_default);
+ sun6i_isp_params_configure_modules(isp_dev,
+ &sun6i_isp_params_config_default);
state->configured = true;
vb2_buffer = &state->pending->v4l2_buffer.vb2_buf;
vb2_buffer_done(vb2_buffer, error ? VB2_BUF_STATE_ERROR :
VB2_BUF_STATE_QUEUED);
+
+ state->pending = NULL;
}
list_for_each_entry(isp_buffer, &state->queue, list) {
struct sun6i_isp_proc_source *source;
struct v4l2_subdev *source_subdev;
struct media_pad *remote_pad;
- /* Initialize to 0 to use both in disable label (ret != 0) and off. */
- int ret = 0;
+ int ret;
/* Source */
if (!on) {
sun6i_isp_proc_irq_disable(isp_dev);
v4l2_subdev_call(source_subdev, video, s_stream, 0);
+ ret = 0;
goto disable;
}
.set_fmt = sun6i_isp_proc_set_fmt,
};
-const struct v4l2_subdev_ops sun6i_isp_proc_subdev_ops = {
+static const struct v4l2_subdev_ops sun6i_isp_proc_subdev_ops = {
.video = &sun6i_isp_proc_video_ops,
.pad = &sun6i_isp_proc_pad_ops,
};
enabled = !proc->source_csi0.expected;
break;
default:
- break;
+ return -EINVAL;
}
source->subdev = remote_subdev;
struct list_head *head = &hdm_ch->pending_list;
struct mbo *mbo;
unsigned long flags;
- struct dim_ch_state_t st;
+ struct dim_ch_state st;
BUG_ON(!hdm_ch);
BUG_ON(!hdm_ch->is_initialized);
static void service_done_flag(struct dim2_hdm *dev, int ch_idx)
{
struct hdm_channel *hdm_ch = dev->hch + ch_idx;
- struct dim_ch_state_t st;
+ struct dim_ch_state st;
struct list_head *head;
struct mbo *mbo;
int done_buffers;
return channel_service(ch);
}
-struct dim_ch_state_t *dim_get_channel_state(struct dim_channel *ch,
- struct dim_ch_state_t *state_ptr)
+struct dim_ch_state *dim_get_channel_state(struct dim_channel *ch,
+ struct dim_ch_state *state_ptr)
{
if (!ch || !state_ptr)
return NULL;
CLK_8192FS = 7,
};
-struct dim_ch_state_t {
+struct dim_ch_state {
bool ready; /* Shows readiness to enqueue next buffer */
u16 done_buffers; /* Number of completed buffers */
};
u8 dim_service_channel(struct dim_channel *ch);
-struct dim_ch_state_t *dim_get_channel_state(struct dim_channel *ch,
- struct dim_ch_state_t *state_ptr);
+struct dim_ch_state *dim_get_channel_state(struct dim_channel *ch,
+ struct dim_ch_state *state_ptr);
u16 dim_dbr_space(struct dim_channel *ch);
*
* Register the i2c client device as a MOST interface
*/
-static int i2c_probe(struct i2c_client *client, const struct i2c_device_id *id)
+static int i2c_probe(struct i2c_client *client)
{
struct hdm_i2c *dev;
int ret, i;
.driver = {
.name = "hdm_i2c",
},
- .probe = i2c_probe,
+ .probe_new = i2c_probe,
.remove = i2c_remove,
.id_table = i2c_id,
};
{
int skb_to_free;
int qos, queues_per_port;
- int total_freed = 0;
int total_remaining = 0;
unsigned long flags;
struct octeon_ethernet *priv = netdev_priv(dev);
MAX_SKB_TO_FREE);
skb_to_free = cvm_oct_adjust_skb_to_free(skb_to_free,
priv->fau + qos * 4);
- total_freed += skb_to_free;
if (skb_to_free > 0) {
struct sk_buff *to_free_list = NULL;
return (void *)(uintptr_t)(physical_address);
}
-static inline uint64_t cvmx_ptr_to_phys(void *ptr)
+static inline phys_addr_t cvmx_ptr_to_phys(void *ptr)
{
return (unsigned long)ptr;
}
return 0;
}
-static int dcon_probe(struct i2c_client *client, const struct i2c_device_id *id)
+static int dcon_probe(struct i2c_client *client)
{
struct dcon_priv *dcon;
int rc, i, j;
},
.class = I2C_CLASS_DDC | I2C_CLASS_HWMON,
.id_table = dcon_idtable,
- .probe = dcon_probe,
+ .probe_new = dcon_probe,
.remove = dcon_remove,
.detect = dcon_detect,
.address_list = normal_i2c,
return beacon_updated;
}
-int rtw_sta_flush(struct adapter *padapter)
+void rtw_sta_flush(struct adapter *padapter)
{
struct list_head *phead, *plist;
- int ret = 0;
struct sta_info *psta = NULL;
struct sta_priv *pstapriv = &padapter->stapriv;
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
u8 bc_addr[ETH_ALEN] = {0xff, 0xff, 0xff, 0xff, 0xff, 0xff};
if ((pmlmeinfo->state & 0x03) != WIFI_FW_AP_STATE)
- return ret;
+ return;
spin_lock_bh(&pstapriv->asoc_list_lock);
phead = &pstapriv->asoc_list;
issue_deauth(padapter, bc_addr, WLAN_REASON_DEAUTH_LEAVING);
associated_clients_update(padapter, true);
-
- return ret;
}
/* called > TSR LEVEL for USB or SDIO Interface*/
static unsigned char *__nat25_find_pppoe_tag(struct pppoe_hdr *ph, unsigned short type)
{
unsigned char *cur_ptr, *start_ptr;
- unsigned short tagLen, tagType;
+ unsigned short tag_len, tag_type;
start_ptr = (unsigned char *)ph->tag;
cur_ptr = (unsigned char *)ph->tag;
while ((cur_ptr - start_ptr) < ntohs(ph->length)) {
/* prevent un-alignment access */
- tagType = (unsigned short)((cur_ptr[0] << 8) + cur_ptr[1]);
- tagLen = (unsigned short)((cur_ptr[2] << 8) + cur_ptr[3]);
- if (tagType == type)
+ tag_type = (unsigned short)((cur_ptr[0] << 8) + cur_ptr[1]);
+ tag_len = (unsigned short)((cur_ptr[2] << 8) + cur_ptr[3]);
+ if (tag_type == type)
return cur_ptr;
- cur_ptr = cur_ptr + TAG_HDR_LEN + tagLen;
+ cur_ptr = cur_ptr + TAG_HDR_LEN + tag_len;
}
return NULL;
}
return 0;
}
-static void __nat25_generate_ipv4_network_addr(unsigned char *networkAddr,
- unsigned int *ipAddr)
+static void __nat25_generate_ipv4_network_addr(unsigned char *addr,
+ unsigned int *ip_addr)
{
- memset(networkAddr, 0, MAX_NETWORK_ADDR_LEN);
+ memset(addr, 0, MAX_NETWORK_ADDR_LEN);
- networkAddr[0] = NAT25_IPV4;
- memcpy(networkAddr + 7, (unsigned char *)ipAddr, 4);
+ addr[0] = NAT25_IPV4;
+ memcpy(addr + 7, (unsigned char *)ip_addr, 4);
}
-static void __nat25_generate_pppoe_network_addr(unsigned char *networkAddr,
+static void __nat25_generate_pppoe_network_addr(unsigned char *addr,
unsigned char *ac_mac, __be16 *sid)
{
- memset(networkAddr, 0, MAX_NETWORK_ADDR_LEN);
+ memset(addr, 0, MAX_NETWORK_ADDR_LEN);
- networkAddr[0] = NAT25_PPPOE;
- memcpy(networkAddr + 1, (unsigned char *)sid, 2);
- memcpy(networkAddr + 3, (unsigned char *)ac_mac, 6);
+ addr[0] = NAT25_PPPOE;
+ memcpy(addr + 1, (unsigned char *)sid, 2);
+ memcpy(addr + 3, (unsigned char *)ac_mac, 6);
}
-static void __nat25_generate_ipv6_network_addr(unsigned char *networkAddr,
- unsigned int *ipAddr)
+static void __nat25_generate_ipv6_network_addr(unsigned char *addr,
+ unsigned int *ip_addr)
{
- memset(networkAddr, 0, MAX_NETWORK_ADDR_LEN);
+ memset(addr, 0, MAX_NETWORK_ADDR_LEN);
- networkAddr[0] = NAT25_IPV6;
- memcpy(networkAddr + 1, (unsigned char *)ipAddr, 16);
+ addr[0] = NAT25_IPV6;
+ memcpy(addr + 1, (unsigned char *)ip_addr, 16);
}
static unsigned char *scan_tlv(unsigned char *data, int len, unsigned char tag, unsigned char len8b)
return 0;
}
-static int __nat25_network_hash(unsigned char *networkAddr)
+static int __nat25_network_hash(unsigned char *addr)
{
- if (networkAddr[0] == NAT25_IPV4) {
+ if (addr[0] == NAT25_IPV4) {
unsigned long x;
- x = networkAddr[7] ^ networkAddr[8] ^ networkAddr[9] ^ networkAddr[10];
+ x = addr[7] ^ addr[8] ^ addr[9] ^ addr[10];
return x & (NAT25_HASH_SIZE - 1);
- } else if (networkAddr[0] == NAT25_IPX) {
+ } else if (addr[0] == NAT25_IPX) {
unsigned long x;
- x = networkAddr[1] ^ networkAddr[2] ^ networkAddr[3] ^ networkAddr[4] ^ networkAddr[5] ^
- networkAddr[6] ^ networkAddr[7] ^ networkAddr[8] ^ networkAddr[9] ^ networkAddr[10];
+ x = addr[1] ^ addr[2] ^ addr[3] ^ addr[4] ^ addr[5] ^
+ addr[6] ^ addr[7] ^ addr[8] ^ addr[9] ^ addr[10];
return x & (NAT25_HASH_SIZE - 1);
- } else if (networkAddr[0] == NAT25_APPLE) {
+ } else if (addr[0] == NAT25_APPLE) {
unsigned long x;
- x = networkAddr[1] ^ networkAddr[2] ^ networkAddr[3];
+ x = addr[1] ^ addr[2] ^ addr[3];
return x & (NAT25_HASH_SIZE - 1);
- } else if (networkAddr[0] == NAT25_PPPOE) {
+ } else if (addr[0] == NAT25_PPPOE) {
unsigned long x;
- x = networkAddr[0] ^ networkAddr[1] ^ networkAddr[2] ^ networkAddr[3] ^ networkAddr[4] ^ networkAddr[5] ^ networkAddr[6] ^ networkAddr[7] ^ networkAddr[8];
+ x = addr[0] ^ addr[1] ^ addr[2] ^ addr[3] ^ addr[4] ^
+ addr[5] ^ addr[6] ^ addr[7] ^ addr[8];
return x & (NAT25_HASH_SIZE - 1);
- } else if (networkAddr[0] == NAT25_IPV6) {
+ } else if (addr[0] == NAT25_IPV6) {
unsigned long x;
- x = networkAddr[1] ^ networkAddr[2] ^ networkAddr[3] ^ networkAddr[4] ^ networkAddr[5] ^
- networkAddr[6] ^ networkAddr[7] ^ networkAddr[8] ^ networkAddr[9] ^ networkAddr[10] ^
- networkAddr[11] ^ networkAddr[12] ^ networkAddr[13] ^ networkAddr[14] ^ networkAddr[15] ^
- networkAddr[16];
+ x = addr[1] ^ addr[2] ^ addr[3] ^ addr[4] ^ addr[5] ^ addr[6] ^
+ addr[7] ^ addr[8] ^ addr[9] ^ addr[10] ^ addr[11] ^ addr[12] ^
+ addr[13] ^ addr[14] ^ addr[15] ^ addr[16];
return x & (NAT25_HASH_SIZE - 1);
} else {
int i;
for (i = 0; i < MAX_NETWORK_ADDR_LEN; i++)
- x ^= networkAddr[i];
+ x ^= addr[i];
return x & (NAT25_HASH_SIZE - 1);
}
}
static void __nat25_db_network_insert(struct adapter *priv,
- unsigned char *macAddr, unsigned char *networkAddr)
+ unsigned char *mac_addr, unsigned char *addr)
{
struct nat25_network_db_entry *db;
int hash;
spin_lock_bh(&priv->br_ext_lock);
- hash = __nat25_network_hash(networkAddr);
+ hash = __nat25_network_hash(addr);
db = priv->nethash[hash];
while (db) {
- if (!memcmp(db->networkAddr, networkAddr, MAX_NETWORK_ADDR_LEN)) {
- memcpy(db->macAddr, macAddr, ETH_ALEN);
+ if (!memcmp(db->networkAddr, addr, MAX_NETWORK_ADDR_LEN)) {
+ memcpy(db->macAddr, mac_addr, ETH_ALEN);
db->ageing_timer = jiffies;
spin_unlock_bh(&priv->br_ext_lock);
return;
spin_unlock_bh(&priv->br_ext_lock);
return;
}
- memcpy(db->networkAddr, networkAddr, MAX_NETWORK_ADDR_LEN);
- memcpy(db->macAddr, macAddr, ETH_ALEN);
+ memcpy(db->networkAddr, addr, MAX_NETWORK_ADDR_LEN);
+ memcpy(db->macAddr, mac_addr, ETH_ALEN);
atomic_set(&db->use_count, 1);
db->ageing_timer = jiffies;
int nat25_db_handle(struct adapter *priv, struct sk_buff *skb, int method)
{
unsigned short protocol;
- unsigned char networkAddr[MAX_NETWORK_ADDR_LEN];
+ unsigned char addr[MAX_NETWORK_ADDR_LEN];
unsigned int tmp;
if (!skb)
if (iph->saddr == 0)
return 0;
tmp = be32_to_cpu(iph->saddr);
- __nat25_generate_ipv4_network_addr(networkAddr, &tmp);
+ __nat25_generate_ipv4_network_addr(addr, &tmp);
/* record source IP address and , source mac address into db */
- __nat25_db_network_insert(priv, skb->data + ETH_ALEN, networkAddr);
+ __nat25_db_network_insert(priv, skb->data + ETH_ALEN, addr);
return 0;
default:
return -1;
memcpy(arp_ptr, GET_MY_HWADDR(priv), ETH_ALEN);
arp_ptr += arp->ar_hln;
sender = (unsigned int *)arp_ptr;
- __nat25_generate_ipv4_network_addr(networkAddr, sender);
- __nat25_db_network_insert(priv, skb->data + ETH_ALEN, networkAddr);
+ __nat25_generate_ipv4_network_addr(addr, sender);
+ __nat25_db_network_insert(priv, skb->data + ETH_ALEN, addr);
return 0;
default:
return -1;
return -1;
}
} else { /* session phase */
- __nat25_generate_pppoe_network_addr(networkAddr, skb->data, &ph->sid);
+ __nat25_generate_pppoe_network_addr(addr, skb->data, &ph->sid);
- __nat25_db_network_insert(priv, skb->data + ETH_ALEN, networkAddr);
+ __nat25_db_network_insert(priv, skb->data + ETH_ALEN, addr);
if (!priv->ethBrExtInfo.addPPPoETag &&
priv->pppoe_connection_in_progress &&
return -1;
case NAT25_INSERT:
if (memcmp(&iph->saddr, "\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0\x0", 16)) {
- __nat25_generate_ipv6_network_addr(networkAddr, (unsigned int *)&iph->saddr);
- __nat25_db_network_insert(priv, skb->data + ETH_ALEN, networkAddr);
+ __nat25_generate_ipv6_network_addr(addr, (unsigned int *)&iph->saddr);
+ __nat25_db_network_insert(priv, skb->data + ETH_ALEN, addr);
if (iph->nexthdr == IPPROTO_ICMPV6 &&
skb->len > (ETH_HLEN + sizeof(*iph) + 4)) {
if (!priv->ethBrExtInfo.dhcp_bcst_disable) {
__be16 protocol = *((__be16 *)(skb->data + 2 * ETH_ALEN));
- if (protocol == __constant_htons(ETH_P_IP)) { /* IP */
+ if (protocol == htons(ETH_P_IP)) { /* IP */
struct iphdr *iph = (struct iphdr *)(skb->data + ETH_HLEN);
if (iph->protocol == IPPROTO_UDP) { /* UDP */
- struct udphdr *udph = (struct udphdr *)((size_t)iph + (iph->ihl << 2));
+ struct udphdr *udph = (void *)iph + (iph->ihl << 2);
- if ((udph->source == __constant_htons(CLIENT_PORT)) &&
- (udph->dest == __constant_htons(SERVER_PORT))) { /* DHCP request */
- struct dhcpMessage *dhcph =
- (struct dhcpMessage *)((size_t)udph + sizeof(struct udphdr));
- u32 cookie = be32_to_cpu((__be32)dhcph->cookie);
+ if ((udph->source == htons(CLIENT_PORT)) &&
+ (udph->dest == htons(SERVER_PORT))) { /* DHCP request */
+ struct dhcpMessage *dhcph = (void *)udph + sizeof(struct udphdr);
+ u32 cookie = be32_to_cpu(dhcph->cookie);
if (cookie == DHCP_MAGIC) { /* match magic word */
if (!(dhcph->flags & htons(BROADCAST_FLAG))) {
}
}
-void *scdb_findEntry(struct adapter *priv, unsigned char *ipAddr)
+void *scdb_findEntry(struct adapter *priv, unsigned char *ip_addr)
{
- unsigned char networkAddr[MAX_NETWORK_ADDR_LEN];
+ unsigned char addr[MAX_NETWORK_ADDR_LEN];
struct nat25_network_db_entry *db;
int hash;
- __nat25_generate_ipv4_network_addr(networkAddr, (unsigned int *)ipAddr);
- hash = __nat25_network_hash(networkAddr);
+ __nat25_generate_ipv4_network_addr(addr, (unsigned int *)ip_addr);
+ hash = __nat25_network_hash(addr);
db = priv->nethash[hash];
while (db) {
- if (!memcmp(db->networkAddr, networkAddr, MAX_NETWORK_ADDR_LEN)) {
+ if (!memcmp(db->networkAddr, addr, MAX_NETWORK_ADDR_LEN))
return (void *)db;
- }
db = db->next_hash;
}
return _SUCCESS;
}
-u32 rtw_init_cmd_priv(struct cmd_priv *pcmdpriv)
+int rtw_init_cmd_priv(struct cmd_priv *pcmdpriv)
{
init_completion(&pcmdpriv->enqueue_cmd);
/* sema_init(&(pcmdpriv->cmd_done_sema), 0); */
GFP_KERNEL);
if (!pcmdpriv->cmd_allocated_buf)
- return _FAIL;
+ return -ENOMEM;
pcmdpriv->cmd_buf = pcmdpriv->cmd_allocated_buf + CMDBUFF_ALIGN_SZ - ((size_t)(pcmdpriv->cmd_allocated_buf) & (CMDBUFF_ALIGN_SZ - 1));
if (!pcmdpriv->rsp_allocated_buf) {
kfree(pcmdpriv->cmd_allocated_buf);
- return _FAIL;
+ return -ENOMEM;
}
pcmdpriv->rsp_buf = pcmdpriv->rsp_allocated_buf + 4 - ((size_t)(pcmdpriv->rsp_allocated_buf) & 3);
pcmdpriv->cmd_done_cnt = 0;
pcmdpriv->rsp_cnt = 0;
- return _SUCCESS;
+ return 0;
}
-u32 rtw_init_evt_priv(struct evt_priv *pevtpriv)
+int rtw_init_evt_priv(struct evt_priv *pevtpriv)
{
- u32 res = _SUCCESS;
-
/* allocate DMA-able/Non-Page memory for cmd_buf and rsp_buf */
atomic_set(&pevtpriv->event_seq, 0);
pevtpriv->c2h_wk_alive = false;
pevtpriv->c2h_queue = rtw_cbuf_alloc(C2H_QUEUE_MAX_LEN + 1);
if (!pevtpriv->c2h_queue)
- res = _FAIL;
+ return -ENOMEM;
- return res;
+ return 0;
}
void rtw_free_cmd_priv(struct cmd_priv *pcmdpriv)
return res;
}
-u8 rtw_setdatarate_cmd(struct adapter *padapter, u8 *rateset)
+int rtw_setdatarate_cmd(struct adapter *padapter, u8 *rateset)
{
struct cmd_obj *ph2c;
struct setdatarate_parm *pbsetdataratepara;
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
- u8 res = _SUCCESS;
ph2c = kzalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (!ph2c) {
- res = _FAIL;
- goto exit;
- }
+ if (!ph2c)
+ return -ENOMEM;
pbsetdataratepara = kzalloc(sizeof(*pbsetdataratepara), GFP_ATOMIC);
if (!pbsetdataratepara) {
kfree(ph2c);
- res = _FAIL;
- goto exit;
+ return -ENOMEM;
}
init_h2fwcmd_w_parm_no_rsp(ph2c, pbsetdataratepara, GEN_CMD_CODE(_SetDataRate));
pbsetdataratepara->mac_id = 5;
memcpy(pbsetdataratepara->datarates, rateset, NumRates);
- res = rtw_enqueue_cmd(pcmdpriv, ph2c);
-exit:
+ if (rtw_enqueue_cmd(pcmdpriv, ph2c) == _FAIL)
+ return -EPERM;
- return res;
+ return 0;
}
void rtw_getbbrfreg_cmdrsp_callback(struct adapter *padapter, struct cmd_obj *pcmd)
if ((*pold_state == Ndis802_11Infrastructure) || (*pold_state == Ndis802_11IBSS)) {
if (check_fwstate(pmlmepriv, _FW_LINKED))
- rtw_indicate_disconnect(padapter); /* will clr Linked_state; before this function, we must have chked whether issue dis-assoc_cmd or not */
+ rtw_indicate_disconnect(padapter); /* will clr Linked_state; before this function, we must have checked whether issue dis-assoc_cmd or not */
}
*pold_state = networktype;
return true;
}
-u8 rtw_set_802_11_disassociate(struct adapter *padapter)
+void rtw_set_802_11_disassociate(struct adapter *padapter)
{
struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
}
spin_unlock_bh(&pmlmepriv->lock);
-
- return true;
}
u8 rtw_set_802_11_bssid_list_scan(struct adapter *padapter, struct ndis_802_11_ssid *pssid, int ssid_max_num)
pLed->BlinkTimes = 0; /* Number of times to toggle led state for blinking. */
- pLed->bLedLinkBlinkInProgress = false;
pLed->bLedScanBlinkInProgress = false;
}
-static void SwLedOn(struct adapter *padapter, struct led_priv *pLed)
+static void SwLedOn(struct led_priv *pLed)
{
- u8 LedCfg;
- int res;
+ struct adapter *padapter = container_of(pLed, struct adapter, ledpriv);
if (padapter->bDriverStopped)
return;
- res = rtw_read8(padapter, REG_LEDCFG2, &LedCfg);
- if (res)
+ if (rtw_write8(padapter, REG_LEDCFG2, BIT(5)) != _SUCCESS)
return;
- rtw_write8(padapter, REG_LEDCFG2, (LedCfg & 0xf0) | BIT(5) | BIT(6)); /* SW control led0 on. */
pLed->bLedOn = true;
}
-static void SwLedOff(struct adapter *padapter, struct led_priv *pLed)
+static void SwLedOff(struct led_priv *pLed)
{
- u8 LedCfg;
- int res;
+ struct adapter *padapter = container_of(pLed, struct adapter, ledpriv);
if (padapter->bDriverStopped)
- goto exit;
-
- res = rtw_read8(padapter, REG_LEDCFG2, &LedCfg);/* 0x4E */
- if (res)
- goto exit;
+ return;
- LedCfg &= 0x90; /* Set to software control. */
- rtw_write8(padapter, REG_LEDCFG2, (LedCfg | BIT(3)));
- res = rtw_read8(padapter, REG_MAC_PINMUX_CFG, &LedCfg);
- if (res)
- goto exit;
+ if (rtw_write8(padapter, REG_LEDCFG2, BIT(5) | BIT(3)) != _SUCCESS)
+ return;
- LedCfg &= 0xFE;
- rtw_write8(padapter, REG_MAC_PINMUX_CFG, LedCfg);
-exit:
pLed->bLedOn = false;
}
{
struct delayed_work *dwork = to_delayed_work(work);
struct led_priv *pLed = container_of(dwork, struct led_priv, blink_work);
- struct adapter *padapter = pLed->padapter;
+ struct adapter *padapter = container_of(pLed, struct adapter, ledpriv);
struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
if (padapter->pwrctrlpriv.rf_pwrstate != rf_on) {
- SwLedOff(padapter, pLed);
+ SwLedOff(pLed);
ResetLedStatus(pLed);
return;
}
if (pLed->bLedOn)
- SwLedOff(padapter, pLed);
+ SwLedOff(pLed);
else
- SwLedOn(padapter, pLed);
+ SwLedOn(pLed);
switch (pLed->CurrLedState) {
case LED_BLINK_SLOWLY:
schedule_delayed_work(&pLed->blink_work, LED_BLINK_LINK_INTVL);
break;
case LED_BLINK_SCAN:
- pLed->BlinkTimes--;
- if (pLed->BlinkTimes == 0) {
- if (check_fwstate(pmlmepriv, _FW_LINKED)) {
- pLed->bLedLinkBlinkInProgress = true;
- pLed->CurrLedState = LED_BLINK_NORMAL;
- schedule_delayed_work(&pLed->blink_work, LED_BLINK_LINK_INTVL);
- } else {
- pLed->CurrLedState = LED_BLINK_SLOWLY;
- schedule_delayed_work(&pLed->blink_work, LED_BLINK_NO_LINK_INTVL);
- }
- pLed->bLedScanBlinkInProgress = false;
- } else {
- schedule_delayed_work(&pLed->blink_work, LED_BLINK_SCAN_INTVL);
- }
- break;
case LED_BLINK_TXRX:
pLed->BlinkTimes--;
if (pLed->BlinkTimes == 0) {
if (check_fwstate(pmlmepriv, _FW_LINKED)) {
- pLed->bLedLinkBlinkInProgress = true;
pLed->CurrLedState = LED_BLINK_NORMAL;
schedule_delayed_work(&pLed->blink_work, LED_BLINK_LINK_INTVL);
} else {
schedule_delayed_work(&pLed->blink_work, LED_BLINK_NO_LINK_INTVL);
}
pLed->bLedBlinkInProgress = false;
+ pLed->bLedScanBlinkInProgress = false;
} else {
- schedule_delayed_work(&pLed->blink_work, LED_BLINK_FASTER_INTVL);
+ schedule_delayed_work(&pLed->blink_work,
+ pLed->CurrLedState == LED_BLINK_SCAN ?
+ LED_BLINK_SCAN_INTVL : LED_BLINK_FASTER_INTVL);
}
break;
case LED_BLINK_WPS:
break;
case LED_BLINK_WPS_STOP: /* WPS success */
if (!pLed->bLedOn) {
- pLed->bLedLinkBlinkInProgress = true;
pLed->CurrLedState = LED_BLINK_NORMAL;
schedule_delayed_work(&pLed->blink_work, LED_BLINK_LINK_INTVL);
{
struct led_priv *pledpriv = &padapter->ledpriv;
- pledpriv->padapter = padapter;
ResetLedStatus(pledpriv);
INIT_DELAYED_WORK(&pledpriv->blink_work, blink_work);
}
cancel_delayed_work_sync(&ledpriv->blink_work);
ResetLedStatus(ledpriv);
- SwLedOff(padapter, ledpriv);
+ SwLedOff(ledpriv);
}
void rtw_led_control(struct adapter *padapter, enum LED_CTL_MODE LedAction)
struct registry_priv *registry_par;
struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
- if ((padapter->bSurpriseRemoved) || (padapter->bDriverStopped) ||
- (!padapter->hw_init_completed))
+ if (!padapter->hw_init_completed)
return;
if (!pLed->bRegUseLed)
cancel_delayed_work(&pLed->blink_work);
- pLed->bLedLinkBlinkInProgress = false;
pLed->bLedBlinkInProgress = false;
pLed->CurrLedState = LED_BLINK_SLOWLY;
schedule_delayed_work(&pLed->blink_work, LED_BLINK_NO_LINK_INTVL);
break;
case LED_CTL_LINK:
- if (!pLed->bLedLinkBlinkInProgress)
- return;
-
if (pLed->CurrLedState == LED_BLINK_SCAN || IS_LED_WPS_BLINKING(pLed))
return;
cancel_delayed_work(&pLed->blink_work);
pLed->bLedBlinkInProgress = false;
- pLed->bLedLinkBlinkInProgress = true;
pLed->CurrLedState = LED_BLINK_NORMAL;
schedule_delayed_work(&pLed->blink_work, LED_BLINK_LINK_INTVL);
cancel_delayed_work(&pLed->blink_work);
- pLed->bLedLinkBlinkInProgress = false;
pLed->bLedBlinkInProgress = false;
pLed->bLedScanBlinkInProgress = true;
cancel_delayed_work(&pLed->blink_work);
- pLed->bLedLinkBlinkInProgress = false;
pLed->bLedBlinkInProgress = true;
pLed->CurrLedState = LED_BLINK_TXRX;
cancel_delayed_work(&pLed->blink_work);
- pLed->bLedLinkBlinkInProgress = false;
pLed->bLedBlinkInProgress = false;
pLed->bLedScanBlinkInProgress = false;
pLed->bLedWPSBlinkInProgress = true;
case LED_CTL_STOP_WPS:
cancel_delayed_work(&pLed->blink_work);
- pLed->bLedLinkBlinkInProgress = false;
pLed->bLedBlinkInProgress = false;
pLed->bLedScanBlinkInProgress = false;
pLed->bLedWPSBlinkInProgress = true;
break;
case LED_CTL_POWER_OFF:
pLed->CurrLedState = RTW_LED_OFF;
- pLed->bLedLinkBlinkInProgress = false;
pLed->bLedBlinkInProgress = false;
pLed->bLedWPSBlinkInProgress = false;
pLed->bLedScanBlinkInProgress = false;
cancel_delayed_work(&pLed->blink_work);
- SwLedOff(padapter, pLed);
+ SwLedOff(pLed);
break;
default:
break;
spin_unlock_bh(&free_queue->lock);
}
-void _rtw_free_network_nolock(struct mlme_priv *pmlmepriv, struct wlan_network *pnetwork)
-{
- struct __queue *free_queue = &pmlmepriv->free_bss_pool;
-
- if (!pnetwork)
- return;
- if (pnetwork->fixed)
- return;
- list_del_init(&pnetwork->list);
- list_add_tail(&pnetwork->list, get_list_head(free_queue));
- pmlmepriv->num_of_scanned--;
-}
-
/*
return the wlan_network with the matching addr
u8 *pbuf;
struct wlan_network *pnetwork;
struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
- int res = _SUCCESS;
/* We don't need to memset padapter->XXX to zero, because adapter is allocated by vzalloc(). */
pbuf = vzalloc(MAX_BSS_CNT * (sizeof(struct wlan_network)));
- if (!pbuf) {
- res = _FAIL;
- goto exit;
- }
+ if (!pbuf)
+ return -ENOMEM;
+
pmlmepriv->free_bss_buf = pbuf;
pnetwork = (struct wlan_network *)pbuf;
rtw_init_mlme_timer(padapter);
-exit:
-
- return res;
+ return 0;
}
void rtw_free_mlme_priv(struct mlme_priv *pmlmepriv)
static void rtw_free_network_nolock(struct mlme_priv *pmlmepriv,
struct wlan_network *pnetwork)
{
+ struct __queue *free_queue = &pmlmepriv->free_bss_pool;
- _rtw_free_network_nolock(pmlmepriv, pnetwork);
-
+ if (!pnetwork)
+ return;
+ if (pnetwork->fixed)
+ return;
+ list_del_init(&pnetwork->list);
+ list_add_tail(&pnetwork->list, get_list_head(free_queue));
+ pmlmepriv->num_of_scanned--;
}
void rtw_free_network_queue(struct adapter *dev, u8 isfreeall)
pdev_network->Rssi = 0;
- switch (pregistrypriv->wireless_mode) {
- case WIRELESS_11B:
- pdev_network->NetworkTypeInUse = (Ndis802_11DS);
- break;
- case WIRELESS_11G:
- case WIRELESS_11BG:
- case WIRELESS_11_24N:
- case WIRELESS_11G_24N:
- case WIRELESS_11BG_24N:
- pdev_network->NetworkTypeInUse = (Ndis802_11OFDM24);
- break;
- default:
- /* TODO */
- break;
- }
-
pdev_network->Configuration.DSConfig = (pregistrypriv->channel);
if (cur_network->network.InfrastructureMode == Ndis802_11IBSS)
#include "../include/rtl8188e_xmit.h"
#include "../include/rtl8188e_dm.h"
-/* response function for each management frame subtype, do not reorder */
-static mlme_handler mlme_sta_tbl[] = {
- OnAssocReq,
- OnAssocRsp,
- OnAssocReq,
- OnAssocRsp,
- OnProbeReq,
- OnProbeRsp,
- NULL,
- NULL,
- OnBeacon,
- NULL,
- OnDisassoc,
- OnAuthClient,
- OnDeAuth,
- OnAction,
-};
-
static u8 null_addr[ETH_ALEN] = {0, 0, 0, 0, 0, 0};
/**************************************************
{0x03}, /* 0x41, RT_CHANNEL_DOMAIN_GLOBAL_DOAMIN_2G */
};
-static struct rt_channel_plan_map RTW_CHANNEL_PLAN_MAP_REALTEK_DEFINE = {0x03}; /* use the conbination for max channel numbers */
+static struct rt_channel_plan_map RTW_CHANNEL_PLAN_MAP_REALTEK_DEFINE = {0x03}; /* use the combination for max channel numbers */
/*
* Search the @param channel_num in given @param channel_set
}
}
-void mgt_dispatcher(struct adapter *padapter, struct recv_frame *precv_frame)
-{
- int index;
- mlme_handler fct;
- struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
- struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)precv_frame->rx_data;
- struct sta_info *psta = rtw_get_stainfo(&padapter->stapriv, hdr->addr2);
-
- if (!ieee80211_is_mgmt(hdr->frame_control))
- return;
-
- /* receive the frames that ra(a1) is my address or ra(a1) is bc address. */
- if (memcmp(hdr->addr1, myid(&padapter->eeprompriv), ETH_ALEN) &&
- !is_broadcast_ether_addr(hdr->addr1))
- return;
-
- index = (le16_to_cpu(hdr->frame_control) & IEEE80211_FCTL_STYPE) >> 4;
- if (index >= ARRAY_SIZE(mlme_sta_tbl))
- return;
- fct = mlme_sta_tbl[index];
-
- if (psta) {
- if (ieee80211_has_retry(hdr->frame_control)) {
- if (precv_frame->attrib.seq_num == psta->RxMgmtFrameSeqNum)
- /* drop the duplicate management frame */
- return;
- }
- psta->RxMgmtFrameSeqNum = precv_frame->attrib.seq_num;
- }
-
- if (ieee80211_is_auth(hdr->frame_control)) {
- if (check_fwstate(pmlmepriv, WIFI_AP_STATE))
- fct = OnAuth;
- else
- fct = OnAuthClient;
- }
-
- if (fct)
- fct(padapter, precv_frame);
-}
-
static u32 p2p_listen_state_process(struct adapter *padapter, unsigned char *da)
{
bool response = true;
return _SUCCESS;
}
-static void update_TSF(struct mlme_ext_priv *pmlmeext, u8 *pframe)
-{
- u8 *pIE;
- __le32 *pbuf;
-
- pIE = pframe + sizeof(struct ieee80211_hdr_3addr);
- pbuf = (__le32 *)pIE;
-
- pmlmeext->TSFValue = le32_to_cpu(*(pbuf + 1));
-
- pmlmeext->TSFValue = pmlmeext->TSFValue << 32;
-
- pmlmeext->TSFValue |= le32_to_cpu(*pbuf);
-}
-
static void correct_TSF(struct adapter *padapter)
{
u8 reg;
*****************************************************************************/
-unsigned int OnProbeReq(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnProbeReq(struct adapter *padapter, struct recv_frame *precv_frame)
{
unsigned int ielen;
unsigned char *p;
report_survey_event(padapter, precv_frame);
p2p_listen_state_process(padapter, get_sa(pframe));
- return _SUCCESS;
+ return;
}
}
}
if (check_fwstate(pmlmepriv, WIFI_STATION_STATE))
- return _SUCCESS;
+ return;
if (!check_fwstate(pmlmepriv, _FW_LINKED) &&
!check_fwstate(pmlmepriv, WIFI_ADHOC_MASTER_STATE | WIFI_AP_STATE))
- return _SUCCESS;
+ return;
p = rtw_get_ie(pframe + WLAN_HDR_A3_LEN + _PROBEREQ_IE_OFFSET_, _SSID_IE_, (int *)&ielen,
len - WLAN_HDR_A3_LEN - _PROBEREQ_IE_OFFSET_);
if ((ielen != 0 && memcmp((void *)(p + 2), (void *)cur->Ssid.Ssid, cur->Ssid.SsidLength)) ||
(ielen == 0 && pmlmeinfo->hidden_ssid_mode))
- return _SUCCESS;
+ return;
_issue_probersp:
check_fwstate(pmlmepriv, WIFI_ADHOC_MASTER_STATE)))
issue_probersp(padapter, get_sa(pframe), is_valid_p2p_probereq);
}
- return _SUCCESS;
}
-unsigned int OnProbeRsp(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnProbeRsp(struct adapter *padapter, struct recv_frame *precv_frame)
{
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
struct wifidirect_info *pwdinfo = &padapter->wdinfo;
}
}
}
- return _SUCCESS;
+ return;
} else if (rtw_p2p_chk_state(pwdinfo, P2P_STATE_GONEGO_ING)) {
if (pwdinfo->nego_req_info.benable) {
if (!memcmp(pwdinfo->nego_req_info.peerDevAddr, GetAddr2Ptr(pframe), ETH_ALEN)) {
}
if (pmlmeext->sitesurvey_res.state == SCAN_PROCESS) {
report_survey_event(padapter, precv_frame);
- return _SUCCESS;
+ return;
}
-
- return _SUCCESS;
}
-unsigned int OnBeacon(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnBeacon(struct adapter *padapter, struct recv_frame *precv_frame)
{
+ struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)precv_frame->rx_data;
int cam_idx;
struct sta_info *psta;
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
u8 *pframe = precv_frame->rx_data;
uint len = precv_frame->len;
struct wlan_bssid_ex *pbss;
- int ret = _SUCCESS;
+ u8 *ie_ptr;
+ u32 ie_len;
+
+ ie_ptr = (u8 *)&mgmt->u.beacon.variable;
+ if (precv_frame->len < offsetof(struct ieee80211_mgmt, u.beacon.variable))
+ return;
+ ie_len = precv_frame->len - offsetof(struct ieee80211_mgmt, u.beacon.variable);
if (pmlmeext->sitesurvey_res.state == SCAN_PROCESS) {
report_survey_event(padapter, precv_frame);
- return _SUCCESS;
+ return;
}
- if (!memcmp(GetAddr3Ptr(pframe), get_my_bssid(&pmlmeinfo->network), ETH_ALEN)) {
- if (pmlmeinfo->state & WIFI_FW_AUTH_NULL) {
- /* we should update current network before auth, or some IE is wrong */
- pbss = kmalloc(sizeof(struct wlan_bssid_ex), GFP_ATOMIC);
- if (pbss) {
- if (collect_bss_info(padapter, precv_frame, pbss) == _SUCCESS) {
- update_network(&pmlmepriv->cur_network.network, pbss, padapter, true);
- rtw_get_bcn_info(&pmlmepriv->cur_network);
- }
- kfree(pbss);
- }
+ if (memcmp(mgmt->bssid, get_my_bssid(&pmlmeinfo->network), ETH_ALEN))
+ return;
- /* check the vendor of the assoc AP */
- pmlmeinfo->assoc_AP_vendor = check_assoc_AP(pframe + sizeof(struct ieee80211_hdr_3addr), len - sizeof(struct ieee80211_hdr_3addr));
+ if (pmlmeinfo->state & WIFI_FW_AUTH_NULL) {
+ /* we should update current network before auth, or some IE is wrong */
+ pbss = kmalloc(sizeof(struct wlan_bssid_ex), GFP_ATOMIC);
+ if (!pbss)
+ return;
- /* update TSF Value */
- update_TSF(pmlmeext, pframe);
+ if (collect_bss_info(padapter, precv_frame, pbss) == _SUCCESS) {
+ update_network(&pmlmepriv->cur_network.network, pbss, padapter, true);
+ rtw_get_bcn_info(&pmlmepriv->cur_network);
+ }
+ kfree(pbss);
- /* start auth */
- start_clnt_auth(padapter);
+ /* check the vendor of the assoc AP */
+ pmlmeinfo->assoc_AP_vendor = check_assoc_AP(pframe + sizeof(struct ieee80211_hdr_3addr), len - sizeof(struct ieee80211_hdr_3addr));
- return _SUCCESS;
+ pmlmeext->TSFValue = le64_to_cpu(mgmt->u.beacon.timestamp);
+
+ /* start auth */
+ start_clnt_auth(padapter);
+
+ return;
+ }
+
+ if (((pmlmeinfo->state & 0x03) == WIFI_FW_STATION_STATE) && (pmlmeinfo->state & WIFI_FW_ASSOC_SUCCESS)) {
+ psta = rtw_get_stainfo(pstapriv, mgmt->sa);
+ if (!psta)
+ return;
+
+ if (rtw_check_bcn_info(padapter, pframe, len) != _SUCCESS) {
+ receive_disconnect(padapter, pmlmeinfo->network.MacAddress, 0);
+ return;
}
+ /* update WMM, ERP in the beacon */
+ /* todo: the timer is used instead of the number of the beacon received */
+ if ((sta_rx_pkts(psta) & 0xf) == 0)
+ update_beacon_info(padapter, ie_ptr, ie_len, psta);
+ process_p2p_ps_ie(padapter, ie_ptr, ie_len);
+ } else if ((pmlmeinfo->state & 0x03) == WIFI_FW_ADHOC_STATE) {
+ psta = rtw_get_stainfo(pstapriv, mgmt->sa);
+ if (psta) {
+ /* update WMM, ERP in the beacon */
+ /* todo: the timer is used instead of the number of the beacon received */
+ if ((sta_rx_pkts(psta) & 0xf) == 0)
+ update_beacon_info(padapter, ie_ptr, ie_len, psta);
+ } else {
+ /* allocate a new CAM entry for IBSS station */
+ cam_idx = allocate_fw_sta_entry(padapter);
+ if (cam_idx == NUM_STA)
+ return;
- if (((pmlmeinfo->state & 0x03) == WIFI_FW_STATION_STATE) && (pmlmeinfo->state & WIFI_FW_ASSOC_SUCCESS)) {
- psta = rtw_get_stainfo(pstapriv, GetAddr2Ptr(pframe));
- if (psta) {
- ret = rtw_check_bcn_info(padapter, pframe, len);
- if (!ret) {
- receive_disconnect(padapter,
- pmlmeinfo->network.MacAddress, 0);
- return _SUCCESS;
- }
- /* update WMM, ERP in the beacon */
- /* todo: the timer is used instead of the number of the beacon received */
- if ((sta_rx_pkts(psta) & 0xf) == 0)
- update_beacon_info(padapter, pframe, len, psta);
- process_p2p_ps_ie(padapter, (pframe + WLAN_HDR_A3_LEN), (len - WLAN_HDR_A3_LEN));
+ /* get supported rate */
+ if (update_sta_support_rate(padapter, ie_ptr, ie_len, cam_idx) == _FAIL) {
+ pmlmeinfo->FW_sta_info[cam_idx].status = 0;
+ return;
}
- } else if ((pmlmeinfo->state & 0x03) == WIFI_FW_ADHOC_STATE) {
- psta = rtw_get_stainfo(pstapriv, GetAddr2Ptr(pframe));
- if (psta) {
- /* update WMM, ERP in the beacon */
- /* todo: the timer is used instead of the number of the beacon received */
- if ((sta_rx_pkts(psta) & 0xf) == 0)
- update_beacon_info(padapter, pframe, len, psta);
- } else {
- /* allocate a new CAM entry for IBSS station */
- cam_idx = allocate_fw_sta_entry(padapter);
- if (cam_idx == NUM_STA)
- goto _END_ONBEACON_;
-
- /* get supported rate */
- if (update_sta_support_rate(padapter, (pframe + WLAN_HDR_A3_LEN + _BEACON_IE_OFFSET_), (len - WLAN_HDR_A3_LEN - _BEACON_IE_OFFSET_), cam_idx) == _FAIL) {
- pmlmeinfo->FW_sta_info[cam_idx].status = 0;
- goto _END_ONBEACON_;
- }
- /* update TSF Value */
- update_TSF(pmlmeext, pframe);
+ pmlmeext->TSFValue = le64_to_cpu(mgmt->u.beacon.timestamp);
- /* report sta add event */
- report_add_sta_event(padapter, GetAddr2Ptr(pframe), cam_idx);
- }
+ report_add_sta_event(padapter, mgmt->sa, cam_idx);
}
}
-
-_END_ONBEACON_:
-
- return _SUCCESS;
}
-unsigned int OnAuth(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnAuth(struct adapter *padapter, struct recv_frame *precv_frame)
{
unsigned int auth_mode, ie_len;
u16 seq;
uint len = precv_frame->len;
if ((pmlmeinfo->state & 0x03) != WIFI_FW_AP_STATE)
- return _FAIL;
+ return;
sa = GetAddr2Ptr(pframe);
if (pstat->state & WIFI_FW_AUTH_SUCCESS)
pstat->auth_seq = 0;
- return _SUCCESS;
+ return;
auth_fail:
memcpy(pstat->hwaddr, sa, 6);
issue_auth(padapter, pstat, (unsigned short)status);
- return _FAIL;
}
-unsigned int OnAuthClient(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnAuthClient(struct adapter *padapter, struct recv_frame *precv_frame)
{
unsigned int seq, len, status, offset;
unsigned char *p;
- unsigned int go2asoc = 0;
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)precv_frame->rx_data;
u8 *pframe = precv_frame->rx_data;
uint pkt_len = precv_frame->len;
/* check A1 matches or not */
- if (memcmp(myid(&padapter->eeprompriv), get_da(pframe), ETH_ALEN))
- return _SUCCESS;
+ if (memcmp(myid(&padapter->eeprompriv), ieee80211_get_DA(hdr), ETH_ALEN))
+ return;
if (!(pmlmeinfo->state & WIFI_FW_AUTH_STATE))
- return _SUCCESS;
+ return;
- offset = (GetPrivacy(pframe)) ? 4 : 0;
+ offset = ieee80211_has_protected(hdr->frame_control) ? 4 : 0;
seq = le16_to_cpu(*(__le16 *)((size_t)pframe + WLAN_HDR_A3_LEN + offset + 2));
status = le16_to_cpu(*(__le16 *)((size_t)pframe + WLAN_HDR_A3_LEN + offset + 4));
}
set_link_timer(pmlmeext, 1);
- goto authclnt_fail;
+ return;
}
if (seq == 2) {
pkt_len - WLAN_HDR_A3_LEN - _AUTH_IE_OFFSET_);
if (!p)
- goto authclnt_fail;
+ return;
memcpy((void *)(pmlmeinfo->chg_txt), (void *)(p + 2), len);
pmlmeinfo->auth_seq = 3;
issue_auth(padapter, NULL, 0);
set_link_timer(pmlmeext, REAUTH_TO);
- return _SUCCESS;
+ return;
} else {
/* open system */
- go2asoc = 1;
+ start_clnt_assoc(padapter);
}
} else if (seq == 4) {
if (pmlmeinfo->auth_algo == dot11AuthAlgrthm_Shared)
- go2asoc = 1;
- else
- goto authclnt_fail;
- } else {
- /* this is also illegal */
- goto authclnt_fail;
+ start_clnt_assoc(padapter);
}
-
- if (go2asoc) {
- start_clnt_assoc(padapter);
- return _SUCCESS;
- }
-authclnt_fail:
- return _FAIL;
}
static void UpdateBrateTbl(u8 *mbrate)
}
}
-unsigned int OnAssocReq(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnAssocReq(struct adapter *padapter, struct recv_frame *precv_frame)
{
u16 capab_info;
struct rtw_ieee802_11_elems elems;
u32 p2pielen = 0;
if ((pmlmeinfo->state & 0x03) != WIFI_FW_AP_STATE)
- return _FAIL;
+ return;
frame_type = GetFrameSubType(pframe);
if (frame_type == WIFI_ASSOCREQ)
ie_offset = _REASOCREQ_IE_OFFSET_;
if (pkt_len < IEEE80211_3ADDR_LEN + ie_offset)
- return _FAIL;
+ return;
pstat = rtw_get_stainfo(pstapriv, GetAddr2Ptr(pframe));
if (pstat == (struct sta_info *)NULL) {
report_add_sta_event(padapter, pstat->hwaddr, pstat->aid);
}
- return _SUCCESS;
+ return;
asoc_class2_error:
issue_deauth(padapter, (void *)GetAddr2Ptr(pframe), status);
- return _FAIL;
+ return;
OnAssocReqFail:
else
issue_asocrsp(padapter, status, pstat, WIFI_REASSOCRSP);
- return _FAIL;
+ return;
}
-unsigned int OnAssocRsp(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnAssocRsp(struct adapter *padapter, struct recv_frame *precv_frame)
{
struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)precv_frame->rx_data;
uint i;
/* check A1 matches or not */
if (memcmp(myid(&padapter->eeprompriv), mgmt->da, ETH_ALEN))
- return _SUCCESS;
+ return;
if (!(pmlmeinfo->state & (WIFI_FW_AUTH_SUCCESS | WIFI_FW_ASSOC_STATE)))
- return _SUCCESS;
+ return;
if (pmlmeinfo->state & WIFI_FW_ASSOC_SUCCESS)
- return _SUCCESS;
+ return;
_cancel_timer_ex(&pmlmeext->link_timer);
report_assoc_result:
report_join_res(padapter, res);
-
- return _SUCCESS;
}
-unsigned int OnDeAuth(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnDeAuth(struct adapter *padapter, struct recv_frame *precv_frame)
{
+ struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)precv_frame->rx_data;
unsigned short reason;
struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
- u8 *pframe = precv_frame->rx_data;
struct wifidirect_info *pwdinfo = &padapter->wdinfo;
- /* check A3 */
- if (!(!memcmp(GetAddr3Ptr(pframe), get_my_bssid(&pmlmeinfo->network), ETH_ALEN)))
- return _SUCCESS;
+ if (memcmp(mgmt->bssid, get_my_bssid(&pmlmeinfo->network), ETH_ALEN))
+ return;
if (pwdinfo->rx_invitereq_info.scan_op_ch_only) {
_cancel_timer_ex(&pwdinfo->reset_ch_sitesurvey);
_set_timer(&pwdinfo->reset_ch_sitesurvey, 10);
}
- reason = le16_to_cpu(*(__le16 *)(pframe + WLAN_HDR_A3_LEN));
+ reason = le16_to_cpu(mgmt->u.disassoc.reason_code);
if (check_fwstate(pmlmepriv, WIFI_AP_STATE)) {
struct sta_info *psta;
struct sta_priv *pstapriv = &padapter->stapriv;
+ u8 updated = 0;
- psta = rtw_get_stainfo(pstapriv, GetAddr2Ptr(pframe));
- if (psta) {
- u8 updated = 0;
-
- spin_lock_bh(&pstapriv->asoc_list_lock);
- if (!list_empty(&psta->asoc_list)) {
- list_del_init(&psta->asoc_list);
- pstapriv->asoc_list_cnt--;
- updated = ap_free_sta(padapter, psta, false, reason);
- }
- spin_unlock_bh(&pstapriv->asoc_list_lock);
+ psta = rtw_get_stainfo(pstapriv, mgmt->sa);
+ if (!psta)
+ return;
- associated_clients_update(padapter, updated);
+ spin_lock_bh(&pstapriv->asoc_list_lock);
+ if (!list_empty(&psta->asoc_list)) {
+ list_del_init(&psta->asoc_list);
+ pstapriv->asoc_list_cnt--;
+ updated = ap_free_sta(padapter, psta, false, reason);
}
+ spin_unlock_bh(&pstapriv->asoc_list_lock);
- return _SUCCESS;
+ associated_clients_update(padapter, updated);
} else {
- int ignore_received_deauth = 0;
+ bool ignore_received_deauth = false;
/* Before sending the auth frame to start the STA/GC mode connection with AP/GO,
* we will send the deauth first.
* However, the Win8.1 with BRCM Wi-Fi will send the deauth with reason code 6 to us after receieving our deauth.
* Added the following code to avoid this case.
*/
- if ((pmlmeinfo->state & WIFI_FW_AUTH_STATE) ||
- (pmlmeinfo->state & WIFI_FW_ASSOC_STATE)) {
+ if (pmlmeinfo->state & (WIFI_FW_AUTH_STATE | WIFI_FW_ASSOC_STATE)) {
if (reason == WLAN_REASON_CLASS2_FRAME_FROM_NONAUTH_STA) {
- ignore_received_deauth = 1;
+ ignore_received_deauth = true;
} else if (reason == WLAN_REASON_PREV_AUTH_NOT_VALID) {
// TODO: 802.11r
- ignore_received_deauth = 1;
+ ignore_received_deauth = true;
}
}
if (!ignore_received_deauth)
- receive_disconnect(padapter, GetAddr3Ptr(pframe), reason);
+ receive_disconnect(padapter, mgmt->bssid, reason);
+
+ pmlmepriv->LinkDetectInfo.bBusyTraffic = false;
}
- pmlmepriv->LinkDetectInfo.bBusyTraffic = false;
- return _SUCCESS;
}
-unsigned int OnDisassoc(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnDisassoc(struct adapter *padapter, struct recv_frame *precv_frame)
{
+ struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)precv_frame->rx_data;
u16 reason;
struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
- u8 *pframe = precv_frame->rx_data;
struct wifidirect_info *pwdinfo = &padapter->wdinfo;
+ struct sta_info *psta;
+ struct sta_priv *pstapriv = &padapter->stapriv;
+ u8 updated = 0;
- /* check A3 */
- if (!(!memcmp(GetAddr3Ptr(pframe), get_my_bssid(&pmlmeinfo->network), ETH_ALEN)))
- return _SUCCESS;
+ if (memcmp(mgmt->bssid, get_my_bssid(&pmlmeinfo->network), ETH_ALEN))
+ return;
if (pwdinfo->rx_invitereq_info.scan_op_ch_only) {
_cancel_timer_ex(&pwdinfo->reset_ch_sitesurvey);
_set_timer(&pwdinfo->reset_ch_sitesurvey, 10);
}
- reason = le16_to_cpu(*(__le16 *)(pframe + WLAN_HDR_A3_LEN));
-
- if (check_fwstate(pmlmepriv, WIFI_AP_STATE)) {
- struct sta_info *psta;
- struct sta_priv *pstapriv = &padapter->stapriv;
-
- psta = rtw_get_stainfo(pstapriv, GetAddr2Ptr(pframe));
- if (psta) {
- u8 updated = 0;
+ reason = le16_to_cpu(mgmt->u.disassoc.reason_code);
- spin_lock_bh(&pstapriv->asoc_list_lock);
- if (!list_empty(&psta->asoc_list)) {
- list_del_init(&psta->asoc_list);
- pstapriv->asoc_list_cnt--;
- updated = ap_free_sta(padapter, psta, false, reason);
- }
- spin_unlock_bh(&pstapriv->asoc_list_lock);
+ if (!check_fwstate(pmlmepriv, WIFI_AP_STATE)) {
+ receive_disconnect(padapter, mgmt->bssid, reason);
+ pmlmepriv->LinkDetectInfo.bBusyTraffic = false;
+ return;
+ }
- associated_clients_update(padapter, updated);
- }
+ psta = rtw_get_stainfo(pstapriv, mgmt->sa);
+ if (!psta)
+ return;
- return _SUCCESS;
- } else {
- receive_disconnect(padapter, GetAddr3Ptr(pframe), reason);
+ spin_lock_bh(&pstapriv->asoc_list_lock);
+ if (!list_empty(&psta->asoc_list)) {
+ list_del_init(&psta->asoc_list);
+ pstapriv->asoc_list_cnt--;
+ updated = ap_free_sta(padapter, psta, false, reason);
}
- pmlmepriv->LinkDetectInfo.bBusyTraffic = false;
- return _SUCCESS;
+ spin_unlock_bh(&pstapriv->asoc_list_lock);
+
+ associated_clients_update(padapter, updated);
}
-unsigned int OnAction_back(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnAction_back(struct adapter *padapter, struct recv_frame *precv_frame)
{
struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)precv_frame->rx_data;
struct sta_info *psta = NULL;
struct recv_reorder_ctrl *preorder_ctrl;
- unsigned char *frame_body;
unsigned short tid;
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
- u8 *pframe = precv_frame->rx_data;
struct sta_priv *pstapriv = &padapter->stapriv;
- /* check RA matches or not */
- if (memcmp(myid(&padapter->eeprompriv), mgmt->da, ETH_ALEN))/* for if1, sta/ap mode */
- return _SUCCESS;
if ((pmlmeinfo->state & 0x03) != WIFI_FW_AP_STATE)
if (!(pmlmeinfo->state & WIFI_FW_ASSOC_SUCCESS))
- return _SUCCESS;
+ return;
psta = rtw_get_stainfo(pstapriv, mgmt->sa);
-
if (!psta)
- return _SUCCESS;
-
- frame_body = (unsigned char *)(pframe + sizeof(struct ieee80211_hdr_3addr));
+ return;
if (!pmlmeinfo->HT_enable)
- return _SUCCESS;
+ return;
/* All union members start with an action code, it's ok to use addba_req. */
switch (mgmt->u.action.u.addba_req.action_code) {
case WLAN_ACTION_ADDBA_REQ:
- memcpy(&pmlmeinfo->ADDBA_req, &frame_body[2], sizeof(struct ADDBA_request));
tid = u16_get_bits(le16_to_cpu(mgmt->u.action.u.addba_req.capab),
IEEE80211_ADDBA_PARAM_TID_MASK);
preorder_ctrl = &psta->recvreorder_ctrl[tid];
preorder_ctrl->indicate_seq = 0xffff;
preorder_ctrl->enable = pmlmeinfo->bAcceptAddbaReq;
-
issue_action_BA(padapter, mgmt->sa, WLAN_ACTION_ADDBA_RESP,
pmlmeinfo->bAcceptAddbaReq ?
- WLAN_STATUS_SUCCESS : WLAN_STATUS_REQUEST_DECLINED);
+ WLAN_STATUS_SUCCESS : WLAN_STATUS_REQUEST_DECLINED, mgmt);
break;
case WLAN_ACTION_ADDBA_RESP:
tid = u16_get_bits(le16_to_cpu(mgmt->u.action.u.addba_resp.capab),
default:
break;
}
-
- return _SUCCESS;
}
static int get_reg_classes_full_count(struct p2p_channels *channel_list)
p2pie[p2pielen++] = 0x09; /* WFA P2P v1.0 */
/* Commented by Albert 20110306 */
- /* According to the P2P Specification, the group negoitation request frame should contain 9 P2P attributes */
+ /* According to the P2P Specification, the group negotiation request frame should contain 9 P2P attributes */
/* 1. P2P Capability */
/* 2. Group Owner Intent */
/* 3. Configuration Timeout */
p2pie[p2pielen++] = 0x09; /* WFA P2P v1.0 */
/* Commented by Albert 20100908 */
- /* According to the P2P Specification, the group negoitation response frame should contain 9 P2P attributes */
+ /* According to the P2P Specification, the group negotiation response frame should contain 9 P2P attributes */
/* 1. Status */
/* 2. P2P Capability */
/* 3. Group Owner Intent */
p2pie[p2pielen++] = 0x09; /* WFA P2P v1.0 */
/* Commented by Albert 20110306 */
- /* According to the P2P Specification, the group negoitation request frame should contain 5 P2P attributes */
+ /* According to the P2P Specification, the group negotiation request frame should contain 5 P2P attributes */
/* 1. Status */
/* 2. P2P Capability */
/* 3. Operating Channel */
dump_mgntframe(padapter, pmgntframe);
}
-static int _issue_probereq_p2p(struct adapter *padapter, u8 *da, int wait_ack)
+inline void issue_probereq_p2p(struct adapter *padapter)
{
- int ret = _FAIL;
struct xmit_frame *pmgntframe;
struct pkt_attrib *pattrib;
unsigned char *pframe;
pmgntframe = alloc_mgtxmitframe(pxmitpriv);
if (!pmgntframe)
- goto exit;
+ return;
/* update attribute */
pattrib = &pmgntframe->attrib;
fctrl = &pwlanhdr->frame_control;
*(fctrl) = 0;
- if (da) {
- memcpy(pwlanhdr->addr1, da, ETH_ALEN);
- memcpy(pwlanhdr->addr3, da, ETH_ALEN);
+ if ((pwdinfo->p2p_info.scan_op_ch_only) || (pwdinfo->rx_invitereq_info.scan_op_ch_only)) {
+ /* This two flags will be set when this is only the P2P client mode. */
+ memcpy(pwlanhdr->addr1, pwdinfo->p2p_peer_interface_addr, ETH_ALEN);
+ memcpy(pwlanhdr->addr3, pwdinfo->p2p_peer_interface_addr, ETH_ALEN);
} else {
- if ((pwdinfo->p2p_info.scan_op_ch_only) || (pwdinfo->rx_invitereq_info.scan_op_ch_only)) {
- /* This two flags will be set when this is only the P2P client mode. */
- memcpy(pwlanhdr->addr1, pwdinfo->p2p_peer_interface_addr, ETH_ALEN);
- memcpy(pwlanhdr->addr3, pwdinfo->p2p_peer_interface_addr, ETH_ALEN);
- } else {
- /* broadcast probe request frame */
- eth_broadcast_addr(pwlanhdr->addr1);
- eth_broadcast_addr(pwlanhdr->addr3);
- }
+ /* broadcast probe request frame */
+ eth_broadcast_addr(pwlanhdr->addr1);
+ eth_broadcast_addr(pwlanhdr->addr3);
}
+
memcpy(pwlanhdr->addr2, mac, ETH_ALEN);
SetSeqNum(pwlanhdr, pmlmeext->mgnt_seq);
pattrib->last_txcmdsz = pattrib->pktlen;
- if (wait_ack) {
- ret = dump_mgntframe_and_wait_ack(padapter, pmgntframe);
- } else {
- dump_mgntframe(padapter, pmgntframe);
- ret = _SUCCESS;
- }
-
-exit:
- return ret;
-}
-
-inline void issue_probereq_p2p(struct adapter *adapter, u8 *da)
-{
- _issue_probereq_p2p(adapter, da, false);
+ dump_mgntframe(padapter, pmgntframe);
}
-static s32 rtw_action_public_decache(struct recv_frame *recv_frame, s32 token)
+static s32 rtw_action_public_decache(struct recv_frame *recv_frame, u8 token)
{
struct adapter *adapter = recv_frame->adapter;
struct mlme_ext_priv *mlmeext = &adapter->mlmeextpriv;
(recv_frame->attrib.frag_num & 0xf);
if (GetRetry(frame)) {
- if (token >= 0) {
- if ((seq_ctrl == mlmeext->action_public_rxseq) &&
- (token == mlmeext->action_public_dialog_token))
- return _FAIL;
- } else {
- if (seq_ctrl == mlmeext->action_public_rxseq)
- return _FAIL;
- }
+ if ((seq_ctrl == mlmeext->action_public_rxseq) &&
+ (token == mlmeext->action_public_dialog_token))
+ return _FAIL;
}
mlmeext->action_public_rxseq = seq_ctrl;
-
- if (token >= 0)
- mlmeext->action_public_dialog_token = token;
-
+ mlmeext->action_public_dialog_token = token;
return _SUCCESS;
}
return _SUCCESS;
}
-static unsigned int on_action_public_vendor(struct recv_frame *precv_frame)
+static void on_action_public_vendor(struct recv_frame *precv_frame)
{
- unsigned int ret = _FAIL;
u8 *pframe = precv_frame->rx_data;
u8 *frame_body = pframe + sizeof(struct ieee80211_hdr_3addr);
- if (!memcmp(frame_body + 2, P2P_OUI, 4)) {
- ret = on_action_public_p2p(precv_frame);
- }
-
- return ret;
+ if (!memcmp(frame_body + 2, P2P_OUI, 4))
+ on_action_public_p2p(precv_frame);
}
-static unsigned int on_action_public_default(struct recv_frame *precv_frame)
+static void on_action_public_default(struct recv_frame *precv_frame)
{
- unsigned int ret = _FAIL;
u8 *pframe = precv_frame->rx_data;
u8 *frame_body = pframe + sizeof(struct ieee80211_hdr_3addr);
u8 token;
token = frame_body[2];
- if (rtw_action_public_decache(precv_frame, token) == _FAIL)
- goto exit;
-
- ret = _SUCCESS;
-
-exit:
- return ret;
+ rtw_action_public_decache(precv_frame, token);
}
-unsigned int on_action_public(struct adapter *padapter, struct recv_frame *precv_frame)
+static void on_action_public(struct adapter *padapter, struct recv_frame *precv_frame)
{
- unsigned int ret = _FAIL;
- u8 *pframe = precv_frame->rx_data;
- u8 *frame_body = pframe + sizeof(struct ieee80211_hdr_3addr);
- u8 category, action;
-
- /* check RA matches or not */
- if (memcmp(myid(&padapter->eeprompriv), GetAddr1Ptr(pframe), ETH_ALEN))
- goto exit;
-
- category = frame_body[0];
- if (category != WLAN_CATEGORY_PUBLIC)
- goto exit;
-
- action = frame_body[1];
- switch (action) {
- case ACT_PUBLIC_VENDOR:
- ret = on_action_public_vendor(precv_frame);
- break;
- default:
- ret = on_action_public_default(precv_frame);
- break;
- }
+ struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)precv_frame->rx_data;
-exit:
- return ret;
+ /* All members of the action enum start with action_code. */
+ if (mgmt->u.action.u.s1g.action_code == WLAN_PUB_ACTION_VENDOR_SPECIFIC)
+ on_action_public_vendor(precv_frame);
+ else
+ on_action_public_default(precv_frame);
}
-unsigned int OnAction_p2p(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnAction_p2p(struct adapter *padapter, struct recv_frame *precv_frame)
{
u8 *frame_body;
- u8 category, OUI_Subtype;
+ u8 OUI_Subtype;
u8 *pframe = precv_frame->rx_data;
uint len = precv_frame->len;
struct wifidirect_info *pwdinfo = &padapter->wdinfo;
- /* check RA matches or not */
- if (memcmp(myid(&padapter->eeprompriv), GetAddr1Ptr(pframe), ETH_ALEN))/* for if1, sta/ap mode */
- return _SUCCESS;
-
frame_body = (unsigned char *)(pframe + sizeof(struct ieee80211_hdr_3addr));
- category = frame_body[0];
- if (category != RTW_WLAN_CATEGORY_P2P)
- return _SUCCESS;
-
if (be32_to_cpu(*((__be32 *)(frame_body + 1))) != P2POUI)
- return _SUCCESS;
+ return;
len -= sizeof(struct ieee80211_hdr_3addr);
OUI_Subtype = frame_body[5];
- switch (OUI_Subtype) {
- case P2P_NOTICE_OF_ABSENCE:
- break;
- case P2P_PRESENCE_REQUEST:
+ if (OUI_Subtype == P2P_PRESENCE_REQUEST)
process_p2p_presence_req(pwdinfo, pframe, len);
- break;
- case P2P_PRESENCE_RESPONSE:
- break;
- case P2P_GO_DISC_REQUEST:
- break;
- default:
- break;
- }
- return _SUCCESS;
}
-unsigned int OnAction(struct adapter *padapter, struct recv_frame *precv_frame)
+static void OnAction(struct adapter *padapter, struct recv_frame *precv_frame)
{
struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)precv_frame->rx_data;
+ if (!ether_addr_equal(myid(&padapter->eeprompriv), mgmt->da))
+ return;
+
switch (mgmt->u.action.category) {
case WLAN_CATEGORY_BACK:
OnAction_back(padapter, precv_frame);
OnAction_p2p(padapter, precv_frame);
break;
}
- return _SUCCESS;
}
struct xmit_frame *alloc_mgtxmitframe(struct xmit_priv *pxmitpriv)
return pmgntframe;
}
+void mgt_dispatcher(struct adapter *padapter, struct recv_frame *precv_frame)
+{
+ mlme_handler mlme_sta_tbl[] = {
+ OnAssocReq,
+ OnAssocRsp,
+ OnAssocReq,
+ OnAssocRsp,
+ OnProbeReq,
+ OnProbeRsp,
+ NULL,
+ NULL,
+ OnBeacon,
+ NULL,
+ OnDisassoc,
+ OnAuthClient,
+ OnDeAuth,
+ OnAction,
+ };
+ int index;
+ mlme_handler fct;
+ struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)precv_frame->rx_data;
+ struct sta_info *psta = rtw_get_stainfo(&padapter->stapriv, hdr->addr2);
+
+ if (!ieee80211_is_mgmt(hdr->frame_control))
+ return;
+
+ /* receive the frames that ra(a1) is my address or ra(a1) is bc address. */
+ if (memcmp(hdr->addr1, myid(&padapter->eeprompriv), ETH_ALEN) &&
+ !is_broadcast_ether_addr(hdr->addr1))
+ return;
+
+ index = (le16_to_cpu(hdr->frame_control) & IEEE80211_FCTL_STYPE) >> 4;
+ if (index >= ARRAY_SIZE(mlme_sta_tbl))
+ return;
+ fct = mlme_sta_tbl[index];
+
+ if (psta) {
+ if (ieee80211_has_retry(hdr->frame_control)) {
+ if (precv_frame->attrib.seq_num == psta->RxMgmtFrameSeqNum)
+ /* drop the duplicate management frame */
+ return;
+ }
+ psta->RxMgmtFrameSeqNum = precv_frame->attrib.seq_num;
+ }
+
+ if (ieee80211_is_auth(hdr->frame_control)) {
+ if (check_fwstate(pmlmepriv, WIFI_AP_STATE))
+ fct = OnAuth;
+ else
+ fct = OnAuthClient;
+ }
+
+ if (fct)
+ fct(padapter, precv_frame);
+}
+
/****************************************************************************
-Following are some TX fuctions for WiFi MLME
+Following are some TX functions for WiFi MLME
*****************************************************************************/
void dump_mgntframe(struct adapter *padapter, struct xmit_frame *pmgntframe)
{
- if (padapter->bSurpriseRemoved || padapter->bDriverStopped)
- return;
-
rtl8188eu_mgnt_xmit(padapter, pmgntframe);
}
u32 timeout_ms = 500;/* 500ms */
struct xmit_priv *pxmitpriv = &padapter->xmitpriv;
- if (padapter->bSurpriseRemoved || padapter->bDriverStopped)
- return -1;
-
mutex_lock(&pxmitpriv->ack_tx_mutex);
pxmitpriv->ack_tx = true;
_issue_probereq(padapter, pssid, da, false);
}
-int issue_probereq_ex(struct adapter *padapter, struct ndis_802_11_ssid *pssid, u8 *da,
- int try_cnt, int wait_ms)
+void issue_probereq_ex(struct adapter *padapter, struct ndis_802_11_ssid *pssid, u8 *da)
{
- int ret;
- int i = 0;
-
- do {
- ret = _issue_probereq(padapter, pssid, da, wait_ms > 0);
-
- i++;
+ int i;
- if (padapter->bDriverStopped || padapter->bSurpriseRemoved)
+ for (i = 0; i < 3; i++) {
+ if (_issue_probereq(padapter, pssid, da, true) == _FAIL)
+ msleep(1);
+ else
break;
-
- if (i < try_cnt && wait_ms > 0 && ret == _FAIL)
- msleep(wait_ms);
-
- } while ((i < try_cnt) && ((ret == _FAIL) || (wait_ms == 0)));
-
- if (ret != _FAIL) {
- ret = _SUCCESS;
- goto exit;
}
-exit:
- return ret;
}
-/* if psta == NULL, indiate we are station(client) now... */
+/* if psta == NULL, indicate we are station (client) now... */
void issue_auth(struct adapter *padapter, struct sta_info *psta, unsigned short status)
{
struct xmit_frame *pmgntframe;
if (!padapter->registrypriv.wifi_spec) {
/* Commented by Kurt 20110629 */
/* In some older APs, WPS handshake */
- /* would be fail if we append vender extensions informations to AP */
+ /* would be fail if we append vendor extension information to AP */
if (!memcmp(pIE->data, WPS_OUI, 4))
pIE->Length = 14;
}
kfree(pmlmepriv->assoc_req);
}
-/* when wait_ack is ture, this function shoule be called at process context */
+/* when wait_ack is true, this function should be called at process context */
static int _issue_nulldata(struct adapter *padapter, unsigned char *da, unsigned int power_mode, int wait_ack)
{
int ret = _FAIL;
return ret;
}
-/* when wait_ms > 0 , this function shoule be called at process context */
+/* when wait_ms > 0, this function should be called at process context */
/* da == NULL for station mode */
int issue_nulldata(struct adapter *padapter, unsigned char *da, unsigned int power_mode, int try_cnt, int wait_ms)
{
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
- /* da == NULL, assum it's null data for sta to ap*/
+ /* da == NULL, assume it's null data for sta to ap*/
if (!da)
da = get_my_bssid(&pmlmeinfo->network);
return ret;
}
-/* when wait_ack is ture, this function shoule be called at process context */
+/* when wait_ack is true, this function should be called at process context */
static int _issue_qos_nulldata(struct adapter *padapter, unsigned char *da, u16 tid, int wait_ack)
{
int ret = _FAIL;
return ret;
}
-/* when wait_ms > 0 , this function shoule be called at process context */
+/* when wait_ms > 0 , this function should be called at process context */
/* da == NULL for station mode */
int issue_qos_nulldata(struct adapter *padapter, unsigned char *da, u16 tid, int try_cnt, int wait_ms)
{
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
- /* da == NULL, assum it's null data for sta to ap*/
+ /* da == NULL, assume it's null data for sta to ap*/
if (!da)
da = get_my_bssid(&pmlmeinfo->network);
return ret;
}
-void issue_action_BA(struct adapter *padapter, unsigned char *raddr, u8 action, u16 status)
+void issue_action_BA(struct adapter *padapter, unsigned char *raddr, u8 action,
+ u16 status, struct ieee80211_mgmt *mgmt_req)
{
u16 start_seq;
u16 BA_starting_seqctrl = 0;
break;
case WLAN_ACTION_ADDBA_RESP:
mgmt->u.action.u.addba_resp.action_code = WLAN_ACTION_ADDBA_RESP;
- mgmt->u.action.u.addba_resp.dialog_token = pmlmeinfo->ADDBA_req.dialog_token;
+ mgmt->u.action.u.addba_resp.dialog_token = mgmt_req->u.action.u.addba_req.dialog_token;
mgmt->u.action.u.addba_resp.status = cpu_to_le16(status);
- capab = le16_to_cpu(pmlmeinfo->ADDBA_req.BA_para_set) & 0x3f;
+ capab = le16_to_cpu(mgmt_req->u.action.u.addba_req.capab) & 0x3f;
capab |= u16_encode_bits(64, IEEE80211_ADDBA_PARAM_BUF_SIZE_MASK);
capab |= u16_encode_bits(pregpriv->ampdu_amsdu, IEEE80211_ADDBA_PARAM_AMSDU_MASK);
mgmt->u.action.u.addba_req.capab = cpu_to_le16(capab);
- mgmt->u.action.u.addba_resp.timeout = pmlmeinfo->ADDBA_req.BA_timeout_value;
+ mgmt->u.action.u.addba_resp.timeout = mgmt_req->u.action.u.addba_req.timeout;
pattrib->pktlen = offsetofend(struct ieee80211_mgmt, u.action.u.addba_resp.timeout);
break;
case WLAN_ACTION_DELBA:
if (initiator == 0) { /* recipient */
for (tid = 0; tid < MAXTID; tid++) {
if (psta->recvreorder_ctrl[tid].enable) {
- issue_action_BA(padapter, addr, WLAN_ACTION_DELBA, (((tid << 1) | initiator) & 0x1F));
+ issue_action_BA(padapter, addr, WLAN_ACTION_DELBA,
+ (((tid << 1) | initiator) & 0x1F), NULL);
psta->recvreorder_ctrl[tid].enable = false;
psta->recvreorder_ctrl[tid].indicate_seq = 0xffff;
}
} else if (initiator == 1) { /* originator */
for (tid = 0; tid < MAXTID; tid++) {
if (psta->htpriv.agg_enable_bitmap & BIT(tid)) {
- issue_action_BA(padapter, addr, WLAN_ACTION_DELBA, (((tid << 1) | initiator) & 0x1F));
+ issue_action_BA(padapter, addr, WLAN_ACTION_DELBA,
+ (((tid << 1) | initiator) & 0x1F), NULL);
psta->htpriv.agg_enable_bitmap &= ~BIT(tid);
psta->htpriv.candidate_tid_bitmap &= ~BIT(tid);
}
/****************************************************************************
-Following are some utitity fuctions for WiFi MLME
+Following are some utility functions for WiFi MLME
*****************************************************************************/
int res;
u8 reg;
- if ((is_client_associated_to_ap(adapter)) ||
+ if ((r8188eu_is_client_associated_to_ap(adapter)) ||
((pmlmeinfo->state & 0x03) == WIFI_FW_ADHOC_STATE)) {
/* enable to rx data frame */
rtw_write16(adapter, REG_RXFLTMAP2, 0xFFFF);
if (ScanType == SCAN_ACTIVE) { /* obey the channel plan setting... */
if (rtw_p2p_chk_state(pwdinfo, P2P_STATE_SCAN) ||
rtw_p2p_chk_state(pwdinfo, P2P_STATE_FIND_PHASE_SEARCH)) {
- issue_probereq_p2p(padapter, NULL);
- issue_probereq_p2p(padapter, NULL);
- issue_probereq_p2p(padapter, NULL);
+ issue_probereq_p2p(padapter);
+ issue_probereq_p2p(padapter);
+ issue_probereq_p2p(padapter);
} else {
int i;
for (i = 0; i < RTW_SSID_SCAN_AMOUNT; i++) {
} else {
/* 20100721:Interrupt scan operation here. */
/* For SW antenna diversity before link, it needs to switch to another antenna and scan again. */
- /* It compares the scan result and select beter one to do connection. */
+ /* It compares the scan result and selects a better one to do connection. */
if (AntDivBeforeLink8188E(padapter)) {
pmlmeext->sitesurvey_res.bss_cnt = 0;
pmlmeext->sitesurvey_res.channel_idx = -1;
Restore_DM_Func_Flag(padapter);
/* Switch_DM_Func(padapter, DYNAMIC_ALL_FUNC_ENABLE, true); */
- if (is_client_associated_to_ap(padapter))
+ if (r8188eu_is_client_associated_to_ap(padapter))
issue_nulldata(padapter, NULL, 0, 3, 500);
rtw_mlme_site_survey_done(padapter);
/* collect bss info from Beacon and Probe request/response frames. */
u8 collect_bss_info(struct adapter *padapter, struct recv_frame *precv_frame, struct wlan_bssid_ex *bssid)
{
+ struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)precv_frame->rx_data;
int i;
u32 len;
u8 *p;
- u16 val16, subtype;
+ u16 val16;
u8 *pframe = precv_frame->rx_data;
u32 packet_len = precv_frame->len;
u8 ie_offset;
memset(bssid, 0, sizeof(struct wlan_bssid_ex));
- subtype = GetFrameSubType(pframe);
-
- if (subtype == WIFI_BEACON) {
+ if (ieee80211_is_beacon(mgmt->frame_control)) {
bssid->Reserved[0] = 1;
ie_offset = _BEACON_IE_OFFSET_;
+ } else if (ieee80211_is_probe_req(mgmt->frame_control)) {
+ ie_offset = _PROBEREQ_IE_OFFSET_;
+ bssid->Reserved[0] = 2;
+ } else if (ieee80211_is_probe_resp(mgmt->frame_control)) {
+ ie_offset = _PROBERSP_IE_OFFSET_;
+ bssid->Reserved[0] = 3;
} else {
- /* FIXME : more type */
- if (subtype == WIFI_PROBEREQ) {
- ie_offset = _PROBEREQ_IE_OFFSET_;
- bssid->Reserved[0] = 2;
- } else if (subtype == WIFI_PROBERSP) {
- ie_offset = _PROBERSP_IE_OFFSET_;
- bssid->Reserved[0] = 3;
- } else {
- bssid->Reserved[0] = 0;
- ie_offset = _FIXED_IE_LENGTH_;
- }
+ bssid->Reserved[0] = 0;
+ ie_offset = _FIXED_IE_LENGTH_;
}
bssid->Length = sizeof(struct wlan_bssid_ex) - MAX_IE_SZ + len;
memcpy(bssid->SupportedRates + i, (p + 2), len);
}
- /* todo: */
- bssid->NetworkTypeInUse = Ndis802_11OFDM24;
-
if (bssid->IELength < 12)
return _FAIL;
/* update wireless mode */
update_wireless_mode(padapter);
- /* udpate capability */
+ /* update capability */
caps = rtw_get_capability((struct wlan_bssid_ex *)pnetwork);
update_capinfo(padapter, caps);
if (caps & cap_IBSS) {/* adhoc master */
/* update wireless mode */
update_wireless_mode(padapter);
- /* udpate capability */
+ /* update capability */
caps = rtw_get_capability((struct wlan_bssid_ex *)pnetwork);
update_capinfo(padapter, caps);
if (caps & cap_ESS) {
pcmd_obj->cmdsz = cmdsz;
pcmd_obj->parmbuf = pevtcmd;
- pcmd_obj->rsp = NULL;
- pcmd_obj->rspsz = 0;
-
pc2h_evt_hdr = (struct C2HEvent_Header *)(pevtcmd);
pc2h_evt_hdr->len = sizeof(struct joinbss_event);
pc2h_evt_hdr->ID = GEN_EVT_CODE(_JoinBss);
/* BCN interval */
rtw_write16(padapter, REG_BCN_INTERVAL, pmlmeinfo->bcn_interval);
- /* udpate capability */
+ /* update capability */
update_capinfo(padapter, pmlmeinfo->capability);
/* WMM, Update EDCA param */
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
- if (is_client_associated_to_ap(padapter) || is_IBSS_empty(padapter)) {
+ if (r8188eu_is_client_associated_to_ap(padapter) || r8188eu_is_ibss_empty(padapter)) {
mlme_disconnect(padapter);
rtw_set_bssid(padapter, null_addr);
rtl8188e_sreset_linked_status_check(padapter);
- if (is_client_associated_to_ap(padapter)) {
+ if (r8188eu_is_client_associated_to_ap(padapter)) {
/* linked infrastructure client mode */
int tx_chk = _SUCCESS, rx_chk = _SUCCESS;
}
if (rx_chk != _SUCCESS)
- issue_probereq_ex(padapter, &pmlmeinfo->network.Ssid, psta->hwaddr, 3, 1);
+ issue_probereq_ex(padapter, &pmlmeinfo->network.Ssid, psta->hwaddr);
if ((tx_chk != _SUCCESS && pmlmeinfo->link_count++ == 0xf) || rx_chk != _SUCCESS) {
tx_chk = issue_nulldata(padapter, psta->hwaddr, 0, 3, 1);
pmlmeinfo->link_count = 0;
}
} /* end of if ((psta = rtw_get_stainfo(pstapriv, passoc_res->network.MacAddress)) != NULL) */
- } else if (is_client_associated_to_ibss(padapter)) {
+ } else if (r8188eu_is_client_associated_to_ibss(padapter)) {
/* linked IBSS mode */
/* for each assoc list entry to check the rx pkt counter */
for (i = IBSS_START_MAC_ID; i < NUM_STA; i++) {
u8 val8;
int res;
- if (is_client_associated_to_ap(padapter))
+ if (r8188eu_is_client_associated_to_ap(padapter))
issue_deauth_ex(padapter, pnetwork->MacAddress, WLAN_REASON_DEAUTH_LEAVING, param->deauth_timeout_ms / 100, 100);
mlme_disconnect(padapter);
pmlmeext->sitesurvey_res.scan_mode = pparm->scan_mode;
/* issue null data if associating to the AP */
- if (is_client_associated_to_ap(padapter)) {
+ if (r8188eu_is_client_associated_to_ap(padapter)) {
pmlmeext->sitesurvey_res.state = SCAN_TXNULL;
issue_nulldata(padapter, NULL, 1, 3, 500);
if (((pmlmeinfo->state & WIFI_FW_ASSOC_SUCCESS) && (pmlmeinfo->HT_enable)) ||
((pmlmeinfo->state & 0x03) == WIFI_FW_AP_STATE)) {
- issue_action_BA(padapter, pparm->addr, WLAN_ACTION_ADDBA_REQ, (u16)pparm->tid);
+ issue_action_BA(padapter, pparm->addr, WLAN_ACTION_ADDBA_REQ, (u16)pparm->tid, NULL);
_set_timer(&psta->addba_retry_timer, ADDBA_TO);
} else {
psta->htpriv.candidate_tid_bitmap &= ~BIT(pparm->tid);
set_channel_bwmode(padapter, pwdinfo->invitereq_info.peer_ch, HAL_PRIME_CHNL_OFFSET_DONT_CARE, HT_CHANNEL_WIDTH_20);
rtw_mlme_under_site_survey(padapter);
- issue_probereq_p2p(padapter, NULL);
+ issue_probereq_p2p(padapter);
_set_timer(&pwdinfo->pre_tx_scan_timer, P2P_TX_PRESCAN_TIMEOUT);
}
set_channel_bwmode(padapter, pwdinfo->tx_prov_disc_info.peer_channel_num[0], HAL_PRIME_CHNL_OFFSET_DONT_CARE, HT_CHANNEL_WIDTH_20);
rtw_mlme_under_site_survey(padapter);
- issue_probereq_p2p(padapter, NULL);
+ issue_probereq_p2p(padapter);
_set_timer(&pwdinfo->pre_tx_scan_timer, P2P_TX_PRESCAN_TIMEOUT);
}
set_channel_bwmode(padapter, pwdinfo->nego_req_info.peer_channel_num[0], HAL_PRIME_CHNL_OFFSET_DONT_CARE, HT_CHANNEL_WIDTH_20);
rtw_mlme_under_site_survey(padapter);
- issue_probereq_p2p(padapter, NULL);
+ issue_probereq_p2p(padapter);
_set_timer(&pwdinfo->pre_tx_scan_timer, P2P_TX_PRESCAN_TIMEOUT);
}
void process_p2p_ps_ie(struct adapter *padapter, u8 *IEs, u32 IELength)
{
- u8 *ies;
- u32 ies_len;
u8 *p2p_ie;
u32 p2p_ielen = 0;
u8 noa_attr[MAX_P2P_IE_LEN] = { 0x00 };/* NoA length should be n*(13) + 2 */
if (rtw_p2p_chk_state(pwdinfo, P2P_STATE_NONE))
return;
- if (IELength <= _BEACON_IE_OFFSET_)
- return;
- ies = IEs + _BEACON_IE_OFFSET_;
- ies_len = IELength - _BEACON_IE_OFFSET_;
-
- p2p_ie = rtw_get_p2p_ie(ies, ies_len, NULL, &p2p_ielen);
+ p2p_ie = rtw_get_p2p_ie(IEs, IELength, NULL, &p2p_ielen);
while (p2p_ie) {
find_p2p = true;
}
/* Get the next P2P IE */
- p2p_ie = rtw_get_p2p_ie(p2p_ie + p2p_ielen, ies_len - (p2p_ie - ies + p2p_ielen), NULL, &p2p_ielen);
+ p2p_ie = rtw_get_p2p_ie(p2p_ie + p2p_ielen, IELength - (p2p_ie - IEs + p2p_ielen), NULL, &p2p_ielen);
}
if (find_p2p) {
if (rtw_p2p_chk_state(pwdinfo, P2P_STATE_TX_PROVISION_DIS_REQ)) {
if (pwdinfo->tx_prov_disc_info.benable) { /* the provision discovery request frame is trigger to send or not */
p2p_protocol_wk_cmd(adapter, P2P_PRE_TX_PROVDISC_PROCESS_WK);
- /* issue_probereq_p2p(adapter, NULL); */
+ /* issue_probereq_p2p(adapter); */
/* _set_timer(&pwdinfo->pre_tx_scan_timer, P2P_TX_PRESCAN_TIMEOUT); */
}
} else if (rtw_p2p_chk_state(pwdinfo, P2P_STATE_GONEGO_ING)) {
err = -1;
break;
}
- msleep(1);
+ mdelay(1);
}
return err;
}
/* check BSSID */
- if (!memcmp(pattrib->bssid, "\x0\x0\x0\x0\x0\x0", ETH_ALEN) ||
- !memcmp(mybssid, "\x0\x0\x0\x0\x0\x0", ETH_ALEN) ||
- (memcmp(pattrib->bssid, mybssid, ETH_ALEN))) {
+ if (is_zero_ether_addr(pattrib->bssid) || is_zero_ether_addr(mybssid) ||
+ (memcmp(pattrib->bssid, mybssid, ETH_ALEN))) {
if (!bmcast)
issue_deauth(adapter, pattrib->bssid, WLAN_REASON_CLASS3_FRAME_FROM_NONASSOC_STA);
if (psta->sleepq_len == 0) {
pstapriv->tim_bitmap &= ~BIT(psta->aid);
- /* upate BCN for TIM IE */
+ /* update BCN for TIM IE */
/* update_BCNTIM(padapter); */
update_beacon(padapter, _TIM_IE_, NULL, false);
}
pstapriv->tim_bitmap &= ~BIT(psta->aid);
- /* upate BCN for TIM IE */
+ /* update BCN for TIM IE */
/* update_BCNTIM(padapter); */
update_beacon(padapter, _TIM_IE_, NULL, false);
}
struct recv_frame *precv_frame)
{
struct sta_info *psta = NULL;
- u8 *ptr = precv_frame->rx_data;
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)precv_frame->rx_data;
struct rx_pkt_attrib *pattrib = &precv_frame->attrib;
struct security_priv *psecuritypriv = &adapter->securitypriv;
if (!psta)
return _FAIL;
- /* psta->rssi = prxcmd->rssi; */
- /* psta->signal_quality = prxcmd->sq; */
precv_frame->psta = psta;
pattrib->amsdu = 0;
pattrib->ack_policy = 0;
/* parsing QC field */
if (pattrib->qos) {
+ struct ieee80211_qos_hdr *qos_hdr = (struct ieee80211_qos_hdr *)hdr;
+
pattrib->priority = ieee80211_get_tid(hdr);
- pattrib->ack_policy = GetAckpolicy((ptr + 24));
- pattrib->amsdu = GetAMsdu((ptr + 24));
- pattrib->hdrlen = 26;
+ pattrib->ack_policy = GetAckpolicy(&qos_hdr->qos_ctrl);
+ pattrib->amsdu = GetAMsdu(&qos_hdr->qos_ctrl);
+ pattrib->hdrlen = sizeof(*qos_hdr);
if (pattrib->priority != 0 && pattrib->priority != 3)
adapter->recvpriv.bIsAnyNonBEPkts = true;
struct recv_priv *precvpriv = &padapter->recvpriv;
struct __queue *pfree_recv_queue = &precvpriv->free_recv_queue;
- int ret = _SUCCESS;
nr_subframes = 0;
prframe->len = 0;
rtw_free_recvframe(prframe, pfree_recv_queue);/* free this recv_frame */
- return ret;
+ return _SUCCESS;
}
static bool check_indicate_seq(struct recv_reorder_ctrl *preorder_ctrl, u16 seq_num)
} else {
if (recvpriv->signal_strength_data.update_req == 0) {/* update_req is clear, means we got rx */
avg_signal_strength = recvpriv->signal_strength_data.avg_val;
- /* after avg_vals are accquired, we can re-stat the signal values */
+ /* after avg_vals are acquired, we can re-stat the signal values */
recvpriv->signal_strength_data.update_req = 1;
}
if (recvpriv->signal_qual_data.update_req == 0) {/* update_req is clear, means we got rx */
avg_signal_qual = recvpriv->signal_qual_data.avg_val;
- /* after avg_vals are accquired, we can re-stat the signal values */
+ /* after avg_vals are acquired, we can re-stat the signal values */
recvpriv->signal_qual_data.update_req = 1;
}
}
-static int aes_cipher(u8 *key, uint hdrlen, u8 *pframe, uint plen)
+static void aes_cipher(u8 *key, uint hdrlen, u8 *pframe, uint plen)
{
uint qc_exists, a4_exists, i, j, payload_remainder,
num_blocks, payload_index;
bitwise_xor(aes_out, padded_buffer, chain_buffer);
for (j = 0; j < 8; j++)
pframe[payload_index++] = chain_buffer[j];
-
- return _SUCCESS;
}
u32 rtw_aes_encrypt(struct adapter *padapter, struct xmit_frame *pxmitframe)
psta->keep_alive_trycnt = 0;
}
-u32 _rtw_init_sta_priv(struct sta_priv *pstapriv)
+int _rtw_init_sta_priv(struct sta_priv *pstapriv)
{
struct sta_info *psta;
s32 i;
pstapriv->pallocated_stainfo_buf = vzalloc(sizeof(struct sta_info) * NUM_STA + 4);
if (!pstapriv->pallocated_stainfo_buf)
- return _FAIL;
+ return -ENOMEM;
pstapriv->pstainfo_buf = pstapriv->pallocated_stainfo_buf + 4 -
((size_t)(pstapriv->pallocated_stainfo_buf) & 3);
pstapriv->expire_to = 3; /* 3*2 = 6 sec */
pstapriv->max_num_sta = NUM_STA;
- return _SUCCESS;
+ return 0;
}
inline int rtw_stainfo_offset(struct sta_priv *stapriv, struct sta_info *sta)
}
/* using pstapriv->sta_hash_lock to protect */
-u32 rtw_free_stainfo(struct adapter *padapter, struct sta_info *psta)
+void rtw_free_stainfo(struct adapter *padapter, struct sta_info *psta)
{
int i;
struct __queue *pfree_sta_queue;
struct sta_priv *pstapriv = &padapter->stapriv;
if (!psta)
- goto exit;
+ return;
pfree_sta_queue = &pstapriv->free_sta_queue;
spin_lock_bh(&pfree_sta_queue->lock);
list_add_tail(&psta->list, get_list_head(pfree_sta_queue));
spin_unlock_bh(&pfree_sta_queue->lock);
-
-exit:
-
- return _SUCCESS;
}
/* free all stainfo which in sta_hash[all] */
if (!hwaddr)
return NULL;
- if (IS_MCAST(hwaddr))
+ if (is_multicast_ether_addr(hwaddr))
addr = bc_addr;
else
addr = hwaddr;
return le16_to_cpu(val);
}
-int is_client_associated_to_ap(struct adapter *padapter)
+bool r8188eu_is_client_associated_to_ap(struct adapter *padapter)
{
struct mlme_ext_priv *pmlmeext;
struct mlme_ext_info *pmlmeinfo;
if (!padapter)
- return _FAIL;
+ return false;
pmlmeext = &padapter->mlmeextpriv;
pmlmeinfo = &pmlmeext->mlmext_info;
if ((pmlmeinfo->state & WIFI_FW_ASSOC_SUCCESS) && ((pmlmeinfo->state & 0x03) == WIFI_FW_STATION_STATE))
return true;
- else
- return _FAIL;
+
+ return false;
}
-int is_client_associated_to_ibss(struct adapter *padapter)
+bool r8188eu_is_client_associated_to_ibss(struct adapter *padapter)
{
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
if ((pmlmeinfo->state & WIFI_FW_ASSOC_SUCCESS) && ((pmlmeinfo->state & 0x03) == WIFI_FW_ADHOC_STATE))
return true;
- else
- return _FAIL;
+
+ return false;
}
-int is_IBSS_empty(struct adapter *padapter)
+bool r8188eu_is_ibss_empty(struct adapter *padapter)
{
unsigned int i;
struct mlme_ext_priv *pmlmeext = &padapter->mlmeextpriv;
for (i = IBSS_START_MAC_ID; i < NUM_STA; i++) {
if (pmlmeinfo->FW_sta_info[i].status == 1)
- return _FAIL;
+ return false;
}
return true;
}
int rtw_check_bcn_info(struct adapter *Adapter, u8 *pframe, u32 packet_len)
{
+ struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)pframe;
unsigned int len;
unsigned char *p;
- unsigned short val16, subtype;
+ unsigned short val16;
struct wlan_network *cur_network = &Adapter->mlmepriv.cur_network;
/* u8 wpa_ie[255], rsn_ie[255]; */
u16 wpa_len = 0, rsn_len = 0;
unsigned short ht_cap_info;
unsigned char ht_info_infos_0;
- if (!is_client_associated_to_ap(Adapter))
+ if (!r8188eu_is_client_associated_to_ap(Adapter))
return true;
len = packet_len - sizeof(struct ieee80211_hdr_3addr);
if (!bssid)
return _FAIL;
- subtype = GetFrameSubType(pframe) >> 4;
-
- if (subtype == WIFI_BEACON)
+ if (ieee80211_is_beacon(mgmt->frame_control))
bssid->Reserved[0] = 1;
bssid->Length = sizeof(struct wlan_bssid_ex) - MAX_IE_SZ + len;
return _FAIL;
}
-void update_beacon_info(struct adapter *padapter, u8 *pframe, uint pkt_len, struct sta_info *psta)
+void update_beacon_info(struct adapter *padapter, u8 *ie_ptr, uint ie_len, struct sta_info *psta)
{
unsigned int i;
- unsigned int len;
struct ndis_802_11_var_ie *pIE;
- len = pkt_len - (_BEACON_IE_OFFSET_ + WLAN_HDR_A3_LEN);
-
- for (i = 0; i < len;) {
- pIE = (struct ndis_802_11_var_ie *)(pframe + (_BEACON_IE_OFFSET_ + WLAN_HDR_A3_LEN) + i);
+ for (i = 0; i < ie_len;) {
+ pIE = (struct ndis_802_11_var_ie *)(ie_ptr + i);
switch (pIE->ElementID) {
case _HT_EXTRA_INFO_IE_: /* HT info */
{
pxmitbuf->pallocated_buf = kzalloc(alloc_sz, GFP_KERNEL);
if (!pxmitbuf->pallocated_buf)
- return _FAIL;
+ return -ENOMEM;
pxmitbuf->pbuf = (u8 *)ALIGN((size_t)(pxmitbuf->pallocated_buf), XMITBUF_ALIGN_SZ);
pxmitbuf->dma_transfer_addr = 0;
pxmitbuf->pxmit_urb = usb_alloc_urb(0, GFP_KERNEL);
if (!pxmitbuf->pxmit_urb) {
kfree(pxmitbuf->pallocated_buf);
- return _FAIL;
+ return -ENOMEM;
}
- return _SUCCESS;
+ return 0;
}
static void rtw_xmit_resource_free(struct adapter *padapter, struct xmit_buf *pxmitbuf,
kfree(pxmitbuf->pallocated_buf);
}
-s32 _rtw_init_xmit_priv(struct xmit_priv *pxmitpriv, struct adapter *padapter)
+int _rtw_init_xmit_priv(struct xmit_priv *pxmitpriv, struct adapter *padapter)
{
int i;
struct xmit_buf *pxmitbuf;
struct xmit_frame *pxframe;
- int res = _SUCCESS;
u32 max_xmit_extbuf_size = MAX_XMIT_EXTBUF_SZ;
u32 num_xmit_extbuf = NR_XMIT_EXTBUFF;
if (!pxmitpriv->pallocated_frame_buf) {
pxmitpriv->pxmit_frame_buf = NULL;
- res = _FAIL;
goto exit;
}
pxmitpriv->pxmit_frame_buf = (u8 *)ALIGN((size_t)(pxmitpriv->pallocated_frame_buf), 4);
pxmitpriv->pallocated_xmitbuf = vzalloc(NR_XMITBUFF * sizeof(struct xmit_buf) + 4);
- if (!pxmitpriv->pallocated_xmitbuf) {
- res = _FAIL;
+ if (!pxmitpriv->pallocated_xmitbuf)
goto free_frame_buf;
- }
pxmitpriv->pxmitbuf = (u8 *)ALIGN((size_t)(pxmitpriv->pallocated_xmitbuf), 4);
/* pxmitpriv->pxmitbuf = pxmitpriv->pallocated_xmitbuf + 4 - */
pxmitbuf->ext_tag = false;
/* Tx buf allocation may fail sometimes, so sleep and retry. */
- res = rtw_xmit_resource_alloc(padapter, pxmitbuf, (MAX_XMITBUF_SZ + XMITBUF_ALIGN_SZ));
- if (res == _FAIL) {
+ if (rtw_xmit_resource_alloc(padapter, pxmitbuf, (MAX_XMITBUF_SZ + XMITBUF_ALIGN_SZ))) {
msleep(10);
- res = rtw_xmit_resource_alloc(padapter, pxmitbuf, (MAX_XMITBUF_SZ + XMITBUF_ALIGN_SZ));
- if (res == _FAIL)
+ if (rtw_xmit_resource_alloc(padapter, pxmitbuf, (MAX_XMITBUF_SZ + XMITBUF_ALIGN_SZ)))
goto free_xmitbuf;
}
pxmitpriv->pallocated_xmit_extbuf = vzalloc(num_xmit_extbuf * sizeof(struct xmit_buf) + 4);
- if (!pxmitpriv->pallocated_xmit_extbuf) {
- res = _FAIL;
+ if (!pxmitpriv->pallocated_xmit_extbuf)
goto free_xmitbuf;
- }
pxmitpriv->pxmit_extbuf = (u8 *)ALIGN((size_t)(pxmitpriv->pallocated_xmit_extbuf), 4);
pxmitbuf->padapter = padapter;
pxmitbuf->ext_tag = true;
- res = rtw_xmit_resource_alloc(padapter, pxmitbuf, max_xmit_extbuf_size + XMITBUF_ALIGN_SZ);
- if (res == _FAIL) {
- res = _FAIL;
+ if (rtw_xmit_resource_alloc(padapter, pxmitbuf, max_xmit_extbuf_size + XMITBUF_ALIGN_SZ))
goto free_xmit_extbuf;
- }
list_add_tail(&pxmitbuf->list, &pxmitpriv->free_xmit_extbuf_queue.queue);
pxmitbuf++;
pxmitpriv->free_xmit_extbuf_cnt = num_xmit_extbuf;
- if (rtw_alloc_hwxmits(padapter)) {
- res = _FAIL;
+ if (rtw_alloc_hwxmits(padapter))
goto free_xmit_extbuf;
- }
rtw_init_hwxmits(pxmitpriv->hwxmits, pxmitpriv->hwxmit_entry);
rtl8188eu_init_xmit_priv(padapter);
- return _SUCCESS;
+ return 0;
free_xmit_extbuf:
pxmitbuf = (struct xmit_buf *)pxmitpriv->pxmit_extbuf;
free_frame_buf:
vfree(pxmitpriv->pallocated_frame_buf);
exit:
- return res;
+ return -ENOMEM;
}
static void rtw_pkt_complete(struct adapter *padapter, struct sk_buff *pkt)
{
uint len;
- len = rtw_remainder_len(pfile);
- len = (rlen > len) ? len : rlen;
+ len = min(rtw_remainder_len(pfile), rlen);
if (rmem)
skb_copy_bits(pfile->pkt, pfile->buf_len - pfile->pkt_len, rmem, len);
spin_lock_bh(&padapter->br_ext_lock);
if (!(skb->data[0] & 1) && br_port &&
memcmp(skb->data + ETH_ALEN, padapter->br_mac, ETH_ALEN) &&
- *((__be16 *)(skb->data + ETH_ALEN * 2)) != __constant_htons(ETH_P_8021Q) &&
- *((__be16 *)(skb->data + ETH_ALEN * 2)) == __constant_htons(ETH_P_IP) &&
+ *((__be16 *)(skb->data + ETH_ALEN * 2)) != htons(ETH_P_8021Q) &&
+ *((__be16 *)(skb->data + ETH_ALEN * 2)) == htons(ETH_P_IP) &&
!memcmp(padapter->scdb_mac, skb->data + ETH_ALEN, ETH_ALEN) && padapter->scdb_entry) {
memcpy(skb->data + ETH_ALEN, GET_MY_HWADDR(padapter), ETH_ALEN);
padapter->scdb_entry->ageing_timer = jiffies;
spin_unlock_bh(&padapter->br_ext_lock);
} else {
- if (*((__be16 *)(skb->data + ETH_ALEN * 2)) == __constant_htons(ETH_P_8021Q)) {
+ if (*((__be16 *)(skb->data + ETH_ALEN * 2)) == htons(ETH_P_8021Q)) {
is_vlan_tag = 1;
vlan_hdr = *((unsigned short *)(skb->data + ETH_ALEN * 2 + 2));
for (i = 0; i < 6; i++)
skb_pull(skb, 4);
}
if (!memcmp(skb->data + ETH_ALEN, padapter->br_mac, ETH_ALEN) &&
- (*((__be16 *)(skb->data + ETH_ALEN * 2)) == __constant_htons(ETH_P_IP)))
+ (*((__be16 *)(skb->data + ETH_ALEN * 2)) == htons(ETH_P_IP)))
memcpy(padapter->br_ip, skb->data + WLAN_ETHHDR_LEN + 12, 4);
- if (*((__be16 *)(skb->data + ETH_ALEN * 2)) == __constant_htons(ETH_P_IP)) {
+ if (*((__be16 *)(skb->data + ETH_ALEN * 2)) == htons(ETH_P_IP)) {
if (memcmp(padapter->scdb_mac, skb->data + ETH_ALEN, ETH_ALEN)) {
padapter->scdb_entry = (struct nat25_network_db_entry *)scdb_findEntry(padapter,
skb->data + WLAN_ETHHDR_LEN + 12);
skb_push(skb, 4);
for (i = 0; i < 6; i++)
*((unsigned short *)(skb->data + i * 2)) = *((unsigned short *)(skb->data + 4 + i * 2));
- *((__be16 *)(skb->data + ETH_ALEN * 2)) = __constant_htons(ETH_P_8021Q);
+ *((__be16 *)(skb->data + ETH_ALEN * 2)) = htons(ETH_P_8021Q);
*((unsigned short *)(skb->data + ETH_ALEN * 2 + 2)) = vlan_hdr;
}
skb_push(skb, 4);
for (i = 0; i < 6; i++)
*((unsigned short *)(skb->data + i * 2)) = *((unsigned short *)(skb->data + 4 + i * 2));
- *((__be16 *)(skb->data + ETH_ALEN * 2)) = __constant_htons(ETH_P_8021Q);
+ *((__be16 *)(skb->data + ETH_ALEN * 2)) = htons(ETH_P_8021Q);
*((unsigned short *)(skb->data + ETH_ALEN * 2 + 2)) = vlan_hdr;
}
}
/*-----------------------------------------------------------------------------
* Function: odm_TxPwrTrackSetPwr88E()
*
- * Overview: 88E change all channel tx power accordign to flag.
+ * Overview: 88E change all channel tx power according to flag.
* OFDM & CCK are all different.
*
* Input: NONE
tmp2 = resulta[c2][i];
}
- diff = (tmp1 > tmp2) ? (tmp1 - tmp2) : (tmp2 - tmp1);
+ diff = abs(tmp1 - tmp2);
if (diff > MAX_TOLERANCE) {
if ((i == 2 || i == 6) && !sim_bitmap) {
if (RegE94 != 0)
patha_fill_iqk(adapt, pathaok, result, final_candidate, (RegEA4 == 0));
-/* To Fix BSOD when final_candidate is 0xff */
-/* by sherry 20120321 */
- if (final_candidate < 4) {
- for (i = 0; i < IQK_Matrix_REG_NUM; i++)
- dm_odm->RFCalibrateInfo.IQKMatrixRegSetting.Value[0][i] = result[final_candidate][i];
- dm_odm->RFCalibrateInfo.IQKMatrixRegSetting.bIQKDone = true;
- }
-
_PHY_SaveADDARegisters(adapt, IQK_BB_REG_92C, dm_odm->RFCalibrateInfo.IQK_BB_backup_recover, 9);
}
#include "../include/drv_types.h"
#include "../include/hal_intf.h"
-uint rtw_hal_init(struct adapter *adapt)
+uint rtw_hal_init(struct adapter *adapt)
{
- uint status = _SUCCESS;
-
adapt->hw_init_completed = false;
- status = rtl8188eu_hal_init(adapt);
+ if (rtl8188eu_hal_init(adapt) != _SUCCESS)
+ return _FAIL;
- if (status == _SUCCESS) {
- adapt->hw_init_completed = true;
+ adapt->hw_init_completed = true;
- if (adapt->registrypriv.notch_filter == 1)
- hal_notch_filter_8188e(adapt, 1);
- } else {
- adapt->hw_init_completed = false;
- }
+ if (adapt->registrypriv.notch_filter == 1)
+ hal_notch_filter_8188e(adapt, 1);
- return status;
+ return _SUCCESS;
}
uint rtw_hal_deinit(struct adapter *adapt)
for (i = 0; i < ODM_ASSOCIATE_ENTRY_NUM; i++) {
pEntry = dm_odm->pODM_StaInfo[i];
if (IS_STA_VALID(pEntry)) {
- /* 2 Caculate RSSI per Antenna */
+ /* 2 Calculate RSSI per Antenna */
Main_RSSI = (dm_fat_tbl->MainAnt_Cnt[i] != 0) ? (dm_fat_tbl->MainAnt_Sum[i] / dm_fat_tbl->MainAnt_Cnt[i]) : 0;
Aux_RSSI = (dm_fat_tbl->AuxAnt_Cnt[i] != 0) ? (dm_fat_tbl->AuxAnt_Sum[i] / dm_fat_tbl->AuxAnt_Cnt[i]) : 0;
TargetAnt = (Main_RSSI >= Aux_RSSI) ? MAIN_ANT : AUX_ANT;
/* 2 Select MaxRSSI for DIG */
- LocalMaxRSSI = (Main_RSSI > Aux_RSSI) ? Main_RSSI : Aux_RSSI;
+ LocalMaxRSSI = max(Main_RSSI, Aux_RSSI);
if ((LocalMaxRSSI > AntDivMaxRSSI) && (LocalMaxRSSI < 40))
AntDivMaxRSSI = LocalMaxRSSI;
if (LocalMaxRSSI > MaxRSSI)
else if ((dm_fat_tbl->RxIdleAnt == AUX_ANT) && (Aux_RSSI == 0))
Aux_RSSI = Main_RSSI;
- LocalMinRSSI = (Main_RSSI > Aux_RSSI) ? Aux_RSSI : Main_RSSI;
+ LocalMinRSSI = min(Main_RSSI, Aux_RSSI);
if (LocalMinRSSI < MinRSSI) {
MinRSSI = LocalMinRSSI;
RxIdleAnt = TargetAnt;
if (PwrState) {
rtw_write8(pAdapter, REG_EFUSE_ACCESS, EFUSE_ACCESS_ON);
- /* 1.2V Power: From VDDON with Power Cut(0x0000h[15]), defualt valid */
+ /* 1.2V Power: From VDDON with Power Cut(0x0000h[15]), default valid */
res = rtw_read16(pAdapter, REG_SYS_ISO_CTRL, &tmpV16);
if (res)
return;
return status;
}
-void
-Hal_EfuseParseIDCode88E(
- struct adapter *padapter,
- u8 *hwinfo
- )
+void Hal_EfuseParseIDCode88E(struct adapter *padapter, u8 *hwinfo)
{
struct eeprom_priv *pEEPROM = &padapter->eeprompriv;
struct net_device *netdev = padapter->pnetdev;
/**
* Function: PHY_QueryBBReg
*
-* OverView: Read "sepcific bits" from BB register
+* Overview: Read "sepcific bits" from BB register
*
* Input:
* struct adapter *Adapter,
/**
* Function: PHY_SetBBReg
*
-* OverView: Write "Specific bits" to BB register (page 8~)
+* Overview: Write "Specific bits" to BB register (page 8~)
*
* Input:
* struct adapter *Adapter,
/**
* Function: phy_RFSerialRead
*
-* OverView: Read regster from RF chips
+* Overview: Read register from RF chips
*
* Input:
* struct adapter *Adapter,
/**
* Function: phy_RFSerialWrite
*
-* OverView: Write data to RF register (page 8~)
+* Overview: Write data to RF register (page 8~)
*
* Input:
* struct adapter *Adapter,
/**
* Function: PHY_QueryRFReg
*
-* OverView: Query "Specific bits" to RF register (page 8~)
+* Overview: Query "Specific bits" to RF register (page 8~)
*
* Input:
* struct adapter *Adapter,
/**
* Function: PHY_SetRFReg
*
-* OverView: Write "Specific bits" to RF register (page 8~)
+* Overview: Write "Specific bits" to RF register (page 8~)
*
* Input:
* struct adapter *Adapter,
/**
* Function: phy_InitBBRFRegisterDefinition
*
-* OverView: Initialize Register definition offset for Radio Path A/B/C/D
+* Overview: Initialize Register definition offset for Radio Path A/B/C/D
*
* Input:
* struct adapter *Adapter,
/* RF Interface (Output and) Enable */
pHalData->PHYRegDef.rfintfe = rFPGA0_XA_RFInterfaceOE; /* 16 MSBs if read 32-bit from 0x860 (16-bit for 0x862) */
- /* Addr of LSSI. Wirte RF register by driver */
+ /* Addr of LSSI. Write RF register by driver */
pHalData->PHYRegDef.rf3wireOffset = rFPGA0_XA_LSSIParameter; /* LSSI Parameter */
/* RF parameter */
if (pattrib->pkt_rpt_type == NORMAL_RX) {
pattrib->pkt_len = le32_to_cpu(prxstat->rxdw0) & 0x00003fff;
+ pattrib->icv_err = (le32_to_cpu(prxstat->rxdw0) >> 15) & 0x1;
pattrib->drvinfo_sz = ((le32_to_cpu(prxstat->rxdw0) >> 16) & 0xf) * 8;
-
+ pattrib->encrypt = (u8)((le32_to_cpu(prxstat->rxdw0) >> 20) & 0x7);
+ pattrib->qos = (le32_to_cpu(prxstat->rxdw0) >> 23) & 0x1;
+ pattrib->shift_sz = (le32_to_cpu(prxstat->rxdw0) >> 24) & 0x3;
pattrib->physt = (le32_to_cpu(prxstat->rxdw0) >> 26) & 0x1;
-
pattrib->bdecrypted = (le32_to_cpu(prxstat->rxdw0) & BIT(27)) ? 0 : 1;
- pattrib->encrypt = (le32_to_cpu(prxstat->rxdw0) >> 20) & 0x7;
- pattrib->qos = (le32_to_cpu(prxstat->rxdw0) >> 23) & 0x1;
pattrib->priority = (le32_to_cpu(prxstat->rxdw1) >> 8) & 0xf;
-
pattrib->amsdu = (le32_to_cpu(prxstat->rxdw1) >> 13) & 0x1;
+ pattrib->mdata = (le32_to_cpu(prxstat->rxdw1) >> 26) & 0x1;
+ pattrib->mfrag = (le32_to_cpu(prxstat->rxdw1) >> 27) & 0x1;
pattrib->seq_num = le32_to_cpu(prxstat->rxdw2) & 0x00000fff;
pattrib->frag_num = (le32_to_cpu(prxstat->rxdw2) >> 12) & 0xf;
- pattrib->mfrag = (le32_to_cpu(prxstat->rxdw1) >> 27) & 0x1;
- pattrib->mdata = (le32_to_cpu(prxstat->rxdw1) >> 26) & 0x1;
pattrib->mcs_rate = le32_to_cpu(prxstat->rxdw3) & 0x3f;
pattrib->rxht = (le32_to_cpu(prxstat->rxdw3) >> 6) & 0x1;
- pattrib->icv_err = (le32_to_cpu(prxstat->rxdw0) >> 15) & 0x1;
- pattrib->shift_sz = (le32_to_cpu(prxstat->rxdw0) >> 24) & 0x3;
} else if (pattrib->pkt_rpt_type == TX_REPORT1) { /* CCX */
pattrib->pkt_len = TX_RPT1_PKT_LEN;
} else if (pattrib->pkt_rpt_type == TX_REPORT2) {
*/
void update_recvframe_phyinfo_88e(struct recv_frame *precvframe, struct phy_stat *pphy_status)
{
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)precvframe->rx_data;
struct adapter *padapter = precvframe->adapter;
struct rx_pkt_attrib *pattrib = &precvframe->attrib;
struct hal_data_8188e *pHalData = &padapter->haldata;
struct phy_info *pPHYInfo = &pattrib->phy_info;
u8 *wlanhdr = precvframe->rx_data;
- __le16 fc = *(__le16 *)wlanhdr;
struct odm_per_pkt_info pkt_info;
u8 *sa = NULL;
struct sta_priv *pstapriv;
struct sta_info *psta;
- pkt_info.bPacketMatchBSSID = ((!ieee80211_is_ctl(fc)) &&
+ pkt_info.bPacketMatchBSSID = ((!ieee80211_is_ctl(hdr->frame_control)) &&
!pattrib->icv_err && !pattrib->crc_err &&
!memcmp(get_hdr_bssid(wlanhdr),
get_bssid(&padapter->mlmepriv), ETH_ALEN));
pkt_info.bPacketToSelf = pkt_info.bPacketMatchBSSID &&
- (!memcmp(get_da(wlanhdr),
- myid(&padapter->eeprompriv), ETH_ALEN));
+ ether_addr_equal(ieee80211_get_DA(hdr),
+ myid(&padapter->eeprompriv));
- pkt_info.bPacketBeacon = pkt_info.bPacketMatchBSSID && ieee80211_is_beacon(fc);
+ pkt_info.bPacketBeacon = pkt_info.bPacketMatchBSSID &&
+ ieee80211_is_beacon(hdr->frame_control);
if (pkt_info.bPacketBeacon) {
if (check_fwstate(&padapter->mlmepriv, WIFI_STATION_STATE))
sa = padapter->mlmepriv.cur_network.network.MacAddress;
/* to do Ad-hoc */
} else {
- sa = get_sa(wlanhdr);
+ sa = ieee80211_get_SA(hdr);
}
pstapriv = &padapter->stapriv;
static s32 update_txdesc(struct xmit_frame *pxmitframe, u8 *pmem, s32 sz, u8 bagg_pkt)
{
- int pull = 0;
uint qsel;
u8 data_rate, pwr_status, offset;
struct adapter *adapt = pxmitframe->padapter;
ODM_SetTxAntByTxInfo_88E(&haldata->odmpriv, pmem, pattrib->mac_id);
rtl8188eu_cal_txdesc_chksum(ptxdesc);
- return pull;
+ return 0;
}
/* for non-agg data frame or management frame */
#define rFPGA0_AdDaClockEn 0x888
#define rFPGA0_AnalogParameter4 0x88c
-#define rFPGA0_XA_LSSIReadBack 0x8a0 /* Tranceiver LSSI Readback */
+#define rFPGA0_XA_LSSIReadBack 0x8a0 /* Transceiver LSSI Readback */
#define rFPGA0_XB_LSSIReadBack 0x8a4
#define rFPGA0_XC_LSSIReadBack 0x8a8
#define rFPGA0_XD_LSSIReadBack 0x8ac
/* RxIQ DC offset, Rx digital filter, DC notch filter */
#define rOFDM0_XARxAFE 0xc10
-#define rOFDM0_XARxIQImbalance 0xc14 /* RxIQ imblance matrix */
+#define rOFDM0_XARxIQImbalance 0xc14 /* RxIQ imbalance matrix */
#define rOFDM0_XBRxAFE 0xc18
#define rOFDM0_XBRxIQImbalance 0xc1c
#define rOFDM0_XCRxAFE 0xc20
s32 bDriverStopped;
s32 bSurpriseRemoved;
- s32 bCardDisableWOHSM;
u8 hw_init_completed;
s8 signal_strength;
#define HP_THERMAL_NUM 8
#define AVG_THERMAL_NUM 8
-#define IQK_Matrix_REG_NUM 8
struct odm_phy_dbg_info {
/* ODM Write,debug info */
ODM_BB_PWR_TRA = BIT(8),
};
-# define ODM_ITRF_USB 0x2
+#define ODM_ITRF_USB 0x2
+#define ODM_CE 0x04
/* ODM_CMNINFO_WM_MODE */
enum odm_wireless_mode {
u8 PTSmoothFactor;
};
-struct ijk_matrix_regs_set {
- bool bIQKDone;
- s32 Value[1][IQK_Matrix_REG_NUM];
-};
-
struct odm_rf_cal {
/* for tx power tracking */
u32 RegA24; /* for TempCCK */
u8 ThermalValue_HP[HP_THERMAL_NUM];
u8 ThermalValue_HP_index;
- struct ijk_matrix_regs_set IQKMatrixRegSetting;
u8 Delta_IQK;
u8 Delta_LCK;
#define MAIN_ANT_CGCS_RX 0
#define AUX_ANT_CGCS_RX 1
+#define SET_TX_DESC_ANTSEL_A_88E(__ptxdesc, __value) \
+ le32p_replace_bits((__le32 *)(__ptxdesc + 8), __value, BIT(24))
+#define SET_TX_DESC_ANTSEL_B_88E(__ptxdesc, __value) \
+ le32p_replace_bits((__le32 *)(__ptxdesc + 8), __value, BIT(25))
+#define SET_TX_DESC_ANTSEL_C_88E(__ptxdesc, __value) \
+ le32p_replace_bits((__le32 *)(__ptxdesc + 28), __value, BIT(29))
+
void ODM_AntennaDiversityInit_88E(struct odm_dm_struct *pDM_Odm);
void ODM_AntennaDiversity_88E(struct odm_dm_struct *pDM_Odm);
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
-/* Copyright(c) 2007 - 2011 Realtek Corporation. */
-
-#ifndef __ODM_TYPES_H__
-#define __ODM_TYPES_H__
-
-#define ODM_CE 0x04 /* BIT(2) */
-
-#define SET_TX_DESC_ANTSEL_A_88E(__ptxdesc, __value) \
- le32p_replace_bits((__le32 *)(__ptxdesc + 8), __value, BIT(24))
-#define SET_TX_DESC_ANTSEL_B_88E(__ptxdesc, __value) \
- le32p_replace_bits((__le32 *)(__ptxdesc + 8), __value, BIT(25))
-#define SET_TX_DESC_ANTSEL_C_88E(__ptxdesc, __value) \
- le32p_replace_bits((__le32 *)(__ptxdesc + 28), __value, BIT(29))
-
-#endif /* __ODM_TYPES_H__ */
int netdev_close(struct net_device *pnetdev);
u8 rtw_init_drv_sw(struct adapter *padapter);
-u8 rtw_free_drv_sw(struct adapter *padapter);
-u8 rtw_reset_drv_sw(struct adapter *padapter);
+void rtw_free_drv_sw(struct adapter *padapter);
+void rtw_reset_drv_sw(struct adapter *padapter);
-u32 rtw_start_drv_threads(struct adapter *padapter);
+int rtw_start_drv_threads(struct adapter *padapter);
void rtw_stop_drv_threads (struct adapter *padapter);
void rtw_cancel_all_timer(struct adapter *padapter);
return (&(queue->queue));
}
-static inline void _set_timer(struct timer_list *ptimer,u32 delay_time)
+static inline void _set_timer(struct timer_list *ptimer, u32 delay_time)
{
mod_timer(ptimer, jiffies + msecs_to_jiffies(delay_time));
}
netif_tx_queue_stopped(netdev_get_tx_queue(pnetdev, 3));
}
-extern int RTW_STATUS_CODE(int error_code);
+int RTW_STATUS_CODE(int error_code);
void *rtw_malloc2d(int h, int w, int size);
#define FUNC_ADPT_FMT "%s(%s)"
#define FUNC_ADPT_ARG(adapter) __func__, adapter->pnetdev->name
-#define rtw_signal_process(pid, sig) kill_pid(find_vpid((pid)),(sig), 1)
+#define rtw_signal_process(pid, sig) kill_pid(find_vpid((pid)), (sig), 1)
/* Macros for handling unaligned memory accesses */
#include "rtl8188e_xmit.h"
#include "rtl8188e_cmd.h"
#include "rtw_efuse.h"
-#include "odm_types.h"
#include "odm.h"
#include "odm_HWConfig.h"
#include "odm_RegDefine11N.h"
/* 9bytes + 1byt + 5bytes and pre 1byte. */
/* For worst case: */
/* | 2byte|----8bytes----|1byte|--7bytes--| 92D */
-/* PG data exclude header, dummy 7 bytes frome CP test and reserved 1byte. */
+/* PG data exclude header, dummy 7 bytes from CP test and reserved 1byte. */
#define EFUSE_OOB_PROTECT_BYTES_88E 18
#define EFUSE_PROTECT_BYTES_BANK 16
void rtl8188e_EfuseParseChnlPlan(struct adapter *padapter, u8 *hwinfo,
bool AutoLoadFail);
-void Hal_ReadAntennaDiversity88E(struct adapter *pAdapter,u8 *PROMContent,
+void Hal_ReadAntennaDiversity88E(struct adapter *pAdapter, u8 *PROMContent,
bool AutoLoadFail);
-void Hal_ReadThermalMeter_88E(struct adapter * dapter, u8 *PROMContent,
+void Hal_ReadThermalMeter_88E(struct adapter *padapter, u8 *PROMContent,
bool AutoloadFail);
void Hal_EfuseParseXtal_8188E(struct adapter *pAdapter, u8 *hwinfo,
bool AutoLoadFail);
#ifndef __RTL8188E_SPEC_H__
#define __RTL8188E_SPEC_H__
-/* 8192C Regsiter offset definition */
+/* 8192C Register offset definition */
#define HAL_PS_TIMER_INT_DELAY 50 /* 50 microseconds */
#define HAL_92C_NAV_UPPER_UNIT 128 /* micro-second */
#define REG_USB_HRPWM 0xFE58
#define REG_USB_HCPWM 0xFE57
-/* 8192C Regsiter Bit and Content definition */
+/* 8192C Register Bit and Content definition */
/* 0x0000h ~ 0x00FFh System Configuration */
/* 2 SYS_ISO_CTRL */
#define HQSEL_HIQ BIT(5)
/* For normal driver, 0x10C */
-#define _TXDMA_HIQ_MAP(x) (((x)&0x3) << 14)
-#define _TXDMA_MGQ_MAP(x) (((x)&0x3) << 12)
-#define _TXDMA_BKQ_MAP(x) (((x)&0x3) << 10)
-#define _TXDMA_BEQ_MAP(x) (((x)&0x3) << 8 )
-#define _TXDMA_VIQ_MAP(x) (((x)&0x3) << 6 )
-#define _TXDMA_VOQ_MAP(x) (((x)&0x3) << 4 )
+#define _TXDMA_HIQ_MAP(x) (((x) & 0x3) << 14)
+#define _TXDMA_MGQ_MAP(x) (((x) & 0x3) << 12)
+#define _TXDMA_BKQ_MAP(x) (((x) & 0x3) << 10)
+#define _TXDMA_BEQ_MAP(x) (((x) & 0x3) << 8)
+#define _TXDMA_VIQ_MAP(x) (((x) & 0x3) << 6)
+#define _TXDMA_VOQ_MAP(x) (((x) & 0x3) << 4)
#define QUEUE_LOW 1
#define QUEUE_NORMAL 2
#define EEPROM_Default_CrystalCap_88E 0x20
#define EEPROM_Default_ThermalMeter_88E 0x18
-/* New EFUSE deafult value */
+/* New EFUSE default value */
#define EEPROM_DEFAULT_24G_INDEX 0x2D
#define EEPROM_DEFAULT_24G_HT20_DIFF 0X02
#define EEPROM_DEFAULT_24G_OFDM_DIFF 0X04
void sta_info_update(struct adapter *padapter, struct sta_info *psta);
u8 ap_free_sta(struct adapter *padapter, struct sta_info *psta,
bool active, u16 reason);
-int rtw_sta_flush(struct adapter *padapter);
+void rtw_sta_flush(struct adapter *padapter);
void start_ap_mode(struct adapter *padapter);
void stop_ap_mode(struct adapter *padapter);
void update_bmc_sta(struct adapter *padapter);
#include "wlan_bssdef.h"
#include "rtw_rf.h"
-#include "rtw_led.h"
-
-#define C2H_MEM_SZ (16*1024)
#include "osdep_service.h"
#include "ieee80211.h" /* <ieee80211/ieee80211.h> */
-#define FREE_CMDOBJ_SZ 128
-
#define MAX_CMDSZ 1024
#define MAX_RSPSZ 512
#define MAX_EVTSZ 1024
int rtw_cmd_thread(void *context);
-u32 rtw_init_cmd_priv(struct cmd_priv *pcmdpriv);
+int rtw_init_cmd_priv(struct cmd_priv *pcmdpriv);
void rtw_free_cmd_priv(struct cmd_priv *pcmdpriv);
-u32 rtw_init_evt_priv(struct evt_priv *pevtpriv);
+int rtw_init_evt_priv(struct evt_priv *pevtpriv);
void rtw_free_evt_priv(struct evt_priv *pevtpriv);
void rtw_evt_notify_isr(struct evt_priv *pevtpriv);
u8 p2p_protocol_wk_cmd(struct adapter *padapter, int intCmdType);
u8 rfintfs;
};
-struct Tx_Beacon_param
-{
+struct Tx_Beacon_param {
struct wlan_bssid_ex network;
};
mac[0] == 0
==> CMD mode, return H2C_SUCCESS.
- The following condition must be ture under CMD mode
+ The following condition must be true under CMD mode
mac[1] == mac[4], mac[2] == mac[3], mac[0]=mac[5]= 0;
s0 == 0x1234, s1 == 0xabcd, w0 == 0x78563412, w1 == 0x5aa5def7;
s2 == (b1 << 8 | b0);
unsigned char *pbuf;
};
-/*------------------- Below are used for RF/BB tunning ---------------------*/
+/*------------------- Below are used for RF/BB tuning ---------------------*/
struct setantenna_parm {
u8 tx_antset;
};
struct getratable_parm {
- uint rsvd;
+ uint rsvd;
};
struct getratable_rsp {
- u8 ss_ForceUp[NumRates];
- u8 ss_ULevel[NumRates];
- u8 ss_DLevel[NumRates];
- u8 count_judge[NumRates];
+ u8 ss_ForceUp[NumRates];
+ u8 ss_ULevel[NumRates];
+ u8 ss_DLevel[NumRates];
+ u8 count_judge[NumRates];
};
/* to get TX,RX retry count */
};
/*H2C Handler index: 59 */
-struct SetChannelPlan_param
-{
+struct SetChannelPlan_param {
u8 channel_plan;
};
/*H2C Handler index: 60 */
-struct LedBlink_param
-{
+struct LedBlink_param {
struct LED_871x *pLed;
};
/*H2C Handler index: 61 */
-struct SetChannelSwitch_param
-{
+struct SetChannelSwitch_param {
u8 new_ch_no;
};
/*H2C Handler index: 62 */
-struct TDLSoption_param
-{
+struct TDLSoption_param {
u8 addr[ETH_ALEN];
u8 option;
};
#define H2C_CMD_OVERFLOW 0x06
#define H2C_RESERVED 0x07
-u8 rtw_sitesurvey_cmd(struct adapter *padapter, struct ndis_802_11_ssid *ssid, int ssid_num);
-u8 rtw_createbss_cmd(struct adapter *padapter);
+u8 rtw_sitesurvey_cmd(struct adapter *padapter, struct ndis_802_11_ssid *ssid, int ssid_num);
+u8 rtw_createbss_cmd(struct adapter *padapter);
u8 rtw_setstakey_cmd(struct adapter *padapter, u8 *psta, u8 unicast_key);
u8 rtw_clearstakey_cmd(struct adapter *padapter, u8 *psta, u8 entry, u8 enqueue);
-u8 rtw_joinbss_cmd(struct adapter *padapter, struct wlan_network* pnetwork);
+u8 rtw_joinbss_cmd(struct adapter *padapter, struct wlan_network *pnetwork);
u8 rtw_disassoc_cmd(struct adapter *padapter, u32 deauth_timeout_ms, bool enqueue);
-u8 rtw_setopmode_cmd(struct adapter *padapter, enum ndis_802_11_network_infra networktype);
-u8 rtw_setdatarate_cmd(struct adapter *padapter, u8 *rateset);
-u8 rtw_setrfintfs_cmd(struct adapter *padapter, u8 mode);
+u8 rtw_setopmode_cmd(struct adapter *padapter, enum ndis_802_11_network_infra networktype);
+int rtw_setdatarate_cmd(struct adapter *padapter, u8 *rateset);
+u8 rtw_setrfintfs_cmd(struct adapter *padapter, u8 mode);
-u8 rtw_gettssi_cmd(struct adapter *padapter, u8 offset,u8 *pval);
-u8 rtw_setfwdig_cmd(struct adapter*padapter, u8 type);
-u8 rtw_setfwra_cmd(struct adapter*padapter, u8 type);
+u8 rtw_gettssi_cmd(struct adapter *padapter, u8 offset, u8 *pval);
+u8 rtw_setfwdig_cmd(struct adapter *padapter, u8 type);
+u8 rtw_setfwra_cmd(struct adapter *padapter, u8 type);
-u8 rtw_addbareq_cmd(struct adapter*padapter, u8 tid, u8 *addr);
+u8 rtw_addbareq_cmd(struct adapter *padapter, u8 tid, u8 *addr);
u8 rtw_dynamic_chk_wk_cmd(struct adapter *adapter);
-u8 rtw_lps_ctrl_wk_cmd(struct adapter*padapter, u8 lps_ctrl_type, u8 enqueue);
-u8 rtw_rpt_timer_cfg_cmd(struct adapter*padapter, u16 minRptTime);
+u8 rtw_lps_ctrl_wk_cmd(struct adapter *padapter, u8 lps_ctrl_type, u8 enqueue);
+u8 rtw_rpt_timer_cfg_cmd(struct adapter *padapter, u16 minRptTime);
- u8 rtw_antenna_select_cmd(struct adapter*padapter, u8 antenna,u8 enqueue);
-u8 rtw_ps_cmd(struct adapter*padapter);
+u8 rtw_antenna_select_cmd(struct adapter *padapter, u8 antenna, u8 enqueue);
+u8 rtw_ps_cmd(struct adapter *padapter);
-u8 rtw_chk_hi_queue_cmd(struct adapter*padapter);
+u8 rtw_chk_hi_queue_cmd(struct adapter *padapter);
u8 rtw_set_chplan_cmd(struct adapter *padapter, u8 chplan);
#define _SetRFReg_CMD_ _Write_RFREG_CMD_
#ifdef _RTW_CMD_C_
-static struct _cmd_callback rtw_cmd_callback[] =
-{
+static struct _cmd_callback rtw_cmd_callback[] = {
{GEN_CMD_CODE(_Read_MACREG), NULL}, /*0*/
{GEN_CMD_CODE(_Write_MACREG), NULL},
{GEN_CMD_CODE(_Read_BBREG), &rtw_getbbrfreg_cmdrsp_callback},
};
uint ioreq_flush(struct adapter *adapter, struct io_queue *ioqueue);
-void sync_ioreq_enqueue(struct io_req *preq,struct io_queue *ioqueue);
+void sync_ioreq_enqueue(struct io_req *preq, struct io_queue *ioqueue);
uint sync_ioreq_flush(struct adapter *adapter, struct io_queue *ioqueue);
uint free_ioreq(struct io_req *preq, struct io_queue *pio_queue);
struct io_req *alloc_ioreq(struct io_queue *pio_q);
u32 _ioreq2rwmem(struct io_queue *pio_q);
void dev_power_down(struct adapter *Adapter, u8 bpwrup);
-#define PlatformEFIOWrite1Byte(_a,_b,_c) \
- rtw_write8(_a,_b,_c)
-#define PlatformEFIOWrite2Byte(_a,_b,_c) \
- rtw_write16(_a,_b,_c)
-#define PlatformEFIOWrite4Byte(_a,_b,_c) \
- rtw_write32(_a,_b,_c)
-
-#define PlatformEFIORead1Byte(_a,_b) \
- rtw_read8(_a,_b)
-#define PlatformEFIORead2Byte(_a,_b) \
- rtw_read16(_a,_b)
-#define PlatformEFIORead4Byte(_a,_b) \
- rtw_read32(_a,_b)
-
#endif /* _RTL8711_IO_H_ */
u8 rtw_set_802_11_authentication_mode(struct adapter *adapt,
enum ndis_802_11_auth_mode authmode);
-u8 rtw_set_802_11_bssid(struct adapter*adapter, u8 *bssid);
+u8 rtw_set_802_11_bssid(struct adapter *adapter, u8 *bssid);
u8 rtw_set_802_11_add_wep(struct adapter *adapter, struct ndis_802_11_wep *wep);
-u8 rtw_set_802_11_disassociate(struct adapter *adapter);
-u8 rtw_set_802_11_bssid_list_scan(struct adapter*adapter,
+void rtw_set_802_11_disassociate(struct adapter *adapter);
+u8 rtw_set_802_11_bssid_list_scan(struct adapter *adapter,
struct ndis_802_11_ssid *pssid,
int ssid_max_num);
u8 rtw_set_802_11_infrastructure_mode(struct adapter *adapter,
};
struct led_priv {
- struct adapter *padapter;
-
bool bRegUseLed;
enum LED_STATE_871x CurrLedState; /* Current LED state. */
u32 BlinkTimes; /* Number of times to toggle led state for blinking. */
- bool bLedLinkBlinkInProgress;
bool bLedScanBlinkInProgress;
struct delayed_work blink_work;
};
struct profile_info {
u8 ssidlen;
- u8 ssid[ WLAN_SSID_MAXLEN ];
- u8 peermac[ ETH_ALEN ];
+ u8 ssid[WLAN_SSID_MAXLEN];
+ u8 peermac[ETH_ALEN];
};
struct tx_invite_req_info {
u8 token;
u8 benable;
- u8 go_ssid[ WLAN_SSID_MAXLEN ];
+ u8 go_ssid[WLAN_SSID_MAXLEN];
u8 ssidlen;
- u8 go_bssid[ ETH_ALEN ];
- u8 peer_macaddr[ ETH_ALEN ];
+ u8 go_bssid[ETH_ALEN];
+ u8 peer_macaddr[ETH_ALEN];
u8 operating_ch; /* This information will be set by using the
* p2p_set op_ch=x */
u8 peer_ch; /* The listen channel for peer P2P device */
};
struct group_id_info {
- u8 go_device_addr[ ETH_ALEN ]; /* The GO's device address of
+ u8 go_device_addr[ETH_ALEN]; /* The GO's device address of
* this P2P group */
- u8 ssid[ WLAN_SSID_MAXLEN ]; /* The SSID of this P2P group */
+ u8 ssid[WLAN_SSID_MAXLEN]; /* The SSID of this P2P group */
};
struct scan_limit_info {
return false;
}
-static inline int get_fwstate(struct mlme_priv *pmlmepriv)
-{
- return pmlmepriv->fw_state;
-}
-
/*
* No Limit on the calling context,
* therefore set it to be the critical section...
{
pmlmepriv->fw_state |= state;
/* FOR HW integration */
- if (_FW_UNDER_SURVEY==state)
+ if (_FW_UNDER_SURVEY == state)
pmlmepriv->bScanInProcess = true;
}
{
pmlmepriv->fw_state &= ~state;
/* FOR HW integration */
- if (_FW_UNDER_SURVEY==state)
+ if (_FW_UNDER_SURVEY == state)
pmlmepriv->bScanInProcess = false;
}
int rtw_restruct_sec_ie(struct adapter *adapter, u8 *in_ie, u8 *out_ie,
uint in_len);
int rtw_restruct_wmm_ie(struct adapter *adapter, u8 *in_ie, u8 *out_ie,
- uint in_len, uint initial_out_len);
+ uint in_len, uint initial_out_len);
void rtw_init_registrypriv_dev_network(struct adapter *adapter);
void rtw_update_registrypriv_dev_network(struct adapter *adapter);
void _rtw_free_network(struct mlme_priv *pmlmepriv,
struct wlan_network *pnetwork, u8 isfreeall);
-void _rtw_free_network_nolock(struct mlme_priv *pmlmepriv,
- struct wlan_network *pnetwork);
-struct wlan_network* _rtw_find_network(struct __queue *scanned_queue, u8 *addr);
+struct wlan_network *_rtw_find_network(struct __queue *scanned_queue, u8 *addr);
void _rtw_free_network_queue(struct adapter *padapter, u8 isfreeall);
SCAN_STATE_MAX,
};
-typedef unsigned int (*mlme_handler)(struct adapter *adapt, struct recv_frame *frame);
+typedef void (*mlme_handler)(struct adapter *adapt, struct recv_frame *frame);
struct ss_res {
int state;
u8 bwmode_updated;
u8 hidden_ssid_mode;
- struct ADDBA_request ADDBA_req;
struct WMM_para_element WMM_param;
struct HT_caps_element HT_caps;
struct HT_info_element HT_info;
void init_mlme_ext_priv(struct adapter *adapter);
int init_hw_mlme_ext(struct adapter *padapter);
void free_mlme_ext_priv (struct mlme_ext_priv *pmlmeext);
-extern struct xmit_frame *alloc_mgtxmitframe(struct xmit_priv *pxmitpriv);
+struct xmit_frame *alloc_mgtxmitframe(struct xmit_priv *pxmitpriv);
unsigned char networktype_to_raid(unsigned char network_type);
u8 judge_network_type(struct adapter *padapter, unsigned char *rate, int len);
u8 *get_my_bssid(struct wlan_bssid_ex *pnetwork);
u16 get_beacon_interval(struct wlan_bssid_ex *bss);
-int is_client_associated_to_ap(struct adapter *padapter);
-int is_client_associated_to_ibss(struct adapter *padapter);
-int is_IBSS_empty(struct adapter *padapter);
+bool r8188eu_is_client_associated_to_ap(struct adapter *padapter);
+bool r8188eu_is_client_associated_to_ibss(struct adapter *padapter);
+bool r8188eu_is_ibss_empty(struct adapter *padapter);
unsigned char check_assoc_AP(u8 *pframe, uint len);
void ERP_IE_handler(struct adapter *padapter, struct ndis_802_11_var_ie *pIE);
void VCS_update(struct adapter *padapter, struct sta_info *psta);
-void update_beacon_info(struct adapter *padapter, u8 *pframe, uint len,
- struct sta_info *psta);
+void update_beacon_info(struct adapter *padapter, u8 *ie_ptr, uint ie_len, struct sta_info *psta);
int rtw_check_bcn_info(struct adapter *Adapter, u8 *pframe, u32 packet_len);
void update_IOT_info(struct adapter *padapter);
void update_capinfo(struct adapter *adapter, u16 updatecap);
void report_surveydone_event(struct adapter *padapter);
void report_del_sta_event(struct adapter *padapter,
unsigned char *addr, unsigned short reason);
-void report_add_sta_event(struct adapter *padapter, unsigned char* addr,
+void report_add_sta_event(struct adapter *padapter, unsigned char *addr,
int cam_idx);
void beacon_timing_control(struct adapter *padapter);
-extern u8 set_tx_beacon_cmd(struct adapter*padapter);
+u8 set_tx_beacon_cmd(struct adapter *padapter);
unsigned int setup_beacon_frame(struct adapter *padapter,
unsigned char *beacon_frame);
void update_mgnt_tx_rate(struct adapter *padapter, u8 rate);
void issue_p2p_provision_request(struct adapter *padapter, u8 *pssid,
u8 ussidlen, u8 *pdev_raddr);
void issue_p2p_GO_request(struct adapter *padapter, u8 *raddr);
-void issue_probereq_p2p(struct adapter *padapter, u8 *da);
+void issue_probereq_p2p(struct adapter *padapter);
void issue_p2p_invitation_response(struct adapter *padapter, u8 *raddr,
u8 dialogToken, u8 success);
-void issue_p2p_invitation_request(struct adapter *padapter, u8* raddr);
+void issue_p2p_invitation_request(struct adapter *padapter, u8 *raddr);
void issue_beacon(struct adapter *padapter, int timeout_ms);
void issue_probersp(struct adapter *padapter, unsigned char *da,
u8 is_valid_p2p_probereq);
unsigned short status);
void issue_probereq(struct adapter *padapter, struct ndis_802_11_ssid *pssid,
u8 *da);
-s32 issue_probereq_ex(struct adapter *adapter, struct ndis_802_11_ssid *pssid,
- u8* da, int try_cnt, int wait_ms);
+void issue_probereq_ex(struct adapter *padapter, struct ndis_802_11_ssid *pssid, u8 *da);
int issue_nulldata(struct adapter *padapter, unsigned char *da,
unsigned int power_mode, int try_cnt, int wait_ms);
int issue_qos_nulldata(struct adapter *padapter, unsigned char *da,
unsigned short reason);
int issue_deauth_ex(struct adapter *padapter, u8 *da, unsigned short reason,
int try_cnt, int wait_ms);
-void issue_action_BA(struct adapter *padapter, unsigned char *raddr, u8 action, u16 status);
+void issue_action_BA(struct adapter *padapter, unsigned char *raddr, u8 action,
+ u16 status, struct ieee80211_mgmt *mgmt_req);
unsigned int send_delba(struct adapter *padapter, u8 initiator, u8 *addr);
unsigned int send_beacon(struct adapter *padapter);
bool get_beacon_valid_bit(struct adapter *adapter);
void start_clnt_join(struct adapter *padapter);
void start_create_ibss(struct adapter *padapter);
-unsigned int OnAssocReq(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int OnAssocRsp(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int OnProbeReq(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int OnProbeRsp(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int OnBeacon(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int OnDisassoc(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int OnAuth(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int OnAuthClient(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int OnDeAuth(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int OnAction(struct adapter *padapter,
- struct recv_frame *precv_frame);
-
-unsigned int OnAction_back(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int on_action_public(struct adapter *padapter,
- struct recv_frame *precv_frame);
-unsigned int OnAction_p2p(struct adapter *padapter,
- struct recv_frame *precv_frame);
-
void mlmeext_joinbss_event_callback(struct adapter *padapter, int join_res);
void mlmeext_sta_del_event_callback(struct adapter *padapter);
void mlmeext_sta_add_event_callback(struct adapter *padapter,
GEN_EVT_CODE(_Survey), /*8*/
GEN_EVT_CODE(_SurveyDone), /*9*/
- GEN_EVT_CODE(_JoinBss) , /*10*/
+ GEN_EVT_CODE(_JoinBss), /*10*/
GEN_EVT_CODE(_AddSTA),
GEN_EVT_CODE(_DelSTA),
GEN_EVT_CODE(_AtimDone),
u8 privacy; /* in frame_ctrl field */
u8 bdecrypted;
u8 encrypt; /* when 0 indicate no encrypt. when non-zero,
- * indicate the encrypt algorith */
+ * indicate the encrypt algorithm */
u8 iv_len;
u8 icv_len;
u8 crc_err;
u8 *precv_buf; /* 4 alignment */
struct __queue free_recv_buf_queue;
u32 free_recv_buf_queue_cnt;
- /* For display the phy informatiom */
+ /* For display the phy information */
u8 is_signal_dbg; /* for debug */
u8 signal_strength_dbg; /* for debug */
s8 rssi;
u32 last_txcmdsz;
u8 nr_frags;
u8 encrypt; /* when 0 indicate no encrypt. when non-zero,
- * indicate the encrypt algorith */
+ * indicate the encrypt algorithm */
u8 iv_len;
u8 icv_len;
u8 iv[18];
s32 rtw_txframes_sta_ac_pending(struct adapter *padapter,
struct pkt_attrib *pattrib);
void rtw_init_hwxmits(struct hw_xmit *phwxmit, int entry);
-s32 _rtw_init_xmit_priv(struct xmit_priv *pxmitpriv, struct adapter *padapter);
+int _rtw_init_xmit_priv(struct xmit_priv *pxmitpriv, struct adapter *padapter);
void _rtw_free_xmit_priv(struct xmit_priv *pxmitpriv);
int rtw_alloc_hwxmits(struct adapter *padapter);
void rtw_free_hwxmits(struct adapter *padapter);
return x;
}
-extern u32 _rtw_init_sta_priv(struct sta_priv *pstapriv);
-extern void _rtw_free_sta_priv(struct sta_priv *pstapriv);
+int _rtw_init_sta_priv(struct sta_priv *pstapriv);
+void _rtw_free_sta_priv(struct sta_priv *pstapriv);
#define stainfo_offset_valid(offset) (offset < NUM_STA && offset >= 0)
int rtw_stainfo_offset(struct sta_priv *stapriv, struct sta_info *sta);
struct sta_info *rtw_get_stainfo_by_offset(struct sta_priv *stapriv, int off);
-extern struct sta_info *rtw_alloc_stainfo(struct sta_priv *stapriv, u8 *hwaddr);
-extern u32 rtw_free_stainfo(struct adapter *adapt, struct sta_info *psta);
-extern void rtw_free_all_stainfo(struct adapter *adapt);
-extern struct sta_info *rtw_get_stainfo(struct sta_priv *stapriv, u8 *hwaddr);
-extern u32 rtw_init_bcmc_stainfo(struct adapter *adapt);
-extern struct sta_info *rtw_get_bcmc_stainfo(struct adapter *padapter);
-extern u8 rtw_access_ctrl(struct adapter *padapter, u8 *mac_addr);
+struct sta_info *rtw_alloc_stainfo(struct sta_priv *stapriv, u8 *hwaddr);
+void rtw_free_stainfo(struct adapter *adapt, struct sta_info *psta);
+void rtw_free_all_stainfo(struct adapter *adapt);
+struct sta_info *rtw_get_stainfo(struct sta_priv *stapriv, u8 *hwaddr);
+u32 rtw_init_bcmc_stainfo(struct adapter *adapt);
+struct sta_info *rtw_get_bcmc_stainfo(struct adapter *padapter);
+u8 rtw_access_ctrl(struct adapter *padapter, u8 *mac_addr);
#endif /* _STA_INFO_H_ */
#define _PWRMGT_ BIT(12)
#define _MORE_DATA_ BIT(13)
#define _PRIVACY_ BIT(14)
-#define _ORDER_ BIT(15)
#define SetToDs(pbuf) \
*(__le16 *)(pbuf) |= cpu_to_le16(_TO_DS_)
#define SetPrivacy(pbuf) \
*(__le16 *)(pbuf) |= cpu_to_le16(_PRIVACY_)
-#define GetPrivacy(pbuf) \
- (((*(__le16 *)(pbuf)) & cpu_to_le16(_PRIVACY_)) != 0)
-
#define GetFrameType(pbuf) \
(le16_to_cpu(*(__le16 *)(pbuf)) & (BIT(3) | BIT(2)))
*(__le16 *)(pbuf) |= cpu_to_le16(type); \
} while (0)
-#define GetTupleCache(pbuf) \
- (cpu_to_le16(*(unsigned short *)((size_t)(pbuf) + 22)))
-
-#define SetFragNum(pbuf, num) \
- do { \
- *(unsigned short *)((size_t)(pbuf) + 22) = \
- ((*(unsigned short *)((size_t)(pbuf) + 22)) & \
- le16_to_cpu(~(0x000f))) | \
- cpu_to_le16(0x0f & (num)); \
- } while (0)
-
#define SetSeqNum(pbuf, num) \
do { \
*(__le16 *)((size_t)(pbuf) + 22) = \
#define GetAMsdu(pbuf) (((le16_to_cpu(*(__le16 *)pbuf)) >> 7) & 0x1)
-#define SetAMsdu(pbuf, amsdu) \
- *(__le16 *)(pbuf) |= cpu_to_le16((amsdu & 1) << 7)
-
-#define GetTid(pbuf) (le16_to_cpu(*(__le16 *)((size_t)(pbuf) + \
- (((GetToDs(pbuf)<<1) | GetFrDs(pbuf)) == 3 ? \
- 30 : 24))) & 0x000f)
-
#define GetAddr1Ptr(pbuf) ((unsigned char *)((size_t)(pbuf) + 4))
#define GetAddr2Ptr(pbuf) ((unsigned char *)((size_t)(pbuf) + 10))
#define GetAddr4Ptr(pbuf) ((unsigned char *)((size_t)(pbuf) + 24))
-static inline bool IS_MCAST(unsigned char *da)
-{
- return (*da) & 0x01;
-}
-
-static inline unsigned char *get_da(unsigned char *pframe)
-{
- unsigned char *da;
- unsigned int to_fr_ds = (GetToDs(pframe) << 1) | GetFrDs(pframe);
-
- switch (to_fr_ds) {
- case 0x00: /* ToDs=0, FromDs=0 */
- da = GetAddr1Ptr(pframe);
- break;
- case 0x01: /* ToDs=0, FromDs=1 */
- da = GetAddr1Ptr(pframe);
- break;
- case 0x02: /* ToDs=1, FromDs=0 */
- da = GetAddr3Ptr(pframe);
- break;
- default: /* ToDs=1, FromDs=1 */
- da = GetAddr3Ptr(pframe);
- break;
- }
- return da;
-}
-
static inline unsigned char *get_sa(unsigned char *pframe)
{
unsigned char *sa;
Below is the definition for 802.11n
------------------------------------------------------------------------------*/
-#define SetOrderBit(pbuf) \
- do { \
- *(unsigned short *)(pbuf) |= cpu_to_le16(_ORDER_); \
- } while (0)
-
-#define GetOrderBit(pbuf) \
- (((*(unsigned short *)(pbuf)) & le16_to_cpu(_ORDER_)) != 0)
-
/**
* struct rtw_ieee80211_bar - HT Block Ack Request
*
struct AC_param ac_param[4];
} __packed;
-struct ADDBA_request {
- unsigned char action_code;
- unsigned char dialog_token;
- __le16 BA_para_set;
- __le16 BA_timeout_value;
- __le16 BA_starting_seqctrl;
-} __packed;
-
#define MAX_AMPDU_FACTOR_64K 3
/* Spatial Multiplexing Power Save Modes */
#define P2P_WILDCARD_SSID_LEN 7
-/* default value, used when: (1)p2p disabed or (2)p2p enabled
+/* default value, used when: (1)p2p disabled or (2)p2p enabled
* but only do 1 scan phase */
#define P2P_FINDPHASE_EX_NONE 0
/* used when p2p enabled and want to do 1 scan phase and
P2P_STATE_TX_PROVISION_DIS_REQ = 6,
P2P_STATE_RX_PROVISION_DIS_RSP = 7,
P2P_STATE_RX_PROVISION_DIS_REQ = 8,
- /* Doing the group owner negoitation handshake */
+ /* Doing the group owner negotiation handshake */
P2P_STATE_GONEGO_ING = 9,
- /* finish the group negoitation handshake with success */
+ /* finish the group negotiation handshake with success */
P2P_STATE_GONEGO_OK = 10,
- /* finish the group negoitation handshake with failure */
+ /* finish the group negotiation handshake with failure */
P2P_STATE_GONEGO_FAIL = 11,
/* receiving the P2P Inviation request and match with the profile. */
P2P_STATE_RECV_INVITE_REQ_MATCH = 12,
P2P_STATE_RECV_INVITE_REQ_JOIN = 19,
/* recveing the P2P Inviation response with failure */
P2P_STATE_RX_INVITE_RESP_FAIL = 20,
- /* receiving p2p negoitation response with information is not available */
+ /* receiving p2p negotiation response with information is not available */
P2P_STATE_RX_INFOR_NOREADY = 21,
- /* sending p2p negoitation response with information is not available */
+ /* sending p2p negotiation response with information is not available */
P2P_STATE_TX_INFOR_NOREADY = 22,
};
u8 Ssid[32];
};
-enum NDIS_802_11_NETWORK_TYPE {
- Ndis802_11FH,
- Ndis802_11DS,
- Ndis802_11OFDM5,
- Ndis802_11OFDM24,
- Ndis802_11NetworkTypeMax /* dummy upper bound */
-};
-
struct ndis_802_11_config_fh {
u32 Length; /* Length of structure */
u32 HopPattern; /* As defined by 802.11, MSB set */
/* MIC check time, 60 seconds. */
#define MIC_CHECK_TIME 60000000
-struct ndis_802_11_auth_evt {
- struct ndis_802_11_status_ind Status;
- struct ndis_802_11_auth_req Request[1];
-};
-
-struct ndis_802_11_test {
- u32 Length;
- u32 Type;
- union {
- struct ndis_802_11_auth_evt AuthenticationEvent;
- NDIS_802_11_RSSI RssiTrigger;
- } tt;
-};
-
#ifndef Ndis802_11APMode
#define Ndis802_11APMode (Ndis802_11InfrastructureMax+1)
#endif
struct ndis_802_11_ssid Ssid;
u32 Privacy;
NDIS_802_11_RSSI Rssi;/* in dBM,raw data ,get from PHY) */
- enum NDIS_802_11_NETWORK_TYPE NetworkTypeInUse;
struct ndis_802_11_config Configuration;
enum ndis_802_11_network_infra InfrastructureMode;
unsigned char SupportedRates[NDIS_802_11_LENGTH_RATES_EX];
#define NUM_PRE_AUTH_KEY 16
#define NUM_PMKID_CACHE NUM_PRE_AUTH_KEY
-/*
-* WPA2
-*/
-
-struct pmkid_candidate {
- unsigned char BSSID[ETH_ALEN];
- u32 Flags;
-};
-
-struct ndis_802_11_pmkid_list {
- u32 Version; /* Version of the structure */
- u32 NumCandidates; /* No. of pmkid candidates */
- struct pmkid_candidate CandidateList[1];
-};
-
-struct ndis_802_11_auth_encrypt {
- enum ndis_802_11_auth_mode AuthModeSupported;
- enum ndis_802_11_wep_status EncryptStatusSupported;
-};
-
-struct ndis_802_11_cap {
- u32 Length;
- u32 Version;
- u32 NoOfPMKIDs;
- u32 NoOfAuthEncryptPairsSupported;
- struct ndis_802_11_auth_encrypt AuthenticationEncryptionSupported[1];
-};
-
u8 key_2char2num(u8 hch, u8 lch);
u8 key_char2num(u8 ch);
u8 str_2char2num(u8 hch, u8 lch);
struct iw_request_info *info,
union iwreq_data *wrqu, char *extra)
{
- int ret = 0;
struct adapter *padapter = (struct adapter *)rtw_netdev_priv(dev);
struct iw_mlme *mlme = (struct iw_mlme *)extra;
switch (mlme->cmd) {
case IW_MLME_DEAUTH:
- if (!rtw_set_802_11_disassociate(padapter))
- ret = -1;
+ rtw_set_802_11_disassociate(padapter);
break;
case IW_MLME_DISASSOC:
- if (!rtw_set_802_11_disassociate(padapter))
- ret = -1;
+ rtw_set_802_11_disassociate(padapter);
break;
default:
return -EOPNOTSUPP;
}
- return ret;
+ return 0;
}
static int rtw_wx_set_scan(struct net_device *dev, struct iw_request_info *a,
struct iw_request_info *a,
union iwreq_data *wrqu, char *extra)
{
- int i, ret = 0;
+ int i;
struct adapter *padapter = (struct adapter *)rtw_netdev_priv(dev);
u8 datarates[NumRates];
u32 target_rate = wrqu->bitrate.value;
}
}
- if (rtw_setdatarate_cmd(padapter, datarates) != _SUCCESS)
- ret = -1;
-
- return ret;
+ return rtw_setdatarate_cmd(padapter, datarates);
}
static int rtw_wx_get_rate(struct net_device *dev,
u32 peer_channel = 0;
/* Commented by Albert 20110304 */
- /* The input data contains two informations. */
+ /* The input data contains two information. */
/* 1. First information is the MAC address which wants to formate with */
/* 2. Second information is the WPS PINCode or "pbc" string for push button method */
/* Format: 00:E0:4C:00:00:05 */
uint p2pielen = 0, attr_contentlen = 0;
struct tx_invite_req_info *pinvite_req_info = &pwdinfo->invitereq_info;
- /* The input data contains two informations. */
+ /* The input data contains two information items. */
/* 1. First information is the P2P device address which you want to send to. */
/* 2. Second information is the group id which combines with GO's mac address, space and GO's ssid. */
/* Command line sample: iwpriv wlan0 p2p_set invite ="00:11:22:33:44:55 00:E0:4C:00:00:05 DIRECT-xy" */
u8 *p2pie;
uint p2pielen = 0, attr_contentlen = 0;
- /* The input data contains two informations. */
+ /* The input data contains two information items. */
/* 1. First information is the MAC address which wants to issue the provisioning discovery request frame. */
/* 2. Second information is the WPS configuration method which wants to discovery */
/* Format: 00:E0:4C:00:00:05_display */
struct iw_request_info *info,
union iwreq_data *wrqu, char *extra)
{
- int ret = 0;
-
if (!memcmp(extra, "enable =", 7)) {
rtw_wext_p2p_enable(dev, info, wrqu, &extra[7]);
} else if (!memcmp(extra, "setDN =", 6)) {
rtw_p2p_set_persistent(dev, info, wrqu, &extra[11]);
}
- return ret;
+ return 0;
}
static int rtw_p2p_get2(struct net_device *dev,
if ((_VENDOR_SPECIFIC_IE_ == probereq_wpsie[0]) &&
(!memcmp(&probereq_wpsie[2], wps_oui, 4))) {
- cp_sz = probereq_wpsie_len > MAX_WPS_IE_LEN ? MAX_WPS_IE_LEN : probereq_wpsie_len;
+ cp_sz = min(probereq_wpsie_len, MAX_WPS_IE_LEN);
pmlmepriv->wps_probe_req_ie_len = 0;
kfree(pmlmepriv->wps_probe_req_ie);
return pnetdev;
}
-u32 rtw_start_drv_threads(struct adapter *padapter)
+int rtw_start_drv_threads(struct adapter *padapter)
{
- u32 _status = _SUCCESS;
-
padapter->cmdThread = kthread_run(rtw_cmd_thread, padapter, "RTW_CMD_THREAD");
if (IS_ERR(padapter->cmdThread))
- _status = _FAIL;
- else
- /* wait for rtw_cmd_thread() to start running */
- wait_for_completion(&padapter->cmdpriv.start_cmd_thread);
+ return PTR_ERR(padapter->cmdThread);
- return _status;
+ /* wait for rtw_cmd_thread() to start running */
+ wait_for_completion(&padapter->cmdpriv.start_cmd_thread);
+
+ return 0;
}
void rtw_stop_drv_threads(struct adapter *padapter)
pmlmepriv->htpriv.ampdu_enable = false;/* set to disabled */
/* security_priv */
- psecuritypriv->binstallGrpkey = _FAIL;
+ psecuritypriv->binstallGrpkey = false;
psecuritypriv->sw_encrypt = pregistrypriv->software_encrypt;
psecuritypriv->sw_decrypt = pregistrypriv->software_decrypt;
psecuritypriv->dot11AuthAlgrthm = dot11AuthAlgrthm_Open; /* open system */
padapter->bShowGetP2PState = 1;
}
-u8 rtw_reset_drv_sw(struct adapter *padapter)
+void rtw_reset_drv_sw(struct adapter *padapter)
{
struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
padapter->mlmeextpriv.sitesurvey_res.state = SCAN_DISABLE;
rtw_set_signal_stat_timer(&padapter->recvpriv);
-
- return _SUCCESS;
}
u8 rtw_init_drv_sw(struct adapter *padapter)
{
- if ((rtw_init_cmd_priv(&padapter->cmdpriv)) == _FAIL) {
+ if (rtw_init_cmd_priv(&padapter->cmdpriv)) {
dev_err(dvobj_to_dev(padapter->dvobj), "rtw_init_cmd_priv failed\n");
return _FAIL;
}
padapter->cmdpriv.padapter = padapter;
- if ((rtw_init_evt_priv(&padapter->evtpriv)) == _FAIL) {
+ if (rtw_init_evt_priv(&padapter->evtpriv)) {
dev_err(dvobj_to_dev(padapter->dvobj), "rtw_init_evt_priv failed\n");
goto free_cmd_priv;
}
- if (rtw_init_mlme_priv(padapter) == _FAIL) {
+ if (rtw_init_mlme_priv(padapter)) {
dev_err(dvobj_to_dev(padapter->dvobj), "rtw_init_mlme_priv failed\n");
goto free_evt_priv;
}
init_mlme_ext_priv(padapter);
- if (_rtw_init_xmit_priv(&padapter->xmitpriv, padapter) == _FAIL) {
+ if (_rtw_init_xmit_priv(&padapter->xmitpriv, padapter)) {
dev_err(dvobj_to_dev(padapter->dvobj), "_rtw_init_xmit_priv failed\n");
goto free_mlme_ext;
}
goto free_xmit_priv;
}
- if (_rtw_init_sta_priv(&padapter->stapriv) == _FAIL) {
+ if (_rtw_init_sta_priv(&padapter->stapriv)) {
dev_err(dvobj_to_dev(padapter->dvobj), "_rtw_init_sta_priv failed\n");
goto free_recv_priv;
}
_cancel_timer_ex(&padapter->recvpriv.signal_stat_timer);
}
-u8 rtw_free_drv_sw(struct adapter *padapter)
+void rtw_free_drv_sw(struct adapter *padapter)
{
/* we can call rtw_p2p_enable here, but: */
/* 1. rtw_p2p_enable may have IO operation */
/* clear pbuddystruct adapter to avoid access wrong pointer. */
if (padapter->pbuddy_adapter)
padapter->pbuddy_adapter->pbuddy_adapter = NULL;
-
- return _SUCCESS;
}
void netdev_br_init(struct net_device *netdev)
if (!padapter->bup) {
padapter->bDriverStopped = false;
padapter->bSurpriseRemoved = false;
- padapter->bCardDisableWOHSM = false;
status = rtw_hal_init(padapter);
if (status == _FAIL)
netdev_dbg(pnetdev, "MAC Address = %pM\n", pnetdev->dev_addr);
- status = rtw_start_drv_threads(padapter);
- if (status == _FAIL) {
+ if (rtw_start_drv_threads(padapter)) {
pr_info("Initialize driver software resource Failed!\n");
goto netdev_open_error;
}
padapter->bDriverStopped = false;
padapter->bSurpriseRemoved = false;
- padapter->bCardDisableWOHSM = false;
status = rtw_hal_init(padapter);
if (status == _FAIL)
void rtw_ips_pwr_down(struct adapter *padapter)
{
- padapter->bCardDisableWOHSM = true;
padapter->net_closed = true;
rtw_led_control(padapter, LED_CTL_POWER_OFF);
rtw_ips_dev_unload(padapter);
- padapter->bCardDisableWOHSM = false;
}
static void rtw_fifo_cleanup(struct adapter *adapter)
-* merge into drivers/net/wireless/rtllib/rtl8192e
+To-do list:
+
+* merge into drivers/net/wireless/realtek/rtlwifi/rtl8192*
* clean up function naming
+* Correct the coding style according to Linux guidelines; please read the document
+ at https://www.kernel.org/doc/html/latest/process/coding-style.html.
+* Remove unnecessary debugging/printing macros; for those that are still needed
+ use the proper kernel API (pr_debug(), dev_dbg(), netdev_dbg()).
+* Remove dead code such as unusued functions, variables, fields, etc..
+* Use in-kernel API and remove unnecessary wrappers where possible.
+* Fix bugs due to code that sleeps in atomic context.
+* Remove the HAL layer and migrate its functionality into the relevant parts of
+ the driver.
+* Switch to use LIB80211.
+* Switch to use MAC80211.
+* Switch to use CFG80211.
+* Improve the error handling of various functions, particularly those that use
+ existing kernel APIs.
};
-struct log_int_8190 {
- u32 nIMR_COMDOK;
- u32 nIMR_MGNTDOK;
- u32 nIMR_HIGH;
- u32 nIMR_VODOK;
- u32 nIMR_VIDOK;
- u32 nIMR_BEDOK;
- u32 nIMR_BKDOK;
- u32 nIMR_ROK;
- u32 nIMR_RCOK;
- u32 nIMR_TBDOK;
- u32 nIMR_BDOK;
- u32 nIMR_RXFOVW;
-};
-
struct phy_ofdm_rx_status_rxsc_sgien_exintfflag {
u8 reserved:4;
u8 rxsc:2;
u8 acm = pAciAifsn->f.acm;
u8 AcmCtrl = rtl92e_readb(dev, AcmHwCtrl);
- AcmCtrl = AcmCtrl | ((priv->AcmMethod == 2) ? 0x0 : 0x1);
-
if (acm) {
switch (eACI) {
case AC0_BE:
priv->rf_chip = RF_8256;
- if (priv->RegChannelPlan == 0xf)
+ if (priv->reg_chnl_plan == 0xf)
priv->ChannelPlan = priv->eeprom_ChannelPlan;
else
- priv->ChannelPlan = priv->RegChannelPlan;
+ priv->ChannelPlan = priv->reg_chnl_plan;
if (priv->eeprom_vid == 0x1186 && priv->eeprom_did == 0x3304)
priv->CustomerID = RT_CID_DLINK;
priv->ChannelPlan = 0x0;
break;
case EEPROM_CID_Nettronix:
- priv->ScanDelay = 100;
priv->CustomerID = RT_CID_Nettronix;
break;
case EEPROM_CID_Pronet:
start:
rtl92e_reset_desc_ring(dev);
priv->Rf_Mode = RF_OP_By_SW_3wire;
- if (priv->ResetProgress == RESET_TYPE_NORESET) {
+ if (priv->rst_progress == RESET_TYPE_NORESET) {
rtl92e_writeb(dev, ANAPAR, 0x37);
mdelay(500);
}
priv->pFirmware->status = FW_STATUS_0_INIT;
- if (priv->RegRfOff)
- priv->rtllib->rf_power_state = rf_off;
-
ulRegRead = rtl92e_readl(dev, CPU_GEN);
if (priv->pFirmware->status == FW_STATUS_0_INIT)
ulRegRead |= CPU_GEN_SYSTEM_RESET;
}
priv->LoopbackMode = RTL819X_NO_LOOPBACK;
- if (priv->ResetProgress == RESET_TYPE_NORESET) {
+ if (priv->rst_progress == RESET_TYPE_NORESET) {
ulRegRead = rtl92e_readl(dev, CPU_GEN);
if (priv->LoopbackMode == RTL819X_NO_LOOPBACK)
ulRegRead = (ulRegRead & CPU_GEN_NO_LOOPBACK_MSK) |
rtl92e_writeb(dev, ACK_TIMEOUT, 0x30);
- if (priv->ResetProgress == RESET_TYPE_NORESET)
+ if (priv->rst_progress == RESET_TYPE_NORESET)
rtl92e_set_wireless_mode(dev, priv->rtllib->mode);
rtl92e_cam_reset(dev);
{
}
}
- if (priv->ResetProgress == RESET_TYPE_NORESET) {
+ if (priv->rst_progress == RESET_TYPE_NORESET) {
rtStatus = rtl92e_config_phy(dev);
if (!rtStatus) {
netdev_info(dev, "RF Config failed\n");
rtl92e_writeb(dev, 0x87, 0x0);
- if (priv->RegRfOff) {
- rtl92e_set_rf_state(dev, rf_off, RF_CHANGE_BY_SW);
- } else if (priv->rtllib->rf_off_reason > RF_CHANGE_BY_PS) {
+ if (priv->rtllib->rf_off_reason > RF_CHANGE_BY_PS) {
rtl92e_set_rf_state(dev, rf_off, priv->rtllib->rf_off_reason);
} else if (priv->rtllib->rf_off_reason >= RF_CHANGE_BY_IPS) {
rtl92e_set_rf_state(dev, rf_off, priv->rtllib->rf_off_reason);
else
priv->Rf_Mode = RF_OP_By_SW_3wire;
- if (priv->ResetProgress == RESET_TYPE_NORESET) {
+ if (priv->rst_progress == RESET_TYPE_NORESET) {
rtl92e_dm_init_txpower_tracking(dev);
if (priv->IC_Cut >= IC_VersionCut_D) {
}
priv->CCKPresentAttentuation_40Mdefault = 0;
priv->CCKPresentAttentuation_difference = 0;
- priv->CCKPresentAttentuation =
+ priv->cck_present_attn =
priv->CCKPresentAttentuation_20Mdefault;
priv->btxpower_tracking = false;
}
reg = rtl92e_readl(dev, RCR);
if (priv->rtllib->state == RTLLIB_LINKED) {
- if (ieee->IntelPromiscuousModeInfo.bPromiscuousOn)
+ if (ieee->intel_promiscuous_md_info.promiscuous_on)
;
else
priv->ReceiveConfig = reg |= RCR_CBSSID;
if (cb_desc->bHwSec) {
static u8 tmp;
- if (!tmp) {
+ if (!tmp)
tmp = 1;
- }
switch (priv->rtllib->pairwise_key_type) {
case KEY_TYPE_WEP40:
case KEY_TYPE_WEP104:
cb_desc->priority);
pdesc->TxFWInfoSize = sizeof(struct tx_fwinfo_8190pci);
- pdesc->DISFB = cb_desc->bTxDisableRateFallBack;
- pdesc->USERATE = cb_desc->bTxUseDriverAssingedRate;
+ pdesc->DISFB = cb_desc->tx_dis_rate_fallback;
+ pdesc->USERATE = cb_desc->tx_use_drv_assinged_rate;
pdesc->FirstSeg = 1;
pdesc->LastSeg = 1;
if (!reset) {
mdelay(150);
- priv->bHwRfOffAction = 2;
+ priv->hw_rf_off_action = 2;
if (!priv->rtllib->bSupportRemoteWakeUp) {
rtl92e_set_rf_off(dev);
for (i = 0; i < MAX_QUEUE_SIZE; i++)
skb_queue_purge(&priv->rtllib->skb_waitQ[i]);
- for (i = 0; i < MAX_QUEUE_SIZE; i++)
- skb_queue_purge(&priv->rtllib->skb_aggQ[i]);
skb_queue_purge(&priv->skb_queue);
}
{
struct r8192_priv *priv = rtllib_priv(dev);
struct rtllib_device *ieee = priv->rtllib;
- u8 *pMcsRate = ieee->dot11HTOperationalRateSet;
+ u8 *pMcsRate = ieee->dot11ht_oper_rate_set;
u32 ratr_value = 0;
u16 rate_config = 0;
u8 rate_index = 0;
break;
case IEEE_N_24G:
case IEEE_N_5G:
- if (ieee->pHTInfo->peer_mimo_ps == 0) {
+ if (ieee->ht_info->peer_mimo_ps == 0) {
ratr_value &= 0x0007F007;
} else {
if (priv->rf_type == RF_1T2R)
break;
}
ratr_value &= 0x0FFFFFFF;
- if (ieee->pHTInfo->cur_tx_bw40mhz &&
- ieee->pHTInfo->bCurShortGI40MHz)
+ if (ieee->ht_info->cur_tx_bw40mhz &&
+ ieee->ht_info->bCurShortGI40MHz)
ratr_value |= 0x80000000;
- else if (!ieee->pHTInfo->cur_tx_bw40mhz &&
- ieee->pHTInfo->bCurShortGI20MHz)
+ else if (!ieee->ht_info->cur_tx_bw40mhz &&
+ ieee->ht_info->bCurShortGI20MHz)
ratr_value |= 0x80000000;
rtl92e_writel(dev, RATR0+rate_index*4, ratr_value);
rtl92e_writeb(dev, UFWP, 1);
SlotIndex = (priv->SilentResetRxSlotIndex++)%SilentResetRxSoltNum;
- if (priv->RxCounter == RegRxCounter) {
+ if (priv->rx_ctr == RegRxCounter) {
priv->SilentResetRxStuckEvent[SlotIndex] = 1;
for (i = 0; i < SilentResetRxSoltNum; i++)
priv->SilentResetRxStuckEvent[SlotIndex] = 0;
}
- priv->RxCounter = RegRxCounter;
+ priv->rx_ctr = RegRxCounter;
return bStuck;
}
rtStatus = rtl92e_check_bb_and_rf(dev,
(enum hw90_block)eCheckItem,
(enum rf90_radio_path)0);
- if (!rtStatus) {
+ if (!rtStatus)
return rtStatus;
- }
}
rtl92e_set_bb_reg(dev, rFPGA0_RFMOD, bCCKEn|bOFDMEn, 0x0);
_rtl92e_phy_config_bb(dev, BaseBand_Config_PHY_REG);
switch (priv->CurrentChannelBW) {
case HT_CHANNEL_WIDTH_20:
- priv->CCKPresentAttentuation =
+ priv->cck_present_attn =
priv->CCKPresentAttentuation_20Mdefault +
priv->CCKPresentAttentuation_difference;
- if (priv->CCKPresentAttentuation >
+ if (priv->cck_present_attn >
(CCKTxBBGainTableLength-1))
- priv->CCKPresentAttentuation =
+ priv->cck_present_attn =
CCKTxBBGainTableLength-1;
- if (priv->CCKPresentAttentuation < 0)
- priv->CCKPresentAttentuation = 0;
+ if (priv->cck_present_attn < 0)
+ priv->cck_present_attn = 0;
if (priv->rtllib->current_network.channel == 14 &&
!priv->bcck_in_ch14) {
break;
case HT_CHANNEL_WIDTH_20_40:
- priv->CCKPresentAttentuation =
+ priv->cck_present_attn =
priv->CCKPresentAttentuation_40Mdefault +
priv->CCKPresentAttentuation_difference;
- if (priv->CCKPresentAttentuation >
+ if (priv->cck_present_attn >
(CCKTxBBGainTableLength - 1))
- priv->CCKPresentAttentuation =
+ priv->cck_present_attn =
CCKTxBBGainTableLength-1;
- if (priv->CCKPresentAttentuation < 0)
- priv->CCKPresentAttentuation = 0;
+ if (priv->cck_present_attn < 0)
+ priv->cck_present_attn = 0;
if (priv->rtllib->current_network.channel == 14 &&
!priv->bcck_in_ch14) {
enum rt_rf_power_state rf_power_state)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- (&(priv->rtllib->PowerSaveControl));
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ (&priv->rtllib->pwr_save_ctrl);
bool bResult = true;
u8 i = 0, QueueID = 0;
struct rtl8192_tx_ring *ring = NULL;
switch (rf_power_state) {
case rf_on:
if ((priv->rtllib->rf_power_state == rf_off) &&
- RT_IN_PS_LEVEL(pPSC, RT_RF_OFF_LEVL_HALT_NIC)) {
+ RT_IN_PS_LEVEL(psc, RT_RF_OFF_LEVL_HALT_NIC)) {
bool rtstatus;
u32 InitilizeCount = 3;
do {
InitilizeCount--;
- priv->RegRfOff = false;
rtstatus = rtl92e_enable_nic(dev);
} while (!rtstatus && (InitilizeCount > 0));
return false;
}
- RT_CLEAR_PS_LEVEL(pPSC,
+ RT_CLEAR_PS_LEVEL(psc,
RT_RF_OFF_LEVL_HALT_NIC);
} else {
rtl92e_writeb(dev, ANAPAR, 0x37);
mdelay(1);
rtl92e_set_bb_reg(dev, rFPGA0_AnalogParameter1,
0x4, 0x1);
- priv->bHwRfOffAction = 0;
+ priv->hw_rf_off_action = 0;
rtl92e_set_bb_reg(dev, rFPGA0_XA_RFInterfaceOE,
BIT4, 0x1);
i++;
}
- if (i >= MAX_DOZE_WAITING_TIMES_9x) {
+ if (i >= MAX_DOZE_WAITING_TIMES_9x)
break;
- }
}
rtl92e_set_rf_off(dev);
break;
i++;
}
- if (i >= MAX_DOZE_WAITING_TIMES_9x) {
+ if (i >= MAX_DOZE_WAITING_TIMES_9x)
break;
- }
}
- if (pPSC->RegRfPsLevel & RT_RF_OFF_LEVL_HALT_NIC &&
- !RT_IN_PS_LEVEL(pPSC, RT_RF_OFF_LEVL_HALT_NIC)) {
+ if (psc->RegRfPsLevel & RT_RF_OFF_LEVL_HALT_NIC &&
+ !RT_IN_PS_LEVEL(psc, RT_RF_OFF_LEVL_HALT_NIC)) {
rtl92e_disable_nic(dev);
- RT_SET_PS_LEVEL(pPSC, RT_RF_OFF_LEVL_HALT_NIC);
- } else if (!(pPSC->RegRfPsLevel &
+ RT_SET_PS_LEVEL(psc, RT_RF_OFF_LEVL_HALT_NIC);
+ } else if (!(psc->RegRfPsLevel &
RT_RF_OFF_LEVL_HALT_NIC)) {
rtl92e_set_rf_off(dev);
}
bool bResult = false;
if (rf_power_state == priv->rtllib->rf_power_state &&
- priv->bHwRfOffAction == 0) {
+ priv->hw_rf_off_action == 0) {
return bResult;
}
{
u32 ulcommand = 0;
- ulcommand |= BIT31|BIT30;
+ ulcommand |= BIT31 | BIT30;
rtl92e_writel(dev, RWCAM, ulcommand);
}
SECR_value |= SCR_TxUseDK;
}
-
ieee->hwsec_active = 1;
- if ((ieee->pHTInfo->iot_action & HT_IOT_ACT_PURE_N_MODE) || !hwwep) {
+ if ((ieee->ht_info->iot_action & HT_IOT_ACT_PURE_N_MODE) || !hwwep) {
ieee->hwsec_active = 0;
SECR_value &= ~SCR_RxDecEnable;
}
enum rt_rf_power_state rt_state;
rt_state = priv->rtllib->rf_power_state;
- if (priv->rtllib->PowerSaveControl.bInactivePs) {
- if (rt_state == rf_off) {
- if (priv->rtllib->rf_off_reason > RF_CHANGE_BY_IPS) {
- netdev_warn(dev, "%s(): RF is OFF.\n",
- __func__);
- return;
- }
- mutex_lock(&priv->rtllib->ips_mutex);
- rtl92e_ips_leave(dev);
- mutex_unlock(&priv->rtllib->ips_mutex);
+ if (rt_state == rf_off) {
+ if (priv->rtllib->rf_off_reason > RF_CHANGE_BY_IPS) {
+ netdev_warn(dev, "%s(): RF is OFF.\n",
+ __func__);
+ return;
}
+ mutex_lock(&priv->rtllib->ips_mutex);
+ rtl92e_ips_leave(dev);
+ mutex_unlock(&priv->rtllib->ips_mutex);
}
priv->rtllib->is_set_key = true;
if (EntryNo >= TOTAL_CAM_ENTRY) {
}
if (DefaultKey)
- usConfig |= BIT15 | (KeyType<<2);
+ usConfig |= BIT15 | (KeyType << 2);
else
- usConfig |= BIT15 | (KeyType<<2) | KeyIndex;
+ usConfig |= BIT15 | (KeyType << 2) | KeyIndex;
for (i = 0; i < CAM_CONTENT_COUNT; i++) {
TargetCommand = i + CAM_CONTENT_COUNT * EntryNo;
- TargetCommand |= BIT31|BIT16;
+ TargetCommand |= BIT31 | BIT16;
if (i == 0) {
- TargetContent = (u32)(*(MacAddr+0)) << 16 |
- (u32)(*(MacAddr+1)) << 24 |
+ TargetContent = (u32)(*(MacAddr + 0)) << 16 |
+ (u32)(*(MacAddr + 1)) << 24 |
(u32)usConfig;
rtl92e_writel(dev, WCAMI, TargetContent);
rtl92e_writel(dev, RWCAM, TargetCommand);
} else if (i == 1) {
- TargetContent = (u32)(*(MacAddr+2)) |
- (u32)(*(MacAddr+3)) << 8 |
- (u32)(*(MacAddr+4)) << 16 |
- (u32)(*(MacAddr+5)) << 24;
+ TargetContent = (u32)(*(MacAddr + 2)) |
+ (u32)(*(MacAddr + 3)) << 8 |
+ (u32)(*(MacAddr + 4)) << 16 |
+ (u32)(*(MacAddr + 5)) << 24;
rtl92e_writel(dev, WCAMI, TargetContent);
rtl92e_writel(dev, RWCAM, TargetCommand);
} else {
if (KeyContent != NULL) {
rtl92e_writel(dev, WCAMI,
- (u32)(*(KeyContent+i-2)));
+ (u32)(*(KeyContent + i - 2)));
rtl92e_writel(dev, RWCAM, TargetCommand);
udelay(100);
}
u8 cur_slot_time = priv->slot_time;
if ((cap & WLAN_CAPABILITY_SHORT_SLOT_TIME) &&
- (!priv->rtllib->pHTInfo->current_rt2rt_long_slot_time)) {
+ (!priv->rtllib->ht_info->current_rt2rt_long_slot_time)) {
if (cur_slot_time != SHORT_SLOT_TIME) {
slot_time_val = SHORT_SLOT_TIME;
priv->rtllib->SetHwRegHandler(dev,
struct rtllib_device *ieee = priv->rtllib;
struct rtllib_network *net = &ieee->current_network;
- if (ieee->pHTInfo->bCurrentHTSupport)
+ if (ieee->ht_info->bCurrentHTSupport)
HT_update_self_and_peer_setting(ieee, net);
- ieee->pHTInfo->current_rt2rt_long_slot_time = net->bssht.bd_rt2rt_long_slot_time;
- ieee->pHTInfo->RT2RT_HT_Mode = net->bssht.rt2rt_ht_mode;
+ ieee->ht_info->current_rt2rt_long_slot_time = net->bssht.bd_rt2rt_long_slot_time;
+ ieee->ht_info->RT2RT_HT_Mode = net->bssht.rt2rt_ht_mode;
_rtl92e_update_cap(dev, net->capability);
}
tcb_desc->queue_index = BEACON_QUEUE;
tcb_desc->data_rate = 2;
tcb_desc->RATRIndex = 7;
- tcb_desc->bTxDisableRateFallBack = 1;
- tcb_desc->bTxUseDriverAssingedRate = 1;
+ tcb_desc->tx_dis_rate_fallback = 1;
+ tcb_desc->tx_use_drv_assinged_rate = 1;
skb_push(pnewskb, priv->rtllib->tx_headroom);
pdesc = &ring->desc[0];
if (ieee->mode == WIRELESS_MODE_N_24G ||
ieee->mode == WIRELESS_MODE_N_5G) {
- memcpy(ieee->Regdot11HTOperationalRateSet,
- ieee->RegHTSuppRateSet, 16);
- memcpy(ieee->Regdot11TxHTOperationalRateSet,
- ieee->RegHTSuppRateSet, 16);
+ memcpy(ieee->reg_dot11ht_oper_rate_set,
+ ieee->reg_ht_supp_rate_set, 16);
+ memcpy(ieee->reg_dot11tx_ht_oper_rate_set,
+ ieee->reg_ht_supp_rate_set, 16);
} else {
- memset(ieee->Regdot11HTOperationalRateSet, 0, 16);
+ memset(ieee->reg_dot11ht_oper_rate_set, 0, 16);
}
}
void rtl92e_set_wireless_mode(struct net_device *dev, u8 wireless_mode)
{
struct r8192_priv *priv = rtllib_priv(dev);
- u8 bSupportMode = _rtl92e_get_supported_wireless_mode(dev);
+ u8 support_mode = _rtl92e_get_supported_wireless_mode(dev);
if ((wireless_mode == WIRELESS_MODE_AUTO) ||
- ((wireless_mode & bSupportMode) == 0)) {
- if (bSupportMode & WIRELESS_MODE_N_24G) {
+ ((wireless_mode & support_mode) == 0)) {
+ if (support_mode & WIRELESS_MODE_N_24G) {
wireless_mode = WIRELESS_MODE_N_24G;
- } else if (bSupportMode & WIRELESS_MODE_N_5G) {
+ } else if (support_mode & WIRELESS_MODE_N_5G) {
wireless_mode = WIRELESS_MODE_N_5G;
- } else if ((bSupportMode & WIRELESS_MODE_A)) {
+ } else if ((support_mode & WIRELESS_MODE_A)) {
wireless_mode = WIRELESS_MODE_A;
- } else if ((bSupportMode & WIRELESS_MODE_G)) {
+ } else if ((support_mode & WIRELESS_MODE_G)) {
wireless_mode = WIRELESS_MODE_G;
- } else if ((bSupportMode & WIRELESS_MODE_B)) {
+ } else if ((support_mode & WIRELESS_MODE_B)) {
wireless_mode = WIRELESS_MODE_B;
} else {
netdev_info(dev,
if ((wireless_mode == WIRELESS_MODE_N_24G) ||
(wireless_mode == WIRELESS_MODE_N_5G)) {
- priv->rtllib->pHTInfo->bEnableHT = 1;
+ priv->rtllib->ht_info->enable_ht = 1;
} else {
- priv->rtllib->pHTInfo->bEnableHT = 0;
+ priv->rtllib->ht_info->enable_ht = 0;
}
_rtl92e_refresh_support_rate(priv);
}
static int _rtl92e_sta_up(struct net_device *dev, bool is_silent_reset)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- (&priv->rtllib->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ (&priv->rtllib->pwr_save_ctrl);
bool init_status;
- priv->bDriverIsGoingToUnload = false;
priv->bdisable_nic = false;
priv->up = 1;
return -1;
}
- RT_CLEAR_PS_LEVEL(pPSC, RT_RF_OFF_LEVL_HALT_NIC);
+ RT_CLEAR_PS_LEVEL(psc, RT_RF_OFF_LEVL_HALT_NIC);
priv->bfirst_init = false;
if (priv->polling_timer_on == 0)
{
struct r8192_priv *priv = rtllib_priv(dev);
unsigned long flags = 0;
- u8 RFInProgressTimeOut = 0;
+ u8 rf_in_progress_timeout = 0;
if (priv->up == 0)
return -1;
if (priv->rtllib->state == RTLLIB_LINKED)
rtl92e_leisure_ps_leave(dev);
- priv->bDriverIsGoingToUnload = true;
priv->up = 0;
priv->rtllib->ieee_up = 0;
priv->bfirst_after_down = true;
spin_lock_irqsave(&priv->rf_ps_lock, flags);
while (priv->rf_change_in_progress) {
spin_unlock_irqrestore(&priv->rf_ps_lock, flags);
- if (RFInProgressTimeOut > 100) {
+ if (rf_in_progress_timeout > 100) {
spin_lock_irqsave(&priv->rf_ps_lock, flags);
break;
}
mdelay(1);
- RFInProgressTimeOut++;
+ rf_in_progress_timeout++;
spin_lock_irqsave(&priv->rf_ps_lock, flags);
}
priv->rf_change_in_progress = true;
static void _rtl92e_init_priv_constant(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- &priv->rtllib->PowerSaveControl;
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ &priv->rtllib->pwr_save_ctrl;
- pPSC->RegMaxLPSAwakeIntvl = 5;
+ psc->reg_max_lps_awake_intvl = 5;
}
static void _rtl92e_init_priv_variable(struct net_device *dev)
struct r8192_priv *priv = rtllib_priv(dev);
u8 i;
- priv->AcmMethod = eAcmWay2_SW;
priv->dot11_current_preamble_mode = PREAMBLE_AUTO;
priv->rtllib->status = 0;
priv->polling_timer_on = 0;
priv->up_first_time = 1;
priv->blinked_ingpio = false;
- priv->bDriverIsGoingToUnload = false;
priv->being_init_adapter = false;
- priv->initialized_at_probe = false;
priv->bdisable_nic = false;
priv->bfirst_init = false;
priv->txringcount = 64;
priv->rxringcount = MAX_RX_COUNT;
priv->irq_enabled = 0;
priv->chan = 1;
- priv->RegChannelPlan = 0xf;
+ priv->reg_chnl_plan = 0xf;
priv->rtllib->mode = WIRELESS_MODE_AUTO;
priv->rtllib->iw_mode = IW_MODE_INFRA;
- priv->rtllib->bNetPromiscuousMode = false;
- priv->rtllib->IntelPromiscuousModeInfo.bPromiscuousOn = false;
- priv->rtllib->IntelPromiscuousModeInfo.bFilterSourceStationFrame =
+ priv->rtllib->net_promiscuous_md = false;
+ priv->rtllib->intel_promiscuous_md_info.promiscuous_on = false;
+ priv->rtllib->intel_promiscuous_md_info.fltr_src_sta_frame =
false;
priv->rtllib->ieee_up = 0;
priv->retry_rts = DEFAULT_RETRY_RTS;
priv->promisc = (dev->flags & IFF_PROMISC) ? 1 : 0;
priv->bcck_in_ch14 = false;
priv->bfsync_processing = false;
- priv->CCKPresentAttentuation = 0;
+ priv->cck_present_attn = 0;
priv->rfa_txpowertrackingindex = 0;
priv->rfc_txpowertrackingindex = 0;
priv->CckPwEnl = 6;
- priv->ScanDelay = 50;
- priv->ResetProgress = RESET_TYPE_NORESET;
- priv->bForcedSilentReset = false;
- priv->bDisableNormalResetCheck = false;
+ priv->rst_progress = RESET_TYPE_NORESET;
priv->force_reset = false;
memset(priv->rtllib->swcamtable, 0, sizeof(struct sw_cam_table) * 32);
-
- memset(&priv->InterruptLog, 0, sizeof(struct log_int_8190));
- priv->RxCounter = 0;
+ priv->rx_ctr = 0;
priv->rtllib->wx_set_enc = 0;
priv->hw_radio_off = false;
- priv->RegRfOff = false;
- priv->isRFOff = false;
- priv->bInPowerSaveMode = false;
priv->rtllib->rf_off_reason = 0;
priv->rf_change_in_progress = false;
- priv->bHwRfOffAction = 0;
+ priv->hw_rf_off_action = 0;
priv->SetRFPowerStateInProgress = false;
- priv->rtllib->PowerSaveControl.bInactivePs = true;
- priv->rtllib->PowerSaveControl.bIPSModeBackup = false;
- priv->rtllib->PowerSaveControl.bLeisurePs = true;
- priv->rtllib->PowerSaveControl.bFwCtrlLPS = false;
+ priv->rtllib->pwr_save_ctrl.bLeisurePs = true;
priv->rtllib->LPSDelayCnt = 0;
priv->rtllib->sta_sleep = LPS_IS_WAKE;
priv->rtllib->rf_power_state = rf_on;
for (i = 0; i < MAX_QUEUE_SIZE; i++)
skb_queue_head_init(&priv->rtllib->skb_waitQ[i]);
- for (i = 0; i < MAX_QUEUE_SIZE; i++)
- skb_queue_head_init(&priv->rtllib->skb_aggQ[i]);
}
static void _rtl92e_init_priv_lock(struct r8192_priv *priv)
struct rtllib_device *ieee = priv->rtllib;
unsigned long flag;
- if (priv->ResetProgress == RESET_TYPE_NORESET) {
- priv->ResetProgress = RESET_TYPE_SILENT;
+ if (priv->rst_progress == RESET_TYPE_NORESET) {
+ priv->rst_progress = RESET_TYPE_SILENT;
spin_lock_irqsave(&priv->rf_ps_lock, flag);
if (priv->rf_change_in_progress) {
rtl92e_cam_restore(dev);
rtl92e_dm_restore_state(dev);
END:
- priv->ResetProgress = RESET_TYPE_NORESET;
+ priv->rst_progress = RESET_TYPE_NORESET;
priv->reset_count++;
-
- priv->bForcedSilentReset = false;
priv->bResetInProgress = false;
rtl92e_writeb(dev, UFWP, 1);
*TotalRxBcnNum = 0;
*TotalRxDataNum = 0;
- SlotIndex = (priv->rtllib->LinkDetectInfo.SlotIndex++) %
- (priv->rtllib->LinkDetectInfo.SlotNum);
- priv->rtllib->LinkDetectInfo.RxBcnNum[SlotIndex] =
- priv->rtllib->LinkDetectInfo.NumRecvBcnInPeriod;
- priv->rtllib->LinkDetectInfo.RxDataNum[SlotIndex] =
- priv->rtllib->LinkDetectInfo.NumRecvDataInPeriod;
- for (i = 0; i < priv->rtllib->LinkDetectInfo.SlotNum; i++) {
- *TotalRxBcnNum += priv->rtllib->LinkDetectInfo.RxBcnNum[i];
- *TotalRxDataNum += priv->rtllib->LinkDetectInfo.RxDataNum[i];
+ SlotIndex = (priv->rtllib->link_detect_info.SlotIndex++) %
+ (priv->rtllib->link_detect_info.SlotNum);
+ priv->rtllib->link_detect_info.RxBcnNum[SlotIndex] =
+ priv->rtllib->link_detect_info.NumRecvBcnInPeriod;
+ priv->rtllib->link_detect_info.RxDataNum[SlotIndex] =
+ priv->rtllib->link_detect_info.NumRecvDataInPeriod;
+ for (i = 0; i < priv->rtllib->link_detect_info.SlotNum; i++) {
+ *TotalRxBcnNum += priv->rtllib->link_detect_info.RxBcnNum[i];
+ *TotalRxDataNum += priv->rtllib->link_detect_info.RxDataNum[i];
}
}
enum reset_type ResetType = RESET_TYPE_NORESET;
static u8 check_reset_cnt;
unsigned long flags;
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- (&priv->rtllib->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ (&priv->rtllib->pwr_save_ctrl);
bool bBusyTraffic = false;
bool bHigherBusyTraffic = false;
bool bHigherBusyRxTraffic = false;
RTLLIB_NOLINK) &&
(ieee->rf_power_state == rf_on) && !ieee->is_set_key &&
(!ieee->proto_stoppping) && !ieee->wx_set_enc) {
- if ((ieee->PowerSaveControl.ReturnPoint ==
+ if ((ieee->pwr_save_ctrl.ReturnPoint ==
IPS_CALLBACK_NONE) &&
- (!ieee->bNetPromiscuousMode)) {
+ (!ieee->net_promiscuous_md)) {
rtl92e_ips_enter(dev);
}
}
}
if ((ieee->state == RTLLIB_LINKED) && (ieee->iw_mode ==
- IW_MODE_INFRA) && (!ieee->bNetPromiscuousMode)) {
- if (ieee->LinkDetectInfo.NumRxOkInPeriod > 100 ||
- ieee->LinkDetectInfo.NumTxOkInPeriod > 100)
+ IW_MODE_INFRA) && (!ieee->net_promiscuous_md)) {
+ if (ieee->link_detect_info.NumRxOkInPeriod > 100 ||
+ ieee->link_detect_info.NumTxOkInPeriod > 100)
bBusyTraffic = true;
- if (ieee->LinkDetectInfo.NumRxOkInPeriod > 4000 ||
- ieee->LinkDetectInfo.NumTxOkInPeriod > 4000) {
+ if (ieee->link_detect_info.NumRxOkInPeriod > 4000 ||
+ ieee->link_detect_info.NumTxOkInPeriod > 4000) {
bHigherBusyTraffic = true;
- if (ieee->LinkDetectInfo.NumRxOkInPeriod > 5000)
+ if (ieee->link_detect_info.NumRxOkInPeriod > 5000)
bHigherBusyRxTraffic = true;
else
bHigherBusyRxTraffic = false;
}
- if (((ieee->LinkDetectInfo.NumRxUnicastOkInPeriod +
- ieee->LinkDetectInfo.NumTxOkInPeriod) > 8) ||
- (ieee->LinkDetectInfo.NumRxUnicastOkInPeriod > 2))
+ if (((ieee->link_detect_info.NumRxUnicastOkInPeriod +
+ ieee->link_detect_info.NumTxOkInPeriod) > 8) ||
+ (ieee->link_detect_info.NumRxUnicastOkInPeriod > 2))
bEnterPS = false;
else
bEnterPS = true;
rtl92e_leisure_ps_leave(dev);
}
- ieee->LinkDetectInfo.NumRxOkInPeriod = 0;
- ieee->LinkDetectInfo.NumTxOkInPeriod = 0;
- ieee->LinkDetectInfo.NumRxUnicastOkInPeriod = 0;
- ieee->LinkDetectInfo.bBusyTraffic = bBusyTraffic;
+ ieee->link_detect_info.NumRxOkInPeriod = 0;
+ ieee->link_detect_info.NumTxOkInPeriod = 0;
+ ieee->link_detect_info.NumRxUnicastOkInPeriod = 0;
+ ieee->link_detect_info.bBusyTraffic = bBusyTraffic;
- ieee->LinkDetectInfo.bHigherBusyTraffic = bHigherBusyTraffic;
- ieee->LinkDetectInfo.bHigherBusyRxTraffic = bHigherBusyRxTraffic;
+ ieee->link_detect_info.bHigherBusyTraffic = bHigherBusyTraffic;
+ ieee->link_detect_info.bHigherBusyRxTraffic = bHigherBusyRxTraffic;
if (ieee->state == RTLLIB_LINKED && ieee->iw_mode == IW_MODE_INFRA) {
u32 TotalRxBcnNum = 0;
priv->check_roaming_cnt = 0;
}
- ieee->LinkDetectInfo.NumRecvBcnInPeriod = 0;
- ieee->LinkDetectInfo.NumRecvDataInPeriod = 0;
+ ieee->link_detect_info.NumRecvBcnInPeriod = 0;
+ ieee->link_detect_info.NumRecvDataInPeriod = 0;
}
spin_lock_irqsave(&priv->tx_lock, flags);
if ((check_reset_cnt++ >= 3) && (!ieee->is_roaming) &&
- (!priv->rf_change_in_progress) && (!pPSC->bSwRfProcessing)) {
+ (!priv->rf_change_in_progress) && (!psc->bSwRfProcessing)) {
ResetType = _rtl92e_if_check_reset(dev);
check_reset_cnt = 3;
}
spin_unlock_irqrestore(&priv->tx_lock, flags);
- if (!priv->bDisableNormalResetCheck && ResetType == RESET_TYPE_NORMAL) {
- priv->ResetProgress = RESET_TYPE_NORMAL;
+ if (ResetType == RESET_TYPE_NORMAL) {
+ priv->rst_progress = RESET_TYPE_NORMAL;
return;
}
- if (((priv->force_reset) || (!priv->bDisableNormalResetCheck &&
- ResetType == RESET_TYPE_SILENT)))
+ if ((priv->force_reset || ResetType == RESET_TYPE_SILENT))
_rtl92e_if_silent_reset(dev);
priv->force_reset = false;
- priv->bForcedSilentReset = false;
priv->bResetInProgress = false;
}
}
tcb_desc->RATRIndex = 7;
- tcb_desc->bTxDisableRateFallBack = 1;
- tcb_desc->bTxUseDriverAssingedRate = 1;
+ tcb_desc->tx_dis_rate_fallback = 1;
+ tcb_desc->tx_use_drv_assinged_rate = 1;
tcb_desc->bTxEnableFwCalcDur = 1;
skb_push(skb, priv->rtllib->tx_headroom);
ret = _rtl92e_tx(dev, skb);
if (inta & IMR_ROK) {
priv->stats.rxint++;
- priv->InterruptLog.nIMR_ROK++;
tasklet_schedule(&priv->irq_rx_tasklet);
}
if (inta & IMR_BKDOK) {
priv->stats.txbkokint++;
- priv->rtllib->LinkDetectInfo.NumTxOkInPeriod++;
+ priv->rtllib->link_detect_info.NumTxOkInPeriod++;
_rtl92e_tx_isr(dev, BK_QUEUE);
}
if (inta & IMR_BEDOK) {
priv->stats.txbeokint++;
- priv->rtllib->LinkDetectInfo.NumTxOkInPeriod++;
+ priv->rtllib->link_detect_info.NumTxOkInPeriod++;
_rtl92e_tx_isr(dev, BE_QUEUE);
}
if (inta & IMR_VIDOK) {
priv->stats.txviokint++;
- priv->rtllib->LinkDetectInfo.NumTxOkInPeriod++;
+ priv->rtllib->link_detect_info.NumTxOkInPeriod++;
_rtl92e_tx_isr(dev, VI_QUEUE);
}
if (inta & IMR_VODOK) {
priv->stats.txvookint++;
- priv->rtllib->LinkDetectInfo.NumTxOkInPeriod++;
+ priv->rtllib->link_detect_info.NumTxOkInPeriod++;
_rtl92e_tx_isr(dev, VO_QUEUE);
}
{
bool init_status = true;
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- (&priv->rtllib->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ (&priv->rtllib->pwr_save_ctrl);
if (!priv->up) {
netdev_warn(dev, "%s(): Driver is already down!\n", __func__);
priv->bdisable_nic = false;
return false;
}
- RT_CLEAR_PS_LEVEL(pPSC, RT_RF_OFF_LEVL_HALT_NIC);
+ RT_CLEAR_PS_LEVEL(psc, RT_RF_OFF_LEVL_HALT_NIC);
priv->bfirst_init = false;
rtl92e_irq_enable(dev);
u32 CurrentShowTxate;
};
-struct channel_access_setting {
- u16 SIFS_Timer;
- u16 DIFS_Timer;
- u16 SlotTimeTimer;
- u16 EIFS_Timer;
- u16 CWminIndex;
- u16 CWmaxIndex;
-};
-
struct init_gain {
u8 xaagccore1;
u8 xbagccore1;
bool bfirst_init;
bool bfirst_after_down;
- bool initialized_at_probe;
bool being_init_adapter;
- bool bDriverIsGoingToUnload;
int irq;
short irq_enabled;
struct delayed_work txpower_tracking_wq;
struct delayed_work rfpath_check_wq;
struct delayed_work gpio_change_rf_wq;
-
- struct channel_access_setting ChannelAccessSetting;
-
struct rtl819x_ops *ops;
struct rtllib_device *rtllib;
struct work_struct reset_wq;
- struct log_int_8190 InterruptLog;
-
enum rt_customer_id CustomerID;
struct bb_reg_definition PHYRegDef[4];
struct rate_adaptive rate_adaptive;
- enum acm_method AcmMethod;
-
struct rt_firmware *pFirmware;
enum rtl819x_loopback LoopbackMode;
short chan;
short sens;
short max_sens;
-
- u8 ScanDelay;
bool ps_force;
u32 irq_mask[2];
bool bTXPowerDataReadFromEEPORM;
- u16 RegChannelPlan;
+ u16 reg_chnl_plan;
u16 ChannelPlan;
-
- bool RegRfOff;
- bool isRFOff;
- bool bInPowerSaveMode;
- u8 bHwRfOffAction;
+ u8 hw_rf_off_action;
bool rf_change_in_progress;
bool SetRFPowerStateInProgress;
u8 CCKPresentAttentuation_20Mdefault;
u8 CCKPresentAttentuation_40Mdefault;
s8 CCKPresentAttentuation_difference;
- s8 CCKPresentAttentuation;
+ s8 cck_present_attn;
long undecorated_smoothed_pwdb;
u32 MCSTxPowerLevelOriginalOffset[6];
u32 reset_count;
- enum reset_type ResetProgress;
- bool bForcedSilentReset;
- bool bDisableNormalResetCheck;
+ enum reset_type rst_progress;
u16 TxCounter;
- u16 RxCounter;
+ u16 rx_ctr;
bool bResetInProgress;
bool force_reset;
bool force_lps;
"PATH=/usr/bin:/bin",
NULL};
- if (priv->ResetProgress == RESET_TYPE_SILENT) {
+ if (priv->rst_progress == RESET_TYPE_SILENT)
return;
- }
-
if (priv->rtllib->state != RTLLIB_LINKED)
return;
call_usermodehelper(ac_dc_script, argv, envp, UMH_WAIT_PROC);
static void _rtl92e_dm_check_rate_adaptive(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_hi_throughput *pHTInfo = priv->rtllib->pHTInfo;
+ struct rt_hi_throughput *ht_info = priv->rtllib->ht_info;
struct rate_adaptive *pra = &priv->rate_adaptive;
u32 currentRATR, targetRATR = 0;
u32 LowRSSIThreshForRA = 0, HighRSSIThreshForRA = 0;
bool bshort_gi_enabled = false;
static u8 ping_rssi_state;
- if (!priv->up) {
+ if (!priv->up)
return;
- }
if (pra->rate_adaptive_disabled)
return;
if (priv->rtllib->state == RTLLIB_LINKED) {
- bshort_gi_enabled = (pHTInfo->cur_tx_bw40mhz &&
- pHTInfo->bCurShortGI40MHz) ||
- (!pHTInfo->cur_tx_bw40mhz &&
- pHTInfo->bCurShortGI20MHz);
+ bshort_gi_enabled = (ht_info->cur_tx_bw40mhz &&
+ ht_info->bCurShortGI40MHz) ||
+ (!ht_info->cur_tx_bw40mhz &&
+ ht_info->bCurShortGI20MHz);
pra->upper_rssi_threshold_ratr =
(pra->upper_rssi_threshold_ratr & (~BIT31)) |
for (j = 0; j <= 30; j++) {
- tx_cmd.Op = TXCMD_SET_TX_PWR_TRACKING;
- tx_cmd.Length = 4;
- tx_cmd.Value = Value;
+ tx_cmd.op = TXCMD_SET_TX_PWR_TRACKING;
+ tx_cmd.length = 4;
+ tx_cmd.value = Value;
rtl92e_send_cmd_pkt(dev, DESC_PACKET_TYPE_NORMAL, (u8 *)&tx_cmd,
sizeof(struct dcmd_txcmd));
mdelay(1);
}
if (priv->CurrentChannelBW == HT_CHANNEL_WIDTH_20)
- priv->CCKPresentAttentuation =
+ priv->cck_present_attn =
priv->CCKPresentAttentuation_20Mdefault +
priv->CCKPresentAttentuation_difference;
else
- priv->CCKPresentAttentuation =
+ priv->cck_present_attn =
priv->CCKPresentAttentuation_40Mdefault +
priv->CCKPresentAttentuation_difference;
- if (priv->CCKPresentAttentuation > (CCKTxBBGainTableLength-1))
- priv->CCKPresentAttentuation = CCKTxBBGainTableLength-1;
- if (priv->CCKPresentAttentuation < 0)
- priv->CCKPresentAttentuation = 0;
+ if (priv->cck_present_attn > (CCKTxBBGainTableLength-1))
+ priv->cck_present_attn = CCKTxBBGainTableLength-1;
+ if (priv->cck_present_attn < 0)
+ priv->cck_present_attn = 0;
- if (priv->CCKPresentAttentuation > -1 &&
- priv->CCKPresentAttentuation < CCKTxBBGainTableLength) {
+ if (priv->cck_present_attn > -1 &&
+ priv->cck_present_attn < CCKTxBBGainTableLength) {
if (priv->rtllib->current_network.channel == 14 &&
!priv->bcck_in_ch14) {
priv->bcck_in_ch14 = true;
tmpRegA = rtl92e_get_bb_reg(dev, rOFDM0_XATxIQImbalance,
bMaskDWord);
for (i = 0; i < OFDM_Table_Length; i++) {
- if (tmpRegA == OFDMSwingTable[i]) {
+ if (tmpRegA == OFDMSwingTable[i])
priv->OFDM_index[0] = i;
- }
}
TempCCk = rtl92e_get_bb_reg(dev, rCCK0_TxFilter1, bMaskByte2);
{
u32 TempVal;
struct r8192_priv *priv = rtllib_priv(dev);
- u8 attenuation = priv->CCKPresentAttentuation;
+ u8 attenuation = priv->cck_present_attn;
TempVal = 0;
if (!bInCH14) {
u32 reg_ratr = priv->rate_adaptive.last_ratr;
u32 ratr_value;
- if (!priv->up) {
+ if (!priv->up)
return;
- }
if (priv->rate_adaptive.rate_adaptive_disabled)
return;
dm_digtable.dig_state = DM_STA_DIG_MAX;
dm_digtable.dig_highpwr_state = DM_STA_DIG_MAX;
- dm_digtable.CurSTAConnectState = DIG_STA_DISCONNECT;
- dm_digtable.PreSTAConnectState = DIG_STA_DISCONNECT;
+ dm_digtable.cur_sta_connect_state = DIG_STA_DISCONNECT;
+ dm_digtable.pre_sta_connect_state = DIG_STA_DISCONNECT;
dm_digtable.rssi_low_thresh = DM_DIG_THRESH_LOW;
dm_digtable.rssi_high_thresh = DM_DIG_THRESH_HIGH;
}
if (priv->rtllib->state == RTLLIB_LINKED)
- dm_digtable.CurSTAConnectState = DIG_STA_CONNECT;
+ dm_digtable.cur_sta_connect_state = DIG_STA_CONNECT;
else
- dm_digtable.CurSTAConnectState = DIG_STA_DISCONNECT;
+ dm_digtable.cur_sta_connect_state = DIG_STA_DISCONNECT;
dm_digtable.rssi_val = priv->undecorated_smoothed_pwdb;
_rtl92e_dm_cs_ratio(dev);
if (dm_digtable.dig_algorithm_switch)
dm_digtable.dig_algorithm_switch = 0;
- dm_digtable.PreSTAConnectState = dm_digtable.CurSTAConnectState;
+ dm_digtable.pre_sta_connect_state = dm_digtable.cur_sta_connect_state;
}
return;
}
- if (dm_digtable.PreSTAConnectState == dm_digtable.CurSTAConnectState) {
- if (dm_digtable.CurSTAConnectState == DIG_STA_CONNECT) {
+ if (dm_digtable.pre_sta_connect_state == dm_digtable.cur_sta_connect_state) {
+ if (dm_digtable.cur_sta_connect_state == DIG_STA_CONNECT) {
long gain_range = dm_digtable.rssi_val + 10 -
dm_digtable.backoff_val;
gain_range = clamp_t(long, gain_range,
reset_cnt = 0;
}
- if (dm_digtable.PreSTAConnectState == dm_digtable.CurSTAConnectState) {
- if (dm_digtable.CurSTAConnectState == DIG_STA_CONNECT) {
+ if (dm_digtable.pre_sta_connect_state == dm_digtable.cur_sta_connect_state) {
+ if (dm_digtable.cur_sta_connect_state == DIG_STA_CONNECT) {
if (dm_digtable.rssi_val >=
dm_digtable.rssi_high_power_highthresh)
dm_digtable.curpd_thstate =
reset_cnt = 0;
}
- if (dm_digtable.PreSTAConnectState == dm_digtable.CurSTAConnectState) {
- if (dm_digtable.CurSTAConnectState == DIG_STA_CONNECT) {
+ if (dm_digtable.pre_sta_connect_state == dm_digtable.cur_sta_connect_state) {
+ if (dm_digtable.cur_sta_connect_state == DIG_STA_CONNECT) {
if (dm_digtable.rssi_val <= dm_digtable.rssi_low_thresh)
dm_digtable.curcs_ratio_state = DIG_CS_RATIO_LOWER;
else if (dm_digtable.rssi_val >= dm_digtable.rssi_high_thresh)
static void _rtl92e_dm_check_edca_turbo(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_hi_throughput *pHTInfo = priv->rtllib->pHTInfo;
+ struct rt_hi_throughput *ht_info = priv->rtllib->ht_info;
static unsigned long lastTxOkCnt;
static unsigned long lastRxOkCnt;
goto dm_CheckEdcaTurbo_EXIT;
if (priv->rtllib->state != RTLLIB_LINKED)
goto dm_CheckEdcaTurbo_EXIT;
- if (priv->rtllib->pHTInfo->iot_action & HT_IOT_ACT_DISABLE_EDCA_TURBO)
+ if (priv->rtllib->ht_info->iot_action & HT_IOT_ACT_DISABLE_EDCA_TURBO)
goto dm_CheckEdcaTurbo_EXIT;
if (!priv->rtllib->bis_any_nonbepkts) {
curTxOkCnt = priv->stats.txbytesunicast - lastTxOkCnt;
curRxOkCnt = priv->stats.rxbytesunicast - lastRxOkCnt;
- if (pHTInfo->iot_action & HT_IOT_ACT_EDCA_BIAS_ON_RX) {
+ if (ht_info->iot_action & HT_IOT_ACT_EDCA_BIAS_ON_RX) {
if (curTxOkCnt > 4*curRxOkCnt) {
if (priv->bis_cur_rdlstate ||
!priv->bcurrent_turbo_EDCA) {
rtl92e_writel(dev, EDCAPARA_BE,
- edca_setting_UL[pHTInfo->IOTPeer]);
+ edca_setting_UL[ht_info->IOTPeer]);
priv->bis_cur_rdlstate = false;
}
} else {
!priv->bcurrent_turbo_EDCA) {
if (priv->rtllib->mode == WIRELESS_MODE_G)
rtl92e_writel(dev, EDCAPARA_BE,
- edca_setting_DL_GMode[pHTInfo->IOTPeer]);
+ edca_setting_DL_GMode[ht_info->IOTPeer]);
else
rtl92e_writel(dev, EDCAPARA_BE,
- edca_setting_DL[pHTInfo->IOTPeer]);
+ edca_setting_DL[ht_info->IOTPeer]);
priv->bis_cur_rdlstate = true;
}
}
!priv->bcurrent_turbo_EDCA) {
if (priv->rtllib->mode == WIRELESS_MODE_G)
rtl92e_writel(dev, EDCAPARA_BE,
- edca_setting_DL_GMode[pHTInfo->IOTPeer]);
+ edca_setting_DL_GMode[ht_info->IOTPeer]);
else
rtl92e_writel(dev, EDCAPARA_BE,
- edca_setting_DL[pHTInfo->IOTPeer]);
+ edca_setting_DL[ht_info->IOTPeer]);
priv->bis_cur_rdlstate = true;
}
} else {
if (priv->bis_cur_rdlstate ||
!priv->bcurrent_turbo_EDCA) {
rtl92e_writel(dev, EDCAPARA_BE,
- edca_setting_UL[pHTInfo->IOTPeer]);
+ edca_setting_UL[ht_info->IOTPeer]);
priv->bis_cur_rdlstate = false;
}
static void _rtl92e_dm_cts_to_self(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv((struct net_device *)dev);
- struct rt_hi_throughput *pHTInfo = priv->rtllib->pHTInfo;
+ struct rt_hi_throughput *ht_info = priv->rtllib->ht_info;
static unsigned long lastTxOkCnt;
static unsigned long lastRxOkCnt;
unsigned long curTxOkCnt = 0;
unsigned long curRxOkCnt = 0;
if (!priv->rtllib->bCTSToSelfEnable) {
- pHTInfo->iot_action &= ~HT_IOT_ACT_FORCED_CTS2SELF;
+ ht_info->iot_action &= ~HT_IOT_ACT_FORCED_CTS2SELF;
return;
}
- if (pHTInfo->IOTPeer == HT_IOT_PEER_BROADCOM) {
+ if (ht_info->IOTPeer == HT_IOT_PEER_BROADCOM) {
curTxOkCnt = priv->stats.txbytesunicast - lastTxOkCnt;
curRxOkCnt = priv->stats.rxbytesunicast - lastRxOkCnt;
if (curRxOkCnt > 4*curTxOkCnt)
- pHTInfo->iot_action &= ~HT_IOT_ACT_FORCED_CTS2SELF;
+ ht_info->iot_action &= ~HT_IOT_ACT_FORCED_CTS2SELF;
else
- pHTInfo->iot_action |= HT_IOT_ACT_FORCED_CTS2SELF;
+ ht_info->iot_action |= HT_IOT_ACT_FORCED_CTS2SELF;
lastTxOkCnt = priv->stats.txbytesunicast;
lastRxOkCnt = priv->stats.rxbytesunicast;
static void _rtl92e_dm_init_wa_broadcom_iot(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv((struct net_device *)dev);
- struct rt_hi_throughput *pHTInfo = priv->rtllib->pHTInfo;
+ struct rt_hi_throughput *ht_info = priv->rtllib->ht_info;
- pHTInfo->bWAIotBroadcom = false;
- pHTInfo->WAIotTH = WAIotTHVal;
+ ht_info->bWAIotBroadcom = false;
+ ht_info->WAIotTH = WAIotTHVal;
}
static void _rtl92e_dm_check_rf_ctrl_gpio(void *data)
if (bActuallySet) {
mdelay(1000);
- priv->bHwRfOffAction = 1;
+ priv->hw_rf_off_action = 1;
rtl92e_set_rf_state(dev, rf_power_state_to_set, RF_CHANGE_BY_HW);
if (priv->hw_radio_off)
argv[1] = "RFOFF";
if (priv->rtllib->state == RTLLIB_LINKED &&
priv->rtllib->bfsync_enable &&
- (priv->rtllib->pHTInfo->iot_action & HT_IOT_ACT_CDD_FSYNC)) {
+ (priv->rtllib->ht_info->iot_action & HT_IOT_ACT_CDD_FSYNC)) {
u32 rate_bitmap;
for (rate_index = 0; rate_index <= 27; rate_index++) {
static void _rtl92e_dm_start_sw_fsync(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv(dev);
- u32 rateIndex;
- u32 rateBitmap;
+ u32 rate_index;
+ u32 rate_bitmap;
priv->rate_record = 0;
priv->ContinueDiffCount = 0;
priv->rtllib->fsync_firstdiff_ratethreshold = 200;
priv->rtllib->fsync_seconddiff_ratethreshold = 200;
}
- for (rateIndex = 0; rateIndex <= 27; rateIndex++) {
- rateBitmap = 1 << rateIndex;
- if (priv->rtllib->fsync_rate_bitmap & rateBitmap)
+ for (rate_index = 0; rate_index <= 27; rate_index++) {
+ rate_bitmap = 1 << rate_index;
+ if (priv->rtllib->fsync_rate_bitmap & rate_bitmap)
priv->rate_record +=
priv->stats.received_rate_histogram[1]
- [rateIndex];
+ [rate_index];
}
if (timer_pending(&priv->fsync_timer))
del_timer_sync(&priv->fsync_timer);
static u32 reset_cnt;
if (priv->rtllib->state == RTLLIB_LINKED &&
- priv->rtllib->pHTInfo->IOTPeer == HT_IOT_PEER_BROADCOM) {
+ priv->rtllib->ht_info->IOTPeer == HT_IOT_PEER_BROADCOM) {
if (priv->rtllib->bfsync_enable == 0) {
switch (priv->rtllib->fsync_state) {
case Default_Fsync:
priv->bDynamicTxLowPower = false;
return;
}
- if ((priv->rtllib->pHTInfo->IOTPeer == HT_IOT_PEER_ATHEROS) &&
+ if ((priv->rtllib->ht_info->IOTPeer == HT_IOT_PEER_ATHEROS) &&
(priv->rtllib->mode == IEEE_G)) {
txhipower_threshold = TX_POWER_ATHEROAP_THRESH_HIGH;
txlowpower_threshold = TX_POWER_ATHEROAP_THRESH_LOW;
u8 dig_state;
u8 dig_highpwr_state;
- u8 CurSTAConnectState;
- u8 PreSTAConnectState;
+ u8 cur_sta_connect_state;
+ u8 pre_sta_connect_state;
u8 curpd_thstate;
u8 prepd_thstate;
struct dcmd_txcmd {
- u32 Op;
- u32 Length;
- u32 Value;
+ u32 op;
+ u32 length;
+ u32 value;
};
/*------------------------------Define structure----------------------------*/
ret = _rtl92e_eeprom_xfer(dev, (addr & 0x3F) | (0x6 << 6), 9);
rtl92e_writeb(dev, EPROM_CMD,
- (EPROM_CMD_NORMAL<<EPROM_CMD_OPERATING_MODE_SHIFT));
+ (EPROM_CMD_NORMAL << EPROM_CMD_OPERATING_MODE_SHIFT));
return ret;
}
static void _rtl92e_ps_update_rf_state(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- &(priv->rtllib->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ &priv->rtllib->pwr_save_ctrl;
- pPSC->bSwRfProcessing = true;
- rtl92e_set_rf_state(dev, pPSC->eInactivePowerState, RF_CHANGE_BY_IPS);
+ psc->bSwRfProcessing = true;
+ rtl92e_set_rf_state(dev, psc->eInactivePowerState, RF_CHANGE_BY_IPS);
- pPSC->bSwRfProcessing = false;
+ psc->bSwRfProcessing = false;
}
void rtl92e_ips_enter(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- &(priv->rtllib->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ &priv->rtllib->pwr_save_ctrl;
enum rt_rf_power_state rt_state;
- if (pPSC->bInactivePs) {
- rt_state = priv->rtllib->rf_power_state;
- if (rt_state == rf_on && !pPSC->bSwRfProcessing &&
- (priv->rtllib->state != RTLLIB_LINKED) &&
- (priv->rtllib->iw_mode != IW_MODE_MASTER)) {
- pPSC->eInactivePowerState = rf_off;
- priv->isRFOff = true;
- priv->bInPowerSaveMode = true;
- _rtl92e_ps_update_rf_state(dev);
- }
+ rt_state = priv->rtllib->rf_power_state;
+ if (rt_state == rf_on && !psc->bSwRfProcessing &&
+ (priv->rtllib->state != RTLLIB_LINKED) &&
+ (priv->rtllib->iw_mode != IW_MODE_MASTER)) {
+ psc->eInactivePowerState = rf_off;
+ _rtl92e_ps_update_rf_state(dev);
}
}
void rtl92e_ips_leave(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- &(priv->rtllib->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ &priv->rtllib->pwr_save_ctrl;
enum rt_rf_power_state rt_state;
- if (pPSC->bInactivePs) {
- rt_state = priv->rtllib->rf_power_state;
- if (rt_state != rf_on && !pPSC->bSwRfProcessing &&
- priv->rtllib->rf_off_reason <= RF_CHANGE_BY_IPS) {
- pPSC->eInactivePowerState = rf_on;
- priv->bInPowerSaveMode = false;
- _rtl92e_ps_update_rf_state(dev);
- }
+ rt_state = priv->rtllib->rf_power_state;
+ if (rt_state != rf_on && !psc->bSwRfProcessing &&
+ priv->rtllib->rf_off_reason <= RF_CHANGE_BY_IPS) {
+ psc->eInactivePowerState = rf_on;
+ _rtl92e_ps_update_rf_state(dev);
}
}
enum rt_rf_power_state rt_state;
rt_state = priv->rtllib->rf_power_state;
-
- if (priv->rtllib->PowerSaveControl.bInactivePs) {
- if (rt_state == rf_off) {
- if (priv->rtllib->rf_off_reason > RF_CHANGE_BY_IPS) {
- netdev_warn(dev, "%s(): RF is OFF.\n",
- __func__);
- return;
- }
- netdev_info(dev, "=========>%s(): rtl92e_ips_leave\n",
+ if (rt_state == rf_off) {
+ if (priv->rtllib->rf_off_reason > RF_CHANGE_BY_IPS) {
+ netdev_warn(dev, "%s(): RF is OFF.\n",
__func__);
- schedule_work(&priv->rtllib->ips_leave_wq);
+ return;
}
+ netdev_info(dev, "=========>%s(): rtl92e_ips_leave\n",
+ __func__);
+ schedule_work(&priv->rtllib->ips_leave_wq);
}
}
void rtl92e_leisure_ps_enter(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- &(priv->rtllib->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ &priv->rtllib->pwr_save_ctrl;
if (!((priv->rtllib->iw_mode == IW_MODE_INFRA) &&
(priv->rtllib->state == RTLLIB_LINKED))
(priv->rtllib->iw_mode == IW_MODE_MASTER))
return;
- if (pPSC->bLeisurePs) {
- if (pPSC->LpsIdleCount >= RT_CHECK_FOR_HANG_PERIOD) {
+ if (psc->bLeisurePs) {
+ if (psc->LpsIdleCount >= RT_CHECK_FOR_HANG_PERIOD) {
if (priv->rtllib->ps == RTLLIB_PS_DISABLED) {
- if (!pPSC->bFwCtrlLPS) {
- if (priv->rtllib->SetFwCmdHandler)
- priv->rtllib->SetFwCmdHandler(
- dev, FW_CMD_LPS_ENTER);
- }
+ if (priv->rtllib->SetFwCmdHandler)
+ priv->rtllib->SetFwCmdHandler(dev, FW_CMD_LPS_ENTER);
_rtl92e_ps_set_mode(dev, RTLLIB_PS_MBCAST |
RTLLIB_PS_UNICAST);
}
} else
- pPSC->LpsIdleCount++;
+ psc->LpsIdleCount++;
}
}
void rtl92e_leisure_ps_leave(struct net_device *dev)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- &(priv->rtllib->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ &priv->rtllib->pwr_save_ctrl;
- if (pPSC->bLeisurePs) {
+ if (psc->bLeisurePs) {
if (priv->rtllib->ps != RTLLIB_PS_DISABLED) {
_rtl92e_ps_set_mode(dev, RTLLIB_PS_DISABLED);
-
- if (!pPSC->bFwCtrlLPS) {
- if (priv->rtllib->SetFwCmdHandler)
- priv->rtllib->SetFwCmdHandler(dev,
- FW_CMD_LPS_LEAVE);
- }
+ if (priv->rtllib->SetFwCmdHandler)
+ priv->rtllib->SetFwCmdHandler(dev, FW_CMD_LPS_LEAVE);
}
}
}
union iwreq_data *wrqu, char *extra)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- (&priv->rtllib->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ (&priv->rtllib->pwr_save_ctrl);
struct rtllib_device *ieee = priv->rtllib;
mutex_lock(&priv->wx_mutex);
if (*extra || priv->force_lps) {
priv->ps_force = false;
- pPSC->bLeisurePs = true;
+ psc->bLeisurePs = true;
} else {
if (priv->rtllib->state == RTLLIB_LINKED)
rtl92e_leisure_ps_leave(dev);
priv->ps_force = true;
- pPSC->bLeisurePs = false;
+ psc->bLeisurePs = false;
ieee->ps = *extra;
}
char *extra)
{
struct r8192_priv *priv = rtllib_priv(dev);
- struct rt_pwr_save_ctrl *pPSC = (struct rt_pwr_save_ctrl *)
- (&priv->rtllib->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = (struct rt_pwr_save_ctrl *)
+ (&priv->rtllib->pwr_save_ctrl);
mutex_lock(&priv->wx_mutex);
netdev_info(dev, "%s(): set lps awake interval ! extra is %d\n",
__func__, *extra);
- pPSC->RegMaxLPSAwakeIntvl = *extra;
+ psc->reg_max_lps_awake_intvl = *extra;
mutex_unlock(&priv->wx_mutex);
return 0;
}
rt_state = priv->rtllib->rf_power_state;
mutex_lock(&priv->wx_mutex);
if (wrqu->mode == IW_MODE_ADHOC || wrqu->mode == IW_MODE_MONITOR ||
- ieee->bNetPromiscuousMode) {
- if (priv->rtllib->PowerSaveControl.bInactivePs) {
- if (rt_state == rf_off) {
- if (priv->rtllib->rf_off_reason >
- RF_CHANGE_BY_IPS) {
- netdev_warn(dev, "%s(): RF is OFF.\n",
- __func__);
- mutex_unlock(&priv->wx_mutex);
- return -1;
- }
- netdev_info(dev,
- "=========>%s(): rtl92e_ips_leave\n",
+ ieee->net_promiscuous_md) {
+ if (rt_state == rf_off) {
+ if (priv->rtllib->rf_off_reason >
+ RF_CHANGE_BY_IPS) {
+ netdev_warn(dev, "%s(): RF is OFF.\n",
__func__);
- mutex_lock(&priv->rtllib->ips_mutex);
- rtl92e_ips_leave(dev);
- mutex_unlock(&priv->rtllib->ips_mutex);
+ mutex_unlock(&priv->wx_mutex);
+ return -1;
}
+ netdev_info(dev,
+ "=========>%s(): rtl92e_ips_leave\n",
+ __func__);
+ mutex_lock(&priv->rtllib->ips_mutex);
+ rtl92e_ips_leave(dev);
+ mutex_unlock(&priv->rtllib->ips_mutex);
}
}
ret = rtllib_wx_set_mode(priv->rtllib, a, wrqu, b);
rt_state = priv->rtllib->rf_power_state;
if (!priv->up)
return -ENETDOWN;
- if (priv->rtllib->LinkDetectInfo.bBusyTraffic == true)
+ if (priv->rtllib->link_detect_info.bBusyTraffic == true)
return -EAGAIN;
if (wrqu->data.flags & IW_SCAN_THIS_ESSID) {
priv->rtllib->FirstIe_InScan = true;
if (priv->rtllib->state != RTLLIB_LINKED) {
- if (priv->rtllib->PowerSaveControl.bInactivePs) {
- if (rt_state == rf_off) {
- if (priv->rtllib->rf_off_reason >
- RF_CHANGE_BY_IPS) {
- netdev_warn(dev, "%s(): RF is OFF.\n",
- __func__);
- mutex_unlock(&priv->wx_mutex);
- return -1;
- }
- mutex_lock(&priv->rtllib->ips_mutex);
- rtl92e_ips_leave(dev);
- mutex_unlock(&priv->rtllib->ips_mutex);
+ if (rt_state == rf_off) {
+ if (priv->rtllib->rf_off_reason >
+ RF_CHANGE_BY_IPS) {
+ netdev_warn(dev, "%s(): RF is OFF.\n",
+ __func__);
+ mutex_unlock(&priv->wx_mutex);
+ return -1;
}
+ mutex_lock(&priv->rtllib->ips_mutex);
+ rtl92e_ips_leave(dev);
+ mutex_unlock(&priv->rtllib->ips_mutex);
}
rtllib_stop_scan(priv->rtllib);
if (priv->rtllib->LedControlHandler)
key, 0);
} else {
if ((ieee->pairwise_key_type == KEY_TYPE_CCMP) &&
- ieee->pHTInfo->bCurrentHTSupport)
+ ieee->ht_info->bCurrentHTSupport)
rtl92e_writeb(dev, 0x173, 1);
rtl92e_set_key(dev, 4, idx, alg,
(u8 *)ieee->ap_mac_addr, 0, key);
u32 info_buf[3];
u32 oid;
- u32 bPromiscuousOn;
- u32 bFilterSourceStationFrame;
+ u32 promiscuous_on;
+ u32 fltr_src_sta_frame;
if (copy_from_user(info_buf, wrqu->data.pointer, sizeof(info_buf)))
return -EFAULT;
oid = info_buf[0];
- bPromiscuousOn = info_buf[1];
- bFilterSourceStationFrame = info_buf[2];
+ promiscuous_on = info_buf[1];
+ fltr_src_sta_frame = info_buf[2];
if (oid == OID_RT_INTEL_PROMISCUOUS_MODE) {
- ieee->IntelPromiscuousModeInfo.bPromiscuousOn =
- (bPromiscuousOn) ? (true) : (false);
- ieee->IntelPromiscuousModeInfo.bFilterSourceStationFrame =
- (bFilterSourceStationFrame) ? (true) : (false);
- (bPromiscuousOn) ?
+ ieee->intel_promiscuous_md_info.promiscuous_on =
+ (promiscuous_on) ? (true) : (false);
+ ieee->intel_promiscuous_md_info.fltr_src_sta_frame =
+ (fltr_src_sta_frame) ? (true) : (false);
+ (promiscuous_on) ?
(rtllib_EnableIntelPromiscuousMode(dev, false)) :
(rtllib_DisableIntelPromiscuousMode(dev, false));
netdev_info(dev,
"=======>%s(), on = %d, filter src sta = %d\n",
- __func__, bPromiscuousOn,
- bFilterSourceStationFrame);
+ __func__, promiscuous_on,
+ fltr_src_sta_frame);
} else {
return -1;
}
mutex_lock(&priv->wx_mutex);
snprintf(extra, 45, "PromiscuousMode:%d, FilterSrcSTAFrame:%d",
- ieee->IntelPromiscuousModeInfo.bPromiscuousOn,
- ieee->IntelPromiscuousModeInfo.bFilterSourceStationFrame);
+ ieee->intel_promiscuous_md_info.promiscuous_on,
+ ieee->intel_promiscuous_md_info.fltr_src_sta_frame);
wrqu->data.length = strlen(extra) + 1;
mutex_unlock(&priv->wx_mutex);
pBA->dialog_token = 0;
pBA->ba_start_seq_ctrl.short_data = 0;
}
+
static struct sk_buff *rtllib_ADDBA(struct rtllib_device *ieee, u8 *Dst,
struct ba_record *pBA,
u16 StatusCode, u8 type)
tag += 2;
if (type == ACT_ADDBAREQ) {
- memcpy(tag, (u8 *)&(pBA->ba_start_seq_ctrl), 2);
+ memcpy(tag, (u8 *)&pBA->ba_start_seq_ctrl, 2);
tag += 2;
}
*tag++ = ACT_CAT_BA;
*tag++ = ACT_DELBA;
-
put_unaligned_le16(DelbaParamSet.short_data, tag);
tag += 2;
skb = rtllib_ADDBA(ieee, dst, pBA, 0, ACT_ADDBAREQ);
- if (skb) {
+ if (skb)
softmac_mgmt_xmit(skb, ieee);
- } else {
+ else
netdev_dbg(ieee->dev, "Failed to generate ADDBAReq packet.\n");
- }
}
static void rtllib_send_ADDBARsp(struct rtllib_device *ieee, u8 *dst,
pBaStartSeqCtrl = (union sequence_control *)(req + 7);
if (!ieee->current_network.qos_data.active ||
- !ieee->pHTInfo->bCurrentHTSupport ||
- (ieee->pHTInfo->iot_action & HT_IOT_ACT_REJECT_ADDBA_REQ)) {
+ !ieee->ht_info->bCurrentHTSupport ||
+ (ieee->ht_info->iot_action & HT_IOT_ACT_REJECT_ADDBA_REQ)) {
rc = ADDBA_STATUS_REFUSED;
netdev_warn(ieee->dev,
"Failed to reply on ADDBA_REQ as some capability is not ready(%d, %d)\n",
ieee->current_network.qos_data.active,
- ieee->pHTInfo->bCurrentHTSupport);
+ ieee->ht_info->bCurrentHTSupport);
goto OnADDBAReq_Fail;
}
- if (!GetTs(ieee, (struct ts_common_info **)(&pTS), dst,
- (u8)(pBaParamSet->field.tid), RX_DIR, true)) {
+ if (!GetTs(ieee, (struct ts_common_info **)&pTS, dst,
+ (u8)(pBaParamSet->field.tid), RX_DIR, true)) {
rc = ADDBA_STATUS_REFUSED;
netdev_warn(ieee->dev, "%s(): can't get TS\n", __func__);
goto OnADDBAReq_Fail;
pBA->ba_start_seq_ctrl = *pBaStartSeqCtrl;
if (ieee->GetHalfNmodeSupportByAPsHandler(ieee->dev) ||
- (ieee->pHTInfo->iot_action & HT_IOT_ACT_ALLOW_PEER_AGG_ONE_PKT))
+ (ieee->ht_info->iot_action & HT_IOT_ACT_ALLOW_PEER_AGG_ONE_PKT))
pBA->ba_param_set.field.buffer_size = 1;
else
pBA->ba_param_set.field.buffer_size = 32;
pBaTimeoutVal = (u16 *)(tag + 7);
if (!ieee->current_network.qos_data.active ||
- !ieee->pHTInfo->bCurrentHTSupport ||
- !ieee->pHTInfo->bCurrentAMPDUEnable) {
+ !ieee->ht_info->bCurrentHTSupport ||
+ !ieee->ht_info->bCurrentAMPDUEnable) {
netdev_warn(ieee->dev,
"reject to ADDBA_RSP as some capability is not ready(%d, %d, %d)\n",
ieee->current_network.qos_data.active,
- ieee->pHTInfo->bCurrentHTSupport,
- ieee->pHTInfo->bCurrentAMPDUEnable);
+ ieee->ht_info->bCurrentHTSupport,
+ ieee->ht_info->bCurrentAMPDUEnable);
ReasonCode = DELBA_REASON_UNKNOWN_BA;
goto OnADDBARsp_Reject;
}
- if (!GetTs(ieee, (struct ts_common_info **)(&pTS), dst,
+ if (!GetTs(ieee, (struct ts_common_info **)&pTS, dst,
(u8)(pBaParamSet->field.tid), TX_DIR, false)) {
netdev_warn(ieee->dev, "%s(): can't get TS\n", __func__);
ReasonCode = DELBA_REASON_UNKNOWN_BA;
goto OnADDBARsp_Reject;
}
-
pAdmittedBA->dialog_token = *pDialogToken;
pAdmittedBA->ba_timeout_value = *pBaTimeoutVal;
pAdmittedBA->ba_start_seq_ctrl = pPendingBA->ba_start_seq_ctrl;
}
if (!ieee->current_network.qos_data.active ||
- !ieee->pHTInfo->bCurrentHTSupport) {
+ !ieee->ht_info->bCurrentHTSupport) {
netdev_warn(ieee->dev,
"received DELBA while QOS or HT is not supported(%d, %d)\n",
ieee->current_network. qos_data.active,
- ieee->pHTInfo->bCurrentHTSupport);
+ ieee->ht_info->bCurrentHTSupport);
return -1;
}
struct rx_ts_record *pRxTs;
if (!GetTs(ieee, (struct ts_common_info **)&pRxTs, dst,
- (u8)pDelBaParamSet->field.tid, RX_DIR, false)) {
+ (u8)pDelBaParamSet->field.tid, RX_DIR, false)) {
netdev_warn(ieee->dev,
"%s(): can't get TS for RXTS. dst:%pM TID:%d\n",
__func__, dst,
struct rt_hi_throughput {
- u8 bEnableHT;
+ u8 enable_ht;
u8 bCurrentHTSupport;
u8 bRegBW40MHz;
void HTUpdateDefaultSetting(struct rtllib_device *ieee)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
- pHTInfo->bRegShortGI20MHz = 1;
- pHTInfo->bRegShortGI40MHz = 1;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
- pHTInfo->bRegBW40MHz = 1;
+ ht_info->bRegShortGI20MHz = 1;
+ ht_info->bRegShortGI40MHz = 1;
- if (pHTInfo->bRegBW40MHz)
- pHTInfo->bRegSuppCCK = 1;
+ ht_info->bRegBW40MHz = 1;
+
+ if (ht_info->bRegBW40MHz)
+ ht_info->bRegSuppCCK = 1;
else
- pHTInfo->bRegSuppCCK = true;
+ ht_info->bRegSuppCCK = true;
- pHTInfo->nAMSDU_MaxSize = 7935UL;
- pHTInfo->bAMSDU_Support = 0;
+ ht_info->nAMSDU_MaxSize = 7935UL;
+ ht_info->bAMSDU_Support = 0;
- pHTInfo->bAMPDUEnable = 1;
- pHTInfo->AMPDU_Factor = 2;
- pHTInfo->MPDU_Density = 0;
+ ht_info->bAMPDUEnable = 1;
+ ht_info->AMPDU_Factor = 2;
+ ht_info->MPDU_Density = 0;
- pHTInfo->self_mimo_ps = 3;
- if (pHTInfo->self_mimo_ps == 2)
- pHTInfo->self_mimo_ps = 3;
- ieee->bTxDisableRateFallBack = 0;
- ieee->bTxUseDriverAssingedRate = 0;
+ ht_info->self_mimo_ps = 3;
+ if (ht_info->self_mimo_ps == 2)
+ ht_info->self_mimo_ps = 3;
+ ieee->tx_dis_rate_fallback = 0;
+ ieee->tx_use_drv_assinged_rate = 0;
ieee->bTxEnableFwCalcDur = 1;
- pHTInfo->reg_rt2rt_aggregation = 1;
+ ht_info->reg_rt2rt_aggregation = 1;
- pHTInfo->reg_rx_reorder_enable = 1;
- pHTInfo->rx_reorder_win_size = 64;
- pHTInfo->rx_reorder_pending_time = 30;
+ ht_info->reg_rx_reorder_enable = 1;
+ ht_info->rx_reorder_win_size = 64;
+ ht_info->rx_reorder_pending_time = 30;
}
static u16 HTMcsToDataRate(struct rtllib_device *ieee, u8 nMcsRate)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
- u8 is40MHz = (pHTInfo->bCurBW40MHz) ? 1 : 0;
- u8 isShortGI = (pHTInfo->bCurBW40MHz) ?
- ((pHTInfo->bCurShortGI40MHz) ? 1 : 0) :
- ((pHTInfo->bCurShortGI20MHz) ? 1 : 0);
+ u8 is40MHz = (ht_info->bCurBW40MHz) ? 1 : 0;
+ u8 isShortGI = (ht_info->bCurBW40MHz) ?
+ ((ht_info->bCurShortGI40MHz) ? 1 : 0) :
+ ((ht_info->bCurShortGI20MHz) ? 1 : 0);
return MCS_DATA_RATE[is40MHz][isShortGI][(nMcsRate & 0x7f)];
}
(net->ralink_cap_exist))
retValue = true;
else if (!memcmp(net->bssid, UNKNOWN_BORADCOM, 3) ||
- !memcmp(net->bssid, LINKSYSWRT330_LINKSYSWRT300_BROADCOM, 3) ||
- !memcmp(net->bssid, LINKSYSWRT350_LINKSYSWRT150_BROADCOM, 3) ||
+ !memcmp(net->bssid, LINKSYSWRT330_LINKSYSWRT300_BROADCOM, 3) ||
+ !memcmp(net->bssid, LINKSYSWRT350_LINKSYSWRT150_BROADCOM, 3) ||
(net->broadcom_cap_exist))
retValue = true;
else if (net->bssht.bd_rt2rt_aggregation)
static void HTIOTPeerDetermine(struct rtllib_device *ieee)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
struct rtllib_network *net = &ieee->current_network;
if (net->bssht.bd_rt2rt_aggregation) {
- pHTInfo->IOTPeer = HT_IOT_PEER_REALTEK;
+ ht_info->IOTPeer = HT_IOT_PEER_REALTEK;
if (net->bssht.rt2rt_ht_mode & RT_HT_CAP_USE_92SE)
- pHTInfo->IOTPeer = HT_IOT_PEER_REALTEK_92SE;
+ ht_info->IOTPeer = HT_IOT_PEER_REALTEK_92SE;
if (net->bssht.rt2rt_ht_mode & RT_HT_CAP_USE_SOFTAP)
- pHTInfo->IOTPeer = HT_IOT_PEER_92U_SOFTAP;
+ ht_info->IOTPeer = HT_IOT_PEER_92U_SOFTAP;
} else if (net->broadcom_cap_exist) {
- pHTInfo->IOTPeer = HT_IOT_PEER_BROADCOM;
+ ht_info->IOTPeer = HT_IOT_PEER_BROADCOM;
} else if (!memcmp(net->bssid, UNKNOWN_BORADCOM, 3) ||
!memcmp(net->bssid, LINKSYSWRT330_LINKSYSWRT300_BROADCOM, 3) ||
!memcmp(net->bssid, LINKSYSWRT350_LINKSYSWRT150_BROADCOM, 3)) {
- pHTInfo->IOTPeer = HT_IOT_PEER_BROADCOM;
+ ht_info->IOTPeer = HT_IOT_PEER_BROADCOM;
} else if ((memcmp(net->bssid, BELKINF5D8233V1_RALINK, 3) == 0) ||
(memcmp(net->bssid, BELKINF5D82334V3_RALINK, 3) == 0) ||
(memcmp(net->bssid, PCI_RALINK, 3) == 0) ||
(memcmp(net->bssid, EDIMAX_RALINK, 3) == 0) ||
(memcmp(net->bssid, AIRLINK_RALINK, 3) == 0) ||
net->ralink_cap_exist) {
- pHTInfo->IOTPeer = HT_IOT_PEER_RALINK;
+ ht_info->IOTPeer = HT_IOT_PEER_RALINK;
} else if ((net->atheros_cap_exist) ||
(memcmp(net->bssid, DLINK_ATHEROS_1, 3) == 0) ||
(memcmp(net->bssid, DLINK_ATHEROS_2, 3) == 0)) {
- pHTInfo->IOTPeer = HT_IOT_PEER_ATHEROS;
+ ht_info->IOTPeer = HT_IOT_PEER_ATHEROS;
} else if ((memcmp(net->bssid, CISCO_BROADCOM, 3) == 0) ||
net->cisco_cap_exist) {
- pHTInfo->IOTPeer = HT_IOT_PEER_CISCO;
+ ht_info->IOTPeer = HT_IOT_PEER_CISCO;
} else if ((memcmp(net->bssid, LINKSYS_MARVELL_4400N, 3) == 0) ||
net->marvell_cap_exist) {
- pHTInfo->IOTPeer = HT_IOT_PEER_MARVELL;
+ ht_info->IOTPeer = HT_IOT_PEER_MARVELL;
} else if (net->airgo_cap_exist) {
- pHTInfo->IOTPeer = HT_IOT_PEER_AIRGO;
+ ht_info->IOTPeer = HT_IOT_PEER_AIRGO;
} else {
- pHTInfo->IOTPeer = HT_IOT_PEER_UNKNOWN;
+ ht_info->IOTPeer = HT_IOT_PEER_UNKNOWN;
}
- netdev_dbg(ieee->dev, "IOTPEER: %x\n", pHTInfo->IOTPeer);
+ netdev_dbg(ieee->dev, "IOTPEER: %x\n", ht_info->IOTPeer);
}
static u8 HTIOTActIsDisableMCS14(struct rtllib_device *ieee, u8 *PeerMacAddr)
{
u8 retValue = 0;
- if (ieee->pHTInfo->IOTPeer == HT_IOT_PEER_BROADCOM)
+ if (ieee->ht_info->IOTPeer == HT_IOT_PEER_BROADCOM)
retValue = 1;
return retValue;
{
u8 retValue = 0;
- if (ieee->pHTInfo->IOTPeer == HT_IOT_PEER_BROADCOM)
+ if (ieee->ht_info->IOTPeer == HT_IOT_PEER_BROADCOM)
retValue = 1;
return retValue;
}
static void HTIOTActDetermineRaFunc(struct rtllib_device *ieee, bool bPeerRx2ss)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
- pHTInfo->iot_ra_func &= HT_IOT_RAFUNC_DISABLE_ALL;
+ ht_info->iot_ra_func &= HT_IOT_RAFUNC_DISABLE_ALL;
- if (pHTInfo->IOTPeer == HT_IOT_PEER_RALINK && !bPeerRx2ss)
- pHTInfo->iot_ra_func |= HT_IOT_RAFUNC_PEER_1R;
+ if (ht_info->IOTPeer == HT_IOT_PEER_RALINK && !bPeerRx2ss)
+ ht_info->iot_ra_func |= HT_IOT_RAFUNC_PEER_1R;
- if (pHTInfo->iot_action & HT_IOT_ACT_AMSDU_ENABLE)
- pHTInfo->iot_ra_func |= HT_IOT_RAFUNC_TX_AMSDU;
+ if (ht_info->iot_action & HT_IOT_ACT_AMSDU_ENABLE)
+ ht_info->iot_ra_func |= HT_IOT_RAFUNC_TX_AMSDU;
}
-void HTResetIOTSetting(struct rt_hi_throughput *pHTInfo)
+void HTResetIOTSetting(struct rt_hi_throughput *ht_info)
{
- pHTInfo->iot_action = 0;
- pHTInfo->IOTPeer = HT_IOT_PEER_UNKNOWN;
- pHTInfo->iot_ra_func = 0;
+ ht_info->iot_action = 0;
+ ht_info->IOTPeer = HT_IOT_PEER_UNKNOWN;
+ ht_info->iot_ra_func = 0;
}
void HTConstructCapabilityElement(struct rtllib_device *ieee, u8 *posHTCap,
u8 *len, u8 IsEncrypt, bool bAssoc)
{
- struct rt_hi_throughput *pHT = ieee->pHTInfo;
+ struct rt_hi_throughput *pHT = ieee->ht_info;
struct ht_capab_ele *pCapELE = NULL;
if (!posHTCap || !pHT) {
netdev_warn(ieee->dev,
- "%s(): posHTCap and pHTInfo are null\n", __func__);
+ "%s(): posHTCap and ht_info are null\n", __func__);
return;
}
memset(posHTCap, 0, *len);
if ((bAssoc) && (pHT->ePeerHTSpecVer == HT_SPEC_VER_EWC)) {
- u8 EWC11NHTCap[] = {0x00, 0x90, 0x4c, 0x33};
+ static const u8 EWC11NHTCap[] = { 0x00, 0x90, 0x4c, 0x33 };
memcpy(posHTCap, EWC11NHTCap, sizeof(EWC11NHTCap));
- pCapELE = (struct ht_capab_ele *)&(posHTCap[4]);
+ pCapELE = (struct ht_capab_ele *)&posHTCap[4];
*len = 30 + 2;
} else {
pCapELE = (struct ht_capab_ele *)posHTCap;
pCapELE->MPDUDensity = 0;
}
- memcpy(pCapELE->MCS, ieee->Regdot11HTOperationalRateSet, 16);
+ memcpy(pCapELE->MCS, ieee->reg_dot11ht_oper_rate_set, 16);
memset(&pCapELE->ExtHTCapInfo, 0, 2);
memset(pCapELE->TxBFCap, 0, 4);
void HTConstructInfoElement(struct rtllib_device *ieee, u8 *posHTInfo,
u8 *len, u8 IsEncrypt)
{
- struct rt_hi_throughput *pHT = ieee->pHTInfo;
+ struct rt_hi_throughput *pHT = ieee->ht_info;
struct ht_info_ele *pHTInfoEle = (struct ht_info_ele *)posHTInfo;
if (!posHTInfo || !pHTInfoEle) {
u8 i;
for (i = 0; i <= 15; i++)
- pOperateMCS[i] = ieee->Regdot11TxHTOperationalRateSet[i] &
+ pOperateMCS[i] = ieee->reg_dot11tx_ht_oper_rate_set[i] &
pSupportMCS[i];
HT_PickMCSRate(ieee, pOperateMCS);
void HTOnAssocRsp(struct rtllib_device *ieee)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
struct ht_capab_ele *pPeerHTCap = NULL;
struct ht_info_ele *pPeerHTInfo = NULL;
u16 nMaxAMSDUSize = 0;
u8 *pMcsFilter = NULL;
- static u8 EWC11NHTCap[] = {0x00, 0x90, 0x4c, 0x33};
- static u8 EWC11NHTInfo[] = {0x00, 0x90, 0x4c, 0x34};
+ static const u8 EWC11NHTCap[] = { 0x00, 0x90, 0x4c, 0x33 };
+ static const u8 EWC11NHTInfo[] = { 0x00, 0x90, 0x4c, 0x34 };
- if (!pHTInfo->bCurrentHTSupport) {
+ if (!ht_info->bCurrentHTSupport) {
netdev_warn(ieee->dev, "%s(): HT_DISABLE\n", __func__);
return;
}
netdev_dbg(ieee->dev, "%s(): HT_ENABLE\n", __func__);
- if (!memcmp(pHTInfo->PeerHTCapBuf, EWC11NHTCap, sizeof(EWC11NHTCap)))
- pPeerHTCap = (struct ht_capab_ele *)(&pHTInfo->PeerHTCapBuf[4]);
+ if (!memcmp(ht_info->PeerHTCapBuf, EWC11NHTCap, sizeof(EWC11NHTCap)))
+ pPeerHTCap = (struct ht_capab_ele *)(&ht_info->PeerHTCapBuf[4]);
else
- pPeerHTCap = (struct ht_capab_ele *)(pHTInfo->PeerHTCapBuf);
+ pPeerHTCap = (struct ht_capab_ele *)(ht_info->PeerHTCapBuf);
- if (!memcmp(pHTInfo->PeerHTInfoBuf, EWC11NHTInfo, sizeof(EWC11NHTInfo)))
+ if (!memcmp(ht_info->PeerHTInfoBuf, EWC11NHTInfo, sizeof(EWC11NHTInfo)))
pPeerHTInfo = (struct ht_info_ele *)
- (&pHTInfo->PeerHTInfoBuf[4]);
+ (&ht_info->PeerHTInfoBuf[4]);
else
- pPeerHTInfo = (struct ht_info_ele *)(pHTInfo->PeerHTInfoBuf);
+ pPeerHTInfo = (struct ht_info_ele *)(ht_info->PeerHTInfoBuf);
#ifdef VERBOSE_DEBUG
print_hex_dump_bytes("%s: ", __func__, DUMP_PREFIX_NONE,
pPeerHTCap, sizeof(struct ht_capab_ele));
#endif
HTSetConnectBwMode(ieee, (enum ht_channel_width)(pPeerHTCap->ChlWidth),
- (enum ht_extchnl_offset)(pPeerHTInfo->ExtChlOffset));
- pHTInfo->cur_tx_bw40mhz = ((pPeerHTInfo->RecommemdedTxWidth == 1) ?
+ (enum ht_extchnl_offset)(pPeerHTInfo->ExtChlOffset));
+ ht_info->cur_tx_bw40mhz = ((pPeerHTInfo->RecommemdedTxWidth == 1) ?
true : false);
- pHTInfo->bCurShortGI20MHz = ((pHTInfo->bRegShortGI20MHz) ?
+ ht_info->bCurShortGI20MHz = ((ht_info->bRegShortGI20MHz) ?
((pPeerHTCap->ShortGI20Mhz == 1) ?
true : false) : false);
- pHTInfo->bCurShortGI40MHz = ((pHTInfo->bRegShortGI40MHz) ?
+ ht_info->bCurShortGI40MHz = ((ht_info->bRegShortGI40MHz) ?
((pPeerHTCap->ShortGI40Mhz == 1) ?
true : false) : false);
- pHTInfo->bCurSuppCCK = ((pHTInfo->bRegSuppCCK) ?
+ ht_info->bCurSuppCCK = ((ht_info->bRegSuppCCK) ?
((pPeerHTCap->DssCCk == 1) ? true :
false) : false);
- pHTInfo->bCurrent_AMSDU_Support = pHTInfo->bAMSDU_Support;
+ ht_info->bCurrent_AMSDU_Support = ht_info->bAMSDU_Support;
nMaxAMSDUSize = (pPeerHTCap->MaxAMSDUSize == 0) ? 3839 : 7935;
- if (pHTInfo->nAMSDU_MaxSize > nMaxAMSDUSize)
- pHTInfo->nCurrent_AMSDU_MaxSize = nMaxAMSDUSize;
+ if (ht_info->nAMSDU_MaxSize > nMaxAMSDUSize)
+ ht_info->nCurrent_AMSDU_MaxSize = nMaxAMSDUSize;
else
- pHTInfo->nCurrent_AMSDU_MaxSize = pHTInfo->nAMSDU_MaxSize;
+ ht_info->nCurrent_AMSDU_MaxSize = ht_info->nAMSDU_MaxSize;
- pHTInfo->bCurrentAMPDUEnable = pHTInfo->bAMPDUEnable;
+ ht_info->bCurrentAMPDUEnable = ht_info->bAMPDUEnable;
if (ieee->rtllib_ap_sec_type &&
- (ieee->rtllib_ap_sec_type(ieee) & (SEC_ALG_WEP | SEC_ALG_TKIP))) {
- if ((pHTInfo->IOTPeer == HT_IOT_PEER_ATHEROS) ||
- (pHTInfo->IOTPeer == HT_IOT_PEER_UNKNOWN))
- pHTInfo->bCurrentAMPDUEnable = false;
+ (ieee->rtllib_ap_sec_type(ieee) & (SEC_ALG_WEP | SEC_ALG_TKIP))) {
+ if ((ht_info->IOTPeer == HT_IOT_PEER_ATHEROS) ||
+ (ht_info->IOTPeer == HT_IOT_PEER_UNKNOWN))
+ ht_info->bCurrentAMPDUEnable = false;
}
- if (!pHTInfo->reg_rt2rt_aggregation) {
- if (pHTInfo->AMPDU_Factor > pPeerHTCap->MaxRxAMPDUFactor)
- pHTInfo->CurrentAMPDUFactor =
+ if (!ht_info->reg_rt2rt_aggregation) {
+ if (ht_info->AMPDU_Factor > pPeerHTCap->MaxRxAMPDUFactor)
+ ht_info->CurrentAMPDUFactor =
pPeerHTCap->MaxRxAMPDUFactor;
else
- pHTInfo->CurrentAMPDUFactor = pHTInfo->AMPDU_Factor;
+ ht_info->CurrentAMPDUFactor = ht_info->AMPDU_Factor;
} else {
if (ieee->current_network.bssht.bd_rt2rt_aggregation) {
if (ieee->pairwise_key_type != KEY_TYPE_NA)
- pHTInfo->CurrentAMPDUFactor =
+ ht_info->CurrentAMPDUFactor =
pPeerHTCap->MaxRxAMPDUFactor;
else
- pHTInfo->CurrentAMPDUFactor = HT_AGG_SIZE_64K;
+ ht_info->CurrentAMPDUFactor = HT_AGG_SIZE_64K;
} else {
- if (pPeerHTCap->MaxRxAMPDUFactor < HT_AGG_SIZE_32K)
- pHTInfo->CurrentAMPDUFactor =
- pPeerHTCap->MaxRxAMPDUFactor;
- else
- pHTInfo->CurrentAMPDUFactor = HT_AGG_SIZE_32K;
+ ht_info->CurrentAMPDUFactor = min_t(u32, pPeerHTCap->MaxRxAMPDUFactor,
+ HT_AGG_SIZE_32K);
}
}
- if (pHTInfo->MPDU_Density > pPeerHTCap->MPDUDensity)
- pHTInfo->current_mpdu_density = pHTInfo->MPDU_Density;
- else
- pHTInfo->current_mpdu_density = pPeerHTCap->MPDUDensity;
- if (pHTInfo->iot_action & HT_IOT_ACT_TX_USE_AMSDU_8K) {
- pHTInfo->bCurrentAMPDUEnable = false;
- pHTInfo->ForcedAMSDUMode = HT_AGG_FORCE_ENABLE;
+ ht_info->current_mpdu_density = max_t(u8, ht_info->MPDU_Density,
+ pPeerHTCap->MPDUDensity);
+ if (ht_info->iot_action & HT_IOT_ACT_TX_USE_AMSDU_8K) {
+ ht_info->bCurrentAMPDUEnable = false;
+ ht_info->ForcedAMSDUMode = HT_AGG_FORCE_ENABLE;
}
- pHTInfo->cur_rx_reorder_enable = pHTInfo->reg_rx_reorder_enable;
+ ht_info->cur_rx_reorder_enable = ht_info->reg_rx_reorder_enable;
if (pPeerHTCap->MCS[0] == 0)
pPeerHTCap->MCS[0] = 0xff;
HTIOTActDetermineRaFunc(ieee, ((pPeerHTCap->MCS[1]) != 0));
- HTFilterMCSRate(ieee, pPeerHTCap->MCS, ieee->dot11HTOperationalRateSet);
+ HTFilterMCSRate(ieee, pPeerHTCap->MCS, ieee->dot11ht_oper_rate_set);
- pHTInfo->peer_mimo_ps = pPeerHTCap->MimoPwrSave;
- if (pHTInfo->peer_mimo_ps == MIMO_PS_STATIC)
+ ht_info->peer_mimo_ps = pPeerHTCap->MimoPwrSave;
+ if (ht_info->peer_mimo_ps == MIMO_PS_STATIC)
pMcsFilter = MCS_FILTER_1SS;
else
pMcsFilter = MCS_FILTER_ALL;
ieee->HTHighestOperaRate = HTGetHighestMCSRate(ieee,
- ieee->dot11HTOperationalRateSet, pMcsFilter);
+ ieee->dot11ht_oper_rate_set,
+ pMcsFilter);
ieee->HTCurrentOperaRate = ieee->HTHighestOperaRate;
- pHTInfo->current_op_mode = pPeerHTInfo->OptMode;
+ ht_info->current_op_mode = pPeerHTInfo->OptMode;
}
void HTInitializeHTInfo(struct rtllib_device *ieee)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
- pHTInfo->bCurrentHTSupport = false;
+ ht_info->bCurrentHTSupport = false;
- pHTInfo->bCurBW40MHz = false;
- pHTInfo->cur_tx_bw40mhz = false;
+ ht_info->bCurBW40MHz = false;
+ ht_info->cur_tx_bw40mhz = false;
- pHTInfo->bCurShortGI20MHz = false;
- pHTInfo->bCurShortGI40MHz = false;
- pHTInfo->forced_short_gi = false;
+ ht_info->bCurShortGI20MHz = false;
+ ht_info->bCurShortGI40MHz = false;
+ ht_info->forced_short_gi = false;
- pHTInfo->bCurSuppCCK = true;
+ ht_info->bCurSuppCCK = true;
- pHTInfo->bCurrent_AMSDU_Support = false;
- pHTInfo->nCurrent_AMSDU_MaxSize = pHTInfo->nAMSDU_MaxSize;
- pHTInfo->current_mpdu_density = pHTInfo->MPDU_Density;
- pHTInfo->CurrentAMPDUFactor = pHTInfo->AMPDU_Factor;
+ ht_info->bCurrent_AMSDU_Support = false;
+ ht_info->nCurrent_AMSDU_MaxSize = ht_info->nAMSDU_MaxSize;
+ ht_info->current_mpdu_density = ht_info->MPDU_Density;
+ ht_info->CurrentAMPDUFactor = ht_info->AMPDU_Factor;
- memset((void *)(&(pHTInfo->SelfHTCap)), 0,
- sizeof(pHTInfo->SelfHTCap));
- memset((void *)(&(pHTInfo->SelfHTInfo)), 0,
- sizeof(pHTInfo->SelfHTInfo));
- memset((void *)(&(pHTInfo->PeerHTCapBuf)), 0,
- sizeof(pHTInfo->PeerHTCapBuf));
- memset((void *)(&(pHTInfo->PeerHTInfoBuf)), 0,
- sizeof(pHTInfo->PeerHTInfoBuf));
+ memset((void *)(&ht_info->SelfHTCap), 0,
+ sizeof(ht_info->SelfHTCap));
+ memset((void *)(&ht_info->SelfHTInfo), 0,
+ sizeof(ht_info->SelfHTInfo));
+ memset((void *)(&ht_info->PeerHTCapBuf), 0,
+ sizeof(ht_info->PeerHTCapBuf));
+ memset((void *)(&ht_info->PeerHTInfoBuf), 0,
+ sizeof(ht_info->PeerHTInfoBuf));
- pHTInfo->sw_bw_in_progress = false;
+ ht_info->sw_bw_in_progress = false;
- pHTInfo->ePeerHTSpecVer = HT_SPEC_VER_IEEE;
+ ht_info->ePeerHTSpecVer = HT_SPEC_VER_IEEE;
- pHTInfo->current_rt2rt_aggregation = false;
- pHTInfo->current_rt2rt_long_slot_time = false;
- pHTInfo->RT2RT_HT_Mode = (enum rt_ht_capability)0;
+ ht_info->current_rt2rt_aggregation = false;
+ ht_info->current_rt2rt_long_slot_time = false;
+ ht_info->RT2RT_HT_Mode = (enum rt_ht_capability)0;
- pHTInfo->IOTPeer = 0;
- pHTInfo->iot_action = 0;
- pHTInfo->iot_ra_func = 0;
+ ht_info->IOTPeer = 0;
+ ht_info->iot_action = 0;
+ ht_info->iot_ra_func = 0;
{
- u8 *RegHTSuppRateSets = &(ieee->RegHTSuppRateSet[0]);
+ u8 *RegHTSuppRateSets = &ieee->reg_ht_supp_rate_set[0];
RegHTSuppRateSets[0] = 0xFF;
RegHTSuppRateSets[1] = 0xFF;
void HTResetSelfAndSavePeerSetting(struct rtllib_device *ieee,
struct rtllib_network *pNetwork)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
u8 bIOTAction = 0;
- /* unmark bEnableHT flag here is the same reason why unmarked in
+ /* unmark enable_ht flag here is the same reason why unmarked in
* function rtllib_softmac_new_net. WB 2008.09.10
*/
if (pNetwork->bssht.bd_support_ht) {
- pHTInfo->bCurrentHTSupport = true;
- pHTInfo->ePeerHTSpecVer = pNetwork->bssht.bd_ht_spec_ver;
+ ht_info->bCurrentHTSupport = true;
+ ht_info->ePeerHTSpecVer = pNetwork->bssht.bd_ht_spec_ver;
if (pNetwork->bssht.bd_ht_cap_len > 0 &&
- pNetwork->bssht.bd_ht_cap_len <= sizeof(pHTInfo->PeerHTCapBuf))
- memcpy(pHTInfo->PeerHTCapBuf,
+ pNetwork->bssht.bd_ht_cap_len <= sizeof(ht_info->PeerHTCapBuf))
+ memcpy(ht_info->PeerHTCapBuf,
pNetwork->bssht.bd_ht_cap_buf,
pNetwork->bssht.bd_ht_cap_len);
if (pNetwork->bssht.bd_ht_info_len > 0 &&
pNetwork->bssht.bd_ht_info_len <=
- sizeof(pHTInfo->PeerHTInfoBuf))
- memcpy(pHTInfo->PeerHTInfoBuf,
+ sizeof(ht_info->PeerHTInfoBuf))
+ memcpy(ht_info->PeerHTInfoBuf,
pNetwork->bssht.bd_ht_info_buf,
pNetwork->bssht.bd_ht_info_len);
- if (pHTInfo->reg_rt2rt_aggregation) {
- pHTInfo->current_rt2rt_aggregation =
+ if (ht_info->reg_rt2rt_aggregation) {
+ ht_info->current_rt2rt_aggregation =
pNetwork->bssht.bd_rt2rt_aggregation;
- pHTInfo->current_rt2rt_long_slot_time =
+ ht_info->current_rt2rt_long_slot_time =
pNetwork->bssht.bd_rt2rt_long_slot_time;
- pHTInfo->RT2RT_HT_Mode = pNetwork->bssht.rt2rt_ht_mode;
+ ht_info->RT2RT_HT_Mode = pNetwork->bssht.rt2rt_ht_mode;
} else {
- pHTInfo->current_rt2rt_aggregation = false;
- pHTInfo->current_rt2rt_long_slot_time = false;
- pHTInfo->RT2RT_HT_Mode = (enum rt_ht_capability)0;
+ ht_info->current_rt2rt_aggregation = false;
+ ht_info->current_rt2rt_long_slot_time = false;
+ ht_info->RT2RT_HT_Mode = (enum rt_ht_capability)0;
}
HTIOTPeerDetermine(ieee);
- pHTInfo->iot_action = 0;
+ ht_info->iot_action = 0;
bIOTAction = HTIOTActIsDisableMCS14(ieee, pNetwork->bssid);
if (bIOTAction)
- pHTInfo->iot_action |= HT_IOT_ACT_DISABLE_MCS14;
+ ht_info->iot_action |= HT_IOT_ACT_DISABLE_MCS14;
bIOTAction = HTIOTActIsDisableMCS15(ieee);
if (bIOTAction)
- pHTInfo->iot_action |= HT_IOT_ACT_DISABLE_MCS15;
+ ht_info->iot_action |= HT_IOT_ACT_DISABLE_MCS15;
bIOTAction = HTIOTActIsDisableMCSTwoSpatialStream(ieee);
if (bIOTAction)
- pHTInfo->iot_action |= HT_IOT_ACT_DISABLE_ALL_2SS;
+ ht_info->iot_action |= HT_IOT_ACT_DISABLE_ALL_2SS;
bIOTAction = HTIOTActIsDisableEDCATurbo(ieee, pNetwork->bssid);
if (bIOTAction)
- pHTInfo->iot_action |= HT_IOT_ACT_DISABLE_EDCA_TURBO;
+ ht_info->iot_action |= HT_IOT_ACT_DISABLE_EDCA_TURBO;
bIOTAction = HTIOTActIsMgntUseCCK6M(ieee, pNetwork);
if (bIOTAction)
- pHTInfo->iot_action |= HT_IOT_ACT_MGNT_USE_CCK_6M;
+ ht_info->iot_action |= HT_IOT_ACT_MGNT_USE_CCK_6M;
bIOTAction = HTIOTActIsCCDFsync(ieee);
if (bIOTAction)
- pHTInfo->iot_action |= HT_IOT_ACT_CDD_FSYNC;
+ ht_info->iot_action |= HT_IOT_ACT_CDD_FSYNC;
} else {
- pHTInfo->bCurrentHTSupport = false;
- pHTInfo->current_rt2rt_aggregation = false;
- pHTInfo->current_rt2rt_long_slot_time = false;
- pHTInfo->RT2RT_HT_Mode = (enum rt_ht_capability)0;
+ ht_info->bCurrentHTSupport = false;
+ ht_info->current_rt2rt_aggregation = false;
+ ht_info->current_rt2rt_long_slot_time = false;
+ ht_info->RT2RT_HT_Mode = (enum rt_ht_capability)0;
- pHTInfo->iot_action = 0;
- pHTInfo->iot_ra_func = 0;
+ ht_info->iot_action = 0;
+ ht_info->iot_ra_func = 0;
}
}
void HT_update_self_and_peer_setting(struct rtllib_device *ieee,
struct rtllib_network *pNetwork)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
struct ht_info_ele *pPeerHTInfo =
(struct ht_info_ele *)pNetwork->bssht.bd_ht_info_buf;
- if (pHTInfo->bCurrentHTSupport) {
+ if (ht_info->bCurrentHTSupport) {
if (pNetwork->bssht.bd_ht_info_len != 0)
- pHTInfo->current_op_mode = pPeerHTInfo->OptMode;
+ ht_info->current_op_mode = pPeerHTInfo->OptMode;
}
}
EXPORT_SYMBOL(HT_update_self_and_peer_setting);
void HTUseDefaultSetting(struct rtllib_device *ieee)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
- if (pHTInfo->bEnableHT) {
- pHTInfo->bCurrentHTSupport = true;
- pHTInfo->bCurSuppCCK = pHTInfo->bRegSuppCCK;
+ if (ht_info->enable_ht) {
+ ht_info->bCurrentHTSupport = true;
+ ht_info->bCurSuppCCK = ht_info->bRegSuppCCK;
- pHTInfo->bCurBW40MHz = pHTInfo->bRegBW40MHz;
- pHTInfo->bCurShortGI20MHz = pHTInfo->bRegShortGI20MHz;
+ ht_info->bCurBW40MHz = ht_info->bRegBW40MHz;
+ ht_info->bCurShortGI20MHz = ht_info->bRegShortGI20MHz;
- pHTInfo->bCurShortGI40MHz = pHTInfo->bRegShortGI40MHz;
+ ht_info->bCurShortGI40MHz = ht_info->bRegShortGI40MHz;
if (ieee->iw_mode == IW_MODE_ADHOC)
ieee->current_network.qos_data.active =
ieee->current_network.qos_data.supported;
- pHTInfo->bCurrent_AMSDU_Support = pHTInfo->bAMSDU_Support;
- pHTInfo->nCurrent_AMSDU_MaxSize = pHTInfo->nAMSDU_MaxSize;
+ ht_info->bCurrent_AMSDU_Support = ht_info->bAMSDU_Support;
+ ht_info->nCurrent_AMSDU_MaxSize = ht_info->nAMSDU_MaxSize;
- pHTInfo->bCurrentAMPDUEnable = pHTInfo->bAMPDUEnable;
- pHTInfo->CurrentAMPDUFactor = pHTInfo->AMPDU_Factor;
+ ht_info->bCurrentAMPDUEnable = ht_info->bAMPDUEnable;
+ ht_info->CurrentAMPDUFactor = ht_info->AMPDU_Factor;
- pHTInfo->current_mpdu_density = pHTInfo->current_mpdu_density;
+ ht_info->current_mpdu_density = ht_info->current_mpdu_density;
- HTFilterMCSRate(ieee, ieee->Regdot11TxHTOperationalRateSet,
- ieee->dot11HTOperationalRateSet);
+ HTFilterMCSRate(ieee, ieee->reg_dot11tx_ht_oper_rate_set,
+ ieee->dot11ht_oper_rate_set);
ieee->HTHighestOperaRate = HTGetHighestMCSRate(ieee,
- ieee->dot11HTOperationalRateSet,
- MCS_FILTER_ALL);
+ ieee->dot11ht_oper_rate_set,
+ MCS_FILTER_ALL);
ieee->HTCurrentOperaRate = ieee->HTHighestOperaRate;
} else {
- pHTInfo->bCurrentHTSupport = false;
+ ht_info->bCurrentHTSupport = false;
}
}
u8 HTCCheck(struct rtllib_device *ieee, u8 *pFrame)
{
- if (ieee->pHTInfo->bCurrentHTSupport) {
+ if (ieee->ht_info->bCurrentHTSupport) {
if ((IsQoSDataFrame(pFrame) && Frame_Order(pFrame)) == 1) {
netdev_dbg(ieee->dev, "HT CONTROL FILED EXIST!!\n");
return true;
static void HTSetConnectBwModeCallback(struct rtllib_device *ieee)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
- if (pHTInfo->bCurBW40MHz) {
- if (pHTInfo->CurSTAExtChnlOffset == HT_EXTCHNL_OFFSET_UPPER)
+ if (ht_info->bCurBW40MHz) {
+ if (ht_info->CurSTAExtChnlOffset == HT_EXTCHNL_OFFSET_UPPER)
ieee->set_chan(ieee->dev,
ieee->current_network.channel + 2);
- else if (pHTInfo->CurSTAExtChnlOffset ==
+ else if (ht_info->CurSTAExtChnlOffset ==
HT_EXTCHNL_OFFSET_LOWER)
ieee->set_chan(ieee->dev,
ieee->current_network.channel - 2);
ieee->current_network.channel);
ieee->SetBWModeHandler(ieee->dev, HT_CHANNEL_WIDTH_20_40,
- pHTInfo->CurSTAExtChnlOffset);
+ ht_info->CurSTAExtChnlOffset);
} else {
ieee->set_chan(ieee->dev, ieee->current_network.channel);
ieee->SetBWModeHandler(ieee->dev, HT_CHANNEL_WIDTH_20,
HT_EXTCHNL_OFFSET_NO_EXT);
}
- pHTInfo->sw_bw_in_progress = false;
+ ht_info->sw_bw_in_progress = false;
}
void HTSetConnectBwMode(struct rtllib_device *ieee,
enum ht_channel_width bandwidth,
enum ht_extchnl_offset Offset)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
- if (!pHTInfo->bRegBW40MHz)
+ if (!ht_info->bRegBW40MHz)
return;
if (ieee->GetHalfNmodeSupportByAPsHandler(ieee->dev))
bandwidth = HT_CHANNEL_WIDTH_20;
- if (pHTInfo->sw_bw_in_progress) {
+ if (ht_info->sw_bw_in_progress) {
pr_info("%s: sw_bw_in_progress!!\n", __func__);
return;
}
Offset = HT_EXTCHNL_OFFSET_NO_EXT;
if (Offset == HT_EXTCHNL_OFFSET_UPPER ||
Offset == HT_EXTCHNL_OFFSET_LOWER) {
- pHTInfo->bCurBW40MHz = true;
- pHTInfo->CurSTAExtChnlOffset = Offset;
+ ht_info->bCurBW40MHz = true;
+ ht_info->CurSTAExtChnlOffset = Offset;
} else {
- pHTInfo->bCurBW40MHz = false;
- pHTInfo->CurSTAExtChnlOffset = HT_EXTCHNL_OFFSET_NO_EXT;
+ ht_info->bCurBW40MHz = false;
+ ht_info->CurSTAExtChnlOffset = HT_EXTCHNL_OFFSET_NO_EXT;
}
} else {
- pHTInfo->bCurBW40MHz = false;
- pHTInfo->CurSTAExtChnlOffset = HT_EXTCHNL_OFFSET_NO_EXT;
+ ht_info->bCurBW40MHz = false;
+ ht_info->CurSTAExtChnlOffset = HT_EXTCHNL_OFFSET_NO_EXT;
}
- netdev_dbg(ieee->dev, "%s():pHTInfo->bCurBW40MHz:%x\n", __func__,
- pHTInfo->bCurBW40MHz);
+ netdev_dbg(ieee->dev, "%s():ht_info->bCurBW40MHz:%x\n", __func__,
+ ht_info->bCurBW40MHz);
- pHTInfo->sw_bw_in_progress = true;
+ ht_info->sw_bw_in_progress = true;
HTSetConnectBwModeCallback(ieee);
}
DIR_BI_DIR = 3,
};
-enum acm_method {
- eAcmWay0_SwAndHw = 0,
- eAcmWay1_HW = 1,
- eAcmWay2_SW = 2,
-};
-
-
struct acm {
u64 UsedTime;
u64 MediumTime;
if (bPktInBuf && (pRxTs->rx_timeout_indicate_seq == 0xffff)) {
pRxTs->rx_timeout_indicate_seq = pRxTs->rx_indicate_seq;
mod_timer(&pRxTs->rx_pkt_pending_timer, jiffies +
- msecs_to_jiffies(ieee->pHTInfo->rx_reorder_pending_time)
+ msecs_to_jiffies(ieee->ht_info->rx_reorder_pending_time)
);
}
spin_unlock_irqrestore(&(ieee->reorder_spinlock), flags);
#define SUPPORT_CKIP_MIC 0x08
#define SUPPORT_CKIP_PK 0x10
#define RT_RF_OFF_LEVL_HALT_NIC BIT3
-#define RT_IN_PS_LEVEL(pPSC, _PS_FLAG) \
- ((pPSC->CurPsLevel & _PS_FLAG) ? true : false)
-#define RT_CLEAR_PS_LEVEL(pPSC, _PS_FLAG) \
- (pPSC->CurPsLevel &= (~(_PS_FLAG)))
-#define RT_SET_PS_LEVEL(pPSC, _PS_FLAG) (pPSC->CurPsLevel |= _PS_FLAG)
+#define RT_IN_PS_LEVEL(psc, _PS_FLAG) \
+ ((psc->CurPsLevel & _PS_FLAG) ? true : false)
+#define RT_CLEAR_PS_LEVEL(psc, _PS_FLAG) \
+ (psc->CurPsLevel &= (~(_PS_FLAG)))
+#define RT_SET_PS_LEVEL(psc, _PS_FLAG) (psc->CurPsLevel |= _PS_FLAG)
/* defined for skb cb field */
/* At most 28 byte */
u8 bFirstSeg:1;
u8 bLastSeg:1;
u8 bEncrypt:1;
- u8 bTxDisableRateFallBack:1;
- u8 bTxUseDriverAssingedRate:1;
+ u8 tx_dis_rate_fallback:1;
+ u8 tx_use_drv_assinged_rate:1;
u8 bHwSec:1;
u8 nStuckCount;
};
struct rt_pwr_save_ctrl {
-
- bool bInactivePs;
- bool bIPSModeBackup;
bool bSwRfProcessing;
enum rt_rf_power_state eInactivePowerState;
enum ips_callback_function ReturnPoint;
bool bLeisurePs;
u8 LpsIdleCount;
- u8 RegMaxLPSAwakeIntvl;
+ u8 reg_max_lps_awake_intvl;
u8 LPSAwakeIntvl;
u32 CurPsLevel;
u32 RegRfPsLevel;
-
- bool bFwCtrlLPS;
-
};
#define RT_RF_CHANGE_SOURCE u32
};
struct rt_intel_promisc_mode {
- bool bPromiscuousOn;
- bool bFilterSourceStationFrame;
+ bool promiscuous_on;
+ bool fltr_src_sta_frame;
};
RT_RF_CHANGE_SOURCE rf_off_reason;
bool is_set_key;
bool wx_set_enc;
- struct rt_hi_throughput *pHTInfo;
+ struct rt_hi_throughput *ht_info;
spinlock_t reorder_spinlock;
- u8 Regdot11HTOperationalRateSet[16];
- u8 Regdot11TxHTOperationalRateSet[16];
- u8 dot11HTOperationalRateSet[16];
- u8 RegHTSuppRateSet[16];
+ u8 reg_dot11ht_oper_rate_set[16];
+ u8 reg_dot11tx_ht_oper_rate_set[16];
+ u8 dot11ht_oper_rate_set[16];
+ u8 reg_ht_supp_rate_set[16];
u8 HTCurrentOperaRate;
u8 HTHighestOperaRate;
- u8 bTxDisableRateFallBack;
- u8 bTxUseDriverAssingedRate;
+ u8 tx_dis_rate_fallback;
+ u8 tx_use_drv_assinged_rate;
u8 bTxEnableFwCalcDur;
atomic_t atm_swbw;
int scan_age;
int iw_mode; /* operating mode (IW_MODE_*) */
- bool bNetPromiscuousMode;
- struct rt_intel_promisc_mode IntelPromiscuousModeInfo;
+ bool net_promiscuous_md;
+ struct rt_intel_promisc_mode intel_promiscuous_md_info;
spinlock_t lock;
spinlock_t wpax_suitlist_lock;
int mgmt_queue_tail;
u8 AsocRetryCount;
struct sk_buff_head skb_waitQ[MAX_QUEUE_SIZE];
- struct sk_buff_head skb_aggQ[MAX_QUEUE_SIZE];
bool bdynamic_txpower_enable;
struct bandwidth_autoswitch bandwidth_auto_switch;
bool FwRWRF;
- struct rt_link_detect LinkDetectInfo;
+ struct rt_link_detect link_detect_info;
bool bIsAggregateFrame;
- struct rt_pwr_save_ctrl PowerSaveControl;
+ struct rt_pwr_save_ctrl pwr_save_ctrl;
/* used if IEEE_SOFTMAC_TX_QUEUE is set */
struct tx_pending tx_pending;
extern u8 MCS_FILTER_ALL[];
extern u16 MCS_DATA_RATE[2][2][77];
u8 HTCCheck(struct rtllib_device *ieee, u8 *pFrame);
-void HTResetIOTSetting(struct rt_hi_throughput *pHTInfo);
+void HTResetIOTSetting(struct rt_hi_throughput *ht_info);
bool IsHTHalfNmodeAPs(struct rtllib_device *ieee);
u16 TxCountToDataRate(struct rtllib_device *ieee, u8 nDataRate);
int rtllib_rx_ADDBAReq(struct rtllib_device *ieee, struct sk_buff *skb);
return NULL;
priv = kzalloc(sizeof(*priv), GFP_ATOMIC);
- if (priv == NULL)
+ if (!priv)
goto fail;
priv->key_idx = key_idx;
return NULL;
}
-
static void rtllib_tkip_deinit(void *priv)
{
struct rtllib_tkip_data *_priv = priv;
kfree_sensitive(priv);
}
-
static inline u16 RotR1(u16 val)
{
return (val >> 1) | (val << 15);
}
-
static inline u8 Lo8(u16 val)
{
return val & 0xff;
}
-
static inline u8 Hi8(u16 val)
{
return val >> 8;
}
-
static inline u16 Lo16(u32 val)
{
return val & 0xffff;
}
-
static inline u16 Hi16(u32 val)
{
return val >> 16;
}
-
static inline u16 Mk16(u8 hi, u8 lo)
{
return lo | (hi << 8);
}
-
static inline u16 Mk16_le(u16 *v)
{
return *v;
}
-
static const u16 Sbox[256] = {
0xC6A5, 0xF884, 0xEE99, 0xF68D, 0xFF0D, 0xD6BD, 0xDEB1, 0x9154,
0x6050, 0x0203, 0xCEA9, 0x567D, 0xE719, 0xB562, 0x4DE6, 0xEC9A,
0x82C3, 0x29B0, 0x5A77, 0x1E11, 0x7BCB, 0xA8FC, 0x6DD6, 0x2C3A,
};
-
static inline u16 _S_(u16 v)
{
u16 t = Sbox[Hi8(v)];
return Sbox[Lo8(v)] ^ ((t << 8) | (t >> 8));
}
-
#define PHASE1_LOOP_COUNT 8
-
static void tkip_mixing_phase1(u16 *TTAK, const u8 *TK, const u8 *TA, u32 IV32)
{
int i, j;
}
}
-
static void tkip_mixing_phase2(u8 *WEPSeed, const u8 *TK, const u16 *TTAK,
u16 IV16)
{
#endif
}
-
static int rtllib_tkip_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
{
struct rtllib_tkip_data *tkey = priv;
if (!tcb_desc->bHwSec) {
if (!tkey->tx_phase1_done) {
tkip_mixing_phase1(tkey->tx_ttak, tkey->key, hdr->addr2,
- tkey->tx_iv32);
+ tkey->tx_iv32);
tkey->tx_phase1_done = 1;
}
tkip_mixing_phase2(rc4key, tkey->key, tkey->tx_ttak,
tkey->tx_iv16);
- } else
+ } else {
tkey->tx_phase1_done = 1;
-
+ }
len = skb->len - hdr_len;
pos = skb_push(skb, 8);
if (!tcb_desc->bHwSec)
return ret;
return 0;
-
-
}
static int rtllib_tkip_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
if (!tcb_desc->bHwSec || (skb->cb[0] == 1)) {
if ((iv32 < tkey->rx_iv32 ||
- (iv32 == tkey->rx_iv32 && iv16 <= tkey->rx_iv16)) &&
- tkey->initialized) {
+ (iv32 == tkey->rx_iv32 && iv16 <= tkey->rx_iv16)) &&
+ tkey->initialized) {
if (net_ratelimit()) {
netdev_dbg(skb->dev,
"Replay detected: STA= %pM previous TSC %08x%04x received TSC %08x%04x\n",
tkey->dot11RSNAStatsTKIPICVErrors++;
return -5;
}
-
}
/* Update real counters only after Michael MIC verification has
return keyidx;
}
-
static int michael_mic(struct crypto_shash *tfm_michael, u8 *key, u8 *hdr,
u8 *data, size_t data_len, u8 *mic)
{
break;
}
- hdr[12] = 0; /* priority */
+ /* priority */
+ hdr[12] = 0;
- hdr[13] = hdr[14] = hdr[15] = 0; /* reserved */
+ /* reserved */
+ hdr[13] = 0;
+ hdr[14] = 0;
+ hdr[15] = 0;
}
-
static int rtllib_michael_mic_add(struct sk_buff *skb, int hdr_len, void *priv)
{
struct rtllib_tkip_data *tkey = priv;
tkey->tx_hdr[12] = *(skb->data + hdr_len - 2) & 0x07;
pos = skb_put(skb, 8);
if (michael_mic(tkey->tx_tfm_michael, &tkey->key[16], tkey->tx_hdr,
- skb->data + hdr_len, skb->len - 8 - hdr_len, pos))
+ skb->data + hdr_len, skb->len - 8 - hdr_len, pos))
return -1;
return 0;
}
-
static void rtllib_michael_mic_failure(struct net_device *dev,
struct rtllib_hdr_4addr *hdr,
int keyidx)
return 0;
}
-
static int rtllib_tkip_set_key(void *key, int len, u8 *seq, void *priv)
{
struct rtllib_tkip_data *tkey = priv;
(seq[3] << 8) | seq[2];
tkey->rx_iv16 = (seq[1] << 8) | seq[0];
}
- } else if (len == 0)
+ } else if (len == 0) {
tkey->key_set = 0;
- else
+ } else {
return -1;
+ }
return 0;
}
-
static int rtllib_tkip_get_key(void *key, int len, u8 *seq, void *priv)
{
struct rtllib_tkip_data *tkey = priv;
return TKIP_KEY_LEN;
}
-
static void rtllib_tkip_print_stats(struct seq_file *m, void *priv)
{
struct rtllib_tkip_data *tkip = priv;
.owner = THIS_MODULE,
};
-
static int __init rtllib_crypto_tkip_init(void)
{
return lib80211_register_crypto_ops(&rtllib_crypt_tkip);
}
-
static void __exit rtllib_crypto_tkip_exit(void)
{
lib80211_unregister_crypto_ops(&rtllib_crypt_tkip);
struct arc4_ctx tx_ctx_arc4;
};
-
static void *prism2_wep_init(int keyidx)
{
struct prism2_wep_data *priv;
return priv;
}
-
static void prism2_wep_deinit(void *priv)
{
kfree_sensitive(priv);
return 0;
}
-
/* Perform WEP decryption on given struct buffer. Buffer includes whole WEP
* part of the frame: IV (4 bytes), encrypted payload (including SNAP header),
* ICV (4 bytes). len includes both IV and ICV.
return 0;
}
-
static int prism2_wep_set_key(void *key, int len, u8 *seq, void *priv)
{
struct prism2_wep_data *wep = priv;
return 0;
}
-
static int prism2_wep_get_key(void *key, int len, u8 *seq, void *priv)
{
struct prism2_wep_data *wep = priv;
return wep->key_len;
}
-
static void prism2_wep_print_stats(struct seq_file *m, void *priv)
{
struct prism2_wep_data *wep = priv;
.owner = THIS_MODULE,
};
-
static int __init rtllib_crypto_wep_init(void)
{
return lib80211_register_crypto_ops(&rtllib_crypt_wep);
}
-
static void __exit rtllib_crypto_wep_exit(void)
{
lib80211_unregister_crypto_ops(&rtllib_crypt_wep);
spin_lock_init(&ieee->lock);
spin_lock_init(&ieee->wpax_suitlist_lock);
spin_lock_init(&ieee->reorder_spinlock);
- atomic_set(&(ieee->atm_swbw), 0);
+ atomic_set(&ieee->atm_swbw, 0);
/* SAM FIXME */
lib80211_crypt_info_init(&ieee->crypt_info, "RTLLIB", &ieee->lock);
if (err)
goto free_crypt_info;
- ieee->pHTInfo = kzalloc(sizeof(struct rt_hi_throughput), GFP_KERNEL);
- if (!ieee->pHTInfo)
+ ieee->ht_info = kzalloc(sizeof(struct rt_hi_throughput), GFP_KERNEL);
+ if (!ieee->ht_info)
goto free_softmac;
HTUpdateDefaultSetting(ieee);
struct rtllib_device *ieee = (struct rtllib_device *)
netdev_priv_rsl(dev);
- kfree(ieee->pHTInfo);
+ kfree(ieee->ht_info);
rtllib_softmac_free(ieee);
lib80211_crypt_info_free(&ieee->crypt_info);
struct rtllib_rxb *prxb,
struct rx_ts_record *pTS, u16 SeqNum)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
struct rx_reorder_entry *pReorderEntry = NULL;
- u8 WinSize = pHTInfo->rx_reorder_win_size;
+ u8 WinSize = ht_info->rx_reorder_win_size;
u16 WinEnd = 0;
u8 index = 0;
bool bMatchWinStart = false, bPktInBuf = false;
netdev_dbg(ieee->dev,
"Packet Drop! IndicateSeq: %d, NewSeq: %d\n",
pTS->rx_indicate_seq, SeqNum);
- pHTInfo->rx_reorder_drop_counter++;
+ ht_info->rx_reorder_drop_counter++;
{
int i;
netdev_dbg(ieee->dev, "%s(): SET rx timeout timer\n", __func__);
pTS->rx_timeout_indicate_seq = pTS->rx_indicate_seq;
mod_timer(&pTS->rx_pkt_pending_timer, jiffies +
- msecs_to_jiffies(pHTInfo->rx_reorder_pending_time));
+ msecs_to_jiffies(ht_info->rx_reorder_pending_time));
}
spin_unlock_irqrestore(&(ieee->reorder_spinlock), flags);
}
sc = le16_to_cpu(hdr->seq_ctl);
frag = WLAN_GET_SEQ_FRAG(sc);
- if (!ieee->pHTInfo->cur_rx_reorder_enable ||
+ if (!ieee->ht_info->cur_rx_reorder_enable ||
!ieee->current_network.qos_data.active ||
!IsDataFrame(skb->data) ||
IsLegacyDataFrame(skb->data)) {
}
/* Filter packets sent by an STA that will be forwarded by AP */
- if (ieee->IntelPromiscuousModeInfo.bPromiscuousOn &&
- ieee->IntelPromiscuousModeInfo.bFilterSourceStationFrame) {
+ if (ieee->intel_promiscuous_md_info.promiscuous_on &&
+ ieee->intel_promiscuous_md_info.fltr_src_sta_frame) {
if ((fc & RTLLIB_FCTL_TODS) && !(fc & RTLLIB_FCTL_FROMDS) &&
!ether_addr_equal(dst, ieee->current_network.bssid) &&
ether_addr_equal(bssid, ieee->current_network.bssid)) {
/* Nullfunc frames may have PS-bit set, so they must be passed to
* hostap_handle_sta_rx() before being dropped here.
*/
- if (!ieee->IntelPromiscuousModeInfo.bPromiscuousOn) {
+ if (!ieee->intel_promiscuous_md_info.promiscuous_on) {
if (stype != RTLLIB_STYPE_DATA &&
stype != RTLLIB_STYPE_DATA_CFACK &&
stype != RTLLIB_STYPE_DATA_CFPOLL &&
if (unicast) {
if (ieee->state == RTLLIB_LINKED) {
- if (((ieee->LinkDetectInfo.NumRxUnicastOkInPeriod +
- ieee->LinkDetectInfo.NumTxOkInPeriod) > 8) ||
- (ieee->LinkDetectInfo.NumRxUnicastOkInPeriod > 2)) {
+ if (((ieee->link_detect_info.NumRxUnicastOkInPeriod +
+ ieee->link_detect_info.NumTxOkInPeriod) > 8) ||
+ (ieee->link_detect_info.NumRxUnicastOkInPeriod > 2)) {
if (ieee->LeisurePSLeave)
ieee->LeisurePSLeave(ieee->dev);
}
multicast = is_multicast_ether_addr(hdr->addr1);
unicast = !multicast;
if (unicast && !ether_addr_equal(dev->dev_addr, hdr->addr1)) {
- if (ieee->bNetPromiscuousMode)
+ if (ieee->net_promiscuous_md)
bToOtherSTA = true;
else
goto rx_dropped;
/* Update statstics for AP roaming */
if (!bToOtherSTA) {
- ieee->LinkDetectInfo.NumRecvDataInPeriod++;
- ieee->LinkDetectInfo.NumRxOkInPeriod++;
+ ieee->link_detect_info.NumRecvDataInPeriod++;
+ ieee->link_detect_info.NumRxOkInPeriod++;
}
/* Data frame - extract src/dst addresses */
else
nr_subframes = 1;
if (unicast)
- ieee->LinkDetectInfo.NumRxUnicastOkInPeriod += nr_subframes;
+ ieee->link_detect_info.NumRxUnicastOkInPeriod += nr_subframes;
rtllib_rx_check_leave_lps(ieee, unicast, nr_subframes);
}
/* Indicate packets to upper layer or Rx Reorder */
- if (!ieee->pHTInfo->cur_rx_reorder_enable || pTS == NULL || bToOtherSTA)
+ if (!ieee->ht_info->cur_rx_reorder_enable || pTS == NULL || bToOtherSTA)
rtllib_rx_indicate_pkt_legacy(ieee, rx_stats, rxb, dst, src);
else
RxReorderIndicatePacket(ieee, rxb, pTS, SeqNum);
hdrlen += 4;
}
- rtllib_monitor_rx(ieee, skb, rx_stats, hdrlen);
ieee->stats.rx_packets++;
ieee->stats.rx_bytes += skb->len;
+ rtllib_monitor_rx(ieee, skb, rx_stats, hdrlen);
return 1;
}
if (rtllib_act_scanning(ieee, false) &&
ieee->FirstIe_InScan)
netdev_info(ieee->dev,
- "Received beacon ContryIE, SSID: <%s>\n",
+ "Received beacon CountryIE, SSID: <%s>\n",
network->ssid);
dot11d_update_country(ieee, addr2,
info_element->len,
}
if (is_beacon(frame_ctl)) {
if (ieee->state >= RTLLIB_LINKED)
- ieee->LinkDetectInfo.NumRecvBcnInPeriod++;
+ ieee->link_detect_info.NumRecvBcnInPeriod++;
}
}
list_for_each_entry(target, &ieee->network_list, list) {
}
-u8
-MgntQuery_TxRateExcludeCCKRates(struct rtllib_device *ieee)
+u8 MgntQuery_TxRateExcludeCCKRates(struct rtllib_device *ieee)
{
u16 i;
u8 QueryRate = 0;
static u8 MgntQuery_MgntFrameTxRate(struct rtllib_device *ieee)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
u8 rate;
- if (pHTInfo->iot_action & HT_IOT_ACT_MGNT_USE_CCK_6M)
+ if (ht_info->iot_action & HT_IOT_ACT_MGNT_USE_CCK_6M)
rate = 0x0c;
else
rate = ieee->basic_rate & 0x7f;
if (rate == 0) {
if (ieee->mode == IEEE_A ||
ieee->mode == IEEE_N_5G ||
- (ieee->mode == IEEE_N_24G && !pHTInfo->bCurSuppCCK))
+ (ieee->mode == IEEE_N_24G && !ht_info->bCurSuppCCK))
rate = 0x0c;
else
rate = 0x02;
tcb_desc->data_rate = MgntQuery_MgntFrameTxRate(ieee);
tcb_desc->RATRIndex = 7;
- tcb_desc->bTxDisableRateFallBack = 1;
- tcb_desc->bTxUseDriverAssingedRate = 1;
+ tcb_desc->tx_dis_rate_fallback = 1;
+ tcb_desc->tx_use_drv_assinged_rate = 1;
if (single) {
if (ieee->queue_stop) {
enqueue_mgmt(ieee, skb);
tcb_desc->data_rate = MgntQuery_MgntFrameTxRate(ieee);
tcb_desc->RATRIndex = 7;
- tcb_desc->bTxDisableRateFallBack = 1;
- tcb_desc->bTxUseDriverAssingedRate = 1;
+ tcb_desc->tx_dis_rate_fallback = 1;
+ tcb_desc->tx_use_drv_assinged_rate = 1;
if (single) {
if (type != RTLLIB_FTYPE_CTL) {
header->seq_ctl = cpu_to_le16(ieee->seq_ctrl[0] << 4);
ieee->SetHwRegHandler(dev, HW_VAR_CECHK_BSSID,
(u8 *)&bFilterOutNonAssociatedBSSID);
- ieee->bNetPromiscuousMode = true;
+ ieee->net_promiscuous_md = true;
}
EXPORT_SYMBOL(rtllib_EnableIntelPromiscuousMode);
ieee->SetHwRegHandler(dev, HW_VAR_CECHK_BSSID,
(u8 *)&bFilterOutNonAssociatedBSSID);
- ieee->bNetPromiscuousMode = false;
+ ieee->net_promiscuous_md = false;
}
EXPORT_SYMBOL(rtllib_DisableIntelPromiscuousMode);
u8 tmp_ht_cap_len = 0;
u8 *tmp_ht_info_buf = NULL;
u8 tmp_ht_info_len = 0;
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
u8 *tmp_generic_ie_buf = NULL;
u8 tmp_generic_ie_len = 0;
if ((ieee->current_network.mode == IEEE_G) ||
(ieee->current_network.mode == IEEE_N_24G &&
- ieee->pHTInfo->bCurSuppCCK)) {
+ ieee->ht_info->bCurSuppCCK)) {
erp_len = 3;
erpinfo_content = 0;
if (ieee->current_network.buseprotection)
crypt = ieee->crypt_info.crypt[ieee->crypt_info.tx_keyidx];
encrypt = ieee->host_encrypt && crypt && crypt->ops &&
((strcmp(crypt->ops->name, "R-WEP") == 0 || wpa_ie_len));
- if (ieee->pHTInfo->bCurrentHTSupport) {
- tmp_ht_cap_buf = (u8 *)&(ieee->pHTInfo->SelfHTCap);
- tmp_ht_cap_len = sizeof(ieee->pHTInfo->SelfHTCap);
- tmp_ht_info_buf = (u8 *)&(ieee->pHTInfo->SelfHTInfo);
- tmp_ht_info_len = sizeof(ieee->pHTInfo->SelfHTInfo);
+ if (ieee->ht_info->bCurrentHTSupport) {
+ tmp_ht_cap_buf = (u8 *)&(ieee->ht_info->SelfHTCap);
+ tmp_ht_cap_len = sizeof(ieee->ht_info->SelfHTCap);
+ tmp_ht_info_buf = (u8 *)&(ieee->ht_info->SelfHTInfo);
+ tmp_ht_info_len = sizeof(ieee->ht_info->SelfHTInfo);
HTConstructCapabilityElement(ieee, tmp_ht_cap_buf,
&tmp_ht_cap_len, encrypt, false);
HTConstructInfoElement(ieee, tmp_ht_info_buf, &tmp_ht_info_len,
encrypt);
- if (pHTInfo->reg_rt2rt_aggregation) {
- tmp_generic_ie_buf = ieee->pHTInfo->sz_rt2rt_agg_buf;
+ if (ht_info->reg_rt2rt_aggregation) {
+ tmp_generic_ie_buf = ieee->ht_info->sz_rt2rt_agg_buf;
tmp_generic_ie_len =
- sizeof(ieee->pHTInfo->sz_rt2rt_agg_buf);
+ sizeof(ieee->ht_info->sz_rt2rt_agg_buf);
HTConstructRT2RTAggElement(ieee, tmp_generic_ie_buf,
&tmp_generic_ie_len);
}
if ((ieee->rtllib_ap_sec_type &&
(ieee->rtllib_ap_sec_type(ieee) & SEC_ALG_TKIP)) ||
ieee->bForcedBgMode) {
- ieee->pHTInfo->bEnableHT = 0;
+ ieee->ht_info->enable_ht = 0;
ieee->mode = WIRELESS_MODE_G;
}
- if (ieee->pHTInfo->bCurrentHTSupport && ieee->pHTInfo->bEnableHT) {
- ht_cap_buf = (u8 *)&(ieee->pHTInfo->SelfHTCap);
- ht_cap_len = sizeof(ieee->pHTInfo->SelfHTCap);
+ if (ieee->ht_info->bCurrentHTSupport && ieee->ht_info->enable_ht) {
+ ht_cap_buf = (u8 *)&(ieee->ht_info->SelfHTCap);
+ ht_cap_len = sizeof(ieee->ht_info->SelfHTCap);
HTConstructCapabilityElement(ieee, ht_cap_buf, &ht_cap_len,
encrypt, true);
- if (ieee->pHTInfo->current_rt2rt_aggregation) {
- realtek_ie_buf = ieee->pHTInfo->sz_rt2rt_agg_buf;
+ if (ieee->ht_info->current_rt2rt_aggregation) {
+ realtek_ie_buf = ieee->ht_info->sz_rt2rt_agg_buf;
realtek_ie_len =
- sizeof(ieee->pHTInfo->sz_rt2rt_agg_buf);
+ sizeof(ieee->ht_info->sz_rt2rt_agg_buf);
HTConstructRT2RTAggElement(ieee, realtek_ie_buf,
&realtek_ie_len);
}
memcpy(tag, osCcxVerNum.Octet, osCcxVerNum.Length);
tag += osCcxVerNum.Length;
}
- if (ieee->pHTInfo->bCurrentHTSupport && ieee->pHTInfo->bEnableHT) {
- if (ieee->pHTInfo->ePeerHTSpecVer != HT_SPEC_VER_EWC) {
+ if (ieee->ht_info->bCurrentHTSupport && ieee->ht_info->enable_ht) {
+ if (ieee->ht_info->ePeerHTSpecVer != HT_SPEC_VER_EWC) {
tag = skb_put(skb, ht_cap_len);
*tag++ = MFIE_TYPE_HT_CAP;
*tag++ = ht_cap_len - 2;
rtllib_TURBO_Info(ieee, &tag);
}
- if (ieee->pHTInfo->bCurrentHTSupport && ieee->pHTInfo->bEnableHT) {
- if (ieee->pHTInfo->ePeerHTSpecVer == HT_SPEC_VER_EWC) {
+ if (ieee->ht_info->bCurrentHTSupport && ieee->ht_info->enable_ht) {
+ if (ieee->ht_info->ePeerHTSpecVer == HT_SPEC_VER_EWC) {
tag = skb_put(skb, ht_cap_len);
*tag++ = MFIE_TYPE_GENERIC;
*tag++ = ht_cap_len - 2;
tag += ht_cap_len - 2;
}
- if (ieee->pHTInfo->current_rt2rt_aggregation) {
+ if (ieee->ht_info->current_rt2rt_aggregation) {
tag = skb_put(skb, realtek_ie_len);
*tag++ = MFIE_TYPE_GENERIC;
*tag++ = realtek_ie_len - 2;
container_of_work_rsl(data,
struct rtllib_device,
associate_complete_wq);
- struct rt_pwr_save_ctrl *pPSC = &(ieee->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = &ieee->pwr_save_ctrl;
netdev_info(ieee->dev, "Associated successfully with %pM\n",
ieee->current_network.bssid);
ieee->SetWirelessMode(ieee->dev, IEEE_B);
netdev_info(ieee->dev, "Using B rates:%d\n", ieee->rate);
}
- if (ieee->pHTInfo->bCurrentHTSupport && ieee->pHTInfo->bEnableHT) {
+ if (ieee->ht_info->bCurrentHTSupport && ieee->ht_info->enable_ht) {
netdev_info(ieee->dev, "Successfully associated, ht enabled\n");
HTOnAssocRsp(ieee);
} else {
netdev_info(ieee->dev,
"Successfully associated, ht not enabled(%d, %d)\n",
- ieee->pHTInfo->bCurrentHTSupport,
- ieee->pHTInfo->bEnableHT);
- memset(ieee->dot11HTOperationalRateSet, 0, 16);
+ ieee->ht_info->bCurrentHTSupport,
+ ieee->ht_info->enable_ht);
+ memset(ieee->dot11ht_oper_rate_set, 0, 16);
}
- ieee->LinkDetectInfo.SlotNum = 2 * (1 +
+ ieee->link_detect_info.SlotNum = 2 * (1 +
ieee->current_network.beacon_interval /
500);
- if (ieee->LinkDetectInfo.NumRecvBcnInPeriod == 0 ||
- ieee->LinkDetectInfo.NumRecvDataInPeriod == 0) {
- ieee->LinkDetectInfo.NumRecvBcnInPeriod = 1;
- ieee->LinkDetectInfo.NumRecvDataInPeriod = 1;
+ if (ieee->link_detect_info.NumRecvBcnInPeriod == 0 ||
+ ieee->link_detect_info.NumRecvDataInPeriod == 0) {
+ ieee->link_detect_info.NumRecvBcnInPeriod = 1;
+ ieee->link_detect_info.NumRecvDataInPeriod = 1;
}
- pPSC->LpsIdleCount = 0;
+ psc->LpsIdleCount = 0;
ieee->link_change(ieee->dev);
if (ieee->is_silent_reset) {
ieee->current_network.ssid,
ieee->current_network.channel,
ieee->current_network.qos_data.supported,
- ieee->pHTInfo->bEnableHT,
+ ieee->ht_info->enable_ht,
ieee->current_network.bssht.bd_support_ht,
ieee->current_network.mode,
ieee->current_network.flags);
!(ieee->softmac_features & IEEE_SOFTMAC_SCAN))
rtllib_stop_scan_syncro(ieee);
- HTResetIOTSetting(ieee->pHTInfo);
+ HTResetIOTSetting(ieee->ht_info);
ieee->wmm_acm = 0;
if (ieee->iw_mode == IW_MODE_INFRA) {
/* Join the network for the first time */
HTResetSelfAndSavePeerSetting(ieee,
&(ieee->current_network));
else
- ieee->pHTInfo->bCurrentHTSupport =
+ ieee->ht_info->bCurrentHTSupport =
false;
ieee->state = RTLLIB_ASSOCIATING;
netdev_info(ieee->dev,
"Using B rates\n");
}
- memset(ieee->dot11HTOperationalRateSet, 0, 16);
+ memset(ieee->dot11ht_oper_rate_set, 0, 16);
ieee->state = RTLLIB_LINKED;
}
}
((ieee->mode == IEEE_G) &&
(ieee->current_network.mode == IEEE_N_24G) &&
(ieee->AsocRetryCount++ < (RT_ASOC_RETRY_LIMIT-1)))) {
- ieee->pHTInfo->iot_action |= HT_IOT_ACT_PURE_N_MODE;
+ ieee->ht_info->iot_action |= HT_IOT_ACT_PURE_N_MODE;
} else {
ieee->AsocRetryCount = 0;
}
{
int timeout;
u8 dtim;
- struct rt_pwr_save_ctrl *pPSC = &(ieee->PowerSaveControl);
+ struct rt_pwr_save_ctrl *psc = &ieee->pwr_save_ctrl;
if (ieee->LPSDelayCnt) {
ieee->LPSDelayCnt--;
if (time) {
if (ieee->bAwakePktSent) {
- pPSC->LPSAwakeIntvl = 1;
+ psc->LPSAwakeIntvl = 1;
} else {
u8 MaxPeriod = 1;
- if (pPSC->LPSAwakeIntvl == 0)
- pPSC->LPSAwakeIntvl = 1;
- if (pPSC->RegMaxLPSAwakeIntvl == 0)
+ if (psc->LPSAwakeIntvl == 0)
+ psc->LPSAwakeIntvl = 1;
+ if (psc->reg_max_lps_awake_intvl == 0)
MaxPeriod = 1;
- else if (pPSC->RegMaxLPSAwakeIntvl == 0xFF)
+ else if (psc->reg_max_lps_awake_intvl == 0xFF)
MaxPeriod = ieee->current_network.dtim_period;
else
- MaxPeriod = pPSC->RegMaxLPSAwakeIntvl;
- pPSC->LPSAwakeIntvl = (pPSC->LPSAwakeIntvl >=
+ MaxPeriod = psc->reg_max_lps_awake_intvl;
+ psc->LPSAwakeIntvl = (psc->LPSAwakeIntvl >=
MaxPeriod) ? MaxPeriod :
- (pPSC->LPSAwakeIntvl + 1);
+ (psc->LPSAwakeIntvl + 1);
}
{
u8 LPSAwakeIntvl_tmp = 0;
u8 count = ieee->current_network.tim.tim_count;
if (count == 0) {
- if (pPSC->LPSAwakeIntvl > period)
+ if (psc->LPSAwakeIntvl > period)
LPSAwakeIntvl_tmp = period +
- (pPSC->LPSAwakeIntvl -
+ (psc->LPSAwakeIntvl -
period) -
- ((pPSC->LPSAwakeIntvl-period) %
+ ((psc->LPSAwakeIntvl-period) %
period);
else
- LPSAwakeIntvl_tmp = pPSC->LPSAwakeIntvl;
+ LPSAwakeIntvl_tmp = psc->LPSAwakeIntvl;
} else {
- if (pPSC->LPSAwakeIntvl >
+ if (psc->LPSAwakeIntvl >
ieee->current_network.tim.tim_count)
LPSAwakeIntvl_tmp = count +
- (pPSC->LPSAwakeIntvl - count) -
- ((pPSC->LPSAwakeIntvl-count)%period);
+ (psc->LPSAwakeIntvl - count) -
+ ((psc->LPSAwakeIntvl-count)%period);
else
- LPSAwakeIntvl_tmp = pPSC->LPSAwakeIntvl;
+ LPSAwakeIntvl_tmp = psc->LPSAwakeIntvl;
}
*time = ieee->current_network.last_dtim_sta_time
{
if (ieee->sta_sleep == LPS_IS_WAKE) {
if (nl) {
- if (ieee->pHTInfo->iot_action &
+ if (ieee->ht_info->iot_action &
HT_IOT_ACT_NULL_DATA_POWER_SAVING) {
ieee->ack_tx_to_ieee = 1;
rtllib_sta_ps_send_null_frame(ieee, 0);
if (ieee->sta_sleep == LPS_IS_SLEEP)
ieee->sta_wake_up(ieee->dev);
if (nl) {
- if (ieee->pHTInfo->iot_action &
+ if (ieee->ht_info->iot_action &
HT_IOT_ACT_NULL_DATA_POWER_SAVING) {
ieee->ack_tx_to_ieee = 1;
rtllib_sta_ps_send_null_frame(ieee, 0);
if ((ieee->sta_sleep == LPS_IS_WAKE) && !success) {
spin_lock_irqsave(&ieee->mgmt_tx_lock, flags2);
- if (ieee->pHTInfo->iot_action &
+ if (ieee->ht_info->iot_action &
HT_IOT_ACT_NULL_DATA_POWER_SAVING)
rtllib_sta_ps_send_null_frame(ieee, 0);
else
kfree(network);
return 1;
}
- memcpy(ieee->pHTInfo->PeerHTCapBuf,
+ memcpy(ieee->ht_info->PeerHTCapBuf,
network->bssht.bd_ht_cap_buf,
network->bssht.bd_ht_cap_len);
- memcpy(ieee->pHTInfo->PeerHTInfoBuf,
+ memcpy(ieee->ht_info->PeerHTInfoBuf,
network->bssht.bd_ht_info_buf,
network->bssht.bd_ht_info_len);
if (ieee->handle_assoc_response != NULL)
if (ieee->open_wep || !challenge) {
ieee->state = RTLLIB_ASSOCIATING_AUTHENTICATED;
ieee->softmac_stats.rx_auth_rs_ok++;
- if (!(ieee->pHTInfo->iot_action & HT_IOT_ACT_PURE_N_MODE)) {
+ if (!(ieee->ht_info->iot_action & HT_IOT_ACT_PURE_N_MODE)) {
if (!ieee->GetNmodeSupportBySecCfg(ieee->dev)) {
if (IsHTHalfNmodeAPs(ieee)) {
bSupportNmode = true;
ieee->state = RTLLIB_ASSOCIATING;
ieee->softmac_stats.reassoc++;
ieee->is_roaming = true;
- ieee->LinkDetectInfo.bBusyTraffic = false;
+ ieee->link_detect_info.bBusyTraffic = false;
rtllib_disassociate(ieee);
RemovePeerTS(ieee, header->addr2);
if (ieee->LedControlHandler != NULL)
if ((ieee->mode == IEEE_N_24G) || (ieee->mode == IEEE_N_5G))
HTUseDefaultSetting(ieee);
else
- ieee->pHTInfo->bCurrentHTSupport = false;
+ ieee->ht_info->bCurrentHTSupport = false;
ieee->SetHwRegHandler(ieee->dev, HW_VAR_MEDIA_STATUS,
(u8 *)(&ieee->state));
if (!ieee->dot11d_info)
return -ENOMEM;
- ieee->LinkDetectInfo.SlotIndex = 0;
- ieee->LinkDetectInfo.SlotNum = 2;
- ieee->LinkDetectInfo.NumRecvBcnInPeriod = 0;
- ieee->LinkDetectInfo.NumRecvDataInPeriod = 0;
- ieee->LinkDetectInfo.NumTxOkInPeriod = 0;
- ieee->LinkDetectInfo.NumRxOkInPeriod = 0;
- ieee->LinkDetectInfo.NumRxUnicastOkInPeriod = 0;
+ ieee->link_detect_info.SlotIndex = 0;
+ ieee->link_detect_info.SlotNum = 2;
+ ieee->link_detect_info.NumRecvBcnInPeriod = 0;
+ ieee->link_detect_info.NumRecvDataInPeriod = 0;
+ ieee->link_detect_info.NumTxOkInPeriod = 0;
+ ieee->link_detect_info.NumRxOkInPeriod = 0;
+ ieee->link_detect_info.NumRxUnicastOkInPeriod = 0;
ieee->bIsAggregateFrame = false;
ieee->assoc_id = 0;
ieee->queue_stop = 0;
ieee->ps = RTLLIB_PS_DISABLED;
ieee->sta_sleep = LPS_IS_WAKE;
- ieee->Regdot11HTOperationalRateSet[0] = 0xff;
- ieee->Regdot11HTOperationalRateSet[1] = 0xff;
- ieee->Regdot11HTOperationalRateSet[4] = 0x01;
+ ieee->reg_dot11ht_oper_rate_set[0] = 0xff;
+ ieee->reg_dot11ht_oper_rate_set[1] = 0xff;
+ ieee->reg_dot11ht_oper_rate_set[4] = 0x01;
- ieee->Regdot11TxHTOperationalRateSet[0] = 0xff;
- ieee->Regdot11TxHTOperationalRateSet[1] = 0xff;
- ieee->Regdot11TxHTOperationalRateSet[4] = 0x01;
+ ieee->reg_dot11tx_ht_oper_rate_set[0] = 0xff;
+ ieee->reg_dot11tx_ht_oper_rate_set[1] = 0xff;
+ ieee->reg_dot11tx_ht_oper_rate_set[4] = 0x01;
ieee->FirstIe_InScan = false;
ieee->actscanning = false;
if (ieee->ScanOperationBackupHandler)
ieee->ScanOperationBackupHandler(ieee->dev, SCAN_OPT_BACKUP);
- if (ieee->pHTInfo->bCurrentHTSupport && ieee->pHTInfo->bEnableHT &&
- ieee->pHTInfo->bCurBW40MHz) {
+ if (ieee->ht_info->bCurrentHTSupport && ieee->ht_info->enable_ht &&
+ ieee->ht_info->bCurBW40MHz) {
b40M = 1;
- chan_offset = ieee->pHTInfo->CurSTAExtChnlOffset;
- bandwidth = (enum ht_channel_width)ieee->pHTInfo->bCurBW40MHz;
+ chan_offset = ieee->ht_info->CurSTAExtChnlOffset;
+ bandwidth = (enum ht_channel_width)ieee->ht_info->bCurBW40MHz;
ieee->SetBWModeHandler(ieee->dev, HT_CHANNEL_WIDTH_20,
HT_EXTCHNL_OFFSET_NO_EXT);
}
/* Notify AP that I wake up again */
rtllib_sta_ps_send_null_frame(ieee, 0);
- if (ieee->LinkDetectInfo.NumRecvBcnInPeriod == 0 ||
- ieee->LinkDetectInfo.NumRecvDataInPeriod == 0) {
- ieee->LinkDetectInfo.NumRecvBcnInPeriod = 1;
- ieee->LinkDetectInfo.NumRecvDataInPeriod = 1;
+ if (ieee->link_detect_info.NumRecvBcnInPeriod == 0 ||
+ ieee->link_detect_info.NumRecvDataInPeriod == 0) {
+ ieee->link_detect_info.NumRecvBcnInPeriod = 1;
+ ieee->link_detect_info.NumRecvDataInPeriod = 1;
}
if (ieee->data_hard_resume)
ieee->ps = RTLLIB_PS_DISABLED;
goto exit;
}
- if (wrqu->power.flags & IW_POWER_TIMEOUT) {
+ if (wrqu->power.flags & IW_POWER_TIMEOUT)
ieee->ps_timeout = wrqu->power.value / 1000;
- }
if (wrqu->power.flags & IW_POWER_PERIOD)
ieee->ps_period = wrqu->power.value / 1000;
return 0;
}
-
void rtllib_txb_free(struct rtllib_txb *txb)
{
if (unlikely(!txb))
struct sk_buff *skb,
struct cb_desc *tcb_desc)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
struct tx_ts_record *pTxTs = NULL;
struct rtllib_hdr_1addr *hdr = (struct rtllib_hdr_1addr *)skb->data;
if (rtllib_act_scanning(ieee, false))
return;
- if (!pHTInfo->bCurrentHTSupport || !pHTInfo->bEnableHT)
+ if (!ht_info->bCurrentHTSupport || !ht_info->enable_ht)
return;
if (!IsQoSDataFrame(skb->data))
return;
if (tcb_desc->bdhcp || ieee->CntAfterLink < 2)
return;
- if (pHTInfo->iot_action & HT_IOT_ACT_TX_NO_AGGREGATION)
+ if (ht_info->iot_action & HT_IOT_ACT_TX_NO_AGGREGATION)
return;
if (!ieee->GetNmodeSupportBySecCfg(ieee->dev))
return;
- if (pHTInfo->bCurrentAMPDUEnable) {
+ if (ht_info->bCurrentAMPDUEnable) {
if (!GetTs(ieee, (struct ts_common_info **)(&pTxTs), hdr->addr1,
- skb->priority, TX_DIR, true)) {
+ skb->priority, TX_DIR, true)) {
netdev_info(ieee->dev, "%s: can't get TS\n", __func__);
return;
}
goto FORCED_AGG_SETTING;
} else if (!pTxTs->bUsingBa) {
if (SN_LESS(pTxTs->TxAdmittedBARecord.ba_start_seq_ctrl.field.seq_num,
- (pTxTs->TxCurSeq+1)%4096))
+ (pTxTs->TxCurSeq + 1) % 4096))
pTxTs->bUsingBa = true;
else
goto FORCED_AGG_SETTING;
}
if (ieee->iw_mode == IW_MODE_INFRA) {
tcb_desc->bAMPDUEnable = true;
- tcb_desc->ampdu_factor = pHTInfo->CurrentAMPDUFactor;
- tcb_desc->ampdu_density = pHTInfo->current_mpdu_density;
+ tcb_desc->ampdu_factor = ht_info->CurrentAMPDUFactor;
+ tcb_desc->ampdu_density = ht_info->current_mpdu_density;
}
}
FORCED_AGG_SETTING:
- switch (pHTInfo->ForcedAMPDUMode) {
+ switch (ht_info->ForcedAMPDUMode) {
case HT_AGG_AUTO:
break;
case HT_AGG_FORCE_ENABLE:
tcb_desc->bAMPDUEnable = true;
- tcb_desc->ampdu_density = pHTInfo->forced_mpdu_density;
- tcb_desc->ampdu_factor = pHTInfo->forced_ampdu_factor;
+ tcb_desc->ampdu_density = ht_info->forced_mpdu_density;
+ tcb_desc->ampdu_factor = ht_info->forced_ampdu_factor;
break;
case HT_AGG_FORCE_DISABLE:
static void rtllib_query_HTCapShortGI(struct rtllib_device *ieee,
struct cb_desc *tcb_desc)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
tcb_desc->bUseShortGI = false;
- if (!pHTInfo->bCurrentHTSupport || !pHTInfo->bEnableHT)
+ if (!ht_info->bCurrentHTSupport || !ht_info->enable_ht)
return;
- if (pHTInfo->forced_short_gi) {
+ if (ht_info->forced_short_gi) {
tcb_desc->bUseShortGI = true;
return;
}
- if (pHTInfo->bCurBW40MHz && pHTInfo->bCurShortGI40MHz)
+ if (ht_info->bCurBW40MHz && ht_info->bCurShortGI40MHz)
tcb_desc->bUseShortGI = true;
- else if (!pHTInfo->bCurBW40MHz && pHTInfo->bCurShortGI20MHz)
+ else if (!ht_info->bCurBW40MHz && ht_info->bCurShortGI20MHz)
tcb_desc->bUseShortGI = true;
}
static void rtllib_query_BandwidthMode(struct rtllib_device *ieee,
struct cb_desc *tcb_desc)
{
- struct rt_hi_throughput *pHTInfo = ieee->pHTInfo;
+ struct rt_hi_throughput *ht_info = ieee->ht_info;
tcb_desc->bPacketBW = false;
- if (!pHTInfo->bCurrentHTSupport || !pHTInfo->bEnableHT)
+ if (!ht_info->bCurrentHTSupport || !ht_info->enable_ht)
return;
if (tcb_desc->bMulticast || tcb_desc->bBroadcast)
if ((tcb_desc->data_rate & 0x80) == 0)
return;
- if (pHTInfo->bCurBW40MHz && pHTInfo->cur_tx_bw40mhz &&
+ if (ht_info->bCurBW40MHz && ht_info->cur_tx_bw40mhz &&
!ieee->bandwidth_auto_switch.bforced_tx20Mhz)
tcb_desc->bPacketBW = true;
}
struct cb_desc *tcb_desc,
struct sk_buff *skb)
{
- struct rt_hi_throughput *pHTInfo;
+ struct rt_hi_throughput *ht_info;
tcb_desc->bRTSSTBC = false;
tcb_desc->bRTSUseShortGI = false;
if (tcb_desc->bBroadcast || tcb_desc->bMulticast)
return;
- if (is_broadcast_ether_addr(skb->data+16))
+ if (is_broadcast_ether_addr(skb->data + 16))
return;
if (ieee->mode < IEEE_N_24G) {
return;
}
- pHTInfo = ieee->pHTInfo;
+ ht_info = ieee->ht_info;
while (true) {
- if (pHTInfo->iot_action & HT_IOT_ACT_FORCED_CTS2SELF) {
+ if (ht_info->iot_action & HT_IOT_ACT_FORCED_CTS2SELF) {
tcb_desc->bCTSEnable = true;
tcb_desc->rts_rate = MGN_24M;
tcb_desc->bRTSEnable = true;
break;
- } else if (pHTInfo->iot_action & (HT_IOT_ACT_FORCED_RTS |
+ } else if (ht_info->iot_action & (HT_IOT_ACT_FORCED_RTS |
HT_IOT_ACT_PURE_N_MODE)) {
tcb_desc->bRTSEnable = true;
tcb_desc->rts_rate = MGN_24M;
tcb_desc->rts_rate = MGN_24M;
break;
}
- if (pHTInfo->bCurrentHTSupport && pHTInfo->bEnableHT) {
- u8 HTOpMode = pHTInfo->current_op_mode;
+ if (ht_info->bCurrentHTSupport && ht_info->enable_ht) {
+ u8 HTOpMode = ht_info->current_op_mode;
- if ((pHTInfo->bCurBW40MHz && (HTOpMode == 2 ||
- HTOpMode == 3)) ||
- (!pHTInfo->bCurBW40MHz && HTOpMode == 3)) {
+ if ((ht_info->bCurBW40MHz && (HTOpMode == 2 ||
+ HTOpMode == 3)) ||
+ (!ht_info->bCurBW40MHz && HTOpMode == 3)) {
tcb_desc->rts_rate = MGN_24M;
tcb_desc->bRTSEnable = true;
break;
tcb_desc->bRTSBW = false;
}
-
static void rtllib_txrate_selectmode(struct rtllib_device *ieee,
struct cb_desc *tcb_desc)
{
- if (ieee->bTxDisableRateFallBack)
- tcb_desc->bTxDisableRateFallBack = true;
+ if (ieee->tx_dis_rate_fallback)
+ tcb_desc->tx_dis_rate_fallback = true;
- if (ieee->bTxUseDriverAssingedRate)
- tcb_desc->bTxUseDriverAssingedRate = true;
- if (!tcb_desc->bTxDisableRateFallBack ||
- !tcb_desc->bTxUseDriverAssingedRate) {
+ if (ieee->tx_use_drv_assinged_rate)
+ tcb_desc->tx_use_drv_assinged_rate = true;
+ if (!tcb_desc->tx_dis_rate_fallback ||
+ !tcb_desc->tx_use_drv_assinged_rate) {
if (ieee->iw_mode == IW_MODE_INFRA ||
ieee->iw_mode == IW_MODE_ADHOC)
tcb_desc->RATRIndex = 0;
struct tx_ts_record *pTS = NULL;
if (!GetTs(ieee, (struct ts_common_info **)(&pTS), dst,
- skb->priority, TX_DIR, true))
+ skb->priority, TX_DIR, true))
return 0;
seqnum = pTS->TxCurSeq;
- pTS->TxCurSeq = (pTS->TxCurSeq+1)%4096;
+ pTS->TxCurSeq = (pTS->TxCurSeq + 1) % 4096;
return seqnum;
}
return 0;
goto success;
}
-
if (likely(ieee->raw_tx == 0)) {
if (unlikely(skb->len < SNAP_SIZE + sizeof(u16))) {
netdev_warn(ieee->dev, "skb too small (%d).\n",
if (skb->len > 282) {
if (ether_type == ETH_P_IP) {
const struct iphdr *ip = (struct iphdr *)
- ((u8 *)skb->data+14);
+ ((u8 *)skb->data + 14);
if (ip->protocol == IPPROTO_UDP) {
struct udphdr *udp;
udp = (struct udphdr *)((u8 *)ip +
(ip->ihl << 2));
if (((((u8 *)udp)[1] == 68) &&
- (((u8 *)udp)[3] == 67)) ||
+ (((u8 *)udp)[3] == 67)) ||
((((u8 *)udp)[1] == 67) &&
(((u8 *)udp)[3] == 68))) {
bdhcp = true;
/* in case we are a client verify acm is not set for this ac */
while (unlikely(ieee->wmm_acm & (0x01 << skb->priority))) {
netdev_info(ieee->dev, "skb->priority = %x\n",
- skb->priority);
+ skb->priority);
if (wme_downgrade_ac(skb))
break;
netdev_info(ieee->dev, "converted skb->priority = %x\n",
- skb->priority);
+ skb->priority);
}
qos_ctl |= skb->priority;
* MOREFRAGS bit to the frame control
*/
if (i != nr_frags - 1) {
- frag_hdr->frame_ctl = cpu_to_le16(
- fc | RTLLIB_FCTL_MOREFRAGS);
+ frag_hdr->frame_ctl = cpu_to_le16(fc |
+ RTLLIB_FCTL_MOREFRAGS);
bytes = bytes_per_frag;
} else {
if ((qos_activated) && (!bIsMulticast)) {
frag_hdr->seq_ctl =
cpu_to_le16(rtllib_query_seqnum(ieee, skb_frag,
- header.addr1));
+ header.addr1));
frag_hdr->seq_ctl =
- cpu_to_le16(le16_to_cpu(frag_hdr->seq_ctl)<<4 | i);
+ cpu_to_le16(le16_to_cpu(frag_hdr->seq_ctl) << 4 | i);
} else {
frag_hdr->seq_ctl =
- cpu_to_le16(ieee->seq_ctrl[0]<<4 | i);
+ cpu_to_le16(ieee->seq_ctrl[0] << 4 | i);
}
/* Put a SNAP header on the first fragment */
if (i == 0) {
- rtllib_put_snap(
- skb_put(skb_frag, SNAP_SIZE +
- sizeof(u16)), ether_type);
+ rtllib_put_snap(skb_put(skb_frag,
+ SNAP_SIZE +
+ sizeof(u16)), ether_type);
bytes -= SNAP_SIZE + sizeof(u16);
}
tcb_desc->priority = skb->priority;
if (ether_type == ETH_P_PAE) {
- if (ieee->pHTInfo->iot_action &
+ if (ieee->ht_info->iot_action &
HT_IOT_ACT_WA_IOT_Broadcom) {
tcb_desc->data_rate =
MgntQuery_TxRateExcludeCCKRates(ieee);
- tcb_desc->bTxDisableRateFallBack = false;
+ tcb_desc->tx_dis_rate_fallback = false;
} else {
tcb_desc->data_rate = ieee->basic_rate;
- tcb_desc->bTxDisableRateFallBack = 1;
+ tcb_desc->tx_dis_rate_fallback = 1;
}
-
tcb_desc->RATRIndex = 7;
- tcb_desc->bTxUseDriverAssingedRate = 1;
+ tcb_desc->tx_use_drv_assinged_rate = 1;
} else {
if (is_multicast_ether_addr(header.addr1))
tcb_desc->bMulticast = 1;
tcb_desc->data_rate = rtllib_current_rate(ieee);
if (bdhcp) {
- if (ieee->pHTInfo->iot_action &
+ if (ieee->ht_info->iot_action &
HT_IOT_ACT_WA_IOT_Broadcom) {
tcb_desc->data_rate =
MgntQuery_TxRateExcludeCCKRates(ieee);
- tcb_desc->bTxDisableRateFallBack = false;
+ tcb_desc->tx_dis_rate_fallback = false;
} else {
tcb_desc->data_rate = MGN_1M;
- tcb_desc->bTxDisableRateFallBack = 1;
+ tcb_desc->tx_dis_rate_fallback = 1;
}
-
tcb_desc->RATRIndex = 7;
- tcb_desc->bTxUseDriverAssingedRate = 1;
+ tcb_desc->tx_use_drv_assinged_rate = 1;
tcb_desc->bdhcp = 1;
}
netif_stop_queue(dev);
stats->tx_errors++;
return 1;
-
}
netdev_tx_t rtllib_xmit(struct sk_buff *skb, struct net_device *dev)
p = custom;
p += scnprintf(p, MAX_CUSTOM_LEN - (p - custom),
" Last beacon: %lums ago",
- (jiffies - network->last_scanned) / (HZ / 100));
+ (100 * (jiffies - network->last_scanned)) / HZ);
iwe.u.data.length = p - custom;
if (iwe.u.data.length)
start = iwe_stream_add_point_rsl(info, start, stop,
escape_essid(network->ssid,
network->ssid_len),
network->bssid,
- (jiffies - network->last_scanned) /
- (HZ / 100));
+ (100 * (jiffies - network->last_scanned)) /
+ HZ);
}
spin_unlock_irqrestore(&ieee->lock, flags);
--- /dev/null
+To-do list:
+
+* Correct the coding style according to Linux guidelines; please read the document
+ at https://www.kernel.org/doc/html/latest/process/coding-style.html.
+* Remove unnecessary debugging/printing macros; for those that are still needed
+ use the proper kernel API (pr_debug(), dev_dbg(), netdev_dbg()).
+* Remove dead code such as unusued functions, variables, fields, etc..
+* Use in-kernel API and remove unnecessary wrappers where possible.
+* Fix bugs due to code that sleeps in atomic context.
+* Remove the HAL layer and migrate its functionality into the relevant parts of
+ the driver.
+* Switch to use LIB80211.
+* Switch to use MAC80211.
+* Switch to use CFG80211.
+* Improve the error handling of various functions, particularly those that use
+ existing kernel APIs.
#define MAX_IE_LEN 0xff
// added for kernel conflict
-#define ieee80211_crypt_deinit_entries ieee80211_crypt_deinit_entries_rsl
-#define ieee80211_crypt_deinit_handler ieee80211_crypt_deinit_handler_rsl
-#define ieee80211_crypt_delayed_deinit ieee80211_crypt_delayed_deinit_rsl
-#define ieee80211_register_crypto_ops ieee80211_register_crypto_ops_rsl
-#define ieee80211_unregister_crypto_ops ieee80211_unregister_crypto_ops_rsl
-#define ieee80211_get_crypto_ops ieee80211_get_crypto_ops_rsl
-
-#define ieee80211_ccmp_null ieee80211_ccmp_null_rsl
-
-#define ieee80211_tkip_null ieee80211_tkip_null_rsl
-
-#define free_ieee80211 free_ieee80211_rsl
-#define alloc_ieee80211 alloc_ieee80211_rsl
-
-#define ieee80211_rx ieee80211_rx_rsl
-#define ieee80211_rx_mgt ieee80211_rx_mgt_rsl
-
-#define ieee80211_get_beacon ieee80211_get_beacon_rsl
#define ieee80211_wake_queue ieee80211_wake_queue_rsl
#define ieee80211_stop_queue ieee80211_stop_queue_rsl
-#define ieee80211_reset_queue ieee80211_reset_queue_rsl
-#define ieee80211_softmac_stop_protocol ieee80211_softmac_stop_protocol_rsl
-#define ieee80211_softmac_start_protocol ieee80211_softmac_start_protocol_rsl
-#define ieee80211_is_shortslot ieee80211_is_shortslot_rsl
-#define ieee80211_is_54g ieee80211_is_54g_rsl
-#define ieee80211_wpa_supplicant_ioctl ieee80211_wpa_supplicant_ioctl_rsl
-#define ieee80211_ps_tx_ack ieee80211_ps_tx_ack_rsl
-#define ieee80211_softmac_xmit ieee80211_softmac_xmit_rsl
-#define ieee80211_stop_send_beacons ieee80211_stop_send_beacons_rsl
#define notify_wx_assoc_event notify_wx_assoc_event_rsl
#define SendDisassociation SendDisassociation_rsl
-#define ieee80211_disassociate ieee80211_disassociate_rsl
-#define ieee80211_start_send_beacons ieee80211_start_send_beacons_rsl
-#define ieee80211_stop_scan ieee80211_stop_scan_rsl
-#define ieee80211_send_probe_requests ieee80211_send_probe_requests_rsl
-#define ieee80211_softmac_scan_syncro ieee80211_softmac_scan_syncro_rsl
-#define ieee80211_start_scan_syncro ieee80211_start_scan_syncro_rsl
-
-#define ieee80211_wx_get_essid ieee80211_wx_get_essid_rsl
-#define ieee80211_wx_set_essid ieee80211_wx_set_essid_rsl
-#define ieee80211_wx_set_rate ieee80211_wx_set_rate_rsl
-#define ieee80211_wx_get_rate ieee80211_wx_get_rate_rsl
-#define ieee80211_wx_set_wap ieee80211_wx_set_wap_rsl
-#define ieee80211_wx_get_wap ieee80211_wx_get_wap_rsl
-#define ieee80211_wx_set_mode ieee80211_wx_set_mode_rsl
-#define ieee80211_wx_get_mode ieee80211_wx_get_mode_rsl
-#define ieee80211_wx_set_scan ieee80211_wx_set_scan_rsl
-#define ieee80211_wx_get_freq ieee80211_wx_get_freq_rsl
-#define ieee80211_wx_set_freq ieee80211_wx_set_freq_rsl
-#define ieee80211_wx_set_rawtx ieee80211_wx_set_rawtx_rsl
-#define ieee80211_wx_get_name ieee80211_wx_get_name_rsl
-#define ieee80211_wx_set_power ieee80211_wx_set_power_rsl
-#define ieee80211_wx_get_power ieee80211_wx_get_power_rsl
-#define ieee80211_wlan_frequencies ieee80211_wlan_frequencies_rsl
-#define ieee80211_wx_set_rts ieee80211_wx_set_rts_rsl
-#define ieee80211_wx_get_rts ieee80211_wx_get_rts_rsl
-
-#define ieee80211_txb_free ieee80211_txb_free_rsl
-
-#define ieee80211_wx_set_gen_ie ieee80211_wx_set_gen_ie_rsl
-#define ieee80211_wx_get_scan ieee80211_wx_get_scan_rsl
-#define ieee80211_wx_set_encode ieee80211_wx_set_encode_rsl
-#define ieee80211_wx_get_encode ieee80211_wx_get_encode_rsl
-#define ieee80211_wx_set_mlme ieee80211_wx_set_mlme_rsl
-#define ieee80211_wx_set_auth ieee80211_wx_set_auth_rsl
-#define ieee80211_wx_set_encode_ext ieee80211_wx_set_encode_ext_rsl
-#define ieee80211_wx_get_encode_ext ieee80211_wx_get_encode_ext_rsl
struct ieee_param {
void softmac_mgmt_xmit(struct sk_buff *skb, struct ieee80211_device *ieee);
/* ieee80211_crypt_ccmp&tkip&wep.c */
-void ieee80211_tkip_null(void);
int ieee80211_crypto_init(void);
void ieee80211_crypto_deinit(void);
{
ieee80211_unregister_crypto_ops(&ieee80211_crypt_tkip);
}
-
-void ieee80211_tkip_null(void)
-{
-// printk("============>%s()\n", __func__);
- return;
-}
ieee->last_packet_time[i] = 0;
}
-/* These function were added to load crypte module autoly */
- ieee80211_tkip_null();
-
return dev;
failed:
#endif
if (ieee->iw_mode == IW_MODE_MONITOR) {
+ unsigned int len = skb->len;
+
ieee80211_monitor_rx(ieee, skb, rx_stats);
stats->rx_packets++;
- stats->rx_bytes += skb->len;
+ stats->rx_bytes += len;
return 1;
}
info_element->data[0] == 0x00 &&
info_element->data[1] == 0x13 &&
info_element->data[2] == 0x74)) {
- netdev_dbg(ieee->dev, "========> athros AP is exist\n");
+ netdev_dbg(ieee->dev, "========> Atheros AP exists\n");
network->atheros_cap_exist = true;
} else
network->atheros_cap_exist = false;
padapter->cmdpriv.padapter = padapter;
ret = r8712_init_evt_priv(&padapter->evtpriv);
if (ret)
- return ret;
+ goto free_cmd;
ret = r8712_init_mlme_priv(padapter);
if (ret)
- return ret;
- _r8712_init_xmit_priv(&padapter->xmitpriv, padapter);
- _r8712_init_recv_priv(&padapter->recvpriv, padapter);
+ goto free_evt;
+ ret = _r8712_init_xmit_priv(&padapter->xmitpriv, padapter);
+ if (ret)
+ goto free_mlme;
+ ret = _r8712_init_recv_priv(&padapter->recvpriv, padapter);
+ if (ret)
+ goto free_xmit;
memset((unsigned char *)&padapter->securitypriv, 0,
sizeof(struct security_priv));
timer_setup(&padapter->securitypriv.tkip_timer,
r8712_use_tkipkey_handler, 0);
ret = _r8712_init_sta_priv(&padapter->stapriv);
if (ret)
- return ret;
+ goto free_recv;
padapter->stapriv.padapter = padapter;
r8712_init_bcmc_stainfo(padapter);
r8712_init_pwrctrl_priv(padapter);
mp871xinit(padapter);
init_default_value(padapter);
r8712_InitSwLeds(padapter);
+
+ return 0;
+
+free_recv:
+ _r8712_free_recv_priv(&padapter->recvpriv);
+free_xmit:
+ _free_xmit_priv(&padapter->xmitpriv);
+free_mlme:
+ r8712_free_mlme_priv(&padapter->mlmepriv);
+free_evt:
+ r8712_free_evt_priv(&padapter->evtpriv);
+free_cmd:
+ r8712_free_cmd_priv(&padapter->cmdpriv);
return ret;
}
#include "drv_types.h"
#include <linux/skbuff.h>
-void _r8712_init_recv_priv(struct recv_priv *precvpriv,
- struct _adapter *padapter);
+int _r8712_init_recv_priv(struct recv_priv *precvpriv,
+ struct _adapter *padapter);
void _r8712_free_recv_priv(struct recv_priv *precvpriv);
void r8712_recv_entry(union recv_frame *precv_frame);
void r8712_recv_indicatepkt(struct _adapter *adapter,
union recv_frame *precv_frame);
void r8712_handle_tkip_mic_err(struct _adapter *padapter, u8 bgroup);
-void r8712_init_recv_priv(struct recv_priv *precvpriv,
- struct _adapter *padapter);
+int r8712_init_recv_priv(struct recv_priv *precvpriv,
+ struct _adapter *padapter);
void r8712_free_recv_priv(struct recv_priv *precvpriv);
void r8712_os_recv_resource_alloc(struct _adapter *padapter,
union recv_frame *precvframe);
static void recv_tasklet(struct tasklet_struct *t);
-void r8712_init_recv_priv(struct recv_priv *precvpriv,
- struct _adapter *padapter)
+int r8712_init_recv_priv(struct recv_priv *precvpriv,
+ struct _adapter *padapter)
{
int i;
struct recv_buf *precvbuf;
precvpriv->pallocated_recv_buf =
kzalloc(NR_RECVBUFF * sizeof(struct recv_buf) + 4, GFP_ATOMIC);
if (!precvpriv->pallocated_recv_buf)
- return;
+ return -ENOMEM;
precvpriv->precv_buf = precvpriv->pallocated_recv_buf + 4 -
((addr_t)(precvpriv->pallocated_recv_buf) & 3);
precvbuf = (struct recv_buf *)precvpriv->precv_buf;
}
pskb = NULL;
}
+ return 0;
}
void r8712_free_recv_priv(struct recv_priv *precvpriv)
#ifdef CONFIG_R8712_TX_AGGR
struct xmit_frame *p2ndxmitframe = NULL;
#else
- int res = _SUCCESS, xcnt = 0;
+ int res = _SUCCESS;
#endif
phwxmits = pxmitpriv->hwxmits;
dump_xframe(padapter, pxmitframe);
else
r8712_free_xmitframe_ex(pxmitpriv, pxmitframe);
- xcnt++;
#endif
} else { /* pxmitframe == NULL && p2ndxmitframe == NULL */
#define _RTL871X_RECV_C_
#include <linux/ip.h>
-#include <linux/slab.h>
#include <linux/if_ether.h>
-#include <linux/kmemleak.h>
#include <linux/etherdevice.h>
#include <linux/ieee80211.h>
#include <net/cfg80211.h>
_init_queue(&psta_recvpriv->defrag_q);
}
-void _r8712_init_recv_priv(struct recv_priv *precvpriv,
- struct _adapter *padapter)
+int _r8712_init_recv_priv(struct recv_priv *precvpriv,
+ struct _adapter *padapter)
{
+ int ret;
sint i;
union recv_frame *precvframe;
sizeof(union recv_frame) + RXFRAME_ALIGN_SZ,
GFP_ATOMIC);
if (!precvpriv->pallocated_frame_buf)
- return;
- kmemleak_not_leak(precvpriv->pallocated_frame_buf);
+ return -ENOMEM;
precvpriv->precv_frame_buf = precvpriv->pallocated_frame_buf +
RXFRAME_ALIGN_SZ -
((addr_t)(precvpriv->pallocated_frame_buf) &
precvframe++;
}
precvpriv->rx_pending_cnt = 1;
- r8712_init_recv_priv(precvpriv, padapter);
+ ret = r8712_init_recv_priv(precvpriv, padapter);
+ if (ret)
+ kfree(precvpriv->pallocated_frame_buf);
+
+ return ret;
}
void _r8712_free_recv_priv(struct recv_priv *precvpriv)
{
u8 tmpidx = 0;
u8 bResult = false;
- u32 efuseValue = 0;
+ u32 efuseValue;
if (bPseudoTest)
return Efuse_Write1ByteToFakeContent(addr, data);
/* parsing HT_CAP_IE */
p = rtw_get_ie(pnetwork->network.ies + _FIXED_IE_LENGTH_, WLAN_EID_HT_CAPABILITY, &len, pnetwork->network.ie_length - _FIXED_IE_LENGTH_);
if (p && len > 0) {
- pht_cap = (struct ieee80211_ht_cap *)(p + 2);
- pnetwork->bcn_info.ht_cap_info = le16_to_cpu(pht_cap->cap_info);
+ pht_cap = (struct ieee80211_ht_cap *)(p + 2);
+ pnetwork->bcn_info.ht_cap_info = le16_to_cpu(pht_cap->cap_info);
} else {
- pnetwork->bcn_info.ht_cap_info = 0;
+ pnetwork->bcn_info.ht_cap_info = 0;
}
/* parsing HT_INFO_IE */
p = rtw_get_ie(pnetwork->network.ies + _FIXED_IE_LENGTH_, WLAN_EID_HT_OPERATION, &len, pnetwork->network.ie_length - _FIXED_IE_LENGTH_);
if (p && len > 0) {
- pht_info = (struct HT_info_element *)(p + 2);
- pnetwork->bcn_info.ht_info_infos_0 = pht_info->infos[0];
+ pht_info = (struct HT_info_element *)(p + 2);
+ pnetwork->bcn_info.ht_info_infos_0 = pht_info->infos[0];
} else {
- pnetwork->bcn_info.ht_info_infos_0 = 0;
+ pnetwork->bcn_info.ht_info_infos_0 = 0;
}
}
goto exit;
} else {
int select_ret;
+
spin_unlock_bh(&(pmlmepriv->scanned_queue.lock));
select_ret = rtw_select_and_join_from_scanned_queue(pmlmepriv);
if (select_ret == _SUCCESS) {
if (check_fwstate(pmlmepriv, _FW_LINKED|WIFI_ADHOC_MASTER_STATE) == true) {
if ((pmlmepriv->assoc_ssid.ssid_length == ssid->ssid_length) &&
(!memcmp(&pmlmepriv->assoc_ssid.ssid, ssid->ssid, ssid->ssid_length))) {
- if ((check_fwstate(pmlmepriv, WIFI_STATION_STATE) == false)) {
+ if (check_fwstate(pmlmepriv, WIFI_STATION_STATE) == false) {
if (rtw_is_same_ibss(padapter, pnetwork) == false) {
/* if in WIFI_ADHOC_MASTER_STATE | WIFI_ADHOC_STATE, create bss or rejoin again */
rtw_disassoc_cmd(padapter, 0, true);
if ((*pold_state == Ndis802_11Infrastructure) || (*pold_state == Ndis802_11IBSS)) {
if (check_fwstate(pmlmepriv, _FW_LINKED) == true)
rtw_indicate_disconnect(padapter); /* will clr Linked_state; before this function, we must have checked whether issue dis-assoc_cmd or not */
- }
+ }
*pold_state = networktype;
u8 rtw_set_802_11_bssid_list_scan(struct adapter *padapter, struct ndis_802_11_ssid *pssid, int ssid_max_num)
{
- struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
+ struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
u8 res = true;
if (!padapter) {
}
/*
-* rtw_get_cur_max_rate -
-* @adapter: pointer to struct adapter structure
-*
-* Return 0 or 100Kbps
-*/
+ * rtw_get_cur_max_rate -
+ * @adapter: pointer to struct adapter structure
+ *
+ * Return 0 or 100Kbps
+ */
u16 rtw_get_cur_max_rate(struct adapter *adapter)
{
int i = 0;
__le16 tmps, tmpd;
if (rtw_bug_check(dst, src, &s_cap, &d_cap) == false)
- return false;
+ return false;
memcpy((u8 *)&tmps, rtw_get_capability_from_ie(src->ies), 2);
memcpy((u8 *)&tmpd, rtw_get_capability_from_ie(dst->ies), 2);
uint ie_len = 0;
if ((desired_encmode == Ndis802_11EncryptionDisabled) && (privacy != 0))
- bselected = false;
+ bselected = false;
if (psecuritypriv->ndisauthtype == Ndis802_11AuthModeWPA2PSK) {
p = rtw_get_ie(pnetwork->network.ies + _BEACON_IE_OFFSET_, WLAN_EID_RSN, &ie_len, (pnetwork->network.ie_length - _BEACON_IE_OFFSET_));
pmlmepriv->to_join = false;
s_ret = rtw_select_and_join_from_scanned_queue(pmlmepriv);
if (s_ret == _SUCCESS) {
- _set_timer(&pmlmepriv->assoc_timer, MAX_JOIN_TIMEOUT);
+ _set_timer(&pmlmepriv->assoc_timer, MAX_JOIN_TIMEOUT);
} else if (s_ret == 2) {/* there is no need to wait for join */
_clr_fwstate_(pmlmepriv, _FW_UNDER_LINKING);
rtw_indicate_connect(adapter);
if (in_ie[i] == 0xDD && in_ie[i+2] == 0x00 && in_ie[i+3] == 0x50 && in_ie[i+4] == 0xF2 && in_ie[i+5] == 0x02 && i+5 < in_len) { /* WMM element ID and OUI */
for (j = i; j < i + 9; j++) {
- out_ie[ielength] = in_ie[j];
- ielength++;
+ out_ie[ielength] = in_ie[j];
+ ielength++;
}
out_ie[initial_out_len + 1] = 0x07;
out_ie[initial_out_len + 6] = 0x00;
if (ie[13] <= 20) {
/* The RSN IE didn't include the PMK ID, append the PMK information */
- ie[ie_len] = 1;
- ie_len++;
- ie[ie_len] = 0; /* PMKID count = 0x0100 */
- ie_len++;
- memcpy(&ie[ie_len], &psecuritypriv->PMKIDList[iEntry].PMKID, 16);
-
- ie_len += 16;
- ie[13] += 18;/* PMKID length = 2+16 */
-
+ ie[ie_len] = 1;
+ ie_len++;
+ ie[ie_len] = 0; /* PMKID count = 0x0100 */
+ ie_len++;
+ memcpy(&ie[ie_len], &psecuritypriv->PMKIDList[iEntry].PMKID, 16);
+ ie_len += 16;
+ ie[13] += 18;/* PMKID length = 2+16 */
}
return ie_len;
}
memcpy(out_ie, in_ie, 12);
ielength = 12;
if ((ndisauthmode == Ndis802_11AuthModeWPA) || (ndisauthmode == Ndis802_11AuthModeWPAPSK))
- authmode = WLAN_EID_VENDOR_SPECIFIC;
+ authmode = WLAN_EID_VENDOR_SPECIFIC;
if ((ndisauthmode == Ndis802_11AuthModeWPA2) || (ndisauthmode == Ndis802_11AuthModeWPA2PSK))
- authmode = WLAN_EID_RSN;
+ authmode = WLAN_EID_RSN;
if (check_fwstate(pmlmepriv, WIFI_UNDER_WPS)) {
memcpy(out_ie+ielength, psecuritypriv->wps_ie, psecuritypriv->wps_ie_len);
u8 val8;
if (is_client_associated_to_ap(padapter))
- issue_deauth_ex(padapter, pnetwork->mac_address, WLAN_REASON_DEAUTH_LEAVING, param->deauth_timeout_ms/100, 100);
+ issue_deauth_ex(padapter, pnetwork->mac_address, WLAN_REASON_DEAUTH_LEAVING, param->deauth_timeout_ms/100, 100);
if (((pmlmeinfo->state&0x03) == WIFI_FW_ADHOC_STATE) || ((pmlmeinfo->state&0x03) == WIFI_FW_AP_STATE)) {
/* Stop BCN */
if (padapter) {
if (pfree_recv_queue == &precvpriv->free_recv_queue)
- precvpriv->free_recvframe_cnt++;
+ precvpriv->free_recvframe_cnt++;
}
spin_unlock_bh(&pfree_recv_queue->lock);
return _SUCCESS;
if (bmcast) {
/* For AP mode, if DA == MCAST, then BSSID should be also MCAST */
if (!IS_MCAST(pattrib->bssid)) {
- ret = _FAIL;
- goto exit;
+ ret = _FAIL;
+ goto exit;
}
} else { /* not mc-frame */
/* For AP mode, if DA is non-MCAST, then it must be BSSID, and bssid == BSSID */
else if (pacl_list->mode == 2)/* deny unless in accept list */
res = match;
else
- res = true;
+ res = true;
return res;
}
pmlmeinfo->HT_caps.u.HT_cap[i] &= (pIE->data[i]);
} else {
/* modify from fw by Thomas 2010/11/17 */
- if ((pmlmeinfo->HT_caps.u.HT_cap_element.AMPDU_para & 0x3) > (pIE->data[i] & 0x3))
- max_AMPDU_len = (pIE->data[i] & 0x3);
- else
- max_AMPDU_len = (pmlmeinfo->HT_caps.u.HT_cap_element.AMPDU_para & 0x3);
+ max_AMPDU_len = min(pmlmeinfo->HT_caps.u.HT_cap_element.AMPDU_para & 0x3,
+ pIE->data[i] & 0x3);
- if ((pmlmeinfo->HT_caps.u.HT_cap_element.AMPDU_para & 0x1c) > (pIE->data[i] & 0x1c))
- min_MPDU_spacing = (pmlmeinfo->HT_caps.u.HT_cap_element.AMPDU_para & 0x1c);
- else
- min_MPDU_spacing = (pIE->data[i] & 0x1c);
+ min_MPDU_spacing = max(pmlmeinfo->HT_caps.u.HT_cap_element.AMPDU_para & 0x1c,
+ pIE->data[i] & 0x1c);
pmlmeinfo->HT_caps.u.HT_cap_element.AMPDU_para = max_AMPDU_len | min_MPDU_spacing;
}
Final_CCK_Swing_Index = 0;
setIqkMatrix_8723B(pDM_Odm, Final_OFDM_Swing_Index, RFPath,
- pDM_Odm->RFCalibrateInfo.IQKMatrixRegSetting[ChannelMappedIndex].Value[0][0],
- pDM_Odm->RFCalibrateInfo.IQKMatrixRegSetting[ChannelMappedIndex].Value[0][1]);
+ pDM_Odm->RFCalibrateInfo.iqk_matrix_regs_setting_value[ChannelMappedIndex][0],
+ pDM_Odm->RFCalibrateInfo.iqk_matrix_regs_setting_value[ChannelMappedIndex][1]);
setCCKFilterCoefficient(pDM_Odm, Final_CCK_Swing_Index);
pDM_Odm->Remnant_OFDMSwingIdx[RFPath] = Final_OFDM_Swing_Index - PwrTrackingLimit_OFDM;
setIqkMatrix_8723B(pDM_Odm, PwrTrackingLimit_OFDM, RFPath,
- pDM_Odm->RFCalibrateInfo.IQKMatrixRegSetting[ChannelMappedIndex].Value[0][0],
- pDM_Odm->RFCalibrateInfo.IQKMatrixRegSetting[ChannelMappedIndex].Value[0][1]);
+ pDM_Odm->RFCalibrateInfo.iqk_matrix_regs_setting_value[ChannelMappedIndex][0],
+ pDM_Odm->RFCalibrateInfo.iqk_matrix_regs_setting_value[ChannelMappedIndex][1]);
pDM_Odm->Modify_TxAGC_Flag_PathA = true;
PHY_SetTxPowerIndexByRateSection(Adapter, RFPath, pHalData->CurrentChannel, OFDM);
pDM_Odm->Remnant_OFDMSwingIdx[RFPath] = Final_OFDM_Swing_Index;
setIqkMatrix_8723B(pDM_Odm, 0, RFPath,
- pDM_Odm->RFCalibrateInfo.IQKMatrixRegSetting[ChannelMappedIndex].Value[0][0],
- pDM_Odm->RFCalibrateInfo.IQKMatrixRegSetting[ChannelMappedIndex].Value[0][1]);
+ pDM_Odm->RFCalibrateInfo.iqk_matrix_regs_setting_value[ChannelMappedIndex][0],
+ pDM_Odm->RFCalibrateInfo.iqk_matrix_regs_setting_value[ChannelMappedIndex][1]);
pDM_Odm->Modify_TxAGC_Flag_PathA = true;
PHY_SetTxPowerIndexByRateSection(Adapter, RFPath, pHalData->CurrentChannel, OFDM);
PHY_SetTxPowerIndexByRateSection(Adapter, RFPath, pHalData->CurrentChannel, HT_MCS0_MCS7);
} else {
setIqkMatrix_8723B(pDM_Odm, Final_OFDM_Swing_Index, RFPath,
- pDM_Odm->RFCalibrateInfo.IQKMatrixRegSetting[ChannelMappedIndex].Value[0][0],
- pDM_Odm->RFCalibrateInfo.IQKMatrixRegSetting[ChannelMappedIndex].Value[0][1]);
+ pDM_Odm->RFCalibrateInfo.iqk_matrix_regs_setting_value[ChannelMappedIndex][0],
+ pDM_Odm->RFCalibrateInfo.iqk_matrix_regs_setting_value[ChannelMappedIndex][1]);
if (pDM_Odm->Modify_TxAGC_Flag_PathA) { /* If TxAGC has changed, reset TxAGC again */
pDM_Odm->Remnant_OFDMSwingIdx[RFPath] = 0;
/* To Fix BSOD when final_candidate is 0xff */
/* by sherry 20120321 */
if (final_candidate < 4) {
- for (i = 0; i < IQK_Matrix_REG_NUM; i++)
- pDM_Odm->RFCalibrateInfo.IQKMatrixRegSetting[0].Value[0][i] = result[final_candidate][i];
- pDM_Odm->RFCalibrateInfo.IQKMatrixRegSetting[0].bIQKDone = true;
+ for (i = 0; i < IQK_MATRIX_REG_NUM; i++)
+ pDM_Odm->RFCalibrateInfo.iqk_matrix_regs_setting_value[0][i] = result[final_candidate][i];
}
_PHY_SaveADDARegisters8723B(padapter, IQK_BB_REG_92C, pDM_Odm->RFCalibrateInfo.IQK_BB_backup_recover, 9);
#define HP_THERMAL_NUM 8
#define AVG_THERMAL_NUM 8
-#define IQK_Matrix_REG_NUM 8
-#define IQK_Matrix_Settings_NUM 14 /* Channels_2_4G_NUM */
+#define IQK_MATRIX_REG_NUM 8
+#define IQK_MATRIX_SETTINGS_NUM 14 /* Channels_2_4G_NUM */
#define DM_Type_ByFW 0
#define DM_Type_ByDriver 1
TYPE_ALNA3 = BIT(3)|BIT(2)|BIT(1)|BIT(0)
};
-struct iqk_matrix_regs_setting { /* _IQK_MATRIX_REGS_SETTING */
- bool bIQKDone;
- s32 Value[3][IQK_Matrix_REG_NUM];
- bool bBWIqkResultSaved[3];
-};
-
/* Remove PATHDIV_PARA struct to odm_PathDiv.h */
struct odm_rf_cal_t { /* ODM_RF_Calibration_Structure */
u8 ThermalValue_HP[HP_THERMAL_NUM];
u8 ThermalValue_HP_index;
- struct iqk_matrix_regs_setting IQKMatrixRegSetting[IQK_Matrix_Settings_NUM];
+ s32 iqk_matrix_regs_setting_value[IQK_MATRIX_SETTINGS_NUM][IQK_MATRIX_REG_NUM];
bool bNeedIQK;
bool bIQKInProgress;
u8 Delta_IQK;
/* Lower bound checking */
/* RSSI Lower bound check */
- if ((pDM_Odm->RSSI_Min-10) > DM_DIG_MIN_NIC)
- RSSI_Lower = pDM_Odm->RSSI_Min-10;
- else
- RSSI_Lower = DM_DIG_MIN_NIC;
+ RSSI_Lower = max(pDM_Odm->RSSI_Min - 10, DM_DIG_MIN_NIC);
/* Upper and Lower Bound checking */
if (CurrentIGI > DM_DIG_MAX_NIC)
wps_ie = rtw_get_wps_ie(buf, ielen, NULL, &wps_ielen);
if (wps_ie && wps_ielen > 0) {
- padapter->securitypriv.wps_ie_len = wps_ielen < MAX_WPS_IE_LEN ? wps_ielen : MAX_WPS_IE_LEN;
+ padapter->securitypriv.wps_ie_len = min_t(uint, wps_ielen, MAX_WPS_IE_LEN);
memcpy(padapter->securitypriv.wps_ie, wps_ie, padapter->securitypriv.wps_ie_len);
set_fwstate(&padapter->mlmepriv, WIFI_UNDER_WPS);
} else {
if (CHK_SD(sd_card)) {
retval = reset_sd(chip);
if (retval != STATUS_SUCCESS) {
- sd_card->sd_lock_status &= ~(SD_UNLOCK_POW_ON | SD_SDR_RST);
+ sd_card->sd_lock_status &=
+ ~(SD_UNLOCK_POW_ON | SD_SDR_RST);
goto sd_execute_write_cmd_failed;
}
}
select VIDEO_NOMODESET
help
Frame buffer driver for the Silicon Motion SM750 chip
- with 2D accelearion and dual head support.
+ with 2D acceleration and dual head support.
This driver is also available as a module. The module will be
called sm750fb. If you want to compile it as a module, say M
.depth = 12,
.mmal_component = COMP_CAMERA,
.ybbp = 1,
- .remove_padding = 1,
+ .remove_padding = true,
}, {
.fourcc = V4L2_PIX_FMT_YUYV,
.mmal = MMAL_ENCODING_YUYV,
.depth = 16,
.mmal_component = COMP_CAMERA,
.ybbp = 2,
- .remove_padding = 0,
+ .remove_padding = false,
}, {
.fourcc = V4L2_PIX_FMT_RGB24,
.mmal = MMAL_ENCODING_RGB24,
.depth = 24,
.mmal_component = COMP_CAMERA,
.ybbp = 3,
- .remove_padding = 0,
+ .remove_padding = false,
}, {
.fourcc = V4L2_PIX_FMT_JPEG,
.flags = V4L2_FMT_FLAG_COMPRESSED,
.depth = 8,
.mmal_component = COMP_IMAGE_ENCODE,
.ybbp = 0,
- .remove_padding = 0,
+ .remove_padding = false,
}, {
.fourcc = V4L2_PIX_FMT_H264,
.flags = V4L2_FMT_FLAG_COMPRESSED,
.depth = 8,
.mmal_component = COMP_VIDEO_ENCODE,
.ybbp = 0,
- .remove_padding = 0,
+ .remove_padding = false,
}, {
.fourcc = V4L2_PIX_FMT_MJPEG,
.flags = V4L2_FMT_FLAG_COMPRESSED,
.depth = 8,
.mmal_component = COMP_VIDEO_ENCODE,
.ybbp = 0,
- .remove_padding = 0,
+ .remove_padding = false,
}, {
.fourcc = V4L2_PIX_FMT_YVYU,
.mmal = MMAL_ENCODING_YVYU,
.depth = 16,
.mmal_component = COMP_CAMERA,
.ybbp = 2,
- .remove_padding = 0,
+ .remove_padding = false,
}, {
.fourcc = V4L2_PIX_FMT_VYUY,
.mmal = MMAL_ENCODING_VYUY,
.depth = 16,
.mmal_component = COMP_CAMERA,
.ybbp = 2,
- .remove_padding = 0,
+ .remove_padding = false,
}, {
.fourcc = V4L2_PIX_FMT_UYVY,
.mmal = MMAL_ENCODING_UYVY,
.depth = 16,
.mmal_component = COMP_CAMERA,
.ybbp = 2,
- .remove_padding = 0,
+ .remove_padding = false,
}, {
.fourcc = V4L2_PIX_FMT_NV12,
.mmal = MMAL_ENCODING_NV12,
.depth = 12,
.mmal_component = COMP_CAMERA,
.ybbp = 1,
- .remove_padding = 1,
+ .remove_padding = true,
}, {
.fourcc = V4L2_PIX_FMT_BGR24,
.mmal = MMAL_ENCODING_BGR24,
.depth = 24,
.mmal_component = COMP_CAMERA,
.ybbp = 3,
- .remove_padding = 0,
+ .remove_padding = false,
}, {
.fourcc = V4L2_PIX_FMT_YVU420,
.mmal = MMAL_ENCODING_YV12,
.depth = 12,
.mmal_component = COMP_CAMERA,
.ybbp = 1,
- .remove_padding = 1,
+ .remove_padding = true,
}, {
.fourcc = V4L2_PIX_FMT_NV21,
.mmal = MMAL_ENCODING_NV21,
.depth = 12,
.mmal_component = COMP_CAMERA,
.ybbp = 1,
- .remove_padding = 1,
+ .remove_padding = true,
}, {
.fourcc = V4L2_PIX_FMT_BGR32,
.mmal = MMAL_ENCODING_BGRA,
.depth = 32,
.mmal_component = COMP_CAMERA,
.ybbp = 4,
- .remove_padding = 0,
+ .remove_padding = false,
},
};
struct vchiq_mmal_port *port = NULL, *camera_port = NULL;
struct vchiq_mmal_component *encode_component = NULL;
struct mmal_fmt *mfmt = get_format(f);
- u32 remove_padding;
+ bool remove_padding;
if (!mfmt)
return -EINVAL;
goto release_msg;
if (rmsg->u.port_info_get_reply.port.is_enabled == 0)
- port->enabled = 0;
+ port->enabled = false;
else
- port->enabled = 1;
+ port->enabled = true;
/* copy the values out of the message */
port->handle = rmsg->u.port_info_get_reply.port_handle;
if (!port->enabled)
return 0;
- port->enabled = 0;
+ port->enabled = false;
ret = port_action_port(instance, port,
MMAL_MSG_PORT_ACTION_TYPE_DISABLE);
if (ret)
goto done;
- port->enabled = 1;
+ port->enabled = true;
if (port->buffer_cb) {
/* send buffer headers to videocore */
pr_err("failed disconnecting src port\n");
goto release_unlock;
}
- src->connected->enabled = 0;
+ src->connected->enabled = false;
src->connected = NULL;
}
for (idx = 0; idx < VCHIQ_MMAL_MAX_COMPONENTS; idx++) {
if (!instance->component[idx].in_use) {
component = &instance->component[idx];
- component->in_use = 1;
+ component->in_use = true;
break;
}
}
destroy_component(instance, component);
unlock:
if (component)
- component->in_use = 0;
+ component->in_use = false;
mutex_unlock(&instance->vchiq_mutex);
return ret;
ret = destroy_component(instance, component);
- component->in_use = 0;
+ component->in_use = false;
mutex_unlock(&instance->vchiq_mutex);
ret = disable_component(instance, component);
if (ret == 0)
- component->enabled = 0;
+ component->enabled = false;
mutex_unlock(&instance->vchiq_mutex);
int status, struct mmal_buffer *buffer);
struct vchiq_mmal_port {
- u32 enabled:1;
+ bool enabled;
u32 handle;
u32 type; /* port type, cached to use on port info set */
u32 index; /* port index, cached to use on port info set */
};
struct vchiq_mmal_component {
- u32 in_use:1;
- u32 enabled:1;
+ bool in_use;
+ bool enabled;
u32 handle; /* VideoCore handle for component */
u32 inputs; /* Number of input ports */
u32 outputs; /* Number of output ports */
#define VME_A64_MAX 0x10000000000000000ULL
#define VME_CRCSR_MAX 0x1000000ULL
-
/* VME Cycle Types */
#define VME_SCT 0x1
#define VME_BLT 0x2
int vme_register_driver(struct vme_driver *, unsigned int);
void vme_unregister_driver(struct vme_driver *);
-
#endif /* _VME_H_ */
return 0;
}
-
static int fake_master_get(struct vme_master_resource *image, int *enabled,
unsigned long long *vme_base, unsigned long long *size,
u32 *aspace, u32 *cycle, u32 *dwidth)
return retval;
}
-
static void fake_lm_check(struct fake_driver *bridge, unsigned long long addr,
u32 aspace, u32 cycle)
{
kfree(bridge->crcsr_kernel);
}
-
static int __init fake_init(void)
{
int retval, i;
struct vme_lm_resource *lm;
/* We need a fake parent device */
- vme_root = __root_device_register("vme", THIS_MODULE);
+ vme_root = root_device_register("vme");
+ if (IS_ERR(vme_root))
+ return PTR_ERR(vme_root);
/* If we want to support more than one bridge at some point, we need to
* dynamically allocate this so we get one per device.
}
-
static void __exit fake_exit(void)
{
struct list_head *pos = NULL;
root_device_unregister(vme_root);
}
-
MODULE_PARM_DESC(geoid, "Set geographical addressing");
module_param(geoid, int, 0);
static int tsi148_probe(struct pci_dev *, const struct pci_device_id *);
static void tsi148_remove(struct pci_dev *);
-
/* Module parameter */
static bool err_chk;
static int geoid;
/* Need granularity before we set the size */
*size = (unsigned long long)((vme_bound - *vme_base) + granularity);
-
if ((ctl & TSI148_LCSR_ITAT_2eSSTM_M) == TSI148_LCSR_ITAT_2eSSTM_160)
*cycle |= VME_2eSST160;
if ((ctl & TSI148_LCSR_ITAT_2eSSTM_M) == TSI148_LCSR_ITAT_2eSSTM_267)
return 0;
}
-
static int tsi148_master_get(struct vme_master_resource *image, int *enabled,
unsigned long long *vme_base, unsigned long long *size, u32 *aspace,
u32 *cycle, u32 *dwidth)
return retval;
}
-
static ssize_t tsi148_master_write(struct vme_master_resource *image, void *buf,
size_t count, loff_t offset)
{
return 0;
err_dma:
+ list_del(&entry->list);
err_dest:
err_source:
err_align:
if ((lm_ctl & TSI148_LCSR_LMAT_AS_M) == TSI148_LCSR_LMAT_AS_A64)
*aspace |= VME_A64;
-
if (lm_ctl & TSI148_LCSR_LMAT_SUPR)
*cycle |= VME_SUPER;
if (lm_ctl & TSI148_LCSR_LMAT_NPRIV)
bridge = tsi148_bridge->driver_priv;
-
dev_dbg(&pdev->dev, "Driver is being unloaded.\n");
/*
* Control and Status Registers
*/
-
/*
* Command/Status Registers (CRG + $004)
*/
#define TSI148_LCSR_IT7_ITOFL 0x3F4
#define TSI148_LCSR_IT7_ITAT 0x3F8
-
#define TSI148_LCSR_IT0 0x300
#define TSI148_LCSR_IT1 0x320
#define TSI148_LCSR_IT2 0x340
#define TSI148_LCSR_DMA0 0x500
#define TSI148_LCSR_DMA1 0x580
-
static const int TSI148_LCSR_DMA[TSI148_MAX_DMA] = { TSI148_LCSR_DMA0,
TSI148_LCSR_DMA1 };
#define TSI148_CSRBSR 0xFF8
#define TSI148_CBAR 0xFFC
-
-
-
/*
* TSI148 Register Bit Definitions
*/
#define TSI148_LCSR_VEAT_AM_M (0x3F<<8) /* Address Mode Mask */
#define TSI148_LCSR_VEAT_XAM_M (0xFF<<0) /* Master AMode Mask */
-
/*
* VMEbus PCI Error Diagnostics PCI/X Attributes Register CRG + $280
*/
*/
#define CRITICAL_PACKET_LEN 256
-static const unsigned short wTimeStampOff[2][MAX_RATE] = {
+static const unsigned short time_stamp_off[2][MAX_RATE] = {
{384, 288, 226, 209, 54, 43, 37, 31, 28, 25, 24, 23}, /* Long Preamble */
{384, 192, 130, 113, 54, 43, 37, 31, 28, 25, 24, 23}, /* Short Preamble */
};
-static const unsigned short wFB_Opt0[2][5] = {
+static const unsigned short fb_opt0[2][5] = {
{RATE_12M, RATE_18M, RATE_24M, RATE_36M, RATE_48M}, /* fallback_rate0 */
{RATE_12M, RATE_12M, RATE_18M, RATE_24M, RATE_36M}, /* fallback_rate1 */
};
-static const unsigned short wFB_Opt1[2][5] = {
+static const unsigned short fb_opt1[2][5] = {
{RATE_12M, RATE_18M, RATE_24M, RATE_24M, RATE_36M}, /* fallback_rate0 */
{RATE_6M, RATE_6M, RATE_12M, RATE_12M, RATE_18M}, /* fallback_rate1 */
};
static __le16 vnt_time_stamp_off(struct vnt_private *priv, u16 rate)
{
- return cpu_to_le16(wTimeStampOff[priv->preamble_type % 2]
+ return cpu_to_le16(time_stamp_off[priv->preamble_type % 2]
[rate % MAX_RATE]);
}
wRate -= RATE_18M;
if (byFBOption == AUTO_FB_0)
- wRate = wFB_Opt0[FB_RATE0][wRate];
+ wRate = fb_opt0[FB_RATE0][wRate];
else
- wRate = wFB_Opt1[FB_RATE0][wRate];
+ wRate = fb_opt1[FB_RATE0][wRate];
uNextPktTime = s_uGetTxRsvTime(pDevice, byPktType,
len, wRate, bNeedAck);
case RTSDUR_BA_F0: /* RTSDuration_ba_f0 */
uCTSTime = bb_get_frame_time(pDevice->preamble_type, byPktType, 14, pDevice->byTopCCKBasicRate);
if ((byFBOption == AUTO_FB_0) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt0[FB_RATE0][wRate - RATE_18M], bNeedAck);
+ uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt0[FB_RATE0][wRate - RATE_18M], bNeedAck);
else if ((byFBOption == AUTO_FB_1) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt1[FB_RATE0][wRate - RATE_18M], bNeedAck);
+ uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt1[FB_RATE0][wRate - RATE_18M], bNeedAck);
break;
case RTSDUR_AA_F0: /* RTSDuration_aa_f0 */
uCTSTime = bb_get_frame_time(pDevice->preamble_type, byPktType, 14, pDevice->byTopOFDMBasicRate);
if ((byFBOption == AUTO_FB_0) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt0[FB_RATE0][wRate - RATE_18M], bNeedAck);
+ uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt0[FB_RATE0][wRate - RATE_18M], bNeedAck);
else if ((byFBOption == AUTO_FB_1) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt1[FB_RATE0][wRate - RATE_18M], bNeedAck);
+ uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt1[FB_RATE0][wRate - RATE_18M], bNeedAck);
break;
case RTSDUR_BA_F1: /* RTSDuration_ba_f1 */
uCTSTime = bb_get_frame_time(pDevice->preamble_type, byPktType, 14, pDevice->byTopCCKBasicRate);
if ((byFBOption == AUTO_FB_0) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt0[FB_RATE1][wRate - RATE_18M], bNeedAck);
+ uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt0[FB_RATE1][wRate - RATE_18M], bNeedAck);
else if ((byFBOption == AUTO_FB_1) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt1[FB_RATE1][wRate - RATE_18M], bNeedAck);
+ uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt1[FB_RATE1][wRate - RATE_18M], bNeedAck);
break;
case RTSDUR_AA_F1: /* RTSDuration_aa_f1 */
uCTSTime = bb_get_frame_time(pDevice->preamble_type, byPktType, 14, pDevice->byTopOFDMBasicRate);
if ((byFBOption == AUTO_FB_0) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt0[FB_RATE1][wRate - RATE_18M], bNeedAck);
+ uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt0[FB_RATE1][wRate - RATE_18M], bNeedAck);
else if ((byFBOption == AUTO_FB_1) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt1[FB_RATE1][wRate - RATE_18M], bNeedAck);
+ uDurTime = uCTSTime + 2 * pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt1[FB_RATE1][wRate - RATE_18M], bNeedAck);
break;
case CTSDUR_BA_F0: /* CTSDuration_ba_f0 */
if ((byFBOption == AUTO_FB_0) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt0[FB_RATE0][wRate - RATE_18M], bNeedAck);
+ uDurTime = pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt0[FB_RATE0][wRate - RATE_18M], bNeedAck);
else if ((byFBOption == AUTO_FB_1) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt1[FB_RATE0][wRate - RATE_18M], bNeedAck);
+ uDurTime = pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt1[FB_RATE0][wRate - RATE_18M], bNeedAck);
break;
case CTSDUR_BA_F1: /* CTSDuration_ba_f1 */
if ((byFBOption == AUTO_FB_0) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt0[FB_RATE1][wRate - RATE_18M], bNeedAck);
+ uDurTime = pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt0[FB_RATE1][wRate - RATE_18M], bNeedAck);
else if ((byFBOption == AUTO_FB_1) && (wRate >= RATE_18M) && (wRate <= RATE_54M))
- uDurTime = pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, wFB_Opt1[FB_RATE1][wRate - RATE_18M], bNeedAck);
+ uDurTime = pDevice->uSIFS + s_uGetTxRsvTime(pDevice, byPktType, cbFrameLength, fb_opt1[FB_RATE1][wRate - RATE_18M], bNeedAck);
break;
--- /dev/null
+To-do list:
+
+* Correct the coding style according to Linux guidelines; please read the document
+ at https://www.kernel.org/doc/html/latest/process/coding-style.html.
+* Remove unnecessary debugging/printing macros; for those that are still needed
+ use the proper kernel API (pr_debug(), dev_dbg(), netdev_dbg()).
+* Remove dead code such as unusued functions, variables, fields, etc..
+* Use in-kernel API and remove unnecessary wrappers where possible.
+* Fix bugs due to code that sleeps in atomic context.
+* Remove the HAL layer and migrate its functionality into the relevant parts of
+ the driver.
+* Switch to use LIB80211.
+* Switch to use MAC80211.
+* Switch to use CFG80211.
+* Improve the error handling of various functions, particularly those that use
+ existing kernel APIs.
#define WLAN_SET_MGMT_CAP_INFO_PBCC(n) ((n) << 6)
#define WLAN_SET_MGMT_CAP_INFO_AGILITY(n) ((n) << 7)
-/*-- Information Element Types --------------------*/
-/* prototype structure, all IEs start with these members */
-
-struct wlan_ie {
- u8 eid;
- u8 len;
-} __packed;
-
-/*-- Service Set Identity (SSID) -----------------*/
-struct wlan_ie_ssid {
- u8 eid;
- u8 len;
- u8 ssid[1]; /* may be zero, ptrs may overlap */
-} __packed;
-
-/*-- Supported Rates -----------------------------*/
-struct wlan_ie_supp_rates {
- u8 eid;
- u8 len;
- u8 rates[1]; /* had better be at LEAST one! */
-} __packed;
-
-/*-- FH Parameter Set ----------------------------*/
-struct wlan_ie_fh_parms {
- u8 eid;
- u8 len;
- u16 dwell;
- u8 hopset;
- u8 hoppattern;
- u8 hopindex;
-} __packed;
-
-/*-- DS Parameter Set ----------------------------*/
-struct wlan_ie_ds_parms {
- u8 eid;
- u8 len;
- u8 curr_ch;
-} __packed;
-
-/*-- CF Parameter Set ----------------------------*/
-
-struct wlan_ie_cf_parms {
- u8 eid;
- u8 len;
- u8 cfp_cnt;
- u8 cfp_period;
- u16 cfp_maxdur;
- u16 cfp_durremaining;
-} __packed;
-
-/*-- TIM ------------------------------------------*/
-struct wlan_ie_tim {
- u8 eid;
- u8 len;
- u8 dtim_cnt;
- u8 dtim_period;
- u8 bitmap_ctl;
- u8 virt_bm[1];
-} __packed;
-
-/*-- IBSS Parameter Set ---------------------------*/
-struct wlan_ie_ibss_parms {
- u8 eid;
- u8 len;
- u16 atim_win;
-} __packed;
-
-/*-- Challenge Text ------------------------------*/
-struct wlan_ie_challenge {
- u8 eid;
- u8 len;
- u8 challenge[1];
-} __packed;
-
-/*-------------------------------------------------*/
-/* Frame Types */
-
-/* prototype structure, all mgmt frame types will start with these members */
-struct wlan_fr_mgmt {
- u16 type;
- u16 len; /* DOES NOT include CRC !!!! */
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- /*-- info elements ----------*/
-};
-
-/*-- Beacon ---------------------------------------*/
-struct wlan_fr_beacon {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- u64 *ts;
- u16 *bcn_int;
- u16 *cap_info;
- /*-- info elements ----------*/
- struct wlan_ie_ssid *ssid;
- struct wlan_ie_supp_rates *supp_rates;
- struct wlan_ie_fh_parms *fh_parms;
- struct wlan_ie_ds_parms *ds_parms;
- struct wlan_ie_cf_parms *cf_parms;
- struct wlan_ie_ibss_parms *ibss_parms;
- struct wlan_ie_tim *tim;
-
-};
-
-/*-- IBSS ATIM ------------------------------------*/
-struct wlan_fr_ibssatim {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
-
- /*-- fixed fields -----------*/
- /*-- info elements ----------*/
-
- /* this frame type has a null body */
-
-};
-
-/*-- Disassociation -------------------------------*/
-struct wlan_fr_disassoc {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- u16 *reason;
-
- /*-- info elements ----------*/
-
-};
-
-/*-- Association Request --------------------------*/
-struct wlan_fr_assocreq {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- u16 *cap_info;
- u16 *listen_int;
- /*-- info elements ----------*/
- struct wlan_ie_ssid *ssid;
- struct wlan_ie_supp_rates *supp_rates;
-
-};
-
-/*-- Association Response -------------------------*/
-struct wlan_fr_assocresp {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- u16 *cap_info;
- u16 *status;
- u16 *aid;
- /*-- info elements ----------*/
- struct wlan_ie_supp_rates *supp_rates;
-
-};
-
-/*-- Reassociation Request ------------------------*/
-struct wlan_fr_reassocreq {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- u16 *cap_info;
- u16 *listen_int;
- u8 *curr_ap;
- /*-- info elements ----------*/
- struct wlan_ie_ssid *ssid;
- struct wlan_ie_supp_rates *supp_rates;
-
-};
-
-/*-- Reassociation Response -----------------------*/
-struct wlan_fr_reassocresp {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- u16 *cap_info;
- u16 *status;
- u16 *aid;
- /*-- info elements ----------*/
- struct wlan_ie_supp_rates *supp_rates;
-
-};
-
-/*-- Probe Request --------------------------------*/
-struct wlan_fr_probereq {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- /*-- info elements ----------*/
- struct wlan_ie_ssid *ssid;
- struct wlan_ie_supp_rates *supp_rates;
-
-};
-
-/*-- Probe Response -------------------------------*/
-struct wlan_fr_proberesp {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- u64 *ts;
- u16 *bcn_int;
- u16 *cap_info;
- /*-- info elements ----------*/
- struct wlan_ie_ssid *ssid;
- struct wlan_ie_supp_rates *supp_rates;
- struct wlan_ie_fh_parms *fh_parms;
- struct wlan_ie_ds_parms *ds_parms;
- struct wlan_ie_cf_parms *cf_parms;
- struct wlan_ie_ibss_parms *ibss_parms;
-};
-
-/*-- Authentication -------------------------------*/
-struct wlan_fr_authen {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- u16 *auth_alg;
- u16 *auth_seq;
- u16 *status;
- /*-- info elements ----------*/
- struct wlan_ie_challenge *challenge;
-
-};
-
-/*-- Deauthenication -----------------------------*/
-struct wlan_fr_deauthen {
- u16 type;
- u16 len;
- u8 *buf;
- struct p80211_hdr *hdr;
- /* used for target specific data, skb in Linux */
- void *priv;
- /*-- fixed fields -----------*/
- u16 *reason;
-
- /*-- info elements ----------*/
-
-};
-
-void wlan_mgmt_encode_beacon(struct wlan_fr_beacon *f);
-void wlan_mgmt_decode_beacon(struct wlan_fr_beacon *f);
-void wlan_mgmt_encode_disassoc(struct wlan_fr_disassoc *f);
-void wlan_mgmt_decode_disassoc(struct wlan_fr_disassoc *f);
-void wlan_mgmt_encode_assocreq(struct wlan_fr_assocreq *f);
-void wlan_mgmt_decode_assocreq(struct wlan_fr_assocreq *f);
-void wlan_mgmt_encode_assocresp(struct wlan_fr_assocresp *f);
-void wlan_mgmt_decode_assocresp(struct wlan_fr_assocresp *f);
-void wlan_mgmt_encode_reassocreq(struct wlan_fr_reassocreq *f);
-void wlan_mgmt_decode_reassocreq(struct wlan_fr_reassocreq *f);
-void wlan_mgmt_encode_reassocresp(struct wlan_fr_reassocresp *f);
-void wlan_mgmt_decode_reassocresp(struct wlan_fr_reassocresp *f);
-void wlan_mgmt_encode_probereq(struct wlan_fr_probereq *f);
-void wlan_mgmt_decode_probereq(struct wlan_fr_probereq *f);
-void wlan_mgmt_encode_proberesp(struct wlan_fr_proberesp *f);
-void wlan_mgmt_decode_proberesp(struct wlan_fr_proberesp *f);
-void wlan_mgmt_encode_authen(struct wlan_fr_authen *f);
-void wlan_mgmt_decode_authen(struct wlan_fr_authen *f);
-void wlan_mgmt_encode_deauthen(struct wlan_fr_deauthen *f);
-void wlan_mgmt_decode_deauthen(struct wlan_fr_deauthen *f);
-
#endif /* _P80211MGMT_H */
wlandev->rx.mgmt++;
switch (fstype) {
case WLAN_FSTYPE_ASSOCREQ:
- /* printk("assocreq"); */
wlandev->rx.assocreq++;
break;
case WLAN_FSTYPE_ASSOCRESP:
- /* printk("assocresp"); */
wlandev->rx.assocresp++;
break;
case WLAN_FSTYPE_REASSOCREQ:
- /* printk("reassocreq"); */
wlandev->rx.reassocreq++;
break;
case WLAN_FSTYPE_REASSOCRESP:
- /* printk("reassocresp"); */
wlandev->rx.reassocresp++;
break;
case WLAN_FSTYPE_PROBEREQ:
- /* printk("probereq"); */
wlandev->rx.probereq++;
break;
case WLAN_FSTYPE_PROBERESP:
- /* printk("proberesp"); */
wlandev->rx.proberesp++;
break;
case WLAN_FSTYPE_BEACON:
- /* printk("beacon"); */
wlandev->rx.beacon++;
break;
case WLAN_FSTYPE_ATIM:
- /* printk("atim"); */
wlandev->rx.atim++;
break;
case WLAN_FSTYPE_DISASSOC:
- /* printk("disassoc"); */
wlandev->rx.disassoc++;
break;
case WLAN_FSTYPE_AUTHEN:
- /* printk("authen"); */
wlandev->rx.authen++;
break;
case WLAN_FSTYPE_DEAUTHEN:
- /* printk("deauthen"); */
wlandev->rx.deauthen++;
break;
default:
- /* printk("unknown"); */
wlandev->rx.mgmt_unknown++;
break;
}
- /* printk("\n"); */
drop = 2;
break;
wlandev->rx.ctl++;
switch (fstype) {
case WLAN_FSTYPE_PSPOLL:
- /* printk("pspoll"); */
wlandev->rx.pspoll++;
break;
case WLAN_FSTYPE_RTS:
- /* printk("rts"); */
wlandev->rx.rts++;
break;
case WLAN_FSTYPE_CTS:
- /* printk("cts"); */
wlandev->rx.cts++;
break;
case WLAN_FSTYPE_ACK:
- /* printk("ack"); */
wlandev->rx.ack++;
break;
case WLAN_FSTYPE_CFEND:
- /* printk("cfend"); */
wlandev->rx.cfend++;
break;
case WLAN_FSTYPE_CFENDCFACK:
- /* printk("cfendcfack"); */
wlandev->rx.cfendcfack++;
break;
default:
- /* printk("unknown"); */
wlandev->rx.ctl_unknown++;
break;
}
- /* printk("\n"); */
drop = 2;
break;
wlandev->rx.cfack_cfpoll++;
break;
default:
- /* printk("unknown"); */
wlandev->rx.data_unknown++;
break;
}
u8 data[MAXLEN_PSTR32];
} __packed;
-/* MAC address array */
-struct p80211macarray {
- u32 cnt;
- u8 data[1][MAXLEN_PSTR6];
-} __packed;
-
/* prototype template */
struct p80211item {
u32 did;
void **return_value)
{
struct acpi_device *adev = acpi_fetch_acpi_dev(handle);
- struct fwnode_reference_args args;
struct fwnode_handle *fwnode;
struct tb_nhi *nhi = data;
struct pci_dev *pdev;
struct device *dev;
- int ret;
if (!adev)
return AE_OK;
- fwnode = acpi_fwnode_handle(adev);
- ret = fwnode_property_get_reference_args(fwnode, "usb4-host-interface",
- NULL, 0, 0, &args);
- if (ret)
+ fwnode = fwnode_find_reference(acpi_fwnode_handle(adev), "usb4-host-interface", 0);
+ if (IS_ERR(fwnode))
return AE_OK;
/* It needs to reference this NHI */
- if (dev_fwnode(&nhi->pdev->dev) != args.fwnode)
+ if (dev_fwnode(&nhi->pdev->dev) != fwnode)
goto out_put;
/*
}
out_put:
- fwnode_handle_put(args.fwnode);
+ fwnode_handle_put(fwnode);
return AE_OK;
}
#include <linux/delay.h>
#include <linux/idr.h>
+#include <linux/module.h>
#include <linux/nvmem-provider.h>
#include <linux/pm_runtime.h>
#include <linux/sched/signal.h>
#include <linux/sizes.h>
#include <linux/slab.h>
-#include <linux/module.h>
+#include <linux/string_helpers.h>
#include "tb.h"
if (ret)
return ret;
- tb_port_dbg(port, "lane %sabled\n", enable ? "en" : "dis");
+ tb_port_dbg(port, "lane %s\n", str_enabled_disabled(enable));
return 0;
}
#define PORT_CS_18_BE BIT(8)
#define PORT_CS_18_TCM BIT(9)
#define PORT_CS_18_CPS BIT(10)
+#define PORT_CS_18_WOCS BIT(16)
+#define PORT_CS_18_WODS BIT(17)
#define PORT_CS_18_WOU4S BIT(18)
#define PORT_CS_19 0x13
#define PORT_CS_19_PC BIT(3)
static void usb4_switch_check_wakes(struct tb_switch *sw)
{
+ bool wakeup_usb4 = false;
+ struct usb4_port *usb4;
struct tb_port *port;
bool wakeup = false;
u32 val;
wakeup = val & (ROUTER_CS_6_WOPS | ROUTER_CS_6_WOUS);
}
- /* Check for any connected downstream ports for USB4 wake */
+ /*
+ * Check for any downstream ports for USB4 wake,
+ * connection wake and disconnection wake.
+ */
tb_switch_for_each_port(sw, port) {
- if (!tb_port_has_remote(port))
+ if (!port->cap_usb4)
continue;
if (tb_port_read(port, &val, TB_CFG_PORT,
port->cap_usb4 + PORT_CS_18, 1))
break;
- tb_port_dbg(port, "USB4 wake: %s\n",
- (val & PORT_CS_18_WOU4S) ? "yes" : "no");
+ tb_port_dbg(port, "USB4 wake: %s, connection wake: %s, disconnection wake: %s\n",
+ (val & PORT_CS_18_WOU4S) ? "yes" : "no",
+ (val & PORT_CS_18_WOCS) ? "yes" : "no",
+ (val & PORT_CS_18_WODS) ? "yes" : "no");
+
+ wakeup_usb4 = val & (PORT_CS_18_WOU4S | PORT_CS_18_WOCS |
+ PORT_CS_18_WODS);
+
+ usb4 = port->usb4;
+ if (device_may_wakeup(&usb4->dev) && wakeup_usb4)
+ pm_wakeup_event(&usb4->dev, 0);
- if (val & PORT_CS_18_WOU4S)
- wakeup = true;
+ wakeup |= wakeup_usb4;
}
if (wakeup)
*/
int usb4_switch_set_wake(struct tb_switch *sw, unsigned int flags)
{
+ struct usb4_port *usb4;
struct tb_port *port;
u64 route = tb_route(sw);
u32 val;
val |= PORT_CS_19_WOU4;
} else {
bool configured = val & PORT_CS_19_PC;
+ usb4 = port->usb4;
- if ((flags & TB_WAKE_ON_CONNECT) && !configured)
+ if (((flags & TB_WAKE_ON_CONNECT) |
+ device_may_wakeup(&usb4->dev)) && !configured)
val |= PORT_CS_19_WOC;
- if ((flags & TB_WAKE_ON_DISCONNECT) && configured)
+ if (((flags & TB_WAKE_ON_DISCONNECT) |
+ device_may_wakeup(&usb4->dev)) && configured)
val |= PORT_CS_19_WOD;
if ((flags & TB_WAKE_ON_USB4) && configured)
val |= PORT_CS_19_WOU4;
}
}
+ if (!tb_is_upstream_port(port))
+ device_set_wakeup_capable(&usb4->dev, true);
+
pm_runtime_no_callbacks(&usb4->dev);
pm_runtime_set_active(&usb4->dev);
pm_runtime_enable(&usb4->dev);
#include <linux/module.h>
#include <linux/pm_runtime.h>
#include <linux/prandom.h>
+#include <linux/string_helpers.h>
#include <linux/utsname.h>
#include <linux/uuid.h>
#include <linux/workqueue.h>
memcpy(&req.src_uuid, src_uuid, sizeof(*src_uuid));
memcpy(&req.dst_uuid, dst_uuid, sizeof(*dst_uuid));
- len = 0;
data_len = 0;
do {
tb_port_update_credits(port);
tb_xdomain_update_link_attributes(xd);
- dev_dbg(&xd->dev, "lane bonding %sabled\n", width == 2 ? "en" : "dis");
+ dev_dbg(&xd->dev, "lane bonding %s\n", str_enabled_disabled(width == 2));
return 0;
}
When not in use, each legacy PTY occupies 12 bytes on 32-bit
architectures and 24 bytes on 64-bit architectures.
+config LEGACY_TIOCSTI
+ bool "Allow legacy TIOCSTI usage"
+ default y
+ help
+ Historically the kernel has allowed TIOCSTI, which will push
+ characters into a controlling TTY. This continues to be used
+ as a malicious privilege escalation mechanism, and provides no
+ meaningful real-world utility any more. Its use is considered
+ a dangerous legacy operation, and can be disabled on most
+ systems.
+
+ Say 'Y here only if you have confirmed that your system's
+ userspace depends on this functionality to continue operating
+ normally.
+
+ This functionality can be changed at runtime with the
+ dev.tty.legacy_tiocsti sysctl. This configuration option sets
+ the default value of the sysctl.
+
config LDISC_AUTOLOAD
bool "Automatically load TTY Line Disciplines"
default y
return 0;
stdout_irq = irq_of_parse_and_map(np, 0);
- if (stdout_irq == NO_IRQ) {
+ if (!stdout_irq) {
pr_err("ehv-bc: no 'interrupts' property in %pOF node\n", np);
return 0;
}
bc->rx_irq = irq_of_parse_and_map(np, 0);
bc->tx_irq = irq_of_parse_and_map(np, 1);
- if ((bc->rx_irq == NO_IRQ) || (bc->tx_irq == NO_IRQ)) {
+ if (!bc->rx_irq || !bc->tx_irq) {
dev_err(&pdev->dev, "no 'interrupts' property in %pOFn node\n",
np);
ret = -ENODEV;
#include "hvc_console.h"
#define hvc_rtas_cookie 0x67781e15
-struct hvc_struct *hvc_rtas_dev;
+static struct hvc_struct *hvc_rtas_dev;
static int rtascons_put_char_token = RTAS_UNKNOWN_SERVICE;
static int rtascons_get_char_token = RTAS_UNKNOWN_SERVICE;
#include <linux/sched/signal.h>
#include <linux/interrupt.h>
#include <linux/tty.h>
+#include <linux/bitfield.h>
#include <linux/ctype.h>
#include <linux/mm.h>
+#include <linux/math.h>
#include <linux/string.h>
#include <linux/slab.h>
#include <linux/poll.h>
#define T1 10 /* 100mS */
#define T2 34 /* 333mS */
+#define T3 10 /* 10s */
#define N2 3 /* Retry 3 times */
+#define K 2 /* outstanding I frames */
+
+#define MAX_T3 255 /* In seconds. */
+#define MAX_WINDOW_SIZE 7 /* Limit of K in error recovery mode. */
/* Use long timers for testing at low speed with debug on */
#ifdef DEBUG_TIMING
*/
#define MAX_MRU 1500
#define MAX_MTU 1500
+#define MIN_MTU (PROT_OVERHEAD + 1)
/* SOF, ADDR, CTRL, LEN1, LEN2, ..., FCS, EOF */
#define PROT_OVERHEAD 7
#define GSM_NET_TX_TIMEOUT (HZ*10)
enum gsm_dlci_state {
DLCI_CLOSED,
+ DLCI_CONFIGURE, /* Sending PN (for adaption > 1) */
DLCI_OPENING, /* Sending SABM not seen UA */
DLCI_OPEN, /* SABM/UA complete */
DLCI_CLOSING, /* Sending DISC not seen UA/DM */
int prev_adaption;
u32 modem_rx; /* Our incoming virtual modem lines */
u32 modem_tx; /* Our outgoing modem lines */
+ unsigned int mtu;
bool dead; /* Refuse re-open */
+ /* Configuration */
+ u8 prio; /* Priority */
+ u8 ftype; /* Frame type */
+ u8 k; /* Window size */
/* Flow control */
bool throttled; /* Private copy of throttle state */
bool constipated; /* Throttle status for outgoing */
struct net_device *net; /* network interface, if created */
};
+/*
+ * Parameter bits used for parameter negotiation according to 3GPP 27.010
+ * chapter 5.4.6.3.1.
+ */
+
+struct gsm_dlci_param_bits {
+ u8 d_bits;
+ u8 i_cl_bits;
+ u8 p_bits;
+ u8 t_bits;
+ __le16 n_bits;
+ u8 na_bits;
+ u8 k_bits;
+};
+
+static_assert(sizeof(struct gsm_dlci_param_bits) == 8);
+
+#define PN_D_FIELD_DLCI GENMASK(5, 0)
+#define PN_I_CL_FIELD_FTYPE GENMASK(3, 0)
+#define PN_I_CL_FIELD_ADAPTION GENMASK(7, 4)
+#define PN_P_FIELD_PRIO GENMASK(5, 0)
+#define PN_T_FIELD_T1 GENMASK(7, 0)
+#define PN_N_FIELD_N1 GENMASK(15, 0)
+#define PN_NA_FIELD_N2 GENMASK(7, 0)
+#define PN_K_FIELD_K GENMASK(2, 0)
+
/* Total number of supported devices */
#define GSM_TTY_MINORS 256
int adaption; /* 1 or 2 supported */
u8 ftype; /* UI or UIH */
int t1, t2; /* Timers in 1/100th of a sec */
+ unsigned int t3; /* Power wake-up timer in seconds. */
int n2; /* Retry count */
+ u8 k; /* Window size */
/* Statistics (not currently exposed) */
unsigned long bad_fcs;
#define INIT_FCS 0xFF
#define GOOD_FCS 0xCF
+static void gsm_dlci_close(struct gsm_dlci *dlci);
static int gsmld_output(struct gsm_mux *gsm, u8 *data, int len);
static int gsm_modem_update(struct gsm_dlci *dlci, u8 brk);
static struct gsm_msg *gsm_data_alloc(struct gsm_mux *gsm, u8 addr, int len,
}
/**
+ * gsm_encode_params - encode DLCI parameters
+ * @dlci: DLCI to encode from
+ * @params: buffer to fill with the encoded parameters
+ *
+ * Encodes the parameters according to GSM 07.10 section 5.4.6.3.1
+ * table 3.
+ */
+static int gsm_encode_params(const struct gsm_dlci *dlci,
+ struct gsm_dlci_param_bits *params)
+{
+ const struct gsm_mux *gsm = dlci->gsm;
+ unsigned int i, cl;
+
+ switch (dlci->ftype) {
+ case UIH:
+ i = 0; /* UIH */
+ break;
+ case UI:
+ i = 1; /* UI */
+ break;
+ default:
+ pr_debug("unsupported frame type %d\n", dlci->ftype);
+ return -EINVAL;
+ }
+
+ switch (dlci->adaption) {
+ case 1: /* Unstructured */
+ cl = 0; /* convergence layer type 1 */
+ break;
+ case 2: /* Unstructured with modem bits. */
+ cl = 1; /* convergence layer type 2 */
+ break;
+ default:
+ pr_debug("unsupported adaption %d\n", dlci->adaption);
+ return -EINVAL;
+ }
+
+ params->d_bits = FIELD_PREP(PN_D_FIELD_DLCI, dlci->addr);
+ /* UIH, convergence layer type 1 */
+ params->i_cl_bits = FIELD_PREP(PN_I_CL_FIELD_FTYPE, i) |
+ FIELD_PREP(PN_I_CL_FIELD_ADAPTION, cl);
+ params->p_bits = FIELD_PREP(PN_P_FIELD_PRIO, dlci->prio);
+ params->t_bits = FIELD_PREP(PN_T_FIELD_T1, gsm->t1);
+ params->n_bits = cpu_to_le16(FIELD_PREP(PN_N_FIELD_N1, dlci->mtu));
+ params->na_bits = FIELD_PREP(PN_NA_FIELD_N2, gsm->n2);
+ params->k_bits = FIELD_PREP(PN_K_FIELD_K, dlci->k);
+
+ return 0;
+}
+
+/**
* gsm_register_devices - register all tty devices for a given mux index
*
* @driver: the tty driver that describes the tty devices
return 0;
/* MTU/MRU count only the data bits but watch adaption mode */
- if ((len + h) > gsm->mtu)
- len = gsm->mtu - h;
+ if ((len + h) > dlci->mtu)
+ len = dlci->mtu - h;
size = len + h;
- msg = gsm_data_alloc(gsm, dlci->addr, size, gsm->ftype);
+ msg = gsm_data_alloc(gsm, dlci->addr, size, dlci->ftype);
if (!msg)
return -ENOMEM;
dp = msg->data;
len = dlci->skb->len + overhead;
/* MTU/MRU count only the data bits */
- if (len > gsm->mtu) {
+ if (len > dlci->mtu) {
if (dlci->adaption == 3) {
/* Over long frame, bin it */
dev_kfree_skb_any(dlci->skb);
dlci->skb = NULL;
return 0;
}
- len = gsm->mtu;
+ len = dlci->mtu;
} else
last = 1;
size = len + overhead;
- msg = gsm_data_alloc(gsm, dlci->addr, size, gsm->ftype);
+ msg = gsm_data_alloc(gsm, dlci->addr, size, dlci->ftype);
if (msg == NULL) {
skb_queue_tail(&dlci->skb_list, dlci->skb);
dlci->skb = NULL;
return -EINVAL;
}
- msg = gsm_data_alloc(gsm, dlci->addr, size, gsm->ftype);
+ msg = gsm_data_alloc(gsm, dlci->addr, size, dlci->ftype);
if (!msg) {
pr_err("%s: gsm_data_alloc error", __func__);
return -ENOMEM;
}
if (!sent)
break;
- };
+ }
return ret;
}
static int gsm_control_command(struct gsm_mux *gsm, int cmd, const u8 *data,
int dlen)
{
- struct gsm_msg *msg = gsm_data_alloc(gsm, 0, dlen + 2, gsm->ftype);
+ struct gsm_msg *msg;
+ msg = gsm_data_alloc(gsm, 0, dlen + 2, gsm->dlci[0]->ftype);
if (msg == NULL)
return -ENOMEM;
int dlen)
{
struct gsm_msg *msg;
- msg = gsm_data_alloc(gsm, 0, dlen + 2, gsm->ftype);
+
+ msg = gsm_data_alloc(gsm, 0, dlen + 2, gsm->dlci[0]->ftype);
if (msg == NULL)
return;
msg->data[0] = (cmd & 0xFE) << 1 | EA; /* Clear C/R */
}
/**
+ * gsm_process_negotiation - process received parameters
+ * @gsm: GSM channel
+ * @addr: DLCI address
+ * @cr: command/response
+ * @params: encoded parameters from the parameter negotiation message
+ *
+ * Used when the response for our parameter negotiation command was
+ * received.
+ */
+static int gsm_process_negotiation(struct gsm_mux *gsm, unsigned int addr,
+ unsigned int cr,
+ const struct gsm_dlci_param_bits *params)
+{
+ struct gsm_dlci *dlci = gsm->dlci[addr];
+ unsigned int ftype, i, adaption, prio, n1, k;
+
+ i = FIELD_GET(PN_I_CL_FIELD_FTYPE, params->i_cl_bits);
+ adaption = FIELD_GET(PN_I_CL_FIELD_ADAPTION, params->i_cl_bits) + 1;
+ prio = FIELD_GET(PN_P_FIELD_PRIO, params->p_bits);
+ n1 = FIELD_GET(PN_N_FIELD_N1, get_unaligned_le16(¶ms->n_bits));
+ k = FIELD_GET(PN_K_FIELD_K, params->k_bits);
+
+ if (n1 < MIN_MTU) {
+ if (debug & DBG_ERRORS)
+ pr_info("%s N1 out of range in PN\n", __func__);
+ return -EINVAL;
+ }
+
+ switch (i) {
+ case 0x00:
+ ftype = UIH;
+ break;
+ case 0x01:
+ ftype = UI;
+ break;
+ case 0x02: /* I frames are not supported */
+ if (debug & DBG_ERRORS)
+ pr_info("%s unsupported I frame request in PN\n",
+ __func__);
+ return -EINVAL;
+ default:
+ if (debug & DBG_ERRORS)
+ pr_info("%s i out of range in PN\n", __func__);
+ return -EINVAL;
+ }
+
+ if (!cr && gsm->initiator) {
+ if (adaption != dlci->adaption) {
+ if (debug & DBG_ERRORS)
+ pr_info("%s invalid adaption %d in PN\n",
+ __func__, adaption);
+ return -EINVAL;
+ }
+ if (prio != dlci->prio) {
+ if (debug & DBG_ERRORS)
+ pr_info("%s invalid priority %d in PN",
+ __func__, prio);
+ return -EINVAL;
+ }
+ if (n1 > gsm->mru || n1 > dlci->mtu) {
+ /* We requested a frame size but the other party wants
+ * to send larger frames. The standard allows only a
+ * smaller response value than requested (5.4.6.3.1).
+ */
+ if (debug & DBG_ERRORS)
+ pr_info("%s invalid N1 %d in PN\n", __func__,
+ n1);
+ return -EINVAL;
+ }
+ dlci->mtu = n1;
+ if (ftype != dlci->ftype) {
+ if (debug & DBG_ERRORS)
+ pr_info("%s invalid i %d in PN\n", __func__, i);
+ return -EINVAL;
+ }
+ if (ftype != UI && ftype != UIH && k > dlci->k) {
+ if (debug & DBG_ERRORS)
+ pr_info("%s invalid k %d in PN\n", __func__, k);
+ return -EINVAL;
+ }
+ dlci->k = k;
+ } else if (cr && !gsm->initiator) {
+ /* Only convergence layer type 1 and 2 are supported. */
+ if (adaption != 1 && adaption != 2) {
+ if (debug & DBG_ERRORS)
+ pr_info("%s invalid adaption %d in PN\n",
+ __func__, adaption);
+ return -EINVAL;
+ }
+ dlci->adaption = adaption;
+ if (n1 > gsm->mru) {
+ /* Propose a smaller value */
+ dlci->mtu = gsm->mru;
+ } else if (n1 > MAX_MTU) {
+ /* Propose a smaller value */
+ dlci->mtu = MAX_MTU;
+ } else {
+ dlci->mtu = n1;
+ }
+ dlci->prio = prio;
+ dlci->ftype = ftype;
+ dlci->k = k;
+ } else {
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/**
* gsm_control_modem - modem status received
* @gsm: GSM channel
* @data: data following command
}
/**
+ * gsm_control_negotiation - parameter negotiation received
+ * @gsm: GSM channel
+ * @cr: command/response flag
+ * @data: data following command
+ * @dlen: data length
+ *
+ * We have received a parameter negotiation message. This is used by
+ * the GSM mux protocol to configure protocol parameters for a new DLCI.
+ */
+static void gsm_control_negotiation(struct gsm_mux *gsm, unsigned int cr,
+ const u8 *data, unsigned int dlen)
+{
+ unsigned int addr;
+ struct gsm_dlci_param_bits pn_reply;
+ struct gsm_dlci *dlci;
+ struct gsm_dlci_param_bits *params;
+
+ if (dlen < sizeof(struct gsm_dlci_param_bits))
+ return;
+
+ /* Invalid DLCI? */
+ params = (struct gsm_dlci_param_bits *)data;
+ addr = FIELD_GET(PN_D_FIELD_DLCI, params->d_bits);
+ if (addr == 0 || addr >= NUM_DLCI || !gsm->dlci[addr])
+ return;
+ dlci = gsm->dlci[addr];
+
+ /* Too late for parameter negotiation? */
+ if ((!cr && dlci->state == DLCI_OPENING) || dlci->state == DLCI_OPEN)
+ return;
+
+ /* Process the received parameters */
+ if (gsm_process_negotiation(gsm, addr, cr, params) != 0) {
+ /* Negotiation failed. Close the link. */
+ if (debug & DBG_ERRORS)
+ pr_info("%s PN failed\n", __func__);
+ gsm_dlci_close(dlci);
+ return;
+ }
+
+ if (cr) {
+ /* Reply command with accepted parameters. */
+ if (gsm_encode_params(dlci, &pn_reply) == 0)
+ gsm_control_reply(gsm, CMD_PN, (const u8 *)&pn_reply,
+ sizeof(pn_reply));
+ else if (debug & DBG_ERRORS)
+ pr_info("%s PN invalid\n", __func__);
+ } else if (dlci->state == DLCI_CONFIGURE) {
+ /* Proceed with link setup by sending SABM before UA */
+ dlci->state = DLCI_OPENING;
+ gsm_command(gsm, dlci->addr, SABM|PF);
+ mod_timer(&dlci->t1, jiffies + gsm->t1 * HZ / 100);
+ } else {
+ if (debug & DBG_ERRORS)
+ pr_info("%s PN in invalid state\n", __func__);
+ }
+}
+
+/**
* gsm_control_rls - remote line status
* @gsm: GSM channel
* @data: data bytes
/* Modem wishes to enter power saving state */
gsm_control_reply(gsm, CMD_PSC, NULL, 0);
break;
+ /* Optional commands */
+ case CMD_PN:
+ /* Modem sends a parameter negotiation command */
+ gsm_control_negotiation(gsm, 1, data, clen);
+ break;
/* Optional unsupported commands */
- case CMD_PN: /* Parameter negotiation */
case CMD_RPN: /* Remote port negotiation */
case CMD_SNC: /* Service negotiation command */
default:
spin_lock_irqsave(&gsm->control_lock, flags);
ctrl = gsm->pending_cmd;
- /* Does the reply match our command */
command |= 1;
+ /* Does the reply match our command */
if (ctrl != NULL && (command == ctrl->cmd || command == CMD_NSC)) {
/* Our command was replied to, kill the retry timer */
del_timer(&gsm->t2_timer);
ctrl->error = -EOPNOTSUPP;
ctrl->done = 1;
wake_up(&gsm->event);
+ /* Or did we receive the PN response to our PN command */
+ } else if (command == CMD_PN) {
+ gsm_control_negotiation(gsm, 0, data, clen);
}
spin_unlock_irqrestore(&gsm->control_lock, flags);
}
}
/**
+ * gsm_dlci_negotiate - start parameter negotiation
+ * @dlci: DLCI to open
+ *
+ * Starts the parameter negotiation for the new DLCI. This needs to be done
+ * before the DLCI initialized the channel via SABM.
+ */
+static int gsm_dlci_negotiate(struct gsm_dlci *dlci)
+{
+ struct gsm_mux *gsm = dlci->gsm;
+ struct gsm_dlci_param_bits params;
+ int ret;
+
+ ret = gsm_encode_params(dlci, ¶ms);
+ if (ret != 0)
+ return ret;
+
+ /* We cannot asynchronous wait for the command response with
+ * gsm_command() and gsm_control_wait() at this point.
+ */
+ ret = gsm_control_command(gsm, CMD_PN, (const u8 *)¶ms,
+ sizeof(params));
+
+ return ret;
+}
+
+/**
* gsm_dlci_t1 - T1 timer expiry
* @t: timer contained in the DLCI that opened
*
struct gsm_mux *gsm = dlci->gsm;
switch (dlci->state) {
+ case DLCI_CONFIGURE:
+ if (dlci->retries && gsm_dlci_negotiate(dlci) == 0) {
+ dlci->retries--;
+ mod_timer(&dlci->t1, jiffies + gsm->t1 * HZ / 100);
+ } else {
+ gsm_dlci_begin_close(dlci); /* prevent half open link */
+ }
+ break;
case DLCI_OPENING:
if (dlci->retries) {
dlci->retries--;
* to the modem which should then reply with a UA or ADM, at which point
* we will move into open state. Opening is done asynchronously with retry
* running off timers and the responses.
+ * Parameter negotiation is performed before SABM if required.
*/
static void gsm_dlci_begin_open(struct gsm_dlci *dlci)
{
- struct gsm_mux *gsm = dlci->gsm;
- if (dlci->state == DLCI_OPEN || dlci->state == DLCI_OPENING)
+ struct gsm_mux *gsm = dlci ? dlci->gsm : NULL;
+ bool need_pn = false;
+
+ if (!gsm)
return;
- dlci->retries = gsm->n2;
- dlci->state = DLCI_OPENING;
- gsm_command(dlci->gsm, dlci->addr, SABM|PF);
- mod_timer(&dlci->t1, jiffies + gsm->t1 * HZ / 100);
+
+ if (dlci->addr != 0) {
+ if (gsm->adaption != 1 || gsm->adaption != dlci->adaption)
+ need_pn = true;
+ if (dlci->prio != (roundup(dlci->addr + 1, 8) - 1))
+ need_pn = true;
+ if (gsm->ftype != dlci->ftype)
+ need_pn = true;
+ }
+
+ switch (dlci->state) {
+ case DLCI_CLOSED:
+ case DLCI_CLOSING:
+ dlci->retries = gsm->n2;
+ if (!need_pn) {
+ dlci->state = DLCI_OPENING;
+ gsm_command(gsm, dlci->addr, SABM|PF);
+ } else {
+ /* Configure DLCI before setup */
+ dlci->state = DLCI_CONFIGURE;
+ if (gsm_dlci_negotiate(dlci) != 0) {
+ gsm_dlci_close(dlci);
+ return;
+ }
+ }
+ mod_timer(&dlci->t1, jiffies + gsm->t1 * HZ / 100);
+ break;
+ default:
+ break;
+ }
}
/**
dlci->gsm = gsm;
dlci->addr = addr;
dlci->adaption = gsm->adaption;
+ dlci->mtu = gsm->mtu;
+ if (addr == 0)
+ dlci->prio = 0;
+ else
+ dlci->prio = roundup(addr + 1, 8) - 1;
+ dlci->ftype = gsm->ftype;
+ dlci->k = gsm->k;
dlci->state = DLCI_CLOSED;
if (addr) {
dlci->data = gsm_dlci_data;
gsm->t1 = T1;
gsm->t2 = T2;
+ gsm->t3 = T3;
gsm->n2 = N2;
+ gsm->k = K;
gsm->ftype = UIH;
gsm->adaption = 1;
gsm->encoding = GSM_ADV_OPT;
c->initiator = gsm->initiator;
c->t1 = gsm->t1;
c->t2 = gsm->t2;
- c->t3 = 0; /* Not supported */
+ c->t3 = gsm->t3;
c->n2 = gsm->n2;
if (gsm->ftype == UIH)
c->i = 1;
pr_debug("Ftype %d i %d\n", gsm->ftype, c->i);
c->mru = gsm->mru;
c->mtu = gsm->mtu;
- c->k = 0;
+ c->k = gsm->k;
}
static int gsm_config(struct gsm_mux *gsm, struct gsm_config *c)
if ((c->adaption != 1 && c->adaption != 2) || c->k)
return -EOPNOTSUPP;
/* Check the MRU/MTU range looks sane */
- if (c->mru > MAX_MRU || c->mtu > MAX_MTU || c->mru < 8 || c->mtu < 8)
+ if (c->mru < MIN_MTU || c->mtu < MIN_MTU)
+ return -EINVAL;
+ if (c->mru > MAX_MRU || c->mtu > MAX_MTU)
+ return -EINVAL;
+ if (c->t3 > MAX_T3)
return -EINVAL;
if (c->n2 > 255)
return -EINVAL;
return -EINVAL;
if (c->initiator > 1)
return -EINVAL;
+ if (c->k > MAX_WINDOW_SIZE)
+ return -EINVAL;
if (c->i == 0 || c->i > 2) /* UIH and UI only */
return -EINVAL;
/*
gsm->t1 = c->t1;
if (c->t2)
gsm->t2 = c->t2;
+ if (c->t3)
+ gsm->t3 = c->t3;
+ if (c->k)
+ gsm->k = c->k;
/*
* FIXME: We need to separate activation/deactivation from adding
pr_err("alloc_netdev failed\n");
return -ENOMEM;
}
- net->mtu = dlci->gsm->mtu;
- net->min_mtu = 8;
- net->max_mtu = dlci->gsm->mtu;
+ net->mtu = dlci->mtu;
+ net->min_mtu = MIN_MTU;
+ net->max_mtu = dlci->mtu;
mux_net = netdev_priv(net);
mux_net->dlci = dlci;
kref_init(&mux_net->ref);
ssize_t retval = 0;
long timeout;
bool packet;
- size_t tail;
+ size_t old_tail;
/*
* Is this a continuation of a read started earler?
}
packet = tty->ctrl.packet;
- tail = ldata->read_tail;
+ old_tail = ldata->read_tail;
add_wait_queue(&tty->read_wait, &wait);
while (nr) {
if (time)
timeout = time;
}
- if (tail != ldata->read_tail)
+ if (old_tail != ldata->read_tail)
n_tty_kick_worker(tty);
up_read(&tty->termios_rwsem);
static irqreturn_t serial21285_tx_chars(int irq, void *dev_id)
{
struct uart_port *port = dev_id;
- struct circ_buf *xmit = &port->state->xmit;
- int count = 256;
-
- if (port->x_char) {
- *CSR_UARTDR = port->x_char;
- port->icount.tx++;
- port->x_char = 0;
- goto out;
- }
- if (uart_circ_empty(xmit) || uart_tx_stopped(port)) {
- serial21285_stop_tx(port);
- goto out;
- }
-
- do {
- *CSR_UARTDR = xmit->buf[xmit->tail];
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- } while (--count > 0 && !(*CSR_UARTFLG & 0x20));
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
+ u8 ch;
- if (uart_circ_empty(xmit))
- serial21285_stop_tx(port);
+ uart_port_tx_limited(port, ch, 256,
+ !(*CSR_UARTFLG & 0x20),
+ *CSR_UARTDR = ch,
+ ({}));
- out:
return IRQ_HANDLED;
}
priv->dma.tx_err = 0;
memcpy(priv->tx_buf, &xmit->buf[xmit->tail], tx_size);
- xmit->tail += tx_size;
- xmit->tail &= UART_XMIT_SIZE - 1;
- p->port.icount.tx += tx_size;
+ uart_xmit_advance(&p->port, tx_size);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(&p->port);
static int __init brcmuart_init(void)
{
+ int ret;
+
brcmuart_debugfs_root = debugfs_create_dir(
brcmuart_platform_driver.driver.name, NULL);
- return platform_driver_register(&brcmuart_platform_driver);
+ ret = platform_driver_register(&brcmuart_platform_driver);
+ if (ret) {
+ debugfs_remove_recursive(brcmuart_debugfs_root);
+ return ret;
+ }
+
+ return 0;
}
module_init(brcmuart_init);
#include <linux/nmi.h>
#include <linux/mutex.h>
#include <linux/slab.h>
+#include <linux/string_helpers.h>
#include <linux/uaccess.h>
#include <linux/io.h>
#ifdef CONFIG_SPARC
serial8250_isa_init_ports();
- pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
- nr_uarts, share_irqs ? "en" : "dis");
+ pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %s\n",
+ nr_uarts, str_enabled_disabled(share_irqs));
#ifdef CONFIG_SPARC
ret = sunserial_register_minors(&serial8250_reg, UART_NR);
spin_unlock_irqrestore(&p->port.lock, flags);
}
-static void __dma_rx_complete(void *param)
+static void __dma_rx_complete(struct uart_8250_port *p)
{
- struct uart_8250_port *p = param;
struct uart_8250_dma *dma = p->dma;
struct tty_port *tty_port = &p->port.state->port;
struct dma_tx_state state;
tty_flip_buffer_push(tty_port);
}
+static void dma_rx_complete(void *param)
+{
+ struct uart_8250_port *p = param;
+ struct uart_8250_dma *dma = p->dma;
+ unsigned long flags;
+
+ __dma_rx_complete(p);
+
+ spin_lock_irqsave(&p->port.lock, flags);
+ if (!dma->rx_running && (serial_lsr_in(p) & UART_LSR_DR))
+ p->dma->rx_dma(p);
+ spin_unlock_irqrestore(&p->port.lock, flags);
+}
+
int serial8250_tx_dma(struct uart_8250_port *p)
{
struct uart_8250_dma *dma = p->dma;
return -EBUSY;
dma->rx_running = 1;
- desc->callback = __dma_rx_complete;
+ desc->callback = dma_rx_complete;
desc->callback_param = p;
dma->rx_cookie = dmaengine_submit(desc);
dev->port.uartclk = be32_to_cpup(prop);
}
-static int __init ingenic_early_console_setup(struct earlycon_device *dev,
+static int __init ingenic_earlycon_setup_tail(struct earlycon_device *dev,
const char *opt)
{
struct uart_port *port = &dev->port;
uart_parse_options(opt, &baud, &parity, &bits, &flow);
}
- ingenic_early_console_setup_clock(dev);
-
if (dev->baud)
baud = dev->baud;
divisor = DIV_ROUND_CLOSEST(port->uartclk, 16 * baud);
return 0;
}
+static int __init ingenic_early_console_setup(struct earlycon_device *dev,
+ const char *opt)
+{
+ ingenic_early_console_setup_clock(dev);
+
+ return ingenic_earlycon_setup_tail(dev, opt);
+}
+
+static int __init jz4750_early_console_setup(struct earlycon_device *dev,
+ const char *opt)
+{
+ /*
+ * JZ4750/55/60 have an optional /2 divider between the EXT
+ * oscillator and some peripherals including UART, which will
+ * be enabled if using a 24 MHz oscillator, and disabled when
+ * using a 12 MHz oscillator.
+ */
+ ingenic_early_console_setup_clock(dev);
+ if (dev->port.uartclk >= 16000000)
+ dev->port.uartclk /= 2;
+
+ return ingenic_earlycon_setup_tail(dev, opt);
+}
+
OF_EARLYCON_DECLARE(jz4740_uart, "ingenic,jz4740-uart",
ingenic_early_console_setup);
+OF_EARLYCON_DECLARE(jz4750_uart, "ingenic,jz4750-uart",
+ jz4750_early_console_setup);
+
OF_EARLYCON_DECLARE(jz4770_uart, "ingenic,jz4770-uart",
ingenic_early_console_setup);
static const struct of_device_id of_match[] = {
{ .compatible = "ingenic,jz4740-uart", .data = &jz4740_uart_config },
+ { .compatible = "ingenic,jz4750-uart", .data = &jz4760_uart_config },
{ .compatible = "ingenic,jz4760-uart", .data = &jz4760_uart_config },
{ .compatible = "ingenic,jz4770-uart", .data = &jz4760_uart_config },
{ .compatible = "ingenic,jz4775-uart", .data = &jz4760_uart_config },
#define UART_HAS_EFR2 BIT(4)
#define UART_HAS_RHR_IT_DIS BIT(5)
#define UART_RX_TIMEOUT_QUIRK BIT(6)
+#define UART_HAS_NATIVE_RS485 BIT(7)
#define OMAP_UART_FCR_RX_TRIG 6
#define OMAP_UART_FCR_TX_TRIG 4
#define UART_OMAP_IER2 0x1B
#define UART_OMAP_IER2_RHR_IT_DIS BIT(2)
+/* Mode Definition Register 3 */
+#define UART_OMAP_MDR3 0x20
+#define UART_OMAP_MDR3_DIR_POL BIT(3)
+#define UART_OMAP_MDR3_DIR_EN BIT(4)
+
/* Enhanced features register 2 */
#define UART_OMAP_EFR2 0x23
#define UART_OMAP_EFR2_TIMEOUT_BEHAVE BIT(6)
int line;
u8 habit;
u8 mdr1;
+ u8 mdr3;
u8 efr;
u8 scr;
u8 wer;
__omap8250_set_mctrl(&up->port, up->port.mctrl);
- if (up->port.rs485.flags & SER_RS485_ENABLED)
+ serial_out(up, UART_OMAP_MDR3, priv->mdr3);
+
+ if (up->port.rs485.flags & SER_RS485_ENABLED &&
+ up->port.rs485_config == serial8250_em485_config)
serial8250_em485_stop_tx(up);
}
pm_runtime_put_autosuspend(port->dev);
}
+static int omap8250_rs485_config(struct uart_port *port,
+ struct ktermios *termios,
+ struct serial_rs485 *rs485)
+{
+ struct omap8250_priv *priv = port->private_data;
+ struct uart_8250_port *up = up_to_u8250p(port);
+ u32 fixed_delay_rts_before_send = 0;
+ u32 fixed_delay_rts_after_send = 0;
+ unsigned int baud;
+
+ /*
+ * There is a fixed delay of 3 bit clock cycles after the TX shift
+ * register is going empty to allow time for the stop bit to transition
+ * through the transceiver before direction is changed to receive.
+ *
+ * Additionally there appears to be a 1 bit clock delay between writing
+ * to the THR register and transmission of the start bit, per page 8783
+ * of the AM65 TRM: https://www.ti.com/lit/ug/spruid7e/spruid7e.pdf
+ */
+ if (priv->quot) {
+ if (priv->mdr1 == UART_OMAP_MDR1_16X_MODE)
+ baud = port->uartclk / (16 * priv->quot);
+ else
+ baud = port->uartclk / (13 * priv->quot);
+
+ fixed_delay_rts_after_send = 3 * MSEC_PER_SEC / baud;
+ fixed_delay_rts_before_send = 1 * MSEC_PER_SEC / baud;
+ }
+
+ /*
+ * Fall back to RS485 software emulation if the UART is missing
+ * hardware support, if the device tree specifies an mctrl_gpio
+ * (indicates that RTS is unavailable due to a pinmux conflict)
+ * or if the requested delays exceed the fixed hardware delays.
+ */
+ if (!(priv->habit & UART_HAS_NATIVE_RS485) ||
+ mctrl_gpio_to_gpiod(up->gpios, UART_GPIO_RTS) ||
+ rs485->delay_rts_after_send > fixed_delay_rts_after_send ||
+ rs485->delay_rts_before_send > fixed_delay_rts_before_send) {
+ priv->mdr3 &= ~UART_OMAP_MDR3_DIR_EN;
+ serial_out(up, UART_OMAP_MDR3, priv->mdr3);
+
+ port->rs485_config = serial8250_em485_config;
+ return serial8250_em485_config(port, termios, rs485);
+ }
+
+ rs485->delay_rts_after_send = fixed_delay_rts_after_send;
+ rs485->delay_rts_before_send = fixed_delay_rts_before_send;
+
+ if (rs485->flags & SER_RS485_ENABLED)
+ priv->mdr3 |= UART_OMAP_MDR3_DIR_EN;
+ else
+ priv->mdr3 &= ~UART_OMAP_MDR3_DIR_EN;
+
+ /*
+ * Retain same polarity semantics as RS485 software emulation,
+ * i.e. SER_RS485_RTS_ON_SEND means driving RTS low on send.
+ */
+ if (rs485->flags & SER_RS485_RTS_ON_SEND)
+ priv->mdr3 &= ~UART_OMAP_MDR3_DIR_POL;
+ else
+ priv->mdr3 |= UART_OMAP_MDR3_DIR_POL;
+
+ serial_out(up, UART_OMAP_MDR3, priv->mdr3);
+
+ return 0;
+}
+
#ifdef CONFIG_SERIAL_8250_DMA
static int omap_8250_rx_dma(struct uart_8250_port *p);
static struct omap8250_platdata am654_platdata = {
.dma_params = &am654_dma,
.habit = UART_HAS_EFR2 | UART_HAS_RHR_IT_DIS |
- UART_RX_TIMEOUT_QUIRK,
+ UART_RX_TIMEOUT_QUIRK | UART_HAS_NATIVE_RS485,
};
static struct omap8250_platdata am33xx_platdata = {
up.port.shutdown = omap_8250_shutdown;
up.port.throttle = omap_8250_throttle;
up.port.unthrottle = omap_8250_unthrottle;
- up.port.rs485_config = serial8250_em485_config;
+ up.port.rs485_config = omap8250_rs485_config;
+ /* same rs485_supported for software emulation and native RS485 */
up.port.rs485_supported = serial8250_em485_supported;
up.rs485_start_tx = serial8250_em485_start_tx;
up.rs485_stop_tx = serial8250_em485_stop_tx;
*/
serial_in(up, UART_SCR);
}
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
if (uart_circ_empty(xmit))
break;
if ((up->capabilities & UART_CAP_HFIFO) &&
config SERIAL_SIFIVE
tristate "SiFive UART support"
depends on OF
+ default SOC_SIFIVE || SOC_CANAAN
select SERIAL_CORE
help
Select this option if you are building a kernel for a device that
config SERIAL_SIFIVE_CONSOLE
bool "Console on SiFive UART"
depends on SERIAL_SIFIVE=y
+ default SOC_SIFIVE || SOC_CANAAN
select SERIAL_CORE_CONSOLE
select SERIAL_EARLYCON
help
#define ALTERA_JTAGUART_CONTROL_AC_MSK 0x00000400
#define ALTERA_JTAGUART_CONTROL_WSPACE_MSK 0xFFFF0000
-/*
- * Local per-uart structure.
- */
-struct altera_jtaguart {
- struct uart_port port;
- unsigned int sigs; /* Local copy of line sigs */
- unsigned long imr; /* Local IMR mirror */
-};
-
static unsigned int altera_jtaguart_tx_space(struct uart_port *port, u32 *ctlp)
{
u32 ctl = readl(port->membase + ALTERA_JTAGUART_CONTROL_REG);
static void altera_jtaguart_start_tx(struct uart_port *port)
{
- struct altera_jtaguart *pp =
- container_of(port, struct altera_jtaguart, port);
-
- pp->imr |= ALTERA_JTAGUART_CONTROL_WE_MSK;
- writel(pp->imr, port->membase + ALTERA_JTAGUART_CONTROL_REG);
+ port->read_status_mask |= ALTERA_JTAGUART_CONTROL_WE_MSK;
+ writel(port->read_status_mask,
+ port->membase + ALTERA_JTAGUART_CONTROL_REG);
}
static void altera_jtaguart_stop_tx(struct uart_port *port)
{
- struct altera_jtaguart *pp =
- container_of(port, struct altera_jtaguart, port);
-
- pp->imr &= ~ALTERA_JTAGUART_CONTROL_WE_MSK;
- writel(pp->imr, port->membase + ALTERA_JTAGUART_CONTROL_REG);
+ port->read_status_mask &= ~ALTERA_JTAGUART_CONTROL_WE_MSK;
+ writel(port->read_status_mask,
+ port->membase + ALTERA_JTAGUART_CONTROL_REG);
}
static void altera_jtaguart_stop_rx(struct uart_port *port)
{
- struct altera_jtaguart *pp =
- container_of(port, struct altera_jtaguart, port);
-
- pp->imr &= ~ALTERA_JTAGUART_CONTROL_RE_MSK;
- writel(pp->imr, port->membase + ALTERA_JTAGUART_CONTROL_REG);
+ port->read_status_mask &= ~ALTERA_JTAGUART_CONTROL_RE_MSK;
+ writel(port->read_status_mask,
+ port->membase + ALTERA_JTAGUART_CONTROL_REG);
}
static void altera_jtaguart_break_ctl(struct uart_port *port, int break_state)
tty_termios_copy_hw(termios, old);
}
-static void altera_jtaguart_rx_chars(struct altera_jtaguart *pp)
+static void altera_jtaguart_rx_chars(struct uart_port *port)
{
- struct uart_port *port = &pp->port;
- unsigned char ch, flag;
+ unsigned char ch;
unsigned long status;
while ((status = readl(port->membase + ALTERA_JTAGUART_DATA_REG)) &
ALTERA_JTAGUART_DATA_RVALID_MSK) {
ch = status & ALTERA_JTAGUART_DATA_DATA_MSK;
- flag = TTY_NORMAL;
port->icount.rx++;
if (uart_handle_sysrq_char(port, ch))
continue;
- uart_insert_char(port, 0, 0, ch, flag);
+ uart_insert_char(port, 0, 0, ch, TTY_NORMAL);
}
tty_flip_buffer_push(&port->state->port);
}
-static void altera_jtaguart_tx_chars(struct altera_jtaguart *pp)
+static void altera_jtaguart_tx_chars(struct uart_port *port)
{
- struct uart_port *port = &pp->port;
- struct circ_buf *xmit = &port->state->xmit;
- unsigned int pending, count;
-
- if (port->x_char) {
- /* Send special char - probably flow control */
- writel(port->x_char, port->membase + ALTERA_JTAGUART_DATA_REG);
- port->x_char = 0;
- port->icount.tx++;
- return;
- }
+ unsigned int count;
+ u8 ch;
- pending = uart_circ_chars_pending(xmit);
- if (pending > 0) {
- count = altera_jtaguart_tx_space(port, NULL);
- if (count > pending)
- count = pending;
- if (count > 0) {
- pending -= count;
- while (count--) {
- writel(xmit->buf[xmit->tail],
- port->membase + ALTERA_JTAGUART_DATA_REG);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- }
- if (pending < WAKEUP_CHARS)
- uart_write_wakeup(port);
- }
- }
+ count = altera_jtaguart_tx_space(port, NULL);
- if (pending == 0)
- altera_jtaguart_stop_tx(port);
+ uart_port_tx_limited(port, ch, count,
+ true,
+ writel(ch, port->membase + ALTERA_JTAGUART_DATA_REG),
+ ({}));
}
static irqreturn_t altera_jtaguart_interrupt(int irq, void *data)
{
struct uart_port *port = data;
- struct altera_jtaguart *pp =
- container_of(port, struct altera_jtaguart, port);
unsigned int isr;
isr = (readl(port->membase + ALTERA_JTAGUART_CONTROL_REG) >>
- ALTERA_JTAGUART_CONTROL_RI_OFF) & pp->imr;
+ ALTERA_JTAGUART_CONTROL_RI_OFF) & port->read_status_mask;
spin_lock(&port->lock);
if (isr & ALTERA_JTAGUART_CONTROL_RE_MSK)
- altera_jtaguart_rx_chars(pp);
+ altera_jtaguart_rx_chars(port);
if (isr & ALTERA_JTAGUART_CONTROL_WE_MSK)
- altera_jtaguart_tx_chars(pp);
+ altera_jtaguart_tx_chars(port);
spin_unlock(&port->lock);
static int altera_jtaguart_startup(struct uart_port *port)
{
- struct altera_jtaguart *pp =
- container_of(port, struct altera_jtaguart, port);
unsigned long flags;
int ret;
spin_lock_irqsave(&port->lock, flags);
/* Enable RX interrupts now */
- pp->imr = ALTERA_JTAGUART_CONTROL_RE_MSK;
- writel(pp->imr, port->membase + ALTERA_JTAGUART_CONTROL_REG);
+ port->read_status_mask = ALTERA_JTAGUART_CONTROL_RE_MSK;
+ writel(port->read_status_mask,
+ port->membase + ALTERA_JTAGUART_CONTROL_REG);
spin_unlock_irqrestore(&port->lock, flags);
static void altera_jtaguart_shutdown(struct uart_port *port)
{
- struct altera_jtaguart *pp =
- container_of(port, struct altera_jtaguart, port);
unsigned long flags;
spin_lock_irqsave(&port->lock, flags);
/* Disable all interrupts now */
- pp->imr = 0;
- writel(pp->imr, port->membase + ALTERA_JTAGUART_CONTROL_REG);
+ port->read_status_mask = 0;
+ writel(port->read_status_mask,
+ port->membase + ALTERA_JTAGUART_CONTROL_REG);
spin_unlock_irqrestore(&port->lock, flags);
};
#define ALTERA_JTAGUART_MAXPORTS 1
-static struct altera_jtaguart altera_jtaguart_ports[ALTERA_JTAGUART_MAXPORTS];
+static struct uart_port altera_jtaguart_ports[ALTERA_JTAGUART_MAXPORTS];
#if defined(CONFIG_SERIAL_ALTERA_JTAGUART_CONSOLE)
static void altera_jtaguart_console_write(struct console *co, const char *s,
unsigned int count)
{
- struct uart_port *port = &(altera_jtaguart_ports + co->index)->port;
+ struct uart_port *port = &altera_jtaguart_ports[co->index];
uart_console_write(port, s, count, altera_jtaguart_console_putc);
}
if (co->index < 0 || co->index >= ALTERA_JTAGUART_MAXPORTS)
return -EINVAL;
- port = &altera_jtaguart_ports[co->index].port;
+ port = &altera_jtaguart_ports[co->index];
if (port->membase == NULL)
return -ENODEV;
return 0;
if (i >= ALTERA_JTAGUART_MAXPORTS)
return -EINVAL;
- port = &altera_jtaguart_ports[i].port;
+ port = &altera_jtaguart_ports[i];
res_mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (res_mem)
if (i == -1)
i = 0;
- port = &altera_jtaguart_ports[i].port;
+ port = &altera_jtaguart_ports[i];
uart_remove_one_port(&altera_jtaguart_driver, port);
iounmap(port->membase);
static void altera_uart_tx_chars(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
-
- if (port->x_char) {
- /* Send special char - probably flow control */
- altera_uart_writel(port, port->x_char, ALTERA_UART_TXDATA_REG);
- port->x_char = 0;
- port->icount.tx++;
- return;
- }
-
- while (altera_uart_readl(port, ALTERA_UART_STATUS_REG) &
- ALTERA_UART_STATUS_TRDY_MSK) {
- if (xmit->head == xmit->tail)
- break;
- altera_uart_writel(port, xmit->buf[xmit->tail],
- ALTERA_UART_TXDATA_REG);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
+ u8 ch;
- if (uart_circ_empty(xmit))
- altera_uart_stop_tx(port);
+ uart_port_tx(port, ch,
+ altera_uart_readl(port, ALTERA_UART_STATUS_REG) &
+ ALTERA_UART_STATUS_TRDY_MSK,
+ altera_uart_writel(port, ch, ALTERA_UART_TXDATA_REG));
}
static irqreturn_t altera_uart_interrupt(int irq, void *data)
{
struct uart_port *port = data;
struct altera_uart *pp = container_of(port, struct altera_uart, port);
+ unsigned long flags;
unsigned int isr;
isr = altera_uart_readl(port, ALTERA_UART_STATUS_REG) & pp->imr;
- spin_lock(&port->lock);
+ spin_lock_irqsave(&port->lock, flags);
if (isr & ALTERA_UART_STATUS_RRDY_MSK)
altera_uart_rx_chars(port);
if (isr & ALTERA_UART_STATUS_TRDY_MSK)
altera_uart_tx_chars(port);
- spin_unlock(&port->lock);
+ spin_unlock_irqrestore(&port->lock, flags);
return IRQ_RETVAL(isr);
}
static void pl010_tx_chars(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
- int count;
+ u8 ch;
- if (port->x_char) {
- writel(port->x_char, port->membase + UART01x_DR);
- port->icount.tx++;
- port->x_char = 0;
- return;
- }
- if (uart_circ_empty(xmit) || uart_tx_stopped(port)) {
- pl010_stop_tx(port);
- return;
- }
-
- count = port->fifosize >> 1;
- do {
- writel(xmit->buf[xmit->tail], port->membase + UART01x_DR);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- } while (--count > 0);
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
-
- if (uart_circ_empty(xmit))
- pl010_stop_tx(port);
+ uart_port_tx_limited(port, ch, port->fifosize >> 1,
+ true,
+ writel(ch, port->membase + UART01x_DR),
+ ({}));
}
static void pl010_modem_status(struct uart_amba_port *uap)
* Now we know that DMA will fire, so advance the ring buffer
* with the stuff we just dispatched.
*/
- xmit->tail = (xmit->tail + count) & (UART_XMIT_SIZE - 1);
- uap->port.icount.tx += count;
+ uart_xmit_advance(&uap->port, count);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(&uap->port);
*/
static inline void pl011_dma_rx_stop(struct uart_amba_port *uap)
{
+ if (!uap->using_rx_dma)
+ return;
+
/* FIXME. Just disable the DMA enable */
uap->dmacr &= ~UART011_RXDMAE;
pl011_write(uap->dmacr, uap, REG_DMACR);
static void pl011_unthrottle_rx(struct uart_port *port)
{
struct uart_amba_port *uap = container_of(port, struct uart_amba_port, port);
+ unsigned long flags;
- pl011_enable_interrupts(uap);
+ spin_lock_irqsave(&uap->port.lock, flags);
+
+ uap->im = UART011_RTIM;
+ if (!pl011_dma_rx_running(uap))
+ uap->im |= UART011_RXIM;
+
+ pl011_write(uap->im, uap, REG_IMSC);
+
+ spin_unlock_irqrestore(&uap->port.lock, flags);
}
static int pl011_startup(struct uart_port *port)
static void apbuart_tx_chars(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
- int count;
-
- if (port->x_char) {
- UART_PUT_CHAR(port, port->x_char);
- port->icount.tx++;
- port->x_char = 0;
- return;
- }
-
- if (uart_circ_empty(xmit) || uart_tx_stopped(port)) {
- apbuart_stop_tx(port);
- return;
- }
-
- /* amba: fill FIFO */
- count = port->fifosize >> 1;
- do {
- UART_PUT_CHAR(port, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- } while (--count > 0);
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
+ u8 ch;
- if (uart_circ_empty(xmit))
- apbuart_stop_tx(port);
+ uart_port_tx_limited(port, ch, port->fifosize >> 1,
+ true,
+ UART_PUT_CHAR(port, ch),
+ ({}));
}
static irqreturn_t apbuart_int(int irq, void *dev_id)
ar933x_uart_putc(up, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- up->port.icount.tx++;
+ uart_xmit_advance(&up->port, 1);
} while (--count > 0);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
sent = 1;
} else if (!uart_circ_empty(xmit)) {
ch = xmit->buf[xmit->tail];
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
while (!(UART_GET_STATUS(port) & TXEMPTY))
cpu_relax();
UART_SET_DATA(port, ch);
static void atmel_stop_tx(struct uart_port *port)
{
struct atmel_uart_port *atmel_port = to_atmel_uart_port(port);
+ bool is_pdc = atmel_use_pdc_tx(port);
+ bool is_dma = is_pdc || atmel_use_dma_tx(port);
- if (atmel_use_pdc_tx(port)) {
+ if (is_pdc) {
/* disable PDC transmit */
atmel_uart_writel(port, ATMEL_PDC_PTCR, ATMEL_PDC_TXTDIS);
}
- /*
- * Disable the transmitter.
- * This is mandatory when DMA is used, otherwise the DMA buffer
- * is fully transmitted.
- */
- atmel_uart_writel(port, ATMEL_US_CR, ATMEL_US_TXDIS);
- atmel_port->tx_stopped = true;
+ if (is_dma) {
+ /*
+ * Disable the transmitter.
+ * This is mandatory when DMA is used, otherwise the DMA buffer
+ * is fully transmitted.
+ */
+ atmel_uart_writel(port, ATMEL_US_CR, ATMEL_US_TXDIS);
+ atmel_port->tx_stopped = true;
+ }
/* Disable interrupts */
atmel_uart_writel(port, ATMEL_US_IDR, atmel_port->tx_done_mask);
if (atmel_uart_is_half_duplex(port))
if (!atomic_read(&atmel_port->tasklet_shutdown))
atmel_start_rx(port);
-
}
/*
static void atmel_start_tx(struct uart_port *port)
{
struct atmel_uart_port *atmel_port = to_atmel_uart_port(port);
+ bool is_pdc = atmel_use_pdc_tx(port);
+ bool is_dma = is_pdc || atmel_use_dma_tx(port);
- if (atmel_use_pdc_tx(port) && (atmel_uart_readl(port, ATMEL_PDC_PTSR)
+ if (is_pdc && (atmel_uart_readl(port, ATMEL_PDC_PTSR)
& ATMEL_PDC_TXTEN))
/* The transmitter is already running. Yes, we
really need this.*/
return;
- if (atmel_use_pdc_tx(port) || atmel_use_dma_tx(port))
- if (atmel_uart_is_half_duplex(port))
- atmel_stop_rx(port);
+ if (is_dma && atmel_uart_is_half_duplex(port))
+ atmel_stop_rx(port);
- if (atmel_use_pdc_tx(port))
+ if (is_pdc) {
/* re-enable PDC transmit */
atmel_uart_writel(port, ATMEL_PDC_PTCR, ATMEL_PDC_TXTEN);
+ }
/* Enable interrupts */
atmel_uart_writel(port, ATMEL_US_IER, atmel_port->tx_done_mask);
- /* re-enable the transmitter */
- atmel_uart_writel(port, ATMEL_US_CR, ATMEL_US_TXEN);
- atmel_port->tx_stopped = false;
+ if (is_dma) {
+ /* re-enable the transmitter */
+ atmel_uart_writel(port, ATMEL_US_CR, ATMEL_US_TXEN);
+ atmel_port->tx_stopped = false;
+ }
}
/*
*/
static void atmel_tx_chars(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
struct atmel_uart_port *atmel_port = to_atmel_uart_port(port);
+ bool pending;
+ u8 ch;
- if (port->x_char &&
- (atmel_uart_readl(port, ATMEL_US_CSR) & ATMEL_US_TXRDY)) {
- atmel_uart_write_char(port, port->x_char);
- port->icount.tx++;
- port->x_char = 0;
- }
- if (uart_circ_empty(xmit) || uart_tx_stopped(port))
- return;
-
- while (atmel_uart_readl(port, ATMEL_US_CSR) & ATMEL_US_TXRDY) {
- atmel_uart_write_char(port, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
-
- if (!uart_circ_empty(xmit)) {
+ pending = uart_port_tx(port, ch,
+ atmel_uart_readl(port, ATMEL_US_CSR) & ATMEL_US_TXRDY,
+ atmel_uart_write_char(port, ch));
+ if (pending) {
/* we still have characters to transmit, so we should continue
* transmitting them when TX is ready, regardless of
* mode or duplexity
if (chan)
dmaengine_terminate_all(chan);
- xmit->tail += atmel_port->tx_len;
- xmit->tail &= UART_XMIT_SIZE - 1;
-
- port->icount.tx += atmel_port->tx_len;
+ uart_xmit_advance(port, atmel_port->tx_len);
spin_lock_irq(&atmel_port->lock_tx);
async_tx_ack(atmel_port->desc_tx);
/* nothing left to transmit? */
if (atmel_uart_readl(port, ATMEL_PDC_TCR))
return;
-
- xmit->tail += pdc->ofs;
- xmit->tail &= UART_XMIT_SIZE - 1;
-
- port->icount.tx += pdc->ofs;
+ uart_xmit_advance(port, pdc->ofs);
pdc->ofs = 0;
/* more to transmit - setup next transfer */
*/
static void bcm_uart_do_tx(struct uart_port *port)
{
- struct circ_buf *xmit;
- unsigned int val, max_count;
-
- if (port->x_char) {
- bcm_uart_writel(port, port->x_char, UART_FIFO_REG);
- port->icount.tx++;
- port->x_char = 0;
- return;
- }
-
- if (uart_tx_stopped(port)) {
- bcm_uart_stop_tx(port);
- return;
- }
-
- xmit = &port->state->xmit;
- if (uart_circ_empty(xmit))
- goto txq_empty;
+ unsigned int val;
+ bool pending;
+ u8 ch;
val = bcm_uart_readl(port, UART_MCTL_REG);
val = (val & UART_MCTL_TXFIFOFILL_MASK) >> UART_MCTL_TXFIFOFILL_SHIFT;
- max_count = port->fifosize - val;
-
- while (max_count--) {
- unsigned int c;
- c = xmit->buf[xmit->tail];
- bcm_uart_writel(port, c, UART_FIFO_REG);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
-
- if (uart_circ_empty(xmit))
- goto txq_empty;
- return;
+ pending = uart_port_tx_limited(port, ch, port->fifosize - val,
+ true,
+ bcm_uart_writel(port, ch, UART_FIFO_REG),
+ ({}));
+ if (pending)
+ return;
-txq_empty:
/* nothing to send, disable transmit interrupt */
val = bcm_uart_readl(port, UART_IR_REG);
val &= ~UART_TX_INT_MASK;
bcm_uart_writel(port, val, UART_IR_REG);
- return;
}
/*
u32 sysflg = 0;
writew(xmit->buf[xmit->tail], port->membase + UARTDR_OFFSET);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
regmap_read(s->syscon, SYSFLG_OFFSET, &sysflg);
if (sysflg & SYSFLG_UTXFF)
p = cpm2cpu_addr(in_be32(&bdp->cbd_bufaddr), pinfo);
while (count < pinfo->tx_fifosize) {
*p++ = xmit->buf[xmit->tail];
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
count++;
if (xmit->head == xmit->tail)
break;
while (!uart_circ_empty(xmit)) {
writeb(xmit->buf[xmit->tail], port->membase + UA_EMI_REC);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
if (digicolor_uart_tx_full(port))
break;
* so we go one char at a time) :-<
*/
tmp = xmit->buf[xmit->tail];
- xmit->tail = (xmit->tail + 1) & (DZ_XMIT_SIZE - 1);
dz_out(dport, DZ_TDR, tmp);
- dport->port.icount.tx++;
+ uart_xmit_advance(&dport->port, 1);
if (uart_circ_chars_pending(xmit) < DZ_WAKEUP_CHARS)
uart_write_wakeup(&dport->port);
while (!uart_circ_empty(xmit)) {
linflex_put_char(sport, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- sport->icount.tx++;
+ uart_xmit_advance(sport, 1);
}
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/of_dma.h>
+#include <linux/pinctrl/consumer.h>
+#include <linux/pm_runtime.h>
#include <linux/serial_core.h>
#include <linux/slab.h>
#include <linux/tty_flip.h>
/* Rx DMA timeout in ms, which is used to calculate Rx ring buffer size */
#define DMA_RX_TIMEOUT (10)
+#define UART_AUTOSUSPEND_TIMEOUT 3000
#define DRIVER_NAME "fsl-lpuart"
#define DEV_NAME "ttyLP"
dma_unmap_sg(chan->device->dev, sgl, sport->dma_tx_nents,
DMA_TO_DEVICE);
- xmit->tail = (xmit->tail + sport->dma_tx_bytes) & (UART_XMIT_SIZE - 1);
-
- sport->port.icount.tx += sport->dma_tx_bytes;
+ uart_xmit_advance(&sport->port, sport->dma_tx_bytes);
sport->dma_tx_in_progress = false;
spin_unlock_irqrestore(&sport->port.lock, flags);
sport->dma_tx_nents, DMA_TO_DEVICE);
sport->dma_tx_in_progress = false;
}
- dmaengine_terminate_all(chan);
+ dmaengine_terminate_async(chan);
}
if (lpuart_is_32(sport)) {
static inline void lpuart_transmit_buffer(struct lpuart_port *sport)
{
- struct circ_buf *xmit = &sport->port.state->xmit;
-
- if (sport->port.x_char) {
- writeb(sport->port.x_char, sport->port.membase + UARTDR);
- sport->port.icount.tx++;
- sport->port.x_char = 0;
- return;
- }
-
- if (lpuart_stopped_or_empty(&sport->port)) {
- lpuart_stop_tx(&sport->port);
- return;
- }
-
- while (!uart_circ_empty(xmit) &&
- (readb(sport->port.membase + UARTTCFIFO) < sport->txfifo_size)) {
- writeb(xmit->buf[xmit->tail], sport->port.membase + UARTDR);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- sport->port.icount.tx++;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(&sport->port);
+ struct uart_port *port = &sport->port;
+ u8 ch;
- if (uart_circ_empty(xmit))
- lpuart_stop_tx(&sport->port);
+ uart_port_tx(port, ch,
+ readb(port->membase + UARTTCFIFO) < sport->txfifo_size,
+ writeb(ch, port->membase + UARTDR));
}
static inline void lpuart32_transmit_buffer(struct lpuart_port *sport)
txcnt &= UARTWATER_COUNT_MASK;
while (!uart_circ_empty(xmit) && (txcnt < sport->txfifo_size)) {
lpuart32_write(&sport->port, xmit->buf[xmit->tail], UARTDATA);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- sport->port.icount.tx++;
+ uart_xmit_advance(&sport->port, 1);
txcnt = lpuart32_read(&sport->port, UARTWATER);
txcnt = txcnt >> UARTWATER_TXCNT_OFF;
txcnt &= UARTWATER_COUNT_MASK;
}
}
+static void
+lpuart_uart_pm(struct uart_port *port, unsigned int state, unsigned int oldstate)
+{
+ switch (state) {
+ case UART_PM_STATE_OFF:
+ pm_runtime_mark_last_busy(port->dev);
+ pm_runtime_put_autosuspend(port->dev);
+ break;
+ default:
+ pm_runtime_get_sync(port->dev);
+ break;
+ }
+}
+
/* return TIOCSER_TEMT when transmitter is not busy */
static unsigned int lpuart_tx_empty(struct uart_port *port)
{
struct lpuart_port, port);
struct dma_chan *chan = sport->dma_rx_chan;
- dmaengine_terminate_all(chan);
+ dmaengine_terminate_sync(chan);
dma_unmap_sg(chan->device->dev, &sport->rx_sgl, 1, DMA_FROM_DEVICE);
kfree(sport->rx_ring.buf);
sport->rx_ring.tail = 0;
sport->lpuart_dma_rx_use = false;
}
+static void lpuart_hw_setup(struct lpuart_port *sport)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&sport->port.lock, flags);
+
+ lpuart_setup_watermark_enable(sport);
+
+ lpuart_rx_dma_startup(sport);
+ lpuart_tx_dma_startup(sport);
+
+ spin_unlock_irqrestore(&sport->port.lock, flags);
+}
+
static int lpuart_startup(struct uart_port *port)
{
struct lpuart_port *sport = container_of(port, struct lpuart_port, port);
- unsigned long flags;
unsigned char temp;
/* determine FIFO size and enable FIFO mode */
UARTPFIFO_FIFOSIZE_MASK);
lpuart_request_dma(sport);
-
- spin_lock_irqsave(&sport->port.lock, flags);
-
- lpuart_setup_watermark_enable(sport);
-
- lpuart_rx_dma_startup(sport);
- lpuart_tx_dma_startup(sport);
-
- spin_unlock_irqrestore(&sport->port.lock, flags);
+ lpuart_hw_setup(sport);
return 0;
}
lpuart32_write(&sport->port, temp, UARTCTRL);
}
+static void lpuart32_hw_setup(struct lpuart_port *sport)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&sport->port.lock, flags);
+
+ lpuart32_setup_watermark_enable(sport);
+
+ lpuart_rx_dma_startup(sport);
+ lpuart_tx_dma_startup(sport);
+
+ lpuart32_configure(sport);
+
+ spin_unlock_irqrestore(&sport->port.lock, flags);
+}
+
static int lpuart32_startup(struct uart_port *port)
{
struct lpuart_port *sport = container_of(port, struct lpuart_port, port);
- unsigned long flags;
unsigned long temp;
/* determine FIFO size */
}
lpuart_request_dma(sport);
+ lpuart32_hw_setup(sport);
- spin_lock_irqsave(&sport->port.lock, flags);
-
- lpuart32_setup_watermark_enable(sport);
-
- lpuart_rx_dma_startup(sport);
- lpuart_tx_dma_startup(sport);
-
- lpuart32_configure(sport);
-
- spin_unlock_irqrestore(&sport->port.lock, flags);
return 0;
}
if (wait_event_interruptible_timeout(sport->dma_wait,
!sport->dma_tx_in_progress, msecs_to_jiffies(300)) <= 0) {
sport->dma_tx_in_progress = false;
- dmaengine_terminate_all(sport->dma_tx_chan);
+ dmaengine_terminate_sync(sport->dma_tx_chan);
}
sport->lpuart_dma_tx_use = false;
}
.startup = lpuart_startup,
.shutdown = lpuart_shutdown,
.set_termios = lpuart_set_termios,
+ .pm = lpuart_uart_pm,
.type = lpuart_type,
.request_port = lpuart_request_port,
.release_port = lpuart_release_port,
.startup = lpuart32_startup,
.shutdown = lpuart32_shutdown,
.set_termios = lpuart32_set_termios,
+ .pm = lpuart_uart_pm,
.type = lpuart_type,
.request_port = lpuart_request_port,
.release_port = lpuart_release_port,
handler = lpuart_int;
}
+ pm_runtime_use_autosuspend(&pdev->dev);
+ pm_runtime_set_autosuspend_delay(&pdev->dev, UART_AUTOSUSPEND_TIMEOUT);
+ pm_runtime_set_active(&pdev->dev);
+ pm_runtime_enable(&pdev->dev);
+
ret = lpuart_global_reset(sport);
if (ret)
goto failed_reset;
failed_attach_port:
failed_get_rs485:
failed_reset:
+ pm_runtime_disable(&pdev->dev);
+ pm_runtime_set_suspended(&pdev->dev);
+ pm_runtime_dont_use_autosuspend(&pdev->dev);
lpuart_disable_clks(sport);
return ret;
}
if (sport->dma_rx_chan)
dma_release_channel(sport->dma_rx_chan);
+ pm_runtime_disable(&pdev->dev);
+ pm_runtime_set_suspended(&pdev->dev);
+ pm_runtime_dont_use_autosuspend(&pdev->dev);
return 0;
}
-static int __maybe_unused lpuart_suspend(struct device *dev)
+static int lpuart_runtime_suspend(struct device *dev)
{
- struct lpuart_port *sport = dev_get_drvdata(dev);
- unsigned long temp;
- bool irq_wake;
+ struct platform_device *pdev = to_platform_device(dev);
+ struct lpuart_port *sport = platform_get_drvdata(pdev);
- if (lpuart_is_32(sport)) {
- /* disable Rx/Tx and interrupts */
- temp = lpuart32_read(&sport->port, UARTCTRL);
- temp &= ~(UARTCTRL_TE | UARTCTRL_TIE | UARTCTRL_TCIE);
- lpuart32_write(&sport->port, temp, UARTCTRL);
- } else {
- /* disable Rx/Tx and interrupts */
- temp = readb(sport->port.membase + UARTCR2);
- temp &= ~(UARTCR2_TE | UARTCR2_TIE | UARTCR2_TCIE);
- writeb(temp, sport->port.membase + UARTCR2);
- }
+ lpuart_disable_clks(sport);
- uart_suspend_port(&lpuart_reg, &sport->port);
+ return 0;
+};
- /* uart_suspend_port() might set wakeup flag */
- irq_wake = irqd_is_wakeup_set(irq_get_irq_data(sport->port.irq));
+static int lpuart_runtime_resume(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct lpuart_port *sport = platform_get_drvdata(pdev);
- if (sport->lpuart_dma_rx_use) {
- /*
- * EDMA driver during suspend will forcefully release any
- * non-idle DMA channels. If port wakeup is enabled or if port
- * is console port or 'no_console_suspend' is set the Rx DMA
- * cannot resume as expected, hence gracefully release the
- * Rx DMA path before suspend and start Rx DMA path on resume.
- */
- if (irq_wake) {
- del_timer_sync(&sport->lpuart_timer);
- lpuart_dma_rx_free(&sport->port);
- }
+ return lpuart_enable_clks(sport);
+};
- /* Disable Rx DMA to use UART port as wakeup source */
- if (lpuart_is_32(sport)) {
- temp = lpuart32_read(&sport->port, UARTBAUD);
- lpuart32_write(&sport->port, temp & ~UARTBAUD_RDMAE,
- UARTBAUD);
+static void serial_lpuart_enable_wakeup(struct lpuart_port *sport, bool on)
+{
+ unsigned int val, baud;
+
+ if (lpuart_is_32(sport)) {
+ val = lpuart32_read(&sport->port, UARTCTRL);
+ baud = lpuart32_read(&sport->port, UARTBAUD);
+ if (on) {
+ /* set rx_watermark to 0 in wakeup source mode */
+ lpuart32_write(&sport->port, 0, UARTWATER);
+ val |= UARTCTRL_RIE;
+ /* clear RXEDGIF flag before enable RXEDGIE interrupt */
+ lpuart32_write(&sport->port, UARTSTAT_RXEDGIF, UARTSTAT);
+ baud |= UARTBAUD_RXEDGIE;
} else {
- writeb(readb(sport->port.membase + UARTCR5) &
- ~UARTCR5_RDMAS, sport->port.membase + UARTCR5);
+ val &= ~UARTCTRL_RIE;
+ baud &= ~UARTBAUD_RXEDGIE;
}
+ lpuart32_write(&sport->port, val, UARTCTRL);
+ lpuart32_write(&sport->port, baud, UARTBAUD);
+ } else {
+ val = readb(sport->port.membase + UARTCR2);
+ if (on)
+ val |= UARTCR2_RIE;
+ else
+ val &= ~UARTCR2_RIE;
+ writeb(val, sport->port.membase + UARTCR2);
}
+}
- if (sport->lpuart_dma_tx_use) {
- sport->dma_tx_in_progress = false;
- dmaengine_terminate_all(sport->dma_tx_chan);
+static bool lpuart_uport_is_active(struct lpuart_port *sport)
+{
+ struct tty_port *port = &sport->port.state->port;
+ struct tty_struct *tty;
+ struct device *tty_dev;
+ int may_wake = 0;
+
+ tty = tty_port_tty_get(port);
+ if (tty) {
+ tty_dev = tty->dev;
+ may_wake = device_may_wakeup(tty_dev);
+ tty_kref_put(tty);
}
- if (sport->port.suspended && !irq_wake)
- lpuart_disable_clks(sport);
+ if ((tty_port_initialized(port) && may_wake) ||
+ (!console_suspend_enabled && uart_console(&sport->port)))
+ return true;
- return 0;
+ return false;
}
-static int __maybe_unused lpuart_resume(struct device *dev)
+static int lpuart_suspend_noirq(struct device *dev)
{
struct lpuart_port *sport = dev_get_drvdata(dev);
bool irq_wake = irqd_is_wakeup_set(irq_get_irq_data(sport->port.irq));
- if (sport->port.suspended && !irq_wake)
- lpuart_enable_clks(sport);
+ if (lpuart_uport_is_active(sport))
+ serial_lpuart_enable_wakeup(sport, !!irq_wake);
- if (lpuart_is_32(sport))
- lpuart32_setup_watermark_enable(sport);
- else
- lpuart_setup_watermark_enable(sport);
+ pinctrl_pm_select_sleep_state(dev);
- if (sport->lpuart_dma_rx_use) {
- if (irq_wake) {
- if (!lpuart_start_rx_dma(sport))
- rx_dma_timer_init(sport);
- else
- sport->lpuart_dma_rx_use = false;
+ return 0;
+}
+
+static int lpuart_resume_noirq(struct device *dev)
+{
+ struct lpuart_port *sport = dev_get_drvdata(dev);
+ unsigned int val;
+
+ pinctrl_pm_select_default_state(dev);
+
+ if (lpuart_uport_is_active(sport)) {
+ serial_lpuart_enable_wakeup(sport, false);
+
+ /* clear the wakeup flags */
+ if (lpuart_is_32(sport)) {
+ val = lpuart32_read(&sport->port, UARTSTAT);
+ lpuart32_write(&sport->port, val, UARTSTAT);
}
}
- lpuart_tx_dma_startup(sport);
+ return 0;
+}
- if (lpuart_is_32(sport))
- lpuart32_configure(sport);
+static int lpuart_suspend(struct device *dev)
+{
+ struct lpuart_port *sport = dev_get_drvdata(dev);
+ unsigned long temp, flags;
+ uart_suspend_port(&lpuart_reg, &sport->port);
+
+ if (lpuart_uport_is_active(sport)) {
+ spin_lock_irqsave(&sport->port.lock, flags);
+ if (lpuart_is_32(sport)) {
+ /* disable Rx/Tx and interrupts */
+ temp = lpuart32_read(&sport->port, UARTCTRL);
+ temp &= ~(UARTCTRL_TE | UARTCTRL_TIE | UARTCTRL_TCIE);
+ lpuart32_write(&sport->port, temp, UARTCTRL);
+ } else {
+ /* disable Rx/Tx and interrupts */
+ temp = readb(sport->port.membase + UARTCR2);
+ temp &= ~(UARTCR2_TE | UARTCR2_TIE | UARTCR2_TCIE);
+ writeb(temp, sport->port.membase + UARTCR2);
+ }
+ spin_unlock_irqrestore(&sport->port.lock, flags);
+
+ if (sport->lpuart_dma_rx_use) {
+ /*
+ * EDMA driver during suspend will forcefully release any
+ * non-idle DMA channels. If port wakeup is enabled or if port
+ * is console port or 'no_console_suspend' is set the Rx DMA
+ * cannot resume as expected, hence gracefully release the
+ * Rx DMA path before suspend and start Rx DMA path on resume.
+ */
+ del_timer_sync(&sport->lpuart_timer);
+ lpuart_dma_rx_free(&sport->port);
+
+ /* Disable Rx DMA to use UART port as wakeup source */
+ spin_lock_irqsave(&sport->port.lock, flags);
+ if (lpuart_is_32(sport)) {
+ temp = lpuart32_read(&sport->port, UARTBAUD);
+ lpuart32_write(&sport->port, temp & ~UARTBAUD_RDMAE,
+ UARTBAUD);
+ } else {
+ writeb(readb(sport->port.membase + UARTCR5) &
+ ~UARTCR5_RDMAS, sport->port.membase + UARTCR5);
+ }
+ spin_unlock_irqrestore(&sport->port.lock, flags);
+ }
+
+ if (sport->lpuart_dma_tx_use) {
+ spin_lock_irqsave(&sport->port.lock, flags);
+ if (lpuart_is_32(sport)) {
+ temp = lpuart32_read(&sport->port, UARTBAUD);
+ temp &= ~UARTBAUD_TDMAE;
+ lpuart32_write(&sport->port, temp, UARTBAUD);
+ } else {
+ temp = readb(sport->port.membase + UARTCR5);
+ temp &= ~UARTCR5_TDMAS;
+ writeb(temp, sport->port.membase + UARTCR5);
+ }
+ spin_unlock_irqrestore(&sport->port.lock, flags);
+ sport->dma_tx_in_progress = false;
+ dmaengine_terminate_sync(sport->dma_tx_chan);
+ }
+ } else if (pm_runtime_active(sport->port.dev)) {
+ lpuart_disable_clks(sport);
+ pm_runtime_disable(sport->port.dev);
+ pm_runtime_set_suspended(sport->port.dev);
+ }
+
+ return 0;
+}
+
+static void lpuart_console_fixup(struct lpuart_port *sport)
+{
+ struct tty_port *port = &sport->port.state->port;
+ struct uart_port *uport = &sport->port;
+ struct ktermios termios;
+
+ /* i.MX7ULP enter VLLS mode that lpuart module power off and registers
+ * all lost no matter the port is wakeup source.
+ * For console port, console baud rate setting lost and print messy
+ * log when enable the console port as wakeup source. To avoid the
+ * issue happen, user should not enable uart port as wakeup source
+ * in VLLS mode, or restore console setting here.
+ */
+ if (is_imx7ulp_lpuart(sport) && lpuart_uport_is_active(sport) &&
+ console_suspend_enabled && uart_console(&sport->port)) {
+
+ mutex_lock(&port->mutex);
+ memset(&termios, 0, sizeof(struct ktermios));
+ termios.c_cflag = uport->cons->cflag;
+ if (port->tty && termios.c_cflag == 0)
+ termios = port->tty->termios;
+ uport->ops->set_termios(uport, &termios, NULL);
+ mutex_unlock(&port->mutex);
+ }
+}
+
+static int lpuart_resume(struct device *dev)
+{
+ struct lpuart_port *sport = dev_get_drvdata(dev);
+ int ret;
+
+ if (lpuart_uport_is_active(sport)) {
+ if (lpuart_is_32(sport))
+ lpuart32_hw_setup(sport);
+ else
+ lpuart_hw_setup(sport);
+ } else if (pm_runtime_active(sport->port.dev)) {
+ ret = lpuart_enable_clks(sport);
+ if (ret)
+ return ret;
+ pm_runtime_set_active(sport->port.dev);
+ pm_runtime_enable(sport->port.dev);
+ }
+
+ lpuart_console_fixup(sport);
uart_resume_port(&lpuart_reg, &sport->port);
return 0;
}
-static SIMPLE_DEV_PM_OPS(lpuart_pm_ops, lpuart_suspend, lpuart_resume);
+static const struct dev_pm_ops lpuart_pm_ops = {
+ RUNTIME_PM_OPS(lpuart_runtime_suspend,
+ lpuart_runtime_resume, NULL)
+ NOIRQ_SYSTEM_SLEEP_PM_OPS(lpuart_suspend_noirq,
+ lpuart_resume_noirq)
+ SYSTEM_SLEEP_PM_OPS(lpuart_suspend, lpuart_resume)
+};
static struct platform_driver lpuart_driver = {
.probe = lpuart_probe,
.driver = {
.name = "fsl-lpuart",
.of_match_table = lpuart_dt_ids,
- .pm = &lpuart_pm_ops,
+ .pm = pm_ptr(&lpuart_pm_ops),
},
};
static void imx_uart_stop_rx(struct uart_port *port)
{
struct imx_port *sport = (struct imx_port *)port;
- u32 ucr1, ucr2, ucr4;
+ u32 ucr1, ucr2, ucr4, uts;
ucr1 = imx_uart_readl(sport, UCR1);
ucr2 = imx_uart_readl(sport, UCR2);
imx_uart_writel(sport, ucr1, UCR1);
imx_uart_writel(sport, ucr4, UCR4);
- ucr2 &= ~UCR2_RXEN;
+ /* See SER_RS485_ENABLED/UTS_LOOP comment in imx_uart_probe() */
+ if (port->rs485.flags & SER_RS485_ENABLED &&
+ port->rs485.flags & SER_RS485_RTS_ON_SEND &&
+ sport->have_rtscts && !sport->have_rtsgpio) {
+ uts = imx_uart_readl(sport, imx_uart_uts_reg(sport));
+ uts |= UTS_LOOP;
+ imx_uart_writel(sport, uts, imx_uart_uts_reg(sport));
+ ucr2 |= UCR2_RXEN;
+ } else {
+ ucr2 &= ~UCR2_RXEN;
+ }
+
imx_uart_writel(sport, ucr2, UCR2);
}
/* send xmit->buf[xmit->tail]
* out the port here */
imx_uart_writel(sport, xmit->buf[xmit->tail], URTX0);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- sport->port.icount.tx++;
+ uart_xmit_advance(&sport->port, 1);
}
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
ucr1 &= ~UCR1_TXDMAEN;
imx_uart_writel(sport, ucr1, UCR1);
- /* update the stat */
- xmit->tail = (xmit->tail + sport->tx_bytes) & (UART_XMIT_SIZE - 1);
- sport->port.icount.tx += sport->tx_bytes;
+ uart_xmit_advance(&sport->port, sport->tx_bytes);
dev_dbg(sport->port.dev, "we finish the TX DMA.\n");
int retval, i;
unsigned long flags;
int dma_is_inited = 0;
- u32 ucr1, ucr2, ucr3, ucr4;
+ u32 ucr1, ucr2, ucr3, ucr4, uts;
retval = clk_prepare_enable(sport->clk_per);
if (retval)
imx_uart_writel(sport, ucr2, UCR2);
}
+ /* See SER_RS485_ENABLED/UTS_LOOP comment in imx_uart_probe() */
+ uts = imx_uart_readl(sport, imx_uart_uts_reg(sport));
+ uts &= ~UTS_LOOP;
+ imx_uart_writel(sport, uts, imx_uart_uts_reg(sport));
+
spin_unlock_irqrestore(&sport->port.lock, flags);
return 0;
{
struct imx_port *sport = (struct imx_port *)port;
unsigned long flags;
- u32 ucr1, ucr2, ucr4;
+ u32 ucr1, ucr2, ucr4, uts;
if (sport->dma_is_enabled) {
dmaengine_terminate_sync(sport->dma_chan_tx);
spin_lock_irqsave(&sport->port.lock, flags);
ucr1 = imx_uart_readl(sport, UCR1);
- ucr1 &= ~(UCR1_TRDYEN | UCR1_RRDYEN | UCR1_RTSDEN | UCR1_UARTEN | UCR1_RXDMAEN | UCR1_ATDMAEN);
+ ucr1 &= ~(UCR1_TRDYEN | UCR1_RRDYEN | UCR1_RTSDEN | UCR1_RXDMAEN | UCR1_ATDMAEN);
+ /* See SER_RS485_ENABLED/UTS_LOOP comment in imx_uart_probe() */
+ if (port->rs485.flags & SER_RS485_ENABLED &&
+ port->rs485.flags & SER_RS485_RTS_ON_SEND &&
+ sport->have_rtscts && !sport->have_rtsgpio) {
+ uts = imx_uart_readl(sport, imx_uart_uts_reg(sport));
+ uts |= UTS_LOOP;
+ imx_uart_writel(sport, uts, imx_uart_uts_reg(sport));
+ ucr1 |= UCR1_UARTEN;
+ } else {
+ ucr1 &= ~UCR1_UARTEN;
+ }
imx_uart_writel(sport, ucr1, UCR1);
ucr4 = imx_uart_readl(sport, UCR4);
void __iomem *base;
u32 dma_buf_conf[2];
int ret = 0;
- u32 ucr1;
+ u32 ucr1, ucr2, uts;
struct resource *res;
int txirq, rxirq, rtsirq;
ucr1 &= ~(UCR1_ADEN | UCR1_TRDYEN | UCR1_IDEN | UCR1_RRDYEN | UCR1_RTSDEN);
imx_uart_writel(sport, ucr1, UCR1);
+ /*
+ * In case RS485 is enabled without GPIO RTS control, the UART IP
+ * is used to control CTS signal. Keep both the UART and Receiver
+ * enabled, otherwise the UART IP pulls CTS signal always HIGH no
+ * matter how the UCR2 CTSC and CTS bits are set. To prevent any
+ * data from being fed into the RX FIFO, enable loopback mode in
+ * UTS register, which disconnects the RX path from external RXD
+ * pin and connects it to the Transceiver, which is disabled, so
+ * no data can be fed to the RX FIFO that way.
+ */
+ if (sport->port.rs485.flags & SER_RS485_ENABLED &&
+ sport->have_rtscts && !sport->have_rtsgpio) {
+ uts = imx_uart_readl(sport, imx_uart_uts_reg(sport));
+ uts |= UTS_LOOP;
+ imx_uart_writel(sport, uts, imx_uart_uts_reg(sport));
+
+ ucr1 = imx_uart_readl(sport, UCR1);
+ ucr1 |= UCR1_UARTEN;
+ imx_uart_writel(sport, ucr1, UCR1);
+
+ ucr2 = imx_uart_readl(sport, UCR2);
+ ucr2 |= UCR2_RXEN;
+ imx_uart_writel(sport, ucr2, UCR2);
+ }
+
if (!imx_uart_is_imx1(sport) && sport->dte_mode) {
/*
* The DCEDTE bit changes the direction of DSR, DCD, DTR and RI
ZSDELAY();
ZS_WSYNC(channel);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- up->port.icount.tx++;
+ uart_xmit_advance(&up->port, 1);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(&up->port);
ZSDELAY();
ZS_WSYNC(channel);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(&up->port);
#define ASCFSTAT_TXFFLMASK 0x3F00
#define ASCFSTAT_TXFREEMASK 0x3F000000
-static void lqasc_tx_chars(struct uart_port *port);
static struct ltq_uart_port *lqasc_port[MAXPORTS];
static struct uart_driver lqasc_reg;
{
unsigned long flags;
struct ltq_uart_port *ltq_port = to_ltq_uart_port(port);
+ u8 ch;
spin_lock_irqsave(<q_port->lock, flags);
- lqasc_tx_chars(port);
+ uart_port_tx(port, ch,
+ lqasc_tx_ready(port),
+ writeb(ch, port->membase + LTQ_ASC_TBUF));
spin_unlock_irqrestore(<q_port->lock, flags);
return;
}
return 0;
}
-static void
-lqasc_tx_chars(struct uart_port *port)
-{
- struct circ_buf *xmit = &port->state->xmit;
- if (uart_tx_stopped(port)) {
- lqasc_stop_tx(port);
- return;
- }
-
- while (lqasc_tx_ready(port)) {
- if (port->x_char) {
- writeb(port->x_char, port->membase + LTQ_ASC_TBUF);
- port->icount.tx++;
- port->x_char = 0;
- continue;
- }
-
- if (uart_circ_empty(xmit))
- break;
-
- writeb(port->state->xmit.buf[port->state->xmit.tail],
- port->membase + LTQ_ASC_TBUF);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
-}
-
static irqreturn_t
lqasc_tx_int(int irq, void *_port)
{
} else if (!uart_circ_empty(xmit)) {
while (xmit->head != xmit->tail) {
ch = xmit->buf[xmit->tail];
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
liteuart_putchar(port, ch);
}
}
tty_flip_buffer_push(tport);
}
-static void serial_lpc32xx_stop_tx(struct uart_port *port);
-
static bool serial_lpc32xx_tx_ready(struct uart_port *port)
{
u32 level = readl(LPC32XX_HSUART_LEVEL(port->membase));
static void __serial_lpc32xx_tx(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
-
- if (port->x_char) {
- writel((u32)port->x_char, LPC32XX_HSUART_FIFO(port->membase));
- port->icount.tx++;
- port->x_char = 0;
- return;
- }
-
- if (uart_circ_empty(xmit) || uart_tx_stopped(port))
- goto exit_tx;
-
- /* Transfer data */
- while (serial_lpc32xx_tx_ready(port)) {
- writel((u32) xmit->buf[xmit->tail],
- LPC32XX_HSUART_FIFO(port->membase));
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
+ u8 ch;
-exit_tx:
- if (uart_circ_empty(xmit))
- serial_lpc32xx_stop_tx(port);
+ uart_port_tx(port, ch,
+ serial_lpc32xx_tx_ready(port),
+ writel(ch, LPC32XX_HSUART_FIFO(port->membase)));
}
static irqreturn_t serial_lpc32xx_interrupt(int irq, void *dev_id)
} else if (!uart_circ_empty(xmit) &&
!uart_tx_stopped(&s->port)) {
tx = xmit->buf[xmit->tail];
- xmit->tail = (xmit->tail + 1) &
- (UART_XMIT_SIZE - 1);
- s->port.icount.tx++;
+ uart_xmit_advance(&s->port, 1);
}
if (tx != 0xffff) {
max3100_calc_parity(s, &tx);
} else {
max310x_batch_write(port, xmit->buf + xmit->tail, to_send);
}
-
- /* Add data to send */
- port->icount.tx += to_send;
- xmit->tail = (xmit->tail + to_send) & (UART_XMIT_SIZE - 1);
+ uart_xmit_advance(port, to_send);
}
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
static void mcf_tx_chars(struct mcf_uart *pp)
{
struct uart_port *port = &pp->port;
- struct circ_buf *xmit = &port->state->xmit;
-
- if (port->x_char) {
- /* Send special char - probably flow control */
- writeb(port->x_char, port->membase + MCFUART_UTB);
- port->x_char = 0;
- port->icount.tx++;
- return;
- }
-
- while (readb(port->membase + MCFUART_USR) & MCFUART_USR_TXREADY) {
- if (uart_circ_empty(xmit))
- break;
- writeb(xmit->buf[xmit->tail], port->membase + MCFUART_UTB);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE -1);
- port->icount.tx++;
- }
+ bool pending;
+ u8 ch;
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
+ pending = uart_port_tx(port, ch,
+ readb(port->membase + MCFUART_USR) & MCFUART_USR_TXREADY,
+ writeb(ch, port->membase + MCFUART_UTB));
- if (uart_circ_empty(xmit)) {
- mcf_stop_tx(port);
- /* Disable TX to negate RTS automatically */
- if (port->rs485.flags & SER_RS485_ENABLED)
- writeb(MCFUART_UCR_TXDISABLE,
- port->membase + MCFUART_UCR);
- }
+ /* Disable TX to negate RTS automatically */
+ if (!pending && (port->rs485.flags & SER_RS485_ENABLED))
+ writeb(MCFUART_UCR_TXDISABLE, port->membase + MCFUART_UCR);
}
/****************************************************************************/
n = min(n, s);
memcpy_toio(port->membase + MEN_Z135_TX_RAM, &xmit->buf[xmit->tail], n);
- xmit->tail = (xmit->tail + n) & (UART_XMIT_SIZE - 1);
-
iowrite32(n & 0x3ff, port->membase + MEN_Z135_TX_CTRL);
-
- port->icount.tx += n;
+ uart_xmit_advance(port, n);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(port);
ch = xmit->buf[xmit->tail];
writel(ch, port->membase + AML_UART_WFIFO);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
}
if (!uart_circ_empty(xmit)) {
do {
writew(xmit->buf[xmit->tail], port->membase + MLB_USIO_REG_DR);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
if (uart_circ_empty(xmit))
break;
static inline bool
mpc52xx_uart_int_tx_chars(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
-
- /* Process out of band chars */
- if (port->x_char) {
- psc_ops->write_char(port, port->x_char);
- port->icount.tx++;
- port->x_char = 0;
- return true;
- }
-
- /* Nothing to do ? */
- if (uart_circ_empty(xmit) || uart_tx_stopped(port)) {
- mpc52xx_uart_stop_tx(port);
- return false;
- }
-
- /* Send chars */
- while (psc_ops->raw_tx_rdy(port)) {
- psc_ops->write_char(port, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- }
-
- /* Wake up */
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
-
- /* Maybe we're done after all */
- if (uart_circ_empty(xmit)) {
- mpc52xx_uart_stop_tx(port);
- return false;
- }
+ u8 ch;
- return true;
+ return uart_port_tx(port, ch,
+ psc_ops->raw_tx_rdy(port),
+ psc_ops->write_char(port, ch));
}
static irqreturn_t
static void mps2_uart_tx_chars(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
-
- while (!(mps2_uart_read8(port, UARTn_STATE) & UARTn_STATE_TX_FULL)) {
- if (port->x_char) {
- mps2_uart_write8(port, port->x_char, UARTn_DATA);
- port->x_char = 0;
- port->icount.tx++;
- continue;
- }
-
- if (uart_circ_empty(xmit) || uart_tx_stopped(port))
- break;
-
- mps2_uart_write8(port, xmit->buf[xmit->tail], UARTn_DATA);
- xmit->tail = (xmit->tail + 1) % UART_XMIT_SIZE;
- port->icount.tx++;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
+ u8 ch;
- if (uart_circ_empty(xmit))
- mps2_uart_stop_tx(port);
+ uart_port_tx(port, ch,
+ mps2_uart_tx_empty(port),
+ mps2_uart_write8(port, ch, UARTn_DATA));
}
static void mps2_uart_start_tx(struct uart_port *port)
}
count = dma->count - state.residue;
- port->icount.tx += count;
+ uart_xmit_advance(port, count);
dma->count = 0;
- xmit->tail += count;
- xmit->tail &= UART_XMIT_SIZE - 1;
-
/* Restore "Tx FIFO below watermark" interrupt */
msm_port->imr |= MSM_UART_IMR_TXLEV;
msm_write(port, msm_port->imr, MSM_UART_IMR);
port->icount.rx++;
}
- /* Mask conditions we're ignorning. */
+ /* Mask conditions we're ignoring. */
sr &= port->read_status_mask;
if (sr & MSM_UART_SR_RX_BREAK)
else
num_chars = 1;
- for (i = 0; i < num_chars; i++) {
+ for (i = 0; i < num_chars; i++)
buf[i] = xmit->buf[xmit->tail + i];
- port->icount.tx++;
- }
iowrite32_rep(tf, buf, 1);
- xmit->tail = (xmit->tail + num_chars) & (UART_XMIT_SIZE - 1);
+ uart_xmit_advance(port, num_chars);
tf_pointer += num_chars;
}
{
}
+static void mux_tx_done(struct uart_port *port)
+{
+ /* FIXME js: really needs to wait? */
+ while (UART_GET_FIFO_CNT(port))
+ udelay(1);
+}
+
/**
* mux_write - Write chars to the mux fifo.
* @port: Ptr to the uart_port.
*/
static void mux_write(struct uart_port *port)
{
- int count;
- struct circ_buf *xmit = &port->state->xmit;
-
- if(port->x_char) {
- UART_PUT_CHAR(port, port->x_char);
- port->icount.tx++;
- port->x_char = 0;
- return;
- }
-
- if(uart_circ_empty(xmit) || uart_tx_stopped(port)) {
- mux_stop_tx(port);
- return;
- }
-
- count = (port->fifosize) - UART_GET_FIFO_CNT(port);
- do {
- UART_PUT_CHAR(port, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- if(uart_circ_empty(xmit))
- break;
-
- } while(--count > 0);
-
- while(UART_GET_FIFO_CNT(port))
- udelay(1);
-
- if(uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
+ u8 ch;
- if (uart_circ_empty(xmit))
- mux_stop_tx(port);
+ uart_port_tx_limited(port, ch,
+ port->fifosize - UART_GET_FIFO_CNT(port),
+ true,
+ UART_PUT_CHAR(port, ch),
+ mux_tx_done(port));
}
/**
if (IS_EXTENDED(port) && !uart_circ_empty(xmit)) {
writel(xmit->buf[xmit->tail], port->membase + UART_TSH(port));
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
}
ctl = readl(port->membase + UART_INTR(port));
static void mvebu_uart_tx_chars(struct uart_port *port, unsigned int status)
{
- struct circ_buf *xmit = &port->state->xmit;
- unsigned int count;
- unsigned int st;
-
- if (port->x_char) {
- writel(port->x_char, port->membase + UART_TSH(port));
- port->icount.tx++;
- port->x_char = 0;
- return;
- }
-
- if (uart_circ_empty(xmit) || uart_tx_stopped(port)) {
- mvebu_uart_stop_tx(port);
- return;
- }
-
- for (count = 0; count < port->fifosize; count++) {
- writel(xmit->buf[xmit->tail], port->membase + UART_TSH(port));
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
-
- if (uart_circ_empty(xmit))
- break;
-
- st = readl(port->membase + UART_STAT);
- if (st & STAT_TX_FIFO_FUL)
- break;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
+ u8 ch;
- if (uart_circ_empty(xmit))
- mvebu_uart_stop_tx(port);
+ uart_port_tx_limited(port, ch, port->fifosize,
+ !(readl(port->membase + UART_STAT) & STAT_TX_FIFO_FUL),
+ writel(ch, port->membase + UART_TSH(port)),
+ ({}));
}
static irqreturn_t mvebu_uart_isr(int irq, void *dev_id)
static void mxs_auart_tx_chars(struct mxs_auart_port *s)
{
struct circ_buf *xmit = &s->port.state->xmit;
+ bool pending;
+ u8 ch;
if (auart_dma_enabled(s)) {
u32 i = 0;
return;
}
-
- while (!(mxs_read(s, REG_STAT) & AUART_STAT_TXFF)) {
- if (s->port.x_char) {
- s->port.icount.tx++;
- mxs_write(s->port.x_char, s, REG_DATA);
- s->port.x_char = 0;
- continue;
- }
- if (!uart_circ_empty(xmit) && !uart_tx_stopped(&s->port)) {
- s->port.icount.tx++;
- mxs_write(xmit->buf[xmit->tail], s, REG_DATA);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- } else
- break;
- }
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(&s->port);
-
- if (uart_circ_empty(&(s->port.state->xmit)))
- mxs_clr(AUART_INTR_TXIEN, s, REG_INTR);
- else
+ pending = uart_port_tx(&s->port, ch,
+ !(mxs_read(s, REG_STAT) & AUART_STAT_TXFF),
+ mxs_write(ch, s, REG_DATA));
+ if (pending)
mxs_set(AUART_INTR_TXIEN, s, REG_INTR);
-
- if (uart_tx_stopped(&s->port))
- mxs_auart_stop_tx(&s->port);
+ else
+ mxs_clr(AUART_INTR_TXIEN, s, REG_INTR);
}
static void mxs_auart_rx_char(struct mxs_auart_port *s)
static void transmit_chars(struct uart_omap_port *up, unsigned int lsr)
{
- struct circ_buf *xmit = &up->port.state->xmit;
- int count;
+ u8 ch;
- if (up->port.x_char) {
- serial_omap_put_char(up, up->port.x_char);
- up->port.icount.tx++;
- up->port.x_char = 0;
- return;
- }
- if (uart_circ_empty(xmit) || uart_tx_stopped(&up->port)) {
- serial_omap_stop_tx(&up->port);
- return;
- }
- count = up->port.fifosize / 4;
- do {
- serial_omap_put_char(up, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- up->port.icount.tx++;
-
- if (uart_circ_empty(xmit))
- break;
- } while (--count > 0);
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(&up->port);
-
- if (uart_circ_empty(xmit))
- serial_omap_stop_tx(&up->port);
+ uart_port_tx_limited(&up->port, ch, up->port.fifosize / 4,
+ true,
+ serial_omap_put_char(up, ch),
+ ({}));
}
static inline void serial_omap_enable_ier_thri(struct uart_omap_port *up)
static void owl_uart_send_chars(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
- unsigned int ch;
-
- if (port->x_char) {
- while (!(owl_uart_read(port, OWL_UART_STAT) & OWL_UART_STAT_TFFU))
- cpu_relax();
- owl_uart_write(port, port->x_char, OWL_UART_TXDAT);
- port->icount.tx++;
- port->x_char = 0;
- }
-
- if (uart_tx_stopped(port))
- return;
-
- while (!(owl_uart_read(port, OWL_UART_STAT) & OWL_UART_STAT_TFFU)) {
- if (uart_circ_empty(xmit))
- break;
+ u8 ch;
- ch = xmit->buf[xmit->tail];
- owl_uart_write(port, ch, OWL_UART_TXDAT);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
-
- if (uart_circ_empty(xmit))
- owl_uart_stop_tx(port);
+ uart_port_tx(port, ch,
+ !(owl_uart_read(port, OWL_UART_STAT) & OWL_UART_STAT_TFFU),
+ owl_uart_write(port, ch, OWL_UART_TXDAT));
}
static void owl_uart_receive_chars(struct uart_port *port)
if (!chan) {
dev_err(priv->port.dev, "%s:dma_request_channel FAILS(Tx)\n",
__func__);
+ pci_dev_put(dma_dev);
return;
}
priv->chan_tx = chan;
__func__);
dma_release_channel(priv->chan_tx);
priv->chan_tx = NULL;
+ pci_dev_put(dma_dev);
return;
}
priv->rx_buf_virt = dma_alloc_coherent(port->dev, port->fifosize,
&priv->rx_buf_dma, GFP_KERNEL);
priv->chan_rx = chan;
+
+ pci_dev_put(dma_dev);
}
static void pch_dma_rx_complete(void *arg)
{
struct eg20t_port *priv = arg;
struct uart_port *port = &priv->port;
- struct circ_buf *xmit = &port->state->xmit;
struct scatterlist *sg = priv->sg_tx_p;
int i;
- for (i = 0; i < priv->nent; i++, sg++) {
- xmit->tail += sg_dma_len(sg);
- port->icount.tx += sg_dma_len(sg);
- }
- xmit->tail &= UART_XMIT_SIZE - 1;
+ for (i = 0; i < priv->nent; i++, sg++)
+ uart_xmit_advance(port, sg_dma_len(sg));
+
async_tx_ack(priv->desc_tx);
dma_unmap_sg(port->dev, sg, priv->orig_nent, DMA_TO_DEVICE);
priv->tx_dma_use = 0;
while (!uart_tx_stopped(port) && !uart_circ_empty(xmit) && fifo_size) {
iowrite8(xmit->buf[xmit->tail], priv->membase + PCH_UART_THR);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
fifo_size--;
tx_empty = 0;
}
pic32_uart_writel(sport, PIC32_UART_TX, c);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
if (uart_circ_empty(xmit))
break;
if (--max_count == 0)
write_zsdata(uap, xmit->buf[xmit->tail]);
zssync(uap);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- uap->port.icount.tx++;
+ uart_xmit_advance(&uap->port, 1);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(&uap->port);
return;
write_zsdata(uap, xmit->buf[xmit->tail]);
zssync(uap);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(&uap->port);
static void transmit_chars(struct uart_pxa_port *up)
{
- struct circ_buf *xmit = &up->port.state->xmit;
- int count;
+ u8 ch;
- if (up->port.x_char) {
- serial_out(up, UART_TX, up->port.x_char);
- up->port.icount.tx++;
- up->port.x_char = 0;
- return;
- }
- if (uart_circ_empty(xmit) || uart_tx_stopped(&up->port)) {
- serial_pxa_stop_tx(&up->port);
- return;
- }
-
- count = up->port.fifosize / 2;
- do {
- serial_out(up, UART_TX, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- up->port.icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- } while (--count > 0);
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(&up->port);
-
-
- if (uart_circ_empty(xmit))
- serial_pxa_stop_tx(&up->port);
+ uart_port_tx_limited(&up->port, ch, up->port.fifosize / 2,
+ true,
+ serial_out(up, UART_TX, ch),
+ ({}));
}
static void serial_pxa_start_tx(struct uart_port *port)
false, true, true);
geni_se_init(&port->se, UART_RX_WM, port->rx_fifo_depth - 2);
geni_se_select_mode(&port->se, GENI_SE_FIFO);
+ qcom_geni_serial_start_rx(uport);
port->setup = true;
return 0;
return ret;
}
+static int qcom_geni_serial_sys_hib_resume(struct device *dev)
+{
+ int ret = 0;
+ struct uart_port *uport;
+ struct qcom_geni_private_data *private_data;
+ struct qcom_geni_serial_port *port = dev_get_drvdata(dev);
+
+ uport = &port->uport;
+ private_data = uport->private_data;
+
+ if (uart_console(uport)) {
+ geni_icc_set_tag(&port->se, 0x7);
+ geni_icc_set_bw(&port->se);
+ ret = uart_resume_port(private_data->drv, uport);
+ /*
+ * For hibernation usecase clients for
+ * console UART won't call port setup during restore,
+ * hence call port setup for console uart.
+ */
+ qcom_geni_serial_port_setup(uport);
+ } else {
+ /*
+ * Peripheral register settings are lost during hibernation.
+ * Update setup flag such that port setup happens again
+ * during next session. Clients of HS-UART will close and
+ * open the port during hibernation.
+ */
+ port->setup = false;
+ }
+ return ret;
+}
+
static const struct dev_pm_ops qcom_geni_serial_pm_ops = {
SET_SYSTEM_SLEEP_PM_OPS(qcom_geni_serial_sys_suspend,
qcom_geni_serial_sys_resume)
+ .restore = qcom_geni_serial_sys_hib_resume,
+ .thaw = qcom_geni_serial_sys_hib_resume,
};
static const struct of_device_id qcom_geni_serial_match_table[] = {
ch = xmit->buf[xmit->tail];
rda_uart_write(port, ch, RDA_UART_RXTX_BUFFER);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
}
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
static void rp2_tx_chars(struct rp2_uart_port *up)
{
- u16 max_tx = FIFO_SIZE - readw(up->base + RP2_TX_FIFO_COUNT);
- struct circ_buf *xmit = &up->port.state->xmit;
+ u8 ch;
- if (uart_tx_stopped(&up->port)) {
- rp2_uart_stop_tx(&up->port);
- return;
- }
-
- for (; max_tx != 0; max_tx--) {
- if (up->port.x_char) {
- writeb(up->port.x_char, up->base + RP2_DATA_BYTE);
- up->port.x_char = 0;
- up->port.icount.tx++;
- continue;
- }
- if (uart_circ_empty(xmit)) {
- rp2_uart_stop_tx(&up->port);
- break;
- }
- writeb(xmit->buf[xmit->tail], up->base + RP2_DATA_BYTE);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- up->port.icount.tx++;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(&up->port);
+ uart_port_tx_limited(&up->port, ch,
+ FIFO_SIZE - readw(up->base + RP2_TX_FIFO_COUNT),
+ true,
+ writeb(ch, up->base + RP2_DATA_BYTE),
+ ({}));
}
static void rp2_ch_interrupt(struct rp2_uart_port *up)
static void sa1100_tx_chars(struct sa1100_port *sport)
{
- struct circ_buf *xmit = &sport->port.state->xmit;
-
- if (sport->port.x_char) {
- UART_PUT_CHAR(sport, sport->port.x_char);
- sport->port.icount.tx++;
- sport->port.x_char = 0;
- return;
- }
+ u8 ch;
/*
* Check the modem control lines before
*/
sa1100_mctrl_check(sport);
- if (uart_circ_empty(xmit) || uart_tx_stopped(&sport->port)) {
- sa1100_stop_tx(&sport->port);
- return;
- }
-
- /*
- * Tried using FIFO (not checking TNF) for fifo fill:
- * still had the '4 bytes repeated' problem.
- */
- while (UART_GET_UTSR1(sport) & UTSR1_TNF) {
- UART_PUT_CHAR(sport, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- sport->port.icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(&sport->port);
-
- if (uart_circ_empty(xmit))
- sa1100_stop_tx(&sport->port);
+ uart_port_tx(&sport->port, ch,
+ UART_GET_UTSR1(sport) & UTSR1_TNF,
+ UART_PUT_CHAR(sport, ch));
}
static irqreturn_t sa1100_int(int irq, void *dev_id)
{
struct s3c24xx_uart_port *ourport = to_ourport(port);
struct s3c24xx_uart_dma *dma = ourport->dma;
- struct circ_buf *xmit = &port->state->xmit;
struct dma_tx_state state;
int count;
DMA_TO_DEVICE);
async_tx_ack(dma->tx_desc);
count = dma->tx_bytes_requested - state.residue;
- xmit->tail = (xmit->tail + count) & (UART_XMIT_SIZE - 1);
- port->icount.tx += count;
+ uart_xmit_advance(port, count);
}
ourport->tx_enabled = 0;
spin_lock_irqsave(&port->lock, flags);
- xmit->tail = (xmit->tail + count) & (UART_XMIT_SIZE - 1);
- port->icount.tx += count;
+ uart_xmit_advance(port, count);
ourport->tx_in_progress = 0;
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
break;
wr_reg(port, S3C2410_UTXH, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
count--;
}
/* Send char. */
if (!stop_tx) {
write_sbdchn(sport, R_DUART_TX_HOLD, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- sport->port.icount.tx++;
+ uart_xmit_advance(&sport->port, 1);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(&sport->port);
}
to_send = (to_send > txlen) ? txlen : to_send;
- /* Add data to send */
- port->icount.tx += to_send;
-
/* Convert to linear buffer */
for (i = 0; i < to_send; ++i) {
s->buf[i] = xmit->buf[xmit->tail];
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
+ uart_xmit_advance(port, 1);
}
sc16is7xx_fifo_write(port, to_send);
break;
sccnxp_port_write(port, SCCNXP_THR_REG, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
}
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
break;
}
tegra_uart_write(tup, xmit->buf[xmit->tail], UART_TX);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- tup->uport.icount.tx++;
+ uart_xmit_advance(&tup->uport, 1);
}
}
if (tup->tx_in_progress != TEGRA_UART_TX_DMA)
return;
- dmaengine_terminate_all(tup->tx_dma_chan);
+ dmaengine_pause(tup->tx_dma_chan);
dmaengine_tx_status(tup->tx_dma_chan, tup->tx_cookie, &state);
+ dmaengine_terminate_all(tup->tx_dma_chan);
count = tup->tx_bytes_requested - state.residue;
async_tx_ack(tup->tx_dma_desc);
uart_xmit_advance(&tup->uport, count);
return;
}
- dmaengine_terminate_all(tup->rx_dma_chan);
+ dmaengine_pause(tup->rx_dma_chan);
dmaengine_tx_status(tup->rx_dma_chan, tup->rx_cookie, &state);
+ dmaengine_terminate_all(tup->rx_dma_chan);
tegra_uart_rx_buffer_push(tup, state.residue);
tup->rx_dma_active = false;
static inline void transmit_chars(struct uart_port *up)
{
- struct circ_buf *xmit = &up->state->xmit;
- int count;
+ u8 ch;
- if (up->x_char) {
- sio_out(up, TXX9_SITFIFO, up->x_char);
- up->icount.tx++;
- up->x_char = 0;
- return;
- }
- if (uart_circ_empty(xmit) || uart_tx_stopped(up)) {
- serial_txx9_stop_tx(up);
- return;
- }
-
- count = TXX9_SIO_TX_FIFO;
- do {
- sio_out(up, TXX9_SITFIFO, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- up->icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- } while (--count > 0);
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(up);
-
- if (uart_circ_empty(xmit))
- serial_txx9_stop_tx(up);
+ uart_port_tx_limited(up, ch, TXX9_SIO_TX_FIFO,
+ true,
+ sio_out(up, TXX9_SITFIFO, ch),
+ ({}));
}
static irqreturn_t serial_txx9_interrupt(int irq, void *dev_id)
spin_lock_irqsave(&port->lock, flags);
- xmit->tail += s->tx_dma_len;
- xmit->tail &= UART_XMIT_SIZE - 1;
-
- port->icount.tx += s->tx_dma_len;
+ uart_xmit_advance(port, s->tx_dma_len);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(port);
*/
static void __ssp_transmit_chars(struct sifive_serial_port *ssp)
{
- struct circ_buf *xmit = &ssp->port.state->xmit;
- int count;
-
- if (ssp->port.x_char) {
- __ssp_transmit_char(ssp, ssp->port.x_char);
- ssp->port.icount.tx++;
- ssp->port.x_char = 0;
- return;
- }
- if (uart_circ_empty(xmit) || uart_tx_stopped(&ssp->port)) {
- sifive_serial_stop_tx(&ssp->port);
- return;
- }
- count = SIFIVE_TX_FIFO_DEPTH;
- do {
- __ssp_transmit_char(ssp, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- ssp->port.icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- } while (--count > 0);
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(&ssp->port);
+ u8 ch;
- if (uart_circ_empty(xmit))
- sifive_serial_stop_tx(&ssp->port);
+ uart_port_tx_limited(&ssp->port, ch, SIFIVE_TX_FIFO_DEPTH,
+ true,
+ __ssp_transmit_char(ssp, ch),
+ ({}));
}
/**
{
struct sprd_uart_port *sp =
container_of(port, struct sprd_uart_port, port);
- struct circ_buf *xmit = &port->state->xmit;
struct dma_tx_state state;
u32 trans_len;
dmaengine_tx_status(sp->tx_dma.chn, sp->tx_dma.cookie, &state);
if (state.residue) {
trans_len = state.residue - sp->tx_dma.phys_addr;
- xmit->tail = (xmit->tail + trans_len) & (UART_XMIT_SIZE - 1);
- port->icount.tx += trans_len;
+ uart_xmit_advance(port, trans_len);
dma_unmap_single(port->dev, sp->tx_dma.phys_addr,
sp->tx_dma.trans_len, DMA_TO_DEVICE);
}
dma_unmap_single(port->dev, sp->tx_dma.phys_addr,
sp->tx_dma.trans_len, DMA_TO_DEVICE);
- xmit->tail = (xmit->tail + sp->tx_dma.trans_len) & (UART_XMIT_SIZE - 1);
- port->icount.tx += sp->tx_dma.trans_len;
+ uart_xmit_advance(port, sp->tx_dma.trans_len);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(port);
static inline void sprd_tx(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
- int count;
-
- if (port->x_char) {
- serial_out(port, SPRD_TXD, port->x_char);
- port->icount.tx++;
- port->x_char = 0;
- return;
- }
-
- if (uart_circ_empty(xmit) || uart_tx_stopped(port)) {
- sprd_stop_tx(port);
- return;
- }
-
- count = THLD_TX_EMPTY;
- do {
- serial_out(port, SPRD_TXD, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- if (uart_circ_empty(xmit))
- break;
- } while (--count > 0);
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
+ u8 ch;
- if (uart_circ_empty(xmit))
- sprd_stop_tx(port);
+ uart_port_tx_limited(port, ch, THLD_TX_EMPTY,
+ true,
+ serial_out(port, SPRD_TXD, ch),
+ ({}));
}
/* this handles the interrupt from one port */
*/
static void asc_transmit_chars(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
- int txroom;
- unsigned char c;
-
- txroom = asc_hw_txroom(port);
-
- if ((txroom != 0) && port->x_char) {
- c = port->x_char;
- port->x_char = 0;
- asc_out(port, ASC_TXBUF, c);
- port->icount.tx++;
- txroom = asc_hw_txroom(port);
- }
-
- if (uart_tx_stopped(port)) {
- /*
- * We should try and stop the hardware here, but I
- * don't think the ASC has any way to do that.
- */
- asc_disable_tx_interrupts(port);
- return;
- }
-
- if (uart_circ_empty(xmit)) {
- asc_disable_tx_interrupts(port);
- return;
- }
-
- if (txroom == 0)
- return;
-
- do {
- c = xmit->buf[xmit->tail];
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- asc_out(port, ASC_TXBUF, c);
- port->icount.tx++;
- txroom--;
- } while ((txroom > 0) && (!uart_circ_empty(xmit)));
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
+ u8 ch;
- if (uart_circ_empty(xmit))
- asc_disable_tx_interrupts(port);
+ uart_port_tx_limited(port, ch, asc_hw_txroom(port),
+ true,
+ asc_out(port, ASC_TXBUF, ch),
+ ({}));
}
static void asc_receive_chars(struct uart_port *port)
if (!(readl_relaxed(port->membase + ofs->isr) & USART_SR_TXE))
break;
writel_relaxed(xmit->buf[xmit->tail], port->membase + ofs->tdr);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
}
/* rely on TXE irq (mask or unmask) for sending remaining data */
stm32_usart_set_bits(port, ofs->cr3, USART_CR3_DMAT);
- xmit->tail = (xmit->tail + count) & (UART_XMIT_SIZE - 1);
- port->icount.tx += count;
+ uart_xmit_advance(port, count);
+
return;
fallback_err:
if (!stm32port->info)
return -EINVAL;
- ret = stm32_usart_init_port(stm32port, pdev);
- if (ret)
- return ret;
-
- if (stm32port->wakeup_src) {
- device_set_wakeup_capable(&pdev->dev, true);
- ret = dev_pm_set_wake_irq(&pdev->dev, stm32port->port.irq);
- if (ret)
- goto err_deinit_port;
- }
-
stm32port->rx_ch = dma_request_chan(&pdev->dev, "rx");
- if (PTR_ERR(stm32port->rx_ch) == -EPROBE_DEFER) {
- ret = -EPROBE_DEFER;
- goto err_wakeirq;
- }
+ if (PTR_ERR(stm32port->rx_ch) == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
+
/* Fall back in interrupt mode for any non-deferral error */
if (IS_ERR(stm32port->rx_ch))
stm32port->rx_ch = NULL;
if (IS_ERR(stm32port->tx_ch))
stm32port->tx_ch = NULL;
+ ret = stm32_usart_init_port(stm32port, pdev);
+ if (ret)
+ goto err_dma_tx;
+
+ if (stm32port->wakeup_src) {
+ device_set_wakeup_capable(&pdev->dev, true);
+ ret = dev_pm_set_wake_irq(&pdev->dev, stm32port->port.irq);
+ if (ret)
+ goto err_deinit_port;
+ }
+
if (stm32port->rx_ch && stm32_usart_of_dma_rx_probe(stm32port, pdev)) {
/* Fall back in interrupt mode */
dma_release_channel(stm32port->rx_ch);
pm_runtime_set_suspended(&pdev->dev);
pm_runtime_put_noidle(&pdev->dev);
- if (stm32port->tx_ch) {
+ if (stm32port->tx_ch)
stm32_usart_of_dma_tx_remove(stm32port, pdev);
- dma_release_channel(stm32port->tx_ch);
- }
-
if (stm32port->rx_ch)
stm32_usart_of_dma_rx_remove(stm32port, pdev);
-err_dma_rx:
- if (stm32port->rx_ch)
- dma_release_channel(stm32port->rx_ch);
-
-err_wakeirq:
if (stm32port->wakeup_src)
dev_pm_clear_wake_irq(&pdev->dev);
stm32_usart_deinit_port(stm32port);
+err_dma_tx:
+ if (stm32port->tx_ch)
+ dma_release_channel(stm32port->tx_ch);
+
+err_dma_rx:
+ if (stm32port->rx_ch)
+ dma_release_channel(stm32port->rx_ch);
+
return ret;
}
if (status != HV_EOK)
break;
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
}
}
status = sun4v_con_write(ra, len, &sent);
if (status != HV_EOK)
break;
- xmit->tail = (xmit->tail + sent) & (UART_XMIT_SIZE - 1);
- port->icount.tx += sent;
+ uart_xmit_advance(port, sent);
}
}
do {
sp_uart_put_char(port, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) % UART_XMIT_SIZE;
- port->icount.tx++;
-
+ uart_xmit_advance(port, 1);
if (uart_circ_empty(xmit))
break;
} while (sunplus_tx_buf_not_full(port));
for (i = 0; i < up->port.fifosize; i++) {
writeb(xmit->buf[xmit->tail],
&up->regs->w.xfifo[i]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- up->port.icount.tx++;
+ uart_xmit_advance(&up->port, 1);
if (uart_circ_empty(xmit))
break;
}
for (i = 0; i < up->port.fifosize; i++) {
writeb(xmit->buf[xmit->tail],
&up->regs->w.xfifo[i]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- up->port.icount.tx++;
+ uart_xmit_advance(&up->port, 1);
if (uart_circ_empty(xmit))
break;
}
}
}
- return platform_driver_register(&sab_driver);
+ err = platform_driver_register(&sab_driver);
+ if (err) {
+ kfree(sunsab_ports);
+ sunsab_ports = NULL;
+ }
+
+ return err;
}
static void __exit sunsab_exit(void)
count = up->port.fifosize;
do {
serial_out(up, UART_TX, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- up->port.icount.tx++;
+ uart_xmit_advance(&up->port, 1);
if (uart_circ_empty(xmit))
break;
} while (--count > 0);
ZSDELAY();
ZS_WSYNC(channel);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- up->port.icount.tx++;
+ uart_xmit_advance(&up->port, 1);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(&up->port);
ZSDELAY();
ZS_WSYNC(channel);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(&up->port);
!uart_circ_empty(xmit)) {
iowrite8(xmit->buf[xmit->tail],
port->membase + TIMBUART_TXFIFO);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
}
dev_dbg(port->dev,
return 0;
uart_out32(xmit->buf[xmit->tail], ULITE_TX, port);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE-1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
/* wake up */
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
p = qe2cpu_addr(be32_to_cpu(bdp->buf), qe_port);
while (count < qe_port->tx_fifosize) {
*p++ = xmit->buf[xmit->tail];
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
+ uart_xmit_advance(port, 1);
count++;
if (xmit->head == xmit->tail)
break;
c = readw(port->membase + VT8500_RXFIFO) & 0x3ff;
- /* Mask conditions we're ignorning. */
+ /* Mask conditions we're ignoring. */
c &= ~port->read_status_mask;
if (c & FER) {
static void handle_tx(struct uart_port *port)
{
- struct circ_buf *xmit = &port->state->xmit;
+ u8 ch;
- if (port->x_char) {
- writeb(port->x_char, port->membase + VT8500_TXFIFO);
- port->icount.tx++;
- port->x_char = 0;
- }
- if (uart_circ_empty(xmit) || uart_tx_stopped(port)) {
- vt8500_stop_tx(port);
- return;
- }
-
- while (vt8500_tx_empty(port)) {
- if (uart_circ_empty(xmit))
- break;
-
- writeb(xmit->buf[xmit->tail], port->membase + VT8500_TXFIFO);
-
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- port->icount.tx++;
- }
-
- if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
- uart_write_wakeup(port);
-
- if (uart_circ_empty(xmit))
- vt8500_stop_tx(port);
+ uart_port_tx(port, ch,
+ vt8500_tx_empty(port),
+ writeb(ch, port->membase + VT8500_TXFIFO));
}
static void vt8500_start_tx(struct uart_port *port)
!(readl(port->membase + CDNS_UART_SR) & CDNS_UART_SR_TXFULL)) {
writel(xmit->buf[xmit->tail], port->membase + CDNS_UART_FIFO);
-
- port->icount.tx++;
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
+ uart_xmit_advance(port, 1);
numbytes--;
}
/* Send char. */
write_zsdata(zport, xmit->buf[xmit->tail]);
- xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
- zport->port.icount.tx++;
+ uart_xmit_advance(&zport->port, 1);
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(&zport->port);
int rc;
unsigned long flags;
- if (!try_module_get(THIS_MODULE))
- return -EBUSY;
-
DBGINFO(("%s hdlcdev_open\n", dev->name));
- /* generic HDLC layer open processing */
- rc = hdlc_open(dev);
- if (rc)
- return rc;
-
/* arbitrate between network and tty opens */
spin_lock_irqsave(&info->netlock, flags);
if (info->port.count != 0 || info->netcount != 0) {
return rc;
}
+ /* generic HDLC layer open processing */
+ rc = hdlc_open(dev);
+ if (rc) {
+ shutdown(info);
+ spin_lock_irqsave(&info->netlock, flags);
+ info->netcount = 0;
+ spin_unlock_irqrestore(&info->netlock, flags);
+ return rc;
+ }
+
/* assert RTS and DTR, apply hardware settings */
info->signals |= SerialSignal_RTS | SerialSignal_DTR;
program_hw(info);
info->netcount=0;
spin_unlock_irqrestore(&info->netlock, flags);
- module_put(THIS_MODULE);
return 0;
}
int __must_check tty_ldisc_init(struct tty_struct *tty);
void tty_ldisc_deinit(struct tty_struct *tty);
-void tty_sysctl_init(void);
+extern int tty_ldisc_autoload;
/* tty_audit.c */
#ifdef CONFIG_AUDIT
#include "tty.h"
#define MIN_TTYB_SIZE 256
-#define TTYB_ALIGN_MASK 255
+#define TTYB_ALIGN_MASK 0xff
/*
* Byte threshold to limit memory consumption for flip buffers.
* logic this must match.
*/
-#define TTY_BUFFER_PAGE (((PAGE_SIZE - sizeof(struct tty_buffer)) / 2) & ~0xFF)
+#define TTY_BUFFER_PAGE (((PAGE_SIZE - sizeof(struct tty_buffer)) / 2) & ~TTYB_ALIGN_MASK)
/**
* tty_buffer_lock_exclusive - gain exclusive access to buffer
p->commit = 0;
p->lookahead = 0;
p->read = 0;
- p->flags = 0;
+ p->flags = true;
}
/**
* __tty_buffer_request_room - grow tty buffer if needed
* @port: tty port
* @size: size desired
- * @flags: buffer flags if new buffer allocated (default = 0)
+ * @flags: buffer has to store flags along character data
*
* Make at least @size bytes of linear space available for the tty buffer.
*
* Returns: the size we managed to find.
*/
static int __tty_buffer_request_room(struct tty_port *port, size_t size,
- int flags)
+ bool flags)
{
struct tty_bufhead *buf = &port->buf;
struct tty_buffer *b, *n;
int left, change;
b = buf->tail;
- if (b->flags & TTYB_NORMAL)
+ if (!b->flags)
left = 2 * b->size - b->used;
else
left = b->size - b->used;
- change = (b->flags & TTYB_NORMAL) && (~flags & TTYB_NORMAL);
+ change = !b->flags && flags;
if (change || left < size) {
/* This is the slow path - looking for new buffers to use */
n = tty_buffer_alloc(port, size);
int tty_buffer_request_room(struct tty_port *port, size_t size)
{
- return __tty_buffer_request_room(port, size, 0);
+ return __tty_buffer_request_room(port, size, true);
}
EXPORT_SYMBOL_GPL(tty_buffer_request_room);
const unsigned char *chars, char flag, size_t size)
{
int copied = 0;
+ bool flags = flag != TTY_NORMAL;
do {
int goal = min_t(size_t, size - copied, TTY_BUFFER_PAGE);
- int flags = (flag == TTY_NORMAL) ? TTYB_NORMAL : 0;
int space = __tty_buffer_request_room(port, goal, flags);
struct tty_buffer *tb = port->buf.tail;
if (unlikely(space == 0))
break;
memcpy(char_buf_ptr(tb, tb->used), chars, space);
- if (~tb->flags & TTYB_NORMAL)
+ if (tb->flags)
memset(flag_buf_ptr(tb, tb->used), flag, space);
tb->used += space;
copied += space;
int __tty_insert_flip_char(struct tty_port *port, unsigned char ch, char flag)
{
struct tty_buffer *tb;
- int flags = (flag == TTY_NORMAL) ? TTYB_NORMAL : 0;
+ bool flags = flag != TTY_NORMAL;
if (!__tty_buffer_request_room(port, 1, flags))
return 0;
tb = port->buf.tail;
- if (~tb->flags & TTYB_NORMAL)
+ if (tb->flags)
*flag_buf_ptr(tb, tb->used) = flag;
*char_buf_ptr(tb, tb->used++) = ch;
int tty_prepare_flip_string(struct tty_port *port, unsigned char **chars,
size_t size)
{
- int space = __tty_buffer_request_room(port, size, TTYB_NORMAL);
+ int space = __tty_buffer_request_room(port, size, false);
if (likely(space)) {
struct tty_buffer *tb = port->buf.tail;
*chars = char_buf_ptr(tb, tb->used);
- if (~tb->flags & TTYB_NORMAL)
+ if (tb->flags)
memset(flag_buf_ptr(tb, tb->used), TTY_NORMAL, space);
tb->used += space;
}
unsigned char *p, *f = NULL;
p = char_buf_ptr(head, head->lookahead);
- if (~head->flags & TTYB_NORMAL)
+ if (head->flags)
f = flag_buf_ptr(head, head->lookahead);
port->client_ops->lookahead_buf(port, p, f, count);
const char *f = NULL;
int n;
- if (~head->flags & TTYB_NORMAL)
+ if (head->flags)
f = flag_buf_ptr(head, head->read);
n = port->client_ops->receive_buf(port, p, f, count);
return retval;
}
+static bool tty_legacy_tiocsti __read_mostly = IS_ENABLED(CONFIG_LEGACY_TIOCSTI);
/**
* tiocsti - fake input character
* @tty: tty to fake input into
char ch, mbz = 0;
struct tty_ldisc *ld;
+ if (!tty_legacy_tiocsti)
+ return -EIO;
+
if ((current->signal->tty != tty) && !capable(CAP_SYS_ADMIN))
return -EPERM;
if (get_user(ch, p))
*fops = tty_fops;
}
-static char *tty_devnode(struct device *dev, umode_t *mode)
+static char *tty_devnode(const struct device *dev, umode_t *mode)
{
if (!mode)
return NULL;
sysfs_notify(&consdev->kobj, NULL, "active");
}
+static struct ctl_table tty_table[] = {
+ {
+ .procname = "legacy_tiocsti",
+ .data = &tty_legacy_tiocsti,
+ .maxlen = sizeof(tty_legacy_tiocsti),
+ .mode = 0644,
+ .proc_handler = proc_dobool,
+ },
+ {
+ .procname = "ldisc_autoload",
+ .data = &tty_ldisc_autoload,
+ .maxlen = sizeof(tty_ldisc_autoload),
+ .mode = 0644,
+ .proc_handler = proc_dointvec,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
+ },
+ { }
+};
+
+static struct ctl_table tty_dir_table[] = {
+ {
+ .procname = "tty",
+ .mode = 0555,
+ .child = tty_table,
+ },
+ { }
+};
+
+static struct ctl_table tty_root_table[] = {
+ {
+ .procname = "dev",
+ .mode = 0555,
+ .child = tty_dir_table,
+ },
+ { }
+};
+
/*
* Ok, now we can initialize the rest of the tty devices and can count
* on memory allocations, interrupts etc..
*/
int __init tty_init(void)
{
- tty_sysctl_init();
+ register_sysctl_table(tty_root_table);
cdev_init(&tty_cdev, &tty_fops);
if (cdev_add(&tty_cdev, MKDEV(TTYAUX_MAJOR, 0), 1) ||
register_chrdev_region(MKDEV(TTYAUX_MAJOR, 0), 1, "/dev/tty") < 0)
#endif
return 0;
}
-
raw_spin_unlock_irqrestore(&tty_ldiscs_lock, flags);
}
-static int tty_ldisc_autoload = IS_BUILTIN(CONFIG_LDISC_AUTOLOAD);
+int tty_ldisc_autoload = IS_BUILTIN(CONFIG_LDISC_AUTOLOAD);
/**
* tty_ldisc_get - take a reference to an ldisc
tty_ldisc_put(tty->ldisc);
tty->ldisc = NULL;
}
-
-static struct ctl_table tty_table[] = {
- {
- .procname = "ldisc_autoload",
- .data = &tty_ldisc_autoload,
- .maxlen = sizeof(tty_ldisc_autoload),
- .mode = 0644,
- .proc_handler = proc_dointvec,
- .extra1 = SYSCTL_ZERO,
- .extra2 = SYSCTL_ONE,
- },
- { }
-};
-
-static struct ctl_table tty_dir_table[] = {
- {
- .procname = "tty",
- .mode = 0555,
- .child = tty_table,
- },
- { }
-};
-
-static struct ctl_table tty_root_table[] = {
- {
- .procname = "dev",
- .mode = 0555,
- .child = tty_dir_table,
- },
- { }
-};
-
-void tty_sysctl_init(void)
-{
- register_sysctl_table(tty_root_table);
-}
unsigned int refcnt;
};
+/* Bits in uio_dmem_genirq_platdata.flags */
+enum {
+ UIO_IRQ_DISABLED = 0,
+};
+
static int uio_dmem_genirq_open(struct uio_info *info, struct inode *inode)
{
struct uio_dmem_genirq_platdata *priv = info->priv;
* remember the state so we can allow user space to enable it later.
*/
- if (!test_and_set_bit(0, &priv->flags))
+ spin_lock(&priv->lock);
+ if (!__test_and_set_bit(UIO_IRQ_DISABLED, &priv->flags))
disable_irq_nosync(irq);
+ spin_unlock(&priv->lock);
return IRQ_HANDLED;
}
* in the interrupt controller, but keep track of the
* state to prevent per-irq depth damage.
*
- * Serialize this operation to support multiple tasks.
+ * Serialize this operation to support multiple tasks and concurrency
+ * with irq handler on SMP systems.
*/
spin_lock_irqsave(&priv->lock, flags);
if (irq_on) {
- if (test_and_clear_bit(0, &priv->flags))
+ if (__test_and_clear_bit(UIO_IRQ_DISABLED, &priv->flags))
enable_irq(dev_info->irq);
- spin_unlock_irqrestore(&priv->lock, flags);
} else {
- if (!test_and_set_bit(0, &priv->flags)) {
- spin_unlock_irqrestore(&priv->lock, flags);
- disable_irq(dev_info->irq);
- }
+ if (!__test_and_set_bit(UIO_IRQ_DISABLED, &priv->flags))
+ disable_irq_nosync(dev_info->irq);
}
+ spin_unlock_irqrestore(&priv->lock, flags);
return 0;
}
info->priv = priv;
info->name = uio_name;
info->version = "0.0.1";
- if (irq != NO_IRQ) {
+ if (irq) {
if (priv->irq_handler) {
info->irq = irq;
info->irq_flags = IRQF_SHARED;
info->handler = priv->irq_handler;
} else {
- irq = NO_IRQ;
+ irq = 0;
dev_warn(priv->dev, "ignoring irq, no handler\n");
}
}
dev_info(priv->dev,
"eLBC/GPCM device (%s) at 0x%llx, bank %d, irq=%d\n",
priv->name, (unsigned long long)res.start, priv->bank,
- irq != NO_IRQ ? irq : -1);
+ irq ? : -1);
return 0;
out_err2:
endif
+comment "USB dual-mode controller drivers"
+
source "drivers/usb/cdns3/Kconfig"
+source "drivers/usb/fotg210/Kconfig"
+
source "drivers/usb/mtu3/Kconfig"
source "drivers/usb/musb/Kconfig"
obj-$(CONFIG_USB_CDNS3) += cdns3/
obj-$(CONFIG_USB_CDNSP_PCI) += cdns3/
+obj-$(CONFIG_USB_FOTG210) += fotg210/
+
obj-$(CONFIG_USB_MON) += mon/
obj-$(CONFIG_USB_MTU3) += mtu3/
if (pci_dev_run_wake(pdev))
pm_runtime_get_noresume(&pdev->dev);
- if (!pci_is_enabled(func)) {
+ if (pci_is_enabled(func)) {
+ cdns_remove(cdnsp);
+ } else {
kfree(cdnsp);
- goto pci_put;
}
- cdns_remove(cdnsp);
-
-pci_put:
pci_dev_put(func);
}
int cdnsp_queue_ctrl_tx(struct cdnsp_device *pdev, struct cdnsp_request *preq)
{
- u32 field, length_field, remainder;
+ u32 field, length_field, zlp = 0;
struct cdnsp_ep *pep = preq->pep;
struct cdnsp_ring *ep_ring;
int num_trbs;
+ u32 maxp;
int ret;
ep_ring = cdnsp_request_to_transfer_ring(pdev, preq);
/* 1 TRB for data, 1 for status */
num_trbs = (pdev->three_stage_setup) ? 2 : 1;
+ maxp = usb_endpoint_maxp(pep->endpoint.desc);
+
+ if (preq->request.zero && preq->request.length &&
+ (preq->request.length % maxp == 0)) {
+ num_trbs++;
+ zlp = 1;
+ }
+
ret = cdnsp_prepare_transfer(pdev, preq, num_trbs);
if (ret)
return ret;
/* If there's data, queue data TRBs */
- if (pdev->ep0_expect_in)
- field = TRB_TYPE(TRB_DATA) | TRB_IOC;
- else
- field = TRB_ISP | TRB_TYPE(TRB_DATA) | TRB_IOC;
-
if (preq->request.length > 0) {
- remainder = cdnsp_td_remainder(pdev, 0, preq->request.length,
- preq->request.length, preq, 1, 0);
+ field = TRB_TYPE(TRB_DATA);
- length_field = TRB_LEN(preq->request.length) |
- TRB_TD_SIZE(remainder) | TRB_INTR_TARGET(0);
+ if (zlp)
+ field |= TRB_CHAIN;
+ else
+ field |= TRB_IOC | (pdev->ep0_expect_in ? 0 : TRB_ISP);
if (pdev->ep0_expect_in)
field |= TRB_DIR_IN;
+ length_field = TRB_LEN(preq->request.length) |
+ TRB_TD_SIZE(zlp) | TRB_INTR_TARGET(0);
+
cdnsp_queue_trb(pdev, ep_ring, true,
lower_32_bits(preq->request.dma),
upper_32_bits(preq->request.dma), length_field,
TRB_SETUPID(pdev->setup_id) |
pdev->setup_speed);
+ if (zlp) {
+ field = TRB_TYPE(TRB_NORMAL) | TRB_IOC;
+
+ if (!pdev->ep0_expect_in)
+ field = TRB_ISP;
+
+ cdnsp_queue_trb(pdev, ep_ring, true,
+ lower_32_bits(preq->request.dma),
+ upper_32_bits(preq->request.dma), 0,
+ field | ep_ring->cycle_state |
+ TRB_SETUPID(pdev->setup_id) |
+ pdev->setup_speed);
+ }
+
pdev->ep0_stage = CDNSP_DATA_STAGE;
}
* struct ci_role_driver - host/gadget role driver
* @start: start this role
* @stop: stop this role
+ * @suspend: system suspend handler for this role
+ * @resume: system resume handler for this role
* @irq: irq handler for this role
* @name: role name string (host/gadget)
*/
struct ci_role_driver {
int (*start)(struct ci_hdrc *);
void (*stop)(struct ci_hdrc *);
+ void (*suspend)(struct ci_hdrc *ci);
+ void (*resume)(struct ci_hdrc *ci, bool power_lost);
irqreturn_t (*irq)(struct ci_hdrc *);
const char *name;
};
data->hsic_pad_regulator =
devm_regulator_get_optional(dev, "hsic");
if (PTR_ERR(data->hsic_pad_regulator) == -ENODEV) {
- /* no pad regualator is needed */
+ /* no pad regulator is needed */
data->hsic_pad_regulator = NULL;
} else if (IS_ERR(data->hsic_pad_regulator))
return dev_err_probe(dev, PTR_ERR(data->hsic_pad_regulator),
ci_hdrc_imx_remove(pdev);
}
-static int __maybe_unused imx_controller_suspend(struct device *dev)
+static int __maybe_unused imx_controller_suspend(struct device *dev,
+ pm_message_t msg)
{
struct ci_hdrc_imx_data *data = dev_get_drvdata(dev);
int ret = 0;
dev_dbg(dev, "at %s\n", __func__);
- ret = imx_usbmisc_hsic_set_clk(data->usbmisc_data, false);
+ ret = imx_usbmisc_suspend(data->usbmisc_data,
+ PMSG_IS_AUTO(msg) || device_may_wakeup(dev));
if (ret) {
- dev_err(dev, "usbmisc hsic_set_clk failed, ret=%d\n", ret);
+ dev_err(dev,
+ "usbmisc suspend failed, ret=%d\n", ret);
return ret;
}
return 0;
}
-static int __maybe_unused imx_controller_resume(struct device *dev)
+static int __maybe_unused imx_controller_resume(struct device *dev,
+ pm_message_t msg)
{
struct ci_hdrc_imx_data *data = dev_get_drvdata(dev);
int ret = 0;
data->in_lpm = false;
- ret = imx_usbmisc_set_wakeup(data->usbmisc_data, false);
+ ret = imx_usbmisc_resume(data->usbmisc_data,
+ PMSG_IS_AUTO(msg) || device_may_wakeup(dev));
if (ret) {
- dev_err(dev, "usbmisc set_wakeup failed, ret=%d\n", ret);
+ dev_err(dev, "usbmisc resume failed, ret=%d\n", ret);
goto clk_disable;
}
- ret = imx_usbmisc_hsic_set_clk(data->usbmisc_data, true);
- if (ret) {
- dev_err(dev, "usbmisc hsic_set_clk failed, ret=%d\n", ret);
- goto hsic_set_clk_fail;
- }
-
return 0;
-hsic_set_clk_fail:
- imx_usbmisc_set_wakeup(data->usbmisc_data, true);
clk_disable:
imx_disable_unprepare_clks(dev);
return ret;
/* The core's suspend doesn't run */
return 0;
- if (device_may_wakeup(dev)) {
- ret = imx_usbmisc_set_wakeup(data->usbmisc_data, true);
- if (ret) {
- dev_err(dev, "usbmisc set_wakeup failed, ret=%d\n",
- ret);
- return ret;
- }
- }
-
- ret = imx_controller_suspend(dev);
+ ret = imx_controller_suspend(dev, PMSG_SUSPEND);
if (ret)
return ret;
int ret;
pinctrl_pm_select_default_state(dev);
- ret = imx_controller_resume(dev);
+ ret = imx_controller_resume(dev, PMSG_RESUME);
if (!ret && data->supports_runtime_pm) {
pm_runtime_disable(dev);
pm_runtime_set_active(dev);
static int __maybe_unused ci_hdrc_imx_runtime_suspend(struct device *dev)
{
struct ci_hdrc_imx_data *data = dev_get_drvdata(dev);
- int ret;
if (data->in_lpm) {
WARN_ON(1);
return 0;
}
- ret = imx_usbmisc_set_wakeup(data->usbmisc_data, true);
- if (ret) {
- dev_err(dev, "usbmisc set_wakeup failed, ret=%d\n", ret);
- return ret;
- }
-
- return imx_controller_suspend(dev);
+ return imx_controller_suspend(dev, PMSG_AUTO_SUSPEND);
}
static int __maybe_unused ci_hdrc_imx_runtime_resume(struct device *dev)
{
- return imx_controller_resume(dev);
+ return imx_controller_resume(dev, PMSG_AUTO_RESUME);
}
static const struct dev_pm_ops ci_hdrc_imx_pm_ops = {
int imx_usbmisc_init(struct imx_usbmisc_data *data);
int imx_usbmisc_init_post(struct imx_usbmisc_data *data);
-int imx_usbmisc_set_wakeup(struct imx_usbmisc_data *data, bool enabled);
int imx_usbmisc_hsic_set_connect(struct imx_usbmisc_data *data);
-int imx_usbmisc_hsic_set_clk(struct imx_usbmisc_data *data, bool on);
int imx_usbmisc_charger_detection(struct imx_usbmisc_data *data, bool connect);
+int imx_usbmisc_suspend(struct imx_usbmisc_data *data, bool wakeup);
+int imx_usbmisc_resume(struct imx_usbmisc_data *data, bool wakeup);
#endif /* __DRIVER_USB_CHIPIDEA_CI_HDRC_IMX_H */
enum usb_role role)
{
struct ci_hdrc *ci = usb_role_switch_get_drvdata(sw);
- struct ci_hdrc_cable *cable = NULL;
- enum usb_role current_role = ci_role_to_usb_role(ci);
- enum ci_role ci_role = usb_role_to_ci_role(role);
- unsigned long flags;
-
- if ((ci_role != CI_ROLE_END && !ci->roles[ci_role]) ||
- (current_role == role))
- return 0;
+ struct ci_hdrc_cable *cable;
- pm_runtime_get_sync(ci->dev);
- /* Stop current role */
- spin_lock_irqsave(&ci->lock, flags);
- if (current_role == USB_ROLE_DEVICE)
+ if (role == USB_ROLE_HOST) {
+ cable = &ci->platdata->id_extcon;
+ cable->changed = true;
+ cable->connected = true;
cable = &ci->platdata->vbus_extcon;
- else if (current_role == USB_ROLE_HOST)
+ cable->changed = true;
+ cable->connected = false;
+ } else if (role == USB_ROLE_DEVICE) {
cable = &ci->platdata->id_extcon;
-
- if (cable) {
cable->changed = true;
cable->connected = false;
- ci_irq(ci);
- spin_unlock_irqrestore(&ci->lock, flags);
- if (ci->wq && role != USB_ROLE_NONE)
- flush_workqueue(ci->wq);
- spin_lock_irqsave(&ci->lock, flags);
- }
-
- cable = NULL;
-
- /* Start target role */
- if (role == USB_ROLE_DEVICE)
cable = &ci->platdata->vbus_extcon;
- else if (role == USB_ROLE_HOST)
- cable = &ci->platdata->id_extcon;
-
- if (cable) {
cable->changed = true;
cable->connected = true;
- ci_irq(ci);
+ } else {
+ cable = &ci->platdata->id_extcon;
+ cable->changed = true;
+ cable->connected = false;
+ cable = &ci->platdata->vbus_extcon;
+ cable->changed = true;
+ cable->connected = false;
}
- spin_unlock_irqrestore(&ci->lock, flags);
- pm_runtime_put_sync(ci->dev);
+ ci_irq(ci);
return 0;
}
+static enum ci_role ci_get_role(struct ci_hdrc *ci)
+{
+ enum ci_role role;
+
+ if (ci->roles[CI_ROLE_HOST] && ci->roles[CI_ROLE_GADGET]) {
+ if (ci->is_otg) {
+ role = ci_otg_role(ci);
+ hw_write_otgsc(ci, OTGSC_IDIE, OTGSC_IDIE);
+ } else {
+ /*
+ * If the controller is not OTG capable, but support
+ * role switch, the defalt role is gadget, and the
+ * user can switch it through debugfs.
+ */
+ role = CI_ROLE_GADGET;
+ }
+ } else {
+ role = ci->roles[CI_ROLE_HOST] ? CI_ROLE_HOST
+ : CI_ROLE_GADGET;
+ }
+
+ return role;
+}
+
static struct usb_role_switch_desc ci_role_switch = {
.set = ci_usb_role_switch_set,
.get = ci_usb_role_switch_get,
}
}
- if (ci->roles[CI_ROLE_HOST] && ci->roles[CI_ROLE_GADGET]) {
- if (ci->is_otg) {
- ci->role = ci_otg_role(ci);
- /* Enable ID change irq */
- hw_write_otgsc(ci, OTGSC_IDIE, OTGSC_IDIE);
- } else {
- /*
- * If the controller is not OTG capable, but support
- * role switch, the defalt role is gadget, and the
- * user can switch it through debugfs.
- */
- ci->role = CI_ROLE_GADGET;
- }
- } else {
- ci->role = ci->roles[CI_ROLE_HOST]
- ? CI_ROLE_HOST
- : CI_ROLE_GADGET;
- }
-
+ ci->role = ci_get_role(ci);
if (!ci_otg_is_fsm_mode(ci)) {
/* only update vbus status for peripheral */
if (ci->role == CI_ROLE_GADGET) {
cable_id = &ci->platdata->id_extcon;
cable_vbus = &ci->platdata->vbus_extcon;
- if (!IS_ERR(cable_id->edev) && ci->is_otg &&
+ if ((!IS_ERR(cable_id->edev) || !IS_ERR(ci->role_switch))
+ && ci->is_otg &&
(otgsc & OTGSC_IDIE) && (otgsc & OTGSC_IDIS))
ci_irq(ci);
- if (!IS_ERR(cable_vbus->edev) && ci->is_otg &&
+ if ((!IS_ERR(cable_vbus->edev) || !IS_ERR(ci->role_switch))
+ && ci->is_otg &&
(otgsc & OTGSC_BSVIE) && (otgsc & OTGSC_BSVIS))
ci_irq(ci);
}
return 0;
}
+ /* Extra routine per role before system suspend */
+ if (ci->role != CI_ROLE_END && ci_role(ci)->suspend)
+ ci_role(ci)->suspend(ci);
+
if (device_may_wakeup(dev)) {
if (ci_otg_is_fsm_mode(ci))
ci_otg_fsm_suspend_for_srp(ci);
return 0;
}
+static void ci_handle_power_lost(struct ci_hdrc *ci)
+{
+ enum ci_role role;
+
+ disable_irq_nosync(ci->irq);
+ if (!ci_otg_is_fsm_mode(ci)) {
+ role = ci_get_role(ci);
+
+ if (ci->role != role) {
+ ci_handle_id_switch(ci);
+ } else if (role == CI_ROLE_GADGET) {
+ if (ci->is_otg && hw_read_otgsc(ci, OTGSC_BSV))
+ usb_gadget_vbus_connect(&ci->gadget);
+ }
+ }
+
+ enable_irq(ci->irq);
+}
+
static int ci_resume(struct device *dev)
{
struct ci_hdrc *ci = dev_get_drvdata(dev);
+ bool power_lost;
int ret;
+ /* Since ASYNCLISTADDR (host mode) and ENDPTLISTADDR (device
+ * mode) share the same register address. We can check if
+ * controller resume from power lost based on this address
+ * due to this register will be reset after power lost.
+ */
+ power_lost = !hw_read(ci, OP_ENDPTLISTADDR, ~0);
+
if (device_may_wakeup(dev))
disable_irq_wake(ci->irq);
if (ret)
return ret;
+ if (power_lost) {
+ /* shutdown and re-init for phy */
+ ci_usb_phy_exit(ci);
+ ci_usb_phy_init(ci);
+ }
+
+ /* Extra routine per role after system resume */
+ if (ci->role != CI_ROLE_END && ci_role(ci)->resume)
+ ci_role(ci)->resume(ci, power_lost);
+
+ if (power_lost)
+ ci_handle_power_lost(ci);
+
if (ci->supports_runtime_pm) {
pm_runtime_disable(dev);
pm_runtime_set_active(dev);
ci_hdrc_free_dma_aligned_buffer(urb);
}
+#ifdef CONFIG_PM_SLEEP
+static void ci_hdrc_host_suspend(struct ci_hdrc *ci)
+{
+ ehci_suspend(ci->hcd, device_may_wakeup(ci->dev));
+}
+
+static void ci_hdrc_host_resume(struct ci_hdrc *ci, bool power_lost)
+{
+ ehci_resume(ci->hcd, power_lost);
+}
+#endif
+
int ci_hdrc_host_init(struct ci_hdrc *ci)
{
struct ci_role_driver *rdrv;
rdrv->start = host_start;
rdrv->stop = host_stop;
+#ifdef CONFIG_PM_SLEEP
+ rdrv->suspend = ci_hdrc_host_suspend;
+ rdrv->resume = ci_hdrc_host_resume;
+#endif
rdrv->irq = host_irq;
rdrv->name = "host";
ci->roles[CI_ROLE_HOST] = rdrv;
return 0;
}
-static void ci_handle_id_switch(struct ci_hdrc *ci)
+void ci_handle_id_switch(struct ci_hdrc *ci)
{
enum ci_role role = ci_otg_role(ci);
void ci_hdrc_otg_destroy(struct ci_hdrc *ci);
enum ci_role ci_otg_role(struct ci_hdrc *ci);
void ci_handle_vbus_change(struct ci_hdrc *ci);
+void ci_handle_id_switch(struct ci_hdrc *ci);
static inline void ci_otg_queue_work(struct ci_hdrc *ci)
{
disable_irq_nosync(ci->irq);
ci->platdata->pins_default);
}
+#ifdef CONFIG_PM_SLEEP
+static void udc_suspend(struct ci_hdrc *ci)
+{
+ /*
+ * Set OP_ENDPTLISTADDR to be non-zero for
+ * checking if controller resume from power lost
+ * in non-host mode.
+ */
+ if (hw_read(ci, OP_ENDPTLISTADDR, ~0) == 0)
+ hw_write(ci, OP_ENDPTLISTADDR, ~0, ~0);
+}
+
+static void udc_resume(struct ci_hdrc *ci, bool power_lost)
+{
+ if (power_lost) {
+ if (ci->is_otg)
+ hw_write_otgsc(ci, OTGSC_BSVIS | OTGSC_BSVIE,
+ OTGSC_BSVIS | OTGSC_BSVIE);
+ if (ci->vbus_active)
+ usb_gadget_vbus_disconnect(&ci->gadget);
+ }
+
+ /* Restore value 0 if it was set for power lost check */
+ if (hw_read(ci, OP_ENDPTLISTADDR, ~0) == 0xFFFFFFFF)
+ hw_write(ci, OP_ENDPTLISTADDR, ~0, 0);
+}
+#endif
+
/**
* ci_hdrc_gadget_init - initialize device related bits
* @ci: the controller
rdrv->start = udc_id_switch_for_device;
rdrv->stop = udc_id_switch_for_host;
+#ifdef CONFIG_PM_SLEEP
+ rdrv->suspend = udc_suspend;
+ rdrv->resume = udc_resume;
+#endif
rdrv->irq = udc_irq;
rdrv->name = "gadget";
int (*hsic_set_clk)(struct imx_usbmisc_data *data, bool enabled);
/* usb charger detection */
int (*charger_detection)(struct imx_usbmisc_data *data);
+ /* It's called when system resume from usb power lost */
+ int (*power_lost_check)(struct imx_usbmisc_data *data);
};
struct imx_usbmisc {
return 0;
}
+static int usbmisc_imx7d_power_lost_check(struct imx_usbmisc_data *data)
+{
+ struct imx_usbmisc *usbmisc = dev_get_drvdata(data->dev);
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&usbmisc->lock, flags);
+ val = readl(usbmisc->base);
+ spin_unlock_irqrestore(&usbmisc->lock, flags);
+ /*
+ * Here use a power on reset value to judge
+ * if the controller experienced a power lost
+ */
+ if (val == 0x30001000)
+ return 1;
+ else
+ return 0;
+}
+
+static int usbmisc_imx6sx_power_lost_check(struct imx_usbmisc_data *data)
+{
+ struct imx_usbmisc *usbmisc = dev_get_drvdata(data->dev);
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&usbmisc->lock, flags);
+ val = readl(usbmisc->base + data->index * 4);
+ spin_unlock_irqrestore(&usbmisc->lock, flags);
+ /*
+ * Here use a power on reset value to judge
+ * if the controller experienced a power lost
+ */
+ if (val == 0x30001000)
+ return 1;
+ else
+ return 0;
+}
+
static const struct usbmisc_ops imx25_usbmisc_ops = {
.init = usbmisc_imx25_init,
.post = usbmisc_imx25_post,
.init = usbmisc_imx6sx_init,
.hsic_set_connect = usbmisc_imx6_hsic_set_connect,
.hsic_set_clk = usbmisc_imx6_hsic_set_clk,
+ .power_lost_check = usbmisc_imx6sx_power_lost_check,
};
static const struct usbmisc_ops imx7d_usbmisc_ops = {
.init = usbmisc_imx7d_init,
.set_wakeup = usbmisc_imx7d_set_wakeup,
.charger_detection = imx7d_charger_detection,
+ .power_lost_check = usbmisc_imx7d_power_lost_check,
};
static const struct usbmisc_ops imx7ulp_usbmisc_ops = {
.set_wakeup = usbmisc_imx7d_set_wakeup,
.hsic_set_connect = usbmisc_imx6_hsic_set_connect,
.hsic_set_clk = usbmisc_imx6_hsic_set_clk,
+ .power_lost_check = usbmisc_imx7d_power_lost_check,
};
static inline bool is_imx53_usbmisc(struct imx_usbmisc_data *data)
int imx_usbmisc_init_post(struct imx_usbmisc_data *data)
{
struct imx_usbmisc *usbmisc;
+ int ret = 0;
if (!data)
return 0;
usbmisc = dev_get_drvdata(data->dev);
- if (!usbmisc->ops->post)
- return 0;
- return usbmisc->ops->post(data);
-}
-EXPORT_SYMBOL_GPL(imx_usbmisc_init_post);
-
-int imx_usbmisc_set_wakeup(struct imx_usbmisc_data *data, bool enabled)
-{
- struct imx_usbmisc *usbmisc;
+ if (usbmisc->ops->post)
+ ret = usbmisc->ops->post(data);
+ if (ret) {
+ dev_err(data->dev, "post init failed, ret=%d\n", ret);
+ return ret;
+ }
- if (!data)
- return 0;
+ if (usbmisc->ops->set_wakeup)
+ ret = usbmisc->ops->set_wakeup(data, false);
+ if (ret) {
+ dev_err(data->dev, "set_wakeup failed, ret=%d\n", ret);
+ return ret;
+ }
- usbmisc = dev_get_drvdata(data->dev);
- if (!usbmisc->ops->set_wakeup)
- return 0;
- return usbmisc->ops->set_wakeup(data, enabled);
+ return 0;
}
-EXPORT_SYMBOL_GPL(imx_usbmisc_set_wakeup);
+EXPORT_SYMBOL_GPL(imx_usbmisc_init_post);
int imx_usbmisc_hsic_set_connect(struct imx_usbmisc_data *data)
{
}
EXPORT_SYMBOL_GPL(imx_usbmisc_hsic_set_connect);
-int imx_usbmisc_hsic_set_clk(struct imx_usbmisc_data *data, bool on)
-{
- struct imx_usbmisc *usbmisc;
-
- if (!data)
- return 0;
-
- usbmisc = dev_get_drvdata(data->dev);
- if (!usbmisc->ops->hsic_set_clk || !data->hsic)
- return 0;
- return usbmisc->ops->hsic_set_clk(data, on);
-}
-EXPORT_SYMBOL_GPL(imx_usbmisc_hsic_set_clk);
-
int imx_usbmisc_charger_detection(struct imx_usbmisc_data *data, bool connect)
{
struct imx_usbmisc *usbmisc;
}
EXPORT_SYMBOL_GPL(imx_usbmisc_charger_detection);
+int imx_usbmisc_suspend(struct imx_usbmisc_data *data, bool wakeup)
+{
+ struct imx_usbmisc *usbmisc;
+ int ret = 0;
+
+ if (!data)
+ return 0;
+
+ usbmisc = dev_get_drvdata(data->dev);
+
+ if (wakeup && usbmisc->ops->set_wakeup)
+ ret = usbmisc->ops->set_wakeup(data, true);
+ if (ret) {
+ dev_err(data->dev, "set_wakeup failed, ret=%d\n", ret);
+ return ret;
+ }
+
+ if (usbmisc->ops->hsic_set_clk && data->hsic)
+ ret = usbmisc->ops->hsic_set_clk(data, false);
+ if (ret) {
+ dev_err(data->dev, "set_wakeup failed, ret=%d\n", ret);
+ return ret;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(imx_usbmisc_suspend);
+
+int imx_usbmisc_resume(struct imx_usbmisc_data *data, bool wakeup)
+{
+ struct imx_usbmisc *usbmisc;
+ int ret = 0;
+
+ if (!data)
+ return 0;
+
+ usbmisc = dev_get_drvdata(data->dev);
+
+ if (usbmisc->ops->power_lost_check)
+ ret = usbmisc->ops->power_lost_check(data);
+ if (ret > 0) {
+ /* re-init if resume from power lost */
+ ret = imx_usbmisc_init(data);
+ if (ret) {
+ dev_err(data->dev, "re-init failed, ret=%d\n", ret);
+ return ret;
+ }
+ }
+
+ if (wakeup && usbmisc->ops->set_wakeup)
+ ret = usbmisc->ops->set_wakeup(data, false);
+ if (ret) {
+ dev_err(data->dev, "set_wakeup failed, ret=%d\n", ret);
+ return ret;
+ }
+
+ if (usbmisc->ops->hsic_set_clk && data->hsic)
+ ret = usbmisc->ops->hsic_set_clk(data, true);
+ if (ret) {
+ dev_err(data->dev, "set_wakeup failed, ret=%d\n", ret);
+ goto hsic_set_clk_fail;
+ }
+
+ return 0;
+
+hsic_set_clk_fail:
+ if (wakeup && usbmisc->ops->set_wakeup)
+ usbmisc->ops->set_wakeup(data, true);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(imx_usbmisc_resume);
+
static const struct of_device_id usbmisc_imx_dt_ids[] = {
{
.compatible = "fsl,imx25-usbmisc",
.llseek = noop_llseek,
};
-static char *usblp_devnode(struct device *dev, umode_t *mode)
+static char *usblp_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "usb/%s", dev_name(dev));
}
/* Test the interface */
ret = ulpi_write(ulpi, ULPI_SCRATCH, 0xaa);
if (ret < 0)
- goto err;
+ return ret;
ret = ulpi_read(ulpi, ULPI_SCRATCH);
if (ret < 0)
desc = (struct usb_ssp_isoc_ep_comp_descriptor *) buffer;
if (desc->bDescriptorType != USB_DT_SSP_ISOC_ENDPOINT_COMP ||
size < USB_DT_SSP_ISOC_EP_COMP_SIZE) {
- dev_warn(ddev, "Invalid SuperSpeedPlus isoc endpoint companion"
+ dev_notice(ddev, "Invalid SuperSpeedPlus isoc endpoint companion"
"for config %d interface %d altsetting %d ep %d.\n",
cfgno, inum, asnum, ep->desc.bEndpointAddress);
return;
if (desc->bDescriptorType != USB_DT_SS_ENDPOINT_COMP ||
size < USB_DT_SS_EP_COMP_SIZE) {
- dev_warn(ddev, "No SuperSpeed endpoint companion for config %d "
+ dev_notice(ddev, "No SuperSpeed endpoint companion for config %d "
" interface %d altsetting %d ep %d: "
"using minimum values\n",
cfgno, inum, asnum, ep->desc.bEndpointAddress);
/* Check the various values */
if (usb_endpoint_xfer_control(&ep->desc) && desc->bMaxBurst != 0) {
- dev_warn(ddev, "Control endpoint with bMaxBurst = %d in "
+ dev_notice(ddev, "Control endpoint with bMaxBurst = %d in "
"config %d interface %d altsetting %d ep %d: "
"setting to zero\n", desc->bMaxBurst,
cfgno, inum, asnum, ep->desc.bEndpointAddress);
ep->ss_ep_comp.bMaxBurst = 0;
} else if (desc->bMaxBurst > 15) {
- dev_warn(ddev, "Endpoint with bMaxBurst = %d in "
+ dev_notice(ddev, "Endpoint with bMaxBurst = %d in "
"config %d interface %d altsetting %d ep %d: "
"setting to 15\n", desc->bMaxBurst,
cfgno, inum, asnum, ep->desc.bEndpointAddress);
if ((usb_endpoint_xfer_control(&ep->desc) ||
usb_endpoint_xfer_int(&ep->desc)) &&
desc->bmAttributes != 0) {
- dev_warn(ddev, "%s endpoint with bmAttributes = %d in "
+ dev_notice(ddev, "%s endpoint with bmAttributes = %d in "
"config %d interface %d altsetting %d ep %d: "
"setting to zero\n",
usb_endpoint_xfer_control(&ep->desc) ? "Control" : "Bulk",
ep->ss_ep_comp.bmAttributes = 0;
} else if (usb_endpoint_xfer_bulk(&ep->desc) &&
desc->bmAttributes > 16) {
- dev_warn(ddev, "Bulk endpoint with more than 65536 streams in "
+ dev_notice(ddev, "Bulk endpoint with more than 65536 streams in "
"config %d interface %d altsetting %d ep %d: "
"setting to max\n",
cfgno, inum, asnum, ep->desc.bEndpointAddress);
} else if (usb_endpoint_xfer_isoc(&ep->desc) &&
!USB_SS_SSP_ISOC_COMP(desc->bmAttributes) &&
USB_SS_MULT(desc->bmAttributes) > 3) {
- dev_warn(ddev, "Isoc endpoint has Mult of %d in "
+ dev_notice(ddev, "Isoc endpoint has Mult of %d in "
"config %d interface %d altsetting %d ep %d: "
"setting to 3\n",
USB_SS_MULT(desc->bmAttributes),
else
max_tx = 999999;
if (le16_to_cpu(desc->wBytesPerInterval) > max_tx) {
- dev_warn(ddev, "%s endpoint with wBytesPerInterval of %d in "
+ dev_notice(ddev, "%s endpoint with wBytesPerInterval of %d in "
"config %d interface %d altsetting %d ep %d: "
"setting to %d\n",
usb_endpoint_xfer_isoc(&ep->desc) ? "Isoc" : "Int",
else if (d->bLength >= USB_DT_ENDPOINT_SIZE)
n = USB_DT_ENDPOINT_SIZE;
else {
- dev_warn(ddev, "config %d interface %d altsetting %d has an "
+ dev_notice(ddev, "config %d interface %d altsetting %d has an "
"invalid endpoint descriptor of length %d, skipping\n",
cfgno, inum, asnum, d->bLength);
goto skip_to_next_endpoint_or_interface_descriptor;
i = d->bEndpointAddress & ~USB_ENDPOINT_DIR_MASK;
if (i >= 16 || i == 0) {
- dev_warn(ddev, "config %d interface %d altsetting %d has an "
+ dev_notice(ddev, "config %d interface %d altsetting %d has an "
"invalid endpoint with address 0x%X, skipping\n",
cfgno, inum, asnum, d->bEndpointAddress);
goto skip_to_next_endpoint_or_interface_descriptor;
/* Check for duplicate endpoint addresses */
if (config_endpoint_is_duplicate(config, inum, asnum, d)) {
- dev_warn(ddev, "config %d interface %d altsetting %d has a duplicate endpoint with address 0x%X, skipping\n",
+ dev_notice(ddev, "config %d interface %d altsetting %d has a duplicate endpoint with address 0x%X, skipping\n",
cfgno, inum, asnum, d->bEndpointAddress);
goto skip_to_next_endpoint_or_interface_descriptor;
}
/* Ignore some endpoints */
if (udev->quirks & USB_QUIRK_ENDPOINT_IGNORE) {
if (usb_endpoint_is_ignored(udev, ifp, d)) {
- dev_warn(ddev, "config %d interface %d altsetting %d has an ignored endpoint with address 0x%X, skipping\n",
+ dev_notice(ddev, "config %d interface %d altsetting %d has an ignored endpoint with address 0x%X, skipping\n",
cfgno, inum, asnum,
d->bEndpointAddress);
goto skip_to_next_endpoint_or_interface_descriptor;
}
}
if (d->bInterval < i || d->bInterval > j) {
- dev_warn(ddev, "config %d interface %d altsetting %d "
+ dev_notice(ddev, "config %d interface %d altsetting %d "
"endpoint 0x%X has an invalid bInterval %d, "
"changing to %d\n",
cfgno, inum, asnum,
* them usable, we will try treating them as Interrupt endpoints.
*/
if (udev->speed == USB_SPEED_LOW && usb_endpoint_xfer_bulk(d)) {
- dev_warn(ddev, "config %d interface %d altsetting %d "
+ dev_notice(ddev, "config %d interface %d altsetting %d "
"endpoint 0x%X is Bulk; changing to Interrupt\n",
cfgno, inum, asnum, d->bEndpointAddress);
endpoint->desc.bmAttributes = USB_ENDPOINT_XFER_INT;
*/
maxp = le16_to_cpu(endpoint->desc.wMaxPacketSize);
if (maxp == 0 && !(usb_endpoint_xfer_isoc(d) && asnum == 0)) {
- dev_warn(ddev, "config %d interface %d altsetting %d endpoint 0x%X has invalid wMaxPacketSize 0\n",
+ dev_notice(ddev, "config %d interface %d altsetting %d endpoint 0x%X has invalid wMaxPacketSize 0\n",
cfgno, inum, asnum, d->bEndpointAddress);
}
j = maxpacket_maxes[usb_endpoint_type(&endpoint->desc)];
if (maxp > j) {
- dev_warn(ddev, "config %d interface %d altsetting %d endpoint 0x%X has invalid maxpacket %d, setting to %d\n",
+ dev_notice(ddev, "config %d interface %d altsetting %d endpoint 0x%X has invalid maxpacket %d, setting to %d\n",
cfgno, inum, asnum, d->bEndpointAddress, maxp, j);
maxp = j;
endpoint->desc.wMaxPacketSize = cpu_to_le16(i | maxp);
*/
if (udev->speed == USB_SPEED_HIGH && usb_endpoint_xfer_bulk(d)) {
if (maxp != 512)
- dev_warn(ddev, "config %d interface %d altsetting %d "
+ dev_notice(ddev, "config %d interface %d altsetting %d "
"bulk endpoint 0x%X has invalid maxpacket %d\n",
cfgno, inum, asnum, d->bEndpointAddress,
maxp);
i < intfc->num_altsetting;
(++i, ++alt)) {
if (alt->desc.bAlternateSetting == asnum) {
- dev_warn(ddev, "Duplicate descriptor for config %d "
+ dev_notice(ddev, "Duplicate descriptor for config %d "
"interface %d altsetting %d, skipping\n",
cfgno, inum, asnum);
goto skip_to_next_interface_descriptor;
num_ep = num_ep_orig = alt->desc.bNumEndpoints;
alt->desc.bNumEndpoints = 0; /* Use as a counter */
if (num_ep > USB_MAXENDPOINTS) {
- dev_warn(ddev, "too many endpoints for config %d interface %d "
+ dev_notice(ddev, "too many endpoints for config %d interface %d "
"altsetting %d: %d, using maximum allowed: %d\n",
cfgno, inum, asnum, num_ep, USB_MAXENDPOINTS);
num_ep = USB_MAXENDPOINTS;
}
if (n != num_ep_orig)
- dev_warn(ddev, "config %d interface %d altsetting %d has %d "
+ dev_notice(ddev, "config %d interface %d altsetting %d has %d "
"endpoint descriptor%s, different from the interface "
"descriptor's value: %d\n",
cfgno, inum, asnum, n, plural(n), num_ep_orig);
if (config->desc.bDescriptorType != USB_DT_CONFIG ||
config->desc.bLength < USB_DT_CONFIG_SIZE ||
config->desc.bLength > size) {
- dev_err(ddev, "invalid descriptor for config index %d: "
+ dev_notice(ddev, "invalid descriptor for config index %d: "
"type = 0x%X, length = %d\n", cfgidx,
config->desc.bDescriptorType, config->desc.bLength);
return -EINVAL;
size -= config->desc.bLength;
if (nintf > USB_MAXINTERFACES) {
- dev_warn(ddev, "config %d has too many interfaces: %d, "
+ dev_notice(ddev, "config %d has too many interfaces: %d, "
"using maximum allowed: %d\n",
cfgno, nintf, USB_MAXINTERFACES);
nintf = USB_MAXINTERFACES;
(buffer2 += header->bLength, size2 -= header->bLength)) {
if (size2 < sizeof(struct usb_descriptor_header)) {
- dev_warn(ddev, "config %d descriptor has %d excess "
+ dev_notice(ddev, "config %d descriptor has %d excess "
"byte%s, ignoring\n",
cfgno, size2, plural(size2));
break;
header = (struct usb_descriptor_header *) buffer2;
if ((header->bLength > size2) || (header->bLength < 2)) {
- dev_warn(ddev, "config %d has an invalid descriptor "
+ dev_notice(ddev, "config %d has an invalid descriptor "
"of length %d, skipping remainder of the config\n",
cfgno, header->bLength);
break;
d = (struct usb_interface_descriptor *) header;
if (d->bLength < USB_DT_INTERFACE_SIZE) {
- dev_warn(ddev, "config %d has an invalid "
+ dev_notice(ddev, "config %d has an invalid "
"interface descriptor of length %d, "
"skipping\n", cfgno, d->bLength);
continue;
if ((dev->quirks & USB_QUIRK_HONOR_BNUMINTERFACES) &&
n >= nintf_orig) {
- dev_warn(ddev, "config %d has more interface "
+ dev_notice(ddev, "config %d has more interface "
"descriptors, than it declares in "
"bNumInterfaces, ignoring interface "
"number: %d\n", cfgno, inum);
}
if (inum >= nintf_orig)
- dev_warn(ddev, "config %d has an invalid "
+ dev_notice(ddev, "config %d has an invalid "
"interface number: %d but max is %d\n",
cfgno, inum, nintf_orig - 1);
d = (struct usb_interface_assoc_descriptor *)header;
if (d->bLength < USB_DT_INTERFACE_ASSOCIATION_SIZE) {
- dev_warn(ddev,
+ dev_notice(ddev,
"config %d has an invalid interface association descriptor of length %d, skipping\n",
cfgno, d->bLength);
continue;
}
if (iad_num == USB_MAXIADS) {
- dev_warn(ddev, "found more Interface "
+ dev_notice(ddev, "found more Interface "
"Association Descriptors "
"than allocated for in "
"configuration %d\n", cfgno);
} else if (header->bDescriptorType == USB_DT_DEVICE ||
header->bDescriptorType == USB_DT_CONFIG)
- dev_warn(ddev, "config %d contains an unexpected "
+ dev_notice(ddev, "config %d contains an unexpected "
"descriptor of type 0x%X, skipping\n",
cfgno, header->bDescriptorType);
config->desc.wTotalLength = cpu_to_le16(buffer2 - buffer0);
if (n != nintf)
- dev_warn(ddev, "config %d has %d interface%s, different from "
+ dev_notice(ddev, "config %d has %d interface%s, different from "
"the descriptor's value: %d\n",
cfgno, n, plural(n), nintf_orig);
else if (n == 0)
- dev_warn(ddev, "config %d has no interfaces?\n", cfgno);
+ dev_notice(ddev, "config %d has no interfaces?\n", cfgno);
config->desc.bNumInterfaces = nintf = n;
/* Check for missing interface numbers */
break;
}
if (j >= nintf)
- dev_warn(ddev, "config %d has no interface number "
+ dev_notice(ddev, "config %d has no interface number "
"%d\n", cfgno, i);
}
for (i = 0; i < nintf; ++i) {
j = nalts[i];
if (j > USB_MAXALTSETTING) {
- dev_warn(ddev, "too many alternate settings for "
+ dev_notice(ddev, "too many alternate settings for "
"config %d interface %d: %d, "
"using maximum allowed: %d\n",
cfgno, inums[i], j, USB_MAXALTSETTING);
break;
}
if (n >= intfc->num_altsetting)
- dev_warn(ddev, "config %d interface %d has no "
+ dev_notice(ddev, "config %d interface %d has no "
"altsetting %d\n", cfgno, inums[i], j);
}
}
int result;
if (ncfg > USB_MAXCONFIG) {
- dev_warn(ddev, "too many configurations: %d, "
+ dev_notice(ddev, "too many configurations: %d, "
"using maximum allowed: %d\n", ncfg, USB_MAXCONFIG);
dev->descriptor.bNumConfigurations = ncfg = USB_MAXCONFIG;
}
"descriptor/%s: %d\n", cfgno, "start", result);
if (result != -EPIPE)
goto err;
- dev_err(ddev, "chopping to %d config(s)\n", cfgno);
+ dev_notice(ddev, "chopping to %d config(s)\n", cfgno);
dev->descriptor.bNumConfigurations = cfgno;
break;
} else if (result < 4) {
goto err;
}
if (result < length) {
- dev_warn(ddev, "config index %d descriptor too short "
+ dev_notice(ddev, "config index %d descriptor too short "
"(expected %i, got %i)\n", cfgno, length, result);
length = result;
}
/* Get BOS descriptor */
ret = usb_get_descriptor(dev, USB_DT_BOS, 0, bos, USB_DT_BOS_SIZE);
if (ret < USB_DT_BOS_SIZE || bos->bLength < USB_DT_BOS_SIZE) {
- dev_err(ddev, "unable to get BOS descriptor or descriptor too short\n");
+ dev_notice(ddev, "unable to get BOS descriptor or descriptor too short\n");
if (ret >= 0)
ret = -ENOMSG;
kfree(bos);
ret = usb_get_descriptor(dev, USB_DT_BOS, 0, buffer, total_len);
if (ret < total_len) {
- dev_err(ddev, "unable to get BOS descriptor set\n");
+ dev_notice(ddev, "unable to get BOS descriptor set\n");
if (ret >= 0)
ret = -ENOMSG;
goto err;
}
if (cap->bDescriptorType != USB_DT_DEVICE_CAPABILITY) {
- dev_warn(ddev, "descriptor type invalid, skip\n");
+ dev_notice(ddev, "descriptor type invalid, skip\n");
continue;
}
struct class *class;
} *usb_class;
-static char *usb_devnode(struct device *dev, umode_t *mode)
+static char *usb_devnode(const struct device *dev, umode_t *mode)
{
struct usb_class_driver *drv;
return retval;
}
+static int hcd_pci_poweroff_late(struct device *dev)
+{
+ struct pci_dev *pci_dev = to_pci_dev(dev);
+ struct usb_hcd *hcd = pci_get_drvdata(pci_dev);
+
+ if (hcd->driver->pci_poweroff_late && !HCD_DEAD(hcd))
+ return hcd->driver->pci_poweroff_late(hcd, device_may_wakeup(dev));
+
+ return 0;
+}
+
static int hcd_pci_resume_noirq(struct device *dev)
{
powermac_set_asic(to_pci_dev(dev), 1);
#define hcd_pci_suspend NULL
#define hcd_pci_suspend_noirq NULL
+#define hcd_pci_poweroff_late NULL
#define hcd_pci_resume_noirq NULL
#define hcd_pci_resume NULL
#define hcd_pci_restore NULL
.thaw_noirq = NULL,
.thaw = hcd_pci_resume,
.poweroff = hcd_pci_suspend,
+ .poweroff_late = hcd_pci_poweroff_late,
.poweroff_noirq = hcd_pci_suspend_noirq,
.restore_noirq = hcd_pci_resume_noirq,
.restore = hcd_pci_restore,
GFP_KERNEL,
DMA_ATTR_WRITE_COMBINE);
- if (IS_ERR(local_mem))
+ if (IS_ERR_OR_NULL(local_mem)) {
+ if (!local_mem)
+ return -ENOMEM;
+
return PTR_ERR(local_mem);
+ }
/*
* Here we pass a dma_addr_t but the arg type is a phys_addr_t.
return status;
}
+/*
+ * hub_port_stop_enumerate - stop USB enumeration or ignore port events
+ * @hub: target hub
+ * @port1: port num of the port
+ * @retries: port retries number of hub_port_init()
+ *
+ * Return:
+ * true: ignore port actions/events or give up connection attempts.
+ * false: keep original behavior.
+ *
+ * This function will be based on retries to check whether the port which is
+ * marked with early_stop attribute would stop enumeration or ignore events.
+ *
+ * Note:
+ * This function didn't change anything if early_stop is not set, and it will
+ * prevent all connection attempts when early_stop is set and the attempts of
+ * the port are more than 1.
+ */
+static bool hub_port_stop_enumerate(struct usb_hub *hub, int port1, int retries)
+{
+ struct usb_port *port_dev = hub->ports[port1 - 1];
+
+ if (port_dev->early_stop) {
+ if (port_dev->ignore_event)
+ return true;
+
+ /*
+ * We want unsuccessful attempts to fail quickly.
+ * Since some devices may need one failure during
+ * port initialization, we allow two tries but no
+ * more.
+ */
+ if (retries < 2)
+ return false;
+
+ port_dev->ignore_event = 1;
+ } else
+ port_dev->ignore_event = 0;
+
+ return port_dev->ignore_event;
+}
+
/* Check if a port is power on */
int usb_port_is_power_on(struct usb_hub *hub, unsigned int portstatus)
{
do_new_scheme = use_new_scheme(udev, retry_counter, port_dev);
for (retries = 0; retries < GET_DESCRIPTOR_TRIES; (++retries, msleep(100))) {
+ if (hub_port_stop_enumerate(hub, port1, retries)) {
+ retval = -ENODEV;
+ break;
+ }
+
if (do_new_scheme) {
struct usb_device_descriptor *buf;
int r = 0;
status = 0;
for (i = 0; i < PORT_INIT_TRIES; i++) {
+ if (hub_port_stop_enumerate(hub, port1, i)) {
+ status = -ENODEV;
+ break;
+ }
+
usb_lock_port(port_dev);
mutex_lock(hcd->address0_mutex);
retry_locked = true;
if (!pm_runtime_active(&port_dev->dev))
return;
+ /* skip port actions if ignore_event and early_stop are true */
+ if (port_dev->ignore_event && port_dev->early_stop)
+ return;
+
if (hub_handle_remote_wakeup(hub, port1, portstatus, portchange))
connect_change = 1;
mutex_lock(hcd->address0_mutex);
for (i = 0; i < PORT_INIT_TRIES; ++i) {
+ if (hub_port_stop_enumerate(parent_hub, port1, i)) {
+ ret = -ENODEV;
+ break;
+ }
/* ep0 maxpacket size may change; let the HCD know about it.
* Other endpoints will be handled by re-enumeration. */
* @is_superspeed cache super-speed status
* @usb3_lpm_u1_permit: whether USB3 U1 LPM is permitted.
* @usb3_lpm_u2_permit: whether USB3 U2 LPM is permitted.
+ * @early_stop: whether port initialization will be stopped earlier.
+ * @ignore_event: whether events of the port are ignored.
*/
struct usb_port {
struct usb_device *child;
u32 over_current_count;
u8 portnum;
u32 quirks;
+ unsigned int early_stop:1;
+ unsigned int ignore_event:1;
unsigned int is_superspeed:1;
unsigned int usb3_lpm_u1_permit:1;
unsigned int usb3_lpm_u2_permit:1;
* Author: Lan Tianyu <tianyu.lan@intel.com>
*/
+#include <linux/kstrtox.h>
#include <linux/slab.h>
#include <linux/pm_qos.h>
#include <linux/component.h>
static const struct attribute_group *port_dev_group[];
+static ssize_t early_stop_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct usb_port *port_dev = to_usb_port(dev);
+
+ return sysfs_emit(buf, "%s\n", port_dev->early_stop ? "yes" : "no");
+}
+
+static ssize_t early_stop_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct usb_port *port_dev = to_usb_port(dev);
+ bool value;
+
+ if (kstrtobool(buf, &value))
+ return -EINVAL;
+
+ if (value)
+ port_dev->early_stop = 1;
+ else
+ port_dev->early_stop = 0;
+
+ return count;
+}
+static DEVICE_ATTR_RW(early_stop);
+
static ssize_t disable_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
bool disabled;
int rc;
- rc = strtobool(buf, &disabled);
+ rc = kstrtobool(buf, &disabled);
if (rc)
return rc;
&dev_attr_quirks.attr,
&dev_attr_over_current_count.attr,
&dev_attr_disable.attr,
+ &dev_attr_early_stop.attr,
NULL,
};
#include <linux/kernel.h>
+#include <linux/kstrtox.h>
#include <linux/string.h>
#include <linux/usb.h>
#include <linux/usb/hcd.h>
if (ret < 0)
return -EINTR;
- ret = strtobool(buf, &value);
+ ret = kstrtobool(buf, &value);
if (!ret) {
udev->usb2_hw_lpm_allowed = value;
int rc = count;
bool val;
- if (strtobool(buf, &val) != 0)
+ if (kstrtobool(buf, &val) != 0)
return -EINVAL;
if (val)
struct usb_interface *intf = to_usb_interface(dev);
bool val;
- if (strtobool(buf, &val) != 0)
+ if (kstrtobool(buf, &val) != 0)
return -EINVAL;
if (val)
hsotg->gadget.dev.of_node = hsotg->dev->of_node;
hsotg->gadget.speed = USB_SPEED_UNKNOWN;
- if (hsotg->dr_mode == USB_DR_MODE_PERIPHERAL) {
+ if (hsotg->dr_mode == USB_DR_MODE_PERIPHERAL ||
+ (hsotg->dr_mode == USB_DR_MODE_OTG && dwc2_is_device_mode(hsotg))) {
ret = dwc2_lowlevel_hw_enable(hsotg);
if (ret)
goto err;
if (!IS_ERR_OR_NULL(hsotg->uphy))
otg_set_peripheral(hsotg->uphy->otg, NULL);
- if (hsotg->dr_mode == USB_DR_MODE_PERIPHERAL)
+ if (hsotg->dr_mode == USB_DR_MODE_PERIPHERAL ||
+ (hsotg->dr_mode == USB_DR_MODE_OTG && dwc2_is_device_mode(hsotg)))
dwc2_lowlevel_hw_disable(hsotg);
return 0;
p->ahbcfg = GAHBCFG_HBSTLEN_INCR16 <<
GAHBCFG_HBSTLEN_SHIFT;
p->power_down = DWC2_POWER_DOWN_PARAM_NONE;
+ p->lpm = false;
+ p->lpm_clock_gating = false;
+ p->besl = false;
+ p->hird_threshold_en = false;
}
static void dwc2_set_ltq_params(struct dwc2_hsotg *hsotg)
reset_control_assert(hsotg->reset);
reset_control_assert(hsotg->reset_ecc);
- return ret;
+ return 0;
}
/**
dwc2_debugfs_init(hsotg);
/* Gadget code manages lowlevel hw on its own */
- if (hsotg->dr_mode == USB_DR_MODE_PERIPHERAL)
+ if (hsotg->dr_mode == USB_DR_MODE_PERIPHERAL ||
+ (hsotg->dr_mode == USB_DR_MODE_OTG && dwc2_is_device_mode(hsotg)))
dwc2_lowlevel_hw_disable(hsotg);
#if IS_ENABLED(CONFIG_USB_DWC2_PERIPHERAL) || \
config USB_DWC3_XILINX
tristate "Xilinx Platforms"
- depends on (ARCH_ZYNQMP || ARCH_VERSAL) && OF
+ depends on (ARCH_ZYNQMP || COMPILE_TEST) && OF
default USB_DWC3
help
Support Xilinx SoCs with DesignWare Core USB3 IP.
- This driver handles both ZynqMP and Versal SoC operations.
+ This driver handles ZynqMP SoC operations.
Say 'Y' or 'M' if you have one such device.
config USB_DWC3_AM62
unsigned long flags;
int ret;
u32 reg;
+ u32 desired_dr_role;
mutex_lock(&dwc->mutex);
+ spin_lock_irqsave(&dwc->lock, flags);
+ desired_dr_role = dwc->desired_dr_role;
+ spin_unlock_irqrestore(&dwc->lock, flags);
pm_runtime_get_sync(dwc->dev);
if (dwc->current_dr_role == DWC3_GCTL_PRTCAP_OTG)
dwc3_otg_update(dwc, 0);
- if (!dwc->desired_dr_role)
+ if (!desired_dr_role)
goto out;
- if (dwc->desired_dr_role == dwc->current_dr_role)
+ if (desired_dr_role == dwc->current_dr_role)
goto out;
- if (dwc->desired_dr_role == DWC3_GCTL_PRTCAP_OTG && dwc->edev)
+ if (desired_dr_role == DWC3_GCTL_PRTCAP_OTG && dwc->edev)
goto out;
switch (dwc->current_dr_role) {
*/
if (dwc->current_dr_role && ((DWC3_IP_IS(DWC3) ||
DWC3_VER_IS_PRIOR(DWC31, 190A)) &&
- dwc->desired_dr_role != DWC3_GCTL_PRTCAP_OTG)) {
+ desired_dr_role != DWC3_GCTL_PRTCAP_OTG)) {
reg = dwc3_readl(dwc->regs, DWC3_GCTL);
reg |= DWC3_GCTL_CORESOFTRESET;
dwc3_writel(dwc->regs, DWC3_GCTL, reg);
spin_lock_irqsave(&dwc->lock, flags);
- dwc3_set_prtcap(dwc, dwc->desired_dr_role);
+ dwc3_set_prtcap(dwc, desired_dr_role);
spin_unlock_irqrestore(&dwc->lock, flags);
- switch (dwc->desired_dr_role) {
+ switch (desired_dr_role) {
case DWC3_GCTL_PRTCAP_HOST:
ret = dwc3_host_init(dwc);
if (ret) {
if (!dwc->ulpi_ready) {
ret = dwc3_core_ulpi_init(dwc);
- if (ret)
+ if (ret) {
+ if (ret == -ETIMEDOUT) {
+ dwc3_core_soft_reset(dwc);
+ ret = -EPROBE_DEFER;
+ }
goto err0;
+ }
dwc->ulpi_ready = true;
}
#define PCI_DEVICE_ID_INTEL_ADLN 0x465e
#define PCI_DEVICE_ID_INTEL_ADLN_PCH 0x54ee
#define PCI_DEVICE_ID_INTEL_ADLS 0x7ae1
-#define PCI_DEVICE_ID_INTEL_RPL 0x460e
+#define PCI_DEVICE_ID_INTEL_RPL 0xa70e
#define PCI_DEVICE_ID_INTEL_RPLS 0x7a61
#define PCI_DEVICE_ID_INTEL_MTLP 0x7ec1
#define PCI_DEVICE_ID_INTEL_MTL 0x7e7e
if (IS_ERR(qcom->icc_path_apps)) {
dev_err(dev, "failed to get apps-usb path: %ld\n",
PTR_ERR(qcom->icc_path_apps));
- return PTR_ERR(qcom->icc_path_apps);
+ ret = PTR_ERR(qcom->icc_path_apps);
+ goto put_path_ddr;
}
max_speed = usb_get_maximum_speed(&qcom->dwc3->dev);
}
if (ret) {
dev_err(dev, "failed to set bandwidth for usb-ddr path: %d\n", ret);
- return ret;
+ goto put_path_apps;
}
ret = icc_set_bw(qcom->icc_path_apps, APPS_USB_AVG_BW, APPS_USB_PEAK_BW);
if (ret) {
dev_err(dev, "failed to set bandwidth for apps-usb path: %d\n", ret);
- return ret;
+ goto put_path_apps;
}
return 0;
+
+put_path_apps:
+ icc_put(qcom->icc_path_apps);
+put_path_ddr:
+ icc_put(qcom->icc_path_ddr);
+ return ret;
}
/**
*/
if (num_trbs_left == 1 || (needs_extra_trb &&
num_trbs_left <= 2 &&
- sg_dma_len(sg_next(s)) >= length))
- must_interrupt = true;
+ sg_dma_len(sg_next(s)) >= length)) {
+ struct dwc3_request *r;
+
+ /* Check if previous requests already set IOC */
+ list_for_each_entry(r, &dep->started_list, list) {
+ if (r != req && !r->request.no_interrupt)
+ break;
+
+ if (r == req)
+ must_interrupt = true;
+ }
+ }
dwc3_prepare_one_trb(dep, req, trb_length, 1, i, false,
must_interrupt);
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+
+config USB_FOTG210
+ tristate "Faraday FOTG210 USB2 Dual Role controller"
+ depends on USB || USB_GADGET
+ depends on HAS_DMA && HAS_IOMEM
+ depends on ARCH_GEMINI || COMPILE_TEST
+ default ARCH_GEMINI
+ select MFD_SYSCON
+ help
+ Faraday FOTG210 is a dual-mode USB controller that can act
+ in both host controller and peripheral controller mode.
+
+if USB_FOTG210
+
+config USB_FOTG210_HCD
+ bool "Faraday FOTG210 USB Host Controller support"
+ depends on USB=y || USB=USB_FOTG210
+ help
+ Faraday FOTG210 is an OTG controller which can be configured as
+ an USB2.0 host. It is designed to meet USB2.0 EHCI specification
+ with minor modification.
+
+ To compile this driver as a module, choose M here: the
+ module will be called fotg210-hcd.
+
+config USB_FOTG210_UDC
+ depends on USB_GADGET=y || USB_GADGET=USB_FOTG210
+ bool "Faraday FOTG210 USB Peripheral Controller support"
+ help
+ Faraday USB2.0 OTG controller which can be configured as
+ high speed or full speed USB device. This driver suppports
+ Bulk Transfer so far.
+
+ Say "y" to link the driver statically, or "m" to build a
+ dynamically linked module called "fotg210-udc".
+
+endif
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+
+# This setup links the different object files into one single
+# module so we don't have to EXPORT() a lot of internal symbols
+# or create unnecessary submodules.
+fotg210-objs-y += fotg210-core.o
+fotg210-objs-$(CONFIG_USB_FOTG210_HCD) += fotg210-hcd.o
+fotg210-objs-$(CONFIG_USB_FOTG210_UDC) += fotg210-udc.o
+fotg210-objs := $(fotg210-objs-y)
+obj-$(CONFIG_USB_FOTG210) += fotg210.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Central probing code for the FOTG210 dual role driver
+ * We register one driver for the hardware and then we decide
+ * whether to proceed with probing the host or the peripheral
+ * driver.
+ */
+#include <linux/bitops.h>
+#include <linux/device.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <linux/usb.h>
+#include <linux/usb/otg.h>
+
+#include "fotg210.h"
+
+/*
+ * Gemini-specific initialization function, only executed on the
+ * Gemini SoC using the global misc control register.
+ *
+ * The gemini USB blocks are connected to either Mini-A (host mode) or
+ * Mini-B (peripheral mode) plugs. There is no role switch support on the
+ * Gemini SoC, just either-or.
+ */
+#define GEMINI_GLOBAL_MISC_CTRL 0x30
+#define GEMINI_MISC_USB0_WAKEUP BIT(14)
+#define GEMINI_MISC_USB1_WAKEUP BIT(15)
+#define GEMINI_MISC_USB0_VBUS_ON BIT(22)
+#define GEMINI_MISC_USB1_VBUS_ON BIT(23)
+#define GEMINI_MISC_USB0_MINI_B BIT(29)
+#define GEMINI_MISC_USB1_MINI_B BIT(30)
+
+static int fotg210_gemini_init(struct device *dev, struct resource *res,
+ enum usb_dr_mode mode)
+{
+ struct device_node *np = dev->of_node;
+ struct regmap *map;
+ bool wakeup;
+ u32 mask, val;
+ int ret;
+
+ map = syscon_regmap_lookup_by_phandle(np, "syscon");
+ if (IS_ERR(map)) {
+ dev_err(dev, "no syscon\n");
+ return PTR_ERR(map);
+ }
+ wakeup = of_property_read_bool(np, "wakeup-source");
+
+ /*
+ * Figure out if this is USB0 or USB1 by simply checking the
+ * physical base address.
+ */
+ mask = 0;
+ if (res->start == 0x69000000) {
+ mask = GEMINI_MISC_USB1_VBUS_ON | GEMINI_MISC_USB1_MINI_B |
+ GEMINI_MISC_USB1_WAKEUP;
+ if (mode == USB_DR_MODE_HOST)
+ val = GEMINI_MISC_USB1_VBUS_ON;
+ else
+ val = GEMINI_MISC_USB1_MINI_B;
+ if (wakeup)
+ val |= GEMINI_MISC_USB1_WAKEUP;
+ } else {
+ mask = GEMINI_MISC_USB0_VBUS_ON | GEMINI_MISC_USB0_MINI_B |
+ GEMINI_MISC_USB0_WAKEUP;
+ if (mode == USB_DR_MODE_HOST)
+ val = GEMINI_MISC_USB0_VBUS_ON;
+ else
+ val = GEMINI_MISC_USB0_MINI_B;
+ if (wakeup)
+ val |= GEMINI_MISC_USB0_WAKEUP;
+ }
+
+ ret = regmap_update_bits(map, GEMINI_GLOBAL_MISC_CTRL, mask, val);
+ if (ret) {
+ dev_err(dev, "failed to initialize Gemini PHY\n");
+ return ret;
+ }
+
+ dev_info(dev, "initialized Gemini PHY in %s mode\n",
+ (mode == USB_DR_MODE_HOST) ? "host" : "gadget");
+ return 0;
+}
+
+static int fotg210_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ enum usb_dr_mode mode;
+ int ret;
+
+ mode = usb_get_dr_mode(dev);
+
+ if (of_device_is_compatible(dev->of_node, "cortina,gemini-usb")) {
+ struct resource *res;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ ret = fotg210_gemini_init(dev, res, mode);
+ if (ret)
+ return ret;
+ }
+
+ if (mode == USB_DR_MODE_PERIPHERAL)
+ ret = fotg210_udc_probe(pdev);
+ else
+ ret = fotg210_hcd_probe(pdev);
+
+ return ret;
+}
+
+static int fotg210_remove(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ enum usb_dr_mode mode;
+
+ mode = usb_get_dr_mode(dev);
+
+ if (mode == USB_DR_MODE_PERIPHERAL)
+ fotg210_udc_remove(pdev);
+ else
+ fotg210_hcd_remove(pdev);
+
+ return 0;
+}
+
+#ifdef CONFIG_OF
+static const struct of_device_id fotg210_of_match[] = {
+ { .compatible = "faraday,fotg210" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, fotg210_of_match);
+#endif
+
+static struct platform_driver fotg210_driver = {
+ .driver = {
+ .name = "fotg210",
+ .of_match_table = of_match_ptr(fotg210_of_match),
+ },
+ .probe = fotg210_probe,
+ .remove = fotg210_remove,
+};
+
+static int __init fotg210_init(void)
+{
+ if (usb_disabled())
+ return -ENODEV;
+
+ if (IS_ENABLED(CONFIG_USB_FOTG210_HCD))
+ fotg210_hcd_init();
+ return platform_driver_register(&fotg210_driver);
+}
+module_init(fotg210_init);
+
+static void __exit fotg210_cleanup(void)
+{
+ platform_driver_unregister(&fotg210_driver);
+ if (IS_ENABLED(CONFIG_USB_FOTG210_HCD))
+ fotg210_hcd_cleanup();
+}
+module_exit(fotg210_cleanup);
+
+MODULE_AUTHOR("Yuan-Hsin Chen, Feng-Hsin Chiang");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("FOTG210 Dual Role Controller Driver");
#include <asm/irq.h>
#include <asm/unaligned.h>
-#define DRIVER_AUTHOR "Yuan-Hsin Chen"
-#define DRIVER_DESC "FOTG210 Host Controller (EHCI) Driver"
+#include "fotg210.h"
+
static const char hcd_name[] = "fotg210_hcd";
#undef FOTG210_URB_TRACE
#define INTR_MASK (STS_IAA | STS_FATAL | STS_PCD | STS_ERR | STS_INT)
-#include "fotg210.h"
+#include "fotg210-hcd.h"
#define fotg210_dbg(fotg210, fmt, args...) \
dev_dbg(fotg210_to_hcd(fotg210)->self.controller, fmt, ## args)
* functions and in order to facilitate role switching we cannot
* give the fotg210 driver exclusive access to those.
*/
-MODULE_DESCRIPTION(DRIVER_DESC);
-MODULE_AUTHOR(DRIVER_AUTHOR);
-MODULE_LICENSE("GPL");
static const struct hc_driver fotg210_fotg210_hc_driver = {
.description = hcd_name,
* then invokes the start() method for the HCD associated with it
* through the hotplug entry's driver_data.
*/
-static int fotg210_hcd_probe(struct platform_device *pdev)
+int fotg210_hcd_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct usb_hcd *hcd;
* @dev: USB Host Controller being removed
*
*/
-static int fotg210_hcd_remove(struct platform_device *pdev)
+int fotg210_hcd_remove(struct platform_device *pdev)
{
struct usb_hcd *hcd = platform_get_drvdata(pdev);
struct fotg210_hcd *fotg210 = hcd_to_fotg210(hcd);
return 0;
}
-#ifdef CONFIG_OF
-static const struct of_device_id fotg210_of_match[] = {
- { .compatible = "faraday,fotg210" },
- {},
-};
-MODULE_DEVICE_TABLE(of, fotg210_of_match);
-#endif
-
-static struct platform_driver fotg210_hcd_driver = {
- .driver = {
- .name = "fotg210-hcd",
- .of_match_table = of_match_ptr(fotg210_of_match),
- },
- .probe = fotg210_hcd_probe,
- .remove = fotg210_hcd_remove,
-};
-
-static int __init fotg210_hcd_init(void)
+int __init fotg210_hcd_init(void)
{
- int retval = 0;
-
if (usb_disabled())
return -ENODEV;
fotg210_debug_root = debugfs_create_dir("fotg210", usb_debug_root);
- retval = platform_driver_register(&fotg210_hcd_driver);
- if (retval < 0)
- goto clean;
- return retval;
-
-clean:
- debugfs_remove(fotg210_debug_root);
- fotg210_debug_root = NULL;
-
- clear_bit(USB_EHCI_LOADED, &usb_hcds_loaded);
- return retval;
+ return 0;
}
-module_init(fotg210_hcd_init);
-static void __exit fotg210_hcd_cleanup(void)
+void __exit fotg210_hcd_cleanup(void)
{
- platform_driver_unregister(&fotg210_hcd_driver);
debugfs_remove(fotg210_debug_root);
clear_bit(USB_EHCI_LOADED, &usb_hcds_loaded);
}
-module_exit(fotg210_hcd_cleanup);
#include <linux/platform_device.h>
#include <linux/usb/ch9.h>
#include <linux/usb/gadget.h>
+#include <linux/clk.h>
+#include <linux/usb/otg.h>
+#include <linux/usb/phy.h>
#include "fotg210.h"
+#include "fotg210-udc.h"
#define DRIVER_DESC "FOTG210 USB Device Controller Driver"
#define DRIVER_VERSION "30-April-2013"
static void fotg210_set_address(struct fotg210_udc *fotg210,
struct usb_ctrlrequest *ctrl)
{
- if (ctrl->wValue >= 0x0100) {
+ if (le16_to_cpu(ctrl->wValue) >= 0x0100) {
fotg210_request_error(fotg210);
} else {
- fotg210_set_dev_addr(fotg210, ctrl->wValue);
+ fotg210_set_dev_addr(fotg210, le16_to_cpu(ctrl->wValue));
fotg210_set_cxdone(fotg210);
}
}
switch (ctrl->bRequestType & USB_RECIP_MASK) {
case USB_RECIP_DEVICE:
- fotg210->ep0_data = 1 << USB_DEVICE_SELF_POWERED;
+ fotg210->ep0_data = cpu_to_le16(1 << USB_DEVICE_SELF_POWERED);
break;
case USB_RECIP_INTERFACE:
- fotg210->ep0_data = 0;
+ fotg210->ep0_data = cpu_to_le16(0);
break;
case USB_RECIP_ENDPOINT:
epnum = ctrl->wIndex & USB_ENDPOINT_NUMBER_MASK;
if (epnum)
fotg210->ep0_data =
- fotg210_is_epnstall(fotg210->ep[epnum])
- << USB_ENDPOINT_HALT;
+ cpu_to_le16(fotg210_is_epnstall(fotg210->ep[epnum])
+ << USB_ENDPOINT_HALT);
else
fotg210_request_error(fotg210);
break;
{
struct fotg210_udc *fotg210 = gadget_to_fotg210(g);
u32 value;
+ int ret;
/* hook up the driver */
driver->driver.bus = NULL;
fotg210->driver = driver;
+ if (!IS_ERR_OR_NULL(fotg210->phy)) {
+ ret = otg_set_peripheral(fotg210->phy->otg,
+ &fotg210->gadget);
+ if (ret)
+ dev_err(fotg210->dev, "can't bind to phy\n");
+ }
+
/* enable device global interrupt */
value = ioread32(fotg210->reg + FOTG210_DMCR);
value |= DMCR_GLINT_EN;
struct fotg210_udc *fotg210 = gadget_to_fotg210(g);
unsigned long flags;
+ if (!IS_ERR_OR_NULL(fotg210->phy))
+ return otg_set_peripheral(fotg210->phy->otg, NULL);
+
spin_lock_irqsave(&fotg210->lock, flags);
fotg210_init(fotg210);
.udc_stop = fotg210_udc_stop,
};
-static int fotg210_udc_remove(struct platform_device *pdev)
+/**
+ * fotg210_phy_event - Called by phy upon VBus event
+ * @nb: notifier block
+ * @action: phy action, is vbus connect or disconnect
+ * @data: the usb_gadget structure in fotg210
+ *
+ * Called by the USB Phy when a cable connect or disconnect is sensed.
+ *
+ * Returns NOTIFY_OK or NOTIFY_DONE
+ */
+static int fotg210_phy_event(struct notifier_block *nb, unsigned long action,
+ void *data)
+{
+ struct usb_gadget *gadget = data;
+
+ if (!gadget)
+ return NOTIFY_DONE;
+
+ switch (action) {
+ case USB_EVENT_VBUS:
+ usb_gadget_vbus_connect(gadget);
+ return NOTIFY_OK;
+ case USB_EVENT_NONE:
+ usb_gadget_vbus_disconnect(gadget);
+ return NOTIFY_OK;
+ default:
+ return NOTIFY_DONE;
+ }
+}
+
+static struct notifier_block fotg210_phy_notifier = {
+ .notifier_call = fotg210_phy_event,
+};
+
+int fotg210_udc_remove(struct platform_device *pdev)
{
struct fotg210_udc *fotg210 = platform_get_drvdata(pdev);
int i;
usb_del_gadget_udc(&fotg210->gadget);
+ if (!IS_ERR_OR_NULL(fotg210->phy)) {
+ usb_unregister_notifier(fotg210->phy, &fotg210_phy_notifier);
+ usb_put_phy(fotg210->phy);
+ }
iounmap(fotg210->reg);
free_irq(platform_get_irq(pdev, 0), fotg210);
fotg210_ep_free_request(&fotg210->ep[0]->ep, fotg210->ep0_req);
for (i = 0; i < FOTG210_MAX_NUM_EP; i++)
kfree(fotg210->ep[i]);
+
+ if (!IS_ERR(fotg210->pclk))
+ clk_disable_unprepare(fotg210->pclk);
+
kfree(fotg210);
return 0;
}
-static int fotg210_udc_probe(struct platform_device *pdev)
+int fotg210_udc_probe(struct platform_device *pdev)
{
- struct resource *res, *ires;
+ struct resource *res;
struct fotg210_udc *fotg210 = NULL;
- struct fotg210_ep *_ep[FOTG210_MAX_NUM_EP];
+ struct device *dev = &pdev->dev;
+ int irq;
int ret = 0;
int i;
return -ENODEV;
}
- ires = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
- if (!ires) {
- pr_err("platform_get_resource IORESOURCE_IRQ error.\n");
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0) {
+ pr_err("could not get irq\n");
return -ENODEV;
}
- ret = -ENOMEM;
-
/* initialize udc */
fotg210 = kzalloc(sizeof(struct fotg210_udc), GFP_KERNEL);
if (fotg210 == NULL)
+ return -ENOMEM;
+
+ fotg210->dev = dev;
+
+ /* It's OK not to supply this clock */
+ fotg210->pclk = devm_clk_get(dev, "PCLK");
+ if (!IS_ERR(fotg210->pclk)) {
+ ret = clk_prepare_enable(fotg210->pclk);
+ if (ret) {
+ dev_err(dev, "failed to enable PCLK\n");
+ goto err;
+ }
+ } else if (PTR_ERR(fotg210->pclk) == -EPROBE_DEFER) {
+ /*
+ * Percolate deferrals, for anything else,
+ * just live without the clocking.
+ */
+ ret = -EPROBE_DEFER;
goto err;
+ }
+
+ fotg210->phy = devm_usb_get_phy_by_phandle(dev, "usb-phy", 0);
+ if (IS_ERR(fotg210->phy)) {
+ ret = PTR_ERR(fotg210->phy);
+ if (ret == -EPROBE_DEFER)
+ goto err_pclk;
+ dev_info(dev, "no PHY found\n");
+ fotg210->phy = NULL;
+ } else {
+ ret = usb_phy_init(fotg210->phy);
+ if (ret)
+ goto err_pclk;
+ dev_info(dev, "found and initialized PHY\n");
+ }
for (i = 0; i < FOTG210_MAX_NUM_EP; i++) {
- _ep[i] = kzalloc(sizeof(struct fotg210_ep), GFP_KERNEL);
- if (_ep[i] == NULL)
+ fotg210->ep[i] = kzalloc(sizeof(struct fotg210_ep), GFP_KERNEL);
+ if (!fotg210->ep[i])
goto err_alloc;
- fotg210->ep[i] = _ep[i];
}
fotg210->reg = ioremap(res->start, resource_size(res));
if (fotg210->reg == NULL) {
- pr_err("ioremap error.\n");
+ dev_err(dev, "ioremap error\n");
goto err_alloc;
}
fotg210->gadget.ops = &fotg210_gadget_ops;
fotg210->gadget.max_speed = USB_SPEED_HIGH;
- fotg210->gadget.dev.parent = &pdev->dev;
- fotg210->gadget.dev.dma_mask = pdev->dev.dma_mask;
+ fotg210->gadget.dev.parent = dev;
+ fotg210->gadget.dev.dma_mask = dev->dma_mask;
fotg210->gadget.name = udc_name;
INIT_LIST_HEAD(&fotg210->gadget.ep_list);
fotg210_disable_unplug(fotg210);
- ret = request_irq(ires->start, fotg210_irq, IRQF_SHARED,
+ ret = request_irq(irq, fotg210_irq, IRQF_SHARED,
udc_name, fotg210);
if (ret < 0) {
- pr_err("request_irq error (%d)\n", ret);
+ dev_err(dev, "request_irq error (%d)\n", ret);
goto err_req;
}
- ret = usb_add_gadget_udc(&pdev->dev, &fotg210->gadget);
+ if (!IS_ERR_OR_NULL(fotg210->phy))
+ usb_register_notifier(fotg210->phy, &fotg210_phy_notifier);
+
+ ret = usb_add_gadget_udc(dev, &fotg210->gadget);
if (ret)
goto err_add_udc;
- dev_info(&pdev->dev, "version %s\n", DRIVER_VERSION);
+ dev_info(dev, "version %s\n", DRIVER_VERSION);
return 0;
err_add_udc:
- free_irq(ires->start, fotg210);
+ if (!IS_ERR_OR_NULL(fotg210->phy))
+ usb_unregister_notifier(fotg210->phy, &fotg210_phy_notifier);
+ free_irq(irq, fotg210);
err_req:
fotg210_ep_free_request(&fotg210->ep[0]->ep, fotg210->ep0_req);
err_alloc:
for (i = 0; i < FOTG210_MAX_NUM_EP; i++)
kfree(fotg210->ep[i]);
- kfree(fotg210);
+err_pclk:
+ if (!IS_ERR(fotg210->pclk))
+ clk_disable_unprepare(fotg210->pclk);
err:
+ kfree(fotg210);
return ret;
}
-
-static struct platform_driver fotg210_driver = {
- .driver = {
- .name = udc_name,
- },
- .probe = fotg210_udc_probe,
- .remove = fotg210_udc_remove,
-};
-
-module_platform_driver(fotg210_driver);
-
-MODULE_AUTHOR("Yuan-Hsin Chen, Feng-Hsin Chiang <john453@faraday-tech.com>");
-MODULE_LICENSE("GPL");
-MODULE_DESCRIPTION(DRIVER_DESC);
struct fotg210_udc {
spinlock_t lock; /* protect the struct */
void __iomem *reg;
+ struct clk *pclk;
unsigned long irq_trigger;
+ struct device *dev;
+ struct usb_phy *phy;
struct usb_gadget gadget;
struct usb_gadget_driver *driver;
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __FOTG210_H
+#define __FOTG210_H
+
+#ifdef CONFIG_USB_FOTG210_HCD
+int fotg210_hcd_probe(struct platform_device *pdev);
+int fotg210_hcd_remove(struct platform_device *pdev);
+int fotg210_hcd_init(void);
+void fotg210_hcd_cleanup(void);
+#else
+static inline int fotg210_hcd_probe(struct platform_device *pdev)
+{
+ return 0;
+}
+static inline int fotg210_hcd_remove(struct platform_device *pdev)
+{
+ return 0;
+}
+static inline int fotg210_hcd_init(void)
+{
+ return 0;
+}
+static inline void fotg210_hcd_cleanup(void)
+{
+}
+#endif
+
+#ifdef CONFIG_USB_FOTG210_UDC
+int fotg210_udc_probe(struct platform_device *pdev);
+int fotg210_udc_remove(struct platform_device *pdev);
+#else
+static inline int fotg210_udc_probe(struct platform_device *pdev)
+{
+ return 0;
+}
+static inline int fotg210_udc_remove(struct platform_device *pdev)
+{
+ return 0;
+}
+#endif
+
+#endif /* __FOTG210_H */
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/device.h>
+#include <linux/kstrtox.h>
#include <linux/nls.h>
#include <linux/usb/composite.h>
#include <linux/usb/gadget_configfs.h>
bool use;
mutex_lock(&gi->lock);
- ret = strtobool(page, &use);
+ ret = kstrtobool(page, &use);
if (!ret) {
gi->use_os_desc = use;
ret = len;
struct usb_composite_dev *cdev = c->cdev;
struct f_ecm *ecm = func_to_ecm(f);
struct usb_string *us;
- int status;
+ int status = 0;
struct usb_ep *ep;
struct f_ecm_opts *ecm_opts;
ecm_opts = container_of(f->fi, struct f_ecm_opts, func_inst);
- /*
- * in drivers/usb/gadget/configfs.c:configfs_composite_bind()
- * configurations are bound in sequence with list_for_each_entry,
- * in each configuration its functions are bound in sequence
- * with list_for_each_entry, so we assume no race condition
- * with regard to ecm_opts->bound access
- */
+ mutex_lock(&ecm_opts->lock);
+
+ gether_set_gadget(ecm_opts->net, cdev->gadget);
+
if (!ecm_opts->bound) {
- mutex_lock(&ecm_opts->lock);
- gether_set_gadget(ecm_opts->net, cdev->gadget);
status = gether_register_netdev(ecm_opts->net);
- mutex_unlock(&ecm_opts->lock);
- if (status)
- return status;
ecm_opts->bound = true;
}
+ mutex_unlock(&ecm_opts->lock);
+ if (status)
+ return status;
+
ecm_string_defs[1].s = ecm->ethaddr;
us = usb_gstrings_attach(cdev, ecm_strings,
wait_queue_head_t write_queue;
struct usb_request *req;
- int minor;
+ struct device dev;
struct cdev cdev;
struct usb_function func;
return container_of(f, struct f_hidg, func);
}
+static void hidg_release(struct device *dev)
+{
+ struct f_hidg *hidg = container_of(dev, struct f_hidg, dev);
+
+ kfree(hidg->set_report_buf);
+ kfree(hidg);
+}
+
/*-------------------------------------------------------------------------*/
/* Static descriptors */
struct usb_ep *ep;
struct f_hidg *hidg = func_to_hidg(f);
struct usb_string *us;
- struct device *device;
int status;
- dev_t dev;
/* maybe allocate device-global string IDs, and patch descriptors */
us = usb_gstrings_attach(c->cdev, ct_func_strings,
/* create char device */
cdev_init(&hidg->cdev, &f_hidg_fops);
- dev = MKDEV(major, hidg->minor);
- status = cdev_add(&hidg->cdev, dev, 1);
+ status = cdev_device_add(&hidg->cdev, &hidg->dev);
if (status)
goto fail_free_descs;
- device = device_create(hidg_class, NULL, dev, NULL,
- "%s%d", "hidg", hidg->minor);
- if (IS_ERR(device)) {
- status = PTR_ERR(device);
- goto del;
- }
-
return 0;
-del:
- cdev_del(&hidg->cdev);
fail_free_descs:
usb_free_all_descriptors(f);
fail:
hidg = func_to_hidg(f);
opts = container_of(f->fi, struct f_hid_opts, func_inst);
- kfree(hidg->report_desc);
- kfree(hidg->set_report_buf);
- kfree(hidg);
+ put_device(&hidg->dev);
mutex_lock(&opts->lock);
--opts->refcnt;
mutex_unlock(&opts->lock);
{
struct f_hidg *hidg = func_to_hidg(f);
- device_destroy(hidg_class, MKDEV(major, hidg->minor));
- cdev_del(&hidg->cdev);
+ cdev_device_del(&hidg->cdev, &hidg->dev);
usb_free_all_descriptors(f);
}
{
struct f_hidg *hidg;
struct f_hid_opts *opts;
+ int ret;
/* allocate and initialize one new instance */
hidg = kzalloc(sizeof(*hidg), GFP_KERNEL);
opts = container_of(fi, struct f_hid_opts, func_inst);
mutex_lock(&opts->lock);
- ++opts->refcnt;
- hidg->minor = opts->minor;
+ device_initialize(&hidg->dev);
+ hidg->dev.release = hidg_release;
+ hidg->dev.class = hidg_class;
+ hidg->dev.devt = MKDEV(major, opts->minor);
+ ret = dev_set_name(&hidg->dev, "hidg%d", opts->minor);
+ if (ret)
+ goto err_unlock;
+
hidg->bInterfaceSubClass = opts->subclass;
hidg->bInterfaceProtocol = opts->protocol;
hidg->report_length = opts->report_length;
hidg->report_desc_length = opts->report_desc_length;
if (opts->report_desc) {
- hidg->report_desc = kmemdup(opts->report_desc,
- opts->report_desc_length,
- GFP_KERNEL);
+ hidg->report_desc = devm_kmemdup(&hidg->dev, opts->report_desc,
+ opts->report_desc_length,
+ GFP_KERNEL);
if (!hidg->report_desc) {
- kfree(hidg);
- mutex_unlock(&opts->lock);
- return ERR_PTR(-ENOMEM);
+ ret = -ENOMEM;
+ goto err_put_device;
}
}
hidg->use_out_ep = !opts->no_out_endpoint;
+ ++opts->refcnt;
mutex_unlock(&opts->lock);
hidg->func.name = "hid";
hidg->qlen = 4;
return &hidg->func;
+
+err_put_device:
+ put_device(&hidg->dev);
+err_unlock:
+ mutex_unlock(&opts->lock);
+ return ERR_PTR(ret);
}
DECLARE_USB_FUNCTION_INIT(hid, hidg_alloc_inst, hidg_alloc);
#include <linux/fcntl.h>
#include <linux/file.h>
#include <linux/fs.h>
+#include <linux/kstrtox.h>
#include <linux/kthread.h>
#include <linux/sched/signal.h>
#include <linux/limits.h>
return -EBUSY;
}
- ret = strtobool(page, &stall);
+ ret = kstrtobool(page, &stall);
if (!ret) {
opts->common->can_stall = stall;
ret = len;
spin_unlock_irqrestore(&dev->lock, flags);
kref_get(&dev->kref);
- DBG(dev, "printer_open returned %x\n", ret);
+
return ret;
}
spin_unlock_irqrestore(&dev->lock, flags);
kref_put(&dev->kref, printer_dev_free);
- DBG(dev, "printer_close\n");
return 0;
}
if (dev->interface < 0)
return;
- DBG(dev, "%s\n", __func__);
-
if (dev->in_ep->desc)
usb_ep_disable(dev->in_ep);
{
struct usb_request *req;
- INFO(dev, "Received Printer Reset Request\n");
-
if (usb_ep_disable(dev->in_ep))
DBG(dev, "Failed to disable USB in_ep\n");
if (usb_ep_disable(dev->out_ep))
{
struct printer_dev *dev = func_to_printer(f);
- DBG(dev, "%s\n", __func__);
-
printer_reset_interface(dev);
}
/* string IDs are assigned dynamically */
-#define UVC_STRING_CONTROL_IDX 0
-#define UVC_STRING_STREAMING_IDX 1
-
static struct usb_string uvc_en_us_strings[] = {
/* [UVC_STRING_CONTROL_IDX].s = DYNAMIC, */
[UVC_STRING_STREAMING_IDX].s = "Video Streaming",
memset(&v4l2_event, 0, sizeof(v4l2_event));
v4l2_event.type = UVC_EVENT_DATA;
- uvc_event->data.length = req->actual;
- memcpy(&uvc_event->data.data, req->buf, req->actual);
+ uvc_event->data.length = min_t(unsigned int, req->actual,
+ sizeof(uvc_event->data.data));
+ memcpy(&uvc_event->data.data, req->buf, uvc_event->data.length);
v4l2_event_queue(&uvc->vdev, &v4l2_event);
}
}
struct uvc_device *uvc = to_uvc(f);
struct v4l2_event v4l2_event;
struct uvc_event *uvc_event = (void *)&v4l2_event.u.data;
+ unsigned int interface = le16_to_cpu(ctrl->wIndex) & 0xff;
+ struct usb_ctrlrequest *mctrl;
if ((ctrl->bRequestType & USB_TYPE_MASK) != USB_TYPE_CLASS) {
uvcg_info(f, "invalid request type\n");
memset(&v4l2_event, 0, sizeof(v4l2_event));
v4l2_event.type = UVC_EVENT_SETUP;
memcpy(&uvc_event->req, ctrl, sizeof(uvc_event->req));
+
+ /* check for the interface number, fixup the interface number in
+ * the ctrl request so the userspace doesn't have to bother with
+ * offset and configfs parsing
+ */
+ mctrl = &uvc_event->req;
+ mctrl->wIndex &= ~cpu_to_le16(0xff);
+ if (interface == uvc->streaming_intf)
+ mctrl->wIndex = cpu_to_le16(UVC_STRING_STREAMING_IDX);
+
v4l2_event_queue(&uvc->vdev, &v4l2_event);
return 0;
#include <linux/blkdev.h>
#include <linux/file.h>
#include <linux/fs.h>
+#include <linux/kstrtox.h>
#include <linux/usb/composite.h>
#include "storage_common.h"
ssize_t rc;
bool ro;
- rc = strtobool(buf, &ro);
+ rc = kstrtobool(buf, &ro);
if (rc)
return rc;
bool nofua;
int ret;
- ret = strtobool(buf, &nofua);
+ ret = kstrtobool(buf, &nofua);
if (ret)
return ret;
bool cdrom;
int ret;
- ret = strtobool(buf, &cdrom);
+ ret = kstrtobool(buf, &cdrom);
if (ret)
return ret;
bool removable;
int ret;
- ret = strtobool(buf, &removable);
+ ret = kstrtobool(buf, &removable);
if (ret)
return ret;
net->max_mtu = GETHER_MAX_MTU_SIZE;
dev->gadget = g;
- SET_NETDEV_DEV(net, &g->dev);
SET_NETDEV_DEVTYPE(net, &gadget_type);
status = register_netdev(net);
struct usb_gadget *g;
int status;
- if (!net->dev.parent)
- return -EINVAL;
dev = netdev_priv(net);
g = dev->gadget;
dev = netdev_priv(net);
dev->gadget = g;
- SET_NETDEV_DEV(net, &g->dev);
}
EXPORT_SYMBOL_GPL(gether_set_gadget);
#include <linux/export.h>
#include <linux/module.h>
#include <linux/console.h>
+#include <linux/kstrtox.h>
#include <linux/kthread.h>
#include <linux/workqueue.h>
#include <linux/kfifo.h>
bool enable;
int ret;
- ret = strtobool(page, &enable);
+ ret = kstrtobool(page, &enable);
if (ret)
return ret;
UVCG_UNCOMPRESSED_ATTR(b_default_frame_index, bDefaultFrameIndex, 8);
UVCG_UNCOMPRESSED_ATTR_RO(b_aspect_ratio_x, bAspectRatioX, 8);
UVCG_UNCOMPRESSED_ATTR_RO(b_aspect_ratio_y, bAspectRatioY, 8);
-UVCG_UNCOMPRESSED_ATTR_RO(bm_interface_flags, bmInterfaceFlags, 8);
+UVCG_UNCOMPRESSED_ATTR_RO(bm_interlace_flags, bmInterlaceFlags, 8);
#undef UVCG_UNCOMPRESSED_ATTR
#undef UVCG_UNCOMPRESSED_ATTR_RO
&uvcg_uncompressed_attr_b_default_frame_index,
&uvcg_uncompressed_attr_b_aspect_ratio_x,
&uvcg_uncompressed_attr_b_aspect_ratio_y,
- &uvcg_uncompressed_attr_bm_interface_flags,
+ &uvcg_uncompressed_attr_bm_interlace_flags,
&uvcg_uncompressed_attr_bma_controls,
NULL,
};
h->desc.bDefaultFrameIndex = 1;
h->desc.bAspectRatioX = 0;
h->desc.bAspectRatioY = 0;
- h->desc.bmInterfaceFlags = 0;
+ h->desc.bmInterlaceFlags = 0;
h->desc.bCopyProtect = 0;
INIT_LIST_HEAD(&h->fmt.frames);
UVCG_MJPEG_ATTR_RO(bm_flags, bmFlags, 8);
UVCG_MJPEG_ATTR_RO(b_aspect_ratio_x, bAspectRatioX, 8);
UVCG_MJPEG_ATTR_RO(b_aspect_ratio_y, bAspectRatioY, 8);
-UVCG_MJPEG_ATTR_RO(bm_interface_flags, bmInterfaceFlags, 8);
+UVCG_MJPEG_ATTR_RO(bm_interlace_flags, bmInterlaceFlags, 8);
#undef UVCG_MJPEG_ATTR
#undef UVCG_MJPEG_ATTR_RO
&uvcg_mjpeg_attr_bm_flags,
&uvcg_mjpeg_attr_b_aspect_ratio_x,
&uvcg_mjpeg_attr_b_aspect_ratio_y,
- &uvcg_mjpeg_attr_bm_interface_flags,
+ &uvcg_mjpeg_attr_bm_interlace_flags,
&uvcg_mjpeg_attr_bma_controls,
NULL,
};
h->desc.bDefaultFrameIndex = 1;
h->desc.bAspectRatioX = 0;
h->desc.bAspectRatioY = 0;
- h->desc.bmInterfaceFlags = 0;
+ h->desc.bmInterlaceFlags = 0;
h->desc.bCopyProtect = 0;
INIT_LIST_HEAD(&h->fmt.frames);
#include <linux/kernel.h>
#include <linux/device.h>
+#include <linux/kstrtox.h>
#include <linux/module.h>
#include <linux/tty.h>
#include <linux/tty_flip.h>
if (!s) /* called for no-arg enable == default */
return 0;
- ret = strtobool(s, &do_enable);
+ ret = kstrtobool(s, &do_enable);
if (ret || enable == do_enable)
return ret;
.bDefaultFrameIndex = 1,
.bAspectRatioX = 0,
.bAspectRatioY = 0,
- .bmInterfaceFlags = 0,
+ .bmInterlaceFlags = 0,
.bCopyProtect = 0,
};
.bDefaultFrameIndex = 1,
.bAspectRatioX = 0,
.bAspectRatioY = 0,
- .bmInterfaceFlags = 0,
+ .bmInterlaceFlags = 0,
.bCopyProtect = 0,
};
config USB_AT91
tristate "Atmel AT91 USB Device Port"
depends on ARCH_AT91
- depends on OF || COMPILE_TEST
+ depends on OF
help
Many Atmel AT91 processors (such as the AT91RM2000) have a
full speed USB Device Port with support for five configurable
help
Faraday usb device controller FUSB300 driver
-config USB_FOTG210_UDC
- depends on HAS_DMA
- tristate "Faraday FOTG210 USB Peripheral Controller"
- help
- Faraday USB2.0 OTG controller which can be configured as
- high speed or full speed USB device. This driver supppors
- Bulk Transfer so far.
-
- Say "y" to link the driver statically, or "m" to build a
- dynamically linked module called "fotg210_udc".
-
config USB_GR_UDC
tristate "Aeroflex Gaisler GRUSBDC USB Peripheral Controller Driver"
depends on HAS_DMA
config USB_GADGET_XILINX
tristate "Xilinx USB Driver"
depends on HAS_DMA
- depends on OF || COMPILE_TEST
+ depends on OF
help
USB peripheral controller driver for Xilinx USB2 device.
Xilinx USB2 device is a soft IP which supports both full
obj-$(CONFIG_USB_MV_UDC) += mv_udc.o
mv_udc-y := mv_udc_core.o
obj-$(CONFIG_USB_FUSB300) += fusb300_udc.o
-obj-$(CONFIG_USB_FOTG210_UDC) += fotg210-udc.o
obj-$(CONFIG_USB_MV_U3D) += mv_u3d_core.o
obj-$(CONFIG_USB_GR_UDC) += gr_udc.o
obj-$(CONFIG_USB_GADGET_XILINX) += udc-xilinx.o
list_del_init(&req->queue);
- if (req->req.status == -EINPROGRESS)
+ if ((req->req.status == -EINPROGRESS) || (status == -EOVERFLOW))
req->req.status = status;
if (req->req.dma) {
{
struct ast_vhub_req *req;
unsigned int len;
+ int status = 0;
u32 stat;
/* Read EP status */
len = VHUB_EP_DMA_TX_SIZE(stat);
/* If not using DMA, copy data out if needed */
- if (!req->req.dma && !ep->epn.is_in && len)
- memcpy(req->req.buf + req->req.actual, ep->buf, len);
-
+ if (!req->req.dma && !ep->epn.is_in && len) {
+ if (req->req.actual + len > req->req.length) {
+ req->last_desc = 1;
+ status = -EOVERFLOW;
+ goto done;
+ } else {
+ memcpy(req->req.buf + req->req.actual, ep->buf, len);
+ }
+ }
/* Adjust size */
req->req.actual += len;
if (len < ep->ep.maxpacket)
req->last_desc = 1;
+done:
/* That's it ? complete the request and pick a new one */
if (req->last_desc >= 0) {
- ast_vhub_done(ep, req, 0);
+ ast_vhub_done(ep, req, status);
req = list_first_entry_or_null(&ep->queue, struct ast_vhub_req,
queue);
static void at91rm9200_udc_pullup(struct at91_udc *udc, int is_on)
{
- if (is_on)
- gpiod_set_value(udc->board.pullup_pin, 1);
- else
- gpiod_set_value(udc->board.pullup_pin, 0);
+ gpiod_set_value(udc->board.pullup_pin, is_on);
}
static const struct at91_udc_caps at91rm9200_udc_caps = {
}
ret = gadget->ops->pullup(gadget, 0);
- if (!ret) {
+ if (!ret)
gadget->connected = 0;
- mutex_lock(&udc_lock);
- if (gadget->udc->driver)
- gadget->udc->driver->disconnect(gadget);
- mutex_unlock(&udc_lock);
- }
+
+ mutex_lock(&udc_lock);
+ if (gadget->udc->driver)
+ gadget->udc->driver->disconnect(gadget);
+ mutex_unlock(&udc_lock);
out:
trace_usb_gadget_disconnect(gadget, ret);
NULL,
};
-static int usb_udc_uevent(struct device *dev, struct kobj_uevent_env *env)
+static int usb_udc_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
- struct usb_udc *udc = container_of(dev, struct usb_udc, dev);
+ const struct usb_udc *udc = container_of(dev, struct usb_udc, dev);
int ret;
ret = add_uevent_var(env, "USB_UDC_NAME=%s", udc->gadget->name);
tristate "Support for additional Renesas xHCI controller with firmware"
help
Say 'Y' to enable the support for the Renesas xHCI controller with
- firmware. Make sure you have the firwmare for the device and
+ firmware. Make sure you have the firmware for the device and
installed on your system for this device to work.
If unsure, say 'N'.
To compile this driver as a module, choose M here: the
module will be called isp1362-hcd.
-config USB_FOTG210_HCD
- tristate "FOTG210 HCD support"
- depends on USB && HAS_DMA && HAS_IOMEM
- help
- Faraday FOTG210 is an OTG controller which can be configured as
- an USB2.0 host. It is designed to meet USB2.0 EHCI specification
- with minor modification.
-
- To compile this driver as a module, choose M here: the
- module will be called fotg210-hcd.
-
config USB_MAX3421_HCD
tristate "MAX3421 HCD (USB-over-SPI) support"
depends on USB && SPI
obj-$(CONFIG_USB_EHCI_MV) += ehci-mv.o
obj-$(CONFIG_USB_HCD_BCMA) += bcma-hcd.o
obj-$(CONFIG_USB_HCD_SSB) += ssb-hcd.o
-obj-$(CONFIG_USB_FOTG210_HCD) += fotg210-hcd.o
obj-$(CONFIG_USB_MAX3421_HCD) += max3421-hcd.o
obj-$(CONFIG_USB_XEN_HCD) += xen-hcd.o
hcd->rsrc_len = resource_size(&res);
irq = irq_of_parse_and_map(dn, 0);
- if (irq == NO_IRQ) {
+ if (!irq) {
dev_err(&op->dev, "%s: irq_of_parse_and_map failed\n",
__FILE__);
rv = -EBUSY;
.remove = ehci_pci_remove,
.shutdown = usb_hcd_pci_shutdown,
-#ifdef CONFIG_PM
.driver = {
- .pm = &usb_hcd_pci_pm_ops
- },
+#ifdef CONFIG_PM
+ .pm = &usb_hcd_pci_pm_ops,
#endif
+ .probe_type = PROBE_PREFER_ASYNCHRONOUS,
+ },
};
static int __init ehci_pci_init(void)
hcd->rsrc_len = resource_size(&res);
irq = irq_of_parse_and_map(dn, 0);
- if (irq == NO_IRQ) {
+ if (!irq) {
dev_err(&op->dev, "%s: irq_of_parse_and_map failed\n",
__FILE__);
rv = -EBUSY;
* acts like a qh would, if EHCI had them for ISO.
*/
struct ehci_iso_stream {
- /* first field matches ehci_hq, but is NULL */
+ /* first field matches ehci_qh, but is NULL */
struct ehci_qh_hw *hw;
u8 bEndpointAddress;
/* USB Host interrupt. */
usb_irq = irq_of_parse_and_map(node, 0);
- if (usb_irq == NO_IRQ) {
+ if (!usb_irq) {
dev_err(dev, "could not get usb irq\n");
ret = -EINVAL;
goto err_usb_irq;
}
irq = irq_of_parse_and_map(dn, 0);
- if (irq == NO_IRQ) {
+ if (!irq) {
dev_err(&op->dev, "%s: irq_of_parse_and_map failed\n",
__FILE__);
rv = -EBUSY;
hcd->rsrc_len = resource_size(&res);
irq = irq_of_parse_and_map(dn, 0);
- if (irq == NO_IRQ) {
+ if (!irq) {
printk(KERN_ERR "%s: irq_of_parse_and_map failed\n", __FILE__);
rv = -EBUSY;
goto err_usb;
*/
#define XHCI_PORT_RZ ((1<<2) | (1<<24) | (0xf<<28))
-/*
+/**
+ * xhci_port_state_to_neutral() - Clean up read portsc value back into writeable
+ * @state: u32 port value read from portsc register to be cleanup up
+ *
* Given a port state, this function returns a value that would result in the
* port being in the same state, if the value was written to the port status
* control register.
* Save Read Only (RO) bits and save read/write bits where
* writing a 0 clears the bit and writing a 1 sets the bit (RWS).
* For all other types (RW1S, RW1CS, RW, and RZ), writing a '0' has no effect.
+ *
+ * Return: u32 value that can be written back to portsc register without
+ * changing port state.
*/
+
u32 xhci_port_state_to_neutral(u32 state)
{
/* Save read-only status and port state */
return (state & XHCI_PORT_RO) | (state & XHCI_PORT_RWS);
}
+EXPORT_SYMBOL_GPL(xhci_port_state_to_neutral);
-/*
- * find slot id based on port number.
- * @port: The one-based port number from one of the two split roothubs.
+/**
+ * xhci_find_slot_id_by_port() - Find slot id of a usb device on a roothub port
+ * @hcd: pointer to hcd of the roothub
+ * @xhci: pointer to xhci structure
+ * @port: one-based port number of the port in this roothub.
+ *
+ * Return: Slot id of the usb device connected to the root port, 0 if not found
*/
+
int xhci_find_slot_id_by_port(struct usb_hcd *hcd, struct xhci_hcd *xhci,
u16 port)
{
return slot_id;
}
+EXPORT_SYMBOL_GPL(xhci_find_slot_id_by_port);
/*
* Stop device
const struct hc_driver *driver;
struct xhci_hcd *xhci;
struct resource *res;
+ struct usb_hcd *usb3_hcd;
struct usb_hcd *hcd;
int ret = -ENODEV;
int wakeup_irq;
xhci = hcd_to_xhci(hcd);
xhci->main_hcd = hcd;
+ xhci->allow_single_roothub = 1;
/*
* imod_interval is the interrupt moderation value in nanoseconds.
xhci->imod_interval = 5000;
device_property_read_u32(dev, "imod-interval-ns", &xhci->imod_interval);
- xhci->shared_hcd = usb_create_shared_hcd(driver, dev,
- dev_name(dev), hcd);
- if (!xhci->shared_hcd) {
- ret = -ENOMEM;
- goto disable_device_wakeup;
- }
-
ret = usb_add_hcd(hcd, irq, IRQF_SHARED);
if (ret)
- goto put_usb3_hcd;
+ goto disable_device_wakeup;
- if (HCC_MAX_PSA(xhci->hcc_params) >= 4 &&
+ if (!xhci_has_one_roothub(xhci)) {
+ xhci->shared_hcd = usb_create_shared_hcd(driver, dev,
+ dev_name(dev), hcd);
+ if (!xhci->shared_hcd) {
+ ret = -ENOMEM;
+ goto dealloc_usb2_hcd;
+ }
+ }
+
+ usb3_hcd = xhci_get_usb3_hcd(xhci);
+ if (usb3_hcd && HCC_MAX_PSA(xhci->hcc_params) >= 4 &&
!(xhci->quirks & XHCI_BROKEN_STREAMS))
- xhci->shared_hcd->can_do_streams = 1;
+ usb3_hcd->can_do_streams = 1;
- ret = usb_add_hcd(xhci->shared_hcd, irq, IRQF_SHARED);
- if (ret)
- goto dealloc_usb2_hcd;
+ if (xhci->shared_hcd) {
+ ret = usb_add_hcd(xhci->shared_hcd, irq, IRQF_SHARED);
+ if (ret)
+ goto put_usb3_hcd;
+ }
if (wakeup_irq > 0) {
ret = dev_pm_set_dedicated_wake_irq_reverse(dev, wakeup_irq);
dealloc_usb3_hcd:
usb_remove_hcd(xhci->shared_hcd);
- xhci->shared_hcd = NULL;
-
-dealloc_usb2_hcd:
- usb_remove_hcd(hcd);
put_usb3_hcd:
- xhci_mtk_sch_exit(mtk);
usb_put_hcd(xhci->shared_hcd);
+dealloc_usb2_hcd:
+ xhci_mtk_sch_exit(mtk);
+ usb_remove_hcd(hcd);
+
disable_device_wakeup:
device_init_wakeup(dev, false);
dev_pm_clear_wake_irq(dev);
device_init_wakeup(dev, false);
- usb_remove_hcd(shared_hcd);
- xhci->shared_hcd = NULL;
+ if (shared_hcd) {
+ usb_remove_hcd(shared_hcd);
+ xhci->shared_hcd = NULL;
+ }
usb_remove_hcd(hcd);
- usb_put_hcd(shared_hcd);
+
+ if (shared_hcd)
+ usb_put_hcd(shared_hcd);
+
usb_put_hcd(hcd);
xhci_mtk_sch_exit(mtk);
clk_bulk_disable_unprepare(BULK_CLKS_NUM, mtk->clks);
struct xhci_hcd_mtk *mtk = dev_get_drvdata(dev);
struct usb_hcd *hcd = mtk->hcd;
struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ struct usb_hcd *shared_hcd = xhci->shared_hcd;
int ret;
xhci_dbg(xhci, "%s: stop port polling\n", __func__);
clear_bit(HCD_FLAG_POLL_RH, &hcd->flags);
del_timer_sync(&hcd->rh_timer);
- clear_bit(HCD_FLAG_POLL_RH, &xhci->shared_hcd->flags);
- del_timer_sync(&xhci->shared_hcd->rh_timer);
+ if (shared_hcd) {
+ clear_bit(HCD_FLAG_POLL_RH, &shared_hcd->flags);
+ del_timer_sync(&shared_hcd->rh_timer);
+ }
ret = xhci_mtk_host_disable(mtk);
if (ret)
restart_poll_rh:
xhci_dbg(xhci, "%s: restart port polling\n", __func__);
- set_bit(HCD_FLAG_POLL_RH, &xhci->shared_hcd->flags);
- usb_hcd_poll_rh_status(xhci->shared_hcd);
+ if (shared_hcd) {
+ set_bit(HCD_FLAG_POLL_RH, &shared_hcd->flags);
+ usb_hcd_poll_rh_status(shared_hcd);
+ }
set_bit(HCD_FLAG_POLL_RH, &hcd->flags);
usb_hcd_poll_rh_status(hcd);
return ret;
struct xhci_hcd_mtk *mtk = dev_get_drvdata(dev);
struct usb_hcd *hcd = mtk->hcd;
struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ struct usb_hcd *shared_hcd = xhci->shared_hcd;
int ret;
usb_wakeup_set(mtk, false);
goto disable_clks;
xhci_dbg(xhci, "%s: restart port polling\n", __func__);
- set_bit(HCD_FLAG_POLL_RH, &xhci->shared_hcd->flags);
- usb_hcd_poll_rh_status(xhci->shared_hcd);
+ if (shared_hcd) {
+ set_bit(HCD_FLAG_POLL_RH, &shared_hcd->flags);
+ usb_hcd_poll_rh_status(shared_hcd);
+ }
set_bit(HCD_FLAG_POLL_RH, &hcd->flags);
usb_hcd_poll_rh_status(hcd);
return 0;
#define PCI_DEVICE_ID_INTEL_TIGER_LAKE_XHCI 0x9a13
#define PCI_DEVICE_ID_INTEL_MAPLE_RIDGE_XHCI 0x1138
#define PCI_DEVICE_ID_INTEL_ALDER_LAKE_PCH_XHCI 0x51ed
+#define PCI_DEVICE_ID_INTEL_ALDER_LAKE_N_PCH_XHCI 0x54ed
#define PCI_DEVICE_ID_AMD_RENOIR_XHCI 0x1639
#define PCI_DEVICE_ID_AMD_PROMONTORYA_4 0x43b9
xhci->quirks |= XHCI_MISSING_CAS;
if (pdev->vendor == PCI_VENDOR_ID_INTEL &&
- pdev->device == PCI_DEVICE_ID_INTEL_ALDER_LAKE_PCH_XHCI)
+ (pdev->device == PCI_DEVICE_ID_INTEL_ALDER_LAKE_PCH_XHCI ||
+ pdev->device == PCI_DEVICE_ID_INTEL_ALDER_LAKE_N_PCH_XHCI))
xhci->quirks |= XHCI_RESET_TO_DEFAULT;
if (pdev->vendor == PCI_VENDOR_ID_INTEL &&
return retval;
}
+static int xhci_pci_poweroff_late(struct usb_hcd *hcd, bool do_wakeup)
+{
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ struct xhci_port *port;
+ struct usb_device *udev;
+ unsigned int slot_id;
+ u32 portsc;
+ int i;
+
+ /*
+ * Systems with XHCI_RESET_TO_DEFAULT quirk have boot firmware that
+ * cause significant boot delay if usb ports are in suspended U3 state
+ * during boot. Some USB devices survive in U3 state over S4 hibernate
+ *
+ * Disable ports that are in U3 if remote wake is not enabled for either
+ * host controller or connected device
+ */
+
+ if (!(xhci->quirks & XHCI_RESET_TO_DEFAULT))
+ return 0;
+
+ for (i = 0; i < HCS_MAX_PORTS(xhci->hcs_params1); i++) {
+ port = &xhci->hw_ports[i];
+ portsc = readl(port->addr);
+
+ if ((portsc & PORT_PLS_MASK) != XDEV_U3)
+ continue;
+
+ slot_id = xhci_find_slot_id_by_port(port->rhub->hcd, xhci,
+ port->hcd_portnum + 1);
+ if (!slot_id || !xhci->devs[slot_id]) {
+ xhci_err(xhci, "No dev for slot_id %d for port %d-%d in U3\n",
+ slot_id, port->rhub->hcd->self.busnum, port->hcd_portnum + 1);
+ continue;
+ }
+
+ udev = xhci->devs[slot_id]->udev;
+
+ /* if wakeup is enabled then don't disable the port */
+ if (udev->do_remote_wakeup && do_wakeup)
+ continue;
+
+ xhci_dbg(xhci, "port %d-%d in U3 without wakeup, disable it\n",
+ port->rhub->hcd->self.busnum, port->hcd_portnum + 1);
+ portsc = xhci_port_state_to_neutral(portsc);
+ writel(portsc | PORT_PE, port->addr);
+ }
+
+ return 0;
+}
+
static void xhci_pci_shutdown(struct usb_hcd *hcd)
{
struct xhci_hcd *xhci = hcd_to_xhci(hcd);
/* suspend and resume implemented later */
.shutdown = usb_hcd_pci_shutdown,
-#ifdef CONFIG_PM
.driver = {
- .pm = &usb_hcd_pci_pm_ops
- },
+#ifdef CONFIG_PM
+ .pm = &usb_hcd_pci_pm_ops,
#endif
+ .probe_type = PROBE_PREFER_ASYNCHRONOUS,
+ },
};
static int __init xhci_pci_init(void)
#ifdef CONFIG_PM
xhci_pci_hc_driver.pci_suspend = xhci_pci_suspend;
xhci_pci_hc_driver.pci_resume = xhci_pci_resume;
+ xhci_pci_hc_driver.pci_poweroff_late = xhci_pci_poweroff_late;
xhci_pci_hc_driver.shutdown = xhci_pci_shutdown;
#endif
return pci_register_driver(&xhci_pci_driver);
}
static int xhci_handle_halted_endpoint(struct xhci_hcd *xhci,
- struct xhci_virt_ep *ep, unsigned int stream_id,
+ struct xhci_virt_ep *ep,
struct xhci_td *td,
enum xhci_ep_reset_type reset_type)
{
td->status = -EPROTO;
}
/* reset ep, reset handler cleans up cancelled tds */
- err = xhci_handle_halted_endpoint(xhci, ep, 0, td,
- reset_type);
+ err = xhci_handle_halted_endpoint(xhci, ep, td, reset_type);
if (err)
break;
ep->ep_state &= ~EP_STOP_CMD_PENDING;
}
/* Almost same procedure as for STALL_ERROR below */
xhci_clear_hub_tt_buffer(xhci, td, ep);
- xhci_handle_halted_endpoint(xhci, ep, ep_ring->stream_id, td,
- EP_HARD_RESET);
+ xhci_handle_halted_endpoint(xhci, ep, td, EP_HARD_RESET);
return 0;
case COMP_STALL_ERROR:
/*
if (ep->ep_index != 0)
xhci_clear_hub_tt_buffer(xhci, td, ep);
- xhci_handle_halted_endpoint(xhci, ep, ep_ring->stream_id, td,
- EP_HARD_RESET);
+ xhci_handle_halted_endpoint(xhci, ep, td, EP_HARD_RESET);
return 0; /* xhci_handle_halted_endpoint marked td cancelled */
default:
switch (trb_comp_code) {
case COMP_SUCCESS:
- ep_ring->err_count = 0;
+ ep->err_count = 0;
/* handle success with untransferred data as short packet */
if (ep_trb != td->last_trb || remaining) {
xhci_warn(xhci, "WARN Successful completion on short TX\n");
break;
case COMP_USB_TRANSACTION_ERROR:
if (xhci->quirks & XHCI_NO_SOFT_RETRY ||
- (ep_ring->err_count++ > MAX_SOFT_RETRY) ||
+ (ep->err_count++ > MAX_SOFT_RETRY) ||
le32_to_cpu(slot_ctx->tt_info) & TT_SLOT)
break;
td->status = 0;
- xhci_handle_halted_endpoint(xhci, ep, ep_ring->stream_id, td,
- EP_SOFT_RESET);
+ xhci_handle_halted_endpoint(xhci, ep, td, EP_SOFT_RESET);
return 0;
default:
/* do nothing */
case COMP_USB_TRANSACTION_ERROR:
case COMP_INVALID_STREAM_TYPE_ERROR:
case COMP_INVALID_STREAM_ID_ERROR:
- xhci_handle_halted_endpoint(xhci, ep, 0, NULL,
- EP_SOFT_RESET);
+ xhci_dbg(xhci, "Stream transaction error ep %u no id\n",
+ ep_index);
+ if (ep->err_count++ > MAX_SOFT_RETRY)
+ xhci_handle_halted_endpoint(xhci, ep, NULL,
+ EP_HARD_RESET);
+ else
+ xhci_handle_halted_endpoint(xhci, ep, NULL,
+ EP_SOFT_RESET);
goto cleanup;
case COMP_RING_UNDERRUN:
case COMP_RING_OVERRUN:
if (trb_comp_code == COMP_STALL_ERROR ||
xhci_requires_manual_halt_cleanup(xhci, ep_ctx,
trb_comp_code)) {
- xhci_handle_halted_endpoint(xhci, ep,
- ep_ring->stream_id,
- NULL,
+ xhci_handle_halted_endpoint(xhci, ep, NULL,
EP_HARD_RESET);
}
goto cleanup;
if (trb_comp_code == COMP_STALL_ERROR ||
xhci_requires_manual_halt_cleanup(xhci, ep_ctx,
trb_comp_code))
- xhci_handle_halted_endpoint(xhci, ep,
- ep_ring->stream_id,
- td, EP_HARD_RESET);
+ xhci_handle_halted_endpoint(xhci, ep, td,
+ EP_HARD_RESET);
goto cleanup;
}
if (!(status & STS_EINT))
goto out;
+ if (status & STS_HCE) {
+ xhci_warn(xhci, "WARNING: Host Controller Error\n");
+ goto out;
+ }
+
if (status & STS_FATAL) {
xhci_warn(xhci, "WARNING: Host System Error\n");
xhci_halt(xhci);
* have to restore the device state to the previous state
*/
struct xhci_ring *new_ring;
+ unsigned int err_count;
unsigned int ep_state;
#define SET_DEQ_PENDING (1 << 0)
#define EP_HALTED (1 << 1) /* For stall handling */
* if we own the TRB (if we are the consumer). See section 4.9.1.
*/
u32 cycle_state;
- unsigned int err_count;
unsigned int stream_id;
unsigned int num_segs;
unsigned int num_trbs_free;
config USB_ONBOARD_HUB
tristate "Onboard USB hub support"
- depends on OF || COMPILE_TEST
+ depends on OF
help
Say Y here if you want to support discrete onboard USB hubs that
don't require an additional control bus for initialization, but
char data[30 *3 + 4];
char *d = data;
int m = (sizeof(data) - 1) / 3 - 1;
- int l = 0;
struct u132_target *target = &ftdi->target[ed];
struct u132_command *command = &ftdi->command[
COMMAND_MASK & ftdi->command_next];
} else if (i++ < m) {
int w = sprintf(d, " %02X", *b++);
d += w;
- l += w;
} else
d += sprintf(d, " ..");
}
int long_stop = 10;
int retry_on_timeout = 5;
int retry_on_empty = 10;
- int err_count = 0;
retval = ftdi_elan_flush_input_fifo(ftdi);
if (retval)
return retval;
continue;
}
} else {
- err_count += 1;
dev_err(&ftdi->udev->dev, "error = %d\n",
retval);
if (read_stop-- > 0) {
.llseek = noop_llseek,
};
-static char *iowarrior_devnode(struct device *dev, umode_t *mode)
+static char *iowarrior_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "usb/%s", dev_name(dev));
}
.llseek = tower_llseek,
};
-static char *legousbtower_devnode(struct device *dev, umode_t *mode)
+static char *legousbtower_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "usb/%s", dev_name(dev));
}
/************************** USB driver **************************/
+#define VENDOR_ID_GENESYS 0x05e3
#define VENDOR_ID_MICROCHIP 0x0424
#define VENDOR_ID_REALTEK 0x0bda
#define VENDOR_ID_TI 0x0451
}
static const struct usb_device_id onboard_hub_id_table[] = {
+ { USB_DEVICE(VENDOR_ID_GENESYS, 0x0608) }, /* Genesys Logic GL850G USB 2.0 */
{ USB_DEVICE(VENDOR_ID_MICROCHIP, 0x2514) }, /* USB2514B USB 2.0 */
{ USB_DEVICE(VENDOR_ID_REALTEK, 0x0411) }, /* RTS5411 USB 3.1 */
{ USB_DEVICE(VENDOR_ID_REALTEK, 0x5411) }, /* RTS5411 USB 2.1 */
.reset_us = 3000,
};
+static const struct onboard_hub_pdata genesys_gl850g_data = {
+ .reset_us = 3,
+};
+
static const struct of_device_id onboard_hub_match[] = {
{ .compatible = "usb424,2514", .data = µchip_usb424_data, },
{ .compatible = "usb451,8140", .data = &ti_tusb8041_data, },
{ .compatible = "usb451,8142", .data = &ti_tusb8041_data, },
+ { .compatible = "usb5e3,608", .data = &genesys_gl850g_data, },
{ .compatible = "usbbda,411", .data = &realtek_rts5411_data, },
{ .compatible = "usbbda,5411", .data = &realtek_rts5411_data, },
{ .compatible = "usbbda,414", .data = &realtek_rts5411_data, },
config USB_SISUSBVGA
tristate "USB 2.0 SVGA dongle support (Net2280/SiS315)"
depends on (USB_MUSB_HDRC || USB_EHCI_HCD)
- select FONT_SUPPORT if USB_SISUSBVGA_CON
help
Say Y here if you intend to attach a USB2VGA dongle based on a
Net2280 and a SiS315 chip.
To compile this driver as a module, choose M here; the module will be
called sisusbvga. If unsure, say N.
-
-config USB_SISUSBVGA_CON
- bool "Text console and mode switching support" if USB_SISUSBVGA
- depends on VT && BROKEN
- select FONT_8x16
- help
- Say Y here if you want a VGA text console via the USB dongle or
- want to support userland applications that utilize the driver's
- display mode switching capabilities.
-
- Note that this console supports VGA/EGA text mode only.
-
- By default, the console part of the driver will not kick in when
- the driver is initialized. If you want the driver to take over
- one or more of the consoles, you need to specify the number of
- the first and last consoles (starting at 1) as driver parameters.
-
- For example, if the driver is compiled as a module:
-
- modprobe sisusbvga first=1 last=5
-
- If you use hotplug, add this to your modutils config files with
- the "options" keyword, such as eg.
-
- options sisusbvga first=1 last=5
-
- If the driver is compiled into the kernel image, the parameters
- must be given in the kernel command like, such as
-
- sisusbvga.first=1 sisusbvga.last=5
-
-
-
#
obj-$(CONFIG_USB_SISUSBVGA) += sisusbvga.o
-
-sisusbvga-y := sisusb.o
-sisusbvga-$(CONFIG_USB_SISUSBVGA_CON) += sisusb_con.o sisusb_init.o
/* Include console and mode switching code? */
-#include <linux/console.h>
#include <linux/vt_kern.h>
#include "sisusb_struct.h"
unsigned char gfxinit; /* graphics core initialized? */
unsigned short chipid, chipvendor;
unsigned short chiprevision;
-#ifdef CONFIG_USB_SISUSBVGA_CON
- struct SiS_Private *SiS_Pr;
- unsigned long scrbuf;
- unsigned int scrbuf_size;
- int haveconsole, con_first, con_last;
- int havethisconsole[MAX_NR_CONSOLES];
- int textmodedestroyed;
- unsigned int sisusb_num_columns; /* real number, not vt's idea */
- int cur_start_addr, con_rolled_over;
- int sisusb_cursor_loc, bad_cursor_pos;
- int sisusb_cursor_size_from;
- int sisusb_cursor_size_to;
- int current_font_height, current_font_512;
- int font_backup_size, font_backup_height, font_backup_512;
- char *font_backup;
- int font_slot;
- struct vc_data *sisusb_display_fg;
- int is_gfx;
- int con_blanked;
-#endif
};
#define to_sisusb_dev(d) container_of(d, struct sisusb_usb_data, kref)
+++ /dev/null
-// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
-/*
- * sisusb - usb kernel driver for SiS315(E) based USB2VGA dongles
- *
- * VGA text mode console part
- *
- * Copyright (C) 2005 by Thomas Winischhofer, Vienna, Austria
- *
- * If distributed as part of the Linux kernel, this code is licensed under the
- * terms of the GPL v2.
- *
- * Otherwise, the following license terms apply:
- *
- * * Redistribution and use in source and binary forms, with or without
- * * modification, are permitted provided that the following conditions
- * * are met:
- * * 1) Redistributions of source code must retain the above copyright
- * * notice, this list of conditions and the following disclaimer.
- * * 2) Redistributions in binary form must reproduce the above copyright
- * * notice, this list of conditions and the following disclaimer in the
- * * documentation and/or other materials provided with the distribution.
- * * 3) The name of the author may not be used to endorse or promote products
- * * derived from this software without specific psisusbr written permission.
- * *
- * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESSED OR
- * * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
- * * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
- * * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
- * * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
- * * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
- * * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- * Author: Thomas Winischhofer <thomas@winischhofer.net>
- *
- * Portions based on vgacon.c which are
- * Created 28 Sep 1997 by Geert Uytterhoeven
- * Rewritten by Martin Mares <mj@ucw.cz>, July 1998
- * based on code Copyright (C) 1991, 1992 Linus Torvalds
- * 1995 Jay Estabrook
- *
- * A note on using in_atomic() in here: We can't handle console
- * calls from non-schedulable context due to our USB-dependend
- * nature. For now, this driver just ignores any calls if it
- * detects this state.
- *
- */
-
-#include <linux/mutex.h>
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/signal.h>
-#include <linux/fs.h>
-#include <linux/usb.h>
-#include <linux/tty.h>
-#include <linux/console.h>
-#include <linux/string.h>
-#include <linux/kd.h>
-#include <linux/init.h>
-#include <linux/vt_kern.h>
-#include <linux/selection.h>
-#include <linux/spinlock.h>
-#include <linux/kref.h>
-#include <linux/ioport.h>
-#include <linux/interrupt.h>
-#include <linux/vmalloc.h>
-
-#include "sisusb.h"
-#include "sisusb_init.h"
-
-/* vc_data -> sisusb conversion table */
-static struct sisusb_usb_data *mysisusbs[MAX_NR_CONSOLES];
-
-/* Forward declaration */
-static const struct consw sisusb_con;
-
-static inline void
-sisusbcon_memsetw(u16 *s, u16 c, unsigned int count)
-{
- memset16(s, c, count / 2);
-}
-
-static inline void
-sisusb_initialize(struct sisusb_usb_data *sisusb)
-{
- /* Reset cursor and start address */
- if (sisusb_setidxreg(sisusb, SISCR, 0x0c, 0x00))
- return;
- if (sisusb_setidxreg(sisusb, SISCR, 0x0d, 0x00))
- return;
- if (sisusb_setidxreg(sisusb, SISCR, 0x0e, 0x00))
- return;
- sisusb_setidxreg(sisusb, SISCR, 0x0f, 0x00);
-}
-
-static inline void
-sisusbcon_set_start_address(struct sisusb_usb_data *sisusb, struct vc_data *c)
-{
- sisusb->cur_start_addr = (c->vc_visible_origin - sisusb->scrbuf) / 2;
-
- sisusb_setidxreg(sisusb, SISCR, 0x0c, (sisusb->cur_start_addr >> 8));
- sisusb_setidxreg(sisusb, SISCR, 0x0d, (sisusb->cur_start_addr & 0xff));
-}
-
-void
-sisusb_set_cursor(struct sisusb_usb_data *sisusb, unsigned int location)
-{
- if (sisusb->sisusb_cursor_loc == location)
- return;
-
- sisusb->sisusb_cursor_loc = location;
-
- /* Hardware bug: Text cursor appears twice or not at all
- * at some positions. Work around it with the cursor skew
- * bits.
- */
-
- if ((location & 0x0007) == 0x0007) {
- sisusb->bad_cursor_pos = 1;
- location--;
- if (sisusb_setidxregandor(sisusb, SISCR, 0x0b, 0x1f, 0x20))
- return;
- } else if (sisusb->bad_cursor_pos) {
- if (sisusb_setidxregand(sisusb, SISCR, 0x0b, 0x1f))
- return;
- sisusb->bad_cursor_pos = 0;
- }
-
- if (sisusb_setidxreg(sisusb, SISCR, 0x0e, (location >> 8)))
- return;
- sisusb_setidxreg(sisusb, SISCR, 0x0f, (location & 0xff));
-}
-
-static inline struct sisusb_usb_data *
-sisusb_get_sisusb(unsigned short console)
-{
- return mysisusbs[console];
-}
-
-static inline int
-sisusb_sisusb_valid(struct sisusb_usb_data *sisusb)
-{
- if (!sisusb->present || !sisusb->ready || !sisusb->sisusb_dev)
- return 0;
-
- return 1;
-}
-
-static struct sisusb_usb_data *
-sisusb_get_sisusb_lock_and_check(unsigned short console)
-{
- struct sisusb_usb_data *sisusb;
-
- /* We can't handle console calls in non-schedulable
- * context due to our locks and the USB transport.
- * So we simply ignore them. This should only affect
- * some calls to printk.
- */
- if (in_atomic())
- return NULL;
-
- sisusb = sisusb_get_sisusb(console);
- if (!sisusb)
- return NULL;
-
- mutex_lock(&sisusb->lock);
-
- if (!sisusb_sisusb_valid(sisusb) ||
- !sisusb->havethisconsole[console]) {
- mutex_unlock(&sisusb->lock);
- return NULL;
- }
-
- return sisusb;
-}
-
-static int
-sisusb_is_inactive(struct vc_data *c, struct sisusb_usb_data *sisusb)
-{
- if (sisusb->is_gfx ||
- sisusb->textmodedestroyed ||
- c->vc_mode != KD_TEXT)
- return 1;
-
- return 0;
-}
-
-/* con_startup console interface routine */
-static const char *
-sisusbcon_startup(void)
-{
- return "SISUSBCON";
-}
-
-/* con_init console interface routine */
-static void
-sisusbcon_init(struct vc_data *c, int init)
-{
- struct sisusb_usb_data *sisusb;
- int cols, rows;
-
- /* This is called by do_take_over_console(),
- * ie by us/under our control. It is
- * only called after text mode and fonts
- * are set up/restored.
- */
-
- sisusb = sisusb_get_sisusb(c->vc_num);
- if (!sisusb)
- return;
-
- mutex_lock(&sisusb->lock);
-
- if (!sisusb_sisusb_valid(sisusb)) {
- mutex_unlock(&sisusb->lock);
- return;
- }
-
- c->vc_can_do_color = 1;
-
- c->vc_complement_mask = 0x7700;
-
- c->vc_hi_font_mask = sisusb->current_font_512 ? 0x0800 : 0;
-
- sisusb->haveconsole = 1;
-
- sisusb->havethisconsole[c->vc_num] = 1;
-
- /* We only support 640x400 */
- c->vc_scan_lines = 400;
-
- c->vc_font.height = sisusb->current_font_height;
-
- /* We only support width = 8 */
- cols = 80;
- rows = c->vc_scan_lines / c->vc_font.height;
-
- /* Increment usage count for our sisusb.
- * Doing so saves us from upping/downing
- * the disconnect semaphore; we can't
- * lose our sisusb until this is undone
- * in con_deinit. For all other console
- * interface functions, it suffices to
- * use sisusb->lock and do a quick check
- * of sisusb for device disconnection.
- */
- kref_get(&sisusb->kref);
-
- if (!*c->uni_pagedict_loc)
- con_set_default_unimap(c);
-
- mutex_unlock(&sisusb->lock);
-
- if (init) {
- c->vc_cols = cols;
- c->vc_rows = rows;
- } else
- vc_resize(c, cols, rows);
-}
-
-/* con_deinit console interface routine */
-static void
-sisusbcon_deinit(struct vc_data *c)
-{
- struct sisusb_usb_data *sisusb;
- int i;
-
- /* This is called by do_take_over_console()
- * and others, ie not under our control.
- */
-
- sisusb = sisusb_get_sisusb(c->vc_num);
- if (!sisusb)
- return;
-
- mutex_lock(&sisusb->lock);
-
- /* Clear ourselves in mysisusbs */
- mysisusbs[c->vc_num] = NULL;
-
- sisusb->havethisconsole[c->vc_num] = 0;
-
- /* Free our font buffer if all consoles are gone */
- if (sisusb->font_backup) {
- for(i = 0; i < MAX_NR_CONSOLES; i++) {
- if (sisusb->havethisconsole[c->vc_num])
- break;
- }
- if (i == MAX_NR_CONSOLES) {
- vfree(sisusb->font_backup);
- sisusb->font_backup = NULL;
- }
- }
-
- mutex_unlock(&sisusb->lock);
-
- /* decrement the usage count on our sisusb */
- kref_put(&sisusb->kref, sisusb_delete);
-}
-
-/* interface routine */
-static u8
-sisusbcon_build_attr(struct vc_data *c, u8 color, enum vc_intensity intensity,
- bool blink, bool underline, bool reverse,
- bool unused)
-{
- u8 attr = color;
-
- if (underline)
- attr = (attr & 0xf0) | c->vc_ulcolor;
- else if (intensity == VCI_HALF_BRIGHT)
- attr = (attr & 0xf0) | c->vc_halfcolor;
-
- if (reverse)
- attr = ((attr) & 0x88) |
- ((((attr) >> 4) |
- ((attr) << 4)) & 0x77);
-
- if (blink)
- attr ^= 0x80;
-
- if (intensity == VCI_BOLD)
- attr ^= 0x08;
-
- return attr;
-}
-
-/* Interface routine */
-static void
-sisusbcon_invert_region(struct vc_data *vc, u16 *p, int count)
-{
- /* Invert a region. This is called with a pointer
- * to the console's internal screen buffer. So we
- * simply do the inversion there and rely on
- * a call to putc(s) to update the real screen.
- */
-
- while (count--) {
- u16 a = *p;
-
- *p++ = ((a) & 0x88ff) |
- (((a) & 0x7000) >> 4) |
- (((a) & 0x0700) << 4);
- }
-}
-
-static inline void *sisusb_vaddr(const struct sisusb_usb_data *sisusb,
- const struct vc_data *c, unsigned int x, unsigned int y)
-{
- return (u16 *)c->vc_origin + y * sisusb->sisusb_num_columns + x;
-}
-
-static inline unsigned long sisusb_haddr(const struct sisusb_usb_data *sisusb,
- const struct vc_data *c, unsigned int x, unsigned int y)
-{
- unsigned long offset = c->vc_origin - sisusb->scrbuf;
-
- /* 2 bytes per each character */
- offset += 2 * (y * sisusb->sisusb_num_columns + x);
-
- return sisusb->vrambase + offset;
-}
-
-/* Interface routine */
-static void
-sisusbcon_putc(struct vc_data *c, int ch, int y, int x)
-{
- struct sisusb_usb_data *sisusb;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return;
-
- /* sisusb->lock is down */
- if (sisusb_is_inactive(c, sisusb)) {
- mutex_unlock(&sisusb->lock);
- return;
- }
-
- sisusb_copy_memory(sisusb, sisusb_vaddr(sisusb, c, x, y),
- sisusb_haddr(sisusb, c, x, y), 2);
-
- mutex_unlock(&sisusb->lock);
-}
-
-/* Interface routine */
-static void
-sisusbcon_putcs(struct vc_data *c, const unsigned short *s,
- int count, int y, int x)
-{
- struct sisusb_usb_data *sisusb;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return;
-
- /* sisusb->lock is down */
-
- /* Need to put the characters into the buffer ourselves,
- * because the vt does this AFTER calling us.
- */
-
- memcpy(sisusb_vaddr(sisusb, c, x, y), s, count * 2);
-
- if (sisusb_is_inactive(c, sisusb)) {
- mutex_unlock(&sisusb->lock);
- return;
- }
-
- sisusb_copy_memory(sisusb, sisusb_vaddr(sisusb, c, x, y),
- sisusb_haddr(sisusb, c, x, y), count * 2);
-
- mutex_unlock(&sisusb->lock);
-}
-
-/* Interface routine */
-static void
-sisusbcon_clear(struct vc_data *c, int y, int x, int height, int width)
-{
- struct sisusb_usb_data *sisusb;
- u16 eattr = c->vc_video_erase_char;
- int i, length, cols;
- u16 *dest;
-
- if (width <= 0 || height <= 0)
- return;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return;
-
- /* sisusb->lock is down */
-
- /* Need to clear buffer ourselves, because the vt does
- * this AFTER calling us.
- */
-
- dest = sisusb_vaddr(sisusb, c, x, y);
-
- cols = sisusb->sisusb_num_columns;
-
- if (width > cols)
- width = cols;
-
- if (x == 0 && width >= c->vc_cols) {
-
- sisusbcon_memsetw(dest, eattr, height * cols * 2);
-
- } else {
-
- for (i = height; i > 0; i--, dest += cols)
- sisusbcon_memsetw(dest, eattr, width * 2);
-
- }
-
- if (sisusb_is_inactive(c, sisusb)) {
- mutex_unlock(&sisusb->lock);
- return;
- }
-
- length = ((height * cols) - x - (cols - width - x)) * 2;
-
-
- sisusb_copy_memory(sisusb, sisusb_vaddr(sisusb, c, x, y),
- sisusb_haddr(sisusb, c, x, y), length);
-
- mutex_unlock(&sisusb->lock);
-}
-
-/* interface routine */
-static int
-sisusbcon_switch(struct vc_data *c)
-{
- struct sisusb_usb_data *sisusb;
- int length;
-
- /* Returnvalue 0 means we have fully restored screen,
- * and vt doesn't need to call do_update_region().
- * Returnvalue != 0 naturally means the opposite.
- */
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return 0;
-
- /* sisusb->lock is down */
-
- /* Don't write to screen if in gfx mode */
- if (sisusb_is_inactive(c, sisusb)) {
- mutex_unlock(&sisusb->lock);
- return 0;
- }
-
- /* That really should not happen. It would mean we are
- * being called while the vc is using its private buffer
- * as origin.
- */
- if (c->vc_origin == (unsigned long)c->vc_screenbuf) {
- mutex_unlock(&sisusb->lock);
- dev_dbg(&sisusb->sisusb_dev->dev, "ASSERT ORIGIN != SCREENBUF!\n");
- return 0;
- }
-
- /* Check that we don't copy too much */
- length = min((int)c->vc_screenbuf_size,
- (int)(sisusb->scrbuf + sisusb->scrbuf_size - c->vc_origin));
-
- /* Restore the screen contents */
- memcpy((u16 *)c->vc_origin, (u16 *)c->vc_screenbuf, length);
-
- sisusb_copy_memory(sisusb, (u8 *)c->vc_origin,
- sisusb_haddr(sisusb, c, 0, 0), length);
-
- mutex_unlock(&sisusb->lock);
-
- return 0;
-}
-
-/* interface routine */
-static void
-sisusbcon_save_screen(struct vc_data *c)
-{
- struct sisusb_usb_data *sisusb;
- int length;
-
- /* Save the current screen contents to vc's private
- * buffer.
- */
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return;
-
- /* sisusb->lock is down */
-
- if (sisusb_is_inactive(c, sisusb)) {
- mutex_unlock(&sisusb->lock);
- return;
- }
-
- /* Check that we don't copy too much */
- length = min((int)c->vc_screenbuf_size,
- (int)(sisusb->scrbuf + sisusb->scrbuf_size - c->vc_origin));
-
- /* Save the screen contents to vc's private buffer */
- memcpy((u16 *)c->vc_screenbuf, (u16 *)c->vc_origin, length);
-
- mutex_unlock(&sisusb->lock);
-}
-
-/* interface routine */
-static void
-sisusbcon_set_palette(struct vc_data *c, const unsigned char *table)
-{
- struct sisusb_usb_data *sisusb;
- int i, j;
-
- /* Return value not used by vt */
-
- if (!con_is_visible(c))
- return;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return;
-
- /* sisusb->lock is down */
-
- if (sisusb_is_inactive(c, sisusb)) {
- mutex_unlock(&sisusb->lock);
- return;
- }
-
- for (i = j = 0; i < 16; i++) {
- if (sisusb_setreg(sisusb, SISCOLIDX, table[i]))
- break;
- if (sisusb_setreg(sisusb, SISCOLDATA, c->vc_palette[j++] >> 2))
- break;
- if (sisusb_setreg(sisusb, SISCOLDATA, c->vc_palette[j++] >> 2))
- break;
- if (sisusb_setreg(sisusb, SISCOLDATA, c->vc_palette[j++] >> 2))
- break;
- }
-
- mutex_unlock(&sisusb->lock);
-}
-
-/* interface routine */
-static int
-sisusbcon_blank(struct vc_data *c, int blank, int mode_switch)
-{
- struct sisusb_usb_data *sisusb;
- u8 sr1, cr17, pmreg, cr63;
- int ret = 0;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return 0;
-
- /* sisusb->lock is down */
-
- if (mode_switch)
- sisusb->is_gfx = blank ? 1 : 0;
-
- if (sisusb_is_inactive(c, sisusb)) {
- mutex_unlock(&sisusb->lock);
- return 0;
- }
-
- switch (blank) {
-
- case 1: /* Normal blanking: Clear screen */
- case -1:
- sisusbcon_memsetw((u16 *)c->vc_origin,
- c->vc_video_erase_char,
- c->vc_screenbuf_size);
- sisusb_copy_memory(sisusb, (u8 *)c->vc_origin,
- sisusb_haddr(sisusb, c, 0, 0),
- c->vc_screenbuf_size);
- sisusb->con_blanked = 1;
- ret = 1;
- break;
-
- default: /* VESA blanking */
- switch (blank) {
- case 0: /* Unblank */
- sr1 = 0x00;
- cr17 = 0x80;
- pmreg = 0x00;
- cr63 = 0x00;
- ret = 1;
- sisusb->con_blanked = 0;
- break;
- case VESA_VSYNC_SUSPEND + 1:
- sr1 = 0x20;
- cr17 = 0x80;
- pmreg = 0x80;
- cr63 = 0x40;
- break;
- case VESA_HSYNC_SUSPEND + 1:
- sr1 = 0x20;
- cr17 = 0x80;
- pmreg = 0x40;
- cr63 = 0x40;
- break;
- case VESA_POWERDOWN + 1:
- sr1 = 0x20;
- cr17 = 0x00;
- pmreg = 0xc0;
- cr63 = 0x40;
- break;
- default:
- mutex_unlock(&sisusb->lock);
- return -EINVAL;
- }
-
- sisusb_setidxregandor(sisusb, SISSR, 0x01, ~0x20, sr1);
- sisusb_setidxregandor(sisusb, SISCR, 0x17, 0x7f, cr17);
- sisusb_setidxregandor(sisusb, SISSR, 0x1f, 0x3f, pmreg);
- sisusb_setidxregandor(sisusb, SISCR, 0x63, 0xbf, cr63);
-
- }
-
- mutex_unlock(&sisusb->lock);
-
- return ret;
-}
-
-/* interface routine */
-static void
-sisusbcon_scrolldelta(struct vc_data *c, int lines)
-{
- struct sisusb_usb_data *sisusb;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return;
-
- /* sisusb->lock is down */
-
- if (sisusb_is_inactive(c, sisusb)) {
- mutex_unlock(&sisusb->lock);
- return;
- }
-
- vc_scrolldelta_helper(c, lines, sisusb->con_rolled_over,
- (void *)sisusb->scrbuf, sisusb->scrbuf_size);
-
- sisusbcon_set_start_address(sisusb, c);
-
- mutex_unlock(&sisusb->lock);
-}
-
-/* Interface routine */
-static void
-sisusbcon_cursor(struct vc_data *c, int mode)
-{
- struct sisusb_usb_data *sisusb;
- int from, to, baseline;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return;
-
- /* sisusb->lock is down */
-
- if (sisusb_is_inactive(c, sisusb)) {
- mutex_unlock(&sisusb->lock);
- return;
- }
-
- if (c->vc_origin != c->vc_visible_origin) {
- c->vc_visible_origin = c->vc_origin;
- sisusbcon_set_start_address(sisusb, c);
- }
-
- if (mode == CM_ERASE) {
- sisusb_setidxregor(sisusb, SISCR, 0x0a, 0x20);
- sisusb->sisusb_cursor_size_to = -1;
- mutex_unlock(&sisusb->lock);
- return;
- }
-
- sisusb_set_cursor(sisusb, (c->vc_pos - sisusb->scrbuf) / 2);
-
- baseline = c->vc_font.height - (c->vc_font.height < 10 ? 1 : 2);
-
- switch (CUR_SIZE(c->vc_cursor_type)) {
- case CUR_BLOCK: from = 1;
- to = c->vc_font.height;
- break;
- case CUR_TWO_THIRDS: from = c->vc_font.height / 3;
- to = baseline;
- break;
- case CUR_LOWER_HALF: from = c->vc_font.height / 2;
- to = baseline;
- break;
- case CUR_LOWER_THIRD: from = (c->vc_font.height * 2) / 3;
- to = baseline;
- break;
- case CUR_NONE: from = 31;
- to = 30;
- break;
- default:
- case CUR_UNDERLINE: from = baseline - 1;
- to = baseline;
- break;
- }
-
- if (sisusb->sisusb_cursor_size_from != from ||
- sisusb->sisusb_cursor_size_to != to) {
-
- sisusb_setidxreg(sisusb, SISCR, 0x0a, from);
- sisusb_setidxregandor(sisusb, SISCR, 0x0b, 0xe0, to);
-
- sisusb->sisusb_cursor_size_from = from;
- sisusb->sisusb_cursor_size_to = to;
- }
-
- mutex_unlock(&sisusb->lock);
-}
-
-static bool
-sisusbcon_scroll_area(struct vc_data *c, struct sisusb_usb_data *sisusb,
- unsigned int t, unsigned int b, enum con_scroll dir,
- unsigned int lines)
-{
- int cols = sisusb->sisusb_num_columns;
- int length = ((b - t) * cols) * 2;
- u16 eattr = c->vc_video_erase_char;
-
- /* sisusb->lock is down */
-
- /* Scroll an area which does not match the
- * visible screen's dimensions. This needs
- * to be done separately, as it does not
- * use hardware panning.
- */
-
- switch (dir) {
-
- case SM_UP:
- memmove(sisusb_vaddr(sisusb, c, 0, t),
- sisusb_vaddr(sisusb, c, 0, t + lines),
- (b - t - lines) * cols * 2);
- sisusbcon_memsetw(sisusb_vaddr(sisusb, c, 0, b - lines),
- eattr, lines * cols * 2);
- break;
-
- case SM_DOWN:
- memmove(sisusb_vaddr(sisusb, c, 0, t + lines),
- sisusb_vaddr(sisusb, c, 0, t),
- (b - t - lines) * cols * 2);
- sisusbcon_memsetw(sisusb_vaddr(sisusb, c, 0, t), eattr,
- lines * cols * 2);
- break;
- }
-
- sisusb_copy_memory(sisusb, sisusb_vaddr(sisusb, c, 0, t),
- sisusb_haddr(sisusb, c, 0, t), length);
-
- mutex_unlock(&sisusb->lock);
-
- return true;
-}
-
-/* Interface routine */
-static bool
-sisusbcon_scroll(struct vc_data *c, unsigned int t, unsigned int b,
- enum con_scroll dir, unsigned int lines)
-{
- struct sisusb_usb_data *sisusb;
- u16 eattr = c->vc_video_erase_char;
- int copyall = 0;
- unsigned long oldorigin;
- unsigned int delta = lines * c->vc_size_row;
-
- /* Returning != 0 means we have done the scrolling successfully.
- * Returning 0 makes vt do the scrolling on its own.
- * Note that con_scroll is only called if the console is
- * visible. In that case, the origin should be our buffer,
- * not the vt's private one.
- */
-
- if (!lines)
- return true;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return false;
-
- /* sisusb->lock is down */
-
- if (sisusb_is_inactive(c, sisusb)) {
- mutex_unlock(&sisusb->lock);
- return false;
- }
-
- /* Special case */
- if (t || b != c->vc_rows)
- return sisusbcon_scroll_area(c, sisusb, t, b, dir, lines);
-
- if (c->vc_origin != c->vc_visible_origin) {
- c->vc_visible_origin = c->vc_origin;
- sisusbcon_set_start_address(sisusb, c);
- }
-
- /* limit amount to maximum realistic size */
- if (lines > c->vc_rows)
- lines = c->vc_rows;
-
- oldorigin = c->vc_origin;
-
- switch (dir) {
-
- case SM_UP:
-
- if (c->vc_scr_end + delta >=
- sisusb->scrbuf + sisusb->scrbuf_size) {
- memcpy((u16 *)sisusb->scrbuf,
- (u16 *)(oldorigin + delta),
- c->vc_screenbuf_size - delta);
- c->vc_origin = sisusb->scrbuf;
- sisusb->con_rolled_over = oldorigin - sisusb->scrbuf;
- copyall = 1;
- } else
- c->vc_origin += delta;
-
- sisusbcon_memsetw(
- (u16 *)(c->vc_origin + c->vc_screenbuf_size - delta),
- eattr, delta);
-
- break;
-
- case SM_DOWN:
-
- if (oldorigin - delta < sisusb->scrbuf) {
- memmove((void *)sisusb->scrbuf + sisusb->scrbuf_size -
- c->vc_screenbuf_size + delta,
- (u16 *)oldorigin,
- c->vc_screenbuf_size - delta);
- c->vc_origin = sisusb->scrbuf +
- sisusb->scrbuf_size -
- c->vc_screenbuf_size;
- sisusb->con_rolled_over = 0;
- copyall = 1;
- } else
- c->vc_origin -= delta;
-
- c->vc_scr_end = c->vc_origin + c->vc_screenbuf_size;
-
- scr_memsetw((u16 *)(c->vc_origin), eattr, delta);
-
- break;
- }
-
- if (copyall)
- sisusb_copy_memory(sisusb,
- (u8 *)c->vc_origin,
- sisusb_haddr(sisusb, c, 0, 0),
- c->vc_screenbuf_size);
- else if (dir == SM_UP)
- sisusb_copy_memory(sisusb,
- (u8 *)c->vc_origin + c->vc_screenbuf_size - delta,
- sisusb_haddr(sisusb, c, 0, 0) +
- c->vc_screenbuf_size - delta,
- delta);
- else
- sisusb_copy_memory(sisusb,
- (u8 *)c->vc_origin,
- sisusb_haddr(sisusb, c, 0, 0),
- delta);
-
- c->vc_scr_end = c->vc_origin + c->vc_screenbuf_size;
- c->vc_visible_origin = c->vc_origin;
-
- sisusbcon_set_start_address(sisusb, c);
-
- c->vc_pos = c->vc_pos - oldorigin + c->vc_origin;
-
- mutex_unlock(&sisusb->lock);
-
- return true;
-}
-
-/* Interface routine */
-static int
-sisusbcon_set_origin(struct vc_data *c)
-{
- struct sisusb_usb_data *sisusb;
-
- /* Returning != 0 means we were successful.
- * Returning 0 will vt make to use its own
- * screenbuffer as the origin.
- */
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return 0;
-
- /* sisusb->lock is down */
-
- if (sisusb_is_inactive(c, sisusb) || sisusb->con_blanked) {
- mutex_unlock(&sisusb->lock);
- return 0;
- }
-
- c->vc_origin = c->vc_visible_origin = sisusb->scrbuf;
-
- sisusbcon_set_start_address(sisusb, c);
-
- sisusb->con_rolled_over = 0;
-
- mutex_unlock(&sisusb->lock);
-
- return true;
-}
-
-/* Interface routine */
-static int
-sisusbcon_resize(struct vc_data *c, unsigned int newcols, unsigned int newrows,
- unsigned int user)
-{
- struct sisusb_usb_data *sisusb;
- int fh;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return -ENODEV;
-
- fh = sisusb->current_font_height;
-
- mutex_unlock(&sisusb->lock);
-
- /* We are quite unflexible as regards resizing. The vt code
- * handles sizes where the line length isn't equal the pitch
- * quite badly. As regards the rows, our panning tricks only
- * work well if the number of rows equals the visible number
- * of rows.
- */
-
- if (newcols != 80 || c->vc_scan_lines / fh != newrows)
- return -EINVAL;
-
- return 0;
-}
-
-int
-sisusbcon_do_font_op(struct sisusb_usb_data *sisusb, int set, int slot,
- u8 *arg, int cmapsz, int ch512, int dorecalc,
- struct vc_data *c, int fh, int uplock)
-{
- int font_select = 0x00, i, err = 0;
- u32 offset = 0;
- u8 dummy;
-
- /* sisusb->lock is down */
-
- /*
- * The default font is kept in slot 0.
- * A user font is loaded in slot 2 (256 ch)
- * or 2+3 (512 ch).
- */
-
- if ((slot != 0 && slot != 2) || !fh) {
- if (uplock)
- mutex_unlock(&sisusb->lock);
- return -EINVAL;
- }
-
- if (set)
- sisusb->font_slot = slot;
-
- /* Default font is always 256 */
- if (slot == 0)
- ch512 = 0;
- else
- offset = 4 * cmapsz;
-
- font_select = (slot == 0) ? 0x00 : (ch512 ? 0x0e : 0x0a);
-
- err |= sisusb_setidxreg(sisusb, SISSR, 0x00, 0x01); /* Reset */
- err |= sisusb_setidxreg(sisusb, SISSR, 0x02, 0x04); /* Write to plane 2 */
- err |= sisusb_setidxreg(sisusb, SISSR, 0x04, 0x07); /* Memory mode a0-bf */
- err |= sisusb_setidxreg(sisusb, SISSR, 0x00, 0x03); /* Reset */
-
- if (err)
- goto font_op_error;
-
- err |= sisusb_setidxreg(sisusb, SISGR, 0x04, 0x03); /* Select plane read 2 */
- err |= sisusb_setidxreg(sisusb, SISGR, 0x05, 0x00); /* Disable odd/even */
- err |= sisusb_setidxreg(sisusb, SISGR, 0x06, 0x00); /* Address range a0-bf */
-
- if (err)
- goto font_op_error;
-
- if (arg) {
- if (set)
- for (i = 0; i < cmapsz; i++) {
- err |= sisusb_writeb(sisusb,
- sisusb->vrambase + offset + i,
- arg[i]);
- if (err)
- break;
- }
- else
- for (i = 0; i < cmapsz; i++) {
- err |= sisusb_readb(sisusb,
- sisusb->vrambase + offset + i,
- &arg[i]);
- if (err)
- break;
- }
-
- /*
- * In 512-character mode, the character map is not contiguous if
- * we want to remain EGA compatible -- which we do
- */
-
- if (ch512) {
- if (set)
- for (i = 0; i < cmapsz; i++) {
- err |= sisusb_writeb(sisusb,
- sisusb->vrambase + offset +
- (2 * cmapsz) + i,
- arg[cmapsz + i]);
- if (err)
- break;
- }
- else
- for (i = 0; i < cmapsz; i++) {
- err |= sisusb_readb(sisusb,
- sisusb->vrambase + offset +
- (2 * cmapsz) + i,
- &arg[cmapsz + i]);
- if (err)
- break;
- }
- }
- }
-
- if (err)
- goto font_op_error;
-
- err |= sisusb_setidxreg(sisusb, SISSR, 0x00, 0x01); /* Reset */
- err |= sisusb_setidxreg(sisusb, SISSR, 0x02, 0x03); /* Write to planes 0+1 */
- err |= sisusb_setidxreg(sisusb, SISSR, 0x04, 0x03); /* Memory mode a0-bf */
- if (set)
- sisusb_setidxreg(sisusb, SISSR, 0x03, font_select);
- err |= sisusb_setidxreg(sisusb, SISSR, 0x00, 0x03); /* Reset end */
-
- if (err)
- goto font_op_error;
-
- err |= sisusb_setidxreg(sisusb, SISGR, 0x04, 0x00); /* Select plane read 0 */
- err |= sisusb_setidxreg(sisusb, SISGR, 0x05, 0x10); /* Enable odd/even */
- err |= sisusb_setidxreg(sisusb, SISGR, 0x06, 0x06); /* Address range b8-bf */
-
- if (err)
- goto font_op_error;
-
- if ((set) && (ch512 != sisusb->current_font_512)) {
-
- /* Font is shared among all our consoles.
- * And so is the hi_font_mask.
- */
- for (i = 0; i < MAX_NR_CONSOLES; i++) {
- struct vc_data *d = vc_cons[i].d;
- if (d && d->vc_sw == &sisusb_con)
- d->vc_hi_font_mask = ch512 ? 0x0800 : 0;
- }
-
- sisusb->current_font_512 = ch512;
-
- /* color plane enable register:
- 256-char: enable intensity bit
- 512-char: disable intensity bit */
- sisusb_getreg(sisusb, SISINPSTAT, &dummy);
- sisusb_setreg(sisusb, SISAR, 0x12);
- sisusb_setreg(sisusb, SISAR, ch512 ? 0x07 : 0x0f);
-
- sisusb_getreg(sisusb, SISINPSTAT, &dummy);
- sisusb_setreg(sisusb, SISAR, 0x20);
- sisusb_getreg(sisusb, SISINPSTAT, &dummy);
- }
-
- if (dorecalc) {
-
- /*
- * Adjust the screen to fit a font of a certain height
- */
-
- unsigned char ovr, vde, fsr;
- int rows = 0, maxscan = 0;
-
- if (c) {
-
- /* Number of video rows */
- rows = c->vc_scan_lines / fh;
- /* Scan lines to actually display-1 */
- maxscan = rows * fh - 1;
-
- /*printk(KERN_DEBUG "sisusb recalc rows %d maxscan %d fh %d sl %d\n",
- rows, maxscan, fh, c->vc_scan_lines);*/
-
- sisusb_getidxreg(sisusb, SISCR, 0x07, &ovr);
- vde = maxscan & 0xff;
- ovr = (ovr & 0xbd) |
- ((maxscan & 0x100) >> 7) |
- ((maxscan & 0x200) >> 3);
- sisusb_setidxreg(sisusb, SISCR, 0x07, ovr);
- sisusb_setidxreg(sisusb, SISCR, 0x12, vde);
-
- }
-
- sisusb_getidxreg(sisusb, SISCR, 0x09, &fsr);
- fsr = (fsr & 0xe0) | (fh - 1);
- sisusb_setidxreg(sisusb, SISCR, 0x09, fsr);
- sisusb->current_font_height = fh;
-
- sisusb->sisusb_cursor_size_from = -1;
- sisusb->sisusb_cursor_size_to = -1;
-
- }
-
- if (uplock)
- mutex_unlock(&sisusb->lock);
-
- if (dorecalc && c) {
- int rows = c->vc_scan_lines / fh;
-
- /* Now adjust our consoles' size */
-
- for (i = 0; i < MAX_NR_CONSOLES; i++) {
- struct vc_data *vc = vc_cons[i].d;
-
- if (vc && vc->vc_sw == &sisusb_con) {
- if (con_is_visible(vc)) {
- vc->vc_sw->con_cursor(vc, CM_DRAW);
- }
- vc->vc_font.height = fh;
- vc_resize(vc, 0, rows);
- }
- }
- }
-
- return 0;
-
-font_op_error:
- if (uplock)
- mutex_unlock(&sisusb->lock);
-
- return -EIO;
-}
-
-/* Interface routine */
-static int
-sisusbcon_font_set(struct vc_data *c, struct console_font *font,
- unsigned int flags)
-{
- struct sisusb_usb_data *sisusb;
- unsigned charcount = font->charcount;
-
- if (font->width != 8 || (charcount != 256 && charcount != 512))
- return -EINVAL;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return -ENODEV;
-
- /* sisusb->lock is down */
-
- /* Save the user-provided font into a buffer. This
- * is used for restoring text mode after quitting
- * from X and for the con_getfont routine.
- */
- if (sisusb->font_backup) {
- if (sisusb->font_backup_size < charcount) {
- vfree(sisusb->font_backup);
- sisusb->font_backup = NULL;
- }
- }
-
- if (!sisusb->font_backup)
- sisusb->font_backup = vmalloc(array_size(charcount, 32));
-
- if (sisusb->font_backup) {
- memcpy(sisusb->font_backup, font->data, array_size(charcount, 32));
- sisusb->font_backup_size = charcount;
- sisusb->font_backup_height = font->height;
- sisusb->font_backup_512 = (charcount == 512) ? 1 : 0;
- }
-
- /* do_font_op ups sisusb->lock */
-
- return sisusbcon_do_font_op(sisusb, 1, 2, font->data,
- 8192, (charcount == 512),
- (!(flags & KD_FONT_FLAG_DONT_RECALC)) ? 1 : 0,
- c, font->height, 1);
-}
-
-/* Interface routine */
-static int
-sisusbcon_font_get(struct vc_data *c, struct console_font *font)
-{
- struct sisusb_usb_data *sisusb;
-
- sisusb = sisusb_get_sisusb_lock_and_check(c->vc_num);
- if (!sisusb)
- return -ENODEV;
-
- /* sisusb->lock is down */
-
- font->width = 8;
- font->height = c->vc_font.height;
- font->charcount = 256;
-
- if (!font->data) {
- mutex_unlock(&sisusb->lock);
- return 0;
- }
-
- if (!sisusb->font_backup) {
- mutex_unlock(&sisusb->lock);
- return -ENODEV;
- }
-
- /* Copy 256 chars only, like vgacon */
- memcpy(font->data, sisusb->font_backup, 256 * 32);
-
- mutex_unlock(&sisusb->lock);
-
- return 0;
-}
-
-/*
- * The console `switch' structure for the sisusb console
- */
-
-static const struct consw sisusb_con = {
- .owner = THIS_MODULE,
- .con_startup = sisusbcon_startup,
- .con_init = sisusbcon_init,
- .con_deinit = sisusbcon_deinit,
- .con_clear = sisusbcon_clear,
- .con_putc = sisusbcon_putc,
- .con_putcs = sisusbcon_putcs,
- .con_cursor = sisusbcon_cursor,
- .con_scroll = sisusbcon_scroll,
- .con_switch = sisusbcon_switch,
- .con_blank = sisusbcon_blank,
- .con_font_set = sisusbcon_font_set,
- .con_font_get = sisusbcon_font_get,
- .con_set_palette = sisusbcon_set_palette,
- .con_scrolldelta = sisusbcon_scrolldelta,
- .con_build_attr = sisusbcon_build_attr,
- .con_invert_region = sisusbcon_invert_region,
- .con_set_origin = sisusbcon_set_origin,
- .con_save_screen = sisusbcon_save_screen,
- .con_resize = sisusbcon_resize,
-};
-
-/* Our very own dummy console driver */
-
-static const char *sisusbdummycon_startup(void)
-{
- return "SISUSBVGADUMMY";
-}
-
-static void sisusbdummycon_init(struct vc_data *vc, int init)
-{
- vc->vc_can_do_color = 1;
- if (init) {
- vc->vc_cols = 80;
- vc->vc_rows = 25;
- } else
- vc_resize(vc, 80, 25);
-}
-
-static void sisusbdummycon_deinit(struct vc_data *vc) { }
-static void sisusbdummycon_clear(struct vc_data *vc, int sy, int sx,
- int height, int width) { }
-static void sisusbdummycon_putc(struct vc_data *vc, int c, int ypos,
- int xpos) { }
-static void sisusbdummycon_putcs(struct vc_data *vc, const unsigned short *s,
- int count, int ypos, int xpos) { }
-static void sisusbdummycon_cursor(struct vc_data *vc, int mode) { }
-
-static bool sisusbdummycon_scroll(struct vc_data *vc, unsigned int top,
- unsigned int bottom, enum con_scroll dir,
- unsigned int lines)
-{
- return false;
-}
-
-static int sisusbdummycon_switch(struct vc_data *vc)
-{
- return 0;
-}
-
-static int sisusbdummycon_blank(struct vc_data *vc, int blank, int mode_switch)
-{
- return 0;
-}
-
-static const struct consw sisusb_dummy_con = {
- .owner = THIS_MODULE,
- .con_startup = sisusbdummycon_startup,
- .con_init = sisusbdummycon_init,
- .con_deinit = sisusbdummycon_deinit,
- .con_clear = sisusbdummycon_clear,
- .con_putc = sisusbdummycon_putc,
- .con_putcs = sisusbdummycon_putcs,
- .con_cursor = sisusbdummycon_cursor,
- .con_scroll = sisusbdummycon_scroll,
- .con_switch = sisusbdummycon_switch,
- .con_blank = sisusbdummycon_blank,
-};
-
-int
-sisusb_console_init(struct sisusb_usb_data *sisusb, int first, int last)
-{
- int i, ret;
-
- mutex_lock(&sisusb->lock);
-
- /* Erm.. that should not happen */
- if (sisusb->haveconsole || !sisusb->SiS_Pr) {
- mutex_unlock(&sisusb->lock);
- return 1;
- }
-
- sisusb->con_first = first;
- sisusb->con_last = last;
-
- if (first > last ||
- first > MAX_NR_CONSOLES ||
- last > MAX_NR_CONSOLES) {
- mutex_unlock(&sisusb->lock);
- return 1;
- }
-
- /* If gfxcore not initialized or no consoles given, quit graciously */
- if (!sisusb->gfxinit || first < 1 || last < 1) {
- mutex_unlock(&sisusb->lock);
- return 0;
- }
-
- sisusb->sisusb_cursor_loc = -1;
- sisusb->sisusb_cursor_size_from = -1;
- sisusb->sisusb_cursor_size_to = -1;
-
- /* Set up text mode (and upload default font) */
- if (sisusb_reset_text_mode(sisusb, 1)) {
- mutex_unlock(&sisusb->lock);
- dev_err(&sisusb->sisusb_dev->dev, "Failed to set up text mode\n");
- return 1;
- }
-
- /* Initialize some gfx registers */
- sisusb_initialize(sisusb);
-
- for (i = first - 1; i <= last - 1; i++) {
- /* Save sisusb for our interface routines */
- mysisusbs[i] = sisusb;
- }
-
- /* Initial console setup */
- sisusb->sisusb_num_columns = 80;
-
- /* Use a 32K buffer (matches b8000-bffff area) */
- sisusb->scrbuf_size = 32 * 1024;
-
- /* Allocate screen buffer */
- if (!(sisusb->scrbuf = (unsigned long)vmalloc(sisusb->scrbuf_size))) {
- mutex_unlock(&sisusb->lock);
- dev_err(&sisusb->sisusb_dev->dev, "Failed to allocate screen buffer\n");
- return 1;
- }
-
- mutex_unlock(&sisusb->lock);
-
- /* Now grab the desired console(s) */
- console_lock();
- ret = do_take_over_console(&sisusb_con, first - 1, last - 1, 0);
- console_unlock();
- if (!ret)
- sisusb->haveconsole = 1;
- else {
- for (i = first - 1; i <= last - 1; i++)
- mysisusbs[i] = NULL;
- }
-
- return ret;
-}
-
-void
-sisusb_console_exit(struct sisusb_usb_data *sisusb)
-{
- int i;
-
- /* This is called if the device is disconnected
- * and while disconnect and lock semaphores
- * are up. This should be save because we
- * can't lose our sisusb any other way but by
- * disconnection (and hence, the disconnect
- * sema is for protecting all other access
- * functions from disconnection, not the
- * other way round).
- */
-
- /* Now what do we do in case of disconnection:
- * One alternative would be to simply call
- * give_up_console(). Nah, not a good idea.
- * give_up_console() is obviously buggy as it
- * only discards the consw pointer from the
- * driver_map, but doesn't adapt vc->vc_sw
- * of the affected consoles. Hence, the next
- * call to any of the console functions will
- * eventually take a trip to oops county.
- * Also, give_up_console for some reason
- * doesn't decrement our module refcount.
- * Instead, we switch our consoles to a private
- * dummy console. This, of course, keeps our
- * refcount up as well, but it works perfectly.
- */
-
- if (sisusb->haveconsole) {
- for (i = 0; i < MAX_NR_CONSOLES; i++)
- if (sisusb->havethisconsole[i]) {
- console_lock();
- do_take_over_console(&sisusb_dummy_con, i, i, 0);
- console_unlock();
- /* At this point, con_deinit for all our
- * consoles is executed by do_take_over_console().
- */
- }
- sisusb->haveconsole = 0;
- }
-
- vfree((void *)sisusb->scrbuf);
- sisusb->scrbuf = 0;
-
- vfree(sisusb->font_backup);
- sisusb->font_backup = NULL;
-}
-
-void __init sisusb_init_concode(void)
-{
- int i;
-
- for (i = 0; i < MAX_NR_CONSOLES; i++)
- mysisusbs[i] = NULL;
-}
+++ /dev/null
-// SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause)
-/*
- * sisusb - usb kernel driver for SiS315(E) based USB2VGA dongles
- *
- * Display mode initializing code
- *
- * Copyright (C) 2001-2005 by Thomas Winischhofer, Vienna, Austria
- *
- * If distributed as part of the Linux kernel, this code is licensed under the
- * terms of the GPL v2.
- *
- * Otherwise, the following license terms apply:
- *
- * * Redistribution and use in source and binary forms, with or without
- * * modification, are permitted provided that the following conditions
- * * are met:
- * * 1) Redistributions of source code must retain the above copyright
- * * notice, this list of conditions and the following disclaimer.
- * * 2) Redistributions in binary form must reproduce the above copyright
- * * notice, this list of conditions and the following disclaimer in the
- * * documentation and/or other materials provided with the distribution.
- * * 3) The name of the author may not be used to endorse or promote products
- * * derived from this software without specific prior written permission.
- * *
- * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
- * * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
- * * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
- * * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
- * * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
- * * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
- * * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- * Author: Thomas Winischhofer <thomas@winischhofer.net>
- *
- */
-
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/errno.h>
-#include <linux/poll.h>
-#include <linux/spinlock.h>
-
-#include "sisusb.h"
-#include "sisusb_init.h"
-#include "sisusb_tables.h"
-
-/*********************************************/
-/* POINTER INITIALIZATION */
-/*********************************************/
-
-static void SiSUSB_InitPtr(struct SiS_Private *SiS_Pr)
-{
- SiS_Pr->SiS_ModeResInfo = SiSUSB_ModeResInfo;
- SiS_Pr->SiS_StandTable = SiSUSB_StandTable;
-
- SiS_Pr->SiS_SModeIDTable = SiSUSB_SModeIDTable;
- SiS_Pr->SiS_EModeIDTable = SiSUSB_EModeIDTable;
- SiS_Pr->SiS_RefIndex = SiSUSB_RefIndex;
- SiS_Pr->SiS_CRT1Table = SiSUSB_CRT1Table;
-
- SiS_Pr->SiS_VCLKData = SiSUSB_VCLKData;
-}
-
-/*********************************************/
-/* HELPER: SetReg, GetReg */
-/*********************************************/
-
-static void
-SiS_SetReg(struct SiS_Private *SiS_Pr, unsigned long port,
- unsigned short index, unsigned short data)
-{
- sisusb_setidxreg(SiS_Pr->sisusb, port, index, data);
-}
-
-static void
-SiS_SetRegByte(struct SiS_Private *SiS_Pr, unsigned long port,
- unsigned short data)
-{
- sisusb_setreg(SiS_Pr->sisusb, port, data);
-}
-
-static unsigned char
-SiS_GetReg(struct SiS_Private *SiS_Pr, unsigned long port, unsigned short index)
-{
- u8 data;
-
- sisusb_getidxreg(SiS_Pr->sisusb, port, index, &data);
-
- return data;
-}
-
-static unsigned char
-SiS_GetRegByte(struct SiS_Private *SiS_Pr, unsigned long port)
-{
- u8 data;
-
- sisusb_getreg(SiS_Pr->sisusb, port, &data);
-
- return data;
-}
-
-static void
-SiS_SetRegANDOR(struct SiS_Private *SiS_Pr, unsigned long port,
- unsigned short index, unsigned short DataAND,
- unsigned short DataOR)
-{
- sisusb_setidxregandor(SiS_Pr->sisusb, port, index, DataAND, DataOR);
-}
-
-static void
-SiS_SetRegAND(struct SiS_Private *SiS_Pr, unsigned long port,
- unsigned short index, unsigned short DataAND)
-{
- sisusb_setidxregand(SiS_Pr->sisusb, port, index, DataAND);
-}
-
-static void
-SiS_SetRegOR(struct SiS_Private *SiS_Pr, unsigned long port,
- unsigned short index, unsigned short DataOR)
-{
- sisusb_setidxregor(SiS_Pr->sisusb, port, index, DataOR);
-}
-
-/*********************************************/
-/* HELPER: DisplayOn, DisplayOff */
-/*********************************************/
-
-static void SiS_DisplayOn(struct SiS_Private *SiS_Pr)
-{
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3c4, 0x01, 0xDF);
-}
-
-/*********************************************/
-/* HELPER: Init Port Addresses */
-/*********************************************/
-
-static void SiSUSBRegInit(struct SiS_Private *SiS_Pr, unsigned long BaseAddr)
-{
- SiS_Pr->SiS_P3c4 = BaseAddr + 0x14;
- SiS_Pr->SiS_P3d4 = BaseAddr + 0x24;
- SiS_Pr->SiS_P3c0 = BaseAddr + 0x10;
- SiS_Pr->SiS_P3ce = BaseAddr + 0x1e;
- SiS_Pr->SiS_P3c2 = BaseAddr + 0x12;
- SiS_Pr->SiS_P3ca = BaseAddr + 0x1a;
- SiS_Pr->SiS_P3c6 = BaseAddr + 0x16;
- SiS_Pr->SiS_P3c7 = BaseAddr + 0x17;
- SiS_Pr->SiS_P3c8 = BaseAddr + 0x18;
- SiS_Pr->SiS_P3c9 = BaseAddr + 0x19;
- SiS_Pr->SiS_P3cb = BaseAddr + 0x1b;
- SiS_Pr->SiS_P3cc = BaseAddr + 0x1c;
- SiS_Pr->SiS_P3cd = BaseAddr + 0x1d;
- SiS_Pr->SiS_P3da = BaseAddr + 0x2a;
- SiS_Pr->SiS_Part1Port = BaseAddr + SIS_CRT2_PORT_04;
-}
-
-/*********************************************/
-/* HELPER: GetSysFlags */
-/*********************************************/
-
-static void SiS_GetSysFlags(struct SiS_Private *SiS_Pr)
-{
- SiS_Pr->SiS_MyCR63 = 0x63;
-}
-
-/*********************************************/
-/* HELPER: Init PCI & Engines */
-/*********************************************/
-
-static void SiSInitPCIetc(struct SiS_Private *SiS_Pr)
-{
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x20, 0xa1);
- /* - Enable 2D (0x40)
- * - Enable 3D (0x02)
- * - Enable 3D vertex command fetch (0x10)
- * - Enable 3D command parser (0x08)
- * - Enable 3D G/L transformation engine (0x80)
- */
- SiS_SetRegOR(SiS_Pr, SiS_Pr->SiS_P3c4, 0x1E, 0xDA);
-}
-
-/*********************************************/
-/* HELPER: SET SEGMENT REGISTERS */
-/*********************************************/
-
-static void SiS_SetSegRegLower(struct SiS_Private *SiS_Pr, unsigned short value)
-{
- unsigned short temp;
-
- value &= 0x00ff;
- temp = SiS_GetRegByte(SiS_Pr, SiS_Pr->SiS_P3cb) & 0xf0;
- temp |= (value >> 4);
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3cb, temp);
- temp = SiS_GetRegByte(SiS_Pr, SiS_Pr->SiS_P3cd) & 0xf0;
- temp |= (value & 0x0f);
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3cd, temp);
-}
-
-static void SiS_SetSegRegUpper(struct SiS_Private *SiS_Pr, unsigned short value)
-{
- unsigned short temp;
-
- value &= 0x00ff;
- temp = SiS_GetRegByte(SiS_Pr, SiS_Pr->SiS_P3cb) & 0x0f;
- temp |= (value & 0xf0);
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3cb, temp);
- temp = SiS_GetRegByte(SiS_Pr, SiS_Pr->SiS_P3cd) & 0x0f;
- temp |= (value << 4);
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3cd, temp);
-}
-
-static void SiS_SetSegmentReg(struct SiS_Private *SiS_Pr, unsigned short value)
-{
- SiS_SetSegRegLower(SiS_Pr, value);
- SiS_SetSegRegUpper(SiS_Pr, value);
-}
-
-static void SiS_ResetSegmentReg(struct SiS_Private *SiS_Pr)
-{
- SiS_SetSegmentReg(SiS_Pr, 0);
-}
-
-static void
-SiS_SetSegmentRegOver(struct SiS_Private *SiS_Pr, unsigned short value)
-{
- unsigned short temp = value >> 8;
-
- temp &= 0x07;
- temp |= (temp << 4);
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x1d, temp);
- SiS_SetSegmentReg(SiS_Pr, value);
-}
-
-static void SiS_ResetSegmentRegOver(struct SiS_Private *SiS_Pr)
-{
- SiS_SetSegmentRegOver(SiS_Pr, 0);
-}
-
-static void SiS_ResetSegmentRegisters(struct SiS_Private *SiS_Pr)
-{
- SiS_ResetSegmentReg(SiS_Pr);
- SiS_ResetSegmentRegOver(SiS_Pr);
-}
-
-/*********************************************/
-/* HELPER: SearchModeID */
-/*********************************************/
-
-static int
-SiS_SearchModeID(struct SiS_Private *SiS_Pr, unsigned short *ModeNo,
- unsigned short *ModeIdIndex)
-{
- if ((*ModeNo) <= 0x13) {
-
- if ((*ModeNo) != 0x03)
- return 0;
-
- (*ModeIdIndex) = 0;
-
- } else {
-
- for (*ModeIdIndex = 0;; (*ModeIdIndex)++) {
-
- if (SiS_Pr->SiS_EModeIDTable[*ModeIdIndex].Ext_ModeID ==
- (*ModeNo))
- break;
-
- if (SiS_Pr->SiS_EModeIDTable[*ModeIdIndex].Ext_ModeID ==
- 0xFF)
- return 0;
- }
-
- }
-
- return 1;
-}
-
-/*********************************************/
-/* HELPER: ENABLE CRT1 */
-/*********************************************/
-
-static void SiS_HandleCRT1(struct SiS_Private *SiS_Pr)
-{
- /* Enable CRT1 gating */
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3d4, SiS_Pr->SiS_MyCR63, 0xbf);
-}
-
-/*********************************************/
-/* HELPER: GetColorDepth */
-/*********************************************/
-
-static unsigned short
-SiS_GetColorDepth(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short ModeIdIndex)
-{
- static const unsigned short ColorDepth[6] = { 1, 2, 4, 4, 6, 8 };
- unsigned short modeflag;
- short index;
-
- if (ModeNo <= 0x13) {
- modeflag = SiS_Pr->SiS_SModeIDTable[ModeIdIndex].St_ModeFlag;
- } else {
- modeflag = SiS_Pr->SiS_EModeIDTable[ModeIdIndex].Ext_ModeFlag;
- }
-
- index = (modeflag & ModeTypeMask) - ModeEGA;
- if (index < 0)
- index = 0;
- return ColorDepth[index];
-}
-
-/*********************************************/
-/* HELPER: GetOffset */
-/*********************************************/
-
-static unsigned short
-SiS_GetOffset(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short ModeIdIndex, unsigned short rrti)
-{
- unsigned short xres, temp, colordepth, infoflag;
-
- infoflag = SiS_Pr->SiS_RefIndex[rrti].Ext_InfoFlag;
- xres = SiS_Pr->SiS_RefIndex[rrti].XRes;
-
- colordepth = SiS_GetColorDepth(SiS_Pr, ModeNo, ModeIdIndex);
-
- temp = xres / 16;
-
- if (infoflag & InterlaceMode)
- temp <<= 1;
-
- temp *= colordepth;
-
- if (xres % 16)
- temp += (colordepth >> 1);
-
- return temp;
-}
-
-/*********************************************/
-/* SEQ */
-/*********************************************/
-
-static void
-SiS_SetSeqRegs(struct SiS_Private *SiS_Pr, unsigned short StandTableIndex)
-{
- unsigned char SRdata;
- int i;
-
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x00, 0x03);
-
- SRdata = SiS_Pr->SiS_StandTable[StandTableIndex].SR[0] | 0x20;
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x01, SRdata);
-
- for (i = 2; i <= 4; i++) {
- SRdata = SiS_Pr->SiS_StandTable[StandTableIndex].SR[i - 1];
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, i, SRdata);
- }
-}
-
-/*********************************************/
-/* MISC */
-/*********************************************/
-
-static void
-SiS_SetMiscRegs(struct SiS_Private *SiS_Pr, unsigned short StandTableIndex)
-{
- unsigned char Miscdata = SiS_Pr->SiS_StandTable[StandTableIndex].MISC;
-
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3c2, Miscdata);
-}
-
-/*********************************************/
-/* CRTC */
-/*********************************************/
-
-static void
-SiS_SetCRTCRegs(struct SiS_Private *SiS_Pr, unsigned short StandTableIndex)
-{
- unsigned char CRTCdata;
- unsigned short i;
-
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3d4, 0x11, 0x7f);
-
- for (i = 0; i <= 0x18; i++) {
- CRTCdata = SiS_Pr->SiS_StandTable[StandTableIndex].CRTC[i];
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3d4, i, CRTCdata);
- }
-}
-
-/*********************************************/
-/* ATT */
-/*********************************************/
-
-static void
-SiS_SetATTRegs(struct SiS_Private *SiS_Pr, unsigned short StandTableIndex)
-{
- unsigned char ARdata;
- unsigned short i;
-
- for (i = 0; i <= 0x13; i++) {
- ARdata = SiS_Pr->SiS_StandTable[StandTableIndex].ATTR[i];
- SiS_GetRegByte(SiS_Pr, SiS_Pr->SiS_P3da);
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3c0, i);
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3c0, ARdata);
- }
- SiS_GetRegByte(SiS_Pr, SiS_Pr->SiS_P3da);
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3c0, 0x14);
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3c0, 0x00);
-
- SiS_GetRegByte(SiS_Pr, SiS_Pr->SiS_P3da);
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3c0, 0x20);
- SiS_GetRegByte(SiS_Pr, SiS_Pr->SiS_P3da);
-}
-
-/*********************************************/
-/* GRC */
-/*********************************************/
-
-static void
-SiS_SetGRCRegs(struct SiS_Private *SiS_Pr, unsigned short StandTableIndex)
-{
- unsigned char GRdata;
- unsigned short i;
-
- for (i = 0; i <= 0x08; i++) {
- GRdata = SiS_Pr->SiS_StandTable[StandTableIndex].GRC[i];
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3ce, i, GRdata);
- }
-
- if (SiS_Pr->SiS_ModeType > ModeVGA) {
- /* 256 color disable */
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3ce, 0x05, 0xBF);
- }
-}
-
-/*********************************************/
-/* CLEAR EXTENDED REGISTERS */
-/*********************************************/
-
-static void SiS_ClearExt1Regs(struct SiS_Private *SiS_Pr, unsigned short ModeNo)
-{
- int i;
-
- for (i = 0x0A; i <= 0x0E; i++) {
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, i, 0x00);
- }
-
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3c4, 0x37, 0xFE);
-}
-
-/*********************************************/
-/* Get rate index */
-/*********************************************/
-
-static unsigned short
-SiS_GetRatePtr(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short ModeIdIndex)
-{
- unsigned short rrti, i, index, temp;
-
- if (ModeNo <= 0x13)
- return 0xFFFF;
-
- index = SiS_GetReg(SiS_Pr, SiS_Pr->SiS_P3d4, 0x33) & 0x0F;
- if (index > 0)
- index--;
-
- rrti = SiS_Pr->SiS_EModeIDTable[ModeIdIndex].REFindex;
- ModeNo = SiS_Pr->SiS_RefIndex[rrti].ModeID;
-
- i = 0;
- do {
- if (SiS_Pr->SiS_RefIndex[rrti + i].ModeID != ModeNo)
- break;
-
- temp =
- SiS_Pr->SiS_RefIndex[rrti + i].Ext_InfoFlag & ModeTypeMask;
- if (temp < SiS_Pr->SiS_ModeType)
- break;
-
- i++;
- index--;
- } while (index != 0xFFFF);
-
- i--;
-
- return (rrti + i);
-}
-
-/*********************************************/
-/* SYNC */
-/*********************************************/
-
-static void SiS_SetCRT1Sync(struct SiS_Private *SiS_Pr, unsigned short rrti)
-{
- unsigned short sync = SiS_Pr->SiS_RefIndex[rrti].Ext_InfoFlag >> 8;
- sync &= 0xC0;
- sync |= 0x2f;
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3c2, sync);
-}
-
-/*********************************************/
-/* CRTC/2 */
-/*********************************************/
-
-static void
-SiS_SetCRT1CRTC(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short ModeIdIndex, unsigned short rrti)
-{
- unsigned char index;
- unsigned short temp, i, j, modeflag;
-
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3d4, 0x11, 0x7f);
-
- modeflag = SiS_Pr->SiS_EModeIDTable[ModeIdIndex].Ext_ModeFlag;
-
- index = SiS_Pr->SiS_RefIndex[rrti].Ext_CRT1CRTC;
-
- for (i = 0, j = 0; i <= 7; i++, j++) {
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3d4, j,
- SiS_Pr->SiS_CRT1Table[index].CR[i]);
- }
- for (j = 0x10; i <= 10; i++, j++) {
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3d4, j,
- SiS_Pr->SiS_CRT1Table[index].CR[i]);
- }
- for (j = 0x15; i <= 12; i++, j++) {
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3d4, j,
- SiS_Pr->SiS_CRT1Table[index].CR[i]);
- }
- for (j = 0x0A; i <= 15; i++, j++) {
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, j,
- SiS_Pr->SiS_CRT1Table[index].CR[i]);
- }
-
- temp = SiS_Pr->SiS_CRT1Table[index].CR[16] & 0xE0;
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x0E, temp);
-
- temp = ((SiS_Pr->SiS_CRT1Table[index].CR[16]) & 0x01) << 5;
- if (modeflag & DoubleScanMode)
- temp |= 0x80;
- SiS_SetRegANDOR(SiS_Pr, SiS_Pr->SiS_P3d4, 0x09, 0x5F, temp);
-
- if (SiS_Pr->SiS_ModeType > ModeVGA)
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3d4, 0x14, 0x4F);
-}
-
-/*********************************************/
-/* OFFSET & PITCH */
-/*********************************************/
-/* (partly overruled by SetPitch() in XF86) */
-/*********************************************/
-
-static void
-SiS_SetCRT1Offset(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short ModeIdIndex, unsigned short rrti)
-{
- unsigned short du = SiS_GetOffset(SiS_Pr, ModeNo, ModeIdIndex, rrti);
- unsigned short infoflag = SiS_Pr->SiS_RefIndex[rrti].Ext_InfoFlag;
- unsigned short temp;
-
- temp = (du >> 8) & 0x0f;
- SiS_SetRegANDOR(SiS_Pr, SiS_Pr->SiS_P3c4, 0x0E, 0xF0, temp);
-
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3d4, 0x13, (du & 0xFF));
-
- if (infoflag & InterlaceMode)
- du >>= 1;
-
- du <<= 5;
- temp = (du >> 8) & 0xff;
- if (du & 0xff)
- temp++;
- temp++;
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x10, temp);
-}
-
-/*********************************************/
-/* VCLK */
-/*********************************************/
-
-static void
-SiS_SetCRT1VCLK(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short rrti)
-{
- unsigned short index = SiS_Pr->SiS_RefIndex[rrti].Ext_CRTVCLK;
- unsigned short clka = SiS_Pr->SiS_VCLKData[index].SR2B;
- unsigned short clkb = SiS_Pr->SiS_VCLKData[index].SR2C;
-
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3c4, 0x31, 0xCF);
-
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x2B, clka);
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x2C, clkb);
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x2D, 0x01);
-}
-
-/*********************************************/
-/* FIFO */
-/*********************************************/
-
-static void
-SiS_SetCRT1FIFO_310(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short mi)
-{
- unsigned short modeflag = SiS_Pr->SiS_EModeIDTable[mi].Ext_ModeFlag;
-
- /* disable auto-threshold */
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3c4, 0x3D, 0xFE);
-
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x08, 0xAE);
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3c4, 0x09, 0xF0);
-
- if (ModeNo <= 0x13)
- return;
-
- if ((!(modeflag & DoubleScanMode)) || (!(modeflag & HalfDCLK))) {
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x08, 0x34);
- SiS_SetRegOR(SiS_Pr, SiS_Pr->SiS_P3c4, 0x3D, 0x01);
- }
-}
-
-/*********************************************/
-/* MODE REGISTERS */
-/*********************************************/
-
-static void
-SiS_SetVCLKState(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short rrti)
-{
- unsigned short data = 0, VCLK = 0, index = 0;
-
- if (ModeNo > 0x13) {
- index = SiS_Pr->SiS_RefIndex[rrti].Ext_CRTVCLK;
- VCLK = SiS_Pr->SiS_VCLKData[index].CLOCK;
- }
-
- if (VCLK >= 166)
- data |= 0x0c;
- SiS_SetRegANDOR(SiS_Pr, SiS_Pr->SiS_P3c4, 0x32, 0xf3, data);
-
- if (VCLK >= 166)
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3c4, 0x1f, 0xe7);
-
- /* DAC speed */
- data = 0x03;
- if (VCLK >= 260)
- data = 0x00;
- else if (VCLK >= 160)
- data = 0x01;
- else if (VCLK >= 135)
- data = 0x02;
-
- SiS_SetRegANDOR(SiS_Pr, SiS_Pr->SiS_P3c4, 0x07, 0xF8, data);
-}
-
-static void
-SiS_SetCRT1ModeRegs(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short ModeIdIndex, unsigned short rrti)
-{
- unsigned short data, infoflag = 0, modeflag;
-
- if (ModeNo <= 0x13)
- modeflag = SiS_Pr->SiS_SModeIDTable[ModeIdIndex].St_ModeFlag;
- else {
- modeflag = SiS_Pr->SiS_EModeIDTable[ModeIdIndex].Ext_ModeFlag;
- infoflag = SiS_Pr->SiS_RefIndex[rrti].Ext_InfoFlag;
- }
-
- /* Disable DPMS */
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3c4, 0x1F, 0x3F);
-
- data = 0;
- if (ModeNo > 0x13) {
- if (SiS_Pr->SiS_ModeType > ModeEGA) {
- data |= 0x02;
- data |= ((SiS_Pr->SiS_ModeType - ModeVGA) << 2);
- }
- if (infoflag & InterlaceMode)
- data |= 0x20;
- }
- SiS_SetRegANDOR(SiS_Pr, SiS_Pr->SiS_P3c4, 0x06, 0xC0, data);
-
- data = 0;
- if (infoflag & InterlaceMode) {
- /* data = (Hsync / 8) - ((Htotal / 8) / 2) + 3 */
- unsigned short hrs =
- (SiS_GetReg(SiS_Pr, SiS_Pr->SiS_P3d4, 0x04) |
- ((SiS_GetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x0b) & 0xc0) << 2))
- - 3;
- unsigned short hto =
- (SiS_GetReg(SiS_Pr, SiS_Pr->SiS_P3d4, 0x00) |
- ((SiS_GetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x0b) & 0x03) << 8))
- + 5;
- data = hrs - (hto >> 1) + 3;
- }
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3d4, 0x19, (data & 0xFF));
- SiS_SetRegANDOR(SiS_Pr, SiS_Pr->SiS_P3d4, 0x1a, 0xFC, (data >> 8));
-
- if (modeflag & HalfDCLK)
- SiS_SetRegOR(SiS_Pr, SiS_Pr->SiS_P3c4, 0x01, 0x08);
-
- data = 0;
- if (modeflag & LineCompareOff)
- data = 0x08;
- SiS_SetRegANDOR(SiS_Pr, SiS_Pr->SiS_P3c4, 0x0F, 0xB7, data);
-
- if ((SiS_Pr->SiS_ModeType == ModeEGA) && (ModeNo > 0x13))
- SiS_SetRegOR(SiS_Pr, SiS_Pr->SiS_P3c4, 0x0F, 0x40);
-
- SiS_SetRegAND(SiS_Pr, SiS_Pr->SiS_P3c4, 0x31, 0xfb);
-
- data = 0x60;
- if (SiS_Pr->SiS_ModeType != ModeText) {
- data ^= 0x60;
- if (SiS_Pr->SiS_ModeType != ModeEGA)
- data ^= 0xA0;
- }
- SiS_SetRegANDOR(SiS_Pr, SiS_Pr->SiS_P3c4, 0x21, 0x1F, data);
-
- SiS_SetVCLKState(SiS_Pr, ModeNo, rrti);
-
- if (SiS_GetReg(SiS_Pr, SiS_Pr->SiS_P3d4, 0x31) & 0x40)
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3d4, 0x52, 0x2c);
- else
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3d4, 0x52, 0x6c);
-}
-
-/*********************************************/
-/* LOAD DAC */
-/*********************************************/
-
-static void
-SiS_WriteDAC(struct SiS_Private *SiS_Pr, unsigned long DACData,
- unsigned short shiftflag, unsigned short dl, unsigned short ah,
- unsigned short al, unsigned short dh)
-{
- unsigned short d1, d2, d3;
-
- switch (dl) {
- case 0:
- d1 = dh;
- d2 = ah;
- d3 = al;
- break;
- case 1:
- d1 = ah;
- d2 = al;
- d3 = dh;
- break;
- default:
- d1 = al;
- d2 = dh;
- d3 = ah;
- }
- SiS_SetRegByte(SiS_Pr, DACData, (d1 << shiftflag));
- SiS_SetRegByte(SiS_Pr, DACData, (d2 << shiftflag));
- SiS_SetRegByte(SiS_Pr, DACData, (d3 << shiftflag));
-}
-
-static void
-SiS_LoadDAC(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short mi)
-{
- unsigned short data, data2, time, i, j, k, m, n, o;
- unsigned short si, di, bx, sf;
- unsigned long DACAddr, DACData;
- const unsigned char *table = NULL;
-
- if (ModeNo < 0x13)
- data = SiS_Pr->SiS_SModeIDTable[mi].St_ModeFlag;
- else
- data = SiS_Pr->SiS_EModeIDTable[mi].Ext_ModeFlag;
-
- data &= DACInfoFlag;
-
- j = time = 64;
- if (data == 0x00)
- table = SiS_MDA_DAC;
- else if (data == 0x08)
- table = SiS_CGA_DAC;
- else if (data == 0x10)
- table = SiS_EGA_DAC;
- else {
- j = 16;
- time = 256;
- table = SiS_VGA_DAC;
- }
-
- DACAddr = SiS_Pr->SiS_P3c8;
- DACData = SiS_Pr->SiS_P3c9;
- sf = 0;
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3c6, 0xFF);
-
- SiS_SetRegByte(SiS_Pr, DACAddr, 0x00);
-
- for (i = 0; i < j; i++) {
- data = table[i];
- for (k = 0; k < 3; k++) {
- data2 = 0;
- if (data & 0x01)
- data2 += 0x2A;
- if (data & 0x02)
- data2 += 0x15;
- SiS_SetRegByte(SiS_Pr, DACData, (data2 << sf));
- data >>= 2;
- }
- }
-
- if (time == 256) {
- for (i = 16; i < 32; i++) {
- data = table[i] << sf;
- for (k = 0; k < 3; k++)
- SiS_SetRegByte(SiS_Pr, DACData, data);
- }
- si = 32;
- for (m = 0; m < 9; m++) {
- di = si;
- bx = si + 4;
- for (n = 0; n < 3; n++) {
- for (o = 0; o < 5; o++) {
- SiS_WriteDAC(SiS_Pr, DACData, sf, n,
- table[di], table[bx],
- table[si]);
- si++;
- }
- si -= 2;
- for (o = 0; o < 3; o++) {
- SiS_WriteDAC(SiS_Pr, DACData, sf, n,
- table[di], table[si],
- table[bx]);
- si--;
- }
- }
- si += 5;
- }
- }
-}
-
-/*********************************************/
-/* SET CRT1 REGISTER GROUP */
-/*********************************************/
-
-static void
-SiS_SetCRT1Group(struct SiS_Private *SiS_Pr, unsigned short ModeNo,
- unsigned short ModeIdIndex)
-{
- unsigned short StandTableIndex, rrti;
-
- SiS_Pr->SiS_CRT1Mode = ModeNo;
-
- if (ModeNo <= 0x13)
- StandTableIndex = 0;
- else
- StandTableIndex = 1;
-
- SiS_ResetSegmentRegisters(SiS_Pr);
- SiS_SetSeqRegs(SiS_Pr, StandTableIndex);
- SiS_SetMiscRegs(SiS_Pr, StandTableIndex);
- SiS_SetCRTCRegs(SiS_Pr, StandTableIndex);
- SiS_SetATTRegs(SiS_Pr, StandTableIndex);
- SiS_SetGRCRegs(SiS_Pr, StandTableIndex);
- SiS_ClearExt1Regs(SiS_Pr, ModeNo);
-
- rrti = SiS_GetRatePtr(SiS_Pr, ModeNo, ModeIdIndex);
-
- if (rrti != 0xFFFF) {
- SiS_SetCRT1Sync(SiS_Pr, rrti);
- SiS_SetCRT1CRTC(SiS_Pr, ModeNo, ModeIdIndex, rrti);
- SiS_SetCRT1Offset(SiS_Pr, ModeNo, ModeIdIndex, rrti);
- SiS_SetCRT1VCLK(SiS_Pr, ModeNo, rrti);
- }
-
- SiS_SetCRT1FIFO_310(SiS_Pr, ModeNo, ModeIdIndex);
-
- SiS_SetCRT1ModeRegs(SiS_Pr, ModeNo, ModeIdIndex, rrti);
-
- SiS_LoadDAC(SiS_Pr, ModeNo, ModeIdIndex);
-
- SiS_DisplayOn(SiS_Pr);
-}
-
-/*********************************************/
-/* SiSSetMode() */
-/*********************************************/
-
-int SiSUSBSetMode(struct SiS_Private *SiS_Pr, unsigned short ModeNo)
-{
- unsigned short ModeIdIndex;
- unsigned long BaseAddr = SiS_Pr->IOAddress;
-
- SiSUSB_InitPtr(SiS_Pr);
- SiSUSBRegInit(SiS_Pr, BaseAddr);
- SiS_GetSysFlags(SiS_Pr);
-
- if (!(SiS_SearchModeID(SiS_Pr, &ModeNo, &ModeIdIndex)))
- return 0;
-
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3c4, 0x05, 0x86);
-
- SiSInitPCIetc(SiS_Pr);
-
- ModeNo &= 0x7f;
-
- SiS_Pr->SiS_ModeType =
- SiS_Pr->SiS_EModeIDTable[ModeIdIndex].Ext_ModeFlag & ModeTypeMask;
-
- SiS_Pr->SiS_SetFlag = LowModeTests;
-
- /* Set mode on CRT1 */
- SiS_SetCRT1Group(SiS_Pr, ModeNo, ModeIdIndex);
-
- SiS_HandleCRT1(SiS_Pr);
-
- SiS_DisplayOn(SiS_Pr);
- SiS_SetRegByte(SiS_Pr, SiS_Pr->SiS_P3c6, 0xFF);
-
- /* Store mode number */
- SiS_SetReg(SiS_Pr, SiS_Pr->SiS_P3d4, 0x34, ModeNo);
-
- return 1;
-}
-
-int SiSUSBSetVESAMode(struct SiS_Private *SiS_Pr, unsigned short VModeNo)
-{
- unsigned short ModeNo = 0;
- int i;
-
- SiSUSB_InitPtr(SiS_Pr);
-
- if (VModeNo == 0x03) {
-
- ModeNo = 0x03;
-
- } else {
-
- i = 0;
- do {
-
- if (SiS_Pr->SiS_EModeIDTable[i].Ext_VESAID == VModeNo) {
- ModeNo = SiS_Pr->SiS_EModeIDTable[i].Ext_ModeID;
- break;
- }
-
- } while (SiS_Pr->SiS_EModeIDTable[i++].Ext_ModeID != 0xff);
-
- }
-
- if (!ModeNo)
- return 0;
-
- return SiSUSBSetMode(SiS_Pr, ModeNo);
-}
+++ /dev/null
-/* SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause) */
-/* $XFree86$ */
-/* $XdotOrg$ */
-/*
- * Data and prototypes for init.c
- *
- * Copyright (C) 2001-2005 by Thomas Winischhofer, Vienna, Austria
- *
- * If distributed as part of the Linux kernel, the following license terms
- * apply:
- *
- * * This program is free software; you can redistribute it and/or modify
- * * it under the terms of the GNU General Public License as published by
- * * the Free Software Foundation; either version 2 of the named License,
- * * or any later version.
- * *
- * * This program is distributed in the hope that it will be useful,
- * * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * * GNU General Public License for more details.
- * *
- * * You should have received a copy of the GNU General Public License
- * * along with this program; if not, write to the Free Software
- * * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA
- *
- * Otherwise, the following license terms apply:
- *
- * * Redistribution and use in source and binary forms, with or without
- * * modification, are permitted provided that the following conditions
- * * are met:
- * * 1) Redistributions of source code must retain the above copyright
- * * notice, this list of conditions and the following disclaimer.
- * * 2) Redistributions in binary form must reproduce the above copyright
- * * notice, this list of conditions and the following disclaimer in the
- * * documentation and/or other materials provided with the distribution.
- * * 3) The name of the author may not be used to endorse or promote products
- * * derived from this software without specific prior written permission.
- * *
- * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
- * * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
- * * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
- * * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
- * * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
- * * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
- * * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- * Author: Thomas Winischhofer <thomas@winischhofer.net>
- *
- */
-
-#ifndef _SISUSB_INIT_H_
-#define _SISUSB_INIT_H_
-
-/* SiS_ModeType */
-#define ModeText 0x00
-#define ModeCGA 0x01
-#define ModeEGA 0x02
-#define ModeVGA 0x03
-#define Mode15Bpp 0x04
-#define Mode16Bpp 0x05
-#define Mode24Bpp 0x06
-#define Mode32Bpp 0x07
-
-#define ModeTypeMask 0x07
-#define IsTextMode 0x07
-
-#define DACInfoFlag 0x0018
-#define MemoryInfoFlag 0x01E0
-#define MemorySizeShift 5
-
-/* modeflag */
-#define Charx8Dot 0x0200
-#define LineCompareOff 0x0400
-#define CRT2Mode 0x0800
-#define HalfDCLK 0x1000
-#define NoSupportSimuTV 0x2000
-#define NoSupportLCDScale 0x4000 /* SiS bridge: No scaling possible (no matter what panel) */
-#define DoubleScanMode 0x8000
-
-/* Infoflag */
-#define SupportTV 0x0008
-#define SupportTV1024 0x0800
-#define SupportCHTV 0x0800
-#define Support64048060Hz 0x0800 /* Special for 640x480 LCD */
-#define SupportHiVision 0x0010
-#define SupportYPbPr750p 0x1000
-#define SupportLCD 0x0020
-#define SupportRAMDAC2 0x0040 /* All (<= 100Mhz) */
-#define SupportRAMDAC2_135 0x0100 /* All except DH (<= 135Mhz) */
-#define SupportRAMDAC2_162 0x0200 /* B, C (<= 162Mhz) */
-#define SupportRAMDAC2_202 0x0400 /* C (<= 202Mhz) */
-#define InterlaceMode 0x0080
-#define SyncPP 0x0000
-#define SyncPN 0x4000
-#define SyncNP 0x8000
-#define SyncNN 0xc000
-
-/* SetFlag */
-#define ProgrammingCRT2 0x0001
-#define LowModeTests 0x0002
-#define LCDVESATiming 0x0008
-#define EnableLVDSDDA 0x0010
-#define SetDispDevSwitchFlag 0x0020
-#define CheckWinDos 0x0040
-#define SetDOSMode 0x0080
-
-/* Index in ModeResInfo table */
-#define SIS_RI_320x200 0
-#define SIS_RI_320x240 1
-#define SIS_RI_320x400 2
-#define SIS_RI_400x300 3
-#define SIS_RI_512x384 4
-#define SIS_RI_640x400 5
-#define SIS_RI_640x480 6
-#define SIS_RI_800x600 7
-#define SIS_RI_1024x768 8
-#define SIS_RI_1280x1024 9
-#define SIS_RI_1600x1200 10
-#define SIS_RI_1920x1440 11
-#define SIS_RI_2048x1536 12
-#define SIS_RI_720x480 13
-#define SIS_RI_720x576 14
-#define SIS_RI_1280x960 15
-#define SIS_RI_800x480 16
-#define SIS_RI_1024x576 17
-#define SIS_RI_1280x720 18
-#define SIS_RI_856x480 19
-#define SIS_RI_1280x768 20
-#define SIS_RI_1400x1050 21
-#define SIS_RI_1152x864 22 /* Up to here SiS conforming */
-#define SIS_RI_848x480 23
-#define SIS_RI_1360x768 24
-#define SIS_RI_1024x600 25
-#define SIS_RI_1152x768 26
-#define SIS_RI_768x576 27
-#define SIS_RI_1360x1024 28
-#define SIS_RI_1680x1050 29
-#define SIS_RI_1280x800 30
-#define SIS_RI_1920x1080 31
-#define SIS_RI_960x540 32
-#define SIS_RI_960x600 33
-
-#define SIS_VIDEO_CAPTURE 0x00 - 0x30
-#define SIS_VIDEO_PLAYBACK 0x02 - 0x30
-#define SIS_CRT2_PORT_04 0x04 - 0x30
-
-int SiSUSBSetMode(struct SiS_Private *SiS_Pr, unsigned short ModeNo);
-int SiSUSBSetVESAMode(struct SiS_Private *SiS_Pr, unsigned short VModeNo);
-
-extern int sisusb_setreg(struct sisusb_usb_data *sisusb, u32 port, u8 data);
-extern int sisusb_getreg(struct sisusb_usb_data *sisusb, u32 port, u8 * data);
-extern int sisusb_setidxreg(struct sisusb_usb_data *sisusb, u32 port,
- u8 index, u8 data);
-extern int sisusb_getidxreg(struct sisusb_usb_data *sisusb, u32 port,
- u8 index, u8 * data);
-extern int sisusb_setidxregandor(struct sisusb_usb_data *sisusb, u32 port,
- u8 idx, u8 myand, u8 myor);
-extern int sisusb_setidxregor(struct sisusb_usb_data *sisusb, u32 port,
- u8 index, u8 myor);
-extern int sisusb_setidxregand(struct sisusb_usb_data *sisusb, u32 port,
- u8 idx, u8 myand);
-
-void sisusb_delete(struct kref *kref);
-int sisusb_writeb(struct sisusb_usb_data *sisusb, u32 adr, u8 data);
-int sisusb_readb(struct sisusb_usb_data *sisusb, u32 adr, u8 * data);
-int sisusb_copy_memory(struct sisusb_usb_data *sisusb, u8 *src,
- u32 dest, int length);
-int sisusb_reset_text_mode(struct sisusb_usb_data *sisusb, int init);
-int sisusbcon_do_font_op(struct sisusb_usb_data *sisusb, int set, int slot,
- u8 * arg, int cmapsz, int ch512, int dorecalc,
- struct vc_data *c, int fh, int uplock);
-void sisusb_set_cursor(struct sisusb_usb_data *sisusb, unsigned int location);
-int sisusb_console_init(struct sisusb_usb_data *sisusb, int first, int last);
-void sisusb_console_exit(struct sisusb_usb_data *sisusb);
-void sisusb_init_concode(void);
-
-#endif
#include <linux/vmalloc.h>
#include "sisusb.h"
-#include "sisusb_init.h"
-
-#ifdef CONFIG_USB_SISUSBVGA_CON
-#include <linux/font.h>
-#endif
#define SISUSB_DONTSYNC
/* Forward declarations / clean-up routines */
-#ifdef CONFIG_USB_SISUSBVGA_CON
-static int sisusb_first_vc;
-static int sisusb_last_vc;
-module_param_named(first, sisusb_first_vc, int, 0);
-module_param_named(last, sisusb_last_vc, int, 0);
-MODULE_PARM_DESC(first, "Number of first console to take over (1 - MAX_NR_CONSOLES)");
-MODULE_PARM_DESC(last, "Number of last console to take over (1 - MAX_NR_CONSOLES)");
-#endif
-
static struct usb_driver sisusb_driver;
static void sisusb_free_buffers(struct sisusb_usb_data *sisusb)
/* High level: Gfx (indexed) register access */
-#ifdef CONFIG_USB_SISUSBVGA_CON
-int sisusb_setreg(struct sisusb_usb_data *sisusb, u32 port, u8 data)
-{
- return sisusb_write_memio_byte(sisusb, SISUSB_TYPE_IO, port, data);
-}
-
-int sisusb_getreg(struct sisusb_usb_data *sisusb, u32 port, u8 *data)
-{
- return sisusb_read_memio_byte(sisusb, SISUSB_TYPE_IO, port, data);
-}
-#endif
-
-int sisusb_setidxreg(struct sisusb_usb_data *sisusb, u32 port,
+static int sisusb_setidxreg(struct sisusb_usb_data *sisusb, u32 port,
u8 index, u8 data)
{
int ret;
return ret;
}
-int sisusb_getidxreg(struct sisusb_usb_data *sisusb, u32 port,
+static int sisusb_getidxreg(struct sisusb_usb_data *sisusb, u32 port,
u8 index, u8 *data)
{
int ret;
return ret;
}
-int sisusb_setidxregandor(struct sisusb_usb_data *sisusb, u32 port, u8 idx,
+static int sisusb_setidxregandor(struct sisusb_usb_data *sisusb, u32 port, u8 idx,
u8 myand, u8 myor)
{
int ret;
return ret;
}
-int sisusb_setidxregor(struct sisusb_usb_data *sisusb, u32 port,
+static int sisusb_setidxregor(struct sisusb_usb_data *sisusb, u32 port,
u8 index, u8 myor)
{
return sisusb_setidxregandor(sisusb, port, index, 0xff, myor);
}
-int sisusb_setidxregand(struct sisusb_usb_data *sisusb, u32 port,
+static int sisusb_setidxregand(struct sisusb_usb_data *sisusb, u32 port,
u8 idx, u8 myand)
{
return sisusb_setidxregandor(sisusb, port, idx, myand, 0x00);
/* Write/read video ram */
-#ifdef CONFIG_USB_SISUSBVGA_CON
-int sisusb_writeb(struct sisusb_usb_data *sisusb, u32 adr, u8 data)
-{
- return sisusb_write_memio_byte(sisusb, SISUSB_TYPE_MEM, adr, data);
-}
-
-int sisusb_readb(struct sisusb_usb_data *sisusb, u32 adr, u8 *data)
-{
- return sisusb_read_memio_byte(sisusb, SISUSB_TYPE_MEM, adr, data);
-}
-
-int sisusb_copy_memory(struct sisusb_usb_data *sisusb, u8 *src,
- u32 dest, int length)
-{
- size_t dummy;
-
- return sisusb_write_mem_bulk(sisusb, dest, src, length,
- NULL, 0, &dummy);
-}
-
-#ifdef SISUSBENDIANTEST
-static int sisusb_read_memory(struct sisusb_usb_data *sisusb, char *dest,
- u32 src, int length)
-{
- size_t dummy;
-
- return sisusb_read_mem_bulk(sisusb, src, dest, length,
- NULL, &dummy);
-}
-#endif
-#endif
-
#ifdef SISUSBENDIANTEST
static void sisusb_testreadwrite(struct sisusb_usb_data *sisusb)
{
return ret;
}
-
-#ifdef CONFIG_USB_SISUSBVGA_CON
-
-/* Set up default text mode:
- * - Set text mode (0x03)
- * - Upload default font
- * - Upload user font (if available)
- */
-
-int sisusb_reset_text_mode(struct sisusb_usb_data *sisusb, int init)
-{
- int ret = 0, slot = sisusb->font_slot, i;
- const struct font_desc *myfont;
- u8 *tempbuf;
- u16 *tempbufb;
- static const char bootstring[] =
- "SiSUSB VGA text console, (C) 2005 Thomas Winischhofer.";
- static const char bootlogo[] = "(o_ //\\ V_/_";
-
- /* sisusb->lock is down */
-
- if (!sisusb->SiS_Pr)
- return 1;
-
- sisusb->SiS_Pr->IOAddress = SISUSB_PCI_IOPORTBASE + 0x30;
- sisusb->SiS_Pr->sisusb = (void *)sisusb;
-
- /* Set mode 0x03 */
- SiSUSBSetMode(sisusb->SiS_Pr, 0x03);
-
- myfont = find_font("VGA8x16");
- if (!myfont)
- return 1;
-
- tempbuf = vmalloc(8192);
- if (!tempbuf)
- return 1;
-
- for (i = 0; i < 256; i++)
- memcpy(tempbuf + (i * 32), myfont->data + (i * 16), 16);
-
- /* Upload default font */
- ret = sisusbcon_do_font_op(sisusb, 1, 0, tempbuf, 8192,
- 0, 1, NULL, 16, 0);
-
- vfree(tempbuf);
-
- /* Upload user font (and reset current slot) */
- if (sisusb->font_backup) {
- ret |= sisusbcon_do_font_op(sisusb, 1, 2, sisusb->font_backup,
- 8192, sisusb->font_backup_512, 1, NULL,
- sisusb->font_backup_height, 0);
- if (slot != 2)
- sisusbcon_do_font_op(sisusb, 1, 0, NULL, 0, 0, 1,
- NULL, 16, 0);
- }
-
- if (init && !sisusb->scrbuf) {
-
- tempbuf = vmalloc(8192);
- if (tempbuf) {
-
- i = 4096;
- tempbufb = (u16 *)tempbuf;
- while (i--)
- *(tempbufb++) = 0x0720;
-
- i = 0;
- tempbufb = (u16 *)tempbuf;
- while (bootlogo[i]) {
- *(tempbufb++) = 0x0700 | bootlogo[i++];
- if (!(i % 4))
- tempbufb += 76;
- }
-
- i = 0;
- tempbufb = (u16 *)tempbuf + 6;
- while (bootstring[i])
- *(tempbufb++) = 0x0700 | bootstring[i++];
-
- ret |= sisusb_copy_memory(sisusb, tempbuf,
- sisusb->vrambase, 8192);
-
- vfree(tempbuf);
-
- }
-
- } else if (sisusb->scrbuf) {
- ret |= sisusb_copy_memory(sisusb, (u8 *)sisusb->scrbuf,
- sisusb->vrambase, sisusb->scrbuf_size);
- }
-
- if (sisusb->sisusb_cursor_size_from >= 0 &&
- sisusb->sisusb_cursor_size_to >= 0) {
- sisusb_setidxreg(sisusb, SISCR, 0x0a,
- sisusb->sisusb_cursor_size_from);
- sisusb_setidxregandor(sisusb, SISCR, 0x0b, 0xe0,
- sisusb->sisusb_cursor_size_to);
- } else {
- sisusb_setidxreg(sisusb, SISCR, 0x0a, 0x2d);
- sisusb_setidxreg(sisusb, SISCR, 0x0b, 0x0e);
- sisusb->sisusb_cursor_size_to = -1;
- }
-
- slot = sisusb->sisusb_cursor_loc;
- if (slot < 0)
- slot = 0;
-
- sisusb->sisusb_cursor_loc = -1;
- sisusb->bad_cursor_pos = 1;
-
- sisusb_set_cursor(sisusb, slot);
-
- sisusb_setidxreg(sisusb, SISCR, 0x0c, (sisusb->cur_start_addr >> 8));
- sisusb_setidxreg(sisusb, SISCR, 0x0d, (sisusb->cur_start_addr & 0xff));
-
- sisusb->textmodedestroyed = 0;
-
- /* sisusb->lock is down */
-
- return ret;
-}
-
-#endif
-
/* fops */
static int sisusb_open(struct inode *inode, struct file *file)
return 0;
}
-void sisusb_delete(struct kref *kref)
+static void sisusb_delete(struct kref *kref)
{
struct sisusb_usb_data *sisusb = to_sisusb_dev(kref);
sisusb->sisusb_dev = NULL;
sisusb_free_buffers(sisusb);
sisusb_free_urbs(sisusb);
-#ifdef CONFIG_USB_SISUSBVGA_CON
- kfree(sisusb->SiS_Pr);
-#endif
kfree(sisusb);
}
case SUCMD_HANDLETEXTMODE:
retval = 0;
-#ifdef CONFIG_USB_SISUSBVGA_CON
- /* Gfx core must be initialized, SiS_Pr must exist */
- if (!sisusb->gfxinit || !sisusb->SiS_Pr)
- return -ENODEV;
-
- switch (y->data0) {
- case 0:
- retval = sisusb_reset_text_mode(sisusb, 0);
- break;
- case 1:
- sisusb->textmodedestroyed = 1;
- break;
- }
-#endif
- break;
-
-#ifdef CONFIG_USB_SISUSBVGA_CON
- case SUCMD_SETMODE:
- /* Gfx core must be initialized, SiS_Pr must exist */
- if (!sisusb->gfxinit || !sisusb->SiS_Pr)
- return -ENODEV;
-
- retval = 0;
-
- sisusb->SiS_Pr->IOAddress = SISUSB_PCI_IOPORTBASE + 0x30;
- sisusb->SiS_Pr->sisusb = (void *)sisusb;
-
- if (SiSUSBSetMode(sisusb->SiS_Pr, y->data3))
- retval = -EINVAL;
-
break;
- case SUCMD_SETVESAMODE:
- /* Gfx core must be initialized, SiS_Pr must exist */
- if (!sisusb->gfxinit || !sisusb->SiS_Pr)
- return -ENODEV;
-
- retval = 0;
-
- sisusb->SiS_Pr->IOAddress = SISUSB_PCI_IOPORTBASE + 0x30;
- sisusb->SiS_Pr->sisusb = (void *)sisusb;
-
- if (SiSUSBSetVESAMode(sisusb->SiS_Pr, y->data3))
- retval = -EINVAL;
-
- break;
-#endif
-
default:
retval = -EINVAL;
}
x.sisusb_vramsize = sisusb->vramsize;
x.sisusb_minor = sisusb->minor;
x.sisusb_fbdevactive = 0;
-#ifdef CONFIG_USB_SISUSBVGA_CON
- x.sisusb_conactive = sisusb->haveconsole ? 1 : 0;
-#else
x.sisusb_conactive = 0;
-#endif
memset(x.sisusb_reserved, 0, sizeof(x.sisusb_reserved));
if (copy_to_user((void __user *)arg, &x, sizeof(x)))
dev_info(&sisusb->sisusb_dev->dev, "Allocated %d output buffers\n",
sisusb->numobufs);
-#ifdef CONFIG_USB_SISUSBVGA_CON
- /* Allocate our SiS_Pr */
- sisusb->SiS_Pr = kmalloc(sizeof(struct SiS_Private), GFP_KERNEL);
- if (!sisusb->SiS_Pr) {
- retval = -ENOMEM;
- goto error_4;
- }
-#endif
-
/* Do remaining init stuff */
init_waitqueue_head(&sisusb->wait_q);
if (dev->speed == USB_SPEED_HIGH || dev->speed >= USB_SPEED_SUPER) {
int initscreen = 1;
-#ifdef CONFIG_USB_SISUSBVGA_CON
- if (sisusb_first_vc > 0 && sisusb_last_vc > 0 &&
- sisusb_first_vc <= sisusb_last_vc &&
- sisusb_last_vc <= MAX_NR_CONSOLES)
- initscreen = 0;
-#endif
if (sisusb_init_gfxdevice(sisusb, initscreen))
dev_err(&sisusb->sisusb_dev->dev,
"Failed to early initialize device\n");
dev_dbg(&sisusb->sisusb_dev->dev, "*** RWTEST END ***\n");
#endif
-#ifdef CONFIG_USB_SISUSBVGA_CON
- sisusb_console_init(sisusb, sisusb_first_vc, sisusb_last_vc);
-#endif
-
return 0;
error_4:
if (!sisusb)
return;
-#ifdef CONFIG_USB_SISUSBVGA_CON
- sisusb_console_exit(sisusb);
-#endif
-
usb_deregister_dev(intf, &usb_sisusb_class);
mutex_lock(&sisusb->lock);
.id_table = sisusb_table,
};
-static int __init usb_sisusb_init(void)
-{
-
-#ifdef CONFIG_USB_SISUSBVGA_CON
- sisusb_init_concode();
-#endif
-
- return usb_register(&sisusb_driver);
-}
-
-static void __exit usb_sisusb_exit(void)
-{
- usb_deregister(&sisusb_driver);
-}
-
-module_init(usb_sisusb_init);
-module_exit(usb_sisusb_exit);
+module_usb_driver(sisusb_driver);
MODULE_AUTHOR("Thomas Winischhofer <thomas@winischhofer.net>");
MODULE_DESCRIPTION("sisusbvga - Driver for Net2280/SiS315-based USB2VGA dongles");
return 0;
}
-static int usb251xb_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int usb251xb_i2c_probe(struct i2c_client *i2c)
{
struct usb251xb *hub;
.of_match_table = of_match_ptr(usb251xb_of_match),
.pm = &usb251xb_pm_ops,
},
- .probe = usb251xb_i2c_probe,
+ .probe_new = usb251xb_i2c_probe,
.id_table = usb251xb_id,
};
return err;
}
-static int usb3503_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int usb3503_i2c_probe(struct i2c_client *i2c)
{
struct usb3503 *hub;
int err;
.pm = pm_ptr(&usb3503_i2c_pm_ops),
.of_match_table = of_match_ptr(usb3503_of_match),
},
- .probe = usb3503_i2c_probe,
+ .probe_new = usb3503_i2c_probe,
.remove = usb3503_i2c_remove,
.id_table = usb3503_id,
};
return usb4604_switch_mode(hub, hub->mode);
}
-static int usb4604_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
+static int usb4604_i2c_probe(struct i2c_client *i2c)
{
struct usb4604 *hub;
.pm = pm_ptr(&usb4604_i2c_pm_ops),
.of_match_table = of_match_ptr(usb4604_of_match),
},
- .probe = usb4604_i2c_probe,
+ .probe_new = usb4604_i2c_probe,
.id_table = usb4604_id,
};
module_i2c_driver(usb4604_i2c_driver);
select GENERIC_PHY
select SUNXI_SRAM
-config USB_MUSB_DAVINCI
- tristate "DaVinci"
- depends on ARCH_DAVINCI_DMx
- depends on NOP_USB_XCEIV
- depends on BROKEN
-
config USB_MUSB_DA8XX
tristate "DA8xx/OMAP-L1x"
depends on ARCH_DAVINCI_DA8XX
depends on OMAP_CONTROL_PHY || !OMAP_CONTROL_PHY
select GENERIC_PHY
-config USB_MUSB_AM35X
- tristate "AM35x"
- depends on ARCH_OMAP
- depends on NOP_USB_XCEIV
-
config USB_MUSB_DSPS
tristate "TI DSPS platforms"
depends on ARCH_OMAP2PLUS || COMPILE_TEST
depends on OF
depends on MIPS || COMPILE_TEST
depends on USB_MUSB_GADGET
- depends on USB=n || USB_OTG_DISABLE_EXTERNAL_HUB
select USB_ROLE_SWITCH
config USB_MUSB_MEDIATEK
help
Enable DMA transfers using Mentor's engine.
-config USB_TI_CPPI_DMA
- bool 'TI CPPI (Davinci)'
- depends on USB_MUSB_DAVINCI
- help
- Enable DMA transfers when TI CPPI DMA is available.
-
config USB_TI_CPPI41_DMA
bool 'TI CPPI 4.1'
depends on (ARCH_OMAP || ARCH_DAVINCI_DA8XX) && DMADEVICES
# Hardware Glue Layer
obj-$(CONFIG_USB_MUSB_OMAP2PLUS) += omap2430.o
-obj-$(CONFIG_USB_MUSB_AM35X) += am35x.o
obj-$(CONFIG_USB_MUSB_DSPS) += musb_dsps.o
obj-$(CONFIG_USB_MUSB_TUSB6010) += tusb6010.o
-obj-$(CONFIG_USB_MUSB_DAVINCI) += davinci.o
obj-$(CONFIG_USB_MUSB_DA8XX) += da8xx.o
obj-$(CONFIG_USB_MUSB_UX500) += ux500.o
obj-$(CONFIG_USB_MUSB_JZ4740) += jz4740.o
# though PIO is always there to back up DMA, and for ep0
musb_hdrc-$(CONFIG_USB_INVENTRA_DMA) += musbhsdma.o
-musb_hdrc-$(CONFIG_USB_TI_CPPI_DMA) += cppi_dma.o
musb_hdrc-$(CONFIG_USB_TUSB_OMAP_DMA) += tusb6010_omap.o
musb_hdrc-$(CONFIG_USB_UX500_DMA) += ux500_dma.o
musb_hdrc-$(CONFIG_USB_TI_CPPI41_DMA) += musb_cppi41.o
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-
-/*
- * Texas Instruments AM35x "glue layer"
- *
- * Copyright (c) 2010, by Texas Instruments
- *
- * Based on the DA8xx "glue layer" code.
- * Copyright (c) 2008-2009, MontaVista Software, Inc. <source@mvista.com>
- *
- * This file is part of the Inventra Controller Driver for Linux.
- */
-
-#include <linux/module.h>
-#include <linux/clk.h>
-#include <linux/err.h>
-#include <linux/io.h>
-#include <linux/platform_device.h>
-#include <linux/dma-mapping.h>
-#include <linux/usb/usb_phy_generic.h>
-#include <linux/platform_data/usb-omap.h>
-
-#include "musb_core.h"
-
-/*
- * AM35x specific definitions
- */
-/* USB 2.0 OTG module registers */
-#define USB_REVISION_REG 0x00
-#define USB_CTRL_REG 0x04
-#define USB_STAT_REG 0x08
-#define USB_EMULATION_REG 0x0c
-/* 0x10 Reserved */
-#define USB_AUTOREQ_REG 0x14
-#define USB_SRP_FIX_TIME_REG 0x18
-#define USB_TEARDOWN_REG 0x1c
-#define EP_INTR_SRC_REG 0x20
-#define EP_INTR_SRC_SET_REG 0x24
-#define EP_INTR_SRC_CLEAR_REG 0x28
-#define EP_INTR_MASK_REG 0x2c
-#define EP_INTR_MASK_SET_REG 0x30
-#define EP_INTR_MASK_CLEAR_REG 0x34
-#define EP_INTR_SRC_MASKED_REG 0x38
-#define CORE_INTR_SRC_REG 0x40
-#define CORE_INTR_SRC_SET_REG 0x44
-#define CORE_INTR_SRC_CLEAR_REG 0x48
-#define CORE_INTR_MASK_REG 0x4c
-#define CORE_INTR_MASK_SET_REG 0x50
-#define CORE_INTR_MASK_CLEAR_REG 0x54
-#define CORE_INTR_SRC_MASKED_REG 0x58
-/* 0x5c Reserved */
-#define USB_END_OF_INTR_REG 0x60
-
-/* Control register bits */
-#define AM35X_SOFT_RESET_MASK 1
-
-/* USB interrupt register bits */
-#define AM35X_INTR_USB_SHIFT 16
-#define AM35X_INTR_USB_MASK (0x1ff << AM35X_INTR_USB_SHIFT)
-#define AM35X_INTR_DRVVBUS 0x100
-#define AM35X_INTR_RX_SHIFT 16
-#define AM35X_INTR_TX_SHIFT 0
-#define AM35X_TX_EP_MASK 0xffff /* EP0 + 15 Tx EPs */
-#define AM35X_RX_EP_MASK 0xfffe /* 15 Rx EPs */
-#define AM35X_TX_INTR_MASK (AM35X_TX_EP_MASK << AM35X_INTR_TX_SHIFT)
-#define AM35X_RX_INTR_MASK (AM35X_RX_EP_MASK << AM35X_INTR_RX_SHIFT)
-
-#define USB_MENTOR_CORE_OFFSET 0x400
-
-struct am35x_glue {
- struct device *dev;
- struct platform_device *musb;
- struct platform_device *phy;
- struct clk *phy_clk;
- struct clk *clk;
-};
-
-/*
- * am35x_musb_enable - enable interrupts
- */
-static void am35x_musb_enable(struct musb *musb)
-{
- void __iomem *reg_base = musb->ctrl_base;
- u32 epmask;
-
- /* Workaround: setup IRQs through both register sets. */
- epmask = ((musb->epmask & AM35X_TX_EP_MASK) << AM35X_INTR_TX_SHIFT) |
- ((musb->epmask & AM35X_RX_EP_MASK) << AM35X_INTR_RX_SHIFT);
-
- musb_writel(reg_base, EP_INTR_MASK_SET_REG, epmask);
- musb_writel(reg_base, CORE_INTR_MASK_SET_REG, AM35X_INTR_USB_MASK);
-
- /* Force the DRVVBUS IRQ so we can start polling for ID change. */
- musb_writel(reg_base, CORE_INTR_SRC_SET_REG,
- AM35X_INTR_DRVVBUS << AM35X_INTR_USB_SHIFT);
-}
-
-/*
- * am35x_musb_disable - disable HDRC and flush interrupts
- */
-static void am35x_musb_disable(struct musb *musb)
-{
- void __iomem *reg_base = musb->ctrl_base;
-
- musb_writel(reg_base, CORE_INTR_MASK_CLEAR_REG, AM35X_INTR_USB_MASK);
- musb_writel(reg_base, EP_INTR_MASK_CLEAR_REG,
- AM35X_TX_INTR_MASK | AM35X_RX_INTR_MASK);
- musb_writel(reg_base, USB_END_OF_INTR_REG, 0);
-}
-
-#define portstate(stmt) stmt
-
-static void am35x_musb_set_vbus(struct musb *musb, int is_on)
-{
- WARN_ON(is_on && is_peripheral_active(musb));
-}
-
-#define POLL_SECONDS 2
-
-static void otg_timer(struct timer_list *t)
-{
- struct musb *musb = from_timer(musb, t, dev_timer);
- void __iomem *mregs = musb->mregs;
- u8 devctl;
- unsigned long flags;
-
- /*
- * We poll because AM35x's won't expose several OTG-critical
- * status change events (from the transceiver) otherwise.
- */
- devctl = musb_readb(mregs, MUSB_DEVCTL);
- dev_dbg(musb->controller, "Poll devctl %02x (%s)\n", devctl,
- usb_otg_state_string(musb->xceiv->otg->state));
-
- spin_lock_irqsave(&musb->lock, flags);
- switch (musb->xceiv->otg->state) {
- case OTG_STATE_A_WAIT_BCON:
- devctl &= ~MUSB_DEVCTL_SESSION;
- musb_writeb(musb->mregs, MUSB_DEVCTL, devctl);
-
- devctl = musb_readb(musb->mregs, MUSB_DEVCTL);
- if (devctl & MUSB_DEVCTL_BDEVICE) {
- musb->xceiv->otg->state = OTG_STATE_B_IDLE;
- MUSB_DEV_MODE(musb);
- } else {
- musb->xceiv->otg->state = OTG_STATE_A_IDLE;
- MUSB_HST_MODE(musb);
- }
- break;
- case OTG_STATE_A_WAIT_VFALL:
- musb->xceiv->otg->state = OTG_STATE_A_WAIT_VRISE;
- musb_writel(musb->ctrl_base, CORE_INTR_SRC_SET_REG,
- MUSB_INTR_VBUSERROR << AM35X_INTR_USB_SHIFT);
- break;
- case OTG_STATE_B_IDLE:
- devctl = musb_readb(mregs, MUSB_DEVCTL);
- if (devctl & MUSB_DEVCTL_BDEVICE)
- mod_timer(&musb->dev_timer, jiffies + POLL_SECONDS * HZ);
- else
- musb->xceiv->otg->state = OTG_STATE_A_IDLE;
- break;
- default:
- break;
- }
- spin_unlock_irqrestore(&musb->lock, flags);
-}
-
-static void am35x_musb_try_idle(struct musb *musb, unsigned long timeout)
-{
- static unsigned long last_timer;
-
- if (timeout == 0)
- timeout = jiffies + msecs_to_jiffies(3);
-
- /* Never idle if active, or when VBUS timeout is not set as host */
- if (musb->is_active || (musb->a_wait_bcon == 0 &&
- musb->xceiv->otg->state == OTG_STATE_A_WAIT_BCON)) {
- dev_dbg(musb->controller, "%s active, deleting timer\n",
- usb_otg_state_string(musb->xceiv->otg->state));
- del_timer(&musb->dev_timer);
- last_timer = jiffies;
- return;
- }
-
- if (time_after(last_timer, timeout) && timer_pending(&musb->dev_timer)) {
- dev_dbg(musb->controller, "Longer idle timer already pending, ignoring...\n");
- return;
- }
- last_timer = timeout;
-
- dev_dbg(musb->controller, "%s inactive, starting idle timer for %u ms\n",
- usb_otg_state_string(musb->xceiv->otg->state),
- jiffies_to_msecs(timeout - jiffies));
- mod_timer(&musb->dev_timer, timeout);
-}
-
-static irqreturn_t am35x_musb_interrupt(int irq, void *hci)
-{
- struct musb *musb = hci;
- void __iomem *reg_base = musb->ctrl_base;
- struct device *dev = musb->controller;
- struct musb_hdrc_platform_data *plat = dev_get_platdata(dev);
- struct omap_musb_board_data *data = plat->board_data;
- unsigned long flags;
- irqreturn_t ret = IRQ_NONE;
- u32 epintr, usbintr;
-
- spin_lock_irqsave(&musb->lock, flags);
-
- /* Get endpoint interrupts */
- epintr = musb_readl(reg_base, EP_INTR_SRC_MASKED_REG);
-
- if (epintr) {
- musb_writel(reg_base, EP_INTR_SRC_CLEAR_REG, epintr);
-
- musb->int_rx =
- (epintr & AM35X_RX_INTR_MASK) >> AM35X_INTR_RX_SHIFT;
- musb->int_tx =
- (epintr & AM35X_TX_INTR_MASK) >> AM35X_INTR_TX_SHIFT;
- }
-
- /* Get usb core interrupts */
- usbintr = musb_readl(reg_base, CORE_INTR_SRC_MASKED_REG);
- if (!usbintr && !epintr)
- goto eoi;
-
- if (usbintr) {
- musb_writel(reg_base, CORE_INTR_SRC_CLEAR_REG, usbintr);
-
- musb->int_usb =
- (usbintr & AM35X_INTR_USB_MASK) >> AM35X_INTR_USB_SHIFT;
- }
- /*
- * DRVVBUS IRQs are the only proxy we have (a very poor one!) for
- * AM35x's missing ID change IRQ. We need an ID change IRQ to
- * switch appropriately between halves of the OTG state machine.
- * Managing DEVCTL.SESSION per Mentor docs requires that we know its
- * value but DEVCTL.BDEVICE is invalid without DEVCTL.SESSION set.
- * Also, DRVVBUS pulses for SRP (but not at 5V) ...
- */
- if (usbintr & (AM35X_INTR_DRVVBUS << AM35X_INTR_USB_SHIFT)) {
- int drvvbus = musb_readl(reg_base, USB_STAT_REG);
- void __iomem *mregs = musb->mregs;
- u8 devctl = musb_readb(mregs, MUSB_DEVCTL);
- int err;
-
- err = musb->int_usb & MUSB_INTR_VBUSERROR;
- if (err) {
- /*
- * The Mentor core doesn't debounce VBUS as needed
- * to cope with device connect current spikes. This
- * means it's not uncommon for bus-powered devices
- * to get VBUS errors during enumeration.
- *
- * This is a workaround, but newer RTL from Mentor
- * seems to allow a better one: "re"-starting sessions
- * without waiting for VBUS to stop registering in
- * devctl.
- */
- musb->int_usb &= ~MUSB_INTR_VBUSERROR;
- musb->xceiv->otg->state = OTG_STATE_A_WAIT_VFALL;
- mod_timer(&musb->dev_timer, jiffies + POLL_SECONDS * HZ);
- WARNING("VBUS error workaround (delay coming)\n");
- } else if (drvvbus) {
- MUSB_HST_MODE(musb);
- musb->xceiv->otg->state = OTG_STATE_A_WAIT_VRISE;
- portstate(musb->port1_status |= USB_PORT_STAT_POWER);
- del_timer(&musb->dev_timer);
- } else {
- musb->is_active = 0;
- MUSB_DEV_MODE(musb);
- musb->xceiv->otg->state = OTG_STATE_B_IDLE;
- portstate(musb->port1_status &= ~USB_PORT_STAT_POWER);
- }
-
- /* NOTE: this must complete power-on within 100 ms. */
- dev_dbg(musb->controller, "VBUS %s (%s)%s, devctl %02x\n",
- drvvbus ? "on" : "off",
- usb_otg_state_string(musb->xceiv->otg->state),
- err ? " ERROR" : "",
- devctl);
- ret = IRQ_HANDLED;
- }
-
- /* Drop spurious RX and TX if device is disconnected */
- if (musb->int_usb & MUSB_INTR_DISCONNECT) {
- musb->int_tx = 0;
- musb->int_rx = 0;
- }
-
- if (musb->int_tx || musb->int_rx || musb->int_usb)
- ret |= musb_interrupt(musb);
-
-eoi:
- /* EOI needs to be written for the IRQ to be re-asserted. */
- if (ret == IRQ_HANDLED || epintr || usbintr) {
- /* clear level interrupt */
- if (data->clear_irq)
- data->clear_irq();
- /* write EOI */
- musb_writel(reg_base, USB_END_OF_INTR_REG, 0);
- }
-
- /* Poll for ID change */
- if (musb->xceiv->otg->state == OTG_STATE_B_IDLE)
- mod_timer(&musb->dev_timer, jiffies + POLL_SECONDS * HZ);
-
- spin_unlock_irqrestore(&musb->lock, flags);
-
- return ret;
-}
-
-static int am35x_musb_set_mode(struct musb *musb, u8 musb_mode)
-{
- struct device *dev = musb->controller;
- struct musb_hdrc_platform_data *plat = dev_get_platdata(dev);
- struct omap_musb_board_data *data = plat->board_data;
- int retval = 0;
-
- if (data->set_mode)
- data->set_mode(musb_mode);
- else
- retval = -EIO;
-
- return retval;
-}
-
-static int am35x_musb_init(struct musb *musb)
-{
- struct device *dev = musb->controller;
- struct musb_hdrc_platform_data *plat = dev_get_platdata(dev);
- struct omap_musb_board_data *data = plat->board_data;
- void __iomem *reg_base = musb->ctrl_base;
- u32 rev;
-
- musb->mregs += USB_MENTOR_CORE_OFFSET;
-
- /* Returns zero if e.g. not clocked */
- rev = musb_readl(reg_base, USB_REVISION_REG);
- if (!rev)
- return -ENODEV;
-
- musb->xceiv = usb_get_phy(USB_PHY_TYPE_USB2);
- if (IS_ERR_OR_NULL(musb->xceiv))
- return -EPROBE_DEFER;
-
- timer_setup(&musb->dev_timer, otg_timer, 0);
-
- /* Reset the musb */
- if (data->reset)
- data->reset();
-
- /* Reset the controller */
- musb_writel(reg_base, USB_CTRL_REG, AM35X_SOFT_RESET_MASK);
-
- /* Start the on-chip PHY and its PLL. */
- if (data->set_phy_power)
- data->set_phy_power(1);
-
- msleep(5);
-
- musb->isr = am35x_musb_interrupt;
-
- /* clear level interrupt */
- if (data->clear_irq)
- data->clear_irq();
-
- return 0;
-}
-
-static int am35x_musb_exit(struct musb *musb)
-{
- struct device *dev = musb->controller;
- struct musb_hdrc_platform_data *plat = dev_get_platdata(dev);
- struct omap_musb_board_data *data = plat->board_data;
-
- del_timer_sync(&musb->dev_timer);
-
- /* Shutdown the on-chip PHY and its PLL. */
- if (data->set_phy_power)
- data->set_phy_power(0);
-
- usb_put_phy(musb->xceiv);
-
- return 0;
-}
-
-/* AM35x supports only 32bit read operation */
-static void am35x_read_fifo(struct musb_hw_ep *hw_ep, u16 len, u8 *dst)
-{
- void __iomem *fifo = hw_ep->fifo;
- u32 val;
- int i;
-
- /* Read for 32bit-aligned destination address */
- if (likely((0x03 & (unsigned long) dst) == 0) && len >= 4) {
- readsl(fifo, dst, len >> 2);
- dst += len & ~0x03;
- len &= 0x03;
- }
- /*
- * Now read the remaining 1 to 3 byte or complete length if
- * unaligned address.
- */
- if (len > 4) {
- for (i = 0; i < (len >> 2); i++) {
- *(u32 *) dst = musb_readl(fifo, 0);
- dst += 4;
- }
- len &= 0x03;
- }
- if (len > 0) {
- val = musb_readl(fifo, 0);
- memcpy(dst, &val, len);
- }
-}
-
-static const struct musb_platform_ops am35x_ops = {
- .quirks = MUSB_DMA_INVENTRA | MUSB_INDEXED_EP,
- .init = am35x_musb_init,
- .exit = am35x_musb_exit,
-
- .read_fifo = am35x_read_fifo,
-#ifdef CONFIG_USB_INVENTRA_DMA
- .dma_init = musbhs_dma_controller_create,
- .dma_exit = musbhs_dma_controller_destroy,
-#endif
- .enable = am35x_musb_enable,
- .disable = am35x_musb_disable,
-
- .set_mode = am35x_musb_set_mode,
- .try_idle = am35x_musb_try_idle,
-
- .set_vbus = am35x_musb_set_vbus,
-};
-
-static const struct platform_device_info am35x_dev_info = {
- .name = "musb-hdrc",
- .id = PLATFORM_DEVID_AUTO,
- .dma_mask = DMA_BIT_MASK(32),
-};
-
-static int am35x_probe(struct platform_device *pdev)
-{
- struct musb_hdrc_platform_data *pdata = dev_get_platdata(&pdev->dev);
- struct platform_device *musb;
- struct am35x_glue *glue;
- struct platform_device_info pinfo;
- struct clk *phy_clk;
- struct clk *clk;
-
- int ret = -ENOMEM;
-
- glue = kzalloc(sizeof(*glue), GFP_KERNEL);
- if (!glue)
- goto err0;
-
- phy_clk = clk_get(&pdev->dev, "fck");
- if (IS_ERR(phy_clk)) {
- dev_err(&pdev->dev, "failed to get PHY clock\n");
- ret = PTR_ERR(phy_clk);
- goto err3;
- }
-
- clk = clk_get(&pdev->dev, "ick");
- if (IS_ERR(clk)) {
- dev_err(&pdev->dev, "failed to get clock\n");
- ret = PTR_ERR(clk);
- goto err4;
- }
-
- ret = clk_enable(phy_clk);
- if (ret) {
- dev_err(&pdev->dev, "failed to enable PHY clock\n");
- goto err5;
- }
-
- ret = clk_enable(clk);
- if (ret) {
- dev_err(&pdev->dev, "failed to enable clock\n");
- goto err6;
- }
-
- glue->dev = &pdev->dev;
- glue->phy_clk = phy_clk;
- glue->clk = clk;
-
- pdata->platform_ops = &am35x_ops;
-
- glue->phy = usb_phy_generic_register();
- if (IS_ERR(glue->phy)) {
- ret = PTR_ERR(glue->phy);
- goto err7;
- }
- platform_set_drvdata(pdev, glue);
-
- pinfo = am35x_dev_info;
- pinfo.parent = &pdev->dev;
- pinfo.res = pdev->resource;
- pinfo.num_res = pdev->num_resources;
- pinfo.data = pdata;
- pinfo.size_data = sizeof(*pdata);
- pinfo.fwnode = of_fwnode_handle(pdev->dev.of_node);
- pinfo.of_node_reused = true;
-
- glue->musb = musb = platform_device_register_full(&pinfo);
- if (IS_ERR(musb)) {
- ret = PTR_ERR(musb);
- dev_err(&pdev->dev, "failed to register musb device: %d\n", ret);
- goto err8;
- }
-
- return 0;
-
-err8:
- usb_phy_generic_unregister(glue->phy);
-
-err7:
- clk_disable(clk);
-
-err6:
- clk_disable(phy_clk);
-
-err5:
- clk_put(clk);
-
-err4:
- clk_put(phy_clk);
-
-err3:
- kfree(glue);
-
-err0:
- return ret;
-}
-
-static int am35x_remove(struct platform_device *pdev)
-{
- struct am35x_glue *glue = platform_get_drvdata(pdev);
-
- platform_device_unregister(glue->musb);
- usb_phy_generic_unregister(glue->phy);
- clk_disable(glue->clk);
- clk_disable(glue->phy_clk);
- clk_put(glue->clk);
- clk_put(glue->phy_clk);
- kfree(glue);
-
- return 0;
-}
-
-#ifdef CONFIG_PM_SLEEP
-static int am35x_suspend(struct device *dev)
-{
- struct am35x_glue *glue = dev_get_drvdata(dev);
- struct musb_hdrc_platform_data *plat = dev_get_platdata(dev);
- struct omap_musb_board_data *data = plat->board_data;
-
- /* Shutdown the on-chip PHY and its PLL. */
- if (data->set_phy_power)
- data->set_phy_power(0);
-
- clk_disable(glue->phy_clk);
- clk_disable(glue->clk);
-
- return 0;
-}
-
-static int am35x_resume(struct device *dev)
-{
- struct am35x_glue *glue = dev_get_drvdata(dev);
- struct musb_hdrc_platform_data *plat = dev_get_platdata(dev);
- struct omap_musb_board_data *data = plat->board_data;
- int ret;
-
- /* Start the on-chip PHY and its PLL. */
- if (data->set_phy_power)
- data->set_phy_power(1);
-
- ret = clk_enable(glue->phy_clk);
- if (ret) {
- dev_err(dev, "failed to enable PHY clock\n");
- return ret;
- }
-
- ret = clk_enable(glue->clk);
- if (ret) {
- dev_err(dev, "failed to enable clock\n");
- return ret;
- }
-
- return 0;
-}
-#endif
-
-static SIMPLE_DEV_PM_OPS(am35x_pm_ops, am35x_suspend, am35x_resume);
-
-static struct platform_driver am35x_driver = {
- .probe = am35x_probe,
- .remove = am35x_remove,
- .driver = {
- .name = "musb-am35x",
- .pm = &am35x_pm_ops,
- },
-};
-
-MODULE_DESCRIPTION("AM35x MUSB Glue Layer");
-MODULE_AUTHOR("Ajay Kumar Gupta <ajay.gupta@ti.com>");
-MODULE_LICENSE("GPL v2");
-module_platform_driver(am35x_driver);
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Copyright (C) 2005-2006 by Texas Instruments
- *
- * This file implements a DMA interface using TI's CPPI DMA.
- * For now it's DaVinci-only, but CPPI isn't specific to DaVinci or USB.
- * The TUSB6020, using VLYNQ, has CPPI that looks much like DaVinci.
- */
-
-#include <linux/module.h>
-#include <linux/platform_device.h>
-#include <linux/slab.h>
-#include <linux/usb.h>
-
-#include "musb_core.h"
-#include "musb_debug.h"
-#include "cppi_dma.h"
-#include "davinci.h"
-
-
-/* CPPI DMA status 7-mar-2006:
- *
- * - See musb_{host,gadget}.c for more info
- *
- * - Correct RX DMA generally forces the engine into irq-per-packet mode,
- * which can easily saturate the CPU under non-mass-storage loads.
- *
- * NOTES 24-aug-2006 (2.6.18-rc4):
- *
- * - peripheral RXDMA wedged in a test with packets of length 512/512/1.
- * evidently after the 1 byte packet was received and acked, the queue
- * of BDs got garbaged so it wouldn't empty the fifo. (rxcsr 0x2003,
- * and RX DMA0: 4 left, 80000000 8feff880, 8feff860 8feff860; 8f321401
- * 004001ff 00000001 .. 8feff860) Host was just getting NAKed on tx
- * of its next (512 byte) packet. IRQ issues?
- *
- * REVISIT: the "transfer DMA" glue between CPPI and USB fifos will
- * evidently also directly update the RX and TX CSRs ... so audit all
- * host and peripheral side DMA code to avoid CSR access after DMA has
- * been started.
- */
-
-/* REVISIT now we can avoid preallocating these descriptors; or
- * more simply, switch to a global freelist not per-channel ones.
- * Note: at full speed, 64 descriptors == 4K bulk data.
- */
-#define NUM_TXCHAN_BD 64
-#define NUM_RXCHAN_BD 64
-
-static inline void cpu_drain_writebuffer(void)
-{
- wmb();
-#ifdef CONFIG_CPU_ARM926T
- /* REVISIT this "should not be needed",
- * but lack of it sure seemed to hurt ...
- */
- asm("mcr p15, 0, r0, c7, c10, 4 @ drain write buffer\n");
-#endif
-}
-
-static inline struct cppi_descriptor *cppi_bd_alloc(struct cppi_channel *c)
-{
- struct cppi_descriptor *bd = c->freelist;
-
- if (bd)
- c->freelist = bd->next;
- return bd;
-}
-
-static inline void
-cppi_bd_free(struct cppi_channel *c, struct cppi_descriptor *bd)
-{
- if (!bd)
- return;
- bd->next = c->freelist;
- c->freelist = bd;
-}
-
-/*
- * Start DMA controller
- *
- * Initialize the DMA controller as necessary.
- */
-
-/* zero out entire rx state RAM entry for the channel */
-static void cppi_reset_rx(struct cppi_rx_stateram __iomem *rx)
-{
- musb_writel(&rx->rx_skipbytes, 0, 0);
- musb_writel(&rx->rx_head, 0, 0);
- musb_writel(&rx->rx_sop, 0, 0);
- musb_writel(&rx->rx_current, 0, 0);
- musb_writel(&rx->rx_buf_current, 0, 0);
- musb_writel(&rx->rx_len_len, 0, 0);
- musb_writel(&rx->rx_cnt_cnt, 0, 0);
-}
-
-/* zero out entire tx state RAM entry for the channel */
-static void cppi_reset_tx(struct cppi_tx_stateram __iomem *tx, u32 ptr)
-{
- musb_writel(&tx->tx_head, 0, 0);
- musb_writel(&tx->tx_buf, 0, 0);
- musb_writel(&tx->tx_current, 0, 0);
- musb_writel(&tx->tx_buf_current, 0, 0);
- musb_writel(&tx->tx_info, 0, 0);
- musb_writel(&tx->tx_rem_len, 0, 0);
- /* musb_writel(&tx->tx_dummy, 0, 0); */
- musb_writel(&tx->tx_complete, 0, ptr);
-}
-
-static void cppi_pool_init(struct cppi *cppi, struct cppi_channel *c)
-{
- int j;
-
- /* initialize channel fields */
- c->head = NULL;
- c->tail = NULL;
- c->last_processed = NULL;
- c->channel.status = MUSB_DMA_STATUS_UNKNOWN;
- c->controller = cppi;
- c->is_rndis = 0;
- c->freelist = NULL;
-
- /* build the BD Free list for the channel */
- for (j = 0; j < NUM_TXCHAN_BD + 1; j++) {
- struct cppi_descriptor *bd;
- dma_addr_t dma;
-
- bd = dma_pool_alloc(cppi->pool, GFP_KERNEL, &dma);
- bd->dma = dma;
- cppi_bd_free(c, bd);
- }
-}
-
-static int cppi_channel_abort(struct dma_channel *);
-
-static void cppi_pool_free(struct cppi_channel *c)
-{
- struct cppi *cppi = c->controller;
- struct cppi_descriptor *bd;
-
- (void) cppi_channel_abort(&c->channel);
- c->channel.status = MUSB_DMA_STATUS_UNKNOWN;
- c->controller = NULL;
-
- /* free all its bds */
- bd = c->last_processed;
- do {
- if (bd)
- dma_pool_free(cppi->pool, bd, bd->dma);
- bd = cppi_bd_alloc(c);
- } while (bd);
- c->last_processed = NULL;
-}
-
-static void cppi_controller_start(struct cppi *controller)
-{
- void __iomem *tibase;
- int i;
-
- /* do whatever is necessary to start controller */
- for (i = 0; i < ARRAY_SIZE(controller->tx); i++) {
- controller->tx[i].transmit = true;
- controller->tx[i].index = i;
- }
- for (i = 0; i < ARRAY_SIZE(controller->rx); i++) {
- controller->rx[i].transmit = false;
- controller->rx[i].index = i;
- }
-
- /* setup BD list on a per channel basis */
- for (i = 0; i < ARRAY_SIZE(controller->tx); i++)
- cppi_pool_init(controller, controller->tx + i);
- for (i = 0; i < ARRAY_SIZE(controller->rx); i++)
- cppi_pool_init(controller, controller->rx + i);
-
- tibase = controller->tibase;
- INIT_LIST_HEAD(&controller->tx_complete);
-
- /* initialise tx/rx channel head pointers to zero */
- for (i = 0; i < ARRAY_SIZE(controller->tx); i++) {
- struct cppi_channel *tx_ch = controller->tx + i;
- struct cppi_tx_stateram __iomem *tx;
-
- INIT_LIST_HEAD(&tx_ch->tx_complete);
-
- tx = tibase + DAVINCI_TXCPPI_STATERAM_OFFSET(i);
- tx_ch->state_ram = tx;
- cppi_reset_tx(tx, 0);
- }
- for (i = 0; i < ARRAY_SIZE(controller->rx); i++) {
- struct cppi_channel *rx_ch = controller->rx + i;
- struct cppi_rx_stateram __iomem *rx;
-
- INIT_LIST_HEAD(&rx_ch->tx_complete);
-
- rx = tibase + DAVINCI_RXCPPI_STATERAM_OFFSET(i);
- rx_ch->state_ram = rx;
- cppi_reset_rx(rx);
- }
-
- /* enable individual cppi channels */
- musb_writel(tibase, DAVINCI_TXCPPI_INTENAB_REG,
- DAVINCI_DMA_ALL_CHANNELS_ENABLE);
- musb_writel(tibase, DAVINCI_RXCPPI_INTENAB_REG,
- DAVINCI_DMA_ALL_CHANNELS_ENABLE);
-
- /* enable tx/rx CPPI control */
- musb_writel(tibase, DAVINCI_TXCPPI_CTRL_REG, DAVINCI_DMA_CTRL_ENABLE);
- musb_writel(tibase, DAVINCI_RXCPPI_CTRL_REG, DAVINCI_DMA_CTRL_ENABLE);
-
- /* disable RNDIS mode, also host rx RNDIS autorequest */
- musb_writel(tibase, DAVINCI_RNDIS_REG, 0);
- musb_writel(tibase, DAVINCI_AUTOREQ_REG, 0);
-}
-
-/*
- * Stop DMA controller
- *
- * De-Init the DMA controller as necessary.
- */
-
-static void cppi_controller_stop(struct cppi *controller)
-{
- void __iomem *tibase;
- int i;
- struct musb *musb;
-
- musb = controller->controller.musb;
-
- tibase = controller->tibase;
- /* DISABLE INDIVIDUAL CHANNEL Interrupts */
- musb_writel(tibase, DAVINCI_TXCPPI_INTCLR_REG,
- DAVINCI_DMA_ALL_CHANNELS_ENABLE);
- musb_writel(tibase, DAVINCI_RXCPPI_INTCLR_REG,
- DAVINCI_DMA_ALL_CHANNELS_ENABLE);
-
- musb_dbg(musb, "Tearing down RX and TX Channels");
- for (i = 0; i < ARRAY_SIZE(controller->tx); i++) {
- /* FIXME restructure of txdma to use bds like rxdma */
- controller->tx[i].last_processed = NULL;
- cppi_pool_free(controller->tx + i);
- }
- for (i = 0; i < ARRAY_SIZE(controller->rx); i++)
- cppi_pool_free(controller->rx + i);
-
- /* in Tx Case proper teardown is supported. We resort to disabling
- * Tx/Rx CPPI after cleanup of Tx channels. Before TX teardown is
- * complete TX CPPI cannot be disabled.
- */
- /*disable tx/rx cppi */
- musb_writel(tibase, DAVINCI_TXCPPI_CTRL_REG, DAVINCI_DMA_CTRL_DISABLE);
- musb_writel(tibase, DAVINCI_RXCPPI_CTRL_REG, DAVINCI_DMA_CTRL_DISABLE);
-}
-
-/* While dma channel is allocated, we only want the core irqs active
- * for fault reports, otherwise we'd get irqs that we don't care about.
- * Except for TX irqs, where dma done != fifo empty and reusable ...
- *
- * NOTE: docs don't say either way, but irq masking **enables** irqs.
- *
- * REVISIT same issue applies to pure PIO usage too, and non-cppi dma...
- */
-static inline void core_rxirq_disable(void __iomem *tibase, unsigned epnum)
-{
- musb_writel(tibase, DAVINCI_USB_INT_MASK_CLR_REG, 1 << (epnum + 8));
-}
-
-static inline void core_rxirq_enable(void __iomem *tibase, unsigned epnum)
-{
- musb_writel(tibase, DAVINCI_USB_INT_MASK_SET_REG, 1 << (epnum + 8));
-}
-
-
-/*
- * Allocate a CPPI Channel for DMA. With CPPI, channels are bound to
- * each transfer direction of a non-control endpoint, so allocating
- * (and deallocating) is mostly a way to notice bad housekeeping on
- * the software side. We assume the irqs are always active.
- */
-static struct dma_channel *
-cppi_channel_allocate(struct dma_controller *c,
- struct musb_hw_ep *ep, u8 transmit)
-{
- struct cppi *controller;
- u8 index;
- struct cppi_channel *cppi_ch;
- void __iomem *tibase;
- struct musb *musb;
-
- controller = container_of(c, struct cppi, controller);
- tibase = controller->tibase;
- musb = c->musb;
-
- /* ep0 doesn't use DMA; remember cppi indices are 0..N-1 */
- index = ep->epnum - 1;
-
- /* return the corresponding CPPI Channel Handle, and
- * probably disable the non-CPPI irq until we need it.
- */
- if (transmit) {
- if (index >= ARRAY_SIZE(controller->tx)) {
- musb_dbg(musb, "no %cX%d CPPI channel", 'T', index);
- return NULL;
- }
- cppi_ch = controller->tx + index;
- } else {
- if (index >= ARRAY_SIZE(controller->rx)) {
- musb_dbg(musb, "no %cX%d CPPI channel", 'R', index);
- return NULL;
- }
- cppi_ch = controller->rx + index;
- core_rxirq_disable(tibase, ep->epnum);
- }
-
- /* REVISIT make this an error later once the same driver code works
- * with the other DMA engine too
- */
- if (cppi_ch->hw_ep)
- musb_dbg(musb, "re-allocating DMA%d %cX channel %p",
- index, transmit ? 'T' : 'R', cppi_ch);
- cppi_ch->hw_ep = ep;
- cppi_ch->channel.status = MUSB_DMA_STATUS_FREE;
- cppi_ch->channel.max_len = 0x7fffffff;
-
- musb_dbg(musb, "Allocate CPPI%d %cX", index, transmit ? 'T' : 'R');
- return &cppi_ch->channel;
-}
-
-/* Release a CPPI Channel. */
-static void cppi_channel_release(struct dma_channel *channel)
-{
- struct cppi_channel *c;
- void __iomem *tibase;
-
- /* REVISIT: for paranoia, check state and abort if needed... */
-
- c = container_of(channel, struct cppi_channel, channel);
- tibase = c->controller->tibase;
- if (!c->hw_ep)
- musb_dbg(c->controller->controller.musb,
- "releasing idle DMA channel %p", c);
- else if (!c->transmit)
- core_rxirq_enable(tibase, c->index + 1);
-
- /* for now, leave its cppi IRQ enabled (we won't trigger it) */
- c->hw_ep = NULL;
- channel->status = MUSB_DMA_STATUS_UNKNOWN;
-}
-
-/* Context: controller irqlocked */
-static void
-cppi_dump_rx(int level, struct cppi_channel *c, const char *tag)
-{
- void __iomem *base = c->controller->mregs;
- struct cppi_rx_stateram __iomem *rx = c->state_ram;
-
- musb_ep_select(base, c->index + 1);
-
- musb_dbg(c->controller->controller.musb,
- "RX DMA%d%s: %d left, csr %04x, "
- "%08x H%08x S%08x C%08x, "
- "B%08x L%08x %08x .. %08x",
- c->index, tag,
- musb_readl(c->controller->tibase,
- DAVINCI_RXCPPI_BUFCNT0_REG + 4 * c->index),
- musb_readw(c->hw_ep->regs, MUSB_RXCSR),
-
- musb_readl(&rx->rx_skipbytes, 0),
- musb_readl(&rx->rx_head, 0),
- musb_readl(&rx->rx_sop, 0),
- musb_readl(&rx->rx_current, 0),
-
- musb_readl(&rx->rx_buf_current, 0),
- musb_readl(&rx->rx_len_len, 0),
- musb_readl(&rx->rx_cnt_cnt, 0),
- musb_readl(&rx->rx_complete, 0)
- );
-}
-
-/* Context: controller irqlocked */
-static void
-cppi_dump_tx(int level, struct cppi_channel *c, const char *tag)
-{
- void __iomem *base = c->controller->mregs;
- struct cppi_tx_stateram __iomem *tx = c->state_ram;
-
- musb_ep_select(base, c->index + 1);
-
- musb_dbg(c->controller->controller.musb,
- "TX DMA%d%s: csr %04x, "
- "H%08x S%08x C%08x %08x, "
- "F%08x L%08x .. %08x",
- c->index, tag,
- musb_readw(c->hw_ep->regs, MUSB_TXCSR),
-
- musb_readl(&tx->tx_head, 0),
- musb_readl(&tx->tx_buf, 0),
- musb_readl(&tx->tx_current, 0),
- musb_readl(&tx->tx_buf_current, 0),
-
- musb_readl(&tx->tx_info, 0),
- musb_readl(&tx->tx_rem_len, 0),
- /* dummy/unused word 6 */
- musb_readl(&tx->tx_complete, 0)
- );
-}
-
-/* Context: controller irqlocked */
-static inline void
-cppi_rndis_update(struct cppi_channel *c, int is_rx,
- void __iomem *tibase, int is_rndis)
-{
- /* we may need to change the rndis flag for this cppi channel */
- if (c->is_rndis != is_rndis) {
- u32 value = musb_readl(tibase, DAVINCI_RNDIS_REG);
- u32 temp = 1 << (c->index);
-
- if (is_rx)
- temp <<= 16;
- if (is_rndis)
- value |= temp;
- else
- value &= ~temp;
- musb_writel(tibase, DAVINCI_RNDIS_REG, value);
- c->is_rndis = is_rndis;
- }
-}
-
-static void cppi_dump_rxbd(const char *tag, struct cppi_descriptor *bd)
-{
- pr_debug("RXBD/%s %08x: "
- "nxt %08x buf %08x off.blen %08x opt.plen %08x\n",
- tag, bd->dma,
- bd->hw_next, bd->hw_bufp, bd->hw_off_len,
- bd->hw_options);
-}
-
-static void cppi_dump_rxq(int level, const char *tag, struct cppi_channel *rx)
-{
- struct cppi_descriptor *bd;
-
- cppi_dump_rx(level, rx, tag);
- if (rx->last_processed)
- cppi_dump_rxbd("last", rx->last_processed);
- for (bd = rx->head; bd; bd = bd->next)
- cppi_dump_rxbd("active", bd);
-}
-
-
-/* NOTE: DaVinci autoreq is ignored except for host side "RNDIS" mode RX;
- * so we won't ever use it (see "CPPI RX Woes" below).
- */
-static inline int cppi_autoreq_update(struct cppi_channel *rx,
- void __iomem *tibase, int onepacket, unsigned n_bds)
-{
- u32 val;
-
-#ifdef RNDIS_RX_IS_USABLE
- u32 tmp;
- /* assert(is_host_active(musb)) */
-
- /* start from "AutoReq never" */
- tmp = musb_readl(tibase, DAVINCI_AUTOREQ_REG);
- val = tmp & ~((0x3) << (rx->index * 2));
-
- /* HCD arranged reqpkt for packet #1. we arrange int
- * for all but the last one, maybe in two segments.
- */
- if (!onepacket) {
-#if 0
- /* use two segments, autoreq "all" then the last "never" */
- val |= ((0x3) << (rx->index * 2));
- n_bds--;
-#else
- /* one segment, autoreq "all-but-last" */
- val |= ((0x1) << (rx->index * 2));
-#endif
- }
-
- if (val != tmp) {
- int n = 100;
-
- /* make sure that autoreq is updated before continuing */
- musb_writel(tibase, DAVINCI_AUTOREQ_REG, val);
- do {
- tmp = musb_readl(tibase, DAVINCI_AUTOREQ_REG);
- if (tmp == val)
- break;
- cpu_relax();
- } while (n-- > 0);
- }
-#endif
-
- /* REQPKT is turned off after each segment */
- if (n_bds && rx->channel.actual_len) {
- void __iomem *regs = rx->hw_ep->regs;
-
- val = musb_readw(regs, MUSB_RXCSR);
- if (!(val & MUSB_RXCSR_H_REQPKT)) {
- val |= MUSB_RXCSR_H_REQPKT | MUSB_RXCSR_H_WZC_BITS;
- musb_writew(regs, MUSB_RXCSR, val);
- /* flush writebuffer */
- val = musb_readw(regs, MUSB_RXCSR);
- }
- }
- return n_bds;
-}
-
-
-/* Buffer enqueuing Logic:
- *
- * - RX builds new queues each time, to help handle routine "early
- * termination" cases (faults, including errors and short reads)
- * more correctly.
- *
- * - for now, TX reuses the same queue of BDs every time
- *
- * REVISIT long term, we want a normal dynamic model.
- * ... the goal will be to append to the
- * existing queue, processing completed "dma buffers" (segments) on the fly.
- *
- * Otherwise we force an IRQ latency between requests, which slows us a lot
- * (especially in "transparent" dma). Unfortunately that model seems to be
- * inherent in the DMA model from the Mentor code, except in the rare case
- * of transfers big enough (~128+ KB) that we could append "middle" segments
- * in the TX paths. (RX can't do this, see below.)
- *
- * That's true even in the CPPI- friendly iso case, where most urbs have
- * several small segments provided in a group and where the "packet at a time"
- * "transparent" DMA model is always correct, even on the RX side.
- */
-
-/*
- * CPPI TX:
- * ========
- * TX is a lot more reasonable than RX; it doesn't need to run in
- * irq-per-packet mode very often. RNDIS mode seems to behave too
- * (except how it handles the exactly-N-packets case). Building a
- * txdma queue with multiple requests (urb or usb_request) looks
- * like it would work ... but fault handling would need much testing.
- *
- * The main issue with TX mode RNDIS relates to transfer lengths that
- * are an exact multiple of the packet length. It appears that there's
- * a hiccup in that case (maybe the DMA completes before the ZLP gets
- * written?) boiling down to not being able to rely on CPPI writing any
- * terminating zero length packet before the next transfer is written.
- * So that's punted to PIO; better yet, gadget drivers can avoid it.
- *
- * Plus, there's allegedly an undocumented constraint that rndis transfer
- * length be a multiple of 64 bytes ... but the chip doesn't act that
- * way, and we really don't _want_ that behavior anyway.
- *
- * On TX, "transparent" mode works ... although experiments have shown
- * problems trying to use the SOP/EOP bits in different USB packets.
- *
- * REVISIT try to handle terminating zero length packets using CPPI
- * instead of doing it by PIO after an IRQ. (Meanwhile, make Ethernet
- * links avoid that issue by forcing them to avoid zlps.)
- */
-static void
-cppi_next_tx_segment(struct musb *musb, struct cppi_channel *tx)
-{
- unsigned maxpacket = tx->maxpacket;
- dma_addr_t addr = tx->buf_dma + tx->offset;
- size_t length = tx->buf_len - tx->offset;
- struct cppi_descriptor *bd;
- unsigned n_bds;
- unsigned i;
- struct cppi_tx_stateram __iomem *tx_ram = tx->state_ram;
- int rndis;
-
- /* TX can use the CPPI "rndis" mode, where we can probably fit this
- * transfer in one BD and one IRQ. The only time we would NOT want
- * to use it is when hardware constraints prevent it, or if we'd
- * trigger the "send a ZLP?" confusion.
- */
- rndis = (maxpacket & 0x3f) == 0
- && length > maxpacket
- && length < 0xffff
- && (length % maxpacket) != 0;
-
- if (rndis) {
- maxpacket = length;
- n_bds = 1;
- } else {
- if (length)
- n_bds = DIV_ROUND_UP(length, maxpacket);
- else
- n_bds = 1;
- n_bds = min(n_bds, (unsigned) NUM_TXCHAN_BD);
- length = min(n_bds * maxpacket, length);
- }
-
- musb_dbg(musb, "TX DMA%d, pktSz %d %s bds %d dma 0x%llx len %u",
- tx->index,
- maxpacket,
- rndis ? "rndis" : "transparent",
- n_bds,
- (unsigned long long)addr, length);
-
- cppi_rndis_update(tx, 0, musb->ctrl_base, rndis);
-
- /* assuming here that channel_program is called during
- * transfer initiation ... current code maintains state
- * for one outstanding request only (no queues, not even
- * the implicit ones of an iso urb).
- */
-
- bd = tx->freelist;
- tx->head = bd;
- tx->last_processed = NULL;
-
- /* FIXME use BD pool like RX side does, and just queue
- * the minimum number for this request.
- */
-
- /* Prepare queue of BDs first, then hand it to hardware.
- * All BDs except maybe the last should be of full packet
- * size; for RNDIS there _is_ only that last packet.
- */
- for (i = 0; i < n_bds; ) {
- if (++i < n_bds && bd->next)
- bd->hw_next = bd->next->dma;
- else
- bd->hw_next = 0;
-
- bd->hw_bufp = tx->buf_dma + tx->offset;
-
- /* FIXME set EOP only on the last packet,
- * SOP only on the first ... avoid IRQs
- */
- if ((tx->offset + maxpacket) <= tx->buf_len) {
- tx->offset += maxpacket;
- bd->hw_off_len = maxpacket;
- bd->hw_options = CPPI_SOP_SET | CPPI_EOP_SET
- | CPPI_OWN_SET | maxpacket;
- } else {
- /* only this one may be a partial USB Packet */
- u32 partial_len;
-
- partial_len = tx->buf_len - tx->offset;
- tx->offset = tx->buf_len;
- bd->hw_off_len = partial_len;
-
- bd->hw_options = CPPI_SOP_SET | CPPI_EOP_SET
- | CPPI_OWN_SET | partial_len;
- if (partial_len == 0)
- bd->hw_options |= CPPI_ZERO_SET;
- }
-
- musb_dbg(musb, "TXBD %p: nxt %08x buf %08x len %04x opt %08x",
- bd, bd->hw_next, bd->hw_bufp,
- bd->hw_off_len, bd->hw_options);
-
- /* update the last BD enqueued to the list */
- tx->tail = bd;
- bd = bd->next;
- }
-
- /* BDs live in DMA-coherent memory, but writes might be pending */
- cpu_drain_writebuffer();
-
- /* Write to the HeadPtr in state RAM to trigger */
- musb_writel(&tx_ram->tx_head, 0, (u32)tx->freelist->dma);
-
- cppi_dump_tx(5, tx, "/S");
-}
-
-/*
- * CPPI RX Woes:
- * =============
- * Consider a 1KB bulk RX buffer in two scenarios: (a) it's fed two 300 byte
- * packets back-to-back, and (b) it's fed two 512 byte packets back-to-back.
- * (Full speed transfers have similar scenarios.)
- *
- * The correct behavior for Linux is that (a) fills the buffer with 300 bytes,
- * and the next packet goes into a buffer that's queued later; while (b) fills
- * the buffer with 1024 bytes. How to do that with CPPI?
- *
- * - RX queues in "rndis" mode -- one single BD -- handle (a) correctly, but
- * (b) loses **BADLY** because nothing (!) happens when that second packet
- * fills the buffer, much less when a third one arrives. (Which makes this
- * not a "true" RNDIS mode. In the RNDIS protocol short-packet termination
- * is optional, and it's fine if peripherals -- not hosts! -- pad messages
- * out to end-of-buffer. Standard PCI host controller DMA descriptors
- * implement that mode by default ... which is no accident.)
- *
- * - RX queues in "transparent" mode -- two BDs with 512 bytes each -- have
- * converse problems: (b) is handled right, but (a) loses badly. CPPI RX
- * ignores SOP/EOP markings and processes both of those BDs; so both packets
- * are loaded into the buffer (with a 212 byte gap between them), and the next
- * buffer queued will NOT get its 300 bytes of data. (It seems like SOP/EOP
- * are intended as outputs for RX queues, not inputs...)
- *
- * - A variant of "transparent" mode -- one BD at a time -- is the only way to
- * reliably make both cases work, with software handling both cases correctly
- * and at the significant penalty of needing an IRQ per packet. (The lack of
- * I/O overlap can be slightly ameliorated by enabling double buffering.)
- *
- * So how to get rid of IRQ-per-packet? The transparent multi-BD case could
- * be used in special cases like mass storage, which sets URB_SHORT_NOT_OK
- * (or maybe its peripheral side counterpart) to flag (a) scenarios as errors
- * with guaranteed driver level fault recovery and scrubbing out what's left
- * of that garbaged datastream.
- *
- * But there seems to be no way to identify the cases where CPPI RNDIS mode
- * is appropriate -- which do NOT include RNDIS host drivers, but do include
- * the CDC Ethernet driver! -- and the documentation is incomplete/wrong.
- * So we can't _ever_ use RX RNDIS mode ... except by using a heuristic
- * that applies best on the peripheral side (and which could fail rudely).
- *
- * Leaving only "transparent" mode; we avoid multi-bd modes in almost all
- * cases other than mass storage class. Otherwise we're correct but slow,
- * since CPPI penalizes our need for a "true RNDIS" default mode.
- */
-
-
-/* Heuristic, intended to kick in for ethernet/rndis peripheral ONLY
- *
- * IFF
- * (a) peripheral mode ... since rndis peripherals could pad their
- * writes to hosts, causing i/o failure; or we'd have to cope with
- * a largely unknowable variety of host side protocol variants
- * (b) and short reads are NOT errors ... since full reads would
- * cause those same i/o failures
- * (c) and read length is
- * - less than 64KB (max per cppi descriptor)
- * - not a multiple of 4096 (g_zero default, full reads typical)
- * - N (>1) packets long, ditto (full reads not EXPECTED)
- * THEN
- * try rx rndis mode
- *
- * Cost of heuristic failing: RXDMA wedges at the end of transfers that
- * fill out the whole buffer. Buggy host side usb network drivers could
- * trigger that, but "in the field" such bugs seem to be all but unknown.
- *
- * So this module parameter lets the heuristic be disabled. When using
- * gadgetfs, the heuristic will probably need to be disabled.
- */
-static bool cppi_rx_rndis = 1;
-
-module_param(cppi_rx_rndis, bool, 0);
-MODULE_PARM_DESC(cppi_rx_rndis, "enable/disable RX RNDIS heuristic");
-
-
-/**
- * cppi_next_rx_segment - dma read for the next chunk of a buffer
- * @musb: the controller
- * @rx: dma channel
- * @onepacket: true unless caller treats short reads as errors, and
- * performs fault recovery above usbcore.
- * Context: controller irqlocked
- *
- * See above notes about why we can't use multi-BD RX queues except in
- * rare cases (mass storage class), and can never use the hardware "rndis"
- * mode (since it's not a "true" RNDIS mode) with complete safety..
- *
- * It's ESSENTIAL that callers specify "onepacket" mode unless they kick in
- * code to recover from corrupted datastreams after each short transfer.
- */
-static void
-cppi_next_rx_segment(struct musb *musb, struct cppi_channel *rx, int onepacket)
-{
- unsigned maxpacket = rx->maxpacket;
- dma_addr_t addr = rx->buf_dma + rx->offset;
- size_t length = rx->buf_len - rx->offset;
- struct cppi_descriptor *bd, *tail;
- unsigned n_bds;
- unsigned i;
- void __iomem *tibase = musb->ctrl_base;
- int is_rndis = 0;
- struct cppi_rx_stateram __iomem *rx_ram = rx->state_ram;
- struct cppi_descriptor *d;
-
- if (onepacket) {
- /* almost every USB driver, host or peripheral side */
- n_bds = 1;
-
- /* maybe apply the heuristic above */
- if (cppi_rx_rndis
- && is_peripheral_active(musb)
- && length > maxpacket
- && (length & ~0xffff) == 0
- && (length & 0x0fff) != 0
- && (length & (maxpacket - 1)) == 0) {
- maxpacket = length;
- is_rndis = 1;
- }
- } else {
- /* virtually nothing except mass storage class */
- if (length > 0xffff) {
- n_bds = 0xffff / maxpacket;
- length = n_bds * maxpacket;
- } else {
- n_bds = DIV_ROUND_UP(length, maxpacket);
- }
- if (n_bds == 1)
- onepacket = 1;
- else
- n_bds = min(n_bds, (unsigned) NUM_RXCHAN_BD);
- }
-
- /* In host mode, autorequest logic can generate some IN tokens; it's
- * tricky since we can't leave REQPKT set in RXCSR after the transfer
- * finishes. So: multipacket transfers involve two or more segments.
- * And always at least two IRQs ... RNDIS mode is not an option.
- */
- if (is_host_active(musb))
- n_bds = cppi_autoreq_update(rx, tibase, onepacket, n_bds);
-
- cppi_rndis_update(rx, 1, musb->ctrl_base, is_rndis);
-
- length = min(n_bds * maxpacket, length);
-
- musb_dbg(musb, "RX DMA%d seg, maxp %d %s bds %d (cnt %d) "
- "dma 0x%llx len %u %u/%u",
- rx->index, maxpacket,
- onepacket
- ? (is_rndis ? "rndis" : "onepacket")
- : "multipacket",
- n_bds,
- musb_readl(tibase,
- DAVINCI_RXCPPI_BUFCNT0_REG + (rx->index * 4))
- & 0xffff,
- (unsigned long long)addr, length,
- rx->channel.actual_len, rx->buf_len);
-
- /* only queue one segment at a time, since the hardware prevents
- * correct queue shutdown after unexpected short packets
- */
- bd = cppi_bd_alloc(rx);
- rx->head = bd;
-
- /* Build BDs for all packets in this segment */
- for (i = 0, tail = NULL; bd && i < n_bds; i++, tail = bd) {
- u32 bd_len;
-
- if (i) {
- bd = cppi_bd_alloc(rx);
- if (!bd)
- break;
- tail->next = bd;
- tail->hw_next = bd->dma;
- }
- bd->hw_next = 0;
-
- /* all but the last packet will be maxpacket size */
- if (maxpacket < length)
- bd_len = maxpacket;
- else
- bd_len = length;
-
- bd->hw_bufp = addr;
- addr += bd_len;
- rx->offset += bd_len;
-
- bd->hw_off_len = (0 /*offset*/ << 16) + bd_len;
- bd->buflen = bd_len;
-
- bd->hw_options = CPPI_OWN_SET | (i == 0 ? length : 0);
- length -= bd_len;
- }
-
- /* we always expect at least one reusable BD! */
- if (!tail) {
- WARNING("rx dma%d -- no BDs? need %d\n", rx->index, n_bds);
- return;
- } else if (i < n_bds)
- WARNING("rx dma%d -- only %d of %d BDs\n", rx->index, i, n_bds);
-
- tail->next = NULL;
- tail->hw_next = 0;
-
- bd = rx->head;
- rx->tail = tail;
-
- /* short reads and other faults should terminate this entire
- * dma segment. we want one "dma packet" per dma segment, not
- * one per USB packet, terminating the whole queue at once...
- * NOTE that current hardware seems to ignore SOP and EOP.
- */
- bd->hw_options |= CPPI_SOP_SET;
- tail->hw_options |= CPPI_EOP_SET;
-
- for (d = rx->head; d; d = d->next)
- cppi_dump_rxbd("S", d);
-
- /* in case the preceding transfer left some state... */
- tail = rx->last_processed;
- if (tail) {
- tail->next = bd;
- tail->hw_next = bd->dma;
- }
-
- core_rxirq_enable(tibase, rx->index + 1);
-
- /* BDs live in DMA-coherent memory, but writes might be pending */
- cpu_drain_writebuffer();
-
- /* REVISIT specs say to write this AFTER the BUFCNT register
- * below ... but that loses badly.
- */
- musb_writel(&rx_ram->rx_head, 0, bd->dma);
-
- /* bufferCount must be at least 3, and zeroes on completion
- * unless it underflows below zero, or stops at two, or keeps
- * growing ... grr.
- */
- i = musb_readl(tibase,
- DAVINCI_RXCPPI_BUFCNT0_REG + (rx->index * 4))
- & 0xffff;
-
- if (!i)
- musb_writel(tibase,
- DAVINCI_RXCPPI_BUFCNT0_REG + (rx->index * 4),
- n_bds + 2);
- else if (n_bds > (i - 3))
- musb_writel(tibase,
- DAVINCI_RXCPPI_BUFCNT0_REG + (rx->index * 4),
- n_bds - (i - 3));
-
- i = musb_readl(tibase,
- DAVINCI_RXCPPI_BUFCNT0_REG + (rx->index * 4))
- & 0xffff;
- if (i < (2 + n_bds)) {
- musb_dbg(musb, "bufcnt%d underrun - %d (for %d)",
- rx->index, i, n_bds);
- musb_writel(tibase,
- DAVINCI_RXCPPI_BUFCNT0_REG + (rx->index * 4),
- n_bds + 2);
- }
-
- cppi_dump_rx(4, rx, "/S");
-}
-
-/**
- * cppi_channel_program - program channel for data transfer
- * @ch: the channel
- * @maxpacket: max packet size
- * @mode: For RX, 1 unless the usb protocol driver promised to treat
- * all short reads as errors and kick in high level fault recovery.
- * For TX, ignored because of RNDIS mode races/glitches.
- * @dma_addr: dma address of buffer
- * @len: length of buffer
- * Context: controller irqlocked
- */
-static int cppi_channel_program(struct dma_channel *ch,
- u16 maxpacket, u8 mode,
- dma_addr_t dma_addr, u32 len)
-{
- struct cppi_channel *cppi_ch;
- struct cppi *controller;
- struct musb *musb;
-
- cppi_ch = container_of(ch, struct cppi_channel, channel);
- controller = cppi_ch->controller;
- musb = controller->controller.musb;
-
- switch (ch->status) {
- case MUSB_DMA_STATUS_BUS_ABORT:
- case MUSB_DMA_STATUS_CORE_ABORT:
- /* fault irq handler should have handled cleanup */
- WARNING("%cX DMA%d not cleaned up after abort!\n",
- cppi_ch->transmit ? 'T' : 'R',
- cppi_ch->index);
- /* WARN_ON(1); */
- break;
- case MUSB_DMA_STATUS_BUSY:
- WARNING("program active channel? %cX DMA%d\n",
- cppi_ch->transmit ? 'T' : 'R',
- cppi_ch->index);
- /* WARN_ON(1); */
- break;
- case MUSB_DMA_STATUS_UNKNOWN:
- musb_dbg(musb, "%cX DMA%d not allocated!",
- cppi_ch->transmit ? 'T' : 'R',
- cppi_ch->index);
- fallthrough;
- case MUSB_DMA_STATUS_FREE:
- break;
- }
-
- ch->status = MUSB_DMA_STATUS_BUSY;
-
- /* set transfer parameters, then queue up its first segment */
- cppi_ch->buf_dma = dma_addr;
- cppi_ch->offset = 0;
- cppi_ch->maxpacket = maxpacket;
- cppi_ch->buf_len = len;
- cppi_ch->channel.actual_len = 0;
-
- /* TX channel? or RX? */
- if (cppi_ch->transmit)
- cppi_next_tx_segment(musb, cppi_ch);
- else
- cppi_next_rx_segment(musb, cppi_ch, mode);
-
- return true;
-}
-
-static bool cppi_rx_scan(struct cppi *cppi, unsigned ch)
-{
- struct cppi_channel *rx = &cppi->rx[ch];
- struct cppi_rx_stateram __iomem *state = rx->state_ram;
- struct cppi_descriptor *bd;
- struct cppi_descriptor *last = rx->last_processed;
- bool completed = false;
- bool acked = false;
- int i;
- dma_addr_t safe2ack;
- void __iomem *regs = rx->hw_ep->regs;
- struct musb *musb = cppi->controller.musb;
-
- cppi_dump_rx(6, rx, "/K");
-
- bd = last ? last->next : rx->head;
- if (!bd)
- return false;
-
- /* run through all completed BDs */
- for (i = 0, safe2ack = musb_readl(&state->rx_complete, 0);
- (safe2ack || completed) && bd && i < NUM_RXCHAN_BD;
- i++, bd = bd->next) {
- u16 len;
-
- /* catch latest BD writes from CPPI */
- rmb();
- if (!completed && (bd->hw_options & CPPI_OWN_SET))
- break;
-
- musb_dbg(musb, "C/RXBD %llx: nxt %08x buf %08x "
- "off.len %08x opt.len %08x (%d)",
- (unsigned long long)bd->dma, bd->hw_next, bd->hw_bufp,
- bd->hw_off_len, bd->hw_options,
- rx->channel.actual_len);
-
- /* actual packet received length */
- if ((bd->hw_options & CPPI_SOP_SET) && !completed)
- len = bd->hw_off_len & CPPI_RECV_PKTLEN_MASK;
- else
- len = 0;
-
- if (bd->hw_options & CPPI_EOQ_MASK)
- completed = true;
-
- if (!completed && len < bd->buflen) {
- /* NOTE: when we get a short packet, RXCSR_H_REQPKT
- * must have been cleared, and no more DMA packets may
- * active be in the queue... TI docs didn't say, but
- * CPPI ignores those BDs even though OWN is still set.
- */
- completed = true;
- musb_dbg(musb, "rx short %d/%d (%d)",
- len, bd->buflen,
- rx->channel.actual_len);
- }
-
- /* If we got here, we expect to ack at least one BD; meanwhile
- * CPPI may completing other BDs while we scan this list...
- *
- * RACE: we can notice OWN cleared before CPPI raises the
- * matching irq by writing that BD as the completion pointer.
- * In such cases, stop scanning and wait for the irq, avoiding
- * lost acks and states where BD ownership is unclear.
- */
- if (bd->dma == safe2ack) {
- musb_writel(&state->rx_complete, 0, safe2ack);
- safe2ack = musb_readl(&state->rx_complete, 0);
- acked = true;
- if (bd->dma == safe2ack)
- safe2ack = 0;
- }
-
- rx->channel.actual_len += len;
-
- cppi_bd_free(rx, last);
- last = bd;
-
- /* stop scanning on end-of-segment */
- if (bd->hw_next == 0)
- completed = true;
- }
- rx->last_processed = last;
-
- /* dma abort, lost ack, or ... */
- if (!acked && last) {
- int csr;
-
- if (safe2ack == 0 || safe2ack == rx->last_processed->dma)
- musb_writel(&state->rx_complete, 0, safe2ack);
- if (safe2ack == 0) {
- cppi_bd_free(rx, last);
- rx->last_processed = NULL;
-
- /* if we land here on the host side, H_REQPKT will
- * be clear and we need to restart the queue...
- */
- WARN_ON(rx->head);
- }
- musb_ep_select(cppi->mregs, rx->index + 1);
- csr = musb_readw(regs, MUSB_RXCSR);
- if (csr & MUSB_RXCSR_DMAENAB) {
- musb_dbg(musb, "list%d %p/%p, last %llx%s, csr %04x",
- rx->index,
- rx->head, rx->tail,
- rx->last_processed
- ? (unsigned long long)
- rx->last_processed->dma
- : 0,
- completed ? ", completed" : "",
- csr);
- cppi_dump_rxq(4, "/what?", rx);
- }
- }
- if (!completed) {
- int csr;
-
- rx->head = bd;
-
- /* REVISIT seems like "autoreq all but EOP" doesn't...
- * setting it here "should" be racey, but seems to work
- */
- csr = musb_readw(rx->hw_ep->regs, MUSB_RXCSR);
- if (is_host_active(cppi->controller.musb)
- && bd
- && !(csr & MUSB_RXCSR_H_REQPKT)) {
- csr |= MUSB_RXCSR_H_REQPKT;
- musb_writew(regs, MUSB_RXCSR,
- MUSB_RXCSR_H_WZC_BITS | csr);
- csr = musb_readw(rx->hw_ep->regs, MUSB_RXCSR);
- }
- } else {
- rx->head = NULL;
- rx->tail = NULL;
- }
-
- cppi_dump_rx(6, rx, completed ? "/completed" : "/cleaned");
- return completed;
-}
-
-irqreturn_t cppi_interrupt(int irq, void *dev_id)
-{
- struct musb *musb = dev_id;
- struct cppi *cppi;
- void __iomem *tibase;
- struct musb_hw_ep *hw_ep = NULL;
- u32 rx, tx;
- int i, index;
- unsigned long flags;
-
- cppi = container_of(musb->dma_controller, struct cppi, controller);
- if (cppi->irq)
- spin_lock_irqsave(&musb->lock, flags);
-
- tibase = musb->ctrl_base;
-
- tx = musb_readl(tibase, DAVINCI_TXCPPI_MASKED_REG);
- rx = musb_readl(tibase, DAVINCI_RXCPPI_MASKED_REG);
-
- if (!tx && !rx) {
- if (cppi->irq)
- spin_unlock_irqrestore(&musb->lock, flags);
- return IRQ_NONE;
- }
-
- musb_dbg(musb, "CPPI IRQ Tx%x Rx%x", tx, rx);
-
- /* process TX channels */
- for (index = 0; tx; tx = tx >> 1, index++) {
- struct cppi_channel *tx_ch;
- struct cppi_tx_stateram __iomem *tx_ram;
- bool completed = false;
- struct cppi_descriptor *bd;
-
- if (!(tx & 1))
- continue;
-
- tx_ch = cppi->tx + index;
- tx_ram = tx_ch->state_ram;
-
- /* FIXME need a cppi_tx_scan() routine, which
- * can also be called from abort code
- */
-
- cppi_dump_tx(5, tx_ch, "/E");
-
- bd = tx_ch->head;
-
- /*
- * If Head is null then this could mean that a abort interrupt
- * that needs to be acknowledged.
- */
- if (NULL == bd) {
- musb_dbg(musb, "null BD");
- musb_writel(&tx_ram->tx_complete, 0, 0);
- continue;
- }
-
- /* run through all completed BDs */
- for (i = 0; !completed && bd && i < NUM_TXCHAN_BD;
- i++, bd = bd->next) {
- u16 len;
-
- /* catch latest BD writes from CPPI */
- rmb();
- if (bd->hw_options & CPPI_OWN_SET)
- break;
-
- musb_dbg(musb, "C/TXBD %p n %x b %x off %x opt %x",
- bd, bd->hw_next, bd->hw_bufp,
- bd->hw_off_len, bd->hw_options);
-
- len = bd->hw_off_len & CPPI_BUFFER_LEN_MASK;
- tx_ch->channel.actual_len += len;
-
- tx_ch->last_processed = bd;
-
- /* write completion register to acknowledge
- * processing of completed BDs, and possibly
- * release the IRQ; EOQ might not be set ...
- *
- * REVISIT use the same ack strategy as rx
- *
- * REVISIT have observed bit 18 set; huh??
- */
- /* if ((bd->hw_options & CPPI_EOQ_MASK)) */
- musb_writel(&tx_ram->tx_complete, 0, bd->dma);
-
- /* stop scanning on end-of-segment */
- if (bd->hw_next == 0)
- completed = true;
- }
-
- /* on end of segment, maybe go to next one */
- if (completed) {
- /* cppi_dump_tx(4, tx_ch, "/complete"); */
-
- /* transfer more, or report completion */
- if (tx_ch->offset >= tx_ch->buf_len) {
- tx_ch->head = NULL;
- tx_ch->tail = NULL;
- tx_ch->channel.status = MUSB_DMA_STATUS_FREE;
-
- hw_ep = tx_ch->hw_ep;
-
- musb_dma_completion(musb, index + 1, 1);
-
- } else {
- /* Bigger transfer than we could fit in
- * that first batch of descriptors...
- */
- cppi_next_tx_segment(musb, tx_ch);
- }
- } else
- tx_ch->head = bd;
- }
-
- /* Start processing the RX block */
- for (index = 0; rx; rx = rx >> 1, index++) {
-
- if (rx & 1) {
- struct cppi_channel *rx_ch;
-
- rx_ch = cppi->rx + index;
-
- /* let incomplete dma segments finish */
- if (!cppi_rx_scan(cppi, index))
- continue;
-
- /* start another dma segment if needed */
- if (rx_ch->channel.actual_len != rx_ch->buf_len
- && rx_ch->channel.actual_len
- == rx_ch->offset) {
- cppi_next_rx_segment(musb, rx_ch, 1);
- continue;
- }
-
- /* all segments completed! */
- rx_ch->channel.status = MUSB_DMA_STATUS_FREE;
-
- hw_ep = rx_ch->hw_ep;
-
- core_rxirq_disable(tibase, index + 1);
- musb_dma_completion(musb, index + 1, 0);
- }
- }
-
- /* write to CPPI EOI register to re-enable interrupts */
- musb_writel(tibase, DAVINCI_CPPI_EOI_REG, 0);
-
- if (cppi->irq)
- spin_unlock_irqrestore(&musb->lock, flags);
-
- return IRQ_HANDLED;
-}
-EXPORT_SYMBOL_GPL(cppi_interrupt);
-
-/* Instantiate a software object representing a DMA controller. */
-struct dma_controller *
-cppi_dma_controller_create(struct musb *musb, void __iomem *mregs)
-{
- struct cppi *controller;
- struct device *dev = musb->controller;
- struct platform_device *pdev = to_platform_device(dev);
- int irq = platform_get_irq_byname(pdev, "dma");
-
- controller = kzalloc(sizeof *controller, GFP_KERNEL);
- if (!controller)
- return NULL;
-
- controller->mregs = mregs;
- controller->tibase = mregs - DAVINCI_BASE_OFFSET;
-
- controller->controller.musb = musb;
- controller->controller.channel_alloc = cppi_channel_allocate;
- controller->controller.channel_release = cppi_channel_release;
- controller->controller.channel_program = cppi_channel_program;
- controller->controller.channel_abort = cppi_channel_abort;
-
- /* NOTE: allocating from on-chip SRAM would give the least
- * contention for memory access, if that ever matters here.
- */
-
- /* setup BufferPool */
- controller->pool = dma_pool_create("cppi",
- controller->controller.musb->controller,
- sizeof(struct cppi_descriptor),
- CPPI_DESCRIPTOR_ALIGN, 0);
- if (!controller->pool) {
- kfree(controller);
- return NULL;
- }
-
- if (irq > 0) {
- if (request_irq(irq, cppi_interrupt, 0, "cppi-dma", musb)) {
- dev_err(dev, "request_irq %d failed!\n", irq);
- musb_dma_controller_destroy(&controller->controller);
- return NULL;
- }
- controller->irq = irq;
- }
-
- cppi_controller_start(controller);
- return &controller->controller;
-}
-EXPORT_SYMBOL_GPL(cppi_dma_controller_create);
-
-/*
- * Destroy a previously-instantiated DMA controller.
- */
-void cppi_dma_controller_destroy(struct dma_controller *c)
-{
- struct cppi *cppi;
-
- cppi = container_of(c, struct cppi, controller);
-
- cppi_controller_stop(cppi);
-
- if (cppi->irq)
- free_irq(cppi->irq, cppi->controller.musb);
-
- /* assert: caller stopped the controller first */
- dma_pool_destroy(cppi->pool);
-
- kfree(cppi);
-}
-EXPORT_SYMBOL_GPL(cppi_dma_controller_destroy);
-
-/*
- * Context: controller irqlocked, endpoint selected
- */
-static int cppi_channel_abort(struct dma_channel *channel)
-{
- struct cppi_channel *cppi_ch;
- struct cppi *controller;
- void __iomem *mbase;
- void __iomem *tibase;
- void __iomem *regs;
- u32 value;
- struct cppi_descriptor *queue;
-
- cppi_ch = container_of(channel, struct cppi_channel, channel);
-
- controller = cppi_ch->controller;
-
- switch (channel->status) {
- case MUSB_DMA_STATUS_BUS_ABORT:
- case MUSB_DMA_STATUS_CORE_ABORT:
- /* from RX or TX fault irq handler */
- case MUSB_DMA_STATUS_BUSY:
- /* the hardware needs shutting down */
- regs = cppi_ch->hw_ep->regs;
- break;
- case MUSB_DMA_STATUS_UNKNOWN:
- case MUSB_DMA_STATUS_FREE:
- return 0;
- default:
- return -EINVAL;
- }
-
- if (!cppi_ch->transmit && cppi_ch->head)
- cppi_dump_rxq(3, "/abort", cppi_ch);
-
- mbase = controller->mregs;
- tibase = controller->tibase;
-
- queue = cppi_ch->head;
- cppi_ch->head = NULL;
- cppi_ch->tail = NULL;
-
- /* REVISIT should rely on caller having done this,
- * and caller should rely on us not changing it.
- * peripheral code is safe ... check host too.
- */
- musb_ep_select(mbase, cppi_ch->index + 1);
-
- if (cppi_ch->transmit) {
- struct cppi_tx_stateram __iomem *tx_ram;
- /* REVISIT put timeouts on these controller handshakes */
-
- cppi_dump_tx(6, cppi_ch, " (teardown)");
-
- /* teardown DMA engine then usb core */
- do {
- value = musb_readl(tibase, DAVINCI_TXCPPI_TEAR_REG);
- } while (!(value & CPPI_TEAR_READY));
- musb_writel(tibase, DAVINCI_TXCPPI_TEAR_REG, cppi_ch->index);
-
- tx_ram = cppi_ch->state_ram;
- do {
- value = musb_readl(&tx_ram->tx_complete, 0);
- } while (0xFFFFFFFC != value);
-
- /* FIXME clean up the transfer state ... here?
- * the completion routine should get called with
- * an appropriate status code.
- */
-
- value = musb_readw(regs, MUSB_TXCSR);
- value &= ~MUSB_TXCSR_DMAENAB;
- value |= MUSB_TXCSR_FLUSHFIFO;
- musb_writew(regs, MUSB_TXCSR, value);
- musb_writew(regs, MUSB_TXCSR, value);
-
- /*
- * 1. Write to completion Ptr value 0x1(bit 0 set)
- * (write back mode)
- * 2. Wait for abort interrupt and then put the channel in
- * compare mode by writing 1 to the tx_complete register.
- */
- cppi_reset_tx(tx_ram, 1);
- cppi_ch->head = NULL;
- musb_writel(&tx_ram->tx_complete, 0, 1);
- cppi_dump_tx(5, cppi_ch, " (done teardown)");
-
- /* REVISIT tx side _should_ clean up the same way
- * as the RX side ... this does no cleanup at all!
- */
-
- } else /* RX */ {
- u16 csr;
-
- /* NOTE: docs don't guarantee any of this works ... we
- * expect that if the usb core stops telling the cppi core
- * to pull more data from it, then it'll be safe to flush
- * current RX DMA state iff any pending fifo transfer is done.
- */
-
- core_rxirq_disable(tibase, cppi_ch->index + 1);
-
- /* for host, ensure ReqPkt is never set again */
- if (is_host_active(cppi_ch->controller->controller.musb)) {
- value = musb_readl(tibase, DAVINCI_AUTOREQ_REG);
- value &= ~((0x3) << (cppi_ch->index * 2));
- musb_writel(tibase, DAVINCI_AUTOREQ_REG, value);
- }
-
- csr = musb_readw(regs, MUSB_RXCSR);
-
- /* for host, clear (just) ReqPkt at end of current packet(s) */
- if (is_host_active(cppi_ch->controller->controller.musb)) {
- csr |= MUSB_RXCSR_H_WZC_BITS;
- csr &= ~MUSB_RXCSR_H_REQPKT;
- } else
- csr |= MUSB_RXCSR_P_WZC_BITS;
-
- /* clear dma enable */
- csr &= ~(MUSB_RXCSR_DMAENAB);
- musb_writew(regs, MUSB_RXCSR, csr);
- csr = musb_readw(regs, MUSB_RXCSR);
-
- /* Quiesce: wait for current dma to finish (if not cleanup).
- * We can't use bit zero of stateram->rx_sop, since that
- * refers to an entire "DMA packet" not just emptying the
- * current fifo. Most segments need multiple usb packets.
- */
- if (channel->status == MUSB_DMA_STATUS_BUSY)
- udelay(50);
-
- /* scan the current list, reporting any data that was
- * transferred and acking any IRQ
- */
- cppi_rx_scan(controller, cppi_ch->index);
-
- /* clobber the existing state once it's idle
- *
- * NOTE: arguably, we should also wait for all the other
- * RX channels to quiesce (how??) and then temporarily
- * disable RXCPPI_CTRL_REG ... but it seems that we can
- * rely on the controller restarting from state ram, with
- * only RXCPPI_BUFCNT state being bogus. BUFCNT will
- * correct itself after the next DMA transfer though.
- *
- * REVISIT does using rndis mode change that?
- */
- cppi_reset_rx(cppi_ch->state_ram);
-
- /* next DMA request _should_ load cppi head ptr */
-
- /* ... we don't "free" that list, only mutate it in place. */
- cppi_dump_rx(5, cppi_ch, " (done abort)");
-
- /* clean up previously pending bds */
- cppi_bd_free(cppi_ch, cppi_ch->last_processed);
- cppi_ch->last_processed = NULL;
-
- while (queue) {
- struct cppi_descriptor *tmp = queue->next;
-
- cppi_bd_free(cppi_ch, queue);
- queue = tmp;
- }
- }
-
- channel->status = MUSB_DMA_STATUS_FREE;
- cppi_ch->buf_dma = 0;
- cppi_ch->offset = 0;
- cppi_ch->buf_len = 0;
- cppi_ch->maxpacket = 0;
- return 0;
-}
-
-/* TBD Queries:
- *
- * Power Management ... probably turn off cppi during suspend, restart;
- * check state ram? Clocking is presumably shared with usb core.
- */
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Copyright (C) 2005-2006 by Texas Instruments
- *
- * This file is part of the Inventra Controller Driver for Linux.
- */
-
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/sched.h>
-#include <linux/list.h>
-#include <linux/delay.h>
-#include <linux/clk.h>
-#include <linux/err.h>
-#include <linux/io.h>
-#include <linux/gpio/consumer.h>
-#include <linux/platform_device.h>
-#include <linux/dma-mapping.h>
-#include <linux/usb/usb_phy_generic.h>
-
-#include <mach/cputype.h>
-#include <mach/hardware.h>
-
-#include <asm/mach-types.h>
-
-#include "musb_core.h"
-
-#include "davinci.h"
-#include "cppi_dma.h"
-
-
-#define USB_PHY_CTRL IO_ADDRESS(USBPHY_CTL_PADDR)
-#define DM355_DEEPSLEEP IO_ADDRESS(DM355_DEEPSLEEP_PADDR)
-
-struct davinci_glue {
- struct device *dev;
- struct platform_device *musb;
- struct clk *clk;
- bool vbus_state;
- struct gpio_desc *vbus;
- struct work_struct vbus_work;
-};
-
-/* REVISIT (PM) we should be able to keep the PHY in low power mode most
- * of the time (24 MHZ oscillator and PLL off, etc) by setting POWER.D0
- * and, when in host mode, autosuspending idle root ports... PHYPLLON
- * (overriding SUSPENDM?) then likely needs to stay off.
- */
-
-static inline void phy_on(void)
-{
- u32 phy_ctrl = __raw_readl(USB_PHY_CTRL);
-
- /* power everything up; start the on-chip PHY and its PLL */
- phy_ctrl &= ~(USBPHY_OSCPDWN | USBPHY_OTGPDWN | USBPHY_PHYPDWN);
- phy_ctrl |= USBPHY_SESNDEN | USBPHY_VBDTCTEN | USBPHY_PHYPLLON;
- __raw_writel(phy_ctrl, USB_PHY_CTRL);
-
- /* wait for PLL to lock before proceeding */
- while ((__raw_readl(USB_PHY_CTRL) & USBPHY_PHYCLKGD) == 0)
- cpu_relax();
-}
-
-static inline void phy_off(void)
-{
- u32 phy_ctrl = __raw_readl(USB_PHY_CTRL);
-
- /* powerdown the on-chip PHY, its PLL, and the OTG block */
- phy_ctrl &= ~(USBPHY_SESNDEN | USBPHY_VBDTCTEN | USBPHY_PHYPLLON);
- phy_ctrl |= USBPHY_OSCPDWN | USBPHY_OTGPDWN | USBPHY_PHYPDWN;
- __raw_writel(phy_ctrl, USB_PHY_CTRL);
-}
-
-static int dma_off = 1;
-
-static void davinci_musb_enable(struct musb *musb)
-{
- u32 tmp, old, val;
-
- /* workaround: setup irqs through both register sets */
- tmp = (musb->epmask & DAVINCI_USB_TX_ENDPTS_MASK)
- << DAVINCI_USB_TXINT_SHIFT;
- musb_writel(musb->ctrl_base, DAVINCI_USB_INT_MASK_SET_REG, tmp);
- old = tmp;
- tmp = (musb->epmask & (0xfffe & DAVINCI_USB_RX_ENDPTS_MASK))
- << DAVINCI_USB_RXINT_SHIFT;
- musb_writel(musb->ctrl_base, DAVINCI_USB_INT_MASK_SET_REG, tmp);
- tmp |= old;
-
- val = ~MUSB_INTR_SOF;
- tmp |= ((val & 0x01ff) << DAVINCI_USB_USBINT_SHIFT);
- musb_writel(musb->ctrl_base, DAVINCI_USB_INT_MASK_SET_REG, tmp);
-
- if (is_dma_capable() && !dma_off)
- printk(KERN_WARNING "%s %s: dma not reactivated\n",
- __FILE__, __func__);
- else
- dma_off = 0;
-
- /* force a DRVVBUS irq so we can start polling for ID change */
- musb_writel(musb->ctrl_base, DAVINCI_USB_INT_SET_REG,
- DAVINCI_INTR_DRVVBUS << DAVINCI_USB_USBINT_SHIFT);
-}
-
-/*
- * Disable the HDRC and flush interrupts
- */
-static void davinci_musb_disable(struct musb *musb)
-{
- /* because we don't set CTRLR.UINT, "important" to:
- * - not read/write INTRUSB/INTRUSBE
- * - (except during initial setup, as workaround)
- * - use INTSETR/INTCLRR instead
- */
- musb_writel(musb->ctrl_base, DAVINCI_USB_INT_MASK_CLR_REG,
- DAVINCI_USB_USBINT_MASK
- | DAVINCI_USB_TXINT_MASK
- | DAVINCI_USB_RXINT_MASK);
- musb_writel(musb->ctrl_base, DAVINCI_USB_EOI_REG, 0);
-
- if (is_dma_capable() && !dma_off)
- WARNING("dma still active\n");
-}
-
-
-#define portstate(stmt) stmt
-
-/*
- * VBUS SWITCHING IS BOARD-SPECIFIC ... at least for the DM6446 EVM,
- * which doesn't wire DRVVBUS to the FET that switches it. Unclear
- * if that's a problem with the DM6446 chip or just with that board.
- *
- * In either case, the DM355 EVM automates DRVVBUS the normal way,
- * when J10 is out, and TI documents it as handling OTG.
- */
-
-/* I2C operations are always synchronous, and require a task context.
- * With unloaded systems, using the shared workqueue seems to suffice
- * to satisfy the 100msec A_WAIT_VRISE timeout...
- */
-static void evm_deferred_drvvbus(struct work_struct *work)
-{
- struct davinci_glue *glue = container_of(work, struct davinci_glue,
- vbus_work);
-
- gpiod_set_value_cansleep(glue->vbus, glue->vbus_state);
- glue->vbus_state = !glue->vbus_state;
-}
-
-static void davinci_musb_source_power(struct musb *musb, int is_on,
- int immediate)
-{
- struct davinci_glue *glue = dev_get_drvdata(musb->controller->parent);
-
- /* This GPIO handling is entirely optional */
- if (!glue->vbus)
- return;
-
- if (is_on)
- is_on = 1;
-
- if (glue->vbus_state == is_on)
- return;
- /* 0/1 vs "-1 == unknown/init" */
- glue->vbus_state = !is_on;
-
- if (machine_is_davinci_evm()) {
- if (immediate)
- gpiod_set_value_cansleep(glue->vbus, glue->vbus_state);
- else
- schedule_work(&glue->vbus_work);
- }
- if (immediate)
- glue->vbus_state = is_on;
-}
-
-static void davinci_musb_set_vbus(struct musb *musb, int is_on)
-{
- WARN_ON(is_on && is_peripheral_active(musb));
- davinci_musb_source_power(musb, is_on, 0);
-}
-
-
-#define POLL_SECONDS 2
-
-static void otg_timer(struct timer_list *t)
-{
- struct musb *musb = from_timer(musb, t, dev_timer);
- void __iomem *mregs = musb->mregs;
- u8 devctl;
- unsigned long flags;
-
- /* We poll because DaVinci's won't expose several OTG-critical
- * status change events (from the transceiver) otherwise.
- */
- devctl = musb_readb(mregs, MUSB_DEVCTL);
- dev_dbg(musb->controller, "poll devctl %02x (%s)\n", devctl,
- usb_otg_state_string(musb->xceiv->otg->state));
-
- spin_lock_irqsave(&musb->lock, flags);
- switch (musb->xceiv->otg->state) {
- case OTG_STATE_A_WAIT_VFALL:
- /* Wait till VBUS falls below SessionEnd (~0.2V); the 1.3 RTL
- * seems to mis-handle session "start" otherwise (or in our
- * case "recover"), in routine "VBUS was valid by the time
- * VBUSERR got reported during enumeration" cases.
- */
- if (devctl & MUSB_DEVCTL_VBUS) {
- mod_timer(&musb->dev_timer, jiffies + POLL_SECONDS * HZ);
- break;
- }
- musb->xceiv->otg->state = OTG_STATE_A_WAIT_VRISE;
- musb_writel(musb->ctrl_base, DAVINCI_USB_INT_SET_REG,
- MUSB_INTR_VBUSERROR << DAVINCI_USB_USBINT_SHIFT);
- break;
- case OTG_STATE_B_IDLE:
- /*
- * There's no ID-changed IRQ, so we have no good way to tell
- * when to switch to the A-Default state machine (by setting
- * the DEVCTL.SESSION flag).
- *
- * Workaround: whenever we're in B_IDLE, try setting the
- * session flag every few seconds. If it works, ID was
- * grounded and we're now in the A-Default state machine.
- *
- * NOTE setting the session flag is _supposed_ to trigger
- * SRP, but clearly it doesn't.
- */
- musb_writeb(mregs, MUSB_DEVCTL,
- devctl | MUSB_DEVCTL_SESSION);
- devctl = musb_readb(mregs, MUSB_DEVCTL);
- if (devctl & MUSB_DEVCTL_BDEVICE)
- mod_timer(&musb->dev_timer, jiffies + POLL_SECONDS * HZ);
- else
- musb->xceiv->otg->state = OTG_STATE_A_IDLE;
- break;
- default:
- break;
- }
- spin_unlock_irqrestore(&musb->lock, flags);
-}
-
-static irqreturn_t davinci_musb_interrupt(int irq, void *__hci)
-{
- unsigned long flags;
- irqreturn_t retval = IRQ_NONE;
- struct musb *musb = __hci;
- struct usb_otg *otg = musb->xceiv->otg;
- void __iomem *tibase = musb->ctrl_base;
- struct cppi *cppi;
- u32 tmp;
-
- spin_lock_irqsave(&musb->lock, flags);
-
- /* NOTE: DaVinci shadows the Mentor IRQs. Don't manage them through
- * the Mentor registers (except for setup), use the TI ones and EOI.
- *
- * Docs describe irq "vector" registers associated with the CPPI and
- * USB EOI registers. These hold a bitmask corresponding to the
- * current IRQ, not an irq handler address. Would using those bits
- * resolve some of the races observed in this dispatch code??
- */
-
- /* CPPI interrupts share the same IRQ line, but have their own
- * mask, state, "vector", and EOI registers.
- */
- cppi = container_of(musb->dma_controller, struct cppi, controller);
- if (is_cppi_enabled(musb) && musb->dma_controller && !cppi->irq)
- retval = cppi_interrupt(irq, __hci);
-
- /* ack and handle non-CPPI interrupts */
- tmp = musb_readl(tibase, DAVINCI_USB_INT_SRC_MASKED_REG);
- musb_writel(tibase, DAVINCI_USB_INT_SRC_CLR_REG, tmp);
- dev_dbg(musb->controller, "IRQ %08x\n", tmp);
-
- musb->int_rx = (tmp & DAVINCI_USB_RXINT_MASK)
- >> DAVINCI_USB_RXINT_SHIFT;
- musb->int_tx = (tmp & DAVINCI_USB_TXINT_MASK)
- >> DAVINCI_USB_TXINT_SHIFT;
- musb->int_usb = (tmp & DAVINCI_USB_USBINT_MASK)
- >> DAVINCI_USB_USBINT_SHIFT;
-
- /* DRVVBUS irqs are the only proxy we have (a very poor one!) for
- * DaVinci's missing ID change IRQ. We need an ID change IRQ to
- * switch appropriately between halves of the OTG state machine.
- * Managing DEVCTL.SESSION per Mentor docs requires we know its
- * value, but DEVCTL.BDEVICE is invalid without DEVCTL.SESSION set.
- * Also, DRVVBUS pulses for SRP (but not at 5V) ...
- */
- if (tmp & (DAVINCI_INTR_DRVVBUS << DAVINCI_USB_USBINT_SHIFT)) {
- int drvvbus = musb_readl(tibase, DAVINCI_USB_STAT_REG);
- void __iomem *mregs = musb->mregs;
- u8 devctl = musb_readb(mregs, MUSB_DEVCTL);
- int err = musb->int_usb & MUSB_INTR_VBUSERROR;
-
- err = musb->int_usb & MUSB_INTR_VBUSERROR;
- if (err) {
- /* The Mentor core doesn't debounce VBUS as needed
- * to cope with device connect current spikes. This
- * means it's not uncommon for bus-powered devices
- * to get VBUS errors during enumeration.
- *
- * This is a workaround, but newer RTL from Mentor
- * seems to allow a better one: "re"starting sessions
- * without waiting (on EVM, a **long** time) for VBUS
- * to stop registering in devctl.
- */
- musb->int_usb &= ~MUSB_INTR_VBUSERROR;
- musb->xceiv->otg->state = OTG_STATE_A_WAIT_VFALL;
- mod_timer(&musb->dev_timer, jiffies + POLL_SECONDS * HZ);
- WARNING("VBUS error workaround (delay coming)\n");
- } else if (drvvbus) {
- MUSB_HST_MODE(musb);
- musb->xceiv->otg->state = OTG_STATE_A_WAIT_VRISE;
- portstate(musb->port1_status |= USB_PORT_STAT_POWER);
- del_timer(&musb->dev_timer);
- } else {
- musb->is_active = 0;
- MUSB_DEV_MODE(musb);
- musb->xceiv->otg->state = OTG_STATE_B_IDLE;
- portstate(musb->port1_status &= ~USB_PORT_STAT_POWER);
- }
-
- /* NOTE: this must complete poweron within 100 msec
- * (OTG_TIME_A_WAIT_VRISE) but we don't check for that.
- */
- davinci_musb_source_power(musb, drvvbus, 0);
- dev_dbg(musb->controller, "VBUS %s (%s)%s, devctl %02x\n",
- drvvbus ? "on" : "off",
- usb_otg_state_string(musb->xceiv->otg->state),
- err ? " ERROR" : "",
- devctl);
- retval = IRQ_HANDLED;
- }
-
- if (musb->int_tx || musb->int_rx || musb->int_usb)
- retval |= musb_interrupt(musb);
-
- /* irq stays asserted until EOI is written */
- musb_writel(tibase, DAVINCI_USB_EOI_REG, 0);
-
- /* poll for ID change */
- if (musb->xceiv->otg->state == OTG_STATE_B_IDLE)
- mod_timer(&musb->dev_timer, jiffies + POLL_SECONDS * HZ);
-
- spin_unlock_irqrestore(&musb->lock, flags);
-
- return retval;
-}
-
-static int davinci_musb_set_mode(struct musb *musb, u8 mode)
-{
- /* EVM can't do this (right?) */
- return -EIO;
-}
-
-static int davinci_musb_init(struct musb *musb)
-{
- void __iomem *tibase = musb->ctrl_base;
- u32 revision;
- int ret = -ENODEV;
-
- musb->xceiv = usb_get_phy(USB_PHY_TYPE_USB2);
- if (IS_ERR_OR_NULL(musb->xceiv)) {
- ret = -EPROBE_DEFER;
- goto unregister;
- }
-
- musb->mregs += DAVINCI_BASE_OFFSET;
-
- /* returns zero if e.g. not clocked */
- revision = musb_readl(tibase, DAVINCI_USB_VERSION_REG);
- if (revision == 0)
- goto fail;
-
- timer_setup(&musb->dev_timer, otg_timer, 0);
-
- davinci_musb_source_power(musb, 0, 1);
-
- /* dm355 EVM swaps D+/D- for signal integrity, and
- * is clocked from the main 24 MHz crystal.
- */
- if (machine_is_davinci_dm355_evm()) {
- u32 phy_ctrl = __raw_readl(USB_PHY_CTRL);
-
- phy_ctrl &= ~(3 << 9);
- phy_ctrl |= USBPHY_DATAPOL;
- __raw_writel(phy_ctrl, USB_PHY_CTRL);
- }
-
- /* On dm355, the default-A state machine needs DRVVBUS control.
- * If we won't be a host, there's no need to turn it on.
- */
- if (cpu_is_davinci_dm355()) {
- u32 deepsleep = __raw_readl(DM355_DEEPSLEEP);
-
- deepsleep &= ~DRVVBUS_FORCE;
- __raw_writel(deepsleep, DM355_DEEPSLEEP);
- }
-
- /* reset the controller */
- musb_writel(tibase, DAVINCI_USB_CTRL_REG, 0x1);
-
- /* start the on-chip PHY and its PLL */
- phy_on();
-
- msleep(5);
-
- /* NOTE: irqs are in mixed mode, not bypass to pure-musb */
- pr_debug("DaVinci OTG revision %08x phy %03x control %02x\n",
- revision, __raw_readl(USB_PHY_CTRL),
- musb_readb(tibase, DAVINCI_USB_CTRL_REG));
-
- musb->isr = davinci_musb_interrupt;
- return 0;
-
-fail:
- usb_put_phy(musb->xceiv);
-unregister:
- usb_phy_generic_unregister();
- return ret;
-}
-
-static int davinci_musb_exit(struct musb *musb)
-{
- int maxdelay = 30;
- u8 devctl, warn = 0;
-
- del_timer_sync(&musb->dev_timer);
-
- /* force VBUS off */
- if (cpu_is_davinci_dm355()) {
- u32 deepsleep = __raw_readl(DM355_DEEPSLEEP);
-
- deepsleep &= ~DRVVBUS_FORCE;
- deepsleep |= DRVVBUS_OVERRIDE;
- __raw_writel(deepsleep, DM355_DEEPSLEEP);
- }
-
- davinci_musb_source_power(musb, 0 /*off*/, 1);
-
- /*
- * delay, to avoid problems with module reload.
- * if there's no peripheral connected, this can take a
- * long time to fall, especially on EVM with huge C133.
- */
- do {
- devctl = musb_readb(musb->mregs, MUSB_DEVCTL);
- if (!(devctl & MUSB_DEVCTL_VBUS))
- break;
- if ((devctl & MUSB_DEVCTL_VBUS) != warn) {
- warn = devctl & MUSB_DEVCTL_VBUS;
- dev_dbg(musb->controller, "VBUS %d\n",
- warn >> MUSB_DEVCTL_VBUS_SHIFT);
- }
- msleep(1000);
- maxdelay--;
- } while (maxdelay > 0);
-
- /* in OTG mode, another host might be connected */
- if (devctl & MUSB_DEVCTL_VBUS)
- dev_dbg(musb->controller, "VBUS off timeout (devctl %02x)\n", devctl);
-
- phy_off();
-
- usb_put_phy(musb->xceiv);
-
- return 0;
-}
-
-static const struct musb_platform_ops davinci_ops = {
- .quirks = MUSB_DMA_CPPI,
- .init = davinci_musb_init,
- .exit = davinci_musb_exit,
-
-#ifdef CONFIG_USB_TI_CPPI_DMA
- .dma_init = cppi_dma_controller_create,
- .dma_exit = cppi_dma_controller_destroy,
-#endif
- .enable = davinci_musb_enable,
- .disable = davinci_musb_disable,
-
- .set_mode = davinci_musb_set_mode,
-
- .set_vbus = davinci_musb_set_vbus,
-};
-
-static const struct platform_device_info davinci_dev_info = {
- .name = "musb-hdrc",
- .id = PLATFORM_DEVID_AUTO,
- .dma_mask = DMA_BIT_MASK(32),
-};
-
-static int davinci_probe(struct platform_device *pdev)
-{
- struct resource musb_resources[3];
- struct musb_hdrc_platform_data *pdata = dev_get_platdata(&pdev->dev);
- struct platform_device *musb;
- struct davinci_glue *glue;
- struct platform_device_info pinfo;
- struct clk *clk;
-
- int ret = -ENOMEM;
-
- glue = devm_kzalloc(&pdev->dev, sizeof(*glue), GFP_KERNEL);
- if (!glue)
- goto err0;
-
- clk = devm_clk_get(&pdev->dev, "usb");
- if (IS_ERR(clk)) {
- dev_err(&pdev->dev, "failed to get clock\n");
- ret = PTR_ERR(clk);
- goto err0;
- }
-
- ret = clk_enable(clk);
- if (ret) {
- dev_err(&pdev->dev, "failed to enable clock\n");
- goto err0;
- }
-
- glue->dev = &pdev->dev;
- glue->clk = clk;
-
- pdata->platform_ops = &davinci_ops;
-
- glue->vbus = devm_gpiod_get_optional(&pdev->dev, NULL, GPIOD_OUT_LOW);
- if (IS_ERR(glue->vbus)) {
- ret = PTR_ERR(glue->vbus);
- goto err0;
- } else {
- glue->vbus_state = -1;
- INIT_WORK(&glue->vbus_work, evm_deferred_drvvbus);
- }
-
- usb_phy_generic_register();
- platform_set_drvdata(pdev, glue);
-
- memset(musb_resources, 0x00, sizeof(*musb_resources) *
- ARRAY_SIZE(musb_resources));
-
- musb_resources[0].name = pdev->resource[0].name;
- musb_resources[0].start = pdev->resource[0].start;
- musb_resources[0].end = pdev->resource[0].end;
- musb_resources[0].flags = pdev->resource[0].flags;
-
- musb_resources[1].name = pdev->resource[1].name;
- musb_resources[1].start = pdev->resource[1].start;
- musb_resources[1].end = pdev->resource[1].end;
- musb_resources[1].flags = pdev->resource[1].flags;
-
- /*
- * For DM6467 3 resources are passed. A placeholder for the 3rd
- * resource is always there, so it's safe to always copy it...
- */
- musb_resources[2].name = pdev->resource[2].name;
- musb_resources[2].start = pdev->resource[2].start;
- musb_resources[2].end = pdev->resource[2].end;
- musb_resources[2].flags = pdev->resource[2].flags;
-
- pinfo = davinci_dev_info;
- pinfo.parent = &pdev->dev;
- pinfo.res = musb_resources;
- pinfo.num_res = ARRAY_SIZE(musb_resources);
- pinfo.data = pdata;
- pinfo.size_data = sizeof(*pdata);
-
- glue->musb = musb = platform_device_register_full(&pinfo);
- if (IS_ERR(musb)) {
- ret = PTR_ERR(musb);
- dev_err(&pdev->dev, "failed to register musb device: %d\n", ret);
- goto err1;
- }
-
- return 0;
-
-err1:
- clk_disable(clk);
-
-err0:
- return ret;
-}
-
-static int davinci_remove(struct platform_device *pdev)
-{
- struct davinci_glue *glue = platform_get_drvdata(pdev);
-
- platform_device_unregister(glue->musb);
- usb_phy_generic_unregister();
- clk_disable(glue->clk);
-
- return 0;
-}
-
-static struct platform_driver davinci_driver = {
- .probe = davinci_probe,
- .remove = davinci_remove,
- .driver = {
- .name = "musb-davinci",
- },
-};
-
-MODULE_DESCRIPTION("DaVinci MUSB Glue Layer");
-MODULE_AUTHOR("Felipe Balbi <balbi@ti.com>");
-MODULE_LICENSE("GPL v2");
-module_platform_driver(davinci_driver);
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * Copyright (C) 2005-2006 by Texas Instruments
- */
-
-#ifndef __MUSB_HDRDF_H__
-#define __MUSB_HDRDF_H__
-
-/*
- * DaVinci-specific definitions
- */
-
-/* Integrated highspeed/otg PHY */
-#define USBPHY_CTL_PADDR 0x01c40034
-#define USBPHY_DATAPOL BIT(11) /* (dm355) switch D+/D- */
-#define USBPHY_PHYCLKGD BIT(8)
-#define USBPHY_SESNDEN BIT(7) /* v(sess_end) comparator */
-#define USBPHY_VBDTCTEN BIT(6) /* v(bus) comparator */
-#define USBPHY_VBUSSENS BIT(5) /* (dm355,ro) is vbus > 0.5V */
-#define USBPHY_PHYPLLON BIT(4) /* override pll suspend */
-#define USBPHY_CLKO1SEL BIT(3)
-#define USBPHY_OSCPDWN BIT(2)
-#define USBPHY_OTGPDWN BIT(1)
-#define USBPHY_PHYPDWN BIT(0)
-
-#define DM355_DEEPSLEEP_PADDR 0x01c40048
-#define DRVVBUS_FORCE BIT(2)
-#define DRVVBUS_OVERRIDE BIT(1)
-
-/* For now include usb OTG module registers here */
-#define DAVINCI_USB_VERSION_REG 0x00
-#define DAVINCI_USB_CTRL_REG 0x04
-#define DAVINCI_USB_STAT_REG 0x08
-#define DAVINCI_RNDIS_REG 0x10
-#define DAVINCI_AUTOREQ_REG 0x14
-#define DAVINCI_USB_INT_SOURCE_REG 0x20
-#define DAVINCI_USB_INT_SET_REG 0x24
-#define DAVINCI_USB_INT_SRC_CLR_REG 0x28
-#define DAVINCI_USB_INT_MASK_REG 0x2c
-#define DAVINCI_USB_INT_MASK_SET_REG 0x30
-#define DAVINCI_USB_INT_MASK_CLR_REG 0x34
-#define DAVINCI_USB_INT_SRC_MASKED_REG 0x38
-#define DAVINCI_USB_EOI_REG 0x3c
-#define DAVINCI_USB_EOI_INTVEC 0x40
-
-/* BEGIN CPPI-generic (?) */
-
-/* CPPI related registers */
-#define DAVINCI_TXCPPI_CTRL_REG 0x80
-#define DAVINCI_TXCPPI_TEAR_REG 0x84
-#define DAVINCI_CPPI_EOI_REG 0x88
-#define DAVINCI_CPPI_INTVEC_REG 0x8c
-#define DAVINCI_TXCPPI_MASKED_REG 0x90
-#define DAVINCI_TXCPPI_RAW_REG 0x94
-#define DAVINCI_TXCPPI_INTENAB_REG 0x98
-#define DAVINCI_TXCPPI_INTCLR_REG 0x9c
-
-#define DAVINCI_RXCPPI_CTRL_REG 0xC0
-#define DAVINCI_RXCPPI_MASKED_REG 0xD0
-#define DAVINCI_RXCPPI_RAW_REG 0xD4
-#define DAVINCI_RXCPPI_INTENAB_REG 0xD8
-#define DAVINCI_RXCPPI_INTCLR_REG 0xDC
-
-#define DAVINCI_RXCPPI_BUFCNT0_REG 0xE0
-#define DAVINCI_RXCPPI_BUFCNT1_REG 0xE4
-#define DAVINCI_RXCPPI_BUFCNT2_REG 0xE8
-#define DAVINCI_RXCPPI_BUFCNT3_REG 0xEC
-
-/* CPPI state RAM entries */
-#define DAVINCI_CPPI_STATERAM_BASE_OFFSET 0x100
-
-#define DAVINCI_TXCPPI_STATERAM_OFFSET(chnum) \
- (DAVINCI_CPPI_STATERAM_BASE_OFFSET + ((chnum) * 0x40))
-#define DAVINCI_RXCPPI_STATERAM_OFFSET(chnum) \
- (DAVINCI_CPPI_STATERAM_BASE_OFFSET + 0x20 + ((chnum) * 0x40))
-
-/* CPPI masks */
-#define DAVINCI_DMA_CTRL_ENABLE 1
-#define DAVINCI_DMA_CTRL_DISABLE 0
-
-#define DAVINCI_DMA_ALL_CHANNELS_ENABLE 0xF
-#define DAVINCI_DMA_ALL_CHANNELS_DISABLE 0xF
-
-/* END CPPI-generic (?) */
-
-#define DAVINCI_USB_TX_ENDPTS_MASK 0x1f /* ep0 + 4 tx */
-#define DAVINCI_USB_RX_ENDPTS_MASK 0x1e /* 4 rx */
-
-#define DAVINCI_USB_USBINT_SHIFT 16
-#define DAVINCI_USB_TXINT_SHIFT 0
-#define DAVINCI_USB_RXINT_SHIFT 8
-
-#define DAVINCI_INTR_DRVVBUS 0x0100
-
-#define DAVINCI_USB_USBINT_MASK 0x01ff0000 /* 8 Mentor, DRVVBUS */
-#define DAVINCI_USB_TXINT_MASK \
- (DAVINCI_USB_TX_ENDPTS_MASK << DAVINCI_USB_TXINT_SHIFT)
-#define DAVINCI_USB_RXINT_MASK \
- (DAVINCI_USB_RX_ENDPTS_MASK << DAVINCI_USB_RXINT_SHIFT)
-
-#define DAVINCI_BASE_OFFSET 0x400
-
-#endif /* __MUSB_HDRDF_H__ */
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/of_device.h>
+#include <linux/phy/phy.h>
#include <linux/platform_device.h>
#include <linux/usb/role.h>
#include <linux/usb/usb_phy_generic.h>
struct jz4740_glue *glue = usb_role_switch_get_drvdata(sw);
struct usb_phy *phy = glue->musb->xceiv;
+ if (!phy)
+ return 0;
+
switch (role) {
case USB_ROLE_NONE:
atomic_notifier_call_chain(&phy->notifier, USB_EVENT_NONE, phy);
.driver_data = glue,
.fwnode = dev_fwnode(dev),
};
+ int err;
glue->musb = musb;
- if (dev->of_node)
- musb->xceiv = devm_usb_get_phy_by_phandle(dev, "phys", 0);
- else
- musb->xceiv = devm_usb_get_phy(dev, USB_PHY_TYPE_USB2);
- if (IS_ERR(musb->xceiv))
- return dev_err_probe(dev, PTR_ERR(musb->xceiv),
- "No transceiver configured\n");
+ if (IS_ENABLED(CONFIG_GENERIC_PHY)) {
+ musb->phy = devm_of_phy_get_by_index(dev, dev->of_node, 0);
+ if (IS_ERR(musb->phy)) {
+ err = PTR_ERR(musb->phy);
+ if (err != -ENODEV) {
+ dev_err(dev, "Unable to get PHY\n");
+ return err;
+ }
+
+ musb->phy = NULL;
+ }
+ }
+
+ if (musb->phy) {
+ err = phy_init(musb->phy);
+ if (err) {
+ dev_err(dev, "Failed to init PHY\n");
+ return err;
+ }
+
+ err = phy_power_on(musb->phy);
+ if (err) {
+ dev_err(dev, "Unable to power on PHY\n");
+ goto err_phy_shutdown;
+ }
+ } else {
+ if (dev->of_node)
+ musb->xceiv = devm_usb_get_phy_by_phandle(dev, "phys", 0);
+ else
+ musb->xceiv = devm_usb_get_phy(dev, USB_PHY_TYPE_USB2);
+ if (IS_ERR(musb->xceiv)) {
+ dev_err(dev, "No transceiver configured\n");
+ return PTR_ERR(musb->xceiv);
+ }
+ }
glue->role_sw = usb_role_switch_register(dev, &role_sw_desc);
if (IS_ERR(glue->role_sw)) {
dev_err(dev, "Failed to register USB role switch\n");
- return PTR_ERR(glue->role_sw);
+ err = PTR_ERR(glue->role_sw);
+ goto err_phy_power_down;
}
/*
musb->isr = jz4740_musb_interrupt;
return 0;
+
+err_phy_power_down:
+ if (musb->phy)
+ phy_power_off(musb->phy);
+err_phy_shutdown:
+ if (musb->phy)
+ phy_exit(musb->phy);
+ return err;
}
static int jz4740_musb_exit(struct musb *musb)
struct jz4740_glue *glue = dev_get_drvdata(musb->controller->parent);
usb_role_switch_unregister(glue->role_sw);
+ if (musb->phy) {
+ phy_power_off(musb->phy);
+ phy_exit(musb->phy);
+ }
return 0;
}
init_data:
musb->is_active = 1;
- musb->xceiv->otg->state = OTG_STATE_A_IDLE;
+ musb_set_state(musb, OTG_STATE_A_IDLE);
MUSB_HST_MODE(musb);
return error;
init_data:
musb->is_active = 0;
- musb->xceiv->otg->state = OTG_STATE_B_IDLE;
+ musb_set_state(musb, OTG_STATE_B_IDLE);
MUSB_DEV_MODE(musb);
return error;
unsigned long flags;
spin_lock_irqsave(&musb->lock, flags);
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_B_WAIT_ACON:
musb_dbg(musb,
"HNP: b_wait_acon timeout; back to b_peripheral");
musb_g_disconnect(musb);
- musb->xceiv->otg->state = OTG_STATE_B_PERIPHERAL;
+ musb_set_state(musb, OTG_STATE_B_PERIPHERAL);
musb->is_active = 0;
break;
case OTG_STATE_A_SUSPEND:
case OTG_STATE_A_WAIT_BCON:
musb_dbg(musb, "HNP: %s timeout",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
musb_platform_set_vbus(musb, 0);
- musb->xceiv->otg->state = OTG_STATE_A_WAIT_VFALL;
+ musb_set_state(musb, OTG_STATE_A_WAIT_VFALL);
break;
default:
musb_dbg(musb, "HNP: Unhandled mode %s",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
}
spin_unlock_irqrestore(&musb->lock, flags);
}
void __iomem *mbase = musb->mregs;
u8 reg;
- musb_dbg(musb, "HNP: stop from %s",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_dbg(musb, "HNP: stop from %s", musb_otg_state_string(musb));
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_PERIPHERAL:
musb_g_disconnect(musb);
- musb_dbg(musb, "HNP: back to %s",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_dbg(musb, "HNP: back to %s", musb_otg_state_string(musb));
break;
case OTG_STATE_B_HOST:
musb_dbg(musb, "HNP: Disabling HR");
if (hcd)
hcd->self.is_b_host = 0;
- musb->xceiv->otg->state = OTG_STATE_B_PERIPHERAL;
+ musb_set_state(musb, OTG_STATE_B_PERIPHERAL);
MUSB_DEV_MODE(musb);
reg = musb_readb(mbase, MUSB_POWER);
reg |= MUSB_POWER_SUSPENDM;
break;
default:
musb_dbg(musb, "HNP: Stopping in unknown state %s",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
}
/*
static void musb_handle_intr_resume(struct musb *musb, u8 devctl)
{
- musb_dbg(musb, "RESUME (%s)",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_dbg(musb, "RESUME (%s)", musb_otg_state_string(musb));
if (devctl & MUSB_DEVCTL_HM) {
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_SUSPEND:
/* remote wakeup? */
musb->port1_status |=
| MUSB_PORT_STAT_RESUME;
musb->rh_timer = jiffies
+ msecs_to_jiffies(USB_RESUME_TIMEOUT);
- musb->xceiv->otg->state = OTG_STATE_A_HOST;
+ musb_set_state(musb, OTG_STATE_A_HOST);
musb->is_active = 1;
musb_host_resume_root_hub(musb);
schedule_delayed_work(&musb->finish_resume_work,
msecs_to_jiffies(USB_RESUME_TIMEOUT));
break;
case OTG_STATE_B_WAIT_ACON:
- musb->xceiv->otg->state = OTG_STATE_B_PERIPHERAL;
+ musb_set_state(musb, OTG_STATE_B_PERIPHERAL);
musb->is_active = 1;
MUSB_DEV_MODE(musb);
break;
default:
WARNING("bogus %s RESUME (%s)\n",
"host",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
}
} else {
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_SUSPEND:
/* possibly DISCONNECT is upcoming */
- musb->xceiv->otg->state = OTG_STATE_A_HOST;
+ musb_set_state(musb, OTG_STATE_A_HOST);
musb_host_resume_root_hub(musb);
break;
case OTG_STATE_B_WAIT_ACON:
default:
WARNING("bogus %s RESUME (%s)\n",
"peripheral",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
}
}
}
return IRQ_HANDLED;
}
- musb_dbg(musb, "SESSION_REQUEST (%s)",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_dbg(musb, "SESSION_REQUEST (%s)", musb_otg_state_string(musb));
/* IRQ arrives from ID pin sense or (later, if VBUS power
* is removed) SRP. responses are time critical:
*/
musb_writeb(mbase, MUSB_DEVCTL, MUSB_DEVCTL_SESSION);
musb->ep0_stage = MUSB_EP0_START;
- musb->xceiv->otg->state = OTG_STATE_A_IDLE;
+ musb_set_state(musb, OTG_STATE_A_IDLE);
MUSB_HST_MODE(musb);
musb_platform_set_vbus(musb, 1);
* REVISIT: do delays from lots of DEBUG_KERNEL checks
* make trouble here, keeping VBUS < 4.4V ?
*/
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_HOST:
/* recovery is dicey once we've gotten past the
* initial stages of enumeration, but if VBUS
dev_printk(ignore ? KERN_DEBUG : KERN_ERR, musb->controller,
"VBUS_ERROR in %s (%02x, %s), retry #%d, port1 %08x\n",
- usb_otg_state_string(musb->xceiv->otg->state),
+ musb_otg_state_string(musb),
devctl,
({ char *s;
switch (devctl & MUSB_DEVCTL_VBUS) {
static void musb_handle_intr_suspend(struct musb *musb, u8 devctl)
{
musb_dbg(musb, "SUSPEND (%s) devctl %02x",
- usb_otg_state_string(musb->xceiv->otg->state), devctl);
+ musb_otg_state_string(musb), devctl);
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_PERIPHERAL:
/* We also come here if the cable is removed, since
* this silicon doesn't report ID-no-longer-grounded.
musb_g_suspend(musb);
musb->is_active = musb->g.b_hnp_enable;
if (musb->is_active) {
- musb->xceiv->otg->state = OTG_STATE_B_WAIT_ACON;
+ musb_set_state(musb, OTG_STATE_B_WAIT_ACON);
musb_dbg(musb, "HNP: Setting timer for b_ase0_brst");
mod_timer(&musb->otg_timer, jiffies
+ msecs_to_jiffies(
+ msecs_to_jiffies(musb->a_wait_bcon));
break;
case OTG_STATE_A_HOST:
- musb->xceiv->otg->state = OTG_STATE_A_SUSPEND;
+ musb_set_state(musb, OTG_STATE_A_SUSPEND);
musb->is_active = musb->hcd->self.b_hnp_enable;
break;
case OTG_STATE_B_HOST:
musb->port1_status |= USB_PORT_STAT_LOW_SPEED;
/* indicate new connection to OTG machine */
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_B_PERIPHERAL:
if (int_usb & MUSB_INTR_SUSPEND) {
musb_dbg(musb, "HNP: SUSPEND+CONNECT, now b_host");
case OTG_STATE_B_WAIT_ACON:
musb_dbg(musb, "HNP: CONNECT, now b_host");
b_host:
- musb->xceiv->otg->state = OTG_STATE_B_HOST;
+ musb_set_state(musb, OTG_STATE_B_HOST);
if (musb->hcd)
musb->hcd->self.is_b_host = 1;
del_timer(&musb->otg_timer);
default:
if ((devctl & MUSB_DEVCTL_VBUS)
== (3 << MUSB_DEVCTL_VBUS_SHIFT)) {
- musb->xceiv->otg->state = OTG_STATE_A_HOST;
+ musb_set_state(musb, OTG_STATE_A_HOST);
if (hcd)
hcd->self.is_b_host = 0;
}
musb_host_poke_root_hub(musb);
musb_dbg(musb, "CONNECT (%s) devctl %02x",
- usb_otg_state_string(musb->xceiv->otg->state), devctl);
+ musb_otg_state_string(musb), devctl);
}
static void musb_handle_intr_disconnect(struct musb *musb, u8 devctl)
{
musb_dbg(musb, "DISCONNECT (%s) as %s, devctl %02x",
- usb_otg_state_string(musb->xceiv->otg->state),
+ musb_otg_state_string(musb),
MUSB_MODE(musb), devctl);
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_HOST:
case OTG_STATE_A_SUSPEND:
musb_host_resume_root_hub(musb);
musb_root_disconnect(musb);
if (musb->hcd)
musb->hcd->self.is_b_host = 0;
- musb->xceiv->otg->state = OTG_STATE_B_PERIPHERAL;
+ musb_set_state(musb, OTG_STATE_B_PERIPHERAL);
MUSB_DEV_MODE(musb);
musb_g_disconnect(musb);
break;
break;
default:
WARNING("unhandled DISCONNECT transition (%s)\n",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
break;
}
}
dev_err(musb->controller, "Babble\n");
musb_recover_from_babble(musb);
} else {
- musb_dbg(musb, "BUS RESET as %s",
- usb_otg_state_string(musb->xceiv->otg->state));
- switch (musb->xceiv->otg->state) {
+ musb_dbg(musb, "BUS RESET as %s", musb_otg_state_string(musb));
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_SUSPEND:
musb_g_reset(musb);
fallthrough;
case OTG_STATE_A_WAIT_BCON: /* OPT TD.4.7-900ms */
/* never use invalid T(a_wait_bcon) */
musb_dbg(musb, "HNP: in %s, %d msec timeout",
- usb_otg_state_string(musb->xceiv->otg->state),
+ musb_otg_state_string(musb),
TA_WAIT_BCON(musb));
mod_timer(&musb->otg_timer, jiffies
+ msecs_to_jiffies(TA_WAIT_BCON(musb)));
break;
case OTG_STATE_B_WAIT_ACON:
musb_dbg(musb, "HNP: RESET (%s), to b_peripheral",
- usb_otg_state_string(musb->xceiv->otg->state));
- musb->xceiv->otg->state = OTG_STATE_B_PERIPHERAL;
+ musb_otg_state_string(musb));
+ musb_set_state(musb, OTG_STATE_B_PERIPHERAL);
musb_g_reset(musb);
break;
case OTG_STATE_B_IDLE:
- musb->xceiv->otg->state = OTG_STATE_B_PERIPHERAL;
+ musb_set_state(musb, OTG_STATE_B_PERIPHERAL);
fallthrough;
case OTG_STATE_B_PERIPHERAL:
musb_g_reset(musb);
break;
default:
musb_dbg(musb, "Unhandled BUS RESET as %s",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
}
}
}
* (c) peripheral initiates, using SRP
*/
if (musb->port_mode != MUSB_HOST &&
- musb->xceiv->otg->state != OTG_STATE_A_WAIT_BCON &&
- (devctl & MUSB_DEVCTL_VBUS) == MUSB_DEVCTL_VBUS) {
+ musb_get_state(musb) != OTG_STATE_A_WAIT_BCON &&
+ (devctl & MUSB_DEVCTL_VBUS) == MUSB_DEVCTL_VBUS) {
musb->is_active = 1;
} else {
devctl |= MUSB_DEVCTL_SESSION;
int ret;
spin_lock_irqsave(&musb->lock, flags);
- ret = sprintf(buf, "%s\n", usb_otg_state_string(musb->xceiv->otg->state));
+ ret = sprintf(buf, "%s\n", musb_otg_state_string(musb));
spin_unlock_irqrestore(&musb->lock, flags);
return ret;
spin_lock_irqsave(&musb->lock, flags);
/* force T(a_wait_bcon) to be zero/unlimited *OR* valid */
musb->a_wait_bcon = val ? max_t(int, val, OTG_TIME_A_WAIT_BCON) : 0 ;
- if (musb->xceiv->otg->state == OTG_STATE_A_WAIT_BCON)
+ if (musb_get_state(musb) == OTG_STATE_A_WAIT_BCON)
musb->is_active = 0;
musb_platform_try_idle(musb, jiffies + msecs_to_jiffies(val));
spin_unlock_irqrestore(&musb->lock, flags);
musb_pm_runtime_check_session(musb);
- if (musb->xceiv->otg->state != musb->xceiv_old_state) {
- musb->xceiv_old_state = musb->xceiv->otg->state;
+ if (musb_get_state(musb) != musb->xceiv_old_state) {
+ musb->xceiv_old_state = musb_get_state(musb);
sysfs_notify(&musb->controller->kobj, NULL, "mode");
}
else
musb->io.set_toggle = musb_default_set_toggle;
- if (!musb->xceiv->io_ops) {
+ if (IS_ENABLED(CONFIG_USB_PHY) && musb->xceiv && !musb->xceiv->io_ops) {
musb->xceiv->io_dev = musb->controller;
musb->xceiv->io_priv = musb->mregs;
musb->xceiv->io_ops = &musb_ulpi_access;
}
MUSB_DEV_MODE(musb);
- musb->xceiv->otg->state = OTG_STATE_B_IDLE;
+ musb_set_state(musb, OTG_STATE_B_IDLE);
switch (musb->port_mode) {
case MUSB_HOST:
struct usb_phy *xceiv;
struct phy *phy;
+ enum usb_otg_state otg_state;
+
int nIrq;
unsigned irq_wake:1;
musb->ops->clear_ep_rxintr(musb, epnum);
}
+static inline void musb_set_state(struct musb *musb,
+ enum usb_otg_state otg_state)
+{
+ if (musb->xceiv)
+ musb->xceiv->otg->state = otg_state;
+ else
+ musb->otg_state = otg_state;
+}
+
+static inline enum usb_otg_state musb_get_state(struct musb *musb)
+{
+ if (musb->xceiv)
+ return musb->xceiv->otg->state;
+
+ return musb->otg_state;
+}
+
+static inline const char *musb_otg_state_string(struct musb *musb)
+{
+ return usb_otg_state_string(musb_get_state(musb));
+}
+
/*
* gets the "dr_mode" property from DT and converts it into musb_mode
* if the property is not found or not recognized returns MUSB_OTG
u8 reg;
int connect;
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_HOST:
case OTG_STATE_A_WAIT_BCON:
pm_runtime_get_sync(musb->controller);
pm_runtime_get_sync(musb->controller);
if (!strncmp(buf, "0", 1)) {
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_HOST:
musb_root_disconnect(musb);
reg = musb_readb(musb->mregs, MUSB_DEVCTL);
break;
}
} else if (!strncmp(buf, "1", 1)) {
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_WAIT_BCON:
/*
* musb_save_context() called in musb_runtime_suspend()
#define musb_dma_cppi41(musb) 0
#endif
-#ifdef CONFIG_USB_TI_CPPI_DMA
-#define musb_dma_cppi(musb) (musb->ops->quirks & MUSB_DMA_CPPI)
-#else
-#define musb_dma_cppi(musb) 0
-#endif
-
#ifdef CONFIG_USB_TUSB_OMAP_DMA
#define tusb_dma_omap(musb) (musb->ops->quirks & MUSB_DMA_TUSB_OMAP)
#else
#define musb_dma_inventra(musb) 0
#endif
-#if defined(CONFIG_USB_TI_CPPI_DMA) || defined(CONFIG_USB_TI_CPPI41_DMA)
-#define is_cppi_enabled(musb) \
- (musb_dma_cppi(musb) || musb_dma_cppi41(musb))
+#if defined(CONFIG_USB_TI_CPPI41_DMA)
+#define is_cppi_enabled(musb) musb_dma_cppi41(musb)
#else
-#define is_cppi_enabled(musb) 0
+#define is_cppi_enabled(musb) 0
#endif
/*
spin_lock_irqsave(&musb->lock, flags);
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_B_PERIPHERAL:
/* NOTE: OTG state machine doesn't include B_SUSPENDED;
* that's part of the standard usb 1.1 state machine, and
break;
}
- spin_unlock_irqrestore(&musb->lock, flags);
- otg_start_srp(musb->xceiv->otg);
- spin_lock_irqsave(&musb->lock, flags);
+ if (musb->xceiv) {
+ spin_unlock_irqrestore(&musb->lock, flags);
+ otg_start_srp(musb->xceiv->otg);
+ spin_lock_irqsave(&musb->lock, flags);
+ }
/* Block idling for at least 1s */
musb_platform_try_idle(musb,
goto done;
default:
musb_dbg(musb, "Unhandled wake: %s",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
goto done;
}
{
struct musb *musb = gadget_to_musb(gadget);
- if (!musb->xceiv->set_power)
- return -EOPNOTSUPP;
return usb_phy_set_power(musb->xceiv, mA);
}
musb->g.speed = USB_SPEED_UNKNOWN;
MUSB_DEV_MODE(musb);
- musb->xceiv->otg->state = OTG_STATE_B_IDLE;
+ musb_set_state(musb, OTG_STATE_B_IDLE);
/* this "gadget" abstracts/virtualizes the controller */
musb->g.name = musb_driver_name;
struct usb_gadget_driver *driver)
{
struct musb *musb = gadget_to_musb(g);
- struct usb_otg *otg = musb->xceiv->otg;
unsigned long flags;
int retval = 0;
spin_lock_irqsave(&musb->lock, flags);
musb->is_active = 1;
- otg_set_peripheral(otg, &musb->g);
- musb->xceiv->otg->state = OTG_STATE_B_IDLE;
+ if (musb->xceiv)
+ otg_set_peripheral(musb->xceiv->otg, &musb->g);
+ else
+ phy_set_mode(musb->phy, PHY_MODE_USB_DEVICE);
+
+ musb_set_state(musb, OTG_STATE_B_IDLE);
spin_unlock_irqrestore(&musb->lock, flags);
musb_start(musb);
* handles power budgeting ... this way also
* ensures HdrcStart is indirectly called.
*/
- if (musb->xceiv->last_event == USB_EVENT_ID)
+ if (musb->xceiv && musb->xceiv->last_event == USB_EVENT_ID)
musb_platform_set_vbus(musb, 1);
pm_runtime_mark_last_busy(musb->controller);
(void) musb_gadget_vbus_draw(&musb->g, 0);
- musb->xceiv->otg->state = OTG_STATE_UNDEFINED;
+ musb_set_state(musb, OTG_STATE_UNDEFINED);
musb_stop(musb);
- otg_set_peripheral(musb->xceiv->otg, NULL);
+
+ if (musb->xceiv)
+ otg_set_peripheral(musb->xceiv->otg, NULL);
+ else
+ phy_set_mode(musb->phy, PHY_MODE_INVALID);
musb->is_active = 0;
musb->gadget_driver = NULL;
void musb_g_resume(struct musb *musb)
{
musb->is_suspended = 0;
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_B_IDLE:
break;
case OTG_STATE_B_WAIT_ACON:
break;
default:
WARNING("unhandled RESUME transition (%s)\n",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
}
}
devctl = musb_readb(musb->mregs, MUSB_DEVCTL);
musb_dbg(musb, "musb_g_suspend: devctl %02x", devctl);
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_B_IDLE:
if ((devctl & MUSB_DEVCTL_VBUS) == MUSB_DEVCTL_VBUS)
- musb->xceiv->otg->state = OTG_STATE_B_PERIPHERAL;
+ musb_set_state(musb, OTG_STATE_B_PERIPHERAL);
break;
case OTG_STATE_B_PERIPHERAL:
musb->is_suspended = 1;
* A_PERIPHERAL may need care too
*/
WARNING("unhandled SUSPEND transition (%s)",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
}
}
spin_lock(&musb->lock);
}
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
default:
musb_dbg(musb, "Unhandled disconnect %s, setting a_idle",
- usb_otg_state_string(musb->xceiv->otg->state));
- musb->xceiv->otg->state = OTG_STATE_A_IDLE;
+ musb_otg_state_string(musb));
+ musb_set_state(musb, OTG_STATE_A_IDLE);
MUSB_HST_MODE(musb);
break;
case OTG_STATE_A_PERIPHERAL:
- musb->xceiv->otg->state = OTG_STATE_A_WAIT_BCON;
+ musb_set_state(musb, OTG_STATE_A_WAIT_BCON);
MUSB_HST_MODE(musb);
break;
case OTG_STATE_B_WAIT_ACON:
case OTG_STATE_B_HOST:
case OTG_STATE_B_PERIPHERAL:
case OTG_STATE_B_IDLE:
- musb->xceiv->otg->state = OTG_STATE_B_IDLE;
+ musb_set_state(musb, OTG_STATE_B_IDLE);
break;
case OTG_STATE_B_SRP_INIT:
break;
* In that case, do not rely on devctl for setting
* peripheral mode.
*/
- musb->xceiv->otg->state = OTG_STATE_B_PERIPHERAL;
+ musb_set_state(musb, OTG_STATE_B_PERIPHERAL);
musb->g.is_a_peripheral = 0;
} else if (devctl & MUSB_DEVCTL_BDEVICE) {
- musb->xceiv->otg->state = OTG_STATE_B_PERIPHERAL;
+ musb_set_state(musb, OTG_STATE_B_PERIPHERAL);
musb->g.is_a_peripheral = 0;
} else {
- musb->xceiv->otg->state = OTG_STATE_A_PERIPHERAL;
+ musb_set_state(musb, OTG_STATE_A_PERIPHERAL);
musb->g.is_a_peripheral = 1;
}
if (!is_host_active(musb))
return 0;
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_SUSPEND:
return 0;
case OTG_STATE_A_WAIT_VRISE:
*/
devctl = musb_readb(musb->mregs, MUSB_DEVCTL);
if ((devctl & MUSB_DEVCTL_VBUS) == MUSB_DEVCTL_VBUS)
- musb->xceiv->otg->state = OTG_STATE_A_WAIT_BCON;
+ musb_set_state(musb, OTG_STATE_A_WAIT_BCON);
break;
default:
break;
if (musb->is_active) {
WARNING("trying to suspend as %s while active\n",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
return -EBUSY;
} else
return 0;
if (musb->port_mode == MUSB_HOST) {
MUSB_HST_MODE(musb);
- musb->xceiv->otg->state = OTG_STATE_A_IDLE;
+ musb_set_state(musb, OTG_STATE_A_IDLE);
}
- otg_set_host(musb->xceiv->otg, &hcd->self);
+
+ if (musb->xceiv) {
+ otg_set_host(musb->xceiv->otg, &hcd->self);
+ musb->xceiv->otg->host = &hcd->self;
+ } else {
+ phy_set_mode(musb->phy, PHY_MODE_USB_HOST);
+ }
+
/* don't support otg protocols */
hcd->self.otg_port = 0;
- musb->xceiv->otg->host = &hcd->self;
hcd->power_budget = 2 * (power_budget ? : 250);
hcd->skip_phy_initialization = 1;
musb->port1_status |= USB_PORT_STAT_C_SUSPEND << 16;
usb_hcd_poll_rh_status(musb->hcd);
/* NOTE: it might really be A_WAIT_BCON ... */
- musb->xceiv->otg->state = OTG_STATE_A_HOST;
+ musb_set_state(musb, OTG_STATE_A_HOST);
spin_unlock_irqrestore(&musb->lock, flags);
}
int musb_port_suspend(struct musb *musb, bool do_suspend)
{
- struct usb_otg *otg = musb->xceiv->otg;
u8 power;
void __iomem *mbase = musb->mregs;
musb_dbg(musb, "Root port suspended, power %02x", power);
musb->port1_status |= USB_PORT_STAT_SUSPEND;
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_HOST:
- musb->xceiv->otg->state = OTG_STATE_A_SUSPEND;
- musb->is_active = otg->host->b_hnp_enable;
+ musb_set_state(musb, OTG_STATE_A_SUSPEND);
+ musb->is_active = musb->xceiv &&
+ musb->xceiv->otg->host->b_hnp_enable;
if (musb->is_active)
mod_timer(&musb->otg_timer, jiffies
+ msecs_to_jiffies(
musb_platform_try_idle(musb, 0);
break;
case OTG_STATE_B_HOST:
- musb->xceiv->otg->state = OTG_STATE_B_WAIT_ACON;
- musb->is_active = otg->host->b_hnp_enable;
+ musb_set_state(musb, OTG_STATE_B_WAIT_ACON);
+ musb->is_active = musb->xceiv &&
+ musb->xceiv->otg->host->b_hnp_enable;
musb_platform_try_idle(musb, 0);
break;
default:
musb_dbg(musb, "bogus rh suspend? %s",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
}
} else if (power & MUSB_POWER_SUSPENDM) {
power &= ~MUSB_POWER_SUSPENDM;
u8 power;
void __iomem *mbase = musb->mregs;
- if (musb->xceiv->otg->state == OTG_STATE_B_IDLE) {
+ if (musb_get_state(musb) == OTG_STATE_B_IDLE) {
musb_dbg(musb, "HNP: Returning from HNP; no hub reset from b_idle");
musb->port1_status &= ~USB_PORT_STAT_RESET;
return;
void musb_root_disconnect(struct musb *musb)
{
- struct usb_otg *otg = musb->xceiv->otg;
-
musb->port1_status = USB_PORT_STAT_POWER
| (USB_PORT_STAT_C_CONNECTION << 16);
usb_hcd_poll_rh_status(musb->hcd);
musb->is_active = 0;
- switch (musb->xceiv->otg->state) {
+ switch (musb_get_state(musb)) {
case OTG_STATE_A_SUSPEND:
- if (otg->host->b_hnp_enable) {
- musb->xceiv->otg->state = OTG_STATE_A_PERIPHERAL;
+ if (musb->xceiv && musb->xceiv->otg->host->b_hnp_enable) {
+ musb_set_state(musb, OTG_STATE_A_PERIPHERAL);
musb->g.is_a_peripheral = 1;
break;
}
fallthrough;
case OTG_STATE_A_HOST:
- musb->xceiv->otg->state = OTG_STATE_A_WAIT_BCON;
+ musb_set_state(musb, OTG_STATE_A_WAIT_BCON);
musb->is_active = 0;
break;
case OTG_STATE_A_WAIT_VFALL:
- musb->xceiv->otg->state = OTG_STATE_B_IDLE;
+ musb_set_state(musb, OTG_STATE_B_IDLE);
break;
default:
musb_dbg(musb, "host disconnect (%s)",
- usb_otg_state_string(musb->xceiv->otg->state));
+ musb_otg_state_string(musb));
}
}
EXPORT_SYMBOL_GPL(musb_root_disconnect);
#include <linux/list.h>
#include <linux/io.h>
#include <linux/of.h>
+#include <linux/of_irq.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#include <linux/pm_runtime.h>
struct device_node *control_node;
struct platform_device *control_pdev;
int ret = -ENOMEM, val;
+ bool populate_irqs = false;
if (!np)
return -ENODEV;
musb->dev.dma_mask = &omap2430_dmamask;
musb->dev.coherent_dma_mask = omap2430_dmamask;
+ /*
+ * Legacy SoCs using omap_device get confused if node is moved
+ * because of interconnect properties mixed into the node.
+ */
+ if (of_get_property(np, "ti,hwmods", NULL)) {
+ dev_warn(&pdev->dev, "please update to probe with ti-sysc\n");
+ populate_irqs = true;
+ } else {
+ device_set_of_node_from_dev(&musb->dev, &pdev->dev);
+ }
+ of_node_put(np);
+
glue->dev = &pdev->dev;
glue->musb = musb;
glue->status = MUSB_UNKNOWN;
goto err2;
}
+ if (populate_irqs) {
+ struct resource musb_res[3];
+ struct resource *res;
+ int i = 0;
+
+ memset(musb_res, 0, sizeof(*musb_res) * ARRAY_SIZE(musb_res));
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res)
+ goto err2;
+
+ musb_res[i].start = res->start;
+ musb_res[i].end = res->end;
+ musb_res[i].flags = res->flags;
+ musb_res[i].name = res->name;
+ i++;
+
+ ret = of_irq_get_byname(np, "mc");
+ if (ret > 0) {
+ musb_res[i].start = ret;
+ musb_res[i].flags = IORESOURCE_IRQ;
+ musb_res[i].name = "mc";
+ i++;
+ }
+
+ ret = of_irq_get_byname(np, "dma");
+ if (ret > 0) {
+ musb_res[i].start = ret;
+ musb_res[i].flags = IORESOURCE_IRQ;
+ musb_res[i].name = "dma";
+ i++;
+ }
+
+ ret = platform_device_add_resources(musb, musb_res, i);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to add IRQ resources\n");
+ goto err2;
+ }
+ }
+
ret = platform_device_add_data(musb, pdata, sizeof(*pdata));
if (ret) {
dev_err(&pdev->dev, "failed to add platform_data\n");
tristate "GPIO based peripheral-only VBUS sensing 'transceiver'"
depends on GPIOLIB || COMPILE_TEST
depends on USB_GADGET || !USB_GADGET # if USB_GADGET=m, this can't be 'y'
+ depends on !USB_CONN_GPIO
select USB_PHY
help
Provides simple GPIO VBUS sensing for controllers with an
internal transceiver via the usb_phy interface, and
optionally control of a D+ pullup GPIO as well as a VBUS
- current limit regulator.
+ current limit regulator. This driver is for devices that do
+ NOT support role switch. OTG devices that can do role switch
+ (master/peripheral) shall use the USB based connection
+ detection driver USB_CONN_GPIO.
config OMAP_OTG
tristate "OMAP USB OTG controller driver"
Provides read/write operations to the ULPI phy register set for
controllers with a viewport register (e.g. Chipidea/ARC controllers).
-config JZ4770_PHY
- tristate "Ingenic SoCs Transceiver Driver"
- depends on MIPS || COMPILE_TEST
- select USB_PHY
- help
- This driver provides PHY support for the USB controller found
- on the JZ-series and X-series SoCs from Ingenic.
-
endmenu
obj-$(CONFIG_USB_ULPI) += phy-ulpi.o
obj-$(CONFIG_USB_ULPI_VIEWPORT) += phy-ulpi-viewport.o
obj-$(CONFIG_KEYSTONE_USB_PHY) += phy-keystone.o
-obj-$(CONFIG_JZ4770_PHY) += phy-jz4770.o
int err = 0;
u32 clk_rate = 0;
- bool needs_vcc = false, needs_clk = false;
+ bool needs_clk = false;
if (dev->of_node) {
struct device_node *node = dev->of_node;
if (of_property_read_u32(node, "clock-frequency", &clk_rate))
clk_rate = 0;
- needs_vcc = of_property_read_bool(node, "vcc-supply");
needs_clk = of_property_read_bool(node, "clocks");
}
nop->gpiod_reset = devm_gpiod_get_optional(dev, "reset",
}
}
- nop->vcc = devm_regulator_get(dev, "vcc");
- if (IS_ERR(nop->vcc)) {
- dev_dbg(dev, "Error getting vcc regulator: %ld\n",
- PTR_ERR(nop->vcc));
- if (needs_vcc)
- return -EPROBE_DEFER;
- }
+ nop->vcc = devm_regulator_get_optional(dev, "vcc");
+ if (IS_ERR(nop->vcc) && PTR_ERR(nop->vcc) != -ENODEV)
+ return dev_err_probe(dev, PTR_ERR(nop->vcc),
+ "could not get vcc regulator\n");
nop->vbus_draw = devm_regulator_get_exclusive(dev, "vbus");
if (PTR_ERR(nop->vbus_draw) == -ENODEV)
static int usb_phy_generic_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
+ struct device_node *dn = dev->of_node;
struct usb_phy_generic *nop;
int err;
platform_set_drvdata(pdev, nop);
+ device_set_wakeup_capable(&pdev->dev,
+ of_property_read_bool(dn, "wakeup-source"));
+
return 0;
}
MODULE_ALIAS("platform:gpio-vbus");
+/*
+ * NOTE: this driver matches against "gpio-usb-b-connector" for
+ * devices that do NOT support role switch.
+ */
+static const struct of_device_id gpio_vbus_of_match[] = {
+ {
+ .compatible = "gpio-usb-b-connector",
+ },
+ {},
+};
+
static struct platform_driver gpio_vbus_driver = {
.driver = {
.name = "gpio-vbus",
#ifdef CONFIG_PM
.pm = &gpio_vbus_dev_pm_ops,
#endif
+ .of_match_table = gpio_vbus_of_match,
},
.probe = gpio_vbus_probe,
.remove = gpio_vbus_remove,
/*-------------------------------------------------------------------------*/
static int
-isp1301_probe(struct i2c_client *i2c, const struct i2c_device_id *id)
+isp1301_probe(struct i2c_client *i2c)
{
int status;
struct isp1301 *isp;
.driver = {
.name = "isp1301_omap",
},
- .probe = isp1301_probe,
+ .probe_new = isp1301_probe,
.remove = isp1301_remove,
.id_table = isp1301_id,
};
return 0;
}
-static int isp1301_probe(struct i2c_client *client,
- const struct i2c_device_id *i2c_id)
+static int isp1301_probe(struct i2c_client *client)
{
struct isp1301 *isp;
struct usb_phy *phy;
.name = DRV_NAME,
.of_match_table = isp1301_of_match,
},
- .probe = isp1301_probe,
+ .probe_new = isp1301_probe,
.remove = isp1301_remove,
.id_table = isp1301_id,
};
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Ingenic SoCs USB PHY driver
- * Copyright (c) Paul Cercueil <paul@crapouillou.net>
- * Copyright (c) 漆鹏振 (Qi Pengzhen) <aric.pzqi@ingenic.com>
- * Copyright (c) 周琰杰 (Zhou Yanjie) <zhouyanjie@wanyeetech.com>
- */
-
-#include <linux/clk.h>
-#include <linux/io.h>
-#include <linux/module.h>
-#include <linux/platform_device.h>
-#include <linux/regulator/consumer.h>
-#include <linux/usb/otg.h>
-#include <linux/usb/phy.h>
-
-/* OTGPHY register offsets */
-#define REG_USBPCR_OFFSET 0x00
-#define REG_USBRDT_OFFSET 0x04
-#define REG_USBVBFIL_OFFSET 0x08
-#define REG_USBPCR1_OFFSET 0x0c
-
-/* bits within the USBPCR register */
-#define USBPCR_USB_MODE BIT(31)
-#define USBPCR_AVLD_REG BIT(30)
-#define USBPCR_COMMONONN BIT(25)
-#define USBPCR_VBUSVLDEXT BIT(24)
-#define USBPCR_VBUSVLDEXTSEL BIT(23)
-#define USBPCR_POR BIT(22)
-#define USBPCR_SIDDQ BIT(21)
-#define USBPCR_OTG_DISABLE BIT(20)
-#define USBPCR_TXPREEMPHTUNE BIT(6)
-
-#define USBPCR_IDPULLUP_LSB 28
-#define USBPCR_IDPULLUP_MASK GENMASK(29, USBPCR_IDPULLUP_LSB)
-#define USBPCR_IDPULLUP_ALWAYS (0x2 << USBPCR_IDPULLUP_LSB)
-#define USBPCR_IDPULLUP_SUSPEND (0x1 << USBPCR_IDPULLUP_LSB)
-#define USBPCR_IDPULLUP_OTG (0x0 << USBPCR_IDPULLUP_LSB)
-
-#define USBPCR_COMPDISTUNE_LSB 17
-#define USBPCR_COMPDISTUNE_MASK GENMASK(19, USBPCR_COMPDISTUNE_LSB)
-#define USBPCR_COMPDISTUNE_DFT (0x4 << USBPCR_COMPDISTUNE_LSB)
-
-#define USBPCR_OTGTUNE_LSB 14
-#define USBPCR_OTGTUNE_MASK GENMASK(16, USBPCR_OTGTUNE_LSB)
-#define USBPCR_OTGTUNE_DFT (0x4 << USBPCR_OTGTUNE_LSB)
-
-#define USBPCR_SQRXTUNE_LSB 11
-#define USBPCR_SQRXTUNE_MASK GENMASK(13, USBPCR_SQRXTUNE_LSB)
-#define USBPCR_SQRXTUNE_DCR_20PCT (0x7 << USBPCR_SQRXTUNE_LSB)
-#define USBPCR_SQRXTUNE_DFT (0x3 << USBPCR_SQRXTUNE_LSB)
-
-#define USBPCR_TXFSLSTUNE_LSB 7
-#define USBPCR_TXFSLSTUNE_MASK GENMASK(10, USBPCR_TXFSLSTUNE_LSB)
-#define USBPCR_TXFSLSTUNE_DCR_50PPT (0xf << USBPCR_TXFSLSTUNE_LSB)
-#define USBPCR_TXFSLSTUNE_DCR_25PPT (0x7 << USBPCR_TXFSLSTUNE_LSB)
-#define USBPCR_TXFSLSTUNE_DFT (0x3 << USBPCR_TXFSLSTUNE_LSB)
-#define USBPCR_TXFSLSTUNE_INC_25PPT (0x1 << USBPCR_TXFSLSTUNE_LSB)
-#define USBPCR_TXFSLSTUNE_INC_50PPT (0x0 << USBPCR_TXFSLSTUNE_LSB)
-
-#define USBPCR_TXHSXVTUNE_LSB 4
-#define USBPCR_TXHSXVTUNE_MASK GENMASK(5, USBPCR_TXHSXVTUNE_LSB)
-#define USBPCR_TXHSXVTUNE_DFT (0x3 << USBPCR_TXHSXVTUNE_LSB)
-#define USBPCR_TXHSXVTUNE_DCR_15MV (0x1 << USBPCR_TXHSXVTUNE_LSB)
-
-#define USBPCR_TXRISETUNE_LSB 4
-#define USBPCR_TXRISETUNE_MASK GENMASK(5, USBPCR_TXRISETUNE_LSB)
-#define USBPCR_TXRISETUNE_DFT (0x3 << USBPCR_TXRISETUNE_LSB)
-
-#define USBPCR_TXVREFTUNE_LSB 0
-#define USBPCR_TXVREFTUNE_MASK GENMASK(3, USBPCR_TXVREFTUNE_LSB)
-#define USBPCR_TXVREFTUNE_INC_25PPT (0x7 << USBPCR_TXVREFTUNE_LSB)
-#define USBPCR_TXVREFTUNE_DFT (0x5 << USBPCR_TXVREFTUNE_LSB)
-
-/* bits within the USBRDTR register */
-#define USBRDT_UTMI_RST BIT(27)
-#define USBRDT_HB_MASK BIT(26)
-#define USBRDT_VBFIL_LD_EN BIT(25)
-#define USBRDT_IDDIG_EN BIT(24)
-#define USBRDT_IDDIG_REG BIT(23)
-#define USBRDT_VBFIL_EN BIT(2)
-
-/* bits within the USBPCR1 register */
-#define USBPCR1_BVLD_REG BIT(31)
-#define USBPCR1_DPPD BIT(29)
-#define USBPCR1_DMPD BIT(28)
-#define USBPCR1_USB_SEL BIT(28)
-#define USBPCR1_WORD_IF_16BIT BIT(19)
-
-enum ingenic_usb_phy_version {
- ID_JZ4770,
- ID_JZ4780,
- ID_X1000,
- ID_X1830,
-};
-
-struct ingenic_soc_info {
- enum ingenic_usb_phy_version version;
-
- void (*usb_phy_init)(struct usb_phy *phy);
-};
-
-struct jz4770_phy {
- const struct ingenic_soc_info *soc_info;
-
- struct usb_phy phy;
- struct usb_otg otg;
- struct device *dev;
- void __iomem *base;
- struct clk *clk;
- struct regulator *vcc_supply;
-};
-
-static inline struct jz4770_phy *otg_to_jz4770_phy(struct usb_otg *otg)
-{
- return container_of(otg, struct jz4770_phy, otg);
-}
-
-static inline struct jz4770_phy *phy_to_jz4770_phy(struct usb_phy *phy)
-{
- return container_of(phy, struct jz4770_phy, phy);
-}
-
-static int ingenic_usb_phy_set_peripheral(struct usb_otg *otg,
- struct usb_gadget *gadget)
-{
- struct jz4770_phy *priv = otg_to_jz4770_phy(otg);
- u32 reg;
-
- if (priv->soc_info->version >= ID_X1000) {
- reg = readl(priv->base + REG_USBPCR1_OFFSET);
- reg |= USBPCR1_BVLD_REG;
- writel(reg, priv->base + REG_USBPCR1_OFFSET);
- }
-
- reg = readl(priv->base + REG_USBPCR_OFFSET);
- reg &= ~USBPCR_USB_MODE;
- reg |= USBPCR_VBUSVLDEXT | USBPCR_VBUSVLDEXTSEL | USBPCR_OTG_DISABLE;
- writel(reg, priv->base + REG_USBPCR_OFFSET);
-
- return 0;
-}
-
-static int ingenic_usb_phy_set_host(struct usb_otg *otg, struct usb_bus *host)
-{
- struct jz4770_phy *priv = otg_to_jz4770_phy(otg);
- u32 reg;
-
- reg = readl(priv->base + REG_USBPCR_OFFSET);
- reg &= ~(USBPCR_VBUSVLDEXT | USBPCR_VBUSVLDEXTSEL | USBPCR_OTG_DISABLE);
- reg |= USBPCR_USB_MODE;
- writel(reg, priv->base + REG_USBPCR_OFFSET);
-
- return 0;
-}
-
-static int ingenic_usb_phy_init(struct usb_phy *phy)
-{
- struct jz4770_phy *priv = phy_to_jz4770_phy(phy);
- int err;
- u32 reg;
-
- err = regulator_enable(priv->vcc_supply);
- if (err) {
- dev_err(priv->dev, "Unable to enable VCC: %d\n", err);
- return err;
- }
-
- err = clk_prepare_enable(priv->clk);
- if (err) {
- dev_err(priv->dev, "Unable to start clock: %d\n", err);
- return err;
- }
-
- priv->soc_info->usb_phy_init(phy);
-
- /* Wait for PHY to reset */
- usleep_range(30, 300);
- reg = readl(priv->base + REG_USBPCR_OFFSET);
- writel(reg & ~USBPCR_POR, priv->base + REG_USBPCR_OFFSET);
- usleep_range(300, 1000);
-
- return 0;
-}
-
-static void ingenic_usb_phy_shutdown(struct usb_phy *phy)
-{
- struct jz4770_phy *priv = phy_to_jz4770_phy(phy);
-
- clk_disable_unprepare(priv->clk);
- regulator_disable(priv->vcc_supply);
-}
-
-static void ingenic_usb_phy_remove(void *phy)
-{
- usb_remove_phy(phy);
-}
-
-static void jz4770_usb_phy_init(struct usb_phy *phy)
-{
- struct jz4770_phy *priv = phy_to_jz4770_phy(phy);
- u32 reg;
-
- reg = USBPCR_AVLD_REG | USBPCR_COMMONONN | USBPCR_IDPULLUP_ALWAYS |
- USBPCR_COMPDISTUNE_DFT | USBPCR_OTGTUNE_DFT | USBPCR_SQRXTUNE_DFT |
- USBPCR_TXFSLSTUNE_DFT | USBPCR_TXRISETUNE_DFT | USBPCR_TXVREFTUNE_DFT |
- USBPCR_POR;
- writel(reg, priv->base + REG_USBPCR_OFFSET);
-}
-
-static void jz4780_usb_phy_init(struct usb_phy *phy)
-{
- struct jz4770_phy *priv = phy_to_jz4770_phy(phy);
- u32 reg;
-
- reg = readl(priv->base + REG_USBPCR1_OFFSET) | USBPCR1_USB_SEL |
- USBPCR1_WORD_IF_16BIT;
- writel(reg, priv->base + REG_USBPCR1_OFFSET);
-
- reg = USBPCR_TXPREEMPHTUNE | USBPCR_COMMONONN | USBPCR_POR;
- writel(reg, priv->base + REG_USBPCR_OFFSET);
-}
-
-static void x1000_usb_phy_init(struct usb_phy *phy)
-{
- struct jz4770_phy *priv = phy_to_jz4770_phy(phy);
- u32 reg;
-
- reg = readl(priv->base + REG_USBPCR1_OFFSET) | USBPCR1_WORD_IF_16BIT;
- writel(reg, priv->base + REG_USBPCR1_OFFSET);
-
- reg = USBPCR_SQRXTUNE_DCR_20PCT | USBPCR_TXPREEMPHTUNE |
- USBPCR_TXHSXVTUNE_DCR_15MV | USBPCR_TXVREFTUNE_INC_25PPT |
- USBPCR_COMMONONN | USBPCR_POR;
- writel(reg, priv->base + REG_USBPCR_OFFSET);
-}
-
-static void x1830_usb_phy_init(struct usb_phy *phy)
-{
- struct jz4770_phy *priv = phy_to_jz4770_phy(phy);
- u32 reg;
-
- /* rdt */
- writel(USBRDT_VBFIL_EN | USBRDT_UTMI_RST, priv->base + REG_USBRDT_OFFSET);
-
- reg = readl(priv->base + REG_USBPCR1_OFFSET) | USBPCR1_WORD_IF_16BIT |
- USBPCR1_DMPD | USBPCR1_DPPD;
- writel(reg, priv->base + REG_USBPCR1_OFFSET);
-
- reg = USBPCR_IDPULLUP_OTG | USBPCR_VBUSVLDEXT | USBPCR_TXPREEMPHTUNE |
- USBPCR_COMMONONN | USBPCR_POR;
- writel(reg, priv->base + REG_USBPCR_OFFSET);
-}
-
-static const struct ingenic_soc_info jz4770_soc_info = {
- .version = ID_JZ4770,
-
- .usb_phy_init = jz4770_usb_phy_init,
-};
-
-static const struct ingenic_soc_info jz4780_soc_info = {
- .version = ID_JZ4780,
-
- .usb_phy_init = jz4780_usb_phy_init,
-};
-
-static const struct ingenic_soc_info x1000_soc_info = {
- .version = ID_X1000,
-
- .usb_phy_init = x1000_usb_phy_init,
-};
-
-static const struct ingenic_soc_info x1830_soc_info = {
- .version = ID_X1830,
-
- .usb_phy_init = x1830_usb_phy_init,
-};
-
-static const struct of_device_id ingenic_usb_phy_of_matches[] = {
- { .compatible = "ingenic,jz4770-phy", .data = &jz4770_soc_info },
- { .compatible = "ingenic,jz4780-phy", .data = &jz4780_soc_info },
- { .compatible = "ingenic,x1000-phy", .data = &x1000_soc_info },
- { .compatible = "ingenic,x1830-phy", .data = &x1830_soc_info },
- { /* sentinel */ }
-};
-MODULE_DEVICE_TABLE(of, ingenic_usb_phy_of_matches);
-
-static int jz4770_phy_probe(struct platform_device *pdev)
-{
- struct device *dev = &pdev->dev;
- struct jz4770_phy *priv;
- int err;
-
- priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
- if (!priv)
- return -ENOMEM;
-
- priv->soc_info = device_get_match_data(&pdev->dev);
- if (!priv->soc_info) {
- dev_err(&pdev->dev, "Error: No device match found\n");
- return -ENODEV;
- }
-
- platform_set_drvdata(pdev, priv);
- priv->dev = dev;
- priv->phy.dev = dev;
- priv->phy.otg = &priv->otg;
- priv->phy.label = "ingenic-usb-phy";
- priv->phy.init = ingenic_usb_phy_init;
- priv->phy.shutdown = ingenic_usb_phy_shutdown;
-
- priv->otg.state = OTG_STATE_UNDEFINED;
- priv->otg.usb_phy = &priv->phy;
- priv->otg.set_host = ingenic_usb_phy_set_host;
- priv->otg.set_peripheral = ingenic_usb_phy_set_peripheral;
-
- priv->base = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(priv->base)) {
- dev_err(dev, "Failed to map registers\n");
- return PTR_ERR(priv->base);
- }
-
- priv->clk = devm_clk_get(dev, NULL);
- if (IS_ERR(priv->clk))
- return dev_err_probe(dev, PTR_ERR(priv->clk),
- "Failed to get clock\n");
-
- priv->vcc_supply = devm_regulator_get(dev, "vcc");
- if (IS_ERR(priv->vcc_supply))
- return dev_err_probe(dev, PTR_ERR(priv->vcc_supply),
- "Failed to get regulator\n");
-
- err = usb_add_phy(&priv->phy, USB_PHY_TYPE_USB2);
- if (err)
- return dev_err_probe(dev, err, "Unable to register PHY\n");
-
- return devm_add_action_or_reset(dev, ingenic_usb_phy_remove, &priv->phy);
-}
-
-static struct platform_driver ingenic_phy_driver = {
- .probe = jz4770_phy_probe,
- .driver = {
- .name = "jz4770-phy",
- .of_match_table = ingenic_usb_phy_of_matches,
- },
-};
-module_platform_driver(ingenic_phy_driver);
-
-MODULE_AUTHOR("周琰杰 (Zhou Yanjie) <zhouyanjie@wanyeetech.com>");
-MODULE_AUTHOR("漆鹏振 (Qi Pengzhen) <aric.pzqi@ingenic.com>");
-MODULE_AUTHOR("Paul Cercueil <paul@crapouillou.net>");
-MODULE_DESCRIPTION("Ingenic SoCs USB PHY driver");
-MODULE_LICENSE("GPL");
}
EXPORT_SYMBOL_GPL(usb_role_switch_get_role);
-static void *usb_role_switch_match(struct fwnode_handle *fwnode, const char *id,
+static void *usb_role_switch_match(const struct fwnode_handle *fwnode, const char *id,
void *data)
{
struct device *dev;
struct fwnode_handle *parent = fwnode_get_parent(fwnode);
struct device *dev;
- if (!parent || !fwnode_property_present(parent, "usb-role-switch"))
+ if (!fwnode_property_present(parent, "usb-role-switch")) {
+ fwnode_handle_put(parent);
return NULL;
+ }
dev = class_find_device_by_fwnode(role_class, parent);
+ fwnode_handle_put(parent);
return dev ? to_role_switch(dev) : ERR_PTR(-EPROBE_DEFER);
}
{ USB_DEVICE(0x16DC, 0x0015) }, /* W-IE-NE-R Plein & Baus GmbH CML Control, Monitoring and Data Logger */
{ USB_DEVICE(0x17A8, 0x0001) }, /* Kamstrup Optical Eye/3-wire */
{ USB_DEVICE(0x17A8, 0x0005) }, /* Kamstrup M-Bus Master MultiPort 250D */
+ { USB_DEVICE(0x17A8, 0x0011) }, /* Kamstrup 444 MHz RF sniffer */
+ { USB_DEVICE(0x17A8, 0x0013) }, /* Kamstrup 870 MHz RF sniffer */
{ USB_DEVICE(0x17A8, 0x0101) }, /* Kamstrup 868 MHz wM-Bus C-Mode Meter Reader (Int Ant) */
{ USB_DEVICE(0x17A8, 0x0102) }, /* Kamstrup 868 MHz wM-Bus C-Mode Meter Reader (Ext Ant) */
{ USB_DEVICE(0x17F4, 0xAAAA) }, /* Wavesense Jazz blood glucose meter */
struct cp210x_serial_private *priv = usb_get_serial_data(serial);
u32 baud;
+ if (tty->termios.c_ospeed == 0)
+ return;
+
/*
* This maps the requested rate to the actual rate, a valid rate on
* cp2102 or cp2103, or to an arbitrary rate in [1M, max_speed].
- *
- * NOTE: B0 is not implemented.
*/
baud = clamp(tty->termios.c_ospeed, priv->min_speed, priv->max_speed);
tty->termios.c_iflag &= ~(IXON | IXOFF);
}
- if (old_termios &&
+ if (tty->termios.c_ospeed != 0 &&
+ old_termios && old_termios->c_ospeed != 0 &&
C_CRTSCTS(tty) == (old_termios->c_cflag & CRTSCTS) &&
I_IXON(tty) == (old_termios->c_iflag & IXON) &&
I_IXOFF(tty) == (old_termios->c_iflag & IXOFF) &&
mutex_lock(&port_priv->mutex);
+ if (tty->termios.c_ospeed == 0) {
+ port_priv->dtr = false;
+ port_priv->rts = false;
+ } else if (old_termios && old_termios->c_ospeed == 0) {
+ port_priv->dtr = true;
+ port_priv->rts = true;
+ }
+
ret = cp210x_read_reg_block(port, CP210X_GET_FLOW, &flow_ctl,
sizeof(flow_ctl));
if (ret)
u16 bits;
int ret;
- if (old_termios && !cp210x_termios_change(&tty->termios, old_termios))
+ if (old_termios && !cp210x_termios_change(&tty->termios, old_termios) &&
+ tty->termios.c_ospeed != 0)
return;
if (!old_termios || tty->termios.c_ospeed != old_termios->c_ospeed)
static int calc_baud_divisor(speed_t baudrate, speed_t clockrate)
{
- if (!baudrate)
- return 0;
-
return DIV_ROUND_CLOSEST(clockrate, baudrate);
}
speed_t baud_list[] = { baudrate, old_baudrate, F81232_DEF_BAUDRATE };
for (i = 0; i < ARRAY_SIZE(baud_list); ++i) {
- idx = f81232_find_clk(baud_list[i]);
+ baudrate = baud_list[i];
+ if (baudrate == 0) {
+ tty_encode_baud_rate(tty, 0, 0);
+ return;
+ }
+
+ idx = f81232_find_clk(baudrate);
if (idx >= 0) {
- baudrate = baud_list[i];
tty_encode_baud_rate(tty, baudrate, baudrate);
break;
}
static u32 f81534_calc_baud_divisor(u32 baudrate, u32 clockrate)
{
- if (!baudrate)
- return 0;
-
/* Round to nearest divisor */
return DIV_ROUND_CLOSEST(clockrate, baudrate);
}
u32 baud_list[] = {baudrate, old_baudrate, F81534_DEFAULT_BAUD_RATE};
for (i = 0; i < ARRAY_SIZE(baud_list); ++i) {
- idx = f81534_find_clk(baud_list[i]);
+ baudrate = baud_list[i];
+ if (baudrate == 0) {
+ tty_encode_baud_rate(tty, 0, 0);
+ return 0;
+ }
+
+ idx = f81534_find_clk(baudrate);
if (idx >= 0) {
- baudrate = baud_list[i];
tty_encode_baud_rate(tty, baudrate, baudrate);
break;
}
#define QUECTEL_PRODUCT_EP06 0x0306
#define QUECTEL_PRODUCT_EM05G 0x030a
#define QUECTEL_PRODUCT_EM060K 0x030b
+#define QUECTEL_PRODUCT_EM05G_SG 0x0311
#define QUECTEL_PRODUCT_EM12 0x0512
#define QUECTEL_PRODUCT_RM500Q 0x0800
#define QUECTEL_PRODUCT_RM520N 0x0801
{ USB_DEVICE_AND_INTERFACE_INFO(QUECTEL_VENDOR_ID, QUECTEL_PRODUCT_EP06, 0xff, 0, 0) },
{ USB_DEVICE_INTERFACE_CLASS(QUECTEL_VENDOR_ID, QUECTEL_PRODUCT_EM05G, 0xff),
.driver_info = RSVD(6) | ZLP },
+ { USB_DEVICE_INTERFACE_CLASS(QUECTEL_VENDOR_ID, QUECTEL_PRODUCT_EM05G_SG, 0xff),
+ .driver_info = RSVD(6) | ZLP },
{ USB_DEVICE_AND_INTERFACE_INFO(QUECTEL_VENDOR_ID, QUECTEL_PRODUCT_EM060K, 0xff, 0x00, 0x40) },
{ USB_DEVICE_AND_INTERFACE_INFO(QUECTEL_VENDOR_ID, QUECTEL_PRODUCT_EM060K, 0xff, 0xff, 0x30) },
{ USB_DEVICE_AND_INTERFACE_INFO(QUECTEL_VENDOR_ID, QUECTEL_PRODUCT_EM060K, 0xff, 0xff, 0x40) },
if (tty->termios.c_ospeed)
lc->dwDTERate = cpu_to_le32(tty->termios.c_ospeed);
- else if (old_termios)
- lc->dwDTERate = cpu_to_le32(old_termios->c_ospeed);
else
lc->dwDTERate = cpu_to_le32(9600);
+ MEDIA_INFO(us).blockshift + MEDIA_INFO(us).pageshift);
MEDIA_INFO(us).pba_to_lba = kcalloc(num_zones, sizeof(u16*), GFP_NOIO);
MEDIA_INFO(us).lba_to_pba = kcalloc(num_zones, sizeof(u16*), GFP_NOIO);
+ if (MEDIA_INFO(us).pba_to_lba == NULL || MEDIA_INFO(us).lba_to_pba == NULL)
+ return USB_STOR_TRANSPORT_ERROR;
if (alauda_reset_media(us) != USB_STOR_XFER_GOOD)
return USB_STOR_TRANSPORT_ERROR;
return PTR_ERR_OR_ZERO(ctx->psy);
}
-static int anx7411_i2c_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int anx7411_i2c_probe(struct i2c_client *client)
{
struct anx7411_data *plat;
struct device *dev = &client->dev;
.of_match_table = anx_match_table,
.pm = &anx7411_pm_ops,
},
- .probe = anx7411_i2c_probe,
+ .probe_new = anx7411_i2c_probe,
.remove = anx7411_i2c_remove,
.id_table = anx7411_id,
if (!adev || !adev->active)
return 0;
- if (!pdev->ops || !pdev->ops->enter)
+ if (!pdev->ops || !pdev->ops->exit)
return -EOPNOTSUPP;
/* Moving to USB Safe State */
EXPORT_SYMBOL_GPL(typec_partner_set_svdm_version);
/**
+ * typec_partner_usb_power_delivery_register - Register Type-C partner USB Power Delivery Support
+ * @partner: Type-C partner device.
+ * @desc: Description of the USB PD contract.
+ *
+ * This routine is a wrapper around usb_power_delivery_register(). It registers
+ * USB Power Delivery Capabilities for a Type-C partner device. Specifically,
+ * it sets the Type-C partner device as a parent for the resulting USB Power Delivery object.
+ *
+ * Returns handle to struct usb_power_delivery or ERR_PTR.
+ */
+struct usb_power_delivery *
+typec_partner_usb_power_delivery_register(struct typec_partner *partner,
+ struct usb_power_delivery_desc *desc)
+{
+ return usb_power_delivery_register(&partner->dev, desc);
+}
+EXPORT_SYMBOL_GPL(typec_partner_usb_power_delivery_register);
+
+/**
* typec_register_partner - Register a USB Type-C Partner
* @port: The USB Type-C Port the partner is connected to
* @desc: Description of the partner
.max_register = 0x0A,
};
-static int hd3ss3220_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int hd3ss3220_probe(struct i2c_client *client)
{
struct typec_capability typec_cap = { };
struct hd3ss3220 *hd3ss3220;
.name = "hd3ss3220",
.of_match_table = of_match_ptr(dev_ids),
},
- .probe = hd3ss3220_probe,
+ .probe_new = hd3ss3220_probe,
.remove = hd3ss3220_remove,
};
return device_match_fwnode(dev, fwnode);
}
-static void *typec_switch_match(struct fwnode_handle *fwnode, const char *id,
- void *data)
+static void *typec_switch_match(const struct fwnode_handle *fwnode,
+ const char *id, void *data)
{
struct device *dev;
return device_match_fwnode(dev, fwnode);
}
-static void *typec_mux_match(struct fwnode_handle *fwnode, const char *id,
- void *data)
+static void *typec_mux_match(const struct fwnode_handle *fwnode,
+ const char *id, void *data)
{
const struct typec_altmode_desc *desc = data;
struct device *dev;
#include "class.h"
#include "retimer.h"
-static bool dev_name_ends_with(struct device *dev, const char *suffix)
-{
- const char *name = dev_name(dev);
- const int name_len = strlen(name);
- const int suffix_len = strlen(suffix);
-
- if (suffix_len > name_len)
- return false;
-
- return strcmp(name + (name_len - suffix_len), suffix) == 0;
-}
-
static int retimer_fwnode_match(struct device *dev, const void *fwnode)
{
- return device_match_fwnode(dev, fwnode) && dev_name_ends_with(dev, "-retimer");
+ return is_typec_retimer(dev) && device_match_fwnode(dev, fwnode);
}
-static void *typec_retimer_match(struct fwnode_handle *fwnode, const char *id, void *data)
+static void *typec_retimer_match(const struct fwnode_handle *fwnode, const char *id, void *data)
{
struct device *dev;
kfree(to_typec_retimer(dev));
}
-static const struct device_type typec_retimer_dev_type = {
+const struct device_type typec_retimer_dev_type = {
.name = "typec_retimer",
.release = typec_retimer_release,
};
#define to_typec_retimer(_dev_) container_of(_dev_, struct typec_retimer, dev)
+const struct device_type typec_retimer_dev_type;
+
+#define is_typec_retimer(dev) ((dev)->type == &typec_retimer_dev_type)
+
#endif /* __USB_TYPEC_RETIMER__ */
return fwnode;
}
-static int fusb302_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int fusb302_probe(struct i2c_client *client)
{
struct fusb302_chip *chip;
struct i2c_adapter *adapter = client->adapter;
.pm = &fusb302_pm_ops,
.of_match_table = of_match_ptr(fusb302_dt_match),
},
- .probe = fusb302_probe,
+ .probe_new = fusb302_probe,
.remove = fusb302_remove,
.id_table = fusb302_i2c_device_id,
};
return ERR_PTR(err);
tcpci->port = tcpm_register_port(tcpci->dev, &tcpci->tcpc);
- if (IS_ERR(tcpci->port))
+ if (IS_ERR(tcpci->port)) {
+ fwnode_handle_put(tcpci->tcpc.fwnode);
return ERR_CAST(tcpci->port);
+ }
return tcpci;
}
void tcpci_unregister_port(struct tcpci *tcpci)
{
tcpm_unregister_port(tcpci->port);
+ fwnode_handle_put(tcpci->tcpc.fwnode);
}
EXPORT_SYMBOL_GPL(tcpci_unregister_port);
-static int tcpci_probe(struct i2c_client *client,
- const struct i2c_device_id *i2c_id)
+static int tcpci_probe(struct i2c_client *client)
{
struct tcpci_chip *chip;
int err;
.name = "tcpci",
.of_match_table = of_match_ptr(tcpci_of_match),
},
- .probe = tcpci_probe,
+ .probe_new = tcpci_probe,
.remove = tcpci_remove,
.id_table = tcpci_id,
};
return -1;
}
-static int max_tcpci_probe(struct i2c_client *client, const struct i2c_device_id *i2c_id)
+static int max_tcpci_probe(struct i2c_client *client)
{
int ret;
struct max_tcpci_chip *chip;
.name = "maxtcpc",
.of_match_table = of_match_ptr(max_tcpci_of_match),
},
- .probe = max_tcpci_probe,
+ .probe_new = max_tcpci_probe,
.remove = max_tcpci_remove,
.id_table = max_tcpci_id,
};
return ret;
}
-static int rt1711h_probe(struct i2c_client *client,
- const struct i2c_device_id *i2c_id)
+static int rt1711h_probe(struct i2c_client *client)
{
int ret;
struct rt1711h_chip *chip;
.name = "rt1711h",
.of_match_table = of_match_ptr(rt1711h_of_match),
},
- .probe = rt1711h_probe,
+ .probe_new = rt1711h_probe,
.remove = rt1711h_remove,
.id_table = rt1711h_id,
};
#include <linux/regmap.h>
#include <linux/interrupt.h>
#include <linux/usb/typec.h>
+#include <linux/usb/typec_altmode.h>
#include <linux/usb/role.h>
#include "tps6598x.h"
typec_set_orientation(tps->port, TYPEC_ORIENTATION_REVERSE);
else
typec_set_orientation(tps->port, TYPEC_ORIENTATION_NORMAL);
+ typec_set_mode(tps->port, TYPEC_STATE_USB);
tps6598x_set_data_role(tps, TPS_STATUS_TO_TYPEC_DATAROLE(status), true);
tps->partner = typec_register_partner(tps->port, &desc);
typec_set_pwr_role(tps->port, TPS_STATUS_TO_TYPEC_PORTROLE(status));
typec_set_vconn_role(tps->port, TPS_STATUS_TO_TYPEC_VCONN(status));
typec_set_orientation(tps->port, TYPEC_ORIENTATION_NONE);
+ typec_set_mode(tps->port, TYPEC_STATE_SAFE);
tps6598x_set_data_role(tps, TPS_STATUS_TO_TYPEC_DATAROLE(status), false);
power_supply_changed(tps->psy);
ret = devm_tps6598_psy_register(tps);
if (ret)
- return ret;
+ goto err_role_put;
tps->port = typec_register_port(&client->dev, &typec_cap);
if (IS_ERR(tps->port)) {
ret = PTR_ERR(tps->port);
goto err_role_put;
}
- fwnode_handle_put(fwnode);
if (status & TPS_STATUS_PLUG_PRESENT) {
ret = tps6598x_read16(tps, TPS_REG_POWER_STATUS, &tps->pwr_status);
if (ret < 0) {
dev_err(tps->dev, "failed to read power status: %d\n", ret);
- goto err_role_put;
+ goto err_unregister_port;
}
ret = tps6598x_connect(tps, status);
if (ret)
irq_handler,
IRQF_SHARED | IRQF_ONESHOT,
dev_name(&client->dev), tps);
- if (ret) {
- tps6598x_disconnect(tps, 0);
- typec_unregister_port(tps->port);
- goto err_role_put;
- }
+ if (ret)
+ goto err_disconnect;
i2c_set_clientdata(client, tps);
+ fwnode_handle_put(fwnode);
return 0;
+err_disconnect:
+ tps6598x_disconnect(tps, 0);
+err_unregister_port:
+ typec_unregister_port(tps->port);
err_role_put:
usb_role_switch_put(tps->role_sw);
err_fwnode_put:
return ret;
}
-int ucsi_resume(struct ucsi *ucsi)
+static void ucsi_resume_work(struct work_struct *work)
{
+ struct ucsi *ucsi = container_of(work, struct ucsi, resume_work);
struct ucsi_connector *con;
u64 command;
int ret;
/* Restore UCSI notification enable mask after system resume */
command = UCSI_SET_NOTIFICATION_ENABLE | ucsi->ntfy;
ret = ucsi_send_command(ucsi, command, NULL, 0);
- if (ret < 0)
- return ret;
+ if (ret < 0) {
+ dev_err(ucsi->dev, "failed to re-enable notifications (%d)\n", ret);
+ return;
+ }
for (con = ucsi->connector; con->port; con++) {
mutex_lock(&con->lock);
- ucsi_check_connection(con);
+ ucsi_partner_task(con, ucsi_check_connection, 1, 0);
mutex_unlock(&con->lock);
}
+}
+int ucsi_resume(struct ucsi *ucsi)
+{
+ queue_work(system_long_wq, &ucsi->resume_work);
return 0;
}
EXPORT_SYMBOL_GPL(ucsi_resume);
if (!ucsi)
return ERR_PTR(-ENOMEM);
+ INIT_WORK(&ucsi->resume_work, ucsi_resume_work);
INIT_DELAYED_WORK(&ucsi->work, ucsi_init_work);
mutex_init(&ucsi->ppm_lock);
ucsi->dev = dev;
/* Make sure that we are not in the middle of driver initialization */
cancel_delayed_work_sync(&ucsi->work);
+ cancel_work_sync(&ucsi->resume_work);
/* Disable notifications */
ucsi->ops->async_write(ucsi, UCSI_CONTROL, &cmd, sizeof(cmd));
struct ucsi_capability cap;
struct ucsi_connector *connector;
+ struct work_struct resume_work;
struct delayed_work work;
int work_count;
#define UCSI_ROLE_SWITCH_RETRY_PER_HZ 10
};
ATTRIBUTE_GROUPS(ucsi_ccg);
-static int ucsi_ccg_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int ucsi_ccg_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
struct ucsi_ccg *uc;
.dev_groups = ucsi_ccg_groups,
.acpi_match_table = amd_i2c_ucsi_match,
},
- .probe = ucsi_ccg_probe,
+ .probe_new = ucsi_ccg_probe,
.remove = ucsi_ccg_remove,
.id_table = ucsi_ccg_device_id,
};
return 0;
}
-static int ucsi_stm32g0_probe(struct i2c_client *client, const struct i2c_device_id *id)
+static int ucsi_stm32g0_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
struct ucsi_stm32g0 *g0;
.of_match_table = of_match_ptr(ucsi_stm32g0_typec_of_match),
.pm = pm_sleep_ptr(&ucsi_stm32g0_pm_ops),
},
- .probe = ucsi_stm32g0_probe,
+ .probe_new = ucsi_stm32g0_probe,
.remove = ucsi_stm32g0_remove,
.id_table = ucsi_stm32g0_typec_i2c_devid
};
/* Initialize the hardware with the devicetree settings. */
ret = wusb3801_hw_init(wusb3801);
if (ret)
- return ret;
+ goto err_put_connector;
wusb3801->cap.revision = USB_TYPEC_REV_1_2;
wusb3801->cap.accessory[0] = TYPEC_ACCESSORY_AUDIO;
int minor = interface->minor;
dev = usb_get_intfdata(interface);
- usb_set_intfdata(interface, NULL);
/* give back our minor */
usb_deregister_dev(interface, &skel_class);
status = sdev->ud.status;
spin_unlock_irq(&sdev->ud.lock);
- return snprintf(buf, PAGE_SIZE, "%d\n", status);
+ return sysfs_emit(buf, "%d\n", status);
}
static DEVICE_ATTR_RO(usbip_status);
} else {
dev_info(dev, "stub down\n");
+ mutex_lock(&sdev->ud.sysfs_lock);
+
spin_lock_irq(&sdev->ud.lock);
if (sdev->ud.status != SDEV_ST_USED)
goto err;
do {
sock->sk->sk_allocation = GFP_NOIO;
+ sock->sk->sk_use_task_frag = false;
result = sock_recvmsg(sock, &msg, MSG_WAITALL);
if (result <= 0)
urb_p->urb->status = -EINPROGRESS;
/* FIXME: more pipe setup to please usbip_common */
- urb_p->urb->pipe &= ~(3 << 30);
+ BUILD_BUG_ON_MSG(PIPE_BULK != 3, "PIPE_* doesn't range from 0 to 3");
+
+ urb_p->urb->pipe &= ~(PIPE_BULK << 30);
switch (urb_p->ep->type) {
case USB_ENDPOINT_XFER_BULK:
urb_p->urb->pipe |= (PIPE_BULK << 30);
status = udc->ud.status;
spin_unlock_irq(&udc->ud.lock);
- return snprintf(out, PAGE_SIZE, "%d\n", status);
+ return sysfs_emit(out, "%d\n", status);
}
static DEVICE_ATTR_RO(usbip_status);
.llseek = noop_llseek,
};
-static char *vduse_devnode(struct device *dev, umode_t *mode)
+static char *vduse_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "vduse/%s", dev_name(dev));
}
}
EXPORT_SYMBOL_GPL(vfio_file_has_dev);
-static char *vfio_devnode(struct device *dev, umode_t *mode)
+static char *vfio_devnode(const struct device *dev, umode_t *mode)
{
return kasprintf(GFP_KERNEL, "vfio/%s", dev_name(dev));
}
.attrs = adp8860_bl_attributes,
};
-static int adp8860_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int adp8860_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct backlight_device *bl;
struct adp8860_bl *data;
struct adp8860_backlight_platform_data *pdata =
.name = KBUILD_MODNAME,
.pm = &adp8860_i2c_pm_ops,
},
- .probe = adp8860_probe,
+ .probe_new = adp8860_probe,
.remove = adp8860_remove,
.id_table = adp8860_id,
};
.attrs = adp8870_bl_attributes,
};
-static int adp8870_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int adp8870_probe(struct i2c_client *client)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct backlight_properties props;
struct backlight_device *bl;
struct adp8870_bl *data;
.name = KBUILD_MODNAME,
.pm = &adp8870_i2c_pm_ops,
},
- .probe = adp8870_probe,
+ .probe_new = adp8870_probe,
.remove = adp8870_remove,
.id_table = adp8870_id,
};
}
}
-static int arcxcnn_probe(struct i2c_client *cl, const struct i2c_device_id *id)
+static int arcxcnn_probe(struct i2c_client *cl)
{
struct arcxcnn *lp;
int ret;
.name = "arcxcnn_bl",
.of_match_table = of_match_ptr(arcxcnn_dt_ids),
},
- .probe = arcxcnn_probe,
+ .probe_new = arcxcnn_probe,
.remove = arcxcnn_remove,
.id_table = arcxcnn_ids,
};
.check_fb = bd6107_backlight_check_fb,
};
-static int bd6107_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int bd6107_probe(struct i2c_client *client)
{
struct bd6107_platform_data *pdata = dev_get_platdata(&client->dev);
struct backlight_device *backlight;
.driver = {
.name = "bd6107",
},
- .probe = bd6107_probe,
+ .probe_new = bd6107_probe,
.remove = bd6107_remove,
.id_table = bd6107_ids,
};
return ret;
}
-static int lm3630a_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int lm3630a_probe(struct i2c_client *client)
{
struct lm3630a_platform_data *pdata = dev_get_platdata(&client->dev);
struct lm3630a_chip *pchip;
.name = LM3630A_NAME,
.of_match_table = lm3630a_match_table,
},
- .probe = lm3630a_probe,
+ .probe_new = lm3630a_probe,
.remove = lm3630a_remove,
.id_table = lm3630a_id,
};
.max_register = REG_MAX,
};
-static int lm3639_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int lm3639_probe(struct i2c_client *client)
{
int ret;
struct lm3639_chip_data *pchip;
.driver = {
.name = LM3639_NAME,
},
- .probe = lm3639_probe,
+ .probe_new = lm3639_probe,
.remove = lm3639_remove,
.id_table = lm3639_id,
};
return 0;
}
-static int lp855x_probe(struct i2c_client *cl, const struct i2c_device_id *id)
+static int lp855x_probe(struct i2c_client *cl)
{
+ const struct i2c_device_id *id = i2c_client_get_device_id(cl);
const struct acpi_device_id *acpi_id = NULL;
struct device *dev = &cl->dev;
struct lp855x *lp;
.of_match_table = of_match_ptr(lp855x_dt_ids),
.acpi_match_table = ACPI_PTR(lp855x_acpi_match),
},
- .probe = lp855x_probe,
+ .probe_new = lp855x_probe,
.remove = lp855x_remove,
.id_table = lp855x_ids,
};
.check_fb = lv5207lp_backlight_check_fb,
};
-static int lv5207lp_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int lv5207lp_probe(struct i2c_client *client)
{
struct lv5207lp_platform_data *pdata = dev_get_platdata(&client->dev);
struct backlight_device *backlight;
.driver = {
.name = "lv5207lp",
},
- .probe = lv5207lp_probe,
+ .probe_new = lv5207lp_probe,
.remove = lv5207lp_remove,
.id_table = lv5207lp_ids,
};
.update_status = tosa_bl_update_status,
};
-static int tosa_bl_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+static int tosa_bl_probe(struct i2c_client *client)
{
struct backlight_properties props;
struct tosa_bl_data *data;
.name = "tosa-bl",
.pm = &tosa_bl_pm_ops,
},
- .probe = tosa_bl_probe,
+ .probe_new = tosa_bl_probe,
.remove = tosa_bl_remove,
.id_table = tosa_bl_id,
};
handle = of_get_property(np, "interrupts", NULL);
irq = irq_of_parse_and_map(np, 0);
- if (!handle || (irq == NO_IRQ)) {
+ if (!handle || !irq) {
pr_err("fsl-hv: no 'interrupts' property in %pOF node\n",
np);
continue;
feature. More information can be found at
<https://www.advantech.com.tw/products/>
+config ADVANTECH_EC_WDT
+ tristate "Advantech Embedded Controller Watchdog Timer"
+ depends on X86
+ help
+ This driver supports Advantech products with ITE based Embedded Controller.
+ It does not support Advantech products with other ECs or without EC.
+
config ALIM1535_WDT
tristate "ALi M1535 PMU Watchdog Timer"
depends on X86 && PCI
# X86 (i386 + ia64 + x86_64) Architecture
obj-$(CONFIG_ACQUIRE_WDT) += acquirewdt.o
obj-$(CONFIG_ADVANTECH_WDT) += advantechwdt.o
+obj-$(CONFIG_ADVANTECH_EC_WDT) += advantech_ec_wdt.o
obj-$(CONFIG_ALIM1535_WDT) += alim1535_wdt.o
obj-$(CONFIG_ALIM7101_WDT) += alim7101_wdt.o
obj-$(CONFIG_EBC_C384_WDT) += ebc-c384_wdt.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Advantech Embedded Controller Watchdog Driver
+ *
+ * This driver supports Advantech products with ITE based Embedded Controller.
+ * It does not support Advantech products with other ECs or without EC.
+ *
+ * Copyright (C) 2022 Advantech Europe B.V.
+ */
+
+#include <linux/delay.h>
+#include <linux/io.h>
+#include <linux/isa.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/watchdog.h>
+
+#define DRIVER_NAME "advantech_ec_wdt"
+
+/* EC IO region */
+#define EC_BASE_ADDR 0x299
+#define EC_ADDR_EXTENT 2
+
+/* EC minimum IO access delay in ms */
+#define EC_MIN_DELAY 10
+
+/* EC interface definitions */
+#define EC_ADDR_CMD (EC_BASE_ADDR + 1)
+#define EC_ADDR_DATA EC_BASE_ADDR
+#define EC_CMD_EC_PROBE 0x30
+#define EC_CMD_COMM 0x89
+#define EC_CMD_WDT_START 0x28
+#define EC_CMD_WDT_STOP 0x29
+#define EC_CMD_WDT_RESET 0x2A
+#define EC_DAT_EN_DLY_H 0x58
+#define EC_DAT_EN_DLY_L 0x59
+#define EC_DAT_RST_DLY_H 0x5E
+#define EC_DAT_RST_DLY_L 0x5F
+#define EC_MAGIC 0x95
+
+/* module parameters */
+#define MIN_TIME 1
+#define MAX_TIME 6000 /* 100 minutes */
+#define DEFAULT_TIME 60
+
+static unsigned int timeout;
+static ktime_t ec_timestamp;
+
+module_param(timeout, uint, 0);
+MODULE_PARM_DESC(timeout,
+ "Default Watchdog timer setting (" __MODULE_STRING(DEFAULT_TIME) "s). The range is from " __MODULE_STRING(MIN_TIME) " to " __MODULE_STRING(MAX_TIME) ".");
+
+static void adv_ec_wdt_timing_gate(void)
+{
+ ktime_t time_cur, time_delta;
+
+ /* ensure minimum delay between IO accesses*/
+ time_cur = ktime_get();
+ time_delta = ktime_to_ms(ktime_sub(time_cur, ec_timestamp));
+ if (time_delta < EC_MIN_DELAY) {
+ time_delta = EC_MIN_DELAY - time_delta;
+ usleep_range(time_delta * 1000, (time_delta + 1) * 1000);
+ }
+ ec_timestamp = ktime_get();
+}
+
+static void adv_ec_wdt_outb(unsigned char value, unsigned short port)
+{
+ adv_ec_wdt_timing_gate();
+ outb(value, port);
+}
+
+static unsigned char adv_ec_wdt_inb(unsigned short port)
+{
+ adv_ec_wdt_timing_gate();
+ return inb(port);
+}
+
+static int adv_ec_wdt_ping(struct watchdog_device *wdd)
+{
+ adv_ec_wdt_outb(EC_CMD_WDT_RESET, EC_ADDR_CMD);
+ return 0;
+}
+
+static int adv_ec_wdt_set_timeout(struct watchdog_device *wdd, unsigned int t)
+{
+ unsigned int val;
+
+ /* scale time to EC 100 ms base */
+ val = t * 10;
+
+ /* reset enable delay, just in case it was set by BIOS etc. */
+ adv_ec_wdt_outb(EC_CMD_COMM, EC_ADDR_CMD);
+ adv_ec_wdt_outb(EC_DAT_EN_DLY_H, EC_ADDR_DATA);
+ adv_ec_wdt_outb(0, EC_ADDR_DATA);
+
+ adv_ec_wdt_outb(EC_CMD_COMM, EC_ADDR_CMD);
+ adv_ec_wdt_outb(EC_DAT_EN_DLY_L, EC_ADDR_DATA);
+ adv_ec_wdt_outb(0, EC_ADDR_DATA);
+
+ /* set reset delay */
+ adv_ec_wdt_outb(EC_CMD_COMM, EC_ADDR_CMD);
+ adv_ec_wdt_outb(EC_DAT_RST_DLY_H, EC_ADDR_DATA);
+ adv_ec_wdt_outb(val >> 8, EC_ADDR_DATA);
+
+ adv_ec_wdt_outb(EC_CMD_COMM, EC_ADDR_CMD);
+ adv_ec_wdt_outb(EC_DAT_RST_DLY_L, EC_ADDR_DATA);
+ adv_ec_wdt_outb(val & 0xFF, EC_ADDR_DATA);
+
+ wdd->timeout = t;
+ return 0;
+}
+
+static int adv_ec_wdt_start(struct watchdog_device *wdd)
+{
+ adv_ec_wdt_set_timeout(wdd, wdd->timeout);
+ adv_ec_wdt_outb(EC_CMD_WDT_START, EC_ADDR_CMD);
+
+ return 0;
+}
+
+static int adv_ec_wdt_stop(struct watchdog_device *wdd)
+{
+ adv_ec_wdt_outb(EC_CMD_WDT_STOP, EC_ADDR_CMD);
+
+ return 0;
+}
+
+static const struct watchdog_info adv_ec_wdt_info = {
+ .identity = DRIVER_NAME,
+ .options = WDIOF_SETTIMEOUT |
+ WDIOF_MAGICCLOSE |
+ WDIOF_KEEPALIVEPING,
+};
+
+static const struct watchdog_ops adv_ec_wdt_ops = {
+ .owner = THIS_MODULE,
+ .start = adv_ec_wdt_start,
+ .stop = adv_ec_wdt_stop,
+ .ping = adv_ec_wdt_ping,
+ .set_timeout = adv_ec_wdt_set_timeout,
+};
+
+static struct watchdog_device adv_ec_wdt_dev = {
+ .info = &adv_ec_wdt_info,
+ .ops = &adv_ec_wdt_ops,
+ .min_timeout = MIN_TIME,
+ .max_timeout = MAX_TIME,
+ .timeout = DEFAULT_TIME,
+};
+
+static int adv_ec_wdt_probe(struct device *dev, unsigned int id)
+{
+ if (!devm_request_region(dev, EC_BASE_ADDR, EC_ADDR_EXTENT, dev_name(dev))) {
+ dev_err(dev, "Unable to lock port addresses (0x%X-0x%X)\n",
+ EC_BASE_ADDR, EC_BASE_ADDR + EC_ADDR_EXTENT);
+ return -EBUSY;
+ }
+
+ watchdog_init_timeout(&adv_ec_wdt_dev, timeout, dev);
+ watchdog_stop_on_reboot(&adv_ec_wdt_dev);
+ watchdog_stop_on_unregister(&adv_ec_wdt_dev);
+
+ return devm_watchdog_register_device(dev, &adv_ec_wdt_dev);
+}
+
+static struct isa_driver adv_ec_wdt_driver = {
+ .probe = adv_ec_wdt_probe,
+ .driver = {
+ .name = DRIVER_NAME,
+ },
+};
+
+static int __init adv_ec_wdt_init(void)
+{
+ unsigned int val;
+
+ /* quick probe for EC */
+ if (!request_region(EC_BASE_ADDR, EC_ADDR_EXTENT, DRIVER_NAME))
+ return -EBUSY;
+
+ adv_ec_wdt_outb(EC_CMD_EC_PROBE, EC_ADDR_CMD);
+ val = adv_ec_wdt_inb(EC_ADDR_DATA);
+ release_region(EC_BASE_ADDR, EC_ADDR_EXTENT);
+
+ if (val != EC_MAGIC)
+ return -ENODEV;
+
+ return isa_register_driver(&adv_ec_wdt_driver, 1);
+}
+
+static void __exit adv_ec_wdt_exit(void)
+{
+ isa_unregister_driver(&adv_ec_wdt_driver);
+}
+
+module_init(adv_ec_wdt_init);
+module_exit(adv_ec_wdt_exit);
+
+MODULE_AUTHOR("Thomas Kastner <thomas.kastner@advantech.com>");
+MODULE_DESCRIPTION("Advantech Embedded Controller Watchdog Device Driver");
+MODULE_LICENSE("GPL");
+MODULE_VERSION("20221019");
+MODULE_ALIAS("isa:" DRIVER_NAME);
* Joel Stanley <joel@jms.id.au>
*/
+#include <linux/bits.h>
#include <linux/delay.h>
+#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/of.h>
+#include <linux/of_irq.h>
#include <linux/platform_device.h>
#include <linux/watchdog.h>
MODULE_PARM_DESC(nowayout, "Watchdog cannot be stopped once started (default="
__MODULE_STRING(WATCHDOG_NOWAYOUT) ")");
+struct aspeed_wdt_config {
+ u32 ext_pulse_width_mask;
+ u32 irq_shift;
+ u32 irq_mask;
+};
+
struct aspeed_wdt {
struct watchdog_device wdd;
void __iomem *base;
u32 ctrl;
-};
-
-struct aspeed_wdt_config {
- u32 ext_pulse_width_mask;
+ const struct aspeed_wdt_config *cfg;
};
static const struct aspeed_wdt_config ast2400_config = {
.ext_pulse_width_mask = 0xff,
+ .irq_shift = 0,
+ .irq_mask = 0,
};
static const struct aspeed_wdt_config ast2500_config = {
.ext_pulse_width_mask = 0xfffff,
+ .irq_shift = 12,
+ .irq_mask = GENMASK(31, 12),
+};
+
+static const struct aspeed_wdt_config ast2600_config = {
+ .ext_pulse_width_mask = 0xfffff,
+ .irq_shift = 0,
+ .irq_mask = GENMASK(31, 10),
};
static const struct of_device_id aspeed_wdt_of_table[] = {
{ .compatible = "aspeed,ast2400-wdt", .data = &ast2400_config },
{ .compatible = "aspeed,ast2500-wdt", .data = &ast2500_config },
- { .compatible = "aspeed,ast2600-wdt", .data = &ast2500_config },
+ { .compatible = "aspeed,ast2600-wdt", .data = &ast2600_config },
{ },
};
MODULE_DEVICE_TABLE(of, aspeed_wdt_of_table);
#define WDT_CTRL_RESET_SYSTEM BIT(1)
#define WDT_CTRL_ENABLE BIT(0)
#define WDT_TIMEOUT_STATUS 0x10
+#define WDT_TIMEOUT_STATUS_IRQ BIT(2)
#define WDT_TIMEOUT_STATUS_BOOT_SECONDARY BIT(1)
#define WDT_CLEAR_TIMEOUT_STATUS 0x14
#define WDT_CLEAR_TIMEOUT_AND_BOOT_CODE_SELECTION BIT(0)
return 0;
}
+static int aspeed_wdt_set_pretimeout(struct watchdog_device *wdd,
+ unsigned int pretimeout)
+{
+ struct aspeed_wdt *wdt = to_aspeed_wdt(wdd);
+ u32 actual = pretimeout * WDT_RATE_1MHZ;
+ u32 s = wdt->cfg->irq_shift;
+ u32 m = wdt->cfg->irq_mask;
+
+ wdd->pretimeout = pretimeout;
+ wdt->ctrl &= ~m;
+ if (pretimeout)
+ wdt->ctrl |= ((actual << s) & m) | WDT_CTRL_WDT_INTR;
+ else
+ wdt->ctrl &= ~WDT_CTRL_WDT_INTR;
+
+ writel(wdt->ctrl, wdt->base + WDT_CTRL);
+
+ return 0;
+}
+
static int aspeed_wdt_restart(struct watchdog_device *wdd,
unsigned long action, void *data)
{
.stop = aspeed_wdt_stop,
.ping = aspeed_wdt_ping,
.set_timeout = aspeed_wdt_set_timeout,
+ .set_pretimeout = aspeed_wdt_set_pretimeout,
.restart = aspeed_wdt_restart,
.owner = THIS_MODULE,
};
.identity = KBUILD_MODNAME,
};
+static const struct watchdog_info aspeed_wdt_pretimeout_info = {
+ .options = WDIOF_KEEPALIVEPING
+ | WDIOF_PRETIMEOUT
+ | WDIOF_MAGICCLOSE
+ | WDIOF_SETTIMEOUT,
+ .identity = KBUILD_MODNAME,
+};
+
+static irqreturn_t aspeed_wdt_irq(int irq, void *arg)
+{
+ struct watchdog_device *wdd = arg;
+ struct aspeed_wdt *wdt = to_aspeed_wdt(wdd);
+ u32 status = readl(wdt->base + WDT_TIMEOUT_STATUS);
+
+ if (status & WDT_TIMEOUT_STATUS_IRQ)
+ watchdog_notify_pretimeout(wdd);
+
+ return IRQ_HANDLED;
+}
+
static int aspeed_wdt_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
- const struct aspeed_wdt_config *config;
const struct of_device_id *ofdid;
struct aspeed_wdt *wdt;
struct device_node *np;
if (!wdt)
return -ENOMEM;
+ np = dev->of_node;
+
+ ofdid = of_match_node(aspeed_wdt_of_table, np);
+ if (!ofdid)
+ return -EINVAL;
+ wdt->cfg = ofdid->data;
+
wdt->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(wdt->base))
return PTR_ERR(wdt->base);
wdt->wdd.info = &aspeed_wdt_info;
+
+ if (wdt->cfg->irq_mask) {
+ int irq = platform_get_irq_optional(pdev, 0);
+
+ if (irq > 0) {
+ ret = devm_request_irq(dev, irq, aspeed_wdt_irq,
+ IRQF_SHARED, dev_name(dev),
+ wdt);
+ if (ret)
+ return ret;
+
+ wdt->wdd.info = &aspeed_wdt_pretimeout_info;
+ }
+ }
+
wdt->wdd.ops = &aspeed_wdt_ops;
wdt->wdd.max_hw_heartbeat_ms = WDT_MAX_TIMEOUT_MS;
wdt->wdd.parent = dev;
watchdog_set_nowayout(&wdt->wdd, nowayout);
- np = dev->of_node;
-
- ofdid = of_match_node(aspeed_wdt_of_table, np);
- if (!ofdid)
- return -EINVAL;
- config = ofdid->data;
-
/*
* On clock rates:
* - ast2400 wdt can run at PCLK, or 1MHz
(of_device_is_compatible(np, "aspeed,ast2600-wdt"))) {
u32 reg = readl(wdt->base + WDT_RESET_WIDTH);
- reg &= config->ext_pulse_width_mask;
+ reg &= wdt->cfg->ext_pulse_width_mask;
if (of_property_read_bool(np, "aspeed,ext-active-high"))
reg |= WDT_ACTIVE_HIGH_MAGIC;
else
writel(reg, wdt->base + WDT_RESET_WIDTH);
- reg &= config->ext_pulse_width_mask;
+ reg &= wdt->cfg->ext_pulse_width_mask;
if (of_property_read_bool(np, "aspeed,ext-push-pull"))
reg |= WDT_PUSH_PULL_MAGIC;
else
}
if (!of_property_read_u32(np, "aspeed,ext-pulse-duration", &duration)) {
- u32 max_duration = config->ext_pulse_width_mask + 1;
+ u32 max_duration = wdt->cfg->ext_pulse_width_mask + 1;
if (duration == 0 || duration > max_duration) {
dev_err(dev, "Invalid pulse duration: %uus\n",
at91_wdt_stop();
}
-#ifdef CONFIG_PM
-
static int at91wdt_suspend(struct platform_device *pdev, pm_message_t message)
{
at91_wdt_stop();
return 0;
}
-#else
-#define at91wdt_suspend NULL
-#define at91wdt_resume NULL
-#endif
-
static const struct of_device_id at91_wdt_dt_ids[] = {
{ .compatible = "atmel,at91rm9200-wdt" },
{ /* sentinel */ }
.probe = at91wdt_probe,
.remove = at91wdt_remove,
.shutdown = at91wdt_shutdown,
- .suspend = at91wdt_suspend,
- .resume = at91wdt_resume,
+ .suspend = pm_ptr(at91wdt_suspend),
+ .resume = pm_ptr(at91wdt_resume),
.driver = {
.name = "atmel_st_watchdog",
.of_match_table = at91_wdt_dt_ids,
return 0;
}
-#ifdef CONFIG_PM
static int db8500_wdt_suspend(struct platform_device *pdev,
pm_message_t state)
{
}
return 0;
}
-#else
-#define db8500_wdt_suspend NULL
-#define db8500_wdt_resume NULL
-#endif
static struct platform_driver db8500_wdt_driver = {
.probe = db8500_wdt_probe,
- .suspend = db8500_wdt_suspend,
- .resume = db8500_wdt_resume,
+ .suspend = pm_ptr(db8500_wdt_suspend),
+ .resume = pm_ptr(db8500_wdt_resume),
.driver = {
.name = "db8500_wdt",
},
return time_left;
}
-static void iTCO_wdt_set_running(struct iTCO_wdt_private *p)
+/* Returns true if the watchdog was running */
+static bool iTCO_wdt_set_running(struct iTCO_wdt_private *p)
{
u16 val;
- /* Bit 11: TCO Timer Halt -> 0 = The TCO timer is * enabled */
+ /* Bit 11: TCO Timer Halt -> 0 = The TCO timer is enabled */
val = inw(TCO1_CNT(p));
- if (!(val & BIT(11)))
+ if (!(val & BIT(11))) {
set_bit(WDOG_HW_RUNNING, &p->wddev.status);
+ return true;
+ }
+ return false;
}
/*
return -ENODEV; /* Cannot reset NO_REBOOT bit */
}
- /* Set the NO_REBOOT bit to prevent later reboots, just for sure */
- p->update_no_reboot_bit(p->no_reboot_priv, true);
-
if (turn_SMI_watchdog_clear_off >= p->iTCO_version) {
/*
* Bit 13: TCO_EN -> 0
watchdog_set_drvdata(&p->wddev, p);
platform_set_drvdata(pdev, p);
- iTCO_wdt_set_running(p);
+ if (!iTCO_wdt_set_running(p)) {
+ /*
+ * If the watchdog was not running set NO_REBOOT now to
+ * prevent later reboots.
+ */
+ p->update_no_reboot_bit(p->no_reboot_priv, true);
+ }
/* Check that the heartbeat value is within it's range;
if not reset to the default */
struct watchdog_device wdd;
unsigned int pretimeout;
struct kempld_wdt_stage stage[KEMPLD_WDT_MAX_STAGES];
-#ifdef CONFIG_PM
u8 pm_status_store;
-#endif
};
#define DEFAULT_TIMEOUT 30 /* seconds */
return 0;
}
-#ifdef CONFIG_PM
/* Disable watchdog if it is active during suspend */
static int kempld_wdt_suspend(struct platform_device *pdev,
pm_message_t message)
else
return kempld_wdt_stop(wdd);
}
-#else
-#define kempld_wdt_suspend NULL
-#define kempld_wdt_resume NULL
-#endif
static struct platform_driver kempld_wdt_driver = {
.driver = {
.name = "kempld-wdt",
},
.probe = kempld_wdt_probe,
- .suspend = kempld_wdt_suspend,
- .resume = kempld_wdt_resume,
+ .suspend = pm_ptr(kempld_wdt_suspend),
+ .resume = pm_ptr(kempld_wdt_resume),
};
module_platform_driver(kempld_wdt_driver);
*/
#include <dt-bindings/reset/mt2712-resets.h>
+#include <dt-bindings/reset/mediatek,mt6795-resets.h>
#include <dt-bindings/reset/mt7986-resets.h>
#include <dt-bindings/reset/mt8183-resets.h>
#include <dt-bindings/reset/mt8186-resets.h>
+#include <dt-bindings/reset/mt8188-resets.h>
#include <dt-bindings/reset/mt8192-resets.h>
#include <dt-bindings/reset/mt8195-resets.h>
#include <linux/delay.h>
.toprgu_sw_rst_num = MT2712_TOPRGU_SW_RST_NUM,
};
+static const struct mtk_wdt_data mt6795_data = {
+ .toprgu_sw_rst_num = MT6795_TOPRGU_SW_RST_NUM,
+};
+
static const struct mtk_wdt_data mt7986_data = {
.toprgu_sw_rst_num = MT7986_TOPRGU_SW_RST_NUM,
};
.toprgu_sw_rst_num = MT8186_TOPRGU_SW_RST_NUM,
};
+static const struct mtk_wdt_data mt8188_data = {
+ .toprgu_sw_rst_num = MT8188_TOPRGU_SW_RST_NUM,
+};
+
static const struct mtk_wdt_data mt8192_data = {
.toprgu_sw_rst_num = MT8192_TOPRGU_SW_RST_NUM,
};
static const struct of_device_id mtk_wdt_dt_ids[] = {
{ .compatible = "mediatek,mt2712-wdt", .data = &mt2712_data },
{ .compatible = "mediatek,mt6589-wdt" },
+ { .compatible = "mediatek,mt6795-wdt", .data = &mt6795_data },
{ .compatible = "mediatek,mt7986-wdt", .data = &mt7986_data },
{ .compatible = "mediatek,mt8183-wdt", .data = &mt8183_data },
{ .compatible = "mediatek,mt8186-wdt", .data = &mt8186_data },
+ { .compatible = "mediatek,mt8188-wdt", .data = &mt8188_data },
{ .compatible = "mediatek,mt8192-wdt", .data = &mt8192_data },
{ .compatible = "mediatek,mt8195-wdt", .data = &mt8195_data },
{ /* sentinel */ }
return 0;
}
-#ifdef CONFIG_PM
-
/* REVISIT ... not clear this is the best way to handle system suspend; and
* it's very inappropriate for selective device suspend (e.g. suspending this
* through sysfs rather than by stopping the watchdog daemon). Also, this
return 0;
}
-#else
-#define omap_wdt_suspend NULL
-#define omap_wdt_resume NULL
-#endif
-
static const struct of_device_id omap_wdt_of_match[] = {
{ .compatible = "ti,omap3-wdt", },
{},
.probe = omap_wdt_probe,
.remove = omap_wdt_remove,
.shutdown = omap_wdt_shutdown,
- .suspend = omap_wdt_suspend,
- .resume = omap_wdt_resume,
+ .suspend = pm_ptr(omap_wdt_suspend),
+ .resume = pm_ptr(omap_wdt_resume),
.driver = {
.name = "omap_wdt",
.of_match_table = omap_wdt_of_match,
struct rn5t618 *rn5t618 = dev_get_drvdata(dev->parent);
struct rn5t618_wdt *wdt;
int min_timeout, max_timeout;
+ int ret;
+ unsigned int val;
wdt = devm_kzalloc(dev, sizeof(struct rn5t618_wdt), GFP_KERNEL);
if (!wdt)
wdt->wdt_dev.timeout = max_timeout;
wdt->wdt_dev.parent = dev;
+ /* Read out previous power-off factor */
+ ret = regmap_read(wdt->rn5t618->regmap, RN5T618_POFFHIS, &val);
+ if (ret)
+ return ret;
+
+ if (val & RN5T618_POFFHIS_VINDET)
+ wdt->wdt_dev.bootstatus = WDIOF_POWERUNDER;
+ else if (val & RN5T618_POFFHIS_WDG)
+ wdt->wdt_dev.bootstatus = WDIOF_CARDRESET;
+
watchdog_set_drvdata(&wdt->wdt_dev, wdt);
watchdog_init_timeout(&wdt->wdt_dev, timeout, dev);
watchdog_set_nowayout(&wdt->wdt_dev, nowayout);
return devm_watchdog_register_device(dev, wdt);
}
-#ifdef CONFIG_PM
static int twl4030_wdt_suspend(struct platform_device *pdev, pm_message_t state)
{
struct watchdog_device *wdt = platform_get_drvdata(pdev);
return 0;
}
-#else
-#define twl4030_wdt_suspend NULL
-#define twl4030_wdt_resume NULL
-#endif
static const struct of_device_id twl_wdt_of_match[] = {
{ .compatible = "ti,twl4030-wdt", },
static struct platform_driver twl4030_wdt_driver = {
.probe = twl4030_wdt_probe,
- .suspend = twl4030_wdt_suspend,
- .resume = twl4030_wdt_resume,
+ .suspend = pm_ptr(twl4030_wdt_suspend),
+ .resume = pm_ptr(twl4030_wdt_resume),
.driver = {
.name = "twl4030_wdt",
.of_match_table = twl_wdt_of_match,
p->dev = dev;
p->count = count;
- if (WARN_ON(dev == WHITEOUT_DEV))
- return -EBUSY;
+ if (WARN_ON(dev == WHITEOUT_DEV)) {
+ error = -EBUSY;
+ goto err;
+ }
error = kobj_map(cdev_map, dev, count, NULL,
exact_match, exact_lock, p);
if (error)
- return error;
+ goto err;
kobject_get(p->kobj.parent);
return 0;
+
+err:
+ kfree_const(p->kobj.name);
+ p->kobj.name = NULL;
+ return error;
}
/**
}
rc = device_add(dev);
- if (rc)
+ if (rc && dev->devt)
cdev_del(cdev);
return rc;
cifs_dbg(FYI, "Socket created\n");
server->ssocket = socket;
socket->sk->sk_allocation = GFP_NOFS;
+ socket->sk->sk_use_task_frag = false;
if (sfamily == AF_INET6)
cifs_reclassify_socket6(socket);
else
if (dlm_config.ci_protocol == DLM_PROTO_SCTP)
sk->sk_state_change = lowcomms_state_change;
sk->sk_allocation = GFP_NOFS;
+ sk->sk_use_task_frag = false;
sk->sk_error_report = lowcomms_error_report;
release_sock(sk);
}
listen_con.sock = sock;
sock->sk->sk_allocation = GFP_NOFS;
+ sock->sk->sk_use_task_frag = false;
sock->sk->sk_data_ready = lowcomms_listen_data_ready;
release_sock(sock->sk);
struct exfat_chain *p_dir, int entry, unsigned short *uniname)
{
int i;
- struct exfat_entry_set_cache *es;
+ struct exfat_entry_set_cache es;
- es = exfat_get_dentry_set(sb, p_dir, entry, ES_ALL_ENTRIES);
- if (!es)
+ if (exfat_get_dentry_set(&es, sb, p_dir, entry, ES_ALL_ENTRIES))
return;
/*
* Third entry : first file-name entry
* So, the index of first file-name dentry should start from 2.
*/
- for (i = 2; i < es->num_entries; i++) {
- struct exfat_dentry *ep = exfat_get_dentry_cached(es, i);
+ for (i = ES_IDX_FIRST_FILENAME; i < es.num_entries; i++) {
+ struct exfat_dentry *ep = exfat_get_dentry_cached(&es, i);
/* end of name entry */
if (exfat_get_entry_type(ep) != TYPE_EXTEND)
uniname += EXFAT_FILE_NAME_LEN;
}
- exfat_free_dentry_set(es, false);
+ exfat_put_dentry_set(&es, false);
}
/* read a directory entry from the opened directory */
static int exfat_readdir(struct inode *inode, loff_t *cpos, struct exfat_dir_entry *dir_entry)
{
- int i, dentries_per_clu, dentries_per_clu_bits = 0, num_ext;
+ int i, dentries_per_clu, num_ext;
unsigned int type, clu_offset, max_dentries;
struct exfat_chain dir, clu;
struct exfat_uni_name uni_name;
EXFAT_B_TO_CLU(i_size_read(inode), sbi), ei->flags);
dentries_per_clu = sbi->dentries_per_clu;
- dentries_per_clu_bits = ilog2(dentries_per_clu);
max_dentries = (unsigned int)min_t(u64, MAX_EXFAT_DENTRIES,
- (u64)sbi->num_clusters << dentries_per_clu_bits);
+ (u64)EXFAT_CLU_TO_DEN(sbi->num_clusters, sbi));
- clu_offset = dentry >> dentries_per_clu_bits;
+ clu_offset = EXFAT_DEN_TO_CLU(dentry, sbi);
exfat_chain_dup(&clu, &dir);
if (clu.flags == ALLOC_NO_FAT_CHAIN) {
dir_entry->entry = dentry;
brelse(bh);
- ei->hint_bmap.off = dentry >> dentries_per_clu_bits;
+ ei->hint_bmap.off = EXFAT_DEN_TO_CLU(dentry, sbi);
ei->hint_bmap.clu = clu.dir;
*cpos = EXFAT_DEN_TO_B(dentry + 1 + num_ext);
return -EINVAL;
/* 1 file entry + 1 stream entry + name entries */
- return ((len - 1) / EXFAT_FILE_NAME_LEN + 3);
+ return ES_ENTRY_NUM(len);
}
unsigned int exfat_get_entry_type(struct exfat_dentry *ep)
unsigned short chksum = 0;
struct exfat_dentry *ep;
- for (i = 0; i < es->num_entries; i++) {
+ for (i = ES_IDX_FILE; i < es->num_entries; i++) {
ep = exfat_get_dentry_cached(es, i);
chksum = exfat_calc_chksum16(ep, DENTRY_SIZE, chksum,
chksum_type);
chksum_type = CS_DEFAULT;
}
- ep = exfat_get_dentry_cached(es, 0);
+ ep = exfat_get_dentry_cached(es, ES_IDX_FILE);
ep->dentry.file.checksum = cpu_to_le16(chksum);
es->modified = true;
}
-int exfat_free_dentry_set(struct exfat_entry_set_cache *es, int sync)
+int exfat_put_dentry_set(struct exfat_entry_set_cache *es, int sync)
{
int i, err = 0;
bforget(es->bh[i]);
else
brelse(es->bh[i]);
- kfree(es);
+
+ if (IS_DYNAMIC_ES(es))
+ kfree(es->bh);
+
return err;
}
* pointer of entry set on success,
* NULL on failure.
*/
-struct exfat_entry_set_cache *exfat_get_dentry_set(struct super_block *sb,
- struct exfat_chain *p_dir, int entry, unsigned int type)
+int exfat_get_dentry_set(struct exfat_entry_set_cache *es,
+ struct super_block *sb, struct exfat_chain *p_dir, int entry,
+ unsigned int type)
{
int ret, i, num_bh;
- unsigned int off, byte_offset, clu = 0;
+ unsigned int off;
sector_t sec;
struct exfat_sb_info *sbi = EXFAT_SB(sb);
- struct exfat_entry_set_cache *es;
struct exfat_dentry *ep;
int num_entries;
enum exfat_validate_dentry_mode mode = ES_MODE_STARTED;
if (p_dir->dir == DIR_DELETED) {
exfat_err(sb, "access to deleted dentry");
- return NULL;
+ return -EIO;
}
- byte_offset = EXFAT_DEN_TO_B(entry);
- ret = exfat_walk_fat_chain(sb, p_dir, byte_offset, &clu);
+ ret = exfat_find_location(sb, p_dir, entry, &sec, &off);
if (ret)
- return NULL;
+ return ret;
- es = kzalloc(sizeof(*es), GFP_KERNEL);
- if (!es)
- return NULL;
+ memset(es, 0, sizeof(*es));
es->sb = sb;
es->modified = false;
-
- /* byte offset in cluster */
- byte_offset = EXFAT_CLU_OFFSET(byte_offset, sbi);
-
- /* byte offset in sector */
- off = EXFAT_BLK_OFFSET(byte_offset, sb);
es->start_off = off;
-
- /* sector offset in cluster */
- sec = EXFAT_B_TO_BLK(byte_offset, sb);
- sec += exfat_cluster_to_sector(sbi, clu);
+ es->bh = es->__bh;
bh = sb_bread(sb, sec);
if (!bh)
- goto free_es;
+ return -EIO;
es->bh[es->num_bh++] = bh;
- ep = exfat_get_dentry_cached(es, 0);
+ ep = exfat_get_dentry_cached(es, ES_IDX_FILE);
if (!exfat_validate_entry(exfat_get_entry_type(ep), &mode))
- goto free_es;
+ goto put_es;
num_entries = type == ES_ALL_ENTRIES ?
ep->dentry.file.num_ext + 1 : type;
es->num_entries = num_entries;
num_bh = EXFAT_B_TO_BLK_ROUND_UP(off + num_entries * DENTRY_SIZE, sb);
+ if (num_bh > ARRAY_SIZE(es->__bh)) {
+ es->bh = kmalloc_array(num_bh, sizeof(*es->bh), GFP_KERNEL);
+ if (!es->bh) {
+ brelse(bh);
+ return -ENOMEM;
+ }
+ es->bh[0] = bh;
+ }
+
for (i = 1; i < num_bh; i++) {
/* get the next sector */
if (exfat_is_last_sector_in_cluster(sbi, sec)) {
+ unsigned int clu = exfat_sector_to_cluster(sbi, sec);
+
if (p_dir->flags == ALLOC_NO_FAT_CHAIN)
clu++;
else if (exfat_get_next_cluster(sb, &clu))
- goto free_es;
+ goto put_es;
sec = exfat_cluster_to_sector(sbi, clu);
} else {
sec++;
bh = sb_bread(sb, sec);
if (!bh)
- goto free_es;
+ goto put_es;
es->bh[es->num_bh++] = bh;
}
/* validate cached dentries */
- for (i = 1; i < num_entries; i++) {
+ for (i = ES_IDX_STREAM; i < num_entries; i++) {
ep = exfat_get_dentry_cached(es, i);
if (!exfat_validate_entry(exfat_get_entry_type(ep), &mode))
- goto free_es;
+ goto put_es;
}
- return es;
+ return 0;
+
+put_es:
+ exfat_put_dentry_set(es, false);
+ return -EIO;
+}
-free_es:
- exfat_free_dentry_set(es, false);
- return NULL;
+static inline void exfat_reset_empty_hint(struct exfat_hint_femp *hint_femp)
+{
+ hint_femp->eidx = EXFAT_HINT_NONE;
+ hint_femp->count = 0;
+}
+
+static inline void exfat_set_empty_hint(struct exfat_inode_info *ei,
+ struct exfat_hint_femp *candi_empty, struct exfat_chain *clu,
+ int dentry, int num_entries, int entry_type)
+{
+ if (ei->hint_femp.eidx == EXFAT_HINT_NONE ||
+ ei->hint_femp.eidx > dentry) {
+ int total_entries = EXFAT_B_TO_DEN(i_size_read(&ei->vfs_inode));
+
+ if (candi_empty->count == 0) {
+ candi_empty->cur = *clu;
+ candi_empty->eidx = dentry;
+ }
+
+ if (entry_type == TYPE_UNUSED)
+ candi_empty->count += total_entries - dentry;
+ else
+ candi_empty->count++;
+
+ if (candi_empty->count == num_entries ||
+ candi_empty->count + candi_empty->eidx == total_entries)
+ ei->hint_femp = *candi_empty;
+ }
}
enum {
*/
int exfat_find_dir_entry(struct super_block *sb, struct exfat_inode_info *ei,
struct exfat_chain *p_dir, struct exfat_uni_name *p_uniname,
- int num_entries, unsigned int type, struct exfat_hint *hint_opt)
+ struct exfat_hint *hint_opt)
{
int i, rewind = 0, dentry = 0, end_eidx = 0, num_ext = 0, len;
int order, step, name_len = 0;
- int dentries_per_clu, num_empty = 0;
+ int dentries_per_clu;
unsigned int entry_type;
unsigned short *uniname = NULL;
struct exfat_chain clu;
struct exfat_hint *hint_stat = &ei->hint_stat;
struct exfat_hint_femp candi_empty;
struct exfat_sb_info *sbi = EXFAT_SB(sb);
+ int num_entries = exfat_calc_num_entries(p_uniname);
+
+ if (num_entries < 0)
+ return num_entries;
dentries_per_clu = sbi->dentries_per_clu;
end_eidx = dentry;
}
- candi_empty.eidx = EXFAT_HINT_NONE;
+ exfat_reset_empty_hint(&ei->hint_femp);
+
rewind:
order = 0;
step = DIRENT_STEP_FILE;
+ exfat_reset_empty_hint(&candi_empty);
+
while (clu.dir != EXFAT_EOF_CLUSTER) {
i = dentry & (dentries_per_clu - 1);
for (; i < dentries_per_clu; i++, dentry++) {
entry_type == TYPE_DELETED) {
step = DIRENT_STEP_FILE;
- num_empty++;
- if (candi_empty.eidx == EXFAT_HINT_NONE &&
- num_empty == 1) {
- exfat_chain_set(&candi_empty.cur,
- clu.dir, clu.size, clu.flags);
- }
-
- if (candi_empty.eidx == EXFAT_HINT_NONE &&
- num_empty >= num_entries) {
- candi_empty.eidx =
- dentry - (num_empty - 1);
- WARN_ON(candi_empty.eidx < 0);
- candi_empty.count = num_empty;
-
- if (ei->hint_femp.eidx ==
- EXFAT_HINT_NONE ||
- candi_empty.eidx <=
- ei->hint_femp.eidx)
- ei->hint_femp = candi_empty;
- }
+ exfat_set_empty_hint(ei, &candi_empty, &clu,
+ dentry, num_entries,
+ entry_type);
brelse(bh);
if (entry_type == TYPE_UNUSED)
continue;
}
- num_empty = 0;
- candi_empty.eidx = EXFAT_HINT_NONE;
+ exfat_reset_empty_hint(&candi_empty);
if (entry_type == TYPE_FILE || entry_type == TYPE_DIR) {
step = DIRENT_STEP_FILE;
hint_opt->clu = clu.dir;
hint_opt->eidx = i;
- if (type == TYPE_ALL || type == entry_type) {
- num_ext = ep->dentry.file.num_ext;
- step = DIRENT_STEP_STRM;
- }
+ num_ext = ep->dentry.file.num_ext;
+ step = DIRENT_STEP_STRM;
brelse(bh);
continue;
}
rewind = 1;
dentry = 0;
clu.dir = p_dir->dir;
- /* reset empty hint */
- num_empty = 0;
- candi_empty.eidx = EXFAT_HINT_NONE;
goto rewind;
}
+ /*
+ * set the EXFAT_EOF_CLUSTER flag to avoid search
+ * from the beginning again when allocated a new cluster
+ */
+ if (ei->hint_femp.eidx == EXFAT_HINT_NONE) {
+ ei->hint_femp.cur.dir = EXFAT_EOF_CLUSTER;
+ ei->hint_femp.eidx = p_dir->size * dentries_per_clu;
+ ei->hint_femp.count = 0;
+ }
+
/* initialized hint_stat */
hint_stat->clu = p_dir->dir;
hint_stat->eidx = 0;
#include <linux/fs.h>
#include <linux/ratelimit.h>
#include <linux/nls.h>
+#include <linux/blkdev.h>
#define EXFAT_ROOT_INO 1
#define ES_2_ENTRIES 2
#define ES_ALL_ENTRIES 0
+#define ES_IDX_FILE 0
+#define ES_IDX_STREAM 1
+#define ES_IDX_FIRST_FILENAME 2
+#define EXFAT_FILENAME_ENTRY_NUM(name_len) \
+ DIV_ROUND_UP(name_len, EXFAT_FILE_NAME_LEN)
+#define ES_IDX_LAST_FILENAME(name_len) \
+ (ES_IDX_FIRST_FILENAME + EXFAT_FILENAME_ENTRY_NUM(name_len) - 1)
+
#define DIR_DELETED 0xFFFF0321
/* type values */
#define TYPE_PADDING 0x0402
#define TYPE_ACLTAB 0x0403
#define TYPE_BENIGN_SEC 0x0800
-#define TYPE_ALL 0x0FFF
#define MAX_CHARSET_SIZE 6 /* max size of multi-byte character */
#define MAX_NAME_LENGTH 255 /* max len of file name excluding NULL */
#define MAX_VFSNAME_BUF_SIZE ((MAX_NAME_LENGTH + 1) * MAX_CHARSET_SIZE)
-/* Enough size to hold 256 dentry (even 512 Byte sector) */
-#define DIR_CACHE_SIZE (256*sizeof(struct exfat_dentry)/512+1)
-
#define EXFAT_HINT_NONE -1
#define EXFAT_MIN_SUBDIR 2
/*
* helpers for block size to dentry size conversion.
*/
-#define EXFAT_B_TO_DEN_IDX(b, sbi) \
- ((b) << ((sbi)->cluster_size_bits - DENTRY_SIZE_BITS))
#define EXFAT_B_TO_DEN(b) ((b) >> DENTRY_SIZE_BITS)
#define EXFAT_DEN_TO_B(b) ((b) << DENTRY_SIZE_BITS)
/*
+ * helpers for cluster size to dentry size conversion.
+ */
+#define EXFAT_CLU_TO_DEN(clu, sbi) \
+ ((clu) << ((sbi)->cluster_size_bits - DENTRY_SIZE_BITS))
+#define EXFAT_DEN_TO_CLU(dentry, sbi) \
+ ((dentry) >> ((sbi)->cluster_size_bits - DENTRY_SIZE_BITS))
+
+/*
* helpers for fat entry.
*/
#define FAT_ENT_SIZE (4)
#define BITS_PER_BYTE_MASK 0x7
#define IGNORED_BITS_REMAINED(clu, clu_base) ((1 << ((clu) - (clu_base))) - 1)
+#define ES_ENTRY_NUM(name_len) (ES_IDX_LAST_FILENAME(name_len) + 1)
+/* 19 entries = 1 file entry + 1 stream entry + 17 filename entries */
+#define ES_MAX_ENTRY_NUM ES_ENTRY_NUM(MAX_NAME_LENGTH)
+
+/*
+ * 19 entries x 32 bytes/entry = 608 bytes.
+ * The 608 bytes are in 3 sectors at most (even 512 Byte sector).
+ */
+#define DIR_CACHE_SIZE \
+ (DIV_ROUND_UP(EXFAT_DEN_TO_B(ES_MAX_ENTRY_NUM), SECTOR_SIZE) + 1)
+
struct exfat_dentry_namebuf {
char *lfn;
int lfnbuf_len; /* usually MAX_UNINAME_BUF_SIZE */
struct exfat_entry_set_cache {
struct super_block *sb;
- bool modified;
unsigned int start_off;
int num_bh;
- struct buffer_head *bh[DIR_CACHE_SIZE];
+ struct buffer_head *__bh[DIR_CACHE_SIZE];
+ struct buffer_head **bh;
unsigned int num_entries;
+ bool modified;
};
+#define IS_DYNAMIC_ES(es) ((es)->__bh != (es)->bh)
+
struct exfat_dir_entry {
struct exfat_chain dir;
int entry;
sbi->data_start_sector;
}
-static inline int exfat_sector_to_cluster(struct exfat_sb_info *sbi,
+static inline unsigned int exfat_sector_to_cluster(struct exfat_sb_info *sbi,
sector_t sec)
{
return ((sec - sbi->data_start_sector) >> sbi->sect_per_clus_bits) +
/* file.c */
extern const struct file_operations exfat_file_operations;
-int __exfat_truncate(struct inode *inode, loff_t new_size);
-void exfat_truncate(struct inode *inode, loff_t size);
+int __exfat_truncate(struct inode *inode);
+void exfat_truncate(struct inode *inode);
int exfat_setattr(struct user_namespace *mnt_userns, struct dentry *dentry,
struct iattr *attr);
int exfat_getattr(struct user_namespace *mnt_userns, const struct path *path,
int exfat_calc_num_entries(struct exfat_uni_name *p_uniname);
int exfat_find_dir_entry(struct super_block *sb, struct exfat_inode_info *ei,
struct exfat_chain *p_dir, struct exfat_uni_name *p_uniname,
- int num_entries, unsigned int type, struct exfat_hint *hint_opt);
+ struct exfat_hint *hint_opt);
int exfat_alloc_new_dir(struct inode *inode, struct exfat_chain *clu);
struct exfat_dentry *exfat_get_dentry(struct super_block *sb,
struct exfat_chain *p_dir, int entry, struct buffer_head **bh);
struct exfat_dentry *exfat_get_dentry_cached(struct exfat_entry_set_cache *es,
int num);
-struct exfat_entry_set_cache *exfat_get_dentry_set(struct super_block *sb,
- struct exfat_chain *p_dir, int entry, unsigned int type);
-int exfat_free_dentry_set(struct exfat_entry_set_cache *es, int sync);
+int exfat_get_dentry_set(struct exfat_entry_set_cache *es,
+ struct super_block *sb, struct exfat_chain *p_dir, int entry,
+ unsigned int type);
+int exfat_put_dentry_set(struct exfat_entry_set_cache *es, int sync);
int exfat_count_dir_entries(struct super_block *sb, struct exfat_chain *p_dir);
/* inode.c */
}
/* resize the file length */
-int __exfat_truncate(struct inode *inode, loff_t new_size)
+int __exfat_truncate(struct inode *inode)
{
unsigned int num_clusters_new, num_clusters_phys;
unsigned int last_clu = EXFAT_FREE_CLUSTER;
exfat_chain_set(&clu, ei->start_clu, num_clusters_phys, ei->flags);
- if (new_size > 0) {
+ if (i_size_read(inode) > 0) {
/*
* Truncate FAT chain num_clusters after the first cluster
* num_clusters = min(new, phys);
ei->start_clu = EXFAT_EOF_CLUSTER;
}
- i_size_write(inode, new_size);
-
if (ei->type == TYPE_FILE)
ei->attr |= ATTR_ARCHIVE;
return 0;
}
-void exfat_truncate(struct inode *inode, loff_t size)
+void exfat_truncate(struct inode *inode)
{
struct super_block *sb = inode->i_sb;
struct exfat_sb_info *sbi = EXFAT_SB(sb);
goto write_size;
}
- err = __exfat_truncate(inode, i_size_read(inode));
+ err = __exfat_truncate(inode);
if (err)
goto write_size;
* __exfat_write_inode() is called from exfat_truncate(), inode
* is already written by it, so mark_inode_dirty() is unneeded.
*/
- exfat_truncate(inode, attr->ia_size);
+ exfat_truncate(inode);
up_write(&EXFAT_I(inode)->truncate_lock);
} else
mark_inode_dirty(inode);
{
unsigned long long on_disk_size;
struct exfat_dentry *ep, *ep2;
- struct exfat_entry_set_cache *es = NULL;
+ struct exfat_entry_set_cache es;
struct super_block *sb = inode->i_sb;
struct exfat_sb_info *sbi = EXFAT_SB(sb);
struct exfat_inode_info *ei = EXFAT_I(inode);
exfat_set_volume_dirty(sb);
/* get the directory entry of given file or directory */
- es = exfat_get_dentry_set(sb, &(ei->dir), ei->entry, ES_ALL_ENTRIES);
- if (!es)
+ if (exfat_get_dentry_set(&es, sb, &(ei->dir), ei->entry, ES_ALL_ENTRIES))
return -EIO;
- ep = exfat_get_dentry_cached(es, 0);
- ep2 = exfat_get_dentry_cached(es, 1);
+ ep = exfat_get_dentry_cached(&es, ES_IDX_FILE);
+ ep2 = exfat_get_dentry_cached(&es, ES_IDX_STREAM);
ep->dentry.file.attr = cpu_to_le16(exfat_make_attr(inode));
ep2->dentry.stream.start_clu = EXFAT_FREE_CLUSTER;
}
- exfat_update_dir_chksum_with_entry_set(es);
- return exfat_free_dentry_set(es, sync);
+ exfat_update_dir_chksum_with_entry_set(&es);
+ return exfat_put_dentry_set(&es, sync);
}
int exfat_write_inode(struct inode *inode, struct writeback_control *wbc)
if (to > i_size_read(inode)) {
truncate_pagecache(inode, i_size_read(inode));
inode->i_mtime = inode->i_ctime = current_time(inode);
- exfat_truncate(inode, EXFAT_I(inode)->i_size_aligned);
+ exfat_truncate(inode);
}
}
if (!inode->i_nlink) {
i_size_write(inode, 0);
mutex_lock(&EXFAT_SB(inode->i_sb)->s_lock);
- __exfat_truncate(inode, 0);
+ __exfat_truncate(inode);
mutex_unlock(&EXFAT_SB(inode->i_sb)->s_lock);
}
if (hint_femp->eidx != EXFAT_HINT_NONE) {
dentry = hint_femp->eidx;
- if (num_entries <= hint_femp->count) {
- hint_femp->eidx = EXFAT_HINT_NONE;
- return dentry;
- }
+ /*
+ * If hint_femp->count is enough, it is needed to check if
+ * there are actual empty entries.
+ * Otherwise, and if "dentry + hint_famp->count" is also equal
+ * to "p_dir->size * dentries_per_clu", it means ENOSPC.
+ */
+ if (dentry + hint_femp->count == p_dir->size * dentries_per_clu &&
+ num_entries > hint_femp->count)
+ return -ENOSPC;
+
+ hint_femp->eidx = EXFAT_HINT_NONE;
exfat_chain_dup(&clu, &hint_femp->cur);
} else {
exfat_chain_dup(&clu, p_dir);
}
}
+ hint_femp->eidx = p_dir->size * dentries_per_clu - num_empty;
+ hint_femp->count = num_empty;
+ if (num_empty == 0)
+ exfat_chain_set(&hint_femp->cur, EXFAT_EOF_CLUSTER, 0,
+ clu.flags);
+
return -ENOSPC;
}
if (exfat_ent_set(sb, last_clu, clu.dir))
return -EIO;
- if (hint_femp.eidx == EXFAT_HINT_NONE) {
- /* the special case that new dentry
- * should be allocated from the start of new cluster
- */
- hint_femp.eidx = EXFAT_B_TO_DEN_IDX(p_dir->size, sbi);
- hint_femp.count = sbi->dentries_per_clu;
-
+ if (hint_femp.cur.dir == EXFAT_EOF_CLUSTER)
exfat_chain_set(&hint_femp.cur, clu.dir, 0, clu.flags);
- }
+
+ hint_femp.count += sbi->dentries_per_clu;
+
hint_femp.cur.size++;
p_dir->size++;
size = EXFAT_CLU_TO_B(p_dir->size, sbi);
static int exfat_find(struct inode *dir, struct qstr *qname,
struct exfat_dir_entry *info)
{
- int ret, dentry, num_entries, count;
+ int ret, dentry, count;
struct exfat_chain cdir;
struct exfat_uni_name uni_name;
struct super_block *sb = dir->i_sb;
struct exfat_sb_info *sbi = EXFAT_SB(sb);
struct exfat_inode_info *ei = EXFAT_I(dir);
struct exfat_dentry *ep, *ep2;
- struct exfat_entry_set_cache *es;
+ struct exfat_entry_set_cache es;
/* for optimized dir & entry to prevent long traverse of cluster chain */
struct exfat_hint hint_opt;
if (ret)
return ret;
- num_entries = exfat_calc_num_entries(&uni_name);
- if (num_entries < 0)
- return num_entries;
-
/* check the validation of hint_stat and initialize it if required */
if (ei->version != (inode_peek_iversion_raw(dir) & 0xffffffff)) {
ei->hint_stat.clu = cdir.dir;
}
/* search the file name for directories */
- dentry = exfat_find_dir_entry(sb, ei, &cdir, &uni_name,
- num_entries, TYPE_ALL, &hint_opt);
-
+ dentry = exfat_find_dir_entry(sb, ei, &cdir, &uni_name, &hint_opt);
if (dentry < 0)
return dentry; /* -error value */
if (cdir.flags & ALLOC_NO_FAT_CHAIN)
cdir.size -= dentry / sbi->dentries_per_clu;
dentry = hint_opt.eidx;
- es = exfat_get_dentry_set(sb, &cdir, dentry, ES_2_ENTRIES);
- if (!es)
+ if (exfat_get_dentry_set(&es, sb, &cdir, dentry, ES_2_ENTRIES))
return -EIO;
- ep = exfat_get_dentry_cached(es, 0);
- ep2 = exfat_get_dentry_cached(es, 1);
+ ep = exfat_get_dentry_cached(&es, ES_IDX_FILE);
+ ep2 = exfat_get_dentry_cached(&es, ES_IDX_STREAM);
info->type = exfat_get_entry_type(ep);
info->attr = le16_to_cpu(ep->dentry.file.attr);
ep->dentry.file.access_time,
ep->dentry.file.access_date,
0);
- exfat_free_dentry_set(es, false);
+ exfat_put_dentry_set(&es, false);
if (ei->start_clu == EXFAT_FREE_CLUSTER) {
exfat_fs_error(sb,
struct exfat_inode_info *new_ei = NULL;
unsigned int new_entry_type = TYPE_UNUSED;
int new_entry = 0;
- struct buffer_head *old_bh, *new_bh = NULL;
+ struct buffer_head *new_bh = NULL;
/* check the validity of pointer parameters */
if (new_path == NULL || strlen(new_path) == 0)
EXFAT_I(old_parent_inode)->flags);
dentry = ei->entry;
- ep = exfat_get_dentry(sb, &olddir, dentry, &old_bh);
- if (!ep) {
- ret = -EIO;
- goto out;
- }
- brelse(old_bh);
-
/* check whether new dir is existing directory and empty */
if (new_inode) {
ret = -EIO;
{
assert_spin_locked(&wb->list_lock);
assert_spin_locked(&inode->i_lock);
+ WARN_ON_ONCE(inode->i_state & I_FREEING);
list_move(&inode->i_io_list, head);
{
assert_spin_locked(&wb->list_lock);
assert_spin_locked(&inode->i_lock);
+ WARN_ON_ONCE(inode->i_state & I_FREEING);
inode->i_state &= ~I_SYNC_QUEUED;
if (wb != &wb->bdi->wb)
{
assert_spin_locked(&wb->list_lock);
assert_spin_locked(&inode->i_lock);
+ WARN_ON_ONCE(inode->i_state & I_FREEING);
inode->i_state &= ~I_SYNC_QUEUED;
list_del_init(&inode->i_io_list);
{
assert_spin_locked(&inode->i_lock);
+ inode->i_state &= ~I_SYNC_QUEUED;
+ /*
+ * When the inode is being freed just don't bother with dirty list
+ * tracking. Flush worker will ignore this inode anyway and it will
+ * trigger assertions in inode_io_list_move_locked().
+ */
+ if (inode->i_state & I_FREEING) {
+ list_del_init(&inode->i_io_list);
+ wb_io_lists_depopulated(wb);
+ return;
+ }
if (!list_empty(&wb->b_dirty)) {
struct inode *tail;
inode->dirtied_when = jiffies;
}
inode_io_list_move_locked(inode, wb, &wb->b_dirty);
- inode->i_state &= ~I_SYNC_QUEUED;
}
static void redirty_tail(struct inode *inode, struct bdi_writeback *wb)
return ret;
}
-#define EXPIRE_DIRTY_ATIME 0x0001
-
/*
* Move expired (dirtied before dirtied_before) dirty inodes from
* @delaying_queue to @dispatch_queue.
return error;
kaddr = kmap_atomic(page);
- if (dsize > gfs2_max_stuffed_size(ip))
- dsize = gfs2_max_stuffed_size(ip);
memcpy(kaddr, dibh->b_data + sizeof(struct gfs2_dinode), dsize);
memset(kaddr + dsize, 0, PAGE_SIZE - dsize);
kunmap_atomic(kaddr);
void *kaddr = kmap(page);
u64 dsize = i_size_read(inode);
- if (dsize > gfs2_max_stuffed_size(ip))
- dsize = gfs2_max_stuffed_size(ip);
-
memcpy(kaddr, dibh->b_data + sizeof(struct gfs2_dinode), dsize);
memset(kaddr + dsize, 0, PAGE_SIZE - dsize);
kunmap(page);
static void __flock_holder_uninit(struct file *file, struct gfs2_holder *fl_gh)
{
- struct gfs2_glock *gl = fl_gh->gh_gl;
+ struct gfs2_glock *gl = gfs2_glock_hold(fl_gh->gh_gl);
/*
* Make sure gfs2_glock_put() won't sleep under the file->f_lock
* spinlock.
*/
- gfs2_glock_hold(gl);
spin_lock(&file->f_lock);
gfs2_holder_uninit(fl_gh);
spin_unlock(&file->f_lock);
*
*/
-void gfs2_glock_hold(struct gfs2_glock *gl)
+struct gfs2_glock *gfs2_glock_hold(struct gfs2_glock *gl)
{
GLOCK_BUG_ON(gl, __lockref_is_dead(&gl->gl_lockref));
lockref_get(&gl->gl_lockref);
+ return gl;
}
/**
if (gl->gl_state == LM_ST_UNLOCKED)
return 0;
- /*
- * Note that demote_ok is used for the lru process of disposing of
- * glocks. For this purpose, we don't care if the glock's holders
- * have the HIF_MAY_DEMOTE flag set or not. If someone is using
- * them, don't demote.
- */
if (!list_empty(&gl->gl_holders))
return 0;
if (glops->go_demote_ok)
struct gfs2_holder *gh, *tmp;
list_for_each_entry_safe(gh, tmp, &gl->gl_holders, gh_list) {
- if (!test_bit(HIF_WAIT, &gh->gh_iflags))
+ if (test_bit(HIF_HOLDER, &gh->gh_iflags))
continue;
if (ret & LM_OUT_ERROR)
gh->gh_error = -EIO;
}
/**
- * demote_incompat_holders - demote incompatible demoteable holders
- * @gl: the glock we want to promote
- * @current_gh: the newly promoted holder
- *
- * We're passing the newly promoted holder in @current_gh, but actually, any of
- * the strong holders would do.
- */
-static void demote_incompat_holders(struct gfs2_glock *gl,
- struct gfs2_holder *current_gh)
-{
- struct gfs2_holder *gh, *tmp;
-
- /*
- * Demote incompatible holders before we make ourselves eligible.
- * (This holder may or may not allow auto-demoting, but we don't want
- * to demote the new holder before it's even granted.)
- */
- list_for_each_entry_safe(gh, tmp, &gl->gl_holders, gh_list) {
- /*
- * Since holders are at the front of the list, we stop when we
- * find the first non-holder.
- */
- if (!test_bit(HIF_HOLDER, &gh->gh_iflags))
- return;
- if (gh == current_gh)
- continue;
- if (test_bit(HIF_MAY_DEMOTE, &gh->gh_iflags) &&
- !may_grant(gl, current_gh, gh)) {
- /*
- * We should not recurse into do_promote because
- * __gfs2_glock_dq only calls handle_callback,
- * gfs2_glock_add_to_lru and __gfs2_glock_queue_work.
- */
- __gfs2_glock_dq(gh);
- }
- }
-}
-
-/**
* find_first_holder - find the first "holder" gh
* @gl: the glock
*/
return NULL;
}
-/**
- * find_first_strong_holder - find the first non-demoteable holder
- * @gl: the glock
- *
- * Find the first holder that doesn't have the HIF_MAY_DEMOTE flag set.
- */
-static inline struct gfs2_holder *
-find_first_strong_holder(struct gfs2_glock *gl)
-{
- struct gfs2_holder *gh;
-
- list_for_each_entry(gh, &gl->gl_holders, gh_list) {
- if (!test_bit(HIF_HOLDER, &gh->gh_iflags))
- return NULL;
- if (!test_bit(HIF_MAY_DEMOTE, &gh->gh_iflags))
- return gh;
- }
- return NULL;
-}
-
/*
* gfs2_instantiate - Call the glops instantiate function
* @gh: The glock holder
static int do_promote(struct gfs2_glock *gl)
{
struct gfs2_holder *gh, *current_gh;
- bool incompat_holders_demoted = false;
- current_gh = find_first_strong_holder(gl);
+ current_gh = find_first_holder(gl);
list_for_each_entry(gh, &gl->gl_holders, gh_list) {
if (test_bit(HIF_HOLDER, &gh->gh_iflags))
continue;
set_bit(HIF_HOLDER, &gh->gh_iflags);
trace_gfs2_promote(gh);
gfs2_holder_wake(gh);
- if (!incompat_holders_demoted) {
+ if (!current_gh)
current_gh = gh;
- demote_incompat_holders(gl, current_gh);
- incompat_holders_demoted = true;
- }
}
return 0;
}
return;
}
+/**
+ * glock_set_object - set the gl_object field of a glock
+ * @gl: the glock
+ * @object: the object
+ */
+void glock_set_object(struct gfs2_glock *gl, void *object)
+{
+ void *prev_object;
+
+ spin_lock(&gl->gl_lockref.lock);
+ prev_object = gl->gl_object;
+ gl->gl_object = object;
+ spin_unlock(&gl->gl_lockref.lock);
+ if (gfs2_assert_warn(gl->gl_name.ln_sbd, prev_object == NULL)) {
+ pr_warn("glock=%u/%llx\n",
+ gl->gl_name.ln_type,
+ (unsigned long long)gl->gl_name.ln_number);
+ gfs2_dump_glock(NULL, gl, true);
+ }
+}
+
+/**
+ * glock_clear_object - clear the gl_object field of a glock
+ * @gl: the glock
+ */
+void glock_clear_object(struct gfs2_glock *gl, void *object)
+{
+ void *prev_object;
+
+ spin_lock(&gl->gl_lockref.lock);
+ prev_object = gl->gl_object;
+ gl->gl_object = NULL;
+ spin_unlock(&gl->gl_lockref.lock);
+ if (gfs2_assert_warn(gl->gl_name.ln_sbd,
+ prev_object == object || prev_object == NULL)) {
+ pr_warn("glock=%u/%llx\n",
+ gl->gl_name.ln_type,
+ (unsigned long long)gl->gl_name.ln_number);
+ gfs2_dump_glock(NULL, gl, true);
+ }
+}
+
void gfs2_inode_remember_delete(struct gfs2_glock *gl, u64 generation)
{
struct gfs2_inode_lvb *ri = (void *)gl->gl_lksb.sb_lvbptr;
ip = NULL;
spin_unlock(&gl->gl_lockref.lock);
if (ip) {
- struct gfs2_glock *inode_gl = NULL;
-
gl->gl_no_formal_ino = ip->i_no_formal_ino;
set_bit(GIF_DEFERRED_DELETE, &ip->i_flags);
d_prune_aliases(&ip->i_inode);
spin_lock(&gl->gl_lockref.lock);
ip = gl->gl_object;
if (ip) {
- inode_gl = ip->i_gl;
- lockref_get(&inode_gl->gl_lockref);
clear_bit(GIF_DEFERRED_DELETE, &ip->i_flags);
+ if (!igrab(&ip->i_inode))
+ ip = NULL;
}
spin_unlock(&gl->gl_lockref.lock);
- if (inode_gl) {
- gfs2_glock_poke(inode_gl);
- gfs2_glock_put(inode_gl);
+ if (ip) {
+ gfs2_glock_poke(ip->i_gl);
+ iput(&ip->i_inode);
}
evicted = !ip;
}
if (gfs2_queue_delete_work(gl, 5 * HZ))
return;
}
+ goto out;
}
inode = gfs2_lookup_by_inum(sdp, no_addr, gl->gl_no_formal_ino,
d_prune_aliases(inode);
iput(inode);
}
+out:
gfs2_glock_put(gl);
}
struct gfs2_holder *gh, unsigned long ip)
{
INIT_LIST_HEAD(&gh->gh_list);
- gh->gh_gl = gl;
+ gh->gh_gl = gfs2_glock_hold(gl);
gh->gh_ip = ip;
gh->gh_owner_pid = get_pid(task_pid(current));
gh->gh_state = state;
gh->gh_flags = flags;
gh->gh_iflags = 0;
- gfs2_glock_hold(gl);
}
/**
if (test_bit(GLF_LOCK, &gl->gl_flags)) {
struct gfs2_holder *current_gh;
- current_gh = find_first_strong_holder(gl);
+ current_gh = find_first_holder(gl);
try_futile = !may_grant(gl, current_gh, gh);
}
if (test_bit(GLF_INVALIDATE_IN_PROGRESS, &gl->gl_flags))
continue;
if (gh->gh_gl->gl_ops->go_type == LM_TYPE_FLOCK)
continue;
- if (test_bit(HIF_MAY_DEMOTE, &gh2->gh_iflags))
- continue;
if (!pid_is_meaningful(gh2))
continue;
goto trap_recursive;
static void __gfs2_glock_dq(struct gfs2_holder *gh)
{
struct gfs2_glock *gl = gh->gh_gl;
- struct gfs2_sbd *sdp = gl->gl_name.ln_sbd;
unsigned delay = 0;
int fast_path = 0;
/*
- * This while loop is similar to function demote_incompat_holders:
- * If the glock is due to be demoted (which may be from another node
- * or even if this holder is GL_NOCACHE), the weak holders are
- * demoted as well, allowing the glock to be demoted.
+ * This holder should not be cached, so mark it for demote.
+ * Note: this should be done before the check for needs_demote
+ * below.
*/
- while (gh) {
- /*
- * If we're in the process of file system withdraw, we cannot
- * just dequeue any glocks until our journal is recovered, lest
- * we introduce file system corruption. We need two exceptions
- * to this rule: We need to allow unlocking of nondisk glocks
- * and the glock for our own journal that needs recovery.
- */
- if (test_bit(SDF_WITHDRAW_RECOVERY, &sdp->sd_flags) &&
- glock_blocked_by_withdraw(gl) &&
- gh->gh_gl != sdp->sd_jinode_gl) {
- sdp->sd_glock_dqs_held++;
- spin_unlock(&gl->gl_lockref.lock);
- might_sleep();
- wait_on_bit(&sdp->sd_flags, SDF_WITHDRAW_RECOVERY,
- TASK_UNINTERRUPTIBLE);
- spin_lock(&gl->gl_lockref.lock);
- }
+ if (gh->gh_flags & GL_NOCACHE)
+ handle_callback(gl, LM_ST_UNLOCKED, 0, false);
- /*
- * This holder should not be cached, so mark it for demote.
- * Note: this should be done before the check for needs_demote
- * below.
- */
- if (gh->gh_flags & GL_NOCACHE)
- handle_callback(gl, LM_ST_UNLOCKED, 0, false);
-
- list_del_init(&gh->gh_list);
- clear_bit(HIF_HOLDER, &gh->gh_iflags);
- trace_gfs2_glock_queue(gh, 0);
+ list_del_init(&gh->gh_list);
+ clear_bit(HIF_HOLDER, &gh->gh_iflags);
+ trace_gfs2_glock_queue(gh, 0);
- /*
- * If there hasn't been a demote request we are done.
- * (Let the remaining holders, if any, keep holding it.)
- */
- if (!needs_demote(gl)) {
- if (list_empty(&gl->gl_holders))
- fast_path = 1;
- break;
- }
- /*
- * If we have another strong holder (we cannot auto-demote)
- * we are done. It keeps holding it until it is done.
- */
- if (find_first_strong_holder(gl))
- break;
-
- /*
- * If we have a weak holder at the head of the list, it
- * (and all others like it) must be auto-demoted. If there
- * are no more weak holders, we exit the while loop.
- */
- gh = find_first_holder(gl);
+ /*
+ * If there hasn't been a demote request we are done.
+ * (Let the remaining holders, if any, keep holding it.)
+ */
+ if (!needs_demote(gl)) {
+ if (list_empty(&gl->gl_holders))
+ fast_path = 1;
}
if (!test_bit(GLF_LFLUSH, &gl->gl_flags) && demote_ok(gl))
void gfs2_glock_dq(struct gfs2_holder *gh)
{
struct gfs2_glock *gl = gh->gh_gl;
+ struct gfs2_sbd *sdp = gl->gl_name.ln_sbd;
spin_lock(&gl->gl_lockref.lock);
+ if (!gfs2_holder_queued(gh)) {
+ /*
+ * May have already been dequeued because the locking request
+ * was GL_ASYNC and it has failed in the meantime.
+ */
+ goto out;
+ }
+
if (list_is_first(&gh->gh_list, &gl->gl_holders) &&
!test_bit(HIF_HOLDER, &gh->gh_iflags)) {
spin_unlock(&gl->gl_lockref.lock);
spin_lock(&gl->gl_lockref.lock);
}
+ /*
+ * If we're in the process of file system withdraw, we cannot just
+ * dequeue any glocks until our journal is recovered, lest we introduce
+ * file system corruption. We need two exceptions to this rule: We need
+ * to allow unlocking of nondisk glocks and the glock for our own
+ * journal that needs recovery.
+ */
+ if (test_bit(SDF_WITHDRAW_RECOVERY, &sdp->sd_flags) &&
+ glock_blocked_by_withdraw(gl) &&
+ gh->gh_gl != sdp->sd_jinode_gl) {
+ sdp->sd_glock_dqs_held++;
+ spin_unlock(&gl->gl_lockref.lock);
+ might_sleep();
+ wait_on_bit(&sdp->sd_flags, SDF_WITHDRAW_RECOVERY,
+ TASK_UNINTERRUPTIBLE);
+ spin_lock(&gl->gl_lockref.lock);
+ }
+
__gfs2_glock_dq(gh);
+out:
spin_unlock(&gl->gl_lockref.lock);
}
if (test_bit(GLF_REPLY_PENDING, &gl->gl_flags))
delay = gl->gl_hold_time;
}
- /*
- * Note 1: We cannot call demote_incompat_holders from handle_callback
- * or gfs2_set_demote due to recursion problems like: gfs2_glock_dq ->
- * handle_callback -> demote_incompat_holders -> gfs2_glock_dq
- * Plus, we only want to demote the holders if the request comes from
- * a remote cluster node because local holder conflicts are resolved
- * elsewhere.
- *
- * Note 2: if a remote node wants this glock in EX mode, lock_dlm will
- * request that we set our state to UNLOCKED. Here we mock up a holder
- * to make it look like someone wants the lock EX locally. Any SH
- * and DF requests should be able to share the lock without demoting.
- *
- * Note 3: We only want to demote the demoteable holders when there
- * are no more strong holders. The demoteable holders might as well
- * keep the glock until the last strong holder is done with it.
- */
- if (!find_first_strong_holder(gl)) {
- struct gfs2_holder mock_gh = {
- .gh_gl = gl,
- .gh_state = (state == LM_ST_UNLOCKED) ?
- LM_ST_EXCLUSIVE : state,
- .gh_iflags = BIT(HIF_HOLDER)
- };
-
- demote_incompat_holders(gl, &mock_gh);
- }
handle_callback(gl, state, delay, true);
__gfs2_glock_queue_work(gl, delay);
spin_unlock(&gl->gl_lockref.lock);
*p++ = 'H';
if (test_bit(HIF_WAIT, &iflags))
*p++ = 'W';
- if (test_bit(HIF_MAY_DEMOTE, &iflags))
- *p++ = 'D';
if (flags & GL_SKIP)
*p++ = 's';
*p = 0;
list_for_each_entry(gh, &gl->gl_holders, gh_list) {
if (!test_bit(HIF_HOLDER, &gh->gh_iflags))
break;
- if (test_bit(HIF_MAY_DEMOTE, &gh->gh_iflags))
- continue;
if (gh->gh_owner_pid == pid)
goto out;
}
extern int gfs2_glock_get(struct gfs2_sbd *sdp, u64 number,
const struct gfs2_glock_operations *glops,
int create, struct gfs2_glock **glp);
-extern void gfs2_glock_hold(struct gfs2_glock *gl);
+extern struct gfs2_glock *gfs2_glock_hold(struct gfs2_glock *gl);
extern void gfs2_glock_put(struct gfs2_glock *gl);
extern void gfs2_glock_queue_put(struct gfs2_glock *gl);
extern void gfs2_register_debugfs(void);
extern void gfs2_unregister_debugfs(void);
+extern void glock_set_object(struct gfs2_glock *gl, void *object);
+extern void glock_clear_object(struct gfs2_glock *gl, void *object);
+
extern const struct lm_lockops gfs2_dlm_ops;
static inline void gfs2_holder_mark_uninitialized(struct gfs2_holder *gh)
return !list_empty(&gh->gh_list);
}
-/**
- * glock_set_object - set the gl_object field of a glock
- * @gl: the glock
- * @object: the object
- */
-static inline void glock_set_object(struct gfs2_glock *gl, void *object)
-{
- spin_lock(&gl->gl_lockref.lock);
- if (gfs2_assert_warn(gl->gl_name.ln_sbd, gl->gl_object == NULL))
- gfs2_dump_glock(NULL, gl, true);
- gl->gl_object = object;
- spin_unlock(&gl->gl_lockref.lock);
-}
-
-/**
- * glock_clear_object - clear the gl_object field of a glock
- * @gl: the glock
- * @object: the object
- *
- * I'd love to similarly add this:
- * else if (gfs2_assert_warn(gl->gl_sbd, gl->gl_object == object))
- * gfs2_dump_glock(NULL, gl, true);
- * Unfortunately, that's not possible because as soon as gfs2_delete_inode
- * frees the block in the rgrp, another process can reassign it for an I_NEW
- * inode in gfs2_create_inode because that calls new_inode, not gfs2_iget.
- * That means gfs2_delete_inode may subsequently try to call this function
- * for a glock that's already pointing to a brand new inode. If we clear the
- * new inode's gl_object, we'll introduce metadata corruption. Function
- * gfs2_delete_inode calls clear_inode which calls gfs2_clear_inode which also
- * tries to clear gl_object, so it's more than just gfs2_delete_inode.
- *
- */
-static inline void glock_clear_object(struct gfs2_glock *gl, void *object)
-{
- spin_lock(&gl->gl_lockref.lock);
- if (gl->gl_object == object)
- gl->gl_object = NULL;
- spin_unlock(&gl->gl_lockref.lock);
-}
-
-static inline void gfs2_holder_allow_demote(struct gfs2_holder *gh)
-{
- struct gfs2_glock *gl = gh->gh_gl;
-
- spin_lock(&gl->gl_lockref.lock);
- set_bit(HIF_MAY_DEMOTE, &gh->gh_iflags);
- spin_unlock(&gl->gl_lockref.lock);
-}
-
-static inline void gfs2_holder_disallow_demote(struct gfs2_holder *gh)
-{
- struct gfs2_glock *gl = gh->gh_gl;
-
- spin_lock(&gl->gl_lockref.lock);
- clear_bit(HIF_MAY_DEMOTE, &gh->gh_iflags);
- spin_unlock(&gl->gl_lockref.lock);
-}
-
extern void gfs2_inode_remember_delete(struct gfs2_glock *gl, u64 generation);
extern bool gfs2_inode_already_deleted(struct gfs2_glock *gl, u64 generation);
struct timespec64 atime;
u16 height, depth;
umode_t mode = be32_to_cpu(str->di_mode);
- bool is_new = ip->i_inode.i_state & I_NEW;
+ struct inode *inode = &ip->i_inode;
+ bool is_new = inode->i_state & I_NEW;
if (unlikely(ip->i_no_addr != be64_to_cpu(str->di_num.no_addr)))
goto corrupt;
- if (unlikely(!is_new && inode_wrong_type(&ip->i_inode, mode)))
+ if (unlikely(!is_new && inode_wrong_type(inode, mode)))
goto corrupt;
ip->i_no_formal_ino = be64_to_cpu(str->di_num.no_formal_ino);
- ip->i_inode.i_mode = mode;
+ inode->i_mode = mode;
if (is_new) {
- ip->i_inode.i_rdev = 0;
+ inode->i_rdev = 0;
switch (mode & S_IFMT) {
case S_IFBLK:
case S_IFCHR:
- ip->i_inode.i_rdev = MKDEV(be32_to_cpu(str->di_major),
- be32_to_cpu(str->di_minor));
+ inode->i_rdev = MKDEV(be32_to_cpu(str->di_major),
+ be32_to_cpu(str->di_minor));
break;
}
}
- i_uid_write(&ip->i_inode, be32_to_cpu(str->di_uid));
- i_gid_write(&ip->i_inode, be32_to_cpu(str->di_gid));
- set_nlink(&ip->i_inode, be32_to_cpu(str->di_nlink));
- i_size_write(&ip->i_inode, be64_to_cpu(str->di_size));
- gfs2_set_inode_blocks(&ip->i_inode, be64_to_cpu(str->di_blocks));
+ i_uid_write(inode, be32_to_cpu(str->di_uid));
+ i_gid_write(inode, be32_to_cpu(str->di_gid));
+ set_nlink(inode, be32_to_cpu(str->di_nlink));
+ i_size_write(inode, be64_to_cpu(str->di_size));
+ gfs2_set_inode_blocks(inode, be64_to_cpu(str->di_blocks));
atime.tv_sec = be64_to_cpu(str->di_atime);
atime.tv_nsec = be32_to_cpu(str->di_atime_nsec);
- if (timespec64_compare(&ip->i_inode.i_atime, &atime) < 0)
- ip->i_inode.i_atime = atime;
- ip->i_inode.i_mtime.tv_sec = be64_to_cpu(str->di_mtime);
- ip->i_inode.i_mtime.tv_nsec = be32_to_cpu(str->di_mtime_nsec);
- ip->i_inode.i_ctime.tv_sec = be64_to_cpu(str->di_ctime);
- ip->i_inode.i_ctime.tv_nsec = be32_to_cpu(str->di_ctime_nsec);
+ if (timespec64_compare(&inode->i_atime, &atime) < 0)
+ inode->i_atime = atime;
+ inode->i_mtime.tv_sec = be64_to_cpu(str->di_mtime);
+ inode->i_mtime.tv_nsec = be32_to_cpu(str->di_mtime_nsec);
+ inode->i_ctime.tv_sec = be64_to_cpu(str->di_ctime);
+ inode->i_ctime.tv_nsec = be32_to_cpu(str->di_ctime_nsec);
ip->i_goal = be64_to_cpu(str->di_goal_meta);
ip->i_generation = be64_to_cpu(str->di_generation);
ip->i_diskflags = be32_to_cpu(str->di_flags);
ip->i_eattr = be64_to_cpu(str->di_eattr);
/* i_diskflags and i_eattr must be set before gfs2_set_inode_flags() */
- gfs2_set_inode_flags(&ip->i_inode);
+ gfs2_set_inode_flags(inode);
height = be16_to_cpu(str->di_height);
if (unlikely(height > GFS2_MAX_META_HEIGHT))
goto corrupt;
ip->i_depth = (u8)depth;
ip->i_entries = be32_to_cpu(str->di_entries);
- if (S_ISREG(ip->i_inode.i_mode))
- gfs2_set_aops(&ip->i_inode);
+ if (gfs2_is_stuffed(ip) && inode->i_size > gfs2_max_stuffed_size(ip))
+ goto corrupt;
+
+ if (S_ISREG(inode->i_mode))
+ gfs2_set_aops(inode);
return 0;
corrupt:
enum {
/* States */
- HIF_MAY_DEMOTE = 1,
HIF_HOLDER = 6, /* Set for gh that "holds" the glock */
HIF_WAIT = 10,
};
if (unlikely(error))
goto fail;
+ /*
+ * The only caller that sets @blktype to GFS2_BLKST_UNLINKED is
+ * delete_work_func(). Make sure not to cancel the delete work
+ * from within itself here.
+ */
if (blktype == GFS2_BLKST_UNLINKED)
extra_flags |= LM_FLAG_TRY;
else
goto out_ipreserv;
error = gfs2_alloc_blocks(ip, &ip->i_no_addr, dblocks, 1, &ip->i_generation);
+ if (error)
+ goto out_trans_end;
+
ip->i_no_formal_ino = ip->i_generation;
ip->i_inode.i_ino = ip->i_no_addr;
ip->i_goal = ip->i_no_addr;
+ if (*dblocks > 1)
+ ip->i_eattr = ip->i_no_addr + 1;
+out_trans_end:
gfs2_trans_end(sdp);
-
out_ipreserv:
gfs2_inplace_release(ip);
out_quota:
* @size: The initial size of the inode (ignored for directories)
* @excl: Force fail if inode exists
*
+ * FIXME: Change to allocate the disk blocks and write them out in the same
+ * transaction. That way, we can no longer end up in a situation in which an
+ * inode is allocated, the node crashes, and the block looks like a valid
+ * inode. (With atomic creates in place, we will also no longer need to zero
+ * the link count and dirty the inode here on failure.)
+ *
* Returns: 0 on success, or error code
*/
{
const struct qstr *name = &dentry->d_name;
struct posix_acl *default_acl, *acl;
- struct gfs2_holder ghs[2];
+ struct gfs2_holder d_gh, gh;
struct inode *inode = NULL;
struct gfs2_inode *dip = GFS2_I(dir), *ip;
struct gfs2_sbd *sdp = GFS2_SB(&dip->i_inode);
struct gfs2_glock *io_gl;
- int error, free_vfs_inode = 1;
+ int error;
u32 aflags = 0;
unsigned blocks = 1;
struct gfs2_diradd da = { .bh = NULL, .save_loc = 1, };
if (error)
goto fail;
- error = gfs2_glock_nq_init(dip->i_gl, LM_ST_EXCLUSIVE, 0, ghs);
+ error = gfs2_glock_nq_init(dip->i_gl, LM_ST_EXCLUSIVE, 0, &d_gh);
if (error)
goto fail;
- gfs2_holder_mark_uninitialized(ghs + 1);
+ gfs2_holder_mark_uninitialized(&gh);
error = create_ok(dip, name, mode);
if (error)
else
error = finish_no_open(file, NULL);
}
- gfs2_glock_dq_uninit(ghs);
+ gfs2_glock_dq_uninit(&d_gh);
goto fail;
} else if (error != -ENOENT) {
goto fail_gunlock;
error = -ENOMEM;
if (!inode)
goto fail_gunlock;
+ ip = GFS2_I(inode);
error = posix_acl_create(dir, &mode, &default_acl, &acl);
if (error)
goto fail_gunlock;
- ip = GFS2_I(inode);
error = gfs2_qa_get(ip);
if (error)
goto fail_free_acls;
goto fail_free_inode;
gfs2_cancel_delete_work(io_gl);
+retry:
error = insert_inode_locked4(inode, ip->i_no_addr, iget_test, &ip->i_no_addr);
- BUG_ON(error);
+ if (error == -EBUSY)
+ goto retry;
+ if (error)
+ goto fail_gunlock2;
error = gfs2_glock_nq_init(io_gl, LM_ST_SHARED, GL_EXACT | GL_NOPID,
&ip->i_iopen_gh);
if (error)
goto fail_gunlock2;
- error = gfs2_glock_nq_init(ip->i_gl, LM_ST_EXCLUSIVE, GL_SKIP, ghs + 1);
+ error = gfs2_glock_nq_init(ip->i_gl, LM_ST_EXCLUSIVE, GL_SKIP, &gh);
if (error)
goto fail_gunlock3;
if (error)
goto fail_gunlock3;
- if (blocks > 1) {
- ip->i_eattr = ip->i_no_addr + 1;
+ if (blocks > 1)
gfs2_init_xattr(ip);
- }
init_dinode(dip, ip, symname);
gfs2_trans_end(sdp);
glock_set_object(io_gl, ip);
gfs2_set_iop(inode);
- free_vfs_inode = 0; /* After this point, the inode is no longer
- considered free. Any failures need to undo
- the gfs2 structures. */
if (default_acl) {
error = __gfs2_set_acl(inode, default_acl, ACL_TYPE_DEFAULT);
if (error)
file->f_mode |= FMODE_CREATED;
error = finish_open(file, dentry, gfs2_open_common);
}
- gfs2_glock_dq_uninit(ghs);
+ gfs2_glock_dq_uninit(&d_gh);
gfs2_qa_put(ip);
- gfs2_glock_dq_uninit(ghs + 1);
+ gfs2_glock_dq_uninit(&gh);
gfs2_glock_put(io_gl);
gfs2_qa_put(dip);
unlock_new_inode(inode);
fail_gunlock2:
gfs2_glock_put(io_gl);
fail_free_inode:
- if (ip->i_gl) {
- if (free_vfs_inode) /* else evict will do the put for us */
- gfs2_glock_put(ip->i_gl);
- }
gfs2_rs_deltree(&ip->i_res);
gfs2_qa_put(ip);
fail_free_acls:
posix_acl_release(acl);
fail_gunlock:
gfs2_dir_no_add(&da);
- gfs2_glock_dq_uninit(ghs);
+ gfs2_glock_dq_uninit(&d_gh);
if (!IS_ERR_OR_NULL(inode)) {
+ set_bit(GIF_ALLOC_FAILED, &ip->i_flags);
clear_nlink(inode);
- if (!free_vfs_inode)
+ if (ip->i_no_addr)
mark_inode_dirty(inode);
- set_bit(free_vfs_inode ? GIF_FREE_VFS_INODE : GIF_ALLOC_FAILED,
- &GFS2_I(inode)->i_flags);
if (inode->i_state & I_NEW)
iget_failed(inode);
else
iput(inode);
}
- if (gfs2_holder_initialized(ghs + 1))
- gfs2_glock_dq_uninit(ghs + 1);
+ if (gfs2_holder_initialized(&gh))
+ gfs2_glock_dq_uninit(&gh);
fail:
gfs2_qa_put(dip);
return error;
struct buffer_head *bh;
int ty;
+ if (!ip->i_gl) {
+ /* This can only happen during incomplete inode creation. */
+ BUG_ON(!test_bit(GIF_ALLOC_FAILED, &ip->i_flags));
+ return;
+ }
+
gfs2_ail1_wipe(sdp, bstart, blen);
while (blen) {
ty = REMOVE_META;
void gfs2_dinode_out(const struct gfs2_inode *ip, void *buf)
{
+ const struct inode *inode = &ip->i_inode;
struct gfs2_dinode *str = buf;
str->di_header.mh_magic = cpu_to_be32(GFS2_MAGIC);
str->di_header.mh_format = cpu_to_be32(GFS2_FORMAT_DI);
str->di_num.no_addr = cpu_to_be64(ip->i_no_addr);
str->di_num.no_formal_ino = cpu_to_be64(ip->i_no_formal_ino);
- str->di_mode = cpu_to_be32(ip->i_inode.i_mode);
- str->di_uid = cpu_to_be32(i_uid_read(&ip->i_inode));
- str->di_gid = cpu_to_be32(i_gid_read(&ip->i_inode));
- str->di_nlink = cpu_to_be32(ip->i_inode.i_nlink);
- str->di_size = cpu_to_be64(i_size_read(&ip->i_inode));
- str->di_blocks = cpu_to_be64(gfs2_get_inode_blocks(&ip->i_inode));
- str->di_atime = cpu_to_be64(ip->i_inode.i_atime.tv_sec);
- str->di_mtime = cpu_to_be64(ip->i_inode.i_mtime.tv_sec);
- str->di_ctime = cpu_to_be64(ip->i_inode.i_ctime.tv_sec);
+ str->di_mode = cpu_to_be32(inode->i_mode);
+ str->di_uid = cpu_to_be32(i_uid_read(inode));
+ str->di_gid = cpu_to_be32(i_gid_read(inode));
+ str->di_nlink = cpu_to_be32(inode->i_nlink);
+ str->di_size = cpu_to_be64(i_size_read(inode));
+ str->di_blocks = cpu_to_be64(gfs2_get_inode_blocks(inode));
+ str->di_atime = cpu_to_be64(inode->i_atime.tv_sec);
+ str->di_mtime = cpu_to_be64(inode->i_mtime.tv_sec);
+ str->di_ctime = cpu_to_be64(inode->i_ctime.tv_sec);
str->di_goal_meta = cpu_to_be64(ip->i_goal);
str->di_goal_data = cpu_to_be64(ip->i_goal);
str->di_flags = cpu_to_be32(ip->i_diskflags);
str->di_height = cpu_to_be16(ip->i_height);
- str->di_payload_format = cpu_to_be32(S_ISDIR(ip->i_inode.i_mode) &&
+ str->di_payload_format = cpu_to_be32(S_ISDIR(inode->i_mode) &&
!(ip->i_diskflags & GFS2_DIF_EXHASH) ?
GFS2_FORMAT_DE : 0);
str->di_depth = cpu_to_be16(ip->i_depth);
str->di_entries = cpu_to_be32(ip->i_entries);
str->di_eattr = cpu_to_be64(ip->i_eattr);
- str->di_atime_nsec = cpu_to_be32(ip->i_inode.i_atime.tv_nsec);
- str->di_mtime_nsec = cpu_to_be32(ip->i_inode.i_mtime.tv_nsec);
- str->di_ctime_nsec = cpu_to_be32(ip->i_inode.i_ctime.tv_nsec);
+ str->di_atime_nsec = cpu_to_be32(inode->i_atime.tv_nsec);
+ str->di_mtime_nsec = cpu_to_be32(inode->i_mtime.tv_nsec);
+ str->di_ctime_nsec = cpu_to_be32(inode->i_ctime.tv_nsec);
}
/**
int need_endtrans = 0;
int ret;
+ if (unlikely(!ip->i_gl)) {
+ /* This can only happen during incomplete inode creation. */
+ BUG_ON(!test_bit(GIF_ALLOC_FAILED, &ip->i_flags));
+ return;
+ }
+
if (unlikely(gfs2_withdrawn(sdp)))
return;
if (!gfs2_glock_is_locked_by_me(ip->i_gl)) {
{
struct gfs2_inode *ip = GFS2_I(inode);
- if (!test_bit(GIF_FREE_VFS_INODE, &ip->i_flags) &&
- inode->i_nlink &&
+ if (inode->i_nlink &&
gfs2_holder_initialized(&ip->i_iopen_gh)) {
struct gfs2_glock *gl = ip->i_iopen_gh.gh_gl;
if (test_bit(GLF_DEMOTE, &gl->gl_flags))
struct inode *inode = &ip->i_inode;
struct gfs2_glock *gl = ip->i_gl;
- truncate_inode_pages(gfs2_glock2aspace(ip->i_gl), 0);
+ if (unlikely(!gl)) {
+ /* This can only happen during incomplete inode creation. */
+ BUG_ON(!test_bit(GIF_ALLOC_FAILED, &ip->i_flags));
+ return;
+ }
+
+ truncate_inode_pages(gfs2_glock2aspace(gl), 0);
truncate_inode_pages(&inode->i_data, 0);
if (atomic_read(&gl->gl_revokes) == 0) {
struct gfs2_sbd *sdp = sb->s_fs_info;
int ret;
- if (test_bit(GIF_ALLOC_FAILED, &ip->i_flags)) {
- BUG_ON(!gfs2_glock_is_locked_by_me(ip->i_gl));
+ if (unlikely(test_bit(GIF_ALLOC_FAILED, &ip->i_flags)))
goto should_delete;
- }
if (test_bit(GIF_DEFERRED_DELETE, &ip->i_flags))
return SHOULD_DEFER_EVICTION;
goto out;
}
- /* We're about to clear the bitmap for the dinode, but as soon as we
- do, gfs2_create_inode can create another inode at the same block
- location and try to set gl_object again. We clear gl_object here so
- that subsequent inode creates don't see an old gl_object. */
- glock_clear_object(ip->i_gl, ip);
+ if (ip->i_gl)
+ gfs2_inode_remember_delete(ip->i_gl, ip->i_no_formal_ino);
+
+ /*
+ * As soon as we clear the bitmap for the dinode, gfs2_create_inode()
+ * can get called to recreate it, or even gfs2_inode_lookup() if the
+ * inode was recreated on another node in the meantime.
+ *
+ * However, inserting the new inode into the inode hash table will not
+ * succeed until the old inode is removed, and that only happens after
+ * ->evict_inode() returns. The new inode is attached to its inode and
+ * iopen glocks after inserting it into the inode hash table, so at
+ * that point we can be sure that both glocks are unused.
+ */
+
ret = gfs2_dinode_dealloc(ip);
- gfs2_inode_remember_delete(ip->i_gl, ip->i_no_formal_ino);
out:
return ret;
}
struct gfs2_holder gh;
int ret;
- if (test_bit(GIF_FREE_VFS_INODE, &ip->i_flags)) {
- clear_inode(inode);
- return;
- }
-
- if (inode->i_nlink || sb_rdonly(sb))
+ if (inode->i_nlink || sb_rdonly(sb) || !ip->i_no_addr)
goto out;
gfs2_holder_mark_uninitialized(&gh);
struct gfs2_glock *gl = ip->i_iopen_gh.gh_gl;
glock_clear_object(gl, ip);
- if (test_bit(HIF_HOLDER, &ip->i_iopen_gh.gh_iflags)) {
- ip->i_iopen_gh.gh_flags |= GL_NOCACHE;
- gfs2_glock_dq(&ip->i_iopen_gh);
- }
gfs2_glock_hold(gl);
- gfs2_holder_uninit(&ip->i_iopen_gh);
+ ip->i_iopen_gh.gh_flags |= GL_NOCACHE;
+ gfs2_glock_dq_uninit(&ip->i_iopen_gh);
gfs2_glock_put_eventually(gl);
}
if (ip->i_gl) {
ip = alloc_inode_sb(sb, gfs2_inode_cachep, GFP_KERNEL);
if (!ip)
return NULL;
+ ip->i_no_addr = 0;
ip->i_flags = 0;
ip->i_gl = NULL;
gfs2_holder_mark_uninitialized(&ip->i_iopen_gh);
ip->i_eattr = 0;
gfs2_add_inode_blocks(&ip->i_inode, -1);
- error = gfs2_meta_inode_buffer(ip, &dibh);
- if (!error) {
- gfs2_trans_add_meta(ip->i_gl, dibh);
- gfs2_dinode_out(ip, dibh->b_data);
- brelse(dibh);
+ if (likely(!test_bit(GIF_ALLOC_FAILED, &ip->i_flags))) {
+ error = gfs2_meta_inode_buffer(ip, &dibh);
+ if (!error) {
+ gfs2_trans_add_meta(ip->i_gl, dibh);
+ gfs2_dinode_out(ip, dibh->b_data);
+ brelse(dibh);
+ }
}
gfs2_trans_end(sdp);
if (error)
return error;
- error = ea_foreach(ip, ea_dealloc_unstuffed, NULL);
- if (error)
- goto out_quota;
-
- if (ip->i_diskflags & GFS2_DIF_EA_INDIRECT) {
- error = ea_dealloc_indirect(ip);
+ if (likely(!test_bit(GIF_ALLOC_FAILED, &ip->i_flags))) {
+ error = ea_foreach(ip, ea_dealloc_unstuffed, NULL);
if (error)
goto out_quota;
+
+ if (ip->i_diskflags & GFS2_DIF_EA_INDIRECT) {
+ error = ea_dealloc_indirect(ip);
+ if (error)
+ goto out_quota;
+ }
}
error = ea_dealloc_block(ip);
* kn_to: /n1/n2/n3 [depth=3]
* result: /../..
*
- * [3] when @kn_to is NULL result will be "(null)"
+ * [3] when @kn_to is %NULL result will be "(null)"
*
- * Returns the length of the full path. If the full length is equal to or
+ * Return: the length of the full path. If the full length is equal to or
* greater than @buflen, @buf contains the truncated path with the trailing
* '\0'. On error, -errno is returned.
*/
* @buflen: size of @buf
*
* Copies the name of @kn into @buf of @buflen bytes. The behavior is
- * similar to strlcpy(). It returns the length of @kn's name and if @buf
- * isn't long enough, it's filled upto @buflen-1 and nul terminated.
+ * similar to strlcpy().
*
- * Fills buffer with "(null)" if @kn is NULL.
+ * Fills buffer with "(null)" if @kn is %NULL.
+ *
+ * Return: the length of @kn's name and if @buf isn't long enough,
+ * it's filled up to @buflen-1 and nul terminated.
*
* This function can be called from any context.
*/
* path (which includes '..'s) as needed to reach from @from to @to is
* returned.
*
- * Returns the length of the full path. If the full length is equal to or
+ * Return: the length of the full path. If the full length is equal to or
* greater than @buflen, @buf contains the truncated path with the trailing
* '\0'. On error, -errno is returned.
*/
*
* Determines @kn's parent, pins and returns it. This function can be
* called from any context.
+ *
+ * Return: parent node of @kn
*/
struct kernfs_node *kernfs_get_parent(struct kernfs_node *kn)
{
}
/**
- * kernfs_name_hash
+ * kernfs_name_hash - calculate hash of @ns + @name
* @name: Null terminated string to hash
* @ns: Namespace tag to hash
*
- * Returns 31 bit hash of ns + name (so it fits in an off_t )
+ * Return: 31-bit hash of ns + name (so it fits in an off_t)
*/
static unsigned int kernfs_name_hash(const char *name, const void *ns)
{
* Locking:
* kernfs_rwsem held exclusive
*
- * RETURNS:
- * 0 on susccess -EEXIST on failure.
+ * Return:
+ * %0 on success, -EEXIST on failure.
*/
static int kernfs_link_sibling(struct kernfs_node *kn)
{
* @kn: kernfs_node of interest
*
* Try to unlink @kn from its sibling rbtree which starts from
- * kn->parent->dir.children. Returns %true if @kn was actually
- * removed, %false if @kn wasn't on the rbtree.
+ * kn->parent->dir.children.
+ *
+ * Return: %true if @kn was actually removed,
+ * %false if @kn wasn't on the rbtree.
*
* Locking:
* kernfs_rwsem held exclusive
* @kn: kernfs_node to get an active reference to
*
* Get an active reference of @kn. This function is noop if @kn
- * is NULL.
+ * is %NULL.
*
- * RETURNS:
- * Pointer to @kn on success, NULL on failure.
+ * Return:
+ * Pointer to @kn on success, %NULL on failure.
*/
struct kernfs_node *kernfs_get_active(struct kernfs_node *kn)
{
* @kn: kernfs_node to put an active reference to
*
* Put an active reference to @kn. This function is noop if @kn
- * is NULL.
+ * is %NULL.
*/
void kernfs_put_active(struct kernfs_node *kn)
{
* kernfs_drain - drain kernfs_node
* @kn: kernfs_node to drain
*
- * Drain existing usages and nuke all existing mmaps of @kn. Mutiple
+ * Drain existing usages and nuke all existing mmaps of @kn. Multiple
* removers may invoke this function concurrently on @kn and all will
* return after draining is complete.
*/
* kernfs_node_from_dentry - determine kernfs_node associated with a dentry
* @dentry: the dentry in question
*
- * Return the kernfs_node associated with @dentry. If @dentry is not a
+ * Return: the kernfs_node associated with @dentry. If @dentry is not a
* kernfs one, %NULL is returned.
*
* While the returned kernfs_node will stay accessible as long as @dentry
* @id's lower 32bits encode ino and upper gen. If the gen portion is
* zero, all generations are matched.
*
- * RETURNS:
- * NULL on failure. Return a kernfs node with reference counter incremented
+ * Return: %NULL on failure,
+ * otherwise a kernfs node with reference counter incremented.
*/
struct kernfs_node *kernfs_find_and_get_node_by_id(struct kernfs_root *root,
u64 id)
* function increments nlink of the parent's inode if @kn is a
* directory and link into the children list of the parent.
*
- * RETURNS:
- * 0 on success, -EEXIST if entry with the given name already
+ * Return:
+ * %0 on success, -EEXIST if entry with the given name already
* exists.
*/
int kernfs_add_one(struct kernfs_node *kn)
* @name: name to look for
* @ns: the namespace tag to use
*
- * Look for kernfs_node with name @name under @parent. Returns pointer to
- * the found kernfs_node on success, %NULL on failure.
+ * Look for kernfs_node with name @name under @parent.
+ *
+ * Return: pointer to the found kernfs_node on success, %NULL on failure.
*/
static struct kernfs_node *kernfs_find_ns(struct kernfs_node *parent,
const unsigned char *name,
* @ns: the namespace tag to use
*
* Look for kernfs_node with name @name under @parent and get a reference
- * if found. This function may sleep and returns pointer to the found
- * kernfs_node on success, %NULL on failure.
+ * if found. This function may sleep.
+ *
+ * Return: pointer to the found kernfs_node on success, %NULL on failure.
*/
struct kernfs_node *kernfs_find_and_get_ns(struct kernfs_node *parent,
const char *name, const void *ns)
* @ns: the namespace tag to use
*
* Look for kernfs_node with path @path under @parent and get a reference
- * if found. This function may sleep and returns pointer to the found
- * kernfs_node on success, %NULL on failure.
+ * if found. This function may sleep.
+ *
+ * Return: pointer to the found kernfs_node on success, %NULL on failure.
*/
struct kernfs_node *kernfs_walk_and_get_ns(struct kernfs_node *parent,
const char *path, const void *ns)
* @flags: KERNFS_ROOT_* flags
* @priv: opaque data associated with the new directory
*
- * Returns the root of the new hierarchy on success, ERR_PTR() value on
+ * Return: the root of the new hierarchy on success, ERR_PTR() value on
* failure.
*/
struct kernfs_root *kernfs_create_root(struct kernfs_syscall_ops *scops,
/**
* kernfs_root_to_node - return the kernfs_node associated with a kernfs_root
* @root: root to use to lookup
+ *
+ * Return: @root's kernfs_node
*/
struct kernfs_node *kernfs_root_to_node(struct kernfs_root *root)
{
* @priv: opaque data associated with the new directory
* @ns: optional namespace tag of the directory
*
- * Returns the created node on success, ERR_PTR() value on failure.
+ * Return: the created node on success, ERR_PTR() value on failure.
*/
struct kernfs_node *kernfs_create_dir_ns(struct kernfs_node *parent,
const char *name, umode_t mode,
* @parent: parent in which to create a new directory
* @name: name of the new directory
*
- * Returns the created node on success, ERR_PTR() value on failure.
+ * Return: the created node on success, ERR_PTR() value on failure.
*/
struct kernfs_node *kernfs_create_empty_dir(struct kernfs_node *parent,
const char *name)
/* If the kernfs parent node has changed discard and
* proceed to ->lookup.
+ *
+ * There's nothing special needed here when getting the
+ * dentry parent, even if a concurrent rename is in
+ * progress. That's because the dentry is negative so
+ * it can only be the target of the rename and it will
+ * be doing a d_move() not a replace. Consequently the
+ * dentry d_parent won't change over the d_move().
+ *
+ * Also kernfs negative dentries transitioning from
+ * negative to positive during revalidate won't happen
+ * because they are invalidated on containing directory
+ * changes and the lookup re-done so that a new positive
+ * dentry can be properly created.
*/
- spin_lock(&dentry->d_lock);
+ root = kernfs_root_from_sb(dentry->d_sb);
+ down_read(&root->kernfs_rwsem);
parent = kernfs_dentry_node(dentry->d_parent);
if (parent) {
- spin_unlock(&dentry->d_lock);
- root = kernfs_root(parent);
- down_read(&root->kernfs_rwsem);
if (kernfs_dir_changed(parent, dentry)) {
up_read(&root->kernfs_rwsem);
return 0;
}
- up_read(&root->kernfs_rwsem);
- } else
- spin_unlock(&dentry->d_lock);
+ }
+ up_read(&root->kernfs_rwsem);
/* The kernfs parent node hasn't changed, leave the
* dentry negative and return success.
* Find the next descendant to visit for post-order traversal of @root's
* descendants. @root is included in the iteration and the last node to be
* visited.
+ *
+ * Return: the next descendant to visit or %NULL when done.
*/
static struct kernfs_node *kernfs_next_descendant_post(struct kernfs_node *pos,
struct kernfs_node *root)
* the whole kernfs_ops which won the arbitration. This can be used to
* guarantee, for example, all concurrent writes to a "delete" file to
* finish only after the whole operation is complete.
+ *
+ * Return: %true if @kn is removed by this call, otherwise %false.
*/
bool kernfs_remove_self(struct kernfs_node *kn)
{
* @ns: namespace tag of the kernfs_node to remove
*
* Look for the kernfs_node with @name and @ns under @parent and remove it.
- * Returns 0 on success, -ENOENT if such entry doesn't exist.
+ *
+ * Return: %0 on success, -ENOENT if such entry doesn't exist.
*/
int kernfs_remove_by_name_ns(struct kernfs_node *parent, const char *name,
const void *ns)
* @new_parent: new parent to put @sd under
* @new_name: new name
* @new_ns: new namespace tag
+ *
+ * Return: %0 on success, -errno on failure.
*/
int kernfs_rename_ns(struct kernfs_node *kn, struct kernfs_node *new_parent,
const char *new_name, const void *new_ns)
* pending queue is implemented as a singly linked list of kernfs_nodes.
* The list is terminated with the self pointer so that whether a
* kernfs_node is on the list or not can be determined by testing the next
- * pointer for NULL.
+ * pointer for %NULL.
*/
#define KERNFS_NOTIFY_EOL ((void *)&kernfs_notify_list)
}
/**
- * of_on - Return the kernfs_open_node of the specified kernfs_open_file
- * @of: taret kernfs_open_file
+ * of_on - Get the kernfs_open_node of the specified kernfs_open_file
+ * @of: target kernfs_open_file
+ *
+ * Return: the kernfs_open_node of the kernfs_open_file
*/
static struct kernfs_open_node *of_on(struct kernfs_open_file *of)
{
* outside RCU read-side critical section.
*
* The caller needs to make sure that kernfs_open_file_mutex is held.
+ *
+ * Return: @kn->attr.open when kernfs_open_file_mutex is held.
*/
static struct kernfs_open_node *
kernfs_deref_open_node_locked(struct kernfs_node *kn)
* If @kn->attr.open exists, increment its reference count; otherwise,
* create one. @of is chained to the files list.
*
- * LOCKING:
+ * Locking:
* Kernel thread context (may sleep).
*
- * RETURNS:
- * 0 on success, -errno on failure.
+ * Return:
+ * %0 on success, -errno on failure.
*/
static int kernfs_get_open_node(struct kernfs_node *kn,
struct kernfs_open_file *of)
* @ns: optional namespace tag of the file
* @key: lockdep key for the file's active_ref, %NULL to disable lockdep
*
- * Returns the created node on success, ERR_PTR() value on error.
+ * Return: the created node on success, ERR_PTR() value on error.
*/
struct kernfs_node *__kernfs_create_file(struct kernfs_node *parent,
const char *name,
* @kn: target node
* @iattr: iattr to set
*
- * Returns 0 on success, -errno on failure.
+ * Return: %0 on success, -errno on failure.
*/
int kernfs_setattr(struct kernfs_node *kn, const struct iattr *iattr)
{
struct kernfs_root *root = kernfs_root(kn);
down_read(&root->kernfs_rwsem);
- spin_lock(&inode->i_lock);
kernfs_refresh_inode(kn, inode);
generic_fillattr(&init_user_ns, inode, stat);
- spin_unlock(&inode->i_lock);
up_read(&root->kernfs_rwsem);
return 0;
* allocated and basics are initialized. New inode is returned
* locked.
*
- * LOCKING:
+ * Locking:
* Kernel thread context (may sleep).
*
- * RETURNS:
- * Pointer to allocated inode on success, NULL on failure.
+ * Return:
+ * Pointer to allocated inode on success, %NULL on failure.
*/
struct inode *kernfs_get_inode(struct super_block *sb, struct kernfs_node *kn)
{
root = kernfs_root(kn);
down_read(&root->kernfs_rwsem);
- spin_lock(&inode->i_lock);
kernfs_refresh_inode(kn, inode);
ret = generic_permission(&init_user_ns, inode, mask);
- spin_unlock(&inode->i_lock);
up_read(&root->kernfs_rwsem);
return ret;
* kernfs_root - find out the kernfs_root a kernfs_node belongs to
* @kn: kernfs_node of interest
*
- * Return the kernfs_root @kn belongs to.
+ * Return: the kernfs_root @kn belongs to.
*/
static inline struct kernfs_root *kernfs_root(struct kernfs_node *kn)
{
* kernfs_root_from_sb - determine kernfs_root associated with a super_block
* @sb: the super_block in question
*
- * Return the kernfs_root associated with @sb. If @sb is not a kernfs one,
+ * Return: the kernfs_root associated with @sb. If @sb is not a kernfs one,
* %NULL is returned.
*/
struct kernfs_root *kernfs_root_from_sb(struct super_block *sb)
* find the next ancestor in the path down to @child, where @parent was the
* ancestor whose descendant we want to find.
*
- * Say the path is /a/b/c/d. @child is d, @parent is NULL. We return the root
+ * Say the path is /a/b/c/d. @child is d, @parent is %NULL. We return the root
* node. If @parent is b, then we return the node for c.
* Passing in d as @parent is not ok.
*/
* kernfs_node_dentry - get a dentry for the given kernfs_node
* @kn: kernfs_node for which a dentry is needed
* @sb: the kernfs super_block
+ *
+ * Return: the dentry pointer
*/
struct dentry *kernfs_node_dentry(struct kernfs_node *kn,
struct super_block *sb)
* kernfs_super_ns - determine the namespace tag of a kernfs super_block
* @sb: super_block of interest
*
- * Return the namespace tag associated with kernfs super_block @sb.
+ * Return: the namespace tag associated with kernfs super_block @sb.
*/
const void *kernfs_super_ns(struct super_block *sb)
{
* implementation, which should set the specified ->@fs_type and ->@flags, and
* specify the hierarchy and namespace tag to mount via ->@root and ->@ns,
* respectively.
+ *
+ * Return: %0 on success, -errno on failure.
*/
int kernfs_get_tree(struct fs_context *fc)
{
* @name: name of the symlink
* @target: target node for the symlink to point to
*
- * Returns the created node on success, ERR_PTR() value on error.
+ * Return: the created node on success, ERR_PTR() value on error.
* Ownership of the link matches ownership of the target.
*/
struct kernfs_node *kernfs_create_link(struct kernfs_node *parent,
}
static const struct kobj_ns_type_operations *nfs_netns_object_child_ns_type(
- struct kobject *kobj)
+ const struct kobject *kobj)
{
return &net_ns_type_operations;
}
kfree(c);
}
-static const void *nfs_netns_client_namespace(struct kobject *kobj)
+static const void *nfs_netns_client_namespace(const struct kobject *kobj)
{
return container_of(kobj, struct nfs_netns_client, kobject)->net;
}
if (key->gc)
__set_bit(NFSD_FILE_GC, &nf->nf_flags);
nf->nf_inode = key->inode;
- /* nf_ref is pre-incremented for hash table */
- refcount_set(&nf->nf_ref, 2);
+ refcount_set(&nf->nf_ref, 1);
nf->nf_may = key->need;
nf->nf_mark = NULL;
}
return false;
}
-static bool
+static void
nfsd_file_free(struct nfsd_file *nf)
{
s64 age = ktime_to_ms(ktime_sub(ktime_get(), nf->nf_birthtime));
- bool flush = false;
trace_nfsd_file_free(nf);
this_cpu_inc(nfsd_file_releases);
this_cpu_add(nfsd_file_total_age, age);
+ nfsd_file_unhash(nf);
+
+ /*
+ * We call fsync here in order to catch writeback errors. It's not
+ * strictly required by the protocol, but an nfsd_file could get
+ * evicted from the cache before a COMMIT comes in. If another
+ * task were to open that file in the interim and scrape the error,
+ * then the client may never see it. By calling fsync here, we ensure
+ * that writeback happens before the entry is freed, and that any
+ * errors reported result in the write verifier changing.
+ */
+ nfsd_file_fsync(nf);
+
if (nf->nf_mark)
nfsd_file_mark_put(nf->nf_mark);
if (nf->nf_file) {
get_file(nf->nf_file);
filp_close(nf->nf_file, NULL);
fput(nf->nf_file);
- flush = true;
}
/*
* WARN and leak it to preserve system stability.
*/
if (WARN_ON_ONCE(!list_empty(&nf->nf_lru)))
- return flush;
+ return;
call_rcu(&nf->nf_rcu, nfsd_file_slab_free);
- return flush;
}
static bool
mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK);
}
-static void nfsd_file_lru_add(struct nfsd_file *nf)
+static bool nfsd_file_lru_add(struct nfsd_file *nf)
{
set_bit(NFSD_FILE_REFERENCED, &nf->nf_flags);
- if (list_lru_add(&nfsd_file_lru, &nf->nf_lru))
+ if (list_lru_add(&nfsd_file_lru, &nf->nf_lru)) {
trace_nfsd_file_lru_add(nf);
+ return true;
+ }
+ return false;
}
-static void nfsd_file_lru_remove(struct nfsd_file *nf)
+static bool nfsd_file_lru_remove(struct nfsd_file *nf)
{
- if (list_lru_del(&nfsd_file_lru, &nf->nf_lru))
+ if (list_lru_del(&nfsd_file_lru, &nf->nf_lru)) {
trace_nfsd_file_lru_del(nf);
+ return true;
+ }
+ return false;
}
struct nfsd_file *
return NULL;
}
-static void
-nfsd_file_unhash_and_queue(struct nfsd_file *nf, struct list_head *dispose)
-{
- trace_nfsd_file_unhash_and_queue(nf);
- if (nfsd_file_unhash(nf)) {
- /* caller must call nfsd_file_dispose_list() later */
- nfsd_file_lru_remove(nf);
- list_add(&nf->nf_lru, dispose);
- }
-}
-
-static void
-nfsd_file_put_noref(struct nfsd_file *nf)
-{
- trace_nfsd_file_put(nf);
-
- if (refcount_dec_and_test(&nf->nf_ref)) {
- WARN_ON(test_bit(NFSD_FILE_HASHED, &nf->nf_flags));
- nfsd_file_lru_remove(nf);
- nfsd_file_free(nf);
- }
-}
-
-static void
-nfsd_file_unhash_and_put(struct nfsd_file *nf)
-{
- if (nfsd_file_unhash(nf))
- nfsd_file_put_noref(nf);
-}
-
+/**
+ * nfsd_file_put - put the reference to a nfsd_file
+ * @nf: nfsd_file of which to put the reference
+ *
+ * Put a reference to a nfsd_file. In the non-GC case, we just put the
+ * reference immediately. In the GC case, if the reference would be
+ * the last one, the put it on the LRU instead to be cleaned up later.
+ */
void
nfsd_file_put(struct nfsd_file *nf)
{
might_sleep();
+ trace_nfsd_file_put(nf);
- if (test_bit(NFSD_FILE_GC, &nf->nf_flags))
- nfsd_file_lru_add(nf);
- else if (refcount_read(&nf->nf_ref) == 2)
- nfsd_file_unhash_and_put(nf);
-
- if (!test_bit(NFSD_FILE_HASHED, &nf->nf_flags)) {
- nfsd_file_fsync(nf);
- nfsd_file_put_noref(nf);
- } else if (nf->nf_file && test_bit(NFSD_FILE_GC, &nf->nf_flags)) {
- nfsd_file_put_noref(nf);
- nfsd_file_schedule_laundrette();
- } else
- nfsd_file_put_noref(nf);
-}
-
-static void
-nfsd_file_dispose_list(struct list_head *dispose)
-{
- struct nfsd_file *nf;
+ if (test_bit(NFSD_FILE_GC, &nf->nf_flags) &&
+ test_bit(NFSD_FILE_HASHED, &nf->nf_flags)) {
+ /*
+ * If this is the last reference (nf_ref == 1), then try to
+ * transfer it to the LRU.
+ */
+ if (refcount_dec_not_one(&nf->nf_ref))
+ return;
+
+ /* Try to add it to the LRU. If that fails, decrement. */
+ if (nfsd_file_lru_add(nf)) {
+ /* If it's still hashed, we're done */
+ if (test_bit(NFSD_FILE_HASHED, &nf->nf_flags)) {
+ nfsd_file_schedule_laundrette();
+ return;
+ }
- while(!list_empty(dispose)) {
- nf = list_first_entry(dispose, struct nfsd_file, nf_lru);
- list_del_init(&nf->nf_lru);
- nfsd_file_fsync(nf);
- nfsd_file_put_noref(nf);
+ /*
+ * We're racing with unhashing, so try to remove it from
+ * the LRU. If removal fails, then someone else already
+ * has our reference.
+ */
+ if (!nfsd_file_lru_remove(nf))
+ return;
+ }
}
+ if (refcount_dec_and_test(&nf->nf_ref))
+ nfsd_file_free(nf);
}
static void
-nfsd_file_dispose_list_sync(struct list_head *dispose)
+nfsd_file_dispose_list(struct list_head *dispose)
{
- bool flush = false;
struct nfsd_file *nf;
- while(!list_empty(dispose)) {
+ while (!list_empty(dispose)) {
nf = list_first_entry(dispose, struct nfsd_file, nf_lru);
list_del_init(&nf->nf_lru);
- nfsd_file_fsync(nf);
- if (!refcount_dec_and_test(&nf->nf_ref))
- continue;
- if (nfsd_file_free(nf))
- flush = true;
+ nfsd_file_free(nf);
}
- if (flush)
- flush_delayed_fput();
}
static void
struct list_head *head = arg;
struct nfsd_file *nf = list_entry(item, struct nfsd_file, nf_lru);
- /*
- * Do a lockless refcount check. The hashtable holds one reference, so
- * we look to see if anything else has a reference, or if any have
- * been put since the shrinker last ran. Those don't get unhashed and
- * released.
- *
- * Note that in the put path, we set the flag and then decrement the
- * counter. Here we check the counter and then test and clear the flag.
- * That order is deliberate to ensure that we can do this locklessly.
- */
- if (refcount_read(&nf->nf_ref) > 1) {
- list_lru_isolate(lru, &nf->nf_lru);
- trace_nfsd_file_gc_in_use(nf);
- return LRU_REMOVED;
- }
+ /* We should only be dealing with GC entries here */
+ WARN_ON_ONCE(!test_bit(NFSD_FILE_GC, &nf->nf_flags));
/*
* Don't throw out files that are still undergoing I/O or
return LRU_SKIP;
}
+ /* If it was recently added to the list, skip it */
if (test_and_clear_bit(NFSD_FILE_REFERENCED, &nf->nf_flags)) {
trace_nfsd_file_gc_referenced(nf);
return LRU_ROTATE;
}
- if (!test_and_clear_bit(NFSD_FILE_HASHED, &nf->nf_flags)) {
- trace_nfsd_file_gc_hashed(nf);
- return LRU_SKIP;
+ /*
+ * Put the reference held on behalf of the LRU. If it wasn't the last
+ * one, then just remove it from the LRU and ignore it.
+ */
+ if (!refcount_dec_and_test(&nf->nf_ref)) {
+ trace_nfsd_file_gc_in_use(nf);
+ list_lru_isolate(lru, &nf->nf_lru);
+ return LRU_REMOVED;
}
+ /* Refcount went to zero. Unhash it and queue it to the dispose list */
+ nfsd_file_unhash(nf);
list_lru_isolate_move(lru, &nf->nf_lru, head);
this_cpu_inc(nfsd_file_evictions);
trace_nfsd_file_gc_disposed(nf);
return LRU_REMOVED;
}
-/*
- * Unhash items on @dispose immediately, then queue them on the
- * disposal workqueue to finish releasing them in the background.
- *
- * cel: Note that between the time list_lru_shrink_walk runs and
- * now, these items are in the hash table but marked unhashed.
- * Why release these outside of lru_cb ? There's no lock ordering
- * problem since lru_cb currently takes no lock.
- */
-static void nfsd_file_gc_dispose_list(struct list_head *dispose)
-{
- struct nfsd_file *nf;
-
- list_for_each_entry(nf, dispose, nf_lru)
- nfsd_file_hash_remove(nf);
- nfsd_file_dispose_list_delayed(dispose);
-}
-
static void
nfsd_file_gc(void)
{
ret = list_lru_walk(&nfsd_file_lru, nfsd_file_lru_cb,
&dispose, list_lru_count(&nfsd_file_lru));
trace_nfsd_file_gc_removed(ret, list_lru_count(&nfsd_file_lru));
- nfsd_file_gc_dispose_list(&dispose);
+ nfsd_file_dispose_list_delayed(&dispose);
}
static void
ret = list_lru_shrink_walk(&nfsd_file_lru, sc,
nfsd_file_lru_cb, &dispose);
trace_nfsd_file_shrinker_removed(ret, list_lru_count(&nfsd_file_lru));
- nfsd_file_gc_dispose_list(&dispose);
+ nfsd_file_dispose_list_delayed(&dispose);
return ret;
}
.seeks = 1,
};
-/*
- * Find all cache items across all net namespaces that match @inode and
- * move them to @dispose. The lookup is atomic wrt nfsd_file_acquire().
+/**
+ * nfsd_file_queue_for_close: try to close out any open nfsd_files for an inode
+ * @inode: inode on which to close out nfsd_files
+ * @dispose: list on which to gather nfsd_files to close out
+ *
+ * An nfsd_file represents a struct file being held open on behalf of nfsd. An
+ * open file however can block other activity (such as leases), or cause
+ * undesirable behavior (e.g. spurious silly-renames when reexporting NFS).
+ *
+ * This function is intended to find open nfsd_files when this sort of
+ * conflicting access occurs and then attempt to close those files out.
+ *
+ * Populates the dispose list with entries that have already had their
+ * refcounts go to zero. The actual free of an nfsd_file can be expensive,
+ * so we leave it up to the caller whether it wants to wait or not.
*/
-static unsigned int
-__nfsd_file_close_inode(struct inode *inode, struct list_head *dispose)
+static void
+nfsd_file_queue_for_close(struct inode *inode, struct list_head *dispose)
{
struct nfsd_file_lookup_key key = {
.type = NFSD_FILE_KEY_INODE,
.inode = inode,
};
- unsigned int count = 0;
struct nfsd_file *nf;
rcu_read_lock();
do {
+ int decrement = 1;
+
nf = rhashtable_lookup(&nfsd_file_rhash_tbl, &key,
nfsd_file_rhash_params);
if (!nf)
break;
- nfsd_file_unhash_and_queue(nf, dispose);
- count++;
+
+ /* If we raced with someone else unhashing, ignore it */
+ if (!nfsd_file_unhash(nf))
+ continue;
+
+ /* If we can't get a reference, ignore it */
+ if (!nfsd_file_get(nf))
+ continue;
+
+ /* Extra decrement if we remove from the LRU */
+ if (nfsd_file_lru_remove(nf))
+ ++decrement;
+
+ /* If refcount goes to 0, then put on the dispose list */
+ if (refcount_sub_and_test(decrement, &nf->nf_ref)) {
+ list_add(&nf->nf_lru, dispose);
+ trace_nfsd_file_closing(nf);
+ }
} while (1);
rcu_read_unlock();
- return count;
}
/**
- * nfsd_file_close_inode_sync - attempt to forcibly close a nfsd_file
+ * nfsd_file_close_inode - attempt a delayed close of a nfsd_file
* @inode: inode of the file to attempt to remove
*
- * Unhash and put, then flush and fput all cache items associated with @inode.
+ * Close out any open nfsd_files that can be reaped for @inode. The
+ * actual freeing is deferred to the dispose_list_delayed infrastructure.
+ *
+ * This is used by the fsnotify callbacks and setlease notifier.
*/
-void
-nfsd_file_close_inode_sync(struct inode *inode)
+static void
+nfsd_file_close_inode(struct inode *inode)
{
LIST_HEAD(dispose);
- unsigned int count;
- count = __nfsd_file_close_inode(inode, &dispose);
- trace_nfsd_file_close_inode_sync(inode, count);
- nfsd_file_dispose_list_sync(&dispose);
+ nfsd_file_queue_for_close(inode, &dispose);
+ nfsd_file_dispose_list_delayed(&dispose);
}
/**
- * nfsd_file_close_inode - attempt a delayed close of a nfsd_file
+ * nfsd_file_close_inode_sync - attempt to forcibly close a nfsd_file
* @inode: inode of the file to attempt to remove
*
- * Unhash and put all cache item associated with @inode.
+ * Close out any open nfsd_files that can be reaped for @inode. The
+ * nfsd_files are closed out synchronously.
+ *
+ * This is called from nfsd_rename and nfsd_unlink to avoid silly-renames
+ * when reexporting NFS.
*/
-static void
-nfsd_file_close_inode(struct inode *inode)
+void
+nfsd_file_close_inode_sync(struct inode *inode)
{
+ struct nfsd_file *nf;
LIST_HEAD(dispose);
- unsigned int count;
- count = __nfsd_file_close_inode(inode, &dispose);
- trace_nfsd_file_close_inode(inode, count);
- nfsd_file_dispose_list_delayed(&dispose);
+ trace_nfsd_file_close(inode);
+
+ nfsd_file_queue_for_close(inode, &dispose);
+ while (!list_empty(&dispose)) {
+ nf = list_first_entry(&dispose, struct nfsd_file, nf_lru);
+ list_del_init(&nf->nf_lru);
+ nfsd_file_free(nf);
+ }
+ flush_delayed_fput();
}
/**
* nfsd_file_delayed_close - close unused nfsd_files
* @work: dummy
*
- * Walk the LRU list and close any entries that have not been used since
+ * Walk the LRU list and destroy any entries that have not been used since
* the last scan.
*/
static void
/* Only close files for F_SETLEASE leases */
if (fl->fl_flags & FL_LEASE)
- nfsd_file_close_inode_sync(file_inode(fl->fl_file));
+ nfsd_file_close_inode(file_inode(fl->fl_file));
return 0;
}
goto out;
}
+/**
+ * __nfsd_file_cache_purge: clean out the cache for shutdown
+ * @net: net-namespace to shut down the cache (may be NULL)
+ *
+ * Walk the nfsd_file cache and close out any that match @net. If @net is NULL,
+ * then close out everything. Called when an nfsd instance is being shut down.
+ */
static void
__nfsd_file_cache_purge(struct net *net)
{
nf = rhashtable_walk_next(&iter);
while (!IS_ERR_OR_NULL(nf)) {
- if (!net || nf->nf_net == net)
- nfsd_file_unhash_and_queue(nf, &dispose);
+ if (!net || nf->nf_net == net) {
+ nfsd_file_unhash(nf);
+ nfsd_file_lru_remove(nf);
+ list_add(&nf->nf_lru, &dispose);
+ }
nf = rhashtable_walk_next(&iter);
}
if (nf)
nf = nfsd_file_get(nf);
rcu_read_unlock();
- if (nf)
+
+ if (nf) {
+ if (nfsd_file_lru_remove(nf))
+ WARN_ON_ONCE(refcount_dec_and_test(&nf->nf_ref));
goto wait_for_construction;
+ }
nf = nfsd_file_alloc(&key, may_flags);
if (!nf) {
goto out;
}
open_retry = false;
- nfsd_file_put_noref(nf);
+ if (refcount_dec_and_test(&nf->nf_ref))
+ nfsd_file_free(nf);
goto retry;
}
- nfsd_file_lru_remove(nf);
this_cpu_inc(nfsd_file_cache_hits);
status = nfserrno(nfsd_open_break_lease(file_inode(nf->nf_file), may_flags));
this_cpu_inc(nfsd_file_acquisitions);
*pnf = nf;
} else {
- nfsd_file_put(nf);
+ if (refcount_dec_and_test(&nf->nf_ref))
+ nfsd_file_free(nf);
nf = NULL;
}
* If construction failed, or we raced with a call to unlink()
* then unhash.
*/
- if (status != nfs_ok || key.inode->i_nlink == 0)
- nfsd_file_unhash_and_put(nf);
+ if (status == nfs_ok && key.inode->i_nlink == 0)
+ status = nfserr_jukebox;
+ if (status != nfs_ok)
+ nfsd_file_unhash(nf);
clear_bit_unlock(NFSD_FILE_PENDING, &nf->nf_flags);
smp_mb__after_atomic();
wake_up_bit(&nf->nf_flags, NFSD_FILE_PENDING);
} else {
if (!conn->cb_xprt)
return -EINVAL;
- clp->cl_cb_conn.cb_xprt = conn->cb_xprt;
clp->cl_cb_session = ses;
args.bc_xprt = conn->cb_xprt;
args.prognumber = clp->cl_cb_session->se_cb_prog;
rpc_shutdown_client(client);
return -ENOMEM;
}
+
+ if (clp->cl_minorversion != 0)
+ clp->cl_cb_conn.cb_xprt = conn->cb_xprt;
clp->cl_cb_client = client;
clp->cl_cb_cred = cred;
rcu_read_lock();
return status;
}
-static void
-nfsd4_interssc_disconnect(struct vfsmount *ss_mnt)
-{
- nfs_do_sb_deactive(ss_mnt->mnt_sb);
- mntput(ss_mnt);
-}
-
/*
* Verify COPY destination stateid.
*
{
}
-static void
-nfsd4_interssc_disconnect(struct vfsmount *ss_mnt)
-{
-}
-
static struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
struct nfs_fh *src_fh,
nfs4_stateid *stateid)
default:
nfserr = nfserr_offload_denied;
}
- nfsd4_interssc_disconnect(copy->ss_mnt);
+ /* ss_mnt will be unmounted by the laundromat */
goto do_callback;
}
nfserr = nfsd4_do_copy(copy, filp, copy->nf_dst->nf_file,
if (async_copy)
cleanup_async_copy(async_copy);
status = nfserrno(-ENOMEM);
- if (nfsd4_ssc_is_inter(copy))
- nfsd4_interssc_disconnect(copy->ss_mnt);
+ /*
+ * source's vfsmount of inter-copy will be unmounted
+ * by the laundromat
+ */
goto out;
}
__print_flags(val, "|", \
{ 1 << NFSD_FILE_HASHED, "HASHED" }, \
{ 1 << NFSD_FILE_PENDING, "PENDING" }, \
- { 1 << NFSD_FILE_REFERENCED, "REFERENCED"}, \
- { 1 << NFSD_FILE_GC, "GC"})
+ { 1 << NFSD_FILE_REFERENCED, "REFERENCED" }, \
+ { 1 << NFSD_FILE_GC, "GC" })
DECLARE_EVENT_CLASS(nfsd_file_class,
TP_PROTO(struct nfsd_file *nf),
DEFINE_NFSD_FILE_EVENT(nfsd_file_free);
DEFINE_NFSD_FILE_EVENT(nfsd_file_unhash);
DEFINE_NFSD_FILE_EVENT(nfsd_file_put);
+DEFINE_NFSD_FILE_EVENT(nfsd_file_closing);
DEFINE_NFSD_FILE_EVENT(nfsd_file_unhash_and_queue);
TRACE_EVENT(nfsd_file_alloc,
__entry->nf_file)
)
-DECLARE_EVENT_CLASS(nfsd_file_search_class,
- TP_PROTO(
- const struct inode *inode,
- unsigned int count
- ),
- TP_ARGS(inode, count),
- TP_STRUCT__entry(
- __field(const struct inode *, inode)
- __field(unsigned int, count)
- ),
- TP_fast_assign(
- __entry->inode = inode;
- __entry->count = count;
- ),
- TP_printk("inode=%p count=%u",
- __entry->inode, __entry->count)
-);
-
-#define DEFINE_NFSD_FILE_SEARCH_EVENT(name) \
-DEFINE_EVENT(nfsd_file_search_class, name, \
- TP_PROTO( \
- const struct inode *inode, \
- unsigned int count \
- ), \
- TP_ARGS(inode, count))
-
-DEFINE_NFSD_FILE_SEARCH_EVENT(nfsd_file_close_inode_sync);
-DEFINE_NFSD_FILE_SEARCH_EVENT(nfsd_file_close_inode);
-
TRACE_EVENT(nfsd_file_is_cached,
TP_PROTO(
const struct inode *inode,
DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_gc_in_use);
DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_gc_writeback);
DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_gc_referenced);
-DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_gc_hashed);
DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_gc_disposed);
DECLARE_EVENT_CLASS(nfsd_file_lruwalk_class,
DEFINE_NFSD_FILE_LRUWALK_EVENT(nfsd_file_gc_removed);
DEFINE_NFSD_FILE_LRUWALK_EVENT(nfsd_file_shrinker_removed);
+TRACE_EVENT(nfsd_file_close,
+ TP_PROTO(
+ const struct inode *inode
+ ),
+ TP_ARGS(inode),
+ TP_STRUCT__entry(
+ __field(const void *, inode)
+ ),
+ TP_fast_assign(
+ __entry->inode = inode;
+ ),
+ TP_printk("inode=%p",
+ __entry->inode
+ )
+);
+
TRACE_EVENT(nfsd_file_fsync,
TP_PROTO(
const struct nfsd_file *nf,
}
/*
- * attr_must_be_resident
- *
- * Return: True if attribute must be resident.
- */
-static inline bool attr_must_be_resident(struct ntfs_sb_info *sbi,
- enum ATTR_TYPE type)
-{
- const struct ATTR_DEF_ENTRY *de;
-
- switch (type) {
- case ATTR_STD:
- case ATTR_NAME:
- case ATTR_ID:
- case ATTR_LABEL:
- case ATTR_VOL_INFO:
- case ATTR_ROOT:
- case ATTR_EA_INFO:
- return true;
- default:
- de = ntfs_query_def(sbi, type);
- if (de && (de->flags & NTFS_ATTR_MUST_BE_RESIDENT))
- return true;
- return false;
- }
-}
-
-/*
* attr_load_runs - Load all runs stored in @attr.
*/
static int attr_load_runs(struct ATTRIB *attr, struct ntfs_inode *ni,
asize = le32_to_cpu(attr->size);
run_off = le16_to_cpu(attr->nres.run_off);
+
+ if (run_off > asize)
+ return -EINVAL;
+
err = run_unpack_ex(run, ni->mi.sbi, ni->mi.rno, svcn, evcn,
vcn ? *vcn : svcn, Add2Ptr(attr, run_off),
asize - run_off);
int attr_allocate_clusters(struct ntfs_sb_info *sbi, struct runs_tree *run,
CLST vcn, CLST lcn, CLST len, CLST *pre_alloc,
enum ALLOCATE_OPT opt, CLST *alen, const size_t fr,
- CLST *new_lcn)
+ CLST *new_lcn, CLST *new_len)
{
int err;
CLST flen, vcn0 = vcn, pre = pre_alloc ? *pre_alloc : 0;
if (err)
goto out;
- if (new_lcn && vcn == vcn0)
- *new_lcn = lcn;
+ if (vcn == vcn0) {
+ /* Return the first fragment. */
+ if (new_lcn)
+ *new_lcn = lcn;
+ if (new_len)
+ *new_len = flen;
+ }
/* Add new fragment into run storage. */
- if (!run_add_entry(run, vcn, lcn, flen, opt == ALLOCATE_MFT)) {
+ if (!run_add_entry(run, vcn, lcn, flen, opt & ALLOCATE_MFT)) {
/* Undo last 'ntfs_look_for_free_space' */
mark_as_free_ex(sbi, lcn, len, false);
err = -ENOMEM;
goto out;
}
+ if (opt & ALLOCATE_ZERO) {
+ u8 shift = sbi->cluster_bits - SECTOR_SHIFT;
+
+ err = blkdev_issue_zeroout(sbi->sb->s_bdev,
+ (sector_t)lcn << shift,
+ (sector_t)flen << shift,
+ GFP_NOFS, 0);
+ if (err)
+ goto out;
+ }
+
vcn += flen;
- if (flen >= len || opt == ALLOCATE_MFT ||
+ if (flen >= len || (opt & ALLOCATE_MFT) ||
(fr && run->count - cnt >= fr)) {
*alen = vcn - vcn0;
return 0;
const char *data = resident_data(attr);
err = attr_allocate_clusters(sbi, run, 0, 0, len, NULL,
- ALLOCATE_DEF, &alen, 0, NULL);
+ ALLOCATE_DEF, &alen, 0, NULL,
+ NULL);
if (err)
goto out1;
CLST alen, vcn, lcn, new_alen, old_alen, svcn, evcn;
CLST next_svcn, pre_alloc = -1, done = 0;
bool is_ext, is_bad = false;
+ bool dirty = false;
u32 align;
struct MFT_REC *rec;
return err;
/* Return if file is still resident. */
- if (!attr_b->non_res)
+ if (!attr_b->non_res) {
+ dirty = true;
goto ok1;
+ }
/* Layout of records may be changed, so do a full search. */
goto again;
if (keep_prealloc && new_size < old_size) {
attr_b->nres.data_size = cpu_to_le64(new_size);
- mi_b->dirty = true;
+ mi_b->dirty = dirty = true;
goto ok;
}
if (new_alloc <= old_alloc) {
attr_b->nres.data_size = cpu_to_le64(new_size);
- mi_b->dirty = true;
+ mi_b->dirty = dirty = true;
goto ok;
}
/* ~3 bytes per fragment. */
err = attr_allocate_clusters(
sbi, run, vcn, lcn, to_allocate, &pre_alloc,
- is_mft ? ALLOCATE_MFT : 0, &alen,
+ is_mft ? ALLOCATE_MFT : ALLOCATE_DEF, &alen,
is_mft ? 0
: (sbi->record_size -
le32_to_cpu(rec->used) + 8) /
3 +
1,
- NULL);
+ NULL, NULL);
if (err)
goto out;
}
next_svcn = le64_to_cpu(attr->nres.evcn) + 1;
new_alloc_tmp = (u64)next_svcn << cluster_bits;
attr_b->nres.alloc_size = cpu_to_le64(new_alloc_tmp);
- mi_b->dirty = true;
+ mi_b->dirty = dirty = true;
if (next_svcn >= vcn && !to_allocate) {
/* Normal way. Update attribute and exit. */
old_valid = old_size = old_alloc = (u64)vcn << cluster_bits;
attr_b->nres.valid_size = attr_b->nres.data_size =
attr_b->nres.alloc_size = cpu_to_le64(old_size);
- mi_b->dirty = true;
+ mi_b->dirty = dirty = true;
goto again_1;
}
attr_b->nres.valid_size =
attr_b->nres.alloc_size;
}
- mi_b->dirty = true;
+ mi_b->dirty = dirty = true;
err = run_deallocate_ex(sbi, run, vcn, evcn - vcn + 1, &dlen,
true);
if (ret)
*ret = attr_b;
- /* Update inode_set_bytes. */
if (((type == ATTR_DATA && !name_len) ||
(type == ATTR_ALLOC && name == I30_NAME))) {
- bool dirty = false;
-
- if (ni->vfs_inode.i_size != new_size) {
- ni->vfs_inode.i_size = new_size;
- dirty = true;
- }
-
+ /* Update inode_set_bytes. */
if (attr_b->non_res) {
new_alloc = le64_to_cpu(attr_b->nres.alloc_size);
if (inode_get_bytes(&ni->vfs_inode) != new_alloc) {
}
}
+ /* Don't forget to update duplicate information in parent. */
if (dirty) {
ni->ni_flags |= NI_FLAG_UPDATE_PARENT;
mark_inode_dirty(&ni->vfs_inode);
return err;
}
+/*
+ * attr_data_get_block - Returns 'lcn' and 'len' for given 'vcn'.
+ *
+ * @new == NULL means just to get current mapping for 'vcn'
+ * @new != NULL means allocate real cluster if 'vcn' maps to hole
+ * @zero - zeroout new allocated clusters
+ *
+ * NOTE:
+ * - @new != NULL is called only for sparsed or compressed attributes.
+ * - new allocated clusters are zeroed via blkdev_issue_zeroout.
+ */
int attr_data_get_block(struct ntfs_inode *ni, CLST vcn, CLST clen, CLST *lcn,
- CLST *len, bool *new)
+ CLST *len, bool *new, bool zero)
{
int err = 0;
struct runs_tree *run = &ni->file.run;
struct ATTRIB *attr = NULL, *attr_b;
struct ATTR_LIST_ENTRY *le, *le_b;
struct mft_inode *mi, *mi_b;
- CLST hint, svcn, to_alloc, evcn1, next_svcn, asize, end;
- u64 total_size;
- u32 clst_per_frame;
- bool ok;
+ CLST hint, svcn, to_alloc, evcn1, next_svcn, asize, end, vcn0, alen;
+ CLST alloc, evcn;
+ unsigned fr;
+ u64 total_size, total_size0;
+ int step = 0;
if (new)
*new = false;
+ /* Try to find in cache. */
down_read(&ni->file.run_lock);
- ok = run_lookup_entry(run, vcn, lcn, len, NULL);
+ if (!run_lookup_entry(run, vcn, lcn, len, NULL))
+ *len = 0;
up_read(&ni->file.run_lock);
- if (ok && (*lcn != SPARSE_LCN || !new)) {
- /* Normal way. */
- return 0;
+ if (*len) {
+ if (*lcn != SPARSE_LCN || !new)
+ return 0; /* Fast normal way without allocation. */
+ else if (clen > *len)
+ clen = *len;
}
- if (!clen)
- clen = 1;
-
- if (ok && clen > *len)
- clen = *len;
-
+ /* No cluster in cache or we need to allocate cluster in hole. */
sbi = ni->mi.sbi;
cluster_bits = sbi->cluster_bits;
asize = le64_to_cpu(attr_b->nres.alloc_size) >> cluster_bits;
if (vcn >= asize) {
- err = -EINVAL;
+ if (new) {
+ err = -EINVAL;
+ } else {
+ *len = 1;
+ *lcn = SPARSE_LCN;
+ }
goto out;
}
- clst_per_frame = 1u << attr_b->nres.c_unit;
- to_alloc = (clen + clst_per_frame - 1) & ~(clst_per_frame - 1);
-
- if (vcn + to_alloc > asize)
- to_alloc = asize - vcn;
-
svcn = le64_to_cpu(attr_b->nres.svcn);
evcn1 = le64_to_cpu(attr_b->nres.evcn) + 1;
evcn1 = le64_to_cpu(attr->nres.evcn) + 1;
}
+ /* Load in cache actual information. */
err = attr_load_runs(attr, ni, run, NULL);
if (err)
goto out;
- if (!ok) {
- ok = run_lookup_entry(run, vcn, lcn, len, NULL);
- if (ok && (*lcn != SPARSE_LCN || !new)) {
- /* Normal way. */
- err = 0;
- goto ok;
- }
+ if (!*len) {
+ if (run_lookup_entry(run, vcn, lcn, len, NULL)) {
+ if (*lcn != SPARSE_LCN || !new)
+ goto ok; /* Slow normal way without allocation. */
- if (!ok && !new) {
- *len = 0;
- err = 0;
+ if (clen > *len)
+ clen = *len;
+ } else if (!new) {
+ /* Here we may return -ENOENT.
+ * In any case caller gets zero length. */
goto ok;
}
-
- if (ok && clen > *len) {
- clen = *len;
- to_alloc = (clen + clst_per_frame - 1) &
- ~(clst_per_frame - 1);
- }
}
if (!is_attr_ext(attr_b)) {
+ /* The code below only for sparsed or compressed attributes. */
err = -EINVAL;
goto out;
}
+ vcn0 = vcn;
+ to_alloc = clen;
+ fr = (sbi->record_size - le32_to_cpu(mi->mrec->used) + 8) / 3 + 1;
+ /* Allocate frame aligned clusters.
+ * ntfs.sys usually uses 16 clusters per frame for sparsed or compressed.
+ * ntfs3 uses 1 cluster per frame for new created sparsed files. */
+ if (attr_b->nres.c_unit) {
+ CLST clst_per_frame = 1u << attr_b->nres.c_unit;
+ CLST cmask = ~(clst_per_frame - 1);
+
+ /* Get frame aligned vcn and to_alloc. */
+ vcn = vcn0 & cmask;
+ to_alloc = ((vcn0 + clen + clst_per_frame - 1) & cmask) - vcn;
+ if (fr < clst_per_frame)
+ fr = clst_per_frame;
+ zero = true;
+
+ /* Check if 'vcn' and 'vcn0' in different attribute segments. */
+ if (vcn < svcn || evcn1 <= vcn) {
+ /* Load attribute for truncated vcn. */
+ attr = ni_find_attr(ni, attr_b, &le, ATTR_DATA, NULL, 0,
+ &vcn, &mi);
+ if (!attr) {
+ err = -EINVAL;
+ goto out;
+ }
+ svcn = le64_to_cpu(attr->nres.svcn);
+ evcn1 = le64_to_cpu(attr->nres.evcn) + 1;
+ err = attr_load_runs(attr, ni, run, NULL);
+ if (err)
+ goto out;
+ }
+ }
+
+ if (vcn + to_alloc > asize)
+ to_alloc = asize - vcn;
+
/* Get the last LCN to allocate from. */
hint = 0;
hint = -1;
}
- err = attr_allocate_clusters(
- sbi, run, vcn, hint + 1, to_alloc, NULL, 0, len,
- (sbi->record_size - le32_to_cpu(mi->mrec->used) + 8) / 3 + 1,
- lcn);
+ /* Allocate and zeroout new clusters. */
+ err = attr_allocate_clusters(sbi, run, vcn, hint + 1, to_alloc, NULL,
+ zero ? ALLOCATE_ZERO : ALLOCATE_DEF, &alen,
+ fr, lcn, len);
if (err)
goto out;
*new = true;
+ step = 1;
- end = vcn + *len;
+ end = vcn + alen;
+ /* Save 'total_size0' to restore if error. */
+ total_size0 = le64_to_cpu(attr_b->nres.total_size);
+ total_size = total_size0 + ((u64)alen << cluster_bits);
- total_size = le64_to_cpu(attr_b->nres.total_size) +
- ((u64)*len << cluster_bits);
+ if (vcn != vcn0) {
+ if (!run_lookup_entry(run, vcn0, lcn, len, NULL)) {
+ err = -EINVAL;
+ goto out;
+ }
+ if (*lcn == SPARSE_LCN) {
+ /* Internal error. Should not happened. */
+ WARN_ON(1);
+ err = -EINVAL;
+ goto out;
+ }
+ /* Check case when vcn0 + len overlaps new allocated clusters. */
+ if (vcn0 + *len > end)
+ *len = end - vcn0;
+ }
repack:
err = mi_pack_runs(mi, attr, run, max(end, evcn1) - svcn);
if (!ni->attr_list.size) {
err = ni_create_attr_list(ni);
if (err)
- goto out;
+ goto undo1;
/* Layout of records is changed. */
le_b = NULL;
attr_b = ni_find_attr(ni, NULL, &le_b, ATTR_DATA, NULL,
}
}
+ /*
+ * The code below may require additional cluster (to extend attribute list)
+ * and / or one MFT record
+ * It is too complex to undo operations if -ENOSPC occurs deep inside
+ * in 'ni_insert_nonresident'.
+ * Return in advance -ENOSPC here if there are no free cluster and no free MFT.
+ */
+ if (!ntfs_check_for_free_space(sbi, 1, 1)) {
+ /* Undo step 1. */
+ err = -ENOSPC;
+ goto undo1;
+ }
+
+ step = 2;
svcn = evcn1;
/* Estimate next attribute. */
attr = ni_find_attr(ni, attr, &le, ATTR_DATA, NULL, 0, &svcn, &mi);
- if (attr) {
- CLST alloc = bytes_to_cluster(
- sbi, le64_to_cpu(attr_b->nres.alloc_size));
- CLST evcn = le64_to_cpu(attr->nres.evcn);
-
- if (end < next_svcn)
- end = next_svcn;
- while (end > evcn) {
- /* Remove segment [svcn : evcn). */
- mi_remove_attr(NULL, mi, attr);
-
- if (!al_remove_le(ni, le)) {
- err = -EINVAL;
- goto out;
- }
+ if (!attr) {
+ /* Insert new attribute segment. */
+ goto ins_ext;
+ }
- if (evcn + 1 >= alloc) {
- /* Last attribute segment. */
- evcn1 = evcn + 1;
- goto ins_ext;
- }
+ /* Try to update existed attribute segment. */
+ alloc = bytes_to_cluster(sbi, le64_to_cpu(attr_b->nres.alloc_size));
+ evcn = le64_to_cpu(attr->nres.evcn);
- if (ni_load_mi(ni, le, &mi)) {
- attr = NULL;
- goto out;
- }
+ if (end < next_svcn)
+ end = next_svcn;
+ while (end > evcn) {
+ /* Remove segment [svcn : evcn). */
+ mi_remove_attr(NULL, mi, attr);
- attr = mi_find_attr(mi, NULL, ATTR_DATA, NULL, 0,
- &le->id);
- if (!attr) {
- err = -EINVAL;
- goto out;
- }
- svcn = le64_to_cpu(attr->nres.svcn);
- evcn = le64_to_cpu(attr->nres.evcn);
+ if (!al_remove_le(ni, le)) {
+ err = -EINVAL;
+ goto out;
}
- if (end < svcn)
- end = svcn;
+ if (evcn + 1 >= alloc) {
+ /* Last attribute segment. */
+ evcn1 = evcn + 1;
+ goto ins_ext;
+ }
- err = attr_load_runs(attr, ni, run, &end);
- if (err)
+ if (ni_load_mi(ni, le, &mi)) {
+ attr = NULL;
goto out;
+ }
- evcn1 = evcn + 1;
- attr->nres.svcn = cpu_to_le64(next_svcn);
- err = mi_pack_runs(mi, attr, run, evcn1 - next_svcn);
- if (err)
+ attr = mi_find_attr(mi, NULL, ATTR_DATA, NULL, 0, &le->id);
+ if (!attr) {
+ err = -EINVAL;
goto out;
+ }
+ svcn = le64_to_cpu(attr->nres.svcn);
+ evcn = le64_to_cpu(attr->nres.evcn);
+ }
- le->vcn = cpu_to_le64(next_svcn);
- ni->attr_list.dirty = true;
- mi->dirty = true;
+ if (end < svcn)
+ end = svcn;
+
+ err = attr_load_runs(attr, ni, run, &end);
+ if (err)
+ goto out;
+
+ evcn1 = evcn + 1;
+ attr->nres.svcn = cpu_to_le64(next_svcn);
+ err = mi_pack_runs(mi, attr, run, evcn1 - next_svcn);
+ if (err)
+ goto out;
+
+ le->vcn = cpu_to_le64(next_svcn);
+ ni->attr_list.dirty = true;
+ mi->dirty = true;
+ next_svcn = le64_to_cpu(attr->nres.evcn) + 1;
- next_svcn = le64_to_cpu(attr->nres.evcn) + 1;
- }
ins_ext:
if (evcn1 > next_svcn) {
err = ni_insert_nonresident(ni, ATTR_DATA, NULL, 0, run,
ok:
run_truncate_around(run, vcn);
out:
+ if (err && step > 1) {
+ /* Too complex to restore. */
+ _ntfs_bad_inode(&ni->vfs_inode);
+ }
up_write(&ni->file.run_lock);
ni_unlock(ni);
return err;
+
+undo1:
+ /* Undo step1. */
+ attr_b->nres.total_size = cpu_to_le64(total_size0);
+ inode_set_bytes(&ni->vfs_inode, total_size0);
+
+ if (run_deallocate_ex(sbi, run, vcn, alen, NULL, false) ||
+ !run_add_entry(run, vcn, SPARSE_LCN, alen, false) ||
+ mi_pack_runs(mi, attr, run, max(end, evcn1) - svcn)) {
+ _ntfs_bad_inode(&ni->vfs_inode);
+ }
+ goto out;
}
int attr_data_read_resident(struct ntfs_inode *ni, struct page *page)
CLST svcn, evcn;
u16 ro;
+ if (!ni) {
+ /* Is record corrupted? */
+ return -ENOENT;
+ }
+
attr = ni_find_attr(ni, NULL, NULL, type, name, name_len, &vcn, NULL);
if (!attr) {
/* Is record corrupted? */
}
ro = le16_to_cpu(attr->nres.run_off);
+
+ if (ro > le32_to_cpu(attr->size))
+ return -EINVAL;
+
err = run_unpack_ex(run, ni->mi.sbi, ni->mi.rno, svcn, evcn, svcn,
Add2Ptr(attr, ro), le32_to_cpu(attr->size) - ro);
if (err < 0)
struct ATTRIB *attr = NULL, *attr_b;
struct ATTR_LIST_ENTRY *le, *le_b;
struct mft_inode *mi, *mi_b;
- CLST svcn, evcn1, next_svcn, lcn, len;
+ CLST svcn, evcn1, next_svcn, len;
CLST vcn, end, clst_data;
u64 total_size, valid_size, data_size;
}
err = attr_allocate_clusters(sbi, run, vcn + clst_data,
- hint + 1, len - clst_data, NULL, 0,
- &alen, 0, &lcn);
+ hint + 1, len - clst_data, NULL,
+ ALLOCATE_DEF, &alen, 0, NULL,
+ NULL);
if (err)
goto out;
u16 le_sz;
u16 roff = le16_to_cpu(attr->nres.run_off);
+ if (roff > le32_to_cpu(attr->size)) {
+ err = -EINVAL;
+ goto out;
+ }
+
run_unpack_ex(RUN_DEALLOCATE, sbi, ni->mi.rno, svcn,
evcn1 - 1, svcn, Add2Ptr(attr, roff),
le32_to_cpu(attr->size) - roff);
return -ENOENT;
if (!attr_b->non_res) {
- u32 data_size = le32_to_cpu(attr->res.data_size);
+ u32 data_size = le32_to_cpu(attr_b->res.data_size);
u32 from, to;
if (vbo > data_size)
if (!attr_b->non_res) {
/* Still resident. */
- char *data = Add2Ptr(attr_b, attr_b->res.data_off);
+ char *data = Add2Ptr(attr_b,
+ le16_to_cpu(attr_b->res.data_off));
memmove(data + bytes, data, bytes);
memset(data, 0, bytes);
if (vbo <= ni->i_valid)
ni->i_valid += bytes;
- attr_b->nres.data_size = le64_to_cpu(data_size + bytes);
- attr_b->nres.alloc_size = le64_to_cpu(alloc_size + bytes);
+ attr_b->nres.data_size = cpu_to_le64(data_size + bytes);
+ attr_b->nres.alloc_size = cpu_to_le64(alloc_size + bytes);
/* ni->valid may be not equal valid_size (temporary). */
if (ni->i_valid > data_size + bytes)
run_init(&ni->attr_list.run);
+ if (run_off > le32_to_cpu(attr->size)) {
+ err = -EINVAL;
+ goto out;
+ }
+
err = run_unpack_ex(&ni->attr_list.run, ni->mi.sbi, ni->mi.rno,
0, le64_to_cpu(attr->nres.evcn), 0,
Add2Ptr(attr, run_off),
*
* Return: True if all bits [bit, bit+nbits) are zeros "0".
*/
-bool are_bits_clear(const ulong *lmap, size_t bit, size_t nbits)
+bool are_bits_clear(const void *lmap, size_t bit, size_t nbits)
{
size_t pos = bit & 7;
const u8 *map = (u8 *)lmap + (bit >> 3);
*
* Return: True if all bits [bit, bit+nbits) are ones "1".
*/
-bool are_bits_set(const ulong *lmap, size_t bit, size_t nbits)
+bool are_bits_set(const void *lmap, size_t bit, size_t nbits)
{
u8 mask;
size_t pos = bit & 7;
*
* Return: -1 if not found.
*/
-static size_t wnd_scan(const ulong *buf, size_t wbit, u32 wpos, u32 wend,
+static size_t wnd_scan(const void *buf, size_t wbit, u32 wpos, u32 wend,
size_t to_alloc, size_t *prev_tail, size_t *b_pos,
size_t *b_len)
{
while (wpos < wend) {
size_t free_len;
u32 free_bits, end;
- u32 used = find_next_zero_bit(buf, wend, wpos);
+ u32 used = find_next_zero_bit_le(buf, wend, wpos);
if (used >= wend) {
if (*b_len < *prev_tail) {
* Now we have a fragment [wpos, wend) staring with 0.
*/
end = wpos + to_alloc - *prev_tail;
- free_bits = find_next_bit(buf, min(end, wend), wpos);
+ free_bits = find_next_bit_le(buf, min(end, wend), wpos);
free_len = *prev_tail + free_bits - wpos;
u8 cluster_bits = sbi->cluster_bits;
u32 wbits = 8 * sb->s_blocksize;
u32 used, frb;
- const ulong *buf;
size_t wpos, wbit, iw, vbo;
struct buffer_head *bh = NULL;
CLST lcn, clen;
goto out;
}
- buf = (ulong *)bh->b_data;
-
- used = bitmap_weight(buf, wbits);
+ used = ntfs_bitmap_weight_le(bh->b_data, wbits);
if (used < wbits) {
frb = wbits - used;
wnd->free_bits[iw] = frb;
wbits = wnd->nbits - wbit;
do {
- used = find_next_zero_bit(buf, wbits, wpos);
+ used = find_next_zero_bit_le(bh->b_data, wbits, wpos);
if (used > wpos && prev_tail) {
wnd_add_free_ext(wnd, wbit + wpos - prev_tail,
break;
}
- frb = find_next_bit(buf, wbits, wpos);
+ frb = find_next_bit_le(bh->b_data, wbits, wpos);
if (frb >= wbits) {
/* Keep last free block. */
prev_tail += frb - wpos;
if (!wnd->bits_last)
wnd->bits_last = wbits;
- wnd->free_bits = kcalloc(wnd->nwnd, sizeof(u16), GFP_NOFS);
+ wnd->free_bits = kcalloc(wnd->nwnd, sizeof(u16), GFP_NOFS | __GFP_NOWARN);
if (!wnd->free_bits)
return -ENOMEM;
while (iw < wnd->nwnd && bits) {
u32 tail, op;
- ulong *buf;
if (iw + 1 == wnd->nwnd)
wbits = wnd->bits_last;
break;
}
- buf = (ulong *)bh->b_data;
-
lock_buffer(bh);
- __bitmap_clear(buf, wbit, op);
+ ntfs_bitmap_clear_le(bh->b_data, wbit, op);
wnd->free_bits[iw] += op;
while (iw < wnd->nwnd && bits) {
u32 tail, op;
- ulong *buf;
if (unlikely(iw + 1 == wnd->nwnd))
wbits = wnd->bits_last;
err = PTR_ERR(bh);
break;
}
- buf = (ulong *)bh->b_data;
lock_buffer(bh);
- __bitmap_set(buf, wbit, op);
+ ntfs_bitmap_set_le(bh->b_data, wbit, op);
wnd->free_bits[iw] -= op;
set_buffer_uptodate(bh);
}
/*
+ * wnd_set_used_safe - Mark the bits range from bit to bit + bits as used.
+ *
+ * Unlikely wnd_set_used/wnd_set_free this function is not full trusted.
+ * It scans every bit in bitmap and marks free bit as used.
+ * @done - how many bits were marked as used.
+ *
+ * NOTE: normally *done should be 0.
+ */
+int wnd_set_used_safe(struct wnd_bitmap *wnd, size_t bit, size_t bits,
+ size_t *done)
+{
+ size_t i, from = 0, len = 0;
+ int err = 0;
+
+ *done = 0;
+ for (i = 0; i < bits; i++) {
+ if (wnd_is_free(wnd, bit + i, 1)) {
+ if (!len)
+ from = bit + i;
+ len += 1;
+ } else if (len) {
+ err = wnd_set_used(wnd, from, len);
+ *done += len;
+ len = 0;
+ if (err)
+ break;
+ }
+ }
+
+ if (len) {
+ /* last fragment. */
+ err = wnd_set_used(wnd, from, len);
+ *done += len;
+ }
+ return err;
+}
+
+/*
* wnd_is_free_hlp
*
* Return: True if all clusters [bit, bit+bits) are free (bitmap only).
if (IS_ERR(bh))
return false;
- ret = are_bits_clear((ulong *)bh->b_data, wbit, op);
+ ret = are_bits_clear(bh->b_data, wbit, op);
put_bh(bh);
if (!ret)
if (IS_ERR(bh))
goto out;
- ret = are_bits_set((ulong *)bh->b_data, wbit, op);
+ ret = are_bits_set(bh->b_data, wbit, op);
put_bh(bh);
if (!ret)
goto out;
size_t fnd, max_alloc, b_len, b_pos;
size_t iw, prev_tail, nwnd, wbit, ebit, zbit, zend;
size_t to_alloc0 = to_alloc;
- const ulong *buf;
const struct e_node *e;
const struct rb_node *pr, *cr;
u8 log2_bits;
continue;
}
- buf = (ulong *)bh->b_data;
-
/* Scan range [wbit, zbit). */
if (wpos < wzbit) {
/* Scan range [wpos, zbit). */
- fnd = wnd_scan(buf, wbit, wpos, wzbit,
- to_alloc, &prev_tail,
- &b_pos, &b_len);
+ fnd = wnd_scan(bh->b_data, wbit, wpos,
+ wzbit, to_alloc,
+ &prev_tail, &b_pos,
+ &b_len);
if (fnd != MINUS_ONE_T) {
put_bh(bh);
goto found;
/* Scan range [zend, ebit). */
if (wzend < wbits) {
- fnd = wnd_scan(buf, wbit,
+ fnd = wnd_scan(bh->b_data, wbit,
max(wzend, wpos), wbits,
to_alloc, &prev_tail,
&b_pos, &b_len);
continue;
}
- buf = (ulong *)bh->b_data;
-
/* Scan range [wpos, eBits). */
- fnd = wnd_scan(buf, wbit, wpos, wbits, to_alloc, &prev_tail,
- &b_pos, &b_len);
+ fnd = wnd_scan(bh->b_data, wbit, wpos, wbits, to_alloc,
+ &prev_tail, &b_pos, &b_len);
put_bh(bh);
if (fnd != MINUS_ONE_T)
goto found;
new_last = wbits;
if (new_wnd != wnd->nwnd) {
- new_free = kmalloc(new_wnd * sizeof(u16), GFP_NOFS);
+ new_free = kmalloc_array(new_wnd, sizeof(u16), GFP_NOFS);
if (!new_free)
return -ENOMEM;
size_t frb;
u64 vbo, lbo, bytes;
struct buffer_head *bh;
- ulong *buf;
if (iw + 1 == new_wnd)
wbits = new_last;
return -EIO;
lock_buffer(bh);
- buf = (ulong *)bh->b_data;
- __bitmap_clear(buf, b0, blocksize * 8 - b0);
- frb = wbits - bitmap_weight(buf, wbits);
+ ntfs_bitmap_clear_le(bh->b_data, b0, blocksize * 8 - b0);
+ frb = wbits - ntfs_bitmap_weight_le(bh->b_data, wbits);
wnd->total_zeroes += frb - wnd->free_bits[iw];
wnd->free_bits[iw] = frb;
CLST lcn_from = bytes_to_cluster(sbi, range->start);
size_t iw = lcn_from >> (sb->s_blocksize_bits + 3);
u32 wbit = lcn_from & (wbits - 1);
- const ulong *buf;
CLST lcn_to;
if (!minlen)
down_read_nested(&wnd->rw_lock, BITMAP_MUTEX_CLUSTERS);
- for (; iw < wnd->nbits; iw++, wbit = 0) {
+ for (; iw < wnd->nwnd; iw++, wbit = 0) {
CLST lcn_wnd = iw * wbits;
struct buffer_head *bh;
break;
}
- buf = (ulong *)bh->b_data;
-
for (; wbit < wbits; wbit++) {
- if (!test_bit(wbit, buf)) {
+ if (!test_bit_le(wbit, bh->b_data)) {
if (!len)
lcn = lcn_wnd + wbit;
len += 1;
return err;
}
+
+#if BITS_PER_LONG == 64
+typedef __le64 bitmap_ulong;
+#define cpu_to_ul(x) cpu_to_le64(x)
+#define ul_to_cpu(x) le64_to_cpu(x)
+#else
+typedef __le32 bitmap_ulong;
+#define cpu_to_ul(x) cpu_to_le32(x)
+#define ul_to_cpu(x) le32_to_cpu(x)
+#endif
+
+void ntfs_bitmap_set_le(void *map, unsigned int start, int len)
+{
+ bitmap_ulong *p = (bitmap_ulong *)map + BIT_WORD(start);
+ const unsigned int size = start + len;
+ int bits_to_set = BITS_PER_LONG - (start % BITS_PER_LONG);
+ bitmap_ulong mask_to_set = cpu_to_ul(BITMAP_FIRST_WORD_MASK(start));
+
+ while (len - bits_to_set >= 0) {
+ *p |= mask_to_set;
+ len -= bits_to_set;
+ bits_to_set = BITS_PER_LONG;
+ mask_to_set = cpu_to_ul(~0UL);
+ p++;
+ }
+ if (len) {
+ mask_to_set &= cpu_to_ul(BITMAP_LAST_WORD_MASK(size));
+ *p |= mask_to_set;
+ }
+}
+
+void ntfs_bitmap_clear_le(void *map, unsigned int start, int len)
+{
+ bitmap_ulong *p = (bitmap_ulong *)map + BIT_WORD(start);
+ const unsigned int size = start + len;
+ int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG);
+ bitmap_ulong mask_to_clear = cpu_to_ul(BITMAP_FIRST_WORD_MASK(start));
+
+ while (len - bits_to_clear >= 0) {
+ *p &= ~mask_to_clear;
+ len -= bits_to_clear;
+ bits_to_clear = BITS_PER_LONG;
+ mask_to_clear = cpu_to_ul(~0UL);
+ p++;
+ }
+ if (len) {
+ mask_to_clear &= cpu_to_ul(BITMAP_LAST_WORD_MASK(size));
+ *p &= ~mask_to_clear;
+ }
+}
+
+unsigned int ntfs_bitmap_weight_le(const void *bitmap, int bits)
+{
+ const ulong *bmp = bitmap;
+ unsigned int k, lim = bits / BITS_PER_LONG;
+ unsigned int w = 0;
+
+ for (k = 0; k < lim; k++)
+ w += hweight_long(bmp[k]);
+
+ if (bits % BITS_PER_LONG) {
+ w += hweight_long(ul_to_cpu(((bitmap_ulong *)bitmap)[k]) &
+ BITMAP_LAST_WORD_MASK(bits));
+ }
+
+ return w;
+}
if (!nls) {
/* UTF-16 -> UTF-8 */
- ret = utf16s_to_utf8s(name, len, UTF16_LITTLE_ENDIAN, buf,
- buf_len);
+ ret = utf16s_to_utf8s((wchar_t *)name, len, UTF16_LITTLE_ENDIAN,
+ buf, buf_len);
buf[ret] = '\0';
return ret;
}
bits = sbi->cluster_bits;
vcn = pos >> bits;
- err = attr_data_get_block(ni, vcn, 0, &lcn, &clen,
- NULL);
+ err = attr_data_get_block(ni, vcn, 1, &lcn, &clen, NULL,
+ false);
if (err)
goto out;
if (lcn == SPARSE_LCN) {
- loff_t vbo = (loff_t)vcn << bits;
- loff_t to = vbo + ((loff_t)clen << bits);
-
- if (to <= new_valid) {
- ni->i_valid = to;
- pos = to;
- goto next;
- }
-
- if (vbo < pos) {
- pos = vbo;
- } else {
- to = (new_valid >> bits) << bits;
- if (pos < to) {
- ni->i_valid = to;
- pos = to;
- goto next;
- }
- }
+ pos = ((loff_t)clen + vcn) << bits;
+ ni->i_valid = pos;
+ goto next;
}
}
struct address_space *mapping = inode->i_mapping;
u32 blocksize = 1 << inode->i_blkbits;
pgoff_t idx = vbo >> PAGE_SHIFT;
- u32 z_start = vbo & (PAGE_SIZE - 1);
+ u32 from = vbo & (PAGE_SIZE - 1);
pgoff_t idx_end = (vbo_to + PAGE_SIZE - 1) >> PAGE_SHIFT;
loff_t page_off;
struct buffer_head *head, *bh;
- u32 bh_next, bh_off, z_end;
+ u32 bh_next, bh_off, to;
sector_t iblock;
struct page *page;
- for (; idx < idx_end; idx += 1, z_start = 0) {
+ for (; idx < idx_end; idx += 1, from = 0) {
page_off = (loff_t)idx << PAGE_SHIFT;
- z_end = (page_off + PAGE_SIZE) > vbo_to ? (vbo_to - page_off)
- : PAGE_SIZE;
+ to = (page_off + PAGE_SIZE) > vbo_to ? (vbo_to - page_off)
+ : PAGE_SIZE;
iblock = page_off >> inode->i_blkbits;
page = find_or_create_page(mapping, idx,
do {
bh_next = bh_off + blocksize;
- if (bh_next <= z_start || bh_off >= z_end)
+ if (bh_next <= from || bh_off >= to)
continue;
if (!buffer_mapped(bh)) {
} while (bh_off = bh_next, iblock += 1,
head != (bh = bh->b_this_page));
- zero_user_segment(page, z_start, z_end);
+ zero_user_segment(page, from, to);
unlock_page(page);
put_page(page);
}
/*
- * ntfs_sparse_cluster - Helper function to zero a new allocated clusters.
- *
- * NOTE: 512 <= cluster size <= 2M
- */
-void ntfs_sparse_cluster(struct inode *inode, struct page *page0, CLST vcn,
- CLST len)
-{
- struct address_space *mapping = inode->i_mapping;
- struct ntfs_sb_info *sbi = inode->i_sb->s_fs_info;
- u64 vbo = (u64)vcn << sbi->cluster_bits;
- u64 bytes = (u64)len << sbi->cluster_bits;
- u32 blocksize = 1 << inode->i_blkbits;
- pgoff_t idx0 = page0 ? page0->index : -1;
- loff_t vbo_clst = vbo & sbi->cluster_mask_inv;
- loff_t end = ntfs_up_cluster(sbi, vbo + bytes);
- pgoff_t idx = vbo_clst >> PAGE_SHIFT;
- u32 from = vbo_clst & (PAGE_SIZE - 1);
- pgoff_t idx_end = (end + PAGE_SIZE - 1) >> PAGE_SHIFT;
- loff_t page_off;
- u32 to;
- bool partial;
- struct page *page;
-
- for (; idx < idx_end; idx += 1, from = 0) {
- page = idx == idx0 ? page0 : grab_cache_page(mapping, idx);
-
- if (!page)
- continue;
-
- page_off = (loff_t)idx << PAGE_SHIFT;
- to = (page_off + PAGE_SIZE) > end ? (end - page_off)
- : PAGE_SIZE;
- partial = false;
-
- if ((from || PAGE_SIZE != to) &&
- likely(!page_has_buffers(page))) {
- create_empty_buffers(page, blocksize, 0);
- }
-
- if (page_has_buffers(page)) {
- struct buffer_head *head, *bh;
- u32 bh_off = 0;
-
- bh = head = page_buffers(page);
- do {
- u32 bh_next = bh_off + blocksize;
-
- if (from <= bh_off && bh_next <= to) {
- set_buffer_uptodate(bh);
- mark_buffer_dirty(bh);
- } else if (!buffer_uptodate(bh)) {
- partial = true;
- }
- bh_off = bh_next;
- } while (head != (bh = bh->b_this_page));
- }
-
- zero_user_segment(page, from, to);
-
- if (!partial) {
- if (!PageUptodate(page))
- SetPageUptodate(page);
- set_page_dirty(page);
- }
-
- if (idx != idx0) {
- unlock_page(page);
- put_page(page);
- }
- cond_resched();
- }
- mark_inode_dirty(inode);
-}
-
-/*
* ntfs_file_mmap - file_operations::mmap
*/
static int ntfs_file_mmap(struct file *file, struct vm_area_struct *vma)
for (; vcn < end; vcn += len) {
err = attr_data_get_block(ni, vcn, 1, &lcn,
- &len, &new);
+ &len, &new, true);
if (err)
goto out;
-
- if (!new)
- continue;
- ntfs_sparse_cluster(inode, NULL, vcn, 1);
}
}
err = ntfs_set_size(inode, end);
if (err)
goto out;
- inode->i_size = end;
}
if (extend_init && !is_compressed(ni)) {
struct ntfs_sb_info *sbi = sb->s_fs_info;
struct ntfs_inode *ni = ntfs_i(inode);
loff_t end = vbo + len;
- loff_t vbo_down = round_down(vbo, PAGE_SIZE);
+ loff_t vbo_down = round_down(vbo, max_t(unsigned long,
+ sbi->cluster_size, PAGE_SIZE));
bool is_supported_holes = is_sparsed(ni) || is_compressed(ni);
loff_t i_size, new_size;
bool map_locked;
u32 frame_size;
loff_t mask, vbo_a, end_a, tmp;
- err = filemap_write_and_wait_range(mapping, vbo, end - 1);
- if (err)
- goto out;
-
- err = filemap_write_and_wait_range(mapping, end, LLONG_MAX);
+ err = filemap_write_and_wait_range(mapping, vbo_down,
+ LLONG_MAX);
if (err)
goto out;
if (err)
goto out;
- /*
- * Allocate clusters, do not change 'valid' size.
- */
- err = ntfs_set_size(inode, new_size);
- if (err)
- goto out;
+ if (new_size > i_size) {
+ /*
+ * Allocate clusters, do not change 'valid' size.
+ */
+ err = ntfs_set_size(inode, new_size);
+ if (err)
+ goto out;
+ }
if (is_supported_holes) {
- CLST vcn_v = ni->i_valid >> sbi->cluster_bits;
CLST vcn = vbo >> sbi->cluster_bits;
CLST cend = bytes_to_cluster(sbi, end);
+ CLST cend_v = bytes_to_cluster(sbi, ni->i_valid);
CLST lcn, clen;
bool new;
+ if (cend_v > cend)
+ cend_v = cend;
+
/*
- * Allocate but do not zero new clusters. (see below comments)
- * This breaks security: One can read unused on-disk areas.
+ * Allocate and zero new clusters.
* Zeroing these clusters may be too long.
- * Maybe we should check here for root rights?
+ */
+ for (; vcn < cend_v; vcn += clen) {
+ err = attr_data_get_block(ni, vcn, cend_v - vcn,
+ &lcn, &clen, &new,
+ true);
+ if (err)
+ goto out;
+ }
+ /*
+ * Allocate but not zero new clusters.
*/
for (; vcn < cend; vcn += clen) {
err = attr_data_get_block(ni, vcn, cend - vcn,
- &lcn, &clen, &new);
+ &lcn, &clen, &new,
+ false);
if (err)
goto out;
- if (!new || vcn >= vcn_v)
- continue;
-
- /*
- * Unwritten area.
- * NTFS is not able to store several unwritten areas.
- * Activate 'ntfs_sparse_cluster' to zero new allocated clusters.
- *
- * Dangerous in case:
- * 1G of sparsed clusters + 1 cluster of data =>
- * valid_size == 1G + 1 cluster
- * fallocate(1G) will zero 1G and this can be very long
- * xfstest 016/086 will fail without 'ntfs_sparse_cluster'.
- */
- ntfs_sparse_cluster(inode, NULL, vcn,
- min(vcn_v - vcn, clen));
}
}
&ni->file.run, i_size, &ni->i_valid,
true, NULL);
ni_unlock(ni);
+ } else if (new_size > i_size) {
+ inode->i_size = new_size;
}
}
goto out;
if (ia_valid & ATTR_SIZE) {
- loff_t oldsize = inode->i_size;
+ loff_t newsize, oldsize;
if (WARN_ON(ni->ni_flags & NI_FLAG_COMPRESSED_MASK)) {
/* Should never be here, see ntfs_file_open(). */
goto out;
}
inode_dio_wait(inode);
+ oldsize = inode->i_size;
+ newsize = attr->ia_size;
- if (attr->ia_size <= oldsize)
- err = ntfs_truncate(inode, attr->ia_size);
- else if (attr->ia_size > oldsize)
- err = ntfs_extend(inode, attr->ia_size, 0, NULL);
+ if (newsize <= oldsize)
+ err = ntfs_truncate(inode, newsize);
+ else
+ err = ntfs_extend(inode, newsize, 0, NULL);
if (err)
goto out;
ni->ni_flags |= NI_FLAG_UPDATE_PARENT;
+ inode->i_size = newsize;
}
setattr_copy(mnt_userns, inode, attr);
frame_vbo = valid & ~(frame_size - 1);
off = valid & (frame_size - 1);
- err = attr_data_get_block(ni, frame << NTFS_LZNT_CUNIT, 0, &lcn,
- &clen, NULL);
+ err = attr_data_get_block(ni, frame << NTFS_LZNT_CUNIT, 1, &lcn,
+ &clen, NULL, false);
if (err)
goto out;
}
if (!mi_p) {
- /* Do not try if not enogh free space. */
+ /* Do not try if not enough free space. */
if (le32_to_cpu(mi->mrec->used) + 8 >= rs)
continue;
}
roff = le16_to_cpu(attr->nres.run_off);
+
+ if (roff > le32_to_cpu(attr->size)) {
+ err = -EINVAL;
+ break;
+ }
+
err = run_unpack(&run, sbi, ni->mi.rno, svcn, evcn, svcn,
Add2Ptr(attr, roff),
le32_to_cpu(attr->size) - roff);
asize = le32_to_cpu(attr->size);
roff = le16_to_cpu(attr->nres.run_off);
+ if (roff > asize)
+ return -EINVAL;
+
/* run==1 means unpack and deallocate. */
run_unpack_ex(RUN_DEALLOCATE, sbi, ni->mi.rno, svcn, evcn, svcn,
Add2Ptr(attr, roff), asize - roff);
{
struct ATTRIB *attr = NULL;
struct ATTR_FILE_NAME *fname;
+ struct le_str *fns;
if (le)
*le = NULL;
if (uni->len != fname->name_len)
goto next;
- if (ntfs_cmp_names_cpu(uni, (struct le_str *)&fname->name_len, NULL,
- false))
+ fns = (struct le_str *)&fname->name_len;
+ if (ntfs_cmp_names_cpu(uni, fns, NULL, false))
goto next;
return fname;
for (vcn = vbo >> sbi->cluster_bits; vcn < end; vcn += clen) {
err = attr_data_get_block(ni, vcn, cend - vcn, &lcn,
- &clen, &new);
+ &clen, &new, false);
if (err)
goto out;
}
asize = le32_to_cpu(attr->size);
roff = le16_to_cpu(attr->nres.run_off);
+ if (roff > asize) {
+ err = -EINVAL;
+ goto out;
+ }
+
/*run==1 Means unpack and deallocate. */
run_unpack_ex(RUN_DEALLOCATE, sbi, ni->mi.rno, svcn, evcn, svcn,
Add2Ptr(attr, roff), asize - roff);
struct NTFS_DE *de)
{
int err;
+ struct ntfs_sb_info *sbi = ni->mi.sbi;
struct ATTRIB *attr;
struct ATTR_LIST_ENTRY *le;
struct mft_inode *mi;
struct ATTR_FILE_NAME *de_name = (struct ATTR_FILE_NAME *)(de + 1);
u16 de_key_size = le16_to_cpu(de->key_size);
+ if (sbi->options->windows_names &&
+ !valid_windows_name(sbi, (struct le_str *)&de_name->name_len))
+ return -EINVAL;
+
+ /* If option "hide_dot_files" then set hidden attribute for dot files. */
+ if (ni->mi.sbi->options->hide_dot_files) {
+ if (de_name->name_len > 0 &&
+ le16_to_cpu(de_name->name[0]) == '.')
+ ni->std_fa |= FILE_ATTRIBUTE_HIDDEN;
+ else
+ ni->std_fa &= ~FILE_ATTRIBUTE_HIDDEN;
+ }
+
mi_get_ref(&ni->mi, &de->ref);
mi_get_ref(&dir_ni->mi, &de_name->home);
memcpy(Add2Ptr(attr, SIZEOF_RESIDENT), de_name, de_key_size);
/* Insert new name into directory. */
- err = indx_insert_entry(&dir_ni->dir, dir_ni, de, ni->mi.sbi, NULL, 0);
+ err = indx_insert_entry(&dir_ni->dir, dir_ni, de, sbi, NULL, 0);
if (err)
ni_remove_attr_le(ni, attr, mi, le);
modified = true;
}
+ /* std attribute is always in primary MFT record. */
if (modified)
ni->mi.dirty = true;
return -EINVAL;
if (!*buffer) {
- to_free = kmalloc(bytes, GFP_NOFS);
+ to_free = kmalloc(log->page_size, GFP_NOFS);
if (!to_free)
return -ENOMEM;
*buffer = to_free;
struct restart_info *info)
{
u32 skip, vbo;
- struct RESTART_HDR *r_page = kmalloc(DefaultLogPageSize, GFP_NOFS);
-
- if (!r_page)
- return -ENOMEM;
+ struct RESTART_HDR *r_page = NULL;
/* Determine which restart area we are looking for. */
if (first) {
/* Loop continuously until we succeed. */
for (; vbo < l_size; vbo = 2 * vbo + skip, skip = 0) {
bool usa_error;
- u32 sys_page_size;
bool brst, bchk;
struct RESTART_AREA *ra;
goto check_result;
}
- /* Read the entire restart area. */
- sys_page_size = le32_to_cpu(r_page->sys_page_size);
- if (DefaultLogPageSize != sys_page_size) {
- kfree(r_page);
- r_page = kzalloc(sys_page_size, GFP_NOFS);
- if (!r_page)
- return -ENOMEM;
-
- if (read_log_page(log, vbo,
- (struct RECORD_PAGE_HDR **)&r_page,
- &usa_error)) {
- /* Ignore any errors. */
- kfree(r_page);
- r_page = NULL;
- continue;
- }
- }
-
if (is_client_area_valid(r_page, usa_error)) {
info->valid_page = true;
ra = Add2Ptr(r_page, le16_to_cpu(r_page->ra_off));
return false;
}
+ if (run_off > asize)
+ return false;
+
if (run_unpack(NULL, sbi, 0, svcn, evcn, svcn,
Add2Ptr(attr, run_off), asize - run_off) < 0) {
return false;
struct NEW_ATTRIBUTE_SIZES *new_sz;
struct ATTR_FILE_NAME *fname;
struct OpenAttr *oa, *oa2;
- u32 nsize, t32, asize, used, esize, bmp_off, bmp_bits;
+ u32 nsize, t32, asize, used, esize, off, bits;
u16 id, id2;
u32 record_size = sbi->record_size;
u64 t64;
break;
case SetBitsInNonresidentBitMap:
- bmp_off =
- le32_to_cpu(((struct BITMAP_RANGE *)data)->bitmap_off);
- bmp_bits = le32_to_cpu(((struct BITMAP_RANGE *)data)->bits);
+ off = le32_to_cpu(((struct BITMAP_RANGE *)data)->bitmap_off);
+ bits = le32_to_cpu(((struct BITMAP_RANGE *)data)->bits);
- if (cbo + (bmp_off + 7) / 8 > lco ||
- cbo + ((bmp_off + bmp_bits + 7) / 8) > lco) {
+ if (cbo + (off + 7) / 8 > lco ||
+ cbo + ((off + bits + 7) / 8) > lco) {
goto dirty_vol;
}
- __bitmap_set(Add2Ptr(buffer_le, roff), bmp_off, bmp_bits);
+ ntfs_bitmap_set_le(Add2Ptr(buffer_le, roff), off, bits);
a_dirty = true;
break;
case ClearBitsInNonresidentBitMap:
- bmp_off =
- le32_to_cpu(((struct BITMAP_RANGE *)data)->bitmap_off);
- bmp_bits = le32_to_cpu(((struct BITMAP_RANGE *)data)->bits);
+ off = le32_to_cpu(((struct BITMAP_RANGE *)data)->bitmap_off);
+ bits = le32_to_cpu(((struct BITMAP_RANGE *)data)->bits);
- if (cbo + (bmp_off + 7) / 8 > lco ||
- cbo + ((bmp_off + bmp_bits + 7) / 8) > lco) {
+ if (cbo + (off + 7) / 8 > lco ||
+ cbo + ((off + bits + 7) / 8) > lco) {
goto dirty_vol;
}
- __bitmap_clear(Add2Ptr(buffer_le, roff), bmp_off, bmp_bits);
+ ntfs_bitmap_clear_le(Add2Ptr(buffer_le, roff), off, bits);
a_dirty = true;
break;
u16 roff = le16_to_cpu(attr->nres.run_off);
CLST svcn = le64_to_cpu(attr->nres.svcn);
+ if (roff > t32) {
+ kfree(oa->attr);
+ oa->attr = NULL;
+ goto fake_attr;
+ }
+
err = run_unpack(&oa->run0, sbi, inode->i_ino, svcn,
le64_to_cpu(attr->nres.evcn), svcn,
Add2Ptr(attr, roff), t32 - roff);
goto out;
}
attr = oa->attr;
- t64 = le64_to_cpu(attr->nres.alloc_size);
- if (size > t64) {
+ if (size > le64_to_cpu(attr->nres.alloc_size)) {
attr->nres.valid_size = attr->nres.data_size =
attr->nres.alloc_size = cpu_to_le64(size);
}
};
#endif
+static const __le16 CON_NAME[3] = {
+ cpu_to_le16('C'), cpu_to_le16('O'), cpu_to_le16('N'),
+};
+
+static const __le16 NUL_NAME[3] = {
+ cpu_to_le16('N'), cpu_to_le16('U'), cpu_to_le16('L'),
+};
+
+static const __le16 AUX_NAME[3] = {
+ cpu_to_le16('A'), cpu_to_le16('U'), cpu_to_le16('X'),
+};
+
+static const __le16 PRN_NAME[3] = {
+ cpu_to_le16('P'), cpu_to_le16('R'), cpu_to_le16('N'),
+};
+
+static const __le16 COM_NAME[3] = {
+ cpu_to_le16('C'), cpu_to_le16('O'), cpu_to_le16('M'),
+};
+
+static const __le16 LPT_NAME[3] = {
+ cpu_to_le16('L'), cpu_to_le16('P'), cpu_to_le16('T'),
+};
+
// clang-format on
/*
}
/*
- * ntfs_query_def
- *
- * Return: Current ATTR_DEF_ENTRY for given attribute type.
- */
-const struct ATTR_DEF_ENTRY *ntfs_query_def(struct ntfs_sb_info *sbi,
- enum ATTR_TYPE type)
-{
- int type_in = le32_to_cpu(type);
- size_t min_idx = 0;
- size_t max_idx = sbi->def_entries - 1;
-
- while (min_idx <= max_idx) {
- size_t i = min_idx + ((max_idx - min_idx) >> 1);
- const struct ATTR_DEF_ENTRY *entry = sbi->def_table + i;
- int diff = le32_to_cpu(entry->type) - type_in;
-
- if (!diff)
- return entry;
- if (diff < 0)
- min_idx = i + 1;
- else if (i)
- max_idx = i - 1;
- else
- return NULL;
- }
- return NULL;
-}
-
-/*
* ntfs_look_for_free_space - Look for a free space in bitmap.
*/
int ntfs_look_for_free_space(struct ntfs_sb_info *sbi, CLST lcn, CLST len,
}
/*
+ * ntfs_check_for_free_space
+ *
+ * Check if it is possible to allocate 'clen' clusters and 'mlen' Mft records
+ */
+bool ntfs_check_for_free_space(struct ntfs_sb_info *sbi, CLST clen, CLST mlen)
+{
+ size_t free, zlen, avail;
+ struct wnd_bitmap *wnd;
+
+ wnd = &sbi->used.bitmap;
+ down_read_nested(&wnd->rw_lock, BITMAP_MUTEX_CLUSTERS);
+ free = wnd_zeroes(wnd);
+ zlen = min_t(size_t, NTFS_MIN_MFT_ZONE, wnd_zone_len(wnd));
+ up_read(&wnd->rw_lock);
+
+ if (free < zlen + clen)
+ return false;
+
+ avail = free - (zlen + clen);
+
+ wnd = &sbi->mft.bitmap;
+ down_read_nested(&wnd->rw_lock, BITMAP_MUTEX_MFT);
+ free = wnd_zeroes(wnd);
+ zlen = wnd_zone_len(wnd);
+ up_read(&wnd->rw_lock);
+
+ if (free >= zlen + mlen)
+ return true;
+
+ return avail >= bytes_to_cluster(sbi, mlen << sbi->record_bits);
+}
+
+/*
* ntfs_extend_mft - Allocate additional MFT records.
*
* sbi->mft.bitmap is locked for write.
struct ATTRIB *attr;
struct wnd_bitmap *wnd = &sbi->mft.bitmap;
- new_mft_total = (wnd->nbits + MFT_INCREASE_CHUNK + 127) & (CLST)~127;
+ new_mft_total = ALIGN(wnd->nbits + NTFS_MFT_INCREASE_STEP, 128);
new_mft_bytes = (u64)new_mft_total << sbi->record_bits;
/* Step 1: Resize $MFT::DATA. */
NULL, 0, NULL, NULL))
goto next;
- __clear_bit(ir - MFT_REC_RESERVED,
+ __clear_bit_le(ir - MFT_REC_RESERVED,
&sbi->mft.reserved_bitmap);
}
}
/* Scan 5 bits for zero. Bit 0 == MFT_REC_RESERVED */
- zbit = find_next_zero_bit(&sbi->mft.reserved_bitmap,
+ zbit = find_next_zero_bit_le(&sbi->mft.reserved_bitmap,
MFT_REC_FREE, MFT_REC_RESERVED);
if (zbit >= MFT_REC_FREE) {
sbi->mft.next_reserved = MFT_REC_FREE;
if (*rno >= MFT_REC_FREE)
wnd_set_used(wnd, *rno, 1);
else if (*rno >= MFT_REC_RESERVED && sbi->mft.reserved_bitmap_inited)
- __set_bit(*rno - MFT_REC_RESERVED, &sbi->mft.reserved_bitmap);
+ __set_bit_le(*rno - MFT_REC_RESERVED, &sbi->mft.reserved_bitmap);
out:
if (!mft)
else
wnd_set_free(wnd, rno, 1);
} else if (rno >= MFT_REC_RESERVED && sbi->mft.reserved_bitmap_inited) {
- __clear_bit(rno - MFT_REC_RESERVED, &sbi->mft.reserved_bitmap);
+ __clear_bit_le(rno - MFT_REC_RESERVED, &sbi->mft.reserved_bitmap);
}
if (rno < wnd_zone_bit(wnd))
if (!(sbi->flags & NTFS_FLAGS_MFTMIRR))
return;
- err = 0;
bytes = sbi->mft.recs_mirr << sbi->record_bits;
block1 = sbi->mft.lbo >> sb->s_blocksize_bits;
block2 = sbi->mft.lbo2 >> sb->s_blocksize_bits;
put_bh(bh1);
bh1 = NULL;
- if (wait)
- err = sync_dirty_buffer(bh2);
+ err = wait ? sync_dirty_buffer(bh2) : 0;
put_bh(bh2);
if (err)
goto out;
}
- root_sdh = resident_data(attr);
+ root_sdh = resident_data_ex(attr, sizeof(struct INDEX_ROOT));
if (root_sdh->type != ATTR_ZERO ||
- root_sdh->rule != NTFS_COLLATION_TYPE_SECURITY_HASH) {
+ root_sdh->rule != NTFS_COLLATION_TYPE_SECURITY_HASH ||
+ offsetof(struct INDEX_ROOT, ihdr) + root_sdh->ihdr.used > attr->res.data_size) {
err = -EINVAL;
goto out;
}
goto out;
}
- root_sii = resident_data(attr);
+ root_sii = resident_data_ex(attr, sizeof(struct INDEX_ROOT));
if (root_sii->type != ATTR_ZERO ||
- root_sii->rule != NTFS_COLLATION_TYPE_UINT) {
+ root_sii->rule != NTFS_COLLATION_TYPE_UINT ||
+ offsetof(struct INDEX_ROOT, ihdr) + root_sii->ihdr.used > attr->res.data_size) {
err = -EINVAL;
goto out;
}
return 0;
}
+
+static inline bool name_has_forbidden_chars(const struct le_str *fname)
+{
+ int i, ch;
+
+ /* check for forbidden chars */
+ for (i = 0; i < fname->len; ++i) {
+ ch = le16_to_cpu(fname->name[i]);
+
+ /* control chars */
+ if (ch < 0x20)
+ return true;
+
+ switch (ch) {
+ /* disallowed by Windows */
+ case '\\':
+ case '/':
+ case ':':
+ case '*':
+ case '?':
+ case '<':
+ case '>':
+ case '|':
+ case '\"':
+ return true;
+
+ default:
+ /* allowed char */
+ break;
+ }
+ }
+
+ /* file names cannot end with space or . */
+ if (fname->len > 0) {
+ ch = le16_to_cpu(fname->name[fname->len - 1]);
+ if (ch == ' ' || ch == '.')
+ return true;
+ }
+
+ return false;
+}
+
+static inline bool is_reserved_name(struct ntfs_sb_info *sbi,
+ const struct le_str *fname)
+{
+ int port_digit;
+ const __le16 *name = fname->name;
+ int len = fname->len;
+ u16 *upcase = sbi->upcase;
+
+ /* check for 3 chars reserved names (device names) */
+ /* name by itself or with any extension is forbidden */
+ if (len == 3 || (len > 3 && le16_to_cpu(name[3]) == '.'))
+ if (!ntfs_cmp_names(name, 3, CON_NAME, 3, upcase, false) ||
+ !ntfs_cmp_names(name, 3, NUL_NAME, 3, upcase, false) ||
+ !ntfs_cmp_names(name, 3, AUX_NAME, 3, upcase, false) ||
+ !ntfs_cmp_names(name, 3, PRN_NAME, 3, upcase, false))
+ return true;
+
+ /* check for 4 chars reserved names (port name followed by 1..9) */
+ /* name by itself or with any extension is forbidden */
+ if (len == 4 || (len > 4 && le16_to_cpu(name[4]) == '.')) {
+ port_digit = le16_to_cpu(name[3]);
+ if (port_digit >= '1' && port_digit <= '9')
+ if (!ntfs_cmp_names(name, 3, COM_NAME, 3, upcase, false) ||
+ !ntfs_cmp_names(name, 3, LPT_NAME, 3, upcase, false))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * valid_windows_name - Check if a file name is valid in Windows.
+ */
+bool valid_windows_name(struct ntfs_sb_info *sbi, const struct le_str *fname)
+{
+ return !name_has_forbidden_chars(fname) &&
+ !is_reserved_name(sbi, fname);
+}
if (l2 < fsize2)
return -1;
- both_case = f2->type != FILE_NAME_DOS /*&& !sbi->options.nocase*/;
+ both_case = f2->type != FILE_NAME_DOS && !sbi->options->nocase;
if (!l1) {
const struct le_str *s2 = (struct le_str *)&f2->name_len;
if (err)
return err;
- __set_bit(bit - bbuf.bit, bbuf.buf);
+ __set_bit_le(bit - bbuf.bit, bbuf.buf);
bmp_buf_put(&bbuf, true);
if (err)
return err;
- __clear_bit(bit - bbuf.bit, bbuf.buf);
+ __clear_bit_le(bit - bbuf.bit, bbuf.buf);
bmp_buf_put(&bbuf, true);
static bool scan_for_free(const ulong *buf, u32 bit, u32 bits, size_t *ret)
{
- size_t pos = find_next_zero_bit(buf, bits, bit);
+ size_t pos = find_next_zero_bit_le(buf, bits, bit);
if (pos >= bits)
return false;
if (!b->non_res) {
u32 nbits = 8 * le32_to_cpu(b->res.data_size);
- size_t pos = find_next_zero_bit(resident_data(b), nbits, 0);
+ size_t pos = find_next_zero_bit_le(resident_data(b), nbits, 0);
if (pos < nbits)
*bit = pos;
static bool scan_for_used(const ulong *buf, u32 bit, u32 bits, size_t *ret)
{
- size_t pos = find_next_bit(buf, bits, bit);
+ size_t pos = find_next_bit_le(buf, bits, bit);
if (pos >= bits)
return false;
if (!b->non_res) {
u32 nbits = le32_to_cpu(b->res.data_size) * 8;
- size_t pos = find_next_bit(resident_data(b), nbits, from);
+ size_t pos = find_next_bit_le(resident_data(b), nbits, from);
if (pos < nbits)
*bit = pos;
return e;
}
+/*
+ * index_hdr_check
+ *
+ * return true if INDEX_HDR is valid
+ */
+static bool index_hdr_check(const struct INDEX_HDR *hdr, u32 bytes)
+{
+ u32 end = le32_to_cpu(hdr->used);
+ u32 tot = le32_to_cpu(hdr->total);
+ u32 off = le32_to_cpu(hdr->de_off);
+
+ if (!IS_ALIGNED(off, 8) || tot > bytes || end > tot ||
+ off + sizeof(struct NTFS_DE) > end) {
+ /* incorrect index buffer. */
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * index_buf_check
+ *
+ * return true if INDEX_BUFFER seems is valid
+ */
+static bool index_buf_check(const struct INDEX_BUFFER *ib, u32 bytes,
+ const CLST *vbn)
+{
+ const struct NTFS_RECORD_HEADER *rhdr = &ib->rhdr;
+ u16 fo = le16_to_cpu(rhdr->fix_off);
+ u16 fn = le16_to_cpu(rhdr->fix_num);
+
+ if (bytes <= offsetof(struct INDEX_BUFFER, ihdr) ||
+ rhdr->sign != NTFS_INDX_SIGNATURE ||
+ fo < sizeof(struct INDEX_BUFFER)
+ /* Check index buffer vbn. */
+ || (vbn && *vbn != le64_to_cpu(ib->vbn)) || (fo % sizeof(short)) ||
+ fo + fn * sizeof(short) >= bytes ||
+ fn != ((bytes >> SECTOR_SHIFT) + 1)) {
+ /* incorrect index buffer. */
+ return false;
+ }
+
+ return index_hdr_check(&ib->ihdr,
+ bytes - offsetof(struct INDEX_BUFFER, ihdr));
+}
+
void fnd_clear(struct ntfs_fnd *fnd)
{
int i;
- for (i = 0; i < fnd->level; i++) {
+ for (i = fnd->level - 1; i >= 0; i--) {
struct indx_node *n = fnd->nodes[i];
if (!n)
static int fnd_push(struct ntfs_fnd *fnd, struct indx_node *n,
struct NTFS_DE *e)
{
- int i;
+ int i = fnd->level;
- i = fnd->level;
if (i < 0 || i >= ARRAY_SIZE(fnd->nodes))
return -EINVAL;
fnd->nodes[i] = n;
u32 t32;
const struct INDEX_ROOT *root = resident_data(attr);
+ t32 = le32_to_cpu(attr->res.data_size);
+ if (t32 <= offsetof(struct INDEX_ROOT, ihdr) ||
+ !index_hdr_check(&root->ihdr,
+ t32 - offsetof(struct INDEX_ROOT, ihdr))) {
+ goto out;
+ }
+
/* Check root fields. */
if (!root->index_block_clst)
- return -EINVAL;
+ goto out;
indx->type = type;
indx->idx2vbn_bits = __ffs(root->index_block_clst);
if (t32 < sbi->cluster_size) {
/* Index record is smaller than a cluster, use 512 blocks. */
if (t32 != root->index_block_clst * SECTOR_SIZE)
- return -EINVAL;
+ goto out;
/* Check alignment to a cluster. */
if ((sbi->cluster_size >> SECTOR_SHIFT) &
(root->index_block_clst - 1)) {
- return -EINVAL;
+ goto out;
}
indx->vbn2vbo_bits = SECTOR_SHIFT;
} else {
/* Index record must be a multiple of cluster size. */
if (t32 != root->index_block_clst << sbi->cluster_bits)
- return -EINVAL;
+ goto out;
indx->vbn2vbo_bits = sbi->cluster_bits;
}
init_rwsem(&indx->run_lock);
indx->cmp = get_cmp_func(root);
- return indx->cmp ? 0 : -EINVAL;
+ if (!indx->cmp)
+ goto out;
+
+ return 0;
+
+out:
+ ntfs_set_state(sbi, NTFS_DIRTY_DIRTY);
+ return -EINVAL;
}
static struct indx_node *indx_new(struct ntfs_index *indx,
goto out;
ok:
+ if (!index_buf_check(ib, bytes, &vbn)) {
+ ntfs_inode_err(&ni->vfs_inode, "directory corrupted");
+ ntfs_set_state(ni->mi.sbi, NTFS_DIRTY_ERROR);
+ err = -EINVAL;
+ goto out;
+ }
+
if (err == -E_NTFS_FIXUP) {
ntfs_write_bh(ni->mi.sbi, &ib->rhdr, &in->nb, 0);
err = 0;
}
+ /* check for index header length */
+ if (offsetof(struct INDEX_BUFFER, ihdr) + ib->ihdr.used > bytes) {
+ err = -EINVAL;
+ goto out;
+ }
+
in->index = ib;
*node = in;
run_init(&run);
- err = attr_allocate_clusters(sbi, &run, 0, 0, len, NULL, 0, &alen, 0,
- NULL);
+ err = attr_allocate_clusters(sbi, &run, 0, 0, len, NULL, ALLOCATE_DEF,
+ &alen, 0, NULL, NULL);
if (err)
goto out;
goto out1;
}
+ if (in->name == I30_NAME)
+ ni->vfs_inode.i_size = data_size;
+
*vbn = bit << indx->idx2vbn_bits;
return 0;
if (err) {
/* Restore root. */
- if (mi_resize_attr(mi, attr, -ds_root))
+ if (mi_resize_attr(mi, attr, -ds_root)) {
memcpy(attr, a_root, asize);
- else {
+ } else {
/* Bug? */
ntfs_set_state(sbi, NTFS_DIRTY_ERROR);
}
if (bit >= nbits)
return 0;
- pos = find_next_bit(bm, nbits, bit);
+ pos = find_next_bit_le(bm, nbits, bit);
if (pos < nbits)
return 0;
} else {
if (err)
return err;
+ if (in->name == I30_NAME)
+ ni->vfs_inode.i_size = new_data;
+
bpb = bitmap_size(bit);
if (bpb * 8 == nbits)
return 0;
fnd->de[level] = e;
indx_write(indx, ni, n, 0);
- /* Check to see if this action created an empty leaf. */
- if (ib_is_leaf(ib) && ib_is_empty(ib))
+ if (ib_is_leaf(ib) && ib_is_empty(ib)) {
+ /* An empty leaf. */
return 0;
+ }
out:
fnd_clear(fnd);
err = attr_set_size(ni, ATTR_ALLOC, in->name, in->name_len,
&indx->alloc_run, 0, NULL, false, NULL);
+ if (in->name == I30_NAME)
+ ni->vfs_inode.i_size = 0;
+
err = ni_remove_attr(ni, ATTR_ALLOC, in->name, in->name_len,
false, NULL);
run_close(&indx->alloc_run);
le16_to_cpu(ref->seq), le16_to_cpu(rec->seq));
goto out;
} else if (!is_rec_inuse(rec)) {
- err = -EINVAL;
+ err = -ESTALE;
ntfs_err(sb, "Inode r=%x is not in use!", (u32)ino);
goto out;
}
goto out;
}
- if (!is_rec_base(rec))
- goto Ok;
+ if (!is_rec_base(rec)) {
+ err = -EINVAL;
+ goto out;
+ }
/* Record should contain $I30 root. */
is_dir = rec->flags & RECORD_FLAG_DIR;
rsize = attr->non_res ? 0 : le32_to_cpu(attr->res.data_size);
asize = le32_to_cpu(attr->size);
+ if (le16_to_cpu(attr->name_off) + attr->name_len > asize)
+ goto out;
+
+ if (attr->non_res) {
+ t64 = le64_to_cpu(attr->nres.alloc_size);
+ if (le64_to_cpu(attr->nres.data_size) > t64 ||
+ le64_to_cpu(attr->nres.valid_size) > t64)
+ goto out;
+ }
+
switch (attr->type) {
case ATTR_STD:
if (attr->non_res ||
attr_unpack_run:
roff = le16_to_cpu(attr->nres.run_off);
+ if (roff > asize) {
+ err = -EINVAL;
+ goto out;
+ }
+
t64 = le64_to_cpu(attr->nres.svcn);
+
err = run_unpack_ex(run, sbi, ino, t64, le64_to_cpu(attr->nres.evcn),
t64, Add2Ptr(attr, roff), asize - roff);
if (err < 0)
inode->i_flags |= S_NOSEC;
}
-Ok:
if (ino == MFT_REC_MFT && !sb->s_root)
sbi->mft.ni = NULL;
_ntfs_bad_inode(inode);
}
+ if (IS_ERR(inode) && name)
+ ntfs_set_state(sb->s_fs_info, NTFS_DIRTY_ERROR);
+
return inode;
}
clear_buffer_new(bh);
clear_buffer_uptodate(bh);
- /* Direct write uses 'create=0'. */
- if (!create && vbo >= ni->i_valid) {
- /* Out of valid. */
- return 0;
- }
-
- if (vbo >= inode->i_size) {
- /* Out of size. */
- return 0;
- }
-
if (is_resident(ni)) {
ni_lock(ni);
err = attr_data_read_resident(ni, page);
off = vbo & sbi->cluster_mask;
new = false;
- err = attr_data_get_block(ni, vcn, 1, &lcn, &len, create ? &new : NULL);
+ err = attr_data_get_block(ni, vcn, 1, &lcn, &len, create ? &new : NULL,
+ create && sbi->cluster_size > PAGE_SIZE);
if (err)
goto out;
WARN_ON(1);
}
- if (new) {
+ if (new)
set_buffer_new(bh);
- if ((len << cluster_bits) > block_size)
- ntfs_sparse_cluster(inode, page, vcn, len);
- }
lbo = ((u64)lcn << cluster_bits) + off;
}
} else if (vbo >= valid) {
/* Read out of valid data. */
- /* Should never be here 'cause already checked. */
clear_buffer_mapped(bh);
} else if (vbo + bytes <= valid) {
/* Normal read. */
dirty = true;
}
+ if (pos + err > inode->i_size) {
+ inode->i_size = pos + err;
+ dirty = true;
+ }
+
if (dirty)
mark_inode_dirty(inode);
}
return ERR_PTR(err);
}
+/*
+ * ntfs_create_inode
+ *
+ * Helper function for:
+ * - ntfs_create
+ * - ntfs_mknod
+ * - ntfs_symlink
+ * - ntfs_mkdir
+ * - ntfs_atomic_open
+ *
+ * NOTE: if fnd != NULL (ntfs_atomic_open) then @dir is locked
+ */
struct inode *ntfs_create_inode(struct user_namespace *mnt_userns,
struct inode *dir, struct dentry *dentry,
const struct cpu_str *uni, umode_t mode,
struct REPARSE_DATA_BUFFER *rp = NULL;
bool rp_inserted = false;
- ni_lock_dir(dir_ni);
+ if (!fnd)
+ ni_lock_dir(dir_ni);
dir_root = indx_get_root(&dir_ni->dir, dir_ni, NULL, NULL);
if (!dir_root) {
fa = FILE_ATTRIBUTE_ARCHIVE;
}
+ /* If option "hide_dot_files" then set hidden attribute for dot files. */
+ if (sbi->options->hide_dot_files && name->name[0] == '.')
+ fa |= FILE_ATTRIBUTE_HIDDEN;
+
if (!(mode & 0222))
fa |= FILE_ATTRIBUTE_READONLY;
mi_get_ref(&ni->mi, &new_de->ref);
fname = (struct ATTR_FILE_NAME *)(new_de + 1);
+
+ if (sbi->options->windows_names &&
+ !valid_windows_name(sbi, (struct le_str *)&fname->name_len)) {
+ err = -EINVAL;
+ goto out4;
+ }
+
mi_get_ref(&dir_ni->mi, &fname->home);
fname->dup.cr_time = fname->dup.m_time = fname->dup.c_time =
fname->dup.a_time = std5->cr_time;
cpu_to_le64(ntfs_up_cluster(sbi, nsize));
err = attr_allocate_clusters(sbi, &ni->file.run, 0, 0,
- clst, NULL, 0, &alen, 0,
- NULL);
+ clst, NULL, ALLOCATE_DEF,
+ &alen, 0, NULL, NULL);
if (err)
goto out5;
goto out6;
/* Unlock parent directory before ntfs_init_acl. */
- ni_unlock(dir_ni);
+ if (!fnd)
+ ni_unlock(dir_ni);
inode->i_generation = le16_to_cpu(rec->seq);
out7:
/* Undo 'indx_insert_entry'. */
- ni_lock_dir(dir_ni);
+ if (!fnd)
+ ni_lock_dir(dir_ni);
indx_delete_entry(&dir_ni->dir, dir_ni, new_de + 1,
le16_to_cpu(new_de->key_size), sbi);
/* ni_unlock(dir_ni); will be called later. */
ntfs_remove_reparse(sbi, IO_REPARSE_TAG_SYMLINK, &new_de->ref);
out5:
- if (S_ISDIR(mode) || run_is_empty(&ni->file.run))
- goto out4;
-
- run_deallocate(sbi, &ni->file.run, false);
+ if (!S_ISDIR(mode))
+ run_deallocate(sbi, &ni->file.run, false);
out4:
clear_rec_inuse(rec);
out1:
if (err) {
- ni_unlock(dir_ni);
+ if (!fnd)
+ ni_unlock(dir_ni);
return ERR_PTR(err);
}
ni_clear(ntfs_i(inode));
}
-static noinline int ntfs_readlink_hlp(struct inode *inode, char *buffer,
+/*
+ * ntfs_translate_junction
+ *
+ * Translate a Windows junction target to the Linux equivalent.
+ * On junctions, targets are always absolute (they include the drive
+ * letter). We have no way of knowing if the target is for the current
+ * mounted device or not so we just assume it is.
+ */
+static int ntfs_translate_junction(const struct super_block *sb,
+ const struct dentry *link_de, char *target,
+ int target_len, int target_max)
+{
+ int tl_len, err = target_len;
+ char *link_path_buffer = NULL, *link_path;
+ char *translated = NULL;
+ char *target_start;
+ int copy_len;
+
+ link_path_buffer = kmalloc(PATH_MAX, GFP_NOFS);
+ if (!link_path_buffer) {
+ err = -ENOMEM;
+ goto out;
+ }
+ /* Get link path, relative to mount point */
+ link_path = dentry_path_raw(link_de, link_path_buffer, PATH_MAX);
+ if (IS_ERR(link_path)) {
+ ntfs_err(sb, "Error getting link path");
+ err = -EINVAL;
+ goto out;
+ }
+
+ translated = kmalloc(PATH_MAX, GFP_NOFS);
+ if (!translated) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ /* Make translated path a relative path to mount point */
+ strcpy(translated, "./");
+ ++link_path; /* Skip leading / */
+ for (tl_len = sizeof("./") - 1; *link_path; ++link_path) {
+ if (*link_path == '/') {
+ if (PATH_MAX - tl_len < sizeof("../")) {
+ ntfs_err(sb,
+ "Link path %s has too many components",
+ link_path);
+ err = -EINVAL;
+ goto out;
+ }
+ strcpy(translated + tl_len, "../");
+ tl_len += sizeof("../") - 1;
+ }
+ }
+
+ /* Skip drive letter */
+ target_start = target;
+ while (*target_start && *target_start != ':')
+ ++target_start;
+
+ if (!*target_start) {
+ ntfs_err(sb, "Link target (%s) missing drive separator",
+ target);
+ err = -EINVAL;
+ goto out;
+ }
+
+ /* Skip drive separator and leading /, if exists */
+ target_start += 1 + (target_start[1] == '/');
+ copy_len = target_len - (target_start - target);
+
+ if (PATH_MAX - tl_len <= copy_len) {
+ ntfs_err(sb, "Link target %s too large for buffer (%d <= %d)",
+ target_start, PATH_MAX - tl_len, copy_len);
+ err = -EINVAL;
+ goto out;
+ }
+
+ /* translated path has a trailing / and target_start does not */
+ strcpy(translated + tl_len, target_start);
+ tl_len += copy_len;
+ if (target_max <= tl_len) {
+ ntfs_err(sb, "Target path %s too large for buffer (%d <= %d)",
+ translated, target_max, tl_len);
+ err = -EINVAL;
+ goto out;
+ }
+ strcpy(target, translated);
+ err = tl_len;
+
+out:
+ kfree(link_path_buffer);
+ kfree(translated);
+ return err;
+}
+
+static noinline int ntfs_readlink_hlp(const struct dentry *link_de,
+ struct inode *inode, char *buffer,
int buflen)
{
int i, err = -EINVAL;
/* Always set last zero. */
buffer[err] = 0;
+
+ /* If this is a junction, translate the link target. */
+ if (rp->ReparseTag == IO_REPARSE_TAG_MOUNT_POINT)
+ err = ntfs_translate_junction(sb, link_de, buffer, err, buflen);
+
out:
kfree(to_free);
return err;
if (!ret)
return ERR_PTR(-ENOMEM);
- err = ntfs_readlink_hlp(inode, ret, PAGE_SIZE);
+ err = ntfs_readlink_hlp(de, inode, ret, PAGE_SIZE);
if (err < 0) {
kfree(ret);
return ERR_PTR(err);
#include <linux/fs.h>
#include <linux/nls.h>
+#include <linux/ctype.h>
+#include <linux/posix_acl.h>
#include "debug.h"
#include "ntfs.h"
ni_lock_dir(dir_ni);
ni_lock(ni);
+ if (dir_ni != new_dir_ni)
+ ni_lock_dir2(new_dir_ni);
is_bad = false;
err = ni_rename(dir_ni, new_dir_ni, ni, de, new_de, &is_bad);
ntfs_sync_inode(inode);
}
+ if (dir_ni != new_dir_ni)
+ ni_unlock(new_dir_ni);
ni_unlock(ni);
ni_unlock(dir_ni);
out:
return err;
}
+/*
+ * ntfs_atomic_open
+ *
+ * inode_operations::atomic_open
+ */
+static int ntfs_atomic_open(struct inode *dir, struct dentry *dentry,
+ struct file *file, u32 flags, umode_t mode)
+{
+ int err;
+ struct inode *inode;
+ struct ntfs_fnd *fnd = NULL;
+ struct ntfs_inode *ni = ntfs_i(dir);
+ struct dentry *d = NULL;
+ struct cpu_str *uni = __getname();
+ bool locked = false;
+
+ if (!uni)
+ return -ENOMEM;
+
+ err = ntfs_nls_to_utf16(ni->mi.sbi, dentry->d_name.name,
+ dentry->d_name.len, uni, NTFS_NAME_LEN,
+ UTF16_HOST_ENDIAN);
+ if (err < 0)
+ goto out;
+
+#ifdef CONFIG_NTFS3_FS_POSIX_ACL
+ if (IS_POSIXACL(dir)) {
+ /*
+ * Load in cache current acl to avoid ni_lock(dir):
+ * ntfs_create_inode -> ntfs_init_acl -> posix_acl_create ->
+ * ntfs_get_acl -> ntfs_get_acl_ex -> ni_lock
+ */
+ struct posix_acl *p = get_inode_acl(dir, ACL_TYPE_DEFAULT);
+
+ if (IS_ERR(p)) {
+ err = PTR_ERR(p);
+ goto out;
+ }
+ posix_acl_release(p);
+ }
+#endif
+
+ if (d_in_lookup(dentry)) {
+ ni_lock_dir(ni);
+ locked = true;
+ fnd = fnd_get();
+ if (!fnd) {
+ err = -ENOMEM;
+ goto out1;
+ }
+
+ d = d_splice_alias(dir_search_u(dir, uni, fnd), dentry);
+ if (IS_ERR(d)) {
+ err = PTR_ERR(d);
+ d = NULL;
+ goto out2;
+ }
+
+ if (d)
+ dentry = d;
+ }
+
+ if (!(flags & O_CREAT) || d_really_is_positive(dentry)) {
+ err = finish_no_open(file, d);
+ goto out2;
+ }
+
+ file->f_mode |= FMODE_CREATED;
+
+ /*
+ * fnd contains tree's path to insert to.
+ * If fnd is not NULL then dir is locked.
+ */
+
+ /*
+ * Unfortunately I don't know how to get here correct 'struct nameidata *nd'
+ * or 'struct user_namespace *mnt_userns'.
+ * See atomic_open in fs/namei.c.
+ * This is why xfstest/633 failed.
+ * Looks like ntfs_atomic_open must accept 'struct user_namespace *mnt_userns' as argument.
+ */
+
+ inode = ntfs_create_inode(&init_user_ns, dir, dentry, uni, mode, 0,
+ NULL, 0, fnd);
+ err = IS_ERR(inode) ? PTR_ERR(inode)
+ : finish_open(file, dentry, ntfs_file_open);
+ dput(d);
+
+out2:
+ fnd_put(fnd);
+out1:
+ if (locked)
+ ni_unlock(ni);
+out:
+ __putname(uni);
+ return err;
+}
+
struct dentry *ntfs3_get_parent(struct dentry *child)
{
struct inode *inode = d_inode(child);
return ERR_PTR(-ENOENT);
}
+/*
+ * dentry_operations::d_hash
+ */
+static int ntfs_d_hash(const struct dentry *dentry, struct qstr *name)
+{
+ struct ntfs_sb_info *sbi;
+ const char *n = name->name;
+ unsigned int len = name->len;
+ unsigned long hash;
+ struct cpu_str *uni;
+ unsigned int c;
+ int err;
+
+ /* First try fast implementation. */
+ hash = init_name_hash(dentry);
+
+ for (;;) {
+ if (!len--) {
+ name->hash = end_name_hash(hash);
+ return 0;
+ }
+
+ c = *n++;
+ if (c >= 0x80)
+ break;
+
+ hash = partial_name_hash(toupper(c), hash);
+ }
+
+ /*
+ * Try slow way with current upcase table
+ */
+ uni = __getname();
+ if (!uni)
+ return -ENOMEM;
+
+ sbi = dentry->d_sb->s_fs_info;
+
+ err = ntfs_nls_to_utf16(sbi, name->name, name->len, uni, NTFS_NAME_LEN,
+ UTF16_HOST_ENDIAN);
+ if (err < 0)
+ goto out;
+
+ if (!err) {
+ err = -EINVAL;
+ goto out;
+ }
+
+ hash = ntfs_names_hash(uni->name, uni->len, sbi->upcase,
+ init_name_hash(dentry));
+ name->hash = end_name_hash(hash);
+ err = 0;
+
+out:
+ __putname(uni);
+ return err;
+}
+
+/*
+ * dentry_operations::d_compare
+ */
+static int ntfs_d_compare(const struct dentry *dentry, unsigned int len1,
+ const char *str, const struct qstr *name)
+{
+ struct ntfs_sb_info *sbi;
+ int ret;
+ const char *n1 = str;
+ const char *n2 = name->name;
+ unsigned int len2 = name->len;
+ unsigned int lm = min(len1, len2);
+ unsigned char c1, c2;
+ struct cpu_str *uni1;
+ struct le_str *uni2;
+
+ /* First try fast implementation. */
+ for (;;) {
+ if (!lm--)
+ return len1 != len2;
+
+ if ((c1 = *n1++) == (c2 = *n2++))
+ continue;
+
+ if (c1 >= 0x80 || c2 >= 0x80)
+ break;
+
+ if (toupper(c1) != toupper(c2))
+ return 1;
+ }
+
+ /*
+ * Try slow way with current upcase table
+ */
+ sbi = dentry->d_sb->s_fs_info;
+ uni1 = __getname();
+ if (!uni1)
+ return -ENOMEM;
+
+ ret = ntfs_nls_to_utf16(sbi, str, len1, uni1, NTFS_NAME_LEN,
+ UTF16_HOST_ENDIAN);
+ if (ret < 0)
+ goto out;
+
+ if (!ret) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ uni2 = Add2Ptr(uni1, 2048);
+
+ ret = ntfs_nls_to_utf16(sbi, name->name, name->len,
+ (struct cpu_str *)uni2, NTFS_NAME_LEN,
+ UTF16_LITTLE_ENDIAN);
+ if (ret < 0)
+ goto out;
+
+ if (!ret) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = !ntfs_cmp_names_cpu(uni1, uni2, sbi->upcase, false) ? 0 : 1;
+
+out:
+ __putname(uni1);
+ return ret;
+}
+
// clang-format off
const struct inode_operations ntfs_dir_inode_operations = {
.lookup = ntfs_lookup,
.setattr = ntfs3_setattr,
.getattr = ntfs_getattr,
.listxattr = ntfs_listxattr,
+ .atomic_open = ntfs_atomic_open,
.fiemap = ntfs_fiemap,
};
.get_inode_acl = ntfs_get_acl,
.set_acl = ntfs_set_acl,
};
+
+const struct dentry_operations ntfs_dentry_ops = {
+ .d_hash = ntfs_d_hash,
+ .d_compare = ntfs_d_compare,
+};
+
// clang-format on
#define COMPRESSION_UNIT 4
#define COMPRESS_MAX_CLUSTER 0x1000
-#define MFT_INCREASE_CHUNK 1024
enum RECORD_NUM {
MFT_REC_MFT = 0,
{
u32 de_off = le32_to_cpu(hdr->de_off);
u32 used = le32_to_cpu(hdr->used);
- struct NTFS_DE *e = Add2Ptr(hdr, de_off);
+ struct NTFS_DE *e;
u16 esize;
- if (de_off >= used || de_off >= le32_to_cpu(hdr->total))
+ if (de_off >= used || de_off + sizeof(struct NTFS_DE) > used )
return NULL;
+ e = Add2Ptr(hdr, de_off);
esize = le16_to_cpu(e->size);
if (esize < sizeof(struct NTFS_DE) || de_off + esize > used)
return NULL;
unsigned sparse : 1; /* Create sparse files. */
unsigned showmeta : 1; /* Show meta files. */
unsigned nohidden : 1; /* Do not show hidden files. */
+ unsigned hide_dot_files : 1; /* Set hidden flag on dot files. */
+ unsigned windows_names : 1; /* Disallow names forbidden by Windows. */
unsigned force : 1; /* RW mount dirty volume. */
unsigned noacsrules : 1; /* Exclude acs rules. */
unsigned prealloc : 1; /* Preallocate space when file is growing. */
+ unsigned nocase : 1; /* case insensitive. */
};
/* Special value to unpack and deallocate. */
enum ALLOCATE_OPT {
ALLOCATE_DEF = 0, // Allocate all clusters.
ALLOCATE_MFT = 1, // Allocate for MFT.
+ ALLOCATE_ZERO = 2, // Zeroout new allocated clusters
};
enum bitmap_mutex_classes {
/* Minimum MFT zone. */
#define NTFS_MIN_MFT_ZONE 100
+/* Step to increase the MFT. */
+#define NTFS_MFT_INCREASE_STEP 1024
/* Ntfs file system in-core superblock data. */
struct ntfs_sb_info {
NTFS_INODE_MUTEX_REPARSE,
NTFS_INODE_MUTEX_NORMAL,
NTFS_INODE_MUTEX_PARENT,
+ NTFS_INODE_MUTEX_PARENT2,
};
/*
int attr_allocate_clusters(struct ntfs_sb_info *sbi, struct runs_tree *run,
CLST vcn, CLST lcn, CLST len, CLST *pre_alloc,
enum ALLOCATE_OPT opt, CLST *alen, const size_t fr,
- CLST *new_lcn);
+ CLST *new_lcn, CLST *new_len);
int attr_make_nonresident(struct ntfs_inode *ni, struct ATTRIB *attr,
struct ATTR_LIST_ENTRY *le, struct mft_inode *mi,
u64 new_size, struct runs_tree *run,
u64 new_size, const u64 *new_valid, bool keep_prealloc,
struct ATTRIB **ret);
int attr_data_get_block(struct ntfs_inode *ni, CLST vcn, CLST clen, CLST *lcn,
- CLST *len, bool *new);
+ CLST *len, bool *new, bool zero);
int attr_data_read_resident(struct ntfs_inode *ni, struct page *page);
int attr_data_write_resident(struct ntfs_inode *ni, struct page *page);
int attr_load_runs_vcn(struct ntfs_inode *ni, enum ATTR_TYPE type,
}
/* Globals from bitfunc.c */
-bool are_bits_clear(const ulong *map, size_t bit, size_t nbits);
-bool are_bits_set(const ulong *map, size_t bit, size_t nbits);
-size_t get_set_bits_ex(const ulong *map, size_t bit, size_t nbits);
+bool are_bits_clear(const void *map, size_t bit, size_t nbits);
+bool are_bits_set(const void *map, size_t bit, size_t nbits);
+size_t get_set_bits_ex(const void *map, size_t bit, size_t nbits);
/* Globals from dir.c */
int ntfs_utf16_to_nls(struct ntfs_sb_info *sbi, const __le16 *name, u32 len,
/* Globals from file.c */
int ntfs_getattr(struct user_namespace *mnt_userns, const struct path *path,
struct kstat *stat, u32 request_mask, u32 flags);
-void ntfs_sparse_cluster(struct inode *inode, struct page *page0, CLST vcn,
- CLST len);
int ntfs3_setattr(struct user_namespace *mnt_userns, struct dentry *dentry,
struct iattr *attr);
int ntfs_file_open(struct inode *inode, struct file *file);
bool simple);
int ntfs_extend_init(struct ntfs_sb_info *sbi);
int ntfs_loadlog_and_replay(struct ntfs_inode *ni, struct ntfs_sb_info *sbi);
-const struct ATTR_DEF_ENTRY *ntfs_query_def(struct ntfs_sb_info *sbi,
- enum ATTR_TYPE Type);
int ntfs_look_for_free_space(struct ntfs_sb_info *sbi, CLST lcn, CLST len,
CLST *new_lcn, CLST *new_len,
enum ALLOCATE_OPT opt);
+bool ntfs_check_for_free_space(struct ntfs_sb_info *sbi, CLST clen, CLST mlen);
int ntfs_look_free_mft(struct ntfs_sb_info *sbi, CLST *rno, bool mft,
struct ntfs_inode *ni, struct mft_inode **mi);
void ntfs_mark_rec_free(struct ntfs_sb_info *sbi, CLST rno, bool is_mft);
const struct MFT_REF *ref);
void mark_as_free_ex(struct ntfs_sb_info *sbi, CLST lcn, CLST len, bool trim);
int run_deallocate(struct ntfs_sb_info *sbi, struct runs_tree *run, bool trim);
+bool valid_windows_name(struct ntfs_sb_info *sbi, const struct le_str *name);
/* Globals from index.c */
int indx_used_bit(struct ntfs_index *indx, struct ntfs_inode *ni, size_t *bit);
extern const struct inode_operations ntfs_dir_inode_operations;
extern const struct inode_operations ntfs_special_inode_operations;
+extern const struct dentry_operations ntfs_dentry_ops;
/* Globals from record.c */
int mi_get(struct ntfs_sb_info *sbi, CLST rno, struct mft_inode **mi);
u32 run_buf_size, CLST *packed_vcns);
int run_unpack(struct runs_tree *run, struct ntfs_sb_info *sbi, CLST ino,
CLST svcn, CLST evcn, CLST vcn, const u8 *run_buf,
- u32 run_buf_size);
+ int run_buf_size);
#ifdef NTFS3_CHECK_FREE_CLST
int run_unpack_ex(struct runs_tree *run, struct ntfs_sb_info *sbi, CLST ino,
CLST svcn, CLST evcn, CLST vcn, const u8 *run_buf,
- u32 run_buf_size);
+ int run_buf_size);
#else
#define run_unpack_ex run_unpack
#endif
int wnd_init(struct wnd_bitmap *wnd, struct super_block *sb, size_t nbits);
int wnd_set_free(struct wnd_bitmap *wnd, size_t bit, size_t bits);
int wnd_set_used(struct wnd_bitmap *wnd, size_t bit, size_t bits);
+int wnd_set_used_safe(struct wnd_bitmap *wnd, size_t bit, size_t bits,
+ size_t *done);
bool wnd_is_free(struct wnd_bitmap *wnd, size_t bit, size_t bits);
bool wnd_is_used(struct wnd_bitmap *wnd, size_t bit, size_t bits);
void wnd_zone_set(struct wnd_bitmap *wnd, size_t Lcn, size_t Len);
int ntfs_trim_fs(struct ntfs_sb_info *sbi, struct fstrim_range *range);
+void ntfs_bitmap_set_le(void *map, unsigned int start, int len);
+void ntfs_bitmap_clear_le(void *map, unsigned int start, int len);
+unsigned int ntfs_bitmap_weight_le(const void *bitmap, int bits);
+
/* Globals from upcase.c */
int ntfs_cmp_names(const __le16 *s1, size_t l1, const __le16 *s2, size_t l2,
const u16 *upcase, bool bothcase);
int ntfs_cmp_names_cpu(const struct cpu_str *uni1, const struct le_str *uni2,
const u16 *upcase, bool bothcase);
+unsigned long ntfs_names_hash(const u16 *name, size_t len, const u16 *upcase,
+ unsigned long hash);
/* globals from xattr.c */
#ifdef CONFIG_NTFS3_FS_POSIX_ACL
mutex_lock_nested(&ni->ni_lock, NTFS_INODE_MUTEX_PARENT);
}
+static inline void ni_lock_dir2(struct ntfs_inode *ni)
+{
+ mutex_lock_nested(&ni->ni_lock, NTFS_INODE_MUTEX_PARENT2);
+}
+
static inline void ni_unlock(struct ntfs_inode *ni)
{
mutex_unlock(&ni->ni_lock);
return NULL;
}
+ if (off + asize < off) {
+ /* overflow check */
+ return NULL;
+ }
+
attr = Add2Ptr(attr, asize);
off += asize;
}
if (t16 + t32 > asize)
return NULL;
+ t32 = sizeof(short) * attr->name_len;
+ if (t32 && le16_to_cpu(attr->name_off) + t32 > t16)
+ return NULL;
+
return attr;
}
return true;
}
+/*
+ * Pack runs in MFT record.
+ * If failed record is not changed.
+ */
int mi_pack_runs(struct mft_inode *mi, struct ATTRIB *attr,
struct runs_tree *run, CLST len)
{
*/
int run_unpack(struct runs_tree *run, struct ntfs_sb_info *sbi, CLST ino,
CLST svcn, CLST evcn, CLST vcn, const u8 *run_buf,
- u32 run_buf_size)
+ int run_buf_size)
{
u64 prev_lcn, vcn64, lcn, next_vcn;
const u8 *run_last, *run_0;
bool is_mft = ino == MFT_REC_MFT;
+ if (run_buf_size < 0)
+ return -EINVAL;
+
/* Check for empty. */
if (evcn + 1 == svcn)
return 0;
*/
int run_unpack_ex(struct runs_tree *run, struct ntfs_sb_info *sbi, CLST ino,
CLST svcn, CLST evcn, CLST vcn, const u8 *run_buf,
- u32 run_buf_size)
+ int run_buf_size)
{
int ret, err;
CLST next_vcn, lcn, len;
if (down_write_trylock(&wnd->rw_lock)) {
/* Mark all zero bits as used in range [lcn, lcn+len). */
- CLST i, lcn_f = 0, len_f = 0;
-
- err = 0;
- for (i = 0; i < len; i++) {
- if (wnd_is_free(wnd, lcn + i, 1)) {
- if (!len_f)
- lcn_f = lcn + i;
- len_f += 1;
- } else if (len_f) {
- err = wnd_set_used(wnd, lcn_f, len_f);
- len_f = 0;
- if (err)
- break;
- }
- }
-
- if (len_f)
- err = wnd_set_used(wnd, lcn_f, len_f);
-
+ size_t done;
+ err = wnd_set_used_safe(wnd, lcn, len, &done);
up_write(&wnd->rw_lock);
if (err)
return err;
* https://docs.microsoft.com/en-us/windows/wsl/file-permissions
* It stores uid/gid/mode/dev in xattr
*
+ * ntfs allows up to 2^64 clusters per volume.
+ * It means you should use 64 bits lcn to operate with ntfs.
+ * Implementation of ntfs.sys uses only 32 bits lcn.
+ * Default ntfs3 uses 32 bits lcn too.
+ * ntfs3 built with CONFIG_NTFS3_64BIT_CLUSTER (ntfs3_64) uses 64 bits per lcn.
+ *
+ *
+ * ntfs limits, cluster size is 4K (2^12)
+ * -----------------------------------------------------------------------------
+ * | Volume size | Clusters | ntfs.sys | ntfs3 | ntfs3_64 | mkntfs | chkdsk |
+ * -----------------------------------------------------------------------------
+ * | < 16T, 2^44 | < 2^32 | yes | yes | yes | yes | yes |
+ * | > 16T, 2^44 | > 2^32 | no | no | yes | yes | yes |
+ * ----------------------------------------------------------|------------------
+ *
+ * To mount large volumes as ntfs one should use large cluster size (up to 2M)
+ * The maximum volume size in this case is 2^32 * 2^21 = 2^53 = 8P
+ *
+ * ntfs limits, cluster size is 2M (2^31)
+ * -----------------------------------------------------------------------------
+ * | < 8P, 2^54 | < 2^32 | yes | yes | yes | yes | yes |
+ * | > 8P, 2^54 | > 2^32 | no | no | yes | yes | yes |
+ * ----------------------------------------------------------|------------------
+ *
*/
#include <linux/blkdev.h>
Opt_force,
Opt_sparse,
Opt_nohidden,
+ Opt_hide_dot_files,
+ Opt_windows_names,
Opt_showmeta,
Opt_acl,
Opt_iocharset,
Opt_prealloc,
Opt_noacsrules,
+ Opt_nocase,
Opt_err,
};
fsparam_flag_no("force", Opt_force),
fsparam_flag_no("sparse", Opt_sparse),
fsparam_flag_no("hidden", Opt_nohidden),
+ fsparam_flag_no("hide_dot_files", Opt_hide_dot_files),
+ fsparam_flag_no("windows_names", Opt_windows_names),
fsparam_flag_no("acl", Opt_acl),
fsparam_flag_no("showmeta", Opt_showmeta),
fsparam_flag_no("prealloc", Opt_prealloc),
fsparam_flag_no("acsrules", Opt_noacsrules),
+ fsparam_flag_no("nocase", Opt_nocase),
fsparam_string("iocharset", Opt_iocharset),
{}
};
case Opt_nohidden:
opts->nohidden = result.negated ? 1 : 0;
break;
+ case Opt_hide_dot_files:
+ opts->hide_dot_files = result.negated ? 0 : 1;
+ break;
+ case Opt_windows_names:
+ opts->windows_names = result.negated ? 0 : 1;
+ break;
case Opt_acl:
if (!result.negated)
#ifdef CONFIG_NTFS3_FS_POSIX_ACL
case Opt_noacsrules:
opts->noacsrules = result.negated ? 1 : 0;
break;
+ case Opt_nocase:
+ opts->nocase = result.negated ? 1 : 0;
+ break;
default:
/* Should not be here unless we forget add case. */
return -EINVAL;
return NULL;
memset(ni, 0, offsetof(struct ntfs_inode, vfs_inode));
-
mutex_init(&ni->ni_lock);
-
return &ni->vfs_inode;
}
-static void ntfs_i_callback(struct rcu_head *head)
+static void ntfs_free_inode(struct inode *inode)
{
- struct inode *inode = container_of(head, struct inode, i_rcu);
struct ntfs_inode *ni = ntfs_i(inode);
mutex_destroy(&ni->ni_lock);
-
kmem_cache_free(ntfs_inode_cachep, ni);
}
-static void ntfs_destroy_inode(struct inode *inode)
-{
- call_rcu(&inode->i_rcu, ntfs_i_callback);
-}
-
static void init_once(void *foo)
{
struct ntfs_inode *ni = foo;
seq_printf(m, ",gid=%u",
from_kgid_munged(user_ns, opts->fs_gid));
if (opts->fmask)
- seq_printf(m, ",fmask=%04o", ~opts->fs_fmask_inv);
+ seq_printf(m, ",fmask=%04o", opts->fs_fmask_inv ^ 0xffff);
if (opts->dmask)
- seq_printf(m, ",dmask=%04o", ~opts->fs_dmask_inv);
+ seq_printf(m, ",dmask=%04o", opts->fs_dmask_inv ^ 0xffff);
if (opts->nls)
seq_printf(m, ",iocharset=%s", opts->nls->charset);
else
seq_puts(m, ",showmeta");
if (opts->nohidden)
seq_puts(m, ",nohidden");
+ if (opts->windows_names)
+ seq_puts(m, ",windows_names");
+ if (opts->hide_dot_files)
+ seq_puts(m, ",hide_dot_files");
if (opts->force)
seq_puts(m, ",force");
if (opts->noacsrules)
static const struct super_operations ntfs_sops = {
.alloc_inode = ntfs_alloc_inode,
- .destroy_inode = ntfs_destroy_inode,
+ .free_inode = ntfs_free_inode,
.evict_inode = ntfs_evict_inode,
.put_super = ntfs_put_super,
.statfs = ntfs_statfs,
if (boot->sectors_per_clusters <= 0x80)
return boot->sectors_per_clusters;
if (boot->sectors_per_clusters >= 0xf4) /* limit shift to 2MB max */
- return 1U << (0 - boot->sectors_per_clusters);
+ return 1U << -(s8)boot->sectors_per_clusters;
return -EINVAL;
}
: (u32)boot->record_size
<< sbi->cluster_bits;
- if (record_size > MAXIMUM_BYTES_PER_MFT)
+ if (record_size > MAXIMUM_BYTES_PER_MFT || record_size < SECTOR_SIZE)
goto out;
sbi->record_bits = blksize_bits(record_size);
struct block_device *bdev = sb->s_bdev;
struct inode *inode;
struct ntfs_inode *ni;
- size_t i, tt;
+ size_t i, tt, bad_len, bad_frags;
CLST vcn, lcn, len;
struct ATTRIB *attr;
const struct VOLUME_INFO *info;
sb->s_export_op = &ntfs_export_ops;
sb->s_time_gran = NTFS_TIME_GRAN; // 100 nsec
sb->s_xattr = ntfs_xattr_handlers;
+ sb->s_d_op = sbi->options->nocase ? &ntfs_dentry_ops : NULL;
sbi->options->nls = ntfs_load_nls(sbi->options->nls_name);
if (IS_ERR(sbi->options->nls)) {
sbi->mft.ni = ni;
- /* Load $BadClus. */
- ref.low = cpu_to_le32(MFT_REC_BADCLUST);
- ref.seq = cpu_to_le16(MFT_REC_BADCLUST);
- inode = ntfs_iget5(sb, &ref, &NAME_BADCLUS);
- if (IS_ERR(inode)) {
- ntfs_err(sb, "Failed to load $BadClus.");
- err = PTR_ERR(inode);
- goto out;
- }
-
- ni = ntfs_i(inode);
-
- for (i = 0; run_get_entry(&ni->file.run, i, &vcn, &lcn, &len); i++) {
- if (lcn == SPARSE_LCN)
- continue;
-
- if (!sbi->bad_clusters)
- ntfs_notice(sb, "Volume contains bad blocks");
-
- sbi->bad_clusters += len;
- }
-
- iput(inode);
-
/* Load $Bitmap. */
ref.low = cpu_to_le32(MFT_REC_BITMAP);
ref.seq = cpu_to_le16(MFT_REC_BITMAP);
if (err)
goto out;
+ /* Load $BadClus. */
+ ref.low = cpu_to_le32(MFT_REC_BADCLUST);
+ ref.seq = cpu_to_le16(MFT_REC_BADCLUST);
+ inode = ntfs_iget5(sb, &ref, &NAME_BADCLUS);
+ if (IS_ERR(inode)) {
+ err = PTR_ERR(inode);
+ ntfs_err(sb, "Failed to load $BadClus (%d).", err);
+ goto out;
+ }
+
+ ni = ntfs_i(inode);
+ bad_len = bad_frags = 0;
+ for (i = 0; run_get_entry(&ni->file.run, i, &vcn, &lcn, &len); i++) {
+ if (lcn == SPARSE_LCN)
+ continue;
+
+ bad_len += len;
+ bad_frags += 1;
+ if (sb_rdonly(sb))
+ continue;
+
+ if (wnd_set_used_safe(&sbi->used.bitmap, lcn, len, &tt) || tt) {
+ /* Bad blocks marked as free in bitmap. */
+ ntfs_set_state(sbi, NTFS_DIRTY_ERROR);
+ }
+ }
+ if (bad_len) {
+ /*
+ * Notice about bad blocks.
+ * In normal cases these blocks are marked as used in bitmap.
+ * And we never allocate space in it.
+ */
+ ntfs_notice(sb,
+ "Volume contains %zu bad blocks in %zu fragments.",
+ bad_len, bad_frags);
+ }
+ iput(inode);
+
/* Load $AttrDef. */
ref.low = cpu_to_le32(MFT_REC_ATTR);
ref.seq = cpu_to_le16(MFT_REC_ATTR);
goto put_inode_out;
}
bytes = inode->i_size;
- sbi->def_table = t = kmalloc(bytes, GFP_NOFS);
+ sbi->def_table = t = kmalloc(bytes, GFP_NOFS | __GFP_NOWARN);
if (!t) {
err = -ENOMEM;
goto put_inode_out;
ref.low = cpu_to_le32(MFT_REC_ROOT);
ref.seq = cpu_to_le16(MFT_REC_ROOT);
inode = ntfs_iget5(sb, &ref, &NAME_ROOT);
- if (IS_ERR(inode)) {
+ if (IS_ERR(inode) || !inode->i_op) {
ntfs_err(sb, "Failed to load root.");
- err = PTR_ERR(inode);
+ err = IS_ERR(inode) ? PTR_ERR(inode) : -EINVAL;
goto out;
}
* Free resources here.
* ntfs_fs_free will be called with fc->s_fs_info = NULL
*/
+ put_mount_options(sbi->options);
put_ntfs(sbi);
sb->s_fs_info = NULL;
static void __exit exit_ntfs_fs(void)
{
- if (ntfs_inode_cachep) {
- rcu_barrier();
- kmem_cache_destroy(ntfs_inode_cachep);
- }
-
+ rcu_barrier();
+ kmem_cache_destroy(ntfs_inode_cachep);
unregister_filesystem(&ntfs_fs_type);
ntfs3_exit_bitmap();
}
diff2 = l1 - l2;
return diff2 ? diff2 : diff1;
}
+
+/* Helper function for ntfs_d_hash. */
+unsigned long ntfs_names_hash(const u16 *name, size_t len, const u16 *upcase,
+ unsigned long hash)
+{
+ while (len--) {
+ unsigned int c = upcase_unicode_char(upcase, *name++);
+ hash = partial_name_hash(c, hash);
+ }
+
+ return hash;
+}
#include "ntfs_fs.h"
// clang-format off
-#define SYSTEM_DOS_ATTRIB "system.dos_attrib"
-#define SYSTEM_NTFS_ATTRIB "system.ntfs_attrib"
-#define SYSTEM_NTFS_SECURITY "system.ntfs_security"
+#define SYSTEM_DOS_ATTRIB "system.dos_attrib"
+#define SYSTEM_NTFS_ATTRIB "system.ntfs_attrib"
+#define SYSTEM_NTFS_ATTRIB_BE "system.ntfs_attrib_be"
+#define SYSTEM_NTFS_SECURITY "system.ntfs_security"
// clang-format on
static inline size_t unpacked_ea_size(const struct EA_FULL *ea)
* Assume there is at least one xattr in the list.
*/
static inline bool find_ea(const struct EA_FULL *ea_all, u32 bytes,
- const char *name, u8 name_len, u32 *off)
+ const char *name, u8 name_len, u32 *off, u32 *ea_sz)
{
- *off = 0;
+ u32 ea_size;
- if (!ea_all || !bytes)
+ *off = 0;
+ if (!ea_all)
return false;
- for (;;) {
+ for (; *off < bytes; *off += ea_size) {
const struct EA_FULL *ea = Add2Ptr(ea_all, *off);
- u32 next_off = *off + unpacked_ea_size(ea);
-
- if (next_off > bytes)
- return false;
-
+ ea_size = unpacked_ea_size(ea);
if (ea->name_len == name_len &&
- !memcmp(ea->name, name, name_len))
+ !memcmp(ea->name, name, name_len)) {
+ if (ea_sz)
+ *ea_sz = ea_size;
return true;
-
- *off = next_off;
- if (next_off >= bytes)
- return false;
+ }
}
+
+ return false;
}
/*
static int ntfs_read_ea(struct ntfs_inode *ni, struct EA_FULL **ea,
size_t add_bytes, const struct EA_INFO **info)
{
- int err;
+ int err = -EINVAL;
struct ntfs_sb_info *sbi = ni->mi.sbi;
struct ATTR_LIST_ENTRY *le = NULL;
struct ATTRIB *attr_info, *attr_ea;
void *ea_p;
- u32 size;
+ u32 size, off, ea_size;
static_assert(le32_to_cpu(ATTR_EA_INFO) < le32_to_cpu(ATTR_EA));
*info = resident_data_ex(attr_info, sizeof(struct EA_INFO));
if (!*info)
- return -EINVAL;
+ goto out;
/* Check Ea limit. */
size = le32_to_cpu((*info)->size);
- if (size > sbi->ea_max_size)
- return -EFBIG;
+ if (size > sbi->ea_max_size) {
+ err = -EFBIG;
+ goto out;
+ }
- if (attr_size(attr_ea) > sbi->ea_max_size)
- return -EFBIG;
+ if (attr_size(attr_ea) > sbi->ea_max_size) {
+ err = -EFBIG;
+ goto out;
+ }
+
+ if (!size) {
+ /* EA info persists, but xattr is empty. Looks like EA problem. */
+ goto out;
+ }
/* Allocate memory for packed Ea. */
- ea_p = kmalloc(size + add_bytes, GFP_NOFS);
+ ea_p = kmalloc(size_add(size, add_bytes), GFP_NOFS);
if (!ea_p)
return -ENOMEM;
- if (!size) {
- /* EA info persists, but xattr is empty. Looks like EA problem. */
- } else if (attr_ea->non_res) {
+ if (attr_ea->non_res) {
struct runs_tree run;
run_init(&run);
run_close(&run);
if (err)
- goto out;
+ goto out1;
} else {
void *p = resident_data_ex(attr_ea, size);
- if (!p) {
- err = -EINVAL;
- goto out;
- }
+ if (!p)
+ goto out1;
memcpy(ea_p, p, size);
}
memset(Add2Ptr(ea_p, size), 0, add_bytes);
+
+ /* Check all attributes for consistency. */
+ for (off = 0; off < size; off += ea_size) {
+ const struct EA_FULL *ef = Add2Ptr(ea_p, off);
+ u32 bytes = size - off;
+
+ /* Check if we can use field ea->size. */
+ if (bytes < sizeof(ef->size))
+ goto out1;
+
+ if (ef->size) {
+ ea_size = le32_to_cpu(ef->size);
+ if (ea_size > bytes)
+ goto out1;
+ continue;
+ }
+
+ /* Check if we can use fields ef->name_len and ef->elength. */
+ if (bytes < offsetof(struct EA_FULL, name))
+ goto out1;
+
+ ea_size = ALIGN(struct_size(ef, name,
+ 1 + ef->name_len +
+ le16_to_cpu(ef->elength)),
+ 4);
+ if (ea_size > bytes)
+ goto out1;
+ }
+
*ea = ea_p;
return 0;
-out:
+out1:
kfree(ea_p);
- *ea = NULL;
+out:
+ ntfs_set_state(sbi, NTFS_DIRTY_DIRTY);
return err;
}
const struct EA_FULL *ea;
u32 off, size;
int err;
+ int ea_size;
size_t ret;
err = ntfs_read_ea(ni, &ea_all, 0, &info);
size = le32_to_cpu(info->size);
/* Enumerate all xattrs. */
- for (ret = 0, off = 0; off < size; off += unpacked_ea_size(ea)) {
+ for (ret = 0, off = 0; off < size; off += ea_size) {
ea = Add2Ptr(ea_all, off);
+ ea_size = unpacked_ea_size(ea);
if (buffer) {
if (ret + ea->name_len + 1 > bytes_per_buffer) {
goto out;
/* Enumerate all xattrs. */
- if (!find_ea(ea_all, le32_to_cpu(info->size), name, name_len, &off)) {
+ if (!find_ea(ea_all, le32_to_cpu(info->size), name, name_len, &off,
+ NULL)) {
err = -ENODATA;
goto out;
}
struct EA_FULL *new_ea;
struct EA_FULL *ea_all = NULL;
size_t add, new_pack;
- u32 off, size;
+ u32 off, size, ea_sz;
__le16 size_pack;
struct ATTRIB *attr;
struct ATTR_LIST_ENTRY *le;
size_pack = ea_info.size_pack;
}
- if (info && find_ea(ea_all, size, name, name_len, &off)) {
+ if (info && find_ea(ea_all, size, name, name_len, &off, &ea_sz)) {
struct EA_FULL *ea;
- size_t ea_sz;
if (flags & XATTR_CREATE) {
err = -EEXIST;
if (ea->flags & FILE_NEED_EA)
le16_add_cpu(&ea_info.count, -1);
- ea_sz = unpacked_ea_size(ea);
-
le16_add_cpu(&ea_info.size_pack, 0 - packed_ea_size(ea));
memmove(ea, Add2Ptr(ea, ea_sz), size - off - ea_sz);
err = 0; /* Removing non existed xattr. */
if (!err) {
set_cached_acl(inode, type, acl);
- if (inode->i_mode != mode) {
- inode->i_mode = mode;
- mark_inode_dirty(inode);
- }
+ inode->i_mode = mode;
+ inode->i_ctime = current_time(inode);
+ mark_inode_dirty(inode);
}
out:
{
int err;
struct ntfs_inode *ni = ntfs_i(inode);
- size_t name_len = strlen(name);
/* Dispatch request. */
- if (name_len == sizeof(SYSTEM_DOS_ATTRIB) - 1 &&
- !memcmp(name, SYSTEM_DOS_ATTRIB, sizeof(SYSTEM_DOS_ATTRIB))) {
+ if (!strcmp(name, SYSTEM_DOS_ATTRIB)) {
/* system.dos_attrib */
if (!buffer) {
err = sizeof(u8);
goto out;
}
- if (name_len == sizeof(SYSTEM_NTFS_ATTRIB) - 1 &&
- !memcmp(name, SYSTEM_NTFS_ATTRIB, sizeof(SYSTEM_NTFS_ATTRIB))) {
+ if (!strcmp(name, SYSTEM_NTFS_ATTRIB) ||
+ !strcmp(name, SYSTEM_NTFS_ATTRIB_BE)) {
/* system.ntfs_attrib */
if (!buffer) {
err = sizeof(u32);
} else {
err = sizeof(u32);
*(u32 *)buffer = le32_to_cpu(ni->std_fa);
+ if (!strcmp(name, SYSTEM_NTFS_ATTRIB_BE))
+ *(u32 *)buffer = cpu_to_be32(*(u32 *)buffer);
}
goto out;
}
- if (name_len == sizeof(SYSTEM_NTFS_SECURITY) - 1 &&
- !memcmp(name, SYSTEM_NTFS_SECURITY, sizeof(SYSTEM_NTFS_SECURITY))) {
+ if (!strcmp(name, SYSTEM_NTFS_SECURITY)) {
/* system.ntfs_security*/
struct SECURITY_DESCRIPTOR_RELATIVE *sd = NULL;
size_t sd_size = 0;
}
/* Deal with NTFS extended attribute. */
- err = ntfs_get_ea(inode, name, name_len, buffer, size, NULL);
+ err = ntfs_get_ea(inode, name, strlen(name), buffer, size, NULL);
out:
return err;
{
int err = -EINVAL;
struct ntfs_inode *ni = ntfs_i(inode);
- size_t name_len = strlen(name);
enum FILE_ATTRIBUTE new_fa;
/* Dispatch request. */
- if (name_len == sizeof(SYSTEM_DOS_ATTRIB) - 1 &&
- !memcmp(name, SYSTEM_DOS_ATTRIB, sizeof(SYSTEM_DOS_ATTRIB))) {
+ if (!strcmp(name, SYSTEM_DOS_ATTRIB)) {
if (sizeof(u8) != size)
goto out;
new_fa = cpu_to_le32(*(u8 *)value);
goto set_new_fa;
}
- if (name_len == sizeof(SYSTEM_NTFS_ATTRIB) - 1 &&
- !memcmp(name, SYSTEM_NTFS_ATTRIB, sizeof(SYSTEM_NTFS_ATTRIB))) {
+ if (!strcmp(name, SYSTEM_NTFS_ATTRIB) ||
+ !strcmp(name, SYSTEM_NTFS_ATTRIB_BE)) {
if (size != sizeof(u32))
goto out;
- new_fa = cpu_to_le32(*(u32 *)value);
+ if (!strcmp(name, SYSTEM_NTFS_ATTRIB_BE))
+ new_fa = cpu_to_le32(be32_to_cpu(*(u32 *)value));
+ else
+ new_fa = cpu_to_le32(*(u32 *)value);
if (S_ISREG(inode->i_mode)) {
/* Process compressed/sparsed in special way. */
goto out;
}
- if (name_len == sizeof(SYSTEM_NTFS_SECURITY) - 1 &&
- !memcmp(name, SYSTEM_NTFS_SECURITY, sizeof(SYSTEM_NTFS_SECURITY))) {
+ if (!strcmp(name, SYSTEM_NTFS_SECURITY)) {
/* system.ntfs_security*/
__le32 security_id;
bool inserted;
}
/* Deal with NTFS extended attribute. */
- err = ntfs_set_ea(inode, name, name_len, value, size, flags, 0);
+ err = ntfs_set_ea(inode, name, strlen(name), value, size, flags, 0);
out:
inode->i_ctime = current_time(inode);
sc->sc_sock = sock; /* freed by sc_kref_release */
sock->sk->sk_allocation = GFP_ATOMIC;
+ sock->sk->sk_use_task_frag = false;
myaddr.sin_family = AF_INET;
myaddr.sin_addr.s_addr = mynode->nd_ipv4_address;
}
do {
struct mount *parent = last_source->mnt_parent;
- if (last_source == first_source)
+ if (peers(last_source, first_source))
break;
done = parent->mnt_master == p;
if (done && peers(n, parent))
#undef pr_fmt
#define pr_fmt(fmt) PMSG_NAME ": " fmt
-static char *pmsg_devnode(struct device *dev, umode_t *mode)
+static char *pmsg_devnode(const struct device *dev, umode_t *mode)
{
if (mode)
*mode = 0220;
DECLARE_TRACEPOINT(rwmmio_post_read);
void log_write_mmio(u64 val, u8 width, volatile void __iomem *addr,
- unsigned long caller_addr);
+ unsigned long caller_addr, unsigned long caller_addr0);
void log_post_write_mmio(u64 val, u8 width, volatile void __iomem *addr,
- unsigned long caller_addr);
+ unsigned long caller_addr, unsigned long caller_addr0);
void log_read_mmio(u8 width, const volatile void __iomem *addr,
- unsigned long caller_addr);
+ unsigned long caller_addr, unsigned long caller_addr0);
void log_post_read_mmio(u64 val, u8 width, const volatile void __iomem *addr,
- unsigned long caller_addr);
+ unsigned long caller_addr, unsigned long caller_addr0);
#else
static inline void log_write_mmio(u64 val, u8 width, volatile void __iomem *addr,
- unsigned long caller_addr) {}
+ unsigned long caller_addr, unsigned long caller_addr0) {}
static inline void log_post_write_mmio(u64 val, u8 width, volatile void __iomem *addr,
- unsigned long caller_addr) {}
+ unsigned long caller_addr, unsigned long caller_addr0) {}
static inline void log_read_mmio(u8 width, const volatile void __iomem *addr,
- unsigned long caller_addr) {}
+ unsigned long caller_addr, unsigned long caller_addr0) {}
static inline void log_post_read_mmio(u64 val, u8 width, const volatile void __iomem *addr,
- unsigned long caller_addr) {}
+ unsigned long caller_addr, unsigned long caller_addr0) {}
#endif /* CONFIG_TRACE_MMIO_ACCESS */
{
u8 val;
- log_read_mmio(8, addr, _THIS_IP_);
+ log_read_mmio(8, addr, _THIS_IP_, _RET_IP_);
__io_br();
val = __raw_readb(addr);
__io_ar(val);
- log_post_read_mmio(val, 8, addr, _THIS_IP_);
+ log_post_read_mmio(val, 8, addr, _THIS_IP_, _RET_IP_);
return val;
}
#endif
{
u16 val;
- log_read_mmio(16, addr, _THIS_IP_);
+ log_read_mmio(16, addr, _THIS_IP_, _RET_IP_);
__io_br();
val = __le16_to_cpu((__le16 __force)__raw_readw(addr));
__io_ar(val);
- log_post_read_mmio(val, 16, addr, _THIS_IP_);
+ log_post_read_mmio(val, 16, addr, _THIS_IP_, _RET_IP_);
return val;
}
#endif
{
u32 val;
- log_read_mmio(32, addr, _THIS_IP_);
+ log_read_mmio(32, addr, _THIS_IP_, _RET_IP_);
__io_br();
val = __le32_to_cpu((__le32 __force)__raw_readl(addr));
__io_ar(val);
- log_post_read_mmio(val, 32, addr, _THIS_IP_);
+ log_post_read_mmio(val, 32, addr, _THIS_IP_, _RET_IP_);
return val;
}
#endif
{
u64 val;
- log_read_mmio(64, addr, _THIS_IP_);
+ log_read_mmio(64, addr, _THIS_IP_, _RET_IP_);
__io_br();
val = __le64_to_cpu(__raw_readq(addr));
__io_ar(val);
- log_post_read_mmio(val, 64, addr, _THIS_IP_);
+ log_post_read_mmio(val, 64, addr, _THIS_IP_, _RET_IP_);
return val;
}
#endif
#define writeb writeb
static inline void writeb(u8 value, volatile void __iomem *addr)
{
- log_write_mmio(value, 8, addr, _THIS_IP_);
+ log_write_mmio(value, 8, addr, _THIS_IP_, _RET_IP_);
__io_bw();
__raw_writeb(value, addr);
__io_aw();
- log_post_write_mmio(value, 8, addr, _THIS_IP_);
+ log_post_write_mmio(value, 8, addr, _THIS_IP_, _RET_IP_);
}
#endif
#define writew writew
static inline void writew(u16 value, volatile void __iomem *addr)
{
- log_write_mmio(value, 16, addr, _THIS_IP_);
+ log_write_mmio(value, 16, addr, _THIS_IP_, _RET_IP_);
__io_bw();
__raw_writew((u16 __force)cpu_to_le16(value), addr);
__io_aw();
- log_post_write_mmio(value, 16, addr, _THIS_IP_);
+ log_post_write_mmio(value, 16, addr, _THIS_IP_, _RET_IP_);
}
#endif
#define writel writel
static inline void writel(u32 value, volatile void __iomem *addr)
{
- log_write_mmio(value, 32, addr, _THIS_IP_);
+ log_write_mmio(value, 32, addr, _THIS_IP_, _RET_IP_);
__io_bw();
__raw_writel((u32 __force)__cpu_to_le32(value), addr);
__io_aw();
- log_post_write_mmio(value, 32, addr, _THIS_IP_);
+ log_post_write_mmio(value, 32, addr, _THIS_IP_, _RET_IP_);
}
#endif
#define writeq writeq
static inline void writeq(u64 value, volatile void __iomem *addr)
{
- log_write_mmio(value, 64, addr, _THIS_IP_);
+ log_write_mmio(value, 64, addr, _THIS_IP_, _RET_IP_);
__io_bw();
__raw_writeq(__cpu_to_le64(value), addr);
__io_aw();
- log_post_write_mmio(value, 64, addr, _THIS_IP_);
+ log_post_write_mmio(value, 64, addr, _THIS_IP_, _RET_IP_);
}
#endif
#endif /* CONFIG_64BIT */
{
u8 val;
- log_read_mmio(8, addr, _THIS_IP_);
+ log_read_mmio(8, addr, _THIS_IP_, _RET_IP_);
val = __raw_readb(addr);
- log_post_read_mmio(val, 8, addr, _THIS_IP_);
+ log_post_read_mmio(val, 8, addr, _THIS_IP_, _RET_IP_);
return val;
}
#endif
{
u16 val;
- log_read_mmio(16, addr, _THIS_IP_);
+ log_read_mmio(16, addr, _THIS_IP_, _RET_IP_);
val = __le16_to_cpu(__raw_readw(addr));
- log_post_read_mmio(val, 16, addr, _THIS_IP_);
+ log_post_read_mmio(val, 16, addr, _THIS_IP_, _RET_IP_);
return val;
}
#endif
{
u32 val;
- log_read_mmio(32, addr, _THIS_IP_);
+ log_read_mmio(32, addr, _THIS_IP_, _RET_IP_);
val = __le32_to_cpu(__raw_readl(addr));
- log_post_read_mmio(val, 32, addr, _THIS_IP_);
+ log_post_read_mmio(val, 32, addr, _THIS_IP_, _RET_IP_);
return val;
}
#endif
{
u64 val;
- log_read_mmio(64, addr, _THIS_IP_);
+ log_read_mmio(64, addr, _THIS_IP_, _RET_IP_);
val = __le64_to_cpu(__raw_readq(addr));
- log_post_read_mmio(val, 64, addr, _THIS_IP_);
+ log_post_read_mmio(val, 64, addr, _THIS_IP_, _RET_IP_);
return val;
}
#endif
#define writeb_relaxed writeb_relaxed
static inline void writeb_relaxed(u8 value, volatile void __iomem *addr)
{
- log_write_mmio(value, 8, addr, _THIS_IP_);
+ log_write_mmio(value, 8, addr, _THIS_IP_, _RET_IP_);
__raw_writeb(value, addr);
- log_post_write_mmio(value, 8, addr, _THIS_IP_);
+ log_post_write_mmio(value, 8, addr, _THIS_IP_, _RET_IP_);
}
#endif
#define writew_relaxed writew_relaxed
static inline void writew_relaxed(u16 value, volatile void __iomem *addr)
{
- log_write_mmio(value, 16, addr, _THIS_IP_);
+ log_write_mmio(value, 16, addr, _THIS_IP_, _RET_IP_);
__raw_writew(cpu_to_le16(value), addr);
- log_post_write_mmio(value, 16, addr, _THIS_IP_);
+ log_post_write_mmio(value, 16, addr, _THIS_IP_, _RET_IP_);
}
#endif
#define writel_relaxed writel_relaxed
static inline void writel_relaxed(u32 value, volatile void __iomem *addr)
{
- log_write_mmio(value, 32, addr, _THIS_IP_);
+ log_write_mmio(value, 32, addr, _THIS_IP_, _RET_IP_);
__raw_writel(__cpu_to_le32(value), addr);
- log_post_write_mmio(value, 32, addr, _THIS_IP_);
+ log_post_write_mmio(value, 32, addr, _THIS_IP_, _RET_IP_);
}
#endif
#define writeq_relaxed writeq_relaxed
static inline void writeq_relaxed(u64 value, volatile void __iomem *addr)
{
- log_write_mmio(value, 64, addr, _THIS_IP_);
+ log_write_mmio(value, 64, addr, _THIS_IP_, _RET_IP_);
__raw_writeq(__cpu_to_le64(value), addr);
- log_post_write_mmio(value, 64, addr, _THIS_IP_);
+ log_post_write_mmio(value, 64, addr, _THIS_IP_, _RET_IP_);
}
#endif
# endif
#endif
+#define BOUNDED_SECTION_PRE_LABEL(_sec_, _label_, _BEGIN_, _END_) \
+ _BEGIN_##_label_ = .; \
+ KEEP(*(_sec_)) \
+ _END_##_label_ = .;
+
+#define BOUNDED_SECTION_POST_LABEL(_sec_, _label_, _BEGIN_, _END_) \
+ _label_##_BEGIN_ = .; \
+ KEEP(*(_sec_)) \
+ _label_##_END_ = .;
+
+#define BOUNDED_SECTION_BY(_sec_, _label_) \
+ BOUNDED_SECTION_PRE_LABEL(_sec_, _label_, __start, __stop)
+
+#define BOUNDED_SECTION(_sec) BOUNDED_SECTION_BY(_sec, _sec)
+
+#define HEADERED_SECTION_PRE_LABEL(_sec_, _label_, _BEGIN_, _END_, _HDR_) \
+ _HDR_##_label_ = .; \
+ KEEP(*(.gnu.linkonce.##_sec_)) \
+ BOUNDED_SECTION_PRE_LABEL(_sec_, _label_, _BEGIN_, _END_)
+
+#define HEADERED_SECTION_POST_LABEL(_sec_, _label_, _BEGIN_, _END_, _HDR_) \
+ _label_##_HDR_ = .; \
+ KEEP(*(.gnu.linkonce.##_sec_)) \
+ BOUNDED_SECTION_POST_LABEL(_sec_, _label_, _BEGIN_, _END_)
+
+#define HEADERED_SECTION_BY(_sec_, _label_) \
+ HEADERED_SECTION_PRE_LABEL(_sec_, _label_, __start, __stop)
+
+#define HEADERED_SECTION(_sec) HEADERED_SECTION_BY(_sec, _sec)
+
#ifdef CONFIG_TRACE_BRANCH_PROFILING
-#define LIKELY_PROFILE() __start_annotated_branch_profile = .; \
- KEEP(*(_ftrace_annotated_branch)) \
- __stop_annotated_branch_profile = .;
+#define LIKELY_PROFILE() \
+ BOUNDED_SECTION_BY(_ftrace_annotated_branch, _annotated_branch_profile)
#else
#define LIKELY_PROFILE()
#endif
#ifdef CONFIG_PROFILE_ALL_BRANCHES
-#define BRANCH_PROFILE() __start_branch_profile = .; \
- KEEP(*(_ftrace_branch)) \
- __stop_branch_profile = .;
+#define BRANCH_PROFILE() \
+ BOUNDED_SECTION_BY(_ftrace_branch, _branch_profile)
#else
#define BRANCH_PROFILE()
#endif
#ifdef CONFIG_KPROBES
-#define KPROBE_BLACKLIST() . = ALIGN(8); \
- __start_kprobe_blacklist = .; \
- KEEP(*(_kprobe_blacklist)) \
- __stop_kprobe_blacklist = .;
+#define KPROBE_BLACKLIST() \
+ . = ALIGN(8); \
+ BOUNDED_SECTION(_kprobe_blacklist)
#else
#define KPROBE_BLACKLIST()
#endif
#ifdef CONFIG_FUNCTION_ERROR_INJECTION
-#define ERROR_INJECT_WHITELIST() STRUCT_ALIGN(); \
- __start_error_injection_whitelist = .; \
- KEEP(*(_error_injection_whitelist)) \
- __stop_error_injection_whitelist = .;
+#define ERROR_INJECT_WHITELIST() \
+ STRUCT_ALIGN(); \
+ BOUNDED_SECTION(_error_injection_whitelist)
#else
#define ERROR_INJECT_WHITELIST()
#endif
#ifdef CONFIG_EVENT_TRACING
-#define FTRACE_EVENTS() . = ALIGN(8); \
- __start_ftrace_events = .; \
- KEEP(*(_ftrace_events)) \
- __stop_ftrace_events = .; \
- __start_ftrace_eval_maps = .; \
- KEEP(*(_ftrace_eval_map)) \
- __stop_ftrace_eval_maps = .;
+#define FTRACE_EVENTS() \
+ . = ALIGN(8); \
+ BOUNDED_SECTION(_ftrace_events) \
+ BOUNDED_SECTION_BY(_ftrace_eval_map, _ftrace_eval_maps)
#else
#define FTRACE_EVENTS()
#endif
#ifdef CONFIG_TRACING
-#define TRACE_PRINTKS() __start___trace_bprintk_fmt = .; \
- KEEP(*(__trace_printk_fmt)) /* Trace_printk fmt' pointer */ \
- __stop___trace_bprintk_fmt = .;
-#define TRACEPOINT_STR() __start___tracepoint_str = .; \
- KEEP(*(__tracepoint_str)) /* Trace_printk fmt' pointer */ \
- __stop___tracepoint_str = .;
+#define TRACE_PRINTKS() BOUNDED_SECTION_BY(__trace_printk_fmt, ___trace_bprintk_fmt)
+#define TRACEPOINT_STR() BOUNDED_SECTION_BY(__tracepoint_str, ___tracepoint_str)
#else
#define TRACE_PRINTKS()
#define TRACEPOINT_STR()
#endif
#ifdef CONFIG_FTRACE_SYSCALLS
-#define TRACE_SYSCALLS() . = ALIGN(8); \
- __start_syscalls_metadata = .; \
- KEEP(*(__syscalls_metadata)) \
- __stop_syscalls_metadata = .;
+#define TRACE_SYSCALLS() \
+ . = ALIGN(8); \
+ BOUNDED_SECTION_BY(__syscalls_metadata, _syscalls_metadata)
#else
#define TRACE_SYSCALLS()
#endif
#ifdef CONFIG_BPF_EVENTS
-#define BPF_RAW_TP() STRUCT_ALIGN(); \
- __start__bpf_raw_tp = .; \
- KEEP(*(__bpf_raw_tp_map)) \
- __stop__bpf_raw_tp = .;
+#define BPF_RAW_TP() STRUCT_ALIGN(); \
+ BOUNDED_SECTION_BY(__bpf_raw_tp_map, __bpf_raw_tp)
#else
#define BPF_RAW_TP()
#endif
#ifdef CONFIG_SERIAL_EARLYCON
-#define EARLYCON_TABLE() . = ALIGN(8); \
- __earlycon_table = .; \
- KEEP(*(__earlycon_table)) \
- __earlycon_table_end = .;
+#define EARLYCON_TABLE() \
+ . = ALIGN(8); \
+ BOUNDED_SECTION_POST_LABEL(__earlycon_table, __earlycon_table, , _end)
#else
#define EARLYCON_TABLE()
#endif
#ifdef CONFIG_SECURITY
-#define LSM_TABLE() . = ALIGN(8); \
- __start_lsm_info = .; \
- KEEP(*(.lsm_info.init)) \
- __end_lsm_info = .;
-#define EARLY_LSM_TABLE() . = ALIGN(8); \
- __start_early_lsm_info = .; \
- KEEP(*(.early_lsm_info.init)) \
- __end_early_lsm_info = .;
+#define LSM_TABLE() \
+ . = ALIGN(8); \
+ BOUNDED_SECTION_PRE_LABEL(.lsm_info.init, _lsm_info, __start, __end)
+
+#define EARLY_LSM_TABLE() \
+ . = ALIGN(8); \
+ BOUNDED_SECTION_PRE_LABEL(.early_lsm_info.init, _early_lsm_info, __start, __end)
#else
#define LSM_TABLE()
#define EARLY_LSM_TABLE()
#ifdef CONFIG_ACPI
#define ACPI_PROBE_TABLE(name) \
. = ALIGN(8); \
- __##name##_acpi_probe_table = .; \
- KEEP(*(__##name##_acpi_probe_table)) \
- __##name##_acpi_probe_table_end = .;
+ BOUNDED_SECTION_POST_LABEL(__##name##_acpi_probe_table, \
+ __##name##_acpi_probe_table,, _end)
#else
#define ACPI_PROBE_TABLE(name)
#endif
#ifdef CONFIG_THERMAL
#define THERMAL_TABLE(name) \
. = ALIGN(8); \
- __##name##_thermal_table = .; \
- KEEP(*(__##name##_thermal_table)) \
- __##name##_thermal_table_end = .;
+ BOUNDED_SECTION_POST_LABEL(__##name##_thermal_table, \
+ __##name##_thermal_table,, _end)
#else
#define THERMAL_TABLE(name)
#endif
*(__tracepoints) \
/* implement dynamic printk debug */ \
. = ALIGN(8); \
- __start___dyndbg_classes = .; \
- KEEP(*(__dyndbg_classes)) \
- __stop___dyndbg_classes = .; \
- __start___dyndbg = .; \
- KEEP(*(__dyndbg)) \
- __stop___dyndbg = .; \
+ BOUNDED_SECTION_BY(__dyndbg_classes, ___dyndbg_classes) \
+ BOUNDED_SECTION_BY(__dyndbg, ___dyndbg) \
LIKELY_PROFILE() \
BRANCH_PROFILE() \
TRACE_PRINTKS() \
#define JUMP_TABLE_DATA \
. = ALIGN(8); \
- __start___jump_table = .; \
- KEEP(*(__jump_table)) \
- __stop___jump_table = .;
+ BOUNDED_SECTION_BY(__jump_table, ___jump_table)
#ifdef CONFIG_HAVE_STATIC_CALL_INLINE
#define STATIC_CALL_DATA \
. = ALIGN(8); \
- __start_static_call_sites = .; \
- KEEP(*(.static_call_sites)) \
- __stop_static_call_sites = .; \
- __start_static_call_tramp_key = .; \
- KEEP(*(.static_call_tramp_key)) \
- __stop_static_call_tramp_key = .;
+ BOUNDED_SECTION_BY(.static_call_sites, _static_call_sites) \
+ BOUNDED_SECTION_BY(.static_call_tramp_key, _static_call_tramp_key)
#else
#define STATIC_CALL_DATA
#endif
#ifdef CONFIG_ARCH_USES_CFI_TRAPS
#define KCFI_TRAPS \
__kcfi_traps : AT(ADDR(__kcfi_traps) - LOAD_OFFSET) { \
- __start___kcfi_traps = .; \
- KEEP(*(.kcfi_traps)) \
- __stop___kcfi_traps = .; \
+ BOUNDED_SECTION_BY(.kcfi_traps, ___kcfi_traps) \
}
#else
#define KCFI_TRAPS
SCHED_DATA \
RO_AFTER_INIT_DATA /* Read only after init */ \
. = ALIGN(8); \
- __start___tracepoints_ptrs = .; \
- KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
- __stop___tracepoints_ptrs = .; \
+ BOUNDED_SECTION_BY(__tracepoints_ptrs, ___tracepoints_ptrs) \
*(__tracepoints_strings)/* Tracepoints: strings */ \
} \
\
\
/* PCI quirks */ \
.pci_fixup : AT(ADDR(.pci_fixup) - LOAD_OFFSET) { \
- __start_pci_fixups_early = .; \
- KEEP(*(.pci_fixup_early)) \
- __end_pci_fixups_early = .; \
- __start_pci_fixups_header = .; \
- KEEP(*(.pci_fixup_header)) \
- __end_pci_fixups_header = .; \
- __start_pci_fixups_final = .; \
- KEEP(*(.pci_fixup_final)) \
- __end_pci_fixups_final = .; \
- __start_pci_fixups_enable = .; \
- KEEP(*(.pci_fixup_enable)) \
- __end_pci_fixups_enable = .; \
- __start_pci_fixups_resume = .; \
- KEEP(*(.pci_fixup_resume)) \
- __end_pci_fixups_resume = .; \
- __start_pci_fixups_resume_early = .; \
- KEEP(*(.pci_fixup_resume_early)) \
- __end_pci_fixups_resume_early = .; \
- __start_pci_fixups_suspend = .; \
- KEEP(*(.pci_fixup_suspend)) \
- __end_pci_fixups_suspend = .; \
- __start_pci_fixups_suspend_late = .; \
- KEEP(*(.pci_fixup_suspend_late)) \
- __end_pci_fixups_suspend_late = .; \
+ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_early, _pci_fixups_early, __start, __end) \
+ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_header, _pci_fixups_header, __start, __end) \
+ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_final, _pci_fixups_final, __start, __end) \
+ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_enable, _pci_fixups_enable, __start, __end) \
+ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_resume, _pci_fixups_resume, __start, __end) \
+ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_suspend, _pci_fixups_suspend, __start, __end) \
+ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_resume_early, _pci_fixups_resume_early, __start, __end) \
+ BOUNDED_SECTION_PRE_LABEL(.pci_fixup_suspend_late, _pci_fixups_suspend_late, __start, __end) \
} \
\
FW_LOADER_BUILT_IN_DATA \
\
/* Built-in module parameters. */ \
__param : AT(ADDR(__param) - LOAD_OFFSET) { \
- __start___param = .; \
- KEEP(*(__param)) \
- __stop___param = .; \
+ BOUNDED_SECTION_BY(__param, ___param) \
} \
\
/* Built-in module versions. */ \
__modver : AT(ADDR(__modver) - LOAD_OFFSET) { \
- __start___modver = .; \
- KEEP(*(__modver)) \
- __stop___modver = .; \
+ BOUNDED_SECTION_BY(__modver, ___modver) \
} \
\
KCFI_TRAPS \
#define EXCEPTION_TABLE(align) \
. = ALIGN(align); \
__ex_table : AT(ADDR(__ex_table) - LOAD_OFFSET) { \
- __start___ex_table = .; \
- KEEP(*(__ex_table)) \
- __stop___ex_table = .; \
+ BOUNDED_SECTION_BY(__ex_table, ___ex_table) \
}
/*
#ifdef CONFIG_DEBUG_INFO_BTF
#define BTF \
.BTF : AT(ADDR(.BTF) - LOAD_OFFSET) { \
- __start_BTF = .; \
- KEEP(*(.BTF)) \
- __stop_BTF = .; \
+ BOUNDED_SECTION_BY(.BTF, _BTF) \
} \
. = ALIGN(4); \
.BTF_ids : AT(ADDR(.BTF_ids) - LOAD_OFFSET) { \
#define BUG_TABLE \
. = ALIGN(8); \
__bug_table : AT(ADDR(__bug_table) - LOAD_OFFSET) { \
- __start___bug_table = .; \
- KEEP(*(__bug_table)) \
- __stop___bug_table = .; \
+ BOUNDED_SECTION_BY(__bug_table, ___bug_table) \
}
#else
#define BUG_TABLE
#define ORC_UNWIND_TABLE \
. = ALIGN(4); \
.orc_unwind_ip : AT(ADDR(.orc_unwind_ip) - LOAD_OFFSET) { \
- __start_orc_unwind_ip = .; \
- KEEP(*(.orc_unwind_ip)) \
- __stop_orc_unwind_ip = .; \
+ BOUNDED_SECTION_BY(.orc_unwind_ip, _orc_unwind_ip) \
} \
. = ALIGN(2); \
.orc_unwind : AT(ADDR(.orc_unwind) - LOAD_OFFSET) { \
- __start_orc_unwind = .; \
- KEEP(*(.orc_unwind)) \
- __stop_orc_unwind = .; \
+ BOUNDED_SECTION_BY(.orc_unwind, _orc_unwind) \
} \
text_size = _etext - _stext; \
. = ALIGN(4); \
#ifdef CONFIG_FW_LOADER
#define FW_LOADER_BUILT_IN_DATA \
.builtin_fw : AT(ADDR(.builtin_fw) - LOAD_OFFSET) ALIGN(8) { \
- __start_builtin_fw = .; \
- KEEP(*(.builtin_fw)) \
- __end_builtin_fw = .; \
+ BOUNDED_SECTION_PRE_LABEL(.builtin_fw, _builtin_fw, __start, __end) \
}
#else
#define FW_LOADER_BUILT_IN_DATA
#define TRACEDATA \
. = ALIGN(4); \
.tracedata : AT(ADDR(.tracedata) - LOAD_OFFSET) { \
- __tracedata_start = .; \
- KEEP(*(.tracedata)) \
- __tracedata_end = .; \
+ BOUNDED_SECTION_POST_LABEL(.tracedata, __tracedata, _start, _end) \
}
#else
#define TRACEDATA
#ifdef CONFIG_PRINTK_INDEX
#define PRINTK_INDEX \
.printk_index : AT(ADDR(.printk_index) - LOAD_OFFSET) { \
- __start_printk_index = .; \
- *(.printk_index) \
- __stop_printk_index = .; \
+ BOUNDED_SECTION_BY(.printk_index, _printk_index) \
}
#else
#define PRINTK_INDEX
#define NOTES \
.notes : AT(ADDR(.notes) - LOAD_OFFSET) { \
- __start_notes = .; \
- KEEP(*(.note.*)) \
- __stop_notes = .; \
+ BOUNDED_SECTION_BY(.note.*, _notes) \
} NOTES_HEADERS \
NOTES_HEADERS_RESTORE
#define INIT_SETUP(initsetup_align) \
. = ALIGN(initsetup_align); \
- __setup_start = .; \
- KEEP(*(.init.setup)) \
- __setup_end = .;
+ BOUNDED_SECTION_POST_LABEL(.init.setup, __setup, _start, _end)
#define INIT_CALLS_LEVEL(level) \
__initcall##level##_start = .; \
__initcall_end = .;
#define CON_INITCALL \
- __con_initcall_start = .; \
- KEEP(*(.con_initcall.init)) \
- __con_initcall_end = .;
+ BOUNDED_SECTION_POST_LABEL(.con_initcall.init, __con_initcall, _start, _end)
/* Alignment must be consistent with (kunit_suite *) in include/kunit/test.h */
#define KUNIT_TABLE() \
. = ALIGN(8); \
- __kunit_suites_start = .; \
- KEEP(*(.kunit_test_suites)) \
- __kunit_suites_end = .;
+ BOUNDED_SECTION_POST_LABEL(.kunit_test_suites, __kunit_suites, _start, _end)
#ifdef CONFIG_BLK_DEV_INITRD
#define INIT_RAM_FS \
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * Copyright (c) 2022 MediaTek Inc.
+ *
+ */
+#ifndef _DT_BINDINGS_GCE_MT8188_H
+#define _DT_BINDINGS_GCE_MT8188_H
+
+#define CMDQ_THR_PRIO_LOWEST 0
+#define CMDQ_THR_PRIO_1 1
+#define CMDQ_THR_PRIO_2 2
+#define CMDQ_THR_PRIO_3 3
+#define CMDQ_THR_PRIO_4 4
+#define CMDQ_THR_PRIO_5 5
+#define CMDQ_THR_PRIO_6 6
+#define CMDQ_THR_PRIO_HIGHEST 7
+
+#define SUBSYS_1400XXXX 0
+#define SUBSYS_1401XXXX 1
+#define SUBSYS_1402XXXX 2
+#define SUBSYS_1c00XXXX 3
+#define SUBSYS_1c01XXXX 4
+#define SUBSYS_1c02XXXX 5
+#define SUBSYS_1c10XXXX 6
+#define SUBSYS_1c11XXXX 7
+#define SUBSYS_1c12XXXX 8
+#define SUBSYS_14f0XXXX 9
+#define SUBSYS_14f1XXXX 10
+#define SUBSYS_14f2XXXX 11
+#define SUBSYS_1800XXXX 12
+#define SUBSYS_1801XXXX 13
+#define SUBSYS_1802XXXX 14
+#define SUBSYS_1803XXXX 15
+#define SUBSYS_1032XXXX 16
+#define SUBSYS_1033XXXX 17
+#define SUBSYS_1600XXXX 18
+#define SUBSYS_1601XXXX 19
+#define SUBSYS_14e0XXXX 20
+#define SUBSYS_1c20XXXX 21
+#define SUBSYS_1c30XXXX 22
+#define SUBSYS_1c40XXXX 23
+#define SUBSYS_1c50XXXX 24
+#define SUBSYS_1c60XXXX 25
+#define SUBSYS_NO_SUPPORT 99
+
+#define CMDQ_EVENT_IMG_SOF 0
+#define CMDQ_EVENT_IMG_TRAW0_CQ_THR_DONE_0 1
+#define CMDQ_EVENT_IMG_TRAW0_CQ_THR_DONE_1 2
+#define CMDQ_EVENT_IMG_TRAW0_CQ_THR_DONE_2 3
+#define CMDQ_EVENT_IMG_TRAW0_CQ_THR_DONE_3 4
+#define CMDQ_EVENT_IMG_TRAW0_CQ_THR_DONE_4 5
+#define CMDQ_EVENT_IMG_TRAW0_CQ_THR_DONE_5 6
+#define CMDQ_EVENT_IMG_TRAW0_CQ_THR_DONE_6 7
+#define CMDQ_EVENT_IMG_TRAW0_CQ_THR_DONE_7 8
+#define CMDQ_EVENT_IMG_TRAW0_CQ_THR_DONE_8 9
+#define CMDQ_EVENT_IMG_TRAW0_CQ_THR_DONE_9 10
+#define CMDQ_EVENT_IMG_TRAW0_DMA_ERROR_INT 11
+#define CMDQ_EVENT_IMG_TRAW1_CQ_THR_DONE_0 12
+#define CMDQ_EVENT_IMG_TRAW1_CQ_THR_DONE_1 13
+#define CMDQ_EVENT_IMG_TRAW1_CQ_THR_DONE_2 14
+#define CMDQ_EVENT_IMG_TRAW1_CQ_THR_DONE_3 15
+#define CMDQ_EVENT_IMG_TRAW1_CQ_THR_DONE_4 16
+#define CMDQ_EVENT_IMG_TRAW1_CQ_THR_DONE_5 17
+#define CMDQ_EVENT_IMG_TRAW1_CQ_THR_DONE_6 18
+#define CMDQ_EVENT_IMG_TRAW1_CQ_THR_DONE_7 19
+#define CMDQ_EVENT_IMG_TRAW1_CQ_THR_DONE_8 20
+#define CMDQ_EVENT_IMG_TRAW1_CQ_THR_DONE_9 21
+#define CMDQ_EVENT_IMG_TRAW1_DMA_ERROR_INT 22
+#define CMDQ_EVENT_IMG_ADL_RESERVED 23
+#define CMDQ_EVENT_IMG_DIP_CQ_THR_DONE_0 24
+#define CMDQ_EVENT_IMG_DIP_CQ_THR_DONE_1 25
+#define CMDQ_EVENT_IMG_DIP_CQ_THR_DONE_2 26
+#define CMDQ_EVENT_IMG_DIP_CQ_THR_DONE_3 27
+#define CMDQ_EVENT_IMG_DIP_CQ_THR_DONE_4 28
+#define CMDQ_EVENT_IMG_DIP_CQ_THR_DONE_5 29
+#define CMDQ_EVENT_IMG_DIP_CQ_THR_DONE_6 30
+#define CMDQ_EVENT_IMG_DIP_CQ_THR_DONE_7 31
+#define CMDQ_EVENT_IMG_DIP_CQ_THR_DONE_8 32
+#define CMDQ_EVENT_IMG_DIP_CQ_THR_DONE_9 33
+#define CMDQ_EVENT_IMG_DIP_DMA_ERR 34
+#define CMDQ_EVENT_IMG_DIP_NR_DMA_ERR 35
+#define CMDQ_EVENT_DIP_DUMMY_0 36
+#define CMDQ_EVENT_DIP_DUMMY_1 37
+#define CMDQ_EVENT_DIP_DUMMY_2 38
+#define CMDQ_EVENT_IMG_WPE_EIS_GCE_FRAME_DONE 39
+#define CMDQ_EVENT_IMG_WPE_EIS_DONE_SYNC_OUT 40
+#define CMDQ_EVENT_IMG_WPE_EIS_CQ_THR_DONE_0 41
+#define CMDQ_EVENT_IMG_WPE_EIS_CQ_THR_DONE_1 42
+#define CMDQ_EVENT_IMG_WPE_EIS_CQ_THR_DONE_2 43
+#define CMDQ_EVENT_IMG_WPE_EIS_CQ_THR_DONE_3 44
+#define CMDQ_EVENT_IMG_WPE_EIS_CQ_THR_DONE_4 45
+#define CMDQ_EVENT_IMG_WPE_EIS_CQ_THR_DONE_5 46
+#define CMDQ_EVENT_IMG_WPE_EIS_CQ_THR_DONE_6 47
+#define CMDQ_EVENT_IMG_WPE_EIS_CQ_THR_DONE_7 48
+#define CMDQ_EVENT_IMG_WPE_EIS_CQ_THR_DONE_8 49
+#define CMDQ_EVENT_IMG_WPE_EIS_CQ_THR_DONE_9 50
+#define CMDQ_EVENT_IMG_PQDIP_A_CQ_THR_DONE_0 51
+#define CMDQ_EVENT_IMG_PQDIP_A_CQ_THR_DONE_1 52
+#define CMDQ_EVENT_IMG_PQDIP_A_CQ_THR_DONE_2 53
+#define CMDQ_EVENT_IMG_PQDIP_A_CQ_THR_DONE_3 54
+#define CMDQ_EVENT_IMG_PQDIP_A_CQ_THR_DONE_4 55
+#define CMDQ_EVENT_IMG_PQDIP_A_CQ_THR_DONE_5 56
+#define CMDQ_EVENT_IMG_PQDIP_A_CQ_THR_DONE_6 57
+#define CMDQ_EVENT_IMG_PQDIP_A_CQ_THR_DONE_7 58
+#define CMDQ_EVENT_IMG_PQDIP_A_CQ_THR_DONE_8 59
+#define CMDQ_EVENT_IMG_PQDIP_A_CQ_THR_DONE_9 60
+#define CMDQ_EVENT_IMG_PQDIP_A_DMA_ERR 61
+#define CMDQ_EVENT_WPE0_DUMMY_0 62
+#define CMDQ_EVENT_WPE0_DUMMY_1 63
+#define CMDQ_EVENT_WPE0_DUMMY_2 64
+#define CMDQ_EVENT_IMG_WPE_TNR_GCE_FRAME_DONE 65
+#define CMDQ_EVENT_IMG_WPE_TNR_DONE_SYNC_OUT 66
+#define CMDQ_EVENT_IMG_WPE_TNR_CQ_THR_DONE_0 67
+#define CMDQ_EVENT_IMG_WPE_TNR_CQ_THR_DONE_1 68
+#define CMDQ_EVENT_IMG_WPE_TNR_CQ_THR_DONE_2 69
+#define CMDQ_EVENT_IMG_WPE_TNR_CQ_THR_DONE_3 70
+#define CMDQ_EVENT_IMG_WPE_TNR_CQ_THR_DONE_4 71
+#define CMDQ_EVENT_IMG_WPE_TNR_CQ_THR_DONE_5 72
+#define CMDQ_EVENT_IMG_WPE_TNR_CQ_THR_DONE_6 73
+#define CMDQ_EVENT_IMG_WPE_TNR_CQ_THR_DONE_7 74
+#define CMDQ_EVENT_IMG_WPE_TNR_CQ_THR_DONE_8 75
+#define CMDQ_EVENT_IMG_WPE_TNR_CQ_THR_DONE_9 76
+#define CMDQ_EVENT_IMG_PQDIP_B_CQ_THR_DONE_0 77
+#define CMDQ_EVENT_IMG_PQDIP_B_CQ_THR_DONE_1 78
+#define CMDQ_EVENT_IMG_PQDIP_B_CQ_THR_DONE_2 79
+#define CMDQ_EVENT_IMG_PQDIP_B_CQ_THR_DONE_3 80
+#define CMDQ_EVENT_IMG_PQDIP_B_CQ_THR_DONE_4 81
+#define CMDQ_EVENT_IMG_PQDIP_B_CQ_THR_DONE_5 82
+#define CMDQ_EVENT_IMG_PQDIP_B_CQ_THR_DONE_6 83
+#define CMDQ_EVENT_IMG_PQDIP_B_CQ_THR_DONE_7 84
+#define CMDQ_EVENT_IMG_PQDIP_B_CQ_THR_DONE_8 85
+#define CMDQ_EVENT_IMG_PQDIP_B_CQ_THR_DONE_9 86
+#define CMDQ_EVENT_IMG_PQDIP_B_DMA_ERR 87
+#define CMDQ_EVENT_WPE1_DUMMY_0 88
+#define CMDQ_EVENT_WPE1_DUMMY_1 89
+#define CMDQ_EVENT_WPE1_DUMMY_2 90
+#define CMDQ_EVENT_IMG_WPE_LITE_GCE_FRAME_DONE 91
+#define CMDQ_EVENT_IMG_WPE_LITE_DONE_SYNC_OUT 92
+#define CMDQ_EVENT_IMG_WPE_LITE_CQ_THR_DONE_0 93
+#define CMDQ_EVENT_IMG_WPE_LITE_CQ_THR_DONE_1 94
+#define CMDQ_EVENT_IMG_WPE_LITE_CQ_THR_DONE_2 95
+#define CMDQ_EVENT_IMG_WPE_LITE_CQ_THR_DONE_3 96
+#define CMDQ_EVENT_IMG_WPE_LITE_CQ_THR_DONE_4 97
+#define CMDQ_EVENT_IMG_WPE_LITE_CQ_THR_DONE_5 98
+#define CMDQ_EVENT_IMG_WPE_LITE_CQ_THR_DONE_6 99
+#define CMDQ_EVENT_IMG_WPE_LITE_CQ_THR_DONE_7 100
+#define CMDQ_EVENT_IMG_WPE_LITE_CQ_THR_DONE_8 101
+#define CMDQ_EVENT_IMG_WPE_LITE_CQ_THR_DONE_9 102
+#define CMDQ_EVENT_IMG_XTRAW_CQ_THR_DONE_0 103
+#define CMDQ_EVENT_IMG_XTRAW_CQ_THR_DONE_1 104
+#define CMDQ_EVENT_IMG_XTRAW_CQ_THR_DONE_2 105
+#define CMDQ_EVENT_IMG_XTRAW_CQ_THR_DONE_3 106
+#define CMDQ_EVENT_IMG_XTRAW_CQ_THR_DONE_4 107
+#define CMDQ_EVENT_IMG_XTRAW_CQ_THR_DONE_5 108
+#define CMDQ_EVENT_IMG_XTRAW_CQ_THR_DONE_6 109
+#define CMDQ_EVENT_IMG_XTRAW_CQ_THR_DONE_7 110
+#define CMDQ_EVENT_IMG_XTRAW_CQ_THR_DONE_8 111
+#define CMDQ_EVENT_IMG_XTRAW_CQ_THR_DONE_9 112
+#define CMDQ_EVENT_IMG_XTRAW_DMA_ERR_EVENT 113
+#define CMDQ_EVENT_WPE2_DUMMY_0 114
+#define CMDQ_EVENT_WPE2_DUMMY_1 115
+#define CMDQ_EVENT_WPE2_DUMMY_2 116
+#define CMDQ_EVENT_IMG_IMGSYS_IPE_DUMMY 117
+#define CMDQ_EVENT_IMG_IMGSYS_IPE_FDVT_DONE 118
+#define CMDQ_EVENT_IMG_IMGSYS_IPE_ME_DONE 119
+#define CMDQ_EVENT_IMG_IMGSYS_IPE_DVS_DONE 120
+#define CMDQ_EVENT_IMG_IMGSYS_IPE_DVP_DONE 121
+#define CMDQ_EVENT_FDVT1_RESERVED 122
+#define CMDQ_EVENT_IMG_ENG_EVENT 123
+#define CMDQ_EVENT_CAMSUBA_SW_PASS1_DONE 129
+#define CMDQ_EVENT_CAMSUBB_SW_PASS1_DONE 130
+#define CMDQ_EVENT_CAMSUBC_SW_PASS1_DONE 131
+#define CMDQ_EVENT_GCAMSV_A_1_SW_PASS1_DONE 132
+#define CMDQ_EVENT_GCAMSV_A_2_SW_PASS1_DONE 133
+#define CMDQ_EVENT_GCAMSV_B_1_SW_PASS1_DONE 134
+#define CMDQ_EVENT_GCAMSV_B_2_SW_PASS1_DONE 135
+#define CMDQ_EVENT_GCAMSV_C_1_SW_PASS1_DONE 136
+#define CMDQ_EVENT_GCAMSV_C_2_SW_PASS1_DONE 137
+#define CMDQ_EVENT_GCAMSV_D_1_SW_PASS1_DONE 138
+#define CMDQ_EVENT_GCAMSV_D_2_SW_PASS1_DONE 139
+#define CMDQ_EVENT_GCAMSV_E_1_SW_PASS1_DONE 140
+#define CMDQ_EVENT_GCAMSV_E_2_SW_PASS1_DONE 141
+#define CMDQ_EVENT_GCAMSV_F_1_SW_PASS1_DONE 142
+#define CMDQ_EVENT_GCAMSV_F_2_SW_PASS1_DONE 143
+#define CMDQ_EVENT_GCAMSV_G_1_SW_PASS1_DONE 144
+#define CMDQ_EVENT_GCAMSV_G_2_SW_PASS1_DONE 145
+#define CMDQ_EVENT_GCAMSV_H_1_SW_PASS1_DONE 146
+#define CMDQ_EVENT_GCAMSV_H_2_SW_PASS1_DONE 147
+#define CMDQ_EVENT_GCAMSV_I_1_SW_PASS1_DONE 148
+#define CMDQ_EVENT_GCAMSV_I_2_SW_PASS1_DONE 149
+#define CMDQ_EVENT_GCAMSV_J_1_SW_PASS1_DONE 150
+#define CMDQ_EVENT_GCAMSV_J_2_SW_PASS1_DONE 151
+#define CMDQ_EVENT_MRAW_0_SW_PASS1_DONE 152
+#define CMDQ_EVENT_MRAW_1_SW_PASS1_DONE 153
+#define CMDQ_EVENT_MRAW_2_SW_PASS1_DONE 154
+#define CMDQ_EVENT_MRAW_3_SW_PASS1_DONE 155
+#define CMDQ_EVENT_SENINF_CAM0_FIFO_FULL 156
+#define CMDQ_EVENT_SENINF_CAM1_FIFO_FULL 157
+#define CMDQ_EVENT_SENINF_CAM2_FIFO_FULL 158
+#define CMDQ_EVENT_SENINF_CAM3_FIFO_FULL 159
+#define CMDQ_EVENT_SENINF_CAM4_FIFO_FULL 160
+#define CMDQ_EVENT_SENINF_CAM5_FIFO_FULL 161
+#define CMDQ_EVENT_SENINF_CAM6_FIFO_FULL 162
+#define CMDQ_EVENT_SENINF_CAM7_FIFO_FULL 163
+#define CMDQ_EVENT_SENINF_CAM8_FIFO_FULL 164
+#define CMDQ_EVENT_SENINF_CAM9_FIFO_FULL 165
+#define CMDQ_EVENT_SENINF_CAM10_FIFO_FULL 166
+#define CMDQ_EVENT_SENINF_CAM11_FIFO_FULL 167
+#define CMDQ_EVENT_SENINF_CAM12_FIFO_FULL 168
+#define CMDQ_EVENT_SENINF_CAM13_FIFO_FULL 169
+#define CMDQ_EVENT_SENINF_CAM14_FIFO_FULL 170
+#define CMDQ_EVENT_SENINF_CAM15_FIFO_FULL 171
+#define CMDQ_EVENT_SENINF_CAM16_FIFO_FULL 172
+#define CMDQ_EVENT_SENINF_CAM17_FIFO_FULL 173
+#define CMDQ_EVENT_SENINF_CAM18_FIFO_FULL 174
+#define CMDQ_EVENT_SENINF_CAM19_FIFO_FULL 175
+#define CMDQ_EVENT_SENINF_CAM20_FIFO_FULL 176
+#define CMDQ_EVENT_SENINF_CAM21_FIFO_FULL 177
+#define CMDQ_EVENT_SENINF_CAM22_FIFO_FULL 178
+#define CMDQ_EVENT_SENINF_CAM23_FIFO_FULL 179
+#define CMDQ_EVENT_SENINF_CAM24_FIFO_FULL 180
+#define CMDQ_EVENT_SENINF_CAM25_FIFO_FULL 181
+#define CMDQ_EVENT_SENINF_CAM26_FIFO_FULL 182
+#define CMDQ_EVENT_TG_OVRUN_MRAW0_INT 183
+#define CMDQ_EVENT_TG_OVRUN_MRAW1_INT 184
+#define CMDQ_EVENT_TG_OVRUN_MRAW2_INT 185
+#define CMDQ_EVENT_TG_OVRUN_MRAW3_INT 186
+#define CMDQ_EVENT_DMA_R1_ERROR_MRAW0_INT 187
+#define CMDQ_EVENT_DMA_R1_ERROR_MRAW1_INT 188
+#define CMDQ_EVENT_DMA_R1_ERROR_MRAW2_INT 189
+#define CMDQ_EVENT_DMA_R1_ERROR_MRAW3_INT 190
+#define CMDQ_EVENT_PDA0_IRQO_EVENT_DONE_D1 191
+#define CMDQ_EVENT_PDA1_IRQO_EVENT_DONE_D1 192
+#define CMDQ_EVENT_CAM_SUBA_TG_INT1 193
+#define CMDQ_EVENT_CAM_SUBA_TG_INT2 194
+#define CMDQ_EVENT_CAM_SUBA_TG_INT3 195
+#define CMDQ_EVENT_CAM_SUBA_TG_INT4 196
+#define CMDQ_EVENT_CAM_SUBB_TG_INT1 197
+#define CMDQ_EVENT_CAM_SUBB_TG_INT2 198
+#define CMDQ_EVENT_CAM_SUBB_TG_INT3 199
+#define CMDQ_EVENT_CAM_SUBB_TG_INT4 200
+#define CMDQ_EVENT_CAM_SUBC_TG_INT1 201
+#define CMDQ_EVENT_CAM_SUBC_TG_INT2 202
+#define CMDQ_EVENT_CAM_SUBC_TG_INT3 203
+#define CMDQ_EVENT_CAM_SUBC_TG_INT4 204
+#define CMDQ_EVENT_CAM_SUBA_IMGO_R1_LOW_LATENCY_LINE_CNT_INT 205
+#define CMDQ_EVENT_CAM_SUBA_YUVO_R1_LOW_LATENCY_LINE_CNT_INT 206
+#define CMDQ_EVENT_CAM_SUBA_YUVO_R3_LOW_LATENCY_LINE_CNT_INT 207
+#define CMDQ_EVENT_CAM_SUBA_DRZS4NO_R1_LOW_LATENCY_LINE_CNT_INT 208
+#define CMDQ_EVENT_CAM_SUBB_IMGO_R1_LOW_LATENCY_LINE_CNT_INT 209
+#define CMDQ_EVENT_CAM_SUBB_YUVO_R1_LOW_LATENCY_LINE_CNT_INT 210
+#define CMDQ_EVENT_CAM_SUBB_YUVO_R3_LOW_LATENCY_LINE_CNT_INT 211
+#define CMDQ_EVENT_CAM_SUBB_DRZS4NO_R1_LOW_LATENCY_LINE_CNT_INT 212
+#define CMDQ_EVENT_CAM_SUBC_IMGO_R1_LOW_LATENCY_LINE_CNT_INT 213
+#define CMDQ_EVENT_CAM_SUBC_YUVO_R1_LOW_LATENCY_LINE_CNT_INT 214
+#define CMDQ_EVENT_CAM_SUBC_YUVO_R3_LOW_LATENCY_LINE_CNT_INT 215
+#define CMDQ_EVENT_CAM_SUBC_DRZS4NO_R1_LOW_LATENCY_LINE_CNT_INT 216
+#define CMDQ_EVENT_RAW_SEL_SOF_SUBA 217
+#define CMDQ_EVENT_RAW_SEL_SOF_SUBB 218
+#define CMDQ_EVENT_RAW_SEL_SOF_SUBC 219
+#define CMDQ_EVENT_CAM_SUBA_RING_BUFFER_OVERFLOW_INT_IN 220
+#define CMDQ_EVENT_CAM_SUBB_RING_BUFFER_OVERFLOW_INT_IN 221
+#define CMDQ_EVENT_CAM_SUBC_RING_BUFFER_OVERFLOW_INT_IN 222
+#define CMDQ_EVENT_VPP0_MDP_RDMA_SOF 256
+#define CMDQ_EVENT_VPP0_MDP_FG_SOF 257
+#define CMDQ_EVENT_VPP0_STITCH_SOF 258
+#define CMDQ_EVENT_VPP0_MDP_HDR_SOF 259
+#define CMDQ_EVENT_VPP0_MDP_AAL_SOF 260
+#define CMDQ_EVENT_VPP0_MDP_RSZ_IN_RSZ_SOF 261
+#define CMDQ_EVENT_VPP0_MDP_TDSHP_SOF 262
+#define CMDQ_EVENT_VPP0_DISP_COLOR_SOF 263
+#define CMDQ_EVENT_VPP0_DISP_OVL_NOAFBC_SOF 264
+#define CMDQ_EVENT_VPP0_VPP_PADDING_IN_PADDING_SOF 265
+#define CMDQ_EVENT_VPP0_MDP_TCC_IN_SOF 266
+#define CMDQ_EVENT_VPP0_MDP_WROT_SOF 267
+#define CMDQ_EVENT_VPP0_WARP0_MMSYS_TOP_RELAY_SOF_PRE 269
+#define CMDQ_EVENT_VPP0_WARP1_MMSYS_TOP_RELAY_SOF_PRE 270
+#define CMDQ_EVENT_VPP0_VPP1_MMSYS_TOP_RELAY_SOF 271
+#define CMDQ_EVENT_VPP0_VPP1_IN_MMSYS_TOP_RELAY_SOF_PRE 272
+#define CMDQ_EVENT_VPP0_DISP_RDMA_SOF 273
+#define CMDQ_EVENT_VPP0_DISP_WDMA_SOF 274
+#define CMDQ_EVENT_VPP0_MDP_HMS_SOF 275
+#define CMDQ_EVENT_VPP0_MDP_RDMA_FRAME_DONE 288
+#define CMDQ_EVENT_VPP0_MDP_FG_TILE_DONE 289
+#define CMDQ_EVENT_VPP0_STITCH_FRAME_DONE 290
+#define CMDQ_EVENT_VPP0_MDP_HDR_FRAME_DONE 291
+#define CMDQ_EVENT_VPP0_MDP_AAL_FRAME_DONE 292
+#define CMDQ_EVENT_VPP0_MDP_RSZ_FRAME_DONE 293
+#define CMDQ_EVENT_VPP0_MDP_TDSHP_FRAME_DONE 294
+#define CMDQ_EVENT_VPP0_DISP_COLOR_FRAME_DONE 295
+#define CMDQ_EVENT_VPP0_DISP_OVL_NOAFBC_FRAME_DONE 296
+#define CMDQ_EVENT_VPP0_VPP_PADDING_IN_PADDING_FRAME_DONE 297
+#define CMDQ_EVENT_VPP0_MDP_TCC_TCC_FRAME_DONE 298
+#define CMDQ_EVENT_VPP0_MDP_WROT_VIDO_WDONE 299
+#define CMDQ_EVENT_VPP0_DISP_RDMA_FRAME_DONE 305
+#define CMDQ_EVENT_VPP0_DISP_WDMA_FRAME_DONE 306
+#define CMDQ_EVENT_VPP0_MDP_HMS_FRAME_DONE 307
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_0 320
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_1 321
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_2 322
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_3 323
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_4 324
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_5 325
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_6 326
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_7 327
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_8 328
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_9 329
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_10 330
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_11 331
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_12 332
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_13 333
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_14 334
+#define CMDQ_EVENT_VPP0_DISP_MUTEX_STREAM_DONE_15 335
+#define CMDQ_EVENT_VPP0_DISP_RDMA_0_UNDERRUN 336
+#define CMDQ_EVENT_VPP0_DISP_RDMA_1_UNDERRUN 337
+#define CMDQ_EVENT_VPP0_U_MERGE4_UNDERRUN 338
+#define CMDQ_EVENT_VPP0_U_VPP_SPLIT_VIDEO_0_OVERFLOW 339
+#define CMDQ_EVENT_VPP0_U_VPP_SPLIT_VIDEO_1_OVERFLOW 340
+#define CMDQ_EVENT_VPP0_DSI_0_UNDERRUN 341
+#define CMDQ_EVENT_VPP0_DSI_1_UNDERRUN 342
+#define CMDQ_EVENT_VPP0_DP_INTF_0 343
+#define CMDQ_EVENT_VPP0_DP_INTF_1 344
+#define CMDQ_EVENT_VPP0_DPI_0 345
+#define CMDQ_EVENT_VPP0_DPI_1 346
+#define CMDQ_EVENT_VPP0_MDP_RDMA_SW_RST_DONE 352
+#define CMDQ_EVENT_VPP0_MDP_RDMA_PM_VALID_EVENT 353
+#define CMDQ_EVENT_VPP0_DISP_OVL_NOAFBC_FRAME_RESET_DONE_PULSE 354
+#define CMDQ_EVENT_VPP0_MDP_WROT_SW_RST_DONE 355
+#define CMDQ_EVENT_VPP0_DISP_OVL_NOAFBC_TARGET_MATCH_0 356
+#define CMDQ_EVENT_VPP0_DISP_OVL_NOAFBC_TARGET_MATCH_1 357
+#define CMDQ_EVENT_VPP0_DISP_OVL_NOAFBC_TARGET_MATCH_2 358
+#define CMDQ_EVENT_VPP0_DISP_OVL_NOAFBC_TARGET_MATCH_3 359
+#define CMDQ_EVENT_VPP0_DISP_OVL_NOAFBC_TARGET_MATCH_4 360
+#define CMDQ_EVENT_VPP0_DISP_OVL_NOAFBC_TARGET_MATCH_5 361
+#define CMDQ_EVENT_VPP0_DISP_OVL_NOAFBC_TARGET_MATCH_6 362
+#define CMDQ_EVENT_VPP0_DISP_RDMA_DISP_RDMA_VALID_EVENT 363
+#define CMDQ_EVENT_VPP0_DISP_RDMA_DISP_RDMA_TARGET_LINE_EVENT 364
+#define CMDQ_EVENT_VPP0_DISP_WDMA_SW_RST_DONE 365
+#define CMDQ_EVENT_VPP0_DISP_WDMA_WDMA_VALID_EVENT 366
+#define CMDQ_EVENT_VPP0_DISP_WDMA_WDMA_TARGET_LINE_EVENT 367
+#define CMDQ_EVENT_VPP1_HDMI_META_SOF 384
+#define CMDQ_EVENT_VPP1_DGI_SOF 385
+#define CMDQ_EVENT_VPP1_VPP_SPLIT_SOF 386
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_TCC_SOF 387
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_RDMA_SOF 388
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_RDMA_SOF 389
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_RDMA_SOF 390
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_FG_SOF 391
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_FG_SOF 392
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_FG_SOF 393
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_HDR_SOF 394
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_HDR_SOF 395
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_HDR_SOF 396
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_AAL_SOF 397
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_AAL_SOF 398
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_AAL_SOF 399
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_RSZ_SOF 400
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_RSZ_SOF 401
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_RSZ_SOF 402
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_TDSHP_SOF 403
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_TDSHP_SOF 404
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_TDSHP_SOF 405
+#define CMDQ_EVENT_VPP1_SVPP2_VPP_MERGE_SOF 406
+#define CMDQ_EVENT_VPP1_SVPP3_VPP_MERGE_SOF 407
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_COLOR_SOF 408
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_COLOR_SOF 409
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_COLOR_SOF 410
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_OVL_SOF 411
+#define CMDQ_EVENT_VPP1_SVPP1_VPP_PAD_SOF 412
+#define CMDQ_EVENT_VPP1_SVPP2_VPP_PAD_SOF 413
+#define CMDQ_EVENT_VPP1_SVPP3_VPP_PAD_SOF 414
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_WROT_SOF 415
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_WROT_SOF 416
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_WROT_SOF 417
+#define CMDQ_EVENT_VPP1_VPP0_DL_IRLY_SOF 418
+#define CMDQ_EVENT_VPP1_VPP0_DL_ORLY_SOF 419
+#define CMDQ_EVENT_VPP1_VDO0_DL_ORLY_0_SOF 420
+#define CMDQ_EVENT_VPP1_VDO0_DL_ORLY_1_SOF 421
+#define CMDQ_EVENT_VPP1_VDO1_DL_ORLY_0_SOF 422
+#define CMDQ_EVENT_VPP1_VDO1_DL_ORLY_1_SOF 423
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_RDMA_FRAME_DONE 424
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_RDMA_FRAME_DONE 425
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_RDMA_FRAME_DONE 426
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_WROT_FRAME_DONE 427
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_WROT_FRAME_DONE 428
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_WROT_FRAME_DONE 429
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_OVL_FRAME_DONE 430
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_RSZ_FRAME_DONE 431
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_RSZ_FRAME_DONE 432
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_RSZ_FRAME_DONE 433
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_FG_TILE_DONE 434
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_FG_TILE_DONE 435
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_FG_TILE_DONE 436
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_HDR_FRAME_DONE 437
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_HDR_FRAME_DONE 438
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_HDR_FRAME_DONE 439
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_AAL_FRAME_DONE 440
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_AAL_FRAME_DONE 441
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_AAL_FRAME_DONE 442
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_TDSHP_FRAME_DONE 443
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_TDSHP_FRAME_DONE 444
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_TDSHP_FRAME_DONE 445
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_COLOR_FRAME_DONE 446
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_COLOR_FRAME_DONE 447
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_COLOR_FRAME_DONE 448
+#define CMDQ_EVENT_VPP1_SVPP1_VPP_PAD_FRAME_DONE 449
+#define CMDQ_EVENT_VPP1_SVPP2_VPP_PAD_FRAME_DONE 450
+#define CMDQ_EVENT_VPP1_SVPP3_VPP_PAD_FRAME_DONE 451
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_TCC_FRAME_DONE 452
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_0 456
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_1 457
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_2 458
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_3 459
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_4 460
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_5 461
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_6 462
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_7 463
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_8 464
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_9 465
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_10 466
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_11 467
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_12 468
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_13 469
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_14 470
+#define CMDQ_EVENT_VPP1_MUTEX_STREAM_DONE_GCE_EVENT_15 471
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_0 472
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_1 473
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_2 474
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_3 475
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_4 476
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_5 477
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_6 478
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_7 479
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_8 480
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_9 481
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_10 482
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_11 483
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_12 484
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_13 485
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_14 486
+#define CMDQ_EVENT_VPP1_MUTEX_BUF_UNDERRUN_GCE_EVENT_15 487
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_0 488
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_1 489
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_2 490
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_3 491
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_4 492
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_5 493
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_6 494
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_7 495
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_8 496
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_9 497
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_10 498
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_11 499
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_12 500
+#define CMDQ_EVENT_VPP1_DGI_SYNC_EVENT_13 501
+#define CMDQ_EVENT_VPP1_SVPP3_VPP_MERGE_GCE_EVENT 502
+#define CMDQ_EVENT_VPP1_SVPP2_VPP_MERGE_GCE_EVENT 503
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_OVL_GCE_EVENT 504
+#define CMDQ_EVENT_VPP1_VPP_SPLIT_DGI_GCE_EVENT 505
+#define CMDQ_EVENT_VPP1_VPP_SPLIT_HDMI_GCE_EVENT 506
+#define CMDQ_EVENT_VPP1_SVPP3_MDP_WROT_SW_RST_DONE_GCE_EVENT 507
+#define CMDQ_EVENT_VPP1_SVPP2_MDP_WROT_SW_RST_DONE_GCE_EVENT 508
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_WROT_SW_RST_DONE_GCE_EVENT 509
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_OVL_NEW_EVENT_0 510
+#define CMDQ_EVENT_VPP1_SVPP1_MDP_OVL_NEW_EVENT_1 511
+#define CMDQ_EVENT_VDO0_DISP_OVL0_SOF 512
+#define CMDQ_EVENT_VDO0_DISP_WDMA0_SOF 513
+#define CMDQ_EVENT_VDO0_DISP_RDMA0_SOF 514
+#define CMDQ_EVENT_VDO0_DISP_COLOR0_SOF 515
+#define CMDQ_EVENT_VDO0_DISP_CCORR0_SOF 516
+#define CMDQ_EVENT_VDO0_DISP_AAL0_SOF 517
+#define CMDQ_EVENT_VDO0_DISP_GAMMA0_SOF 518
+#define CMDQ_EVENT_VDO0_DISP_DITHER0_SOF 519
+#define CMDQ_EVENT_VDO0_DSI0_SOF 520
+#define CMDQ_EVENT_VDO0_DSC_WRAP0C0_SOF 521
+#define CMDQ_EVENT_VDO0_DISP_OVL1_SOF 522
+#define CMDQ_EVENT_VDO0_DISP_WDMA1_SOF 523
+#define CMDQ_EVENT_VDO0_DISP_RDMA1_SOF 524
+#define CMDQ_EVENT_VDO0_DISP_COLOR1_SOF 525
+#define CMDQ_EVENT_VDO0_DISP_CCORR1_SOF 526
+#define CMDQ_EVENT_VDO0_DISP_AAL1_SOF 527
+#define CMDQ_EVENT_VDO0_DISP_GAMMA1_SOF 528
+#define CMDQ_EVENT_VDO0_DISP_DITHER1_SOF 529
+#define CMDQ_EVENT_VDO0_DSI1_SOF 530
+#define CMDQ_EVENT_VDO0_DSC_WRAP0C1_SOF 531
+#define CMDQ_EVENT_VDO0_VPP_MERGE0_SOF 532
+#define CMDQ_EVENT_VDO0_DP_INTF0_SOF 533
+#define CMDQ_EVENT_VDO0_DISP_DPI0_SOF 534
+#define CMDQ_EVENT_VDO0_DISP_DPI1_SOF 535
+#define CMDQ_EVENT_VDO0_DISP_POSTMASK0_SOF 536
+#define CMDQ_EVENT_VDO0_MDP_WROT0_SOF 537
+#define CMDQ_EVENT_VDO0_DISP_RSZ0_SOF 538
+#define CMDQ_EVENT_VDO0_VPP1_DL_RELAY0_SOF 539
+#define CMDQ_EVENT_VDO0_VPP1_DL_RELAY1_SOF 540
+#define CMDQ_EVENT_VDO0_VDO1_DL_RELAY2_SOF 541
+#define CMDQ_EVENT_VDO0_VDO0_DL_RELAY3_SOF 542
+#define CMDQ_EVENT_VDO0_VDO0_DL_RELAY4_SOF 543
+#define CMDQ_EVENT_VDO0_DISP_PWM0_SOF 544
+#define CMDQ_EVENT_VDO0_DISP_PWM1_SOF 545
+#define CMDQ_EVENT_VDO0_DISP_OVL0_O_FRAME_DONE 546
+#define CMDQ_EVENT_VDO0_DISP_WDMA0_FRAME_DONE 547
+#define CMDQ_EVENT_VDO0_DISP_RDMA0_FRAME_DONE 548
+#define CMDQ_EVENT_VDO0_DISP_COLOR0_O_FRAME_DONE 549
+#define CMDQ_EVENT_VDO0_DISP_CCORR0_O_FRAME_DONE 550
+#define CMDQ_EVENT_VDO0_DISP_AAL0_O_FRAME_DONE 551
+#define CMDQ_EVENT_VDO0_DISP_GAMMA0_O_FRAME_DONE 552
+#define CMDQ_EVENT_VDO0_DISP_DITHER0_O_FRAME_DONE 553
+#define CMDQ_EVENT_VDO0_DSI0_FRAME_DONE 554
+#define CMDQ_EVENT_VDO0_DSC_WRAP0_O_FRAME_DONE_0 555
+#define CMDQ_EVENT_VDO0_DISP_OVL1_O_FRAME_DONE 556
+#define CMDQ_EVENT_VDO0_DISP_WDMA1_O_FRAME_DONE 557
+#define CMDQ_EVENT_VDO0_DISP_RDMA1_O_FRAME_DONE 558
+#define CMDQ_EVENT_VDO0_DISP_COLOR1_O_FRAME_DONE 559
+#define CMDQ_EVENT_VDO0_DISP_CCORR1_O_FRAME_DONE 560
+#define CMDQ_EVENT_VDO0_DISP_AAL1_O_FRAME_DONE 561
+#define CMDQ_EVENT_VDO0_DISP_GAMMA1_O_FRAME_DONE 562
+#define CMDQ_EVENT_VDO0_DISP_DITHER1_O_FRAME_DONE 563
+#define CMDQ_EVENT_VDO0_DSI1_FRAME_DONE 564
+#define CMDQ_EVENT_VDO0_DSC_WRAP0_O_FRAME_DONE_1 565
+#define CMDQ_EVENT_VDO0_DP_INTF0_FRAME_DONE 567
+#define CMDQ_EVENT_VDO0_DISP_DPI0_O_FRAME_DONE 568
+#define CMDQ_EVENT_VDO0_DISP_DPI1_O_FRAME_DONE 569
+#define CMDQ_EVENT_VDO0_DISP_POSTMASK0_O_FRAME_DONE 570
+#define CMDQ_EVENT_VDO0_MDP_WROT0_O_FRAME_DONE 571
+#define CMDQ_EVENT_VDO0_DISP_RSZ0_O_FRAME_DONE 572
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_0 574
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_1 575
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_2 576
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_3 577
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_4 578
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_5 579
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_6 580
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_7 581
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_8 582
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_9 583
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_10 584
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_11 585
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_12 586
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_13 587
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_14 588
+#define CMDQ_EVENT_VDO0_DISP_STREAM_DONE_15 589
+#define CMDQ_EVENT_VDO0_DISP_RDMA_0_UNDERRUN 590
+#define CMDQ_EVENT_VDO0_DISP_RDMA_1_UNDERRUN 591
+#define CMDQ_EVENT_VDO0_U_MERGE4_UNDERRUN 592
+#define CMDQ_EVENT_VDO0_DSI_0_UNDERRUN 595
+#define CMDQ_EVENT_VDO0_DSI_1_UNDERRUN 596
+#define CMDQ_EVENT_VDO0_DP_INTF_0 597
+#define CMDQ_EVENT_VDO0_DP_INTF_1 598
+#define CMDQ_EVENT_VDO0_DPI_0 599
+#define CMDQ_EVENT_VDO0_DPI_1 600
+#define CMDQ_EVENT_VDO0_DISP_SMIASSERT_ENG_EVENT 606
+#define CMDQ_EVENT_VDO0_DSI0_O_DSI_IRQ_EVENT_MM 607
+#define CMDQ_EVENT_VDO0_DSI0_TE_ENG_EVENT_MM 608
+#define CMDQ_EVENT_VDO0_DSI0_O_DSI_DONE_EVENT_MM 609
+#define CMDQ_EVENT_VDO0_DSI0_O_DSI_VACTL_EVENT_MM 610
+#define CMDQ_EVENT_VDO0_DSI1_O_DSI_IRQ_EVENT_MM 611
+#define CMDQ_EVENT_VDO0_DSI1_TE_ENG_EVENT_MM 612
+#define CMDQ_EVENT_VDO0_DSI1_O_DSI_DONE_EVENT_MM 613
+#define CMDQ_EVENT_VDO0_DSI1_O_DSI_VACTL_EVENT_MM 614
+#define CMDQ_EVENT_VDO0_DP_INTF0_O_DP_VSYNC_START_EVENT_MM 615
+#define CMDQ_EVENT_VDO0_DP_INTF0_O_DP_VSYNC_END_EVENT_MM 616
+#define CMDQ_EVENT_VDO0_DP_INTF0_O_DP_VDE_START_EVENT_MM 617
+#define CMDQ_EVENT_VDO0_DP_INTF0_O_DP_VDE_END_EVENT_MM 618
+#define CMDQ_EVENT_VDO0_DP_INTF0_O_VACT_TARGET_LINE_EVENT_MM 619
+#define CMDQ_EVENT_VDO0_DP_INTF0_O_LAST_SAFE_BLANK_EVENT_MM 620
+#define CMDQ_EVENT_VDO0_DP_INTF0_O_LAST_LINE_EVENT_MM 621
+#define CMDQ_EVENT_VDO0_DP_INTF0_O_TRIGGER_LOOP_CLEAR_EVENT_MM 622
+#define CMDQ_EVENT_VDO0_DP_INTF0_O_TARGET_LINE_0_EVENT_MM 623
+#define CMDQ_EVENT_VDO0_DP_INTF0_O_TARGET_LINE_1_EVENT_MM 624
+#define CMDQ_EVENT_VDO0_DISP_POSTMASK0_O_FRAME_RESET_DONE_PULSE 625
+#define CMDQ_EVENT_VDO0_VPP_MERGE0_O_VPP_MERGE_EVENT 626
+#define CMDQ_EVENT_VDO0_DISP_OVL0_O_FRAME_RESET_DONE_PULSE 627
+#define CMDQ_EVENT_VDO0_DISP_RDMA0_O_DISP_RDMA_TARGET_LINE_EVENT 628
+#define CMDQ_EVENT_VDO0_DISP_WDMA0_O_WDMA_TARGET_LINE_EVENT 629
+#define CMDQ_EVENT_VDO0_DISP_WDMA0_O_SW_RST_DONE 630
+#define CMDQ_EVENT_VDO0_DISP_OVL0_O_TARGET_MATCH_EVENT_0 631
+#define CMDQ_EVENT_VDO0_DISP_OVL0_O_TARGET_MATCH_EVENT_1 632
+#define CMDQ_EVENT_VDO0_DISP_OVL0_O_TARGET_MATCH_EVENT_2 633
+#define CMDQ_EVENT_VDO0_DISP_OVL0_O_TARGET_MATCH_EVENT_3 634
+#define CMDQ_EVENT_VDO0_DISP_OVL0_O_TARGET_MATCH_EVENT_4 635
+#define CMDQ_EVENT_VDO0_DISP_OVL0_O_TARGET_MATCH_EVENT_5 636
+#define CMDQ_EVENT_VDO0_DISP_OVL0_O_TARGET_MATCH_EVENT_6 637
+#define CMDQ_EVENT_VDO0_MDP_WROT0_O_SW_RST_DONE 638
+#define CMDQ_EVENT_VDO0_RESERVED 639
+#define CMDQ_EVENT_VDO1_MDP_RDMA0_SOF 640
+#define CMDQ_EVENT_VDO1_MDP_RDMA1_SOF 641
+#define CMDQ_EVENT_VDO1_MDP_RDMA2_SOF 642
+#define CMDQ_EVENT_VDO1_MDP_RDMA3_SOF 643
+#define CMDQ_EVENT_VDO1_MDP_RDMA4_SOF 644
+#define CMDQ_EVENT_VDO1_MDP_RDMA5_SOF 645
+#define CMDQ_EVENT_VDO1_MDP_RDMA6_SOF 646
+#define CMDQ_EVENT_VDO1_MDP_RDMA7_SOF 647
+#define CMDQ_EVENT_VDO1_DISP_PADDING0_SOF 648
+#define CMDQ_EVENT_VDO1_DISP_PADDING1_SOF 649
+#define CMDQ_EVENT_VDO1_DISP_PADDING2_SOF 650
+#define CMDQ_EVENT_VDO1_DISP_PADDING3_SOF 651
+#define CMDQ_EVENT_VDO1_DISP_PADDING4_SOF 652
+#define CMDQ_EVENT_VDO1_DISP_PADDING5_SOF 653
+#define CMDQ_EVENT_VDO1_DISP_PADDING6_SOF 654
+#define CMDQ_EVENT_VDO1_DISP_PADDING7_SOF 655
+#define CMDQ_EVENT_VDO1_DISP_RSZ0_SOF 656
+#define CMDQ_EVENT_VDO1_DISP_RSZ1_SOF 657
+#define CMDQ_EVENT_VDO1_DISP_RSZ2_SOF 658
+#define CMDQ_EVENT_VDO1_DISP_RSZ3_SOF 659
+#define CMDQ_EVENT_VDO1_VPP_MERGE0_SOF 660
+#define CMDQ_EVENT_VDO1_VPP_MERGE1_SOF 661
+#define CMDQ_EVENT_VDO1_VPP_MERGE2_SOF 662
+#define CMDQ_EVENT_VDO1_VPP_MERGE3_SOF 663
+#define CMDQ_EVENT_VDO1_VPP_MERGE4_SOF 664
+#define CMDQ_EVENT_VDO1_VPP2_DL_RELAY_SOF 665
+#define CMDQ_EVENT_VDO1_VPP3_DL_RELAY_SOF 666
+#define CMDQ_EVENT_VDO0_DSC_DL_ASYNC_SOF 667
+#define CMDQ_EVENT_VDO0_MERGE_DL_ASYNC_SOF 668
+#define CMDQ_EVENT_VDO1_OUT_DL_RELAY_SOF 669
+#define CMDQ_EVENT_VDO1_DISP_MIXER_SOF 670
+#define CMDQ_EVENT_VDO1_HDR_VDO_FE0_SOF 671
+#define CMDQ_EVENT_VDO1_HDR_VDO_FE1_SOF 672
+#define CMDQ_EVENT_VDO1_HDR_GFX_FE0_SOF 673
+#define CMDQ_EVENT_VDO1_HDR_GFX_FE1_SOF 674
+#define CMDQ_EVENT_VDO1_HDR_VDO_BE0_SOF 675
+#define CMDQ_EVENT_VDO1_HDR_MLOAD_SOF 676
+#define CMDQ_EVENT_VDO1_DPI0_EXT_SOF 677
+#define CMDQ_EVENT_VDO1_DPI1_EXT_SOF 678
+#define CMDQ_EVENT_VDO1_DP_INTF_EXT_EXT_SOF 679
+#define CMDQ_EVENT_VDO1_MDP_RDMA0_FRAME_DONE 680
+#define CMDQ_EVENT_VDO1_MDP_RDMA1_FRAME_DONE 681
+#define CMDQ_EVENT_VDO1_MDP_RDMA2_FRAME_DONE 682
+#define CMDQ_EVENT_VDO1_MDP_RDMA3_FRAME_DONE 683
+#define CMDQ_EVENT_VDO1_MDP_RDMA4_FRAME_DONE 684
+#define CMDQ_EVENT_VDO1_MDP_RDMA5_FRAME_DONE 685
+#define CMDQ_EVENT_VDO1_MDP_RDMA6_FRAME_DONE 686
+#define CMDQ_EVENT_VDO1_MDP_RDMA7_FRAME_DONE 687
+#define CMDQ_EVENT_VDO1_DISP_PADDING0_FRAME_DONE 688
+#define CMDQ_EVENT_VDO1_DISP_PADDING1_FRAME_DONE 689
+#define CMDQ_EVENT_VDO1_DISP_PADDING2_FRAME_DONE 690
+#define CMDQ_EVENT_VDO1_DISP_PADDING3_FRAME_DONE 691
+#define CMDQ_EVENT_VDO1_DISP_PADDING4_FRAME_DONE 692
+#define CMDQ_EVENT_VDO1_DISP_PADDING5_FRAME_DONE 693
+#define CMDQ_EVENT_VDO1_DISP_PADDING6_FRAME_DONE 694
+#define CMDQ_EVENT_VDO1_DISP_PADDING7_FRAME_DONE 695
+#define CMDQ_EVENT_VDO1_DISP_RSZ0_FRAME_DONE 696
+#define CMDQ_EVENT_VDO1_DISP_RSZ1_FRAME_DONE 697
+#define CMDQ_EVENT_VDO1_DISP_RSZ2_FRAME_DONE 698
+#define CMDQ_EVENT_VDO1_DISP_RSZ3_FRAME_DONE 699
+#define CMDQ_EVENT_VDO1_VPP_MERGE0_FRAME_DONE 700
+#define CMDQ_EVENT_VDO1_VPP_MERGE1_FRAME_DONE 701
+#define CMDQ_EVENT_VDO1_VPP_MERGE2_FRAME_DONE 702
+#define CMDQ_EVENT_VDO1_VPP_MERGE3_FRAME_DONE 703
+#define CMDQ_EVENT_VDO1_VPP_MERGE4_FRAME_DONE 704
+#define CMDQ_EVENT_VDO1_DPI0_FRAME_DONE 705
+#define CMDQ_EVENT_VDO1_DPI1_FRAME_DONE 706
+#define CMDQ_EVENT_VDO1_DP_INTF0_FRAME_DONE 707
+#define CMDQ_EVENT_VDO1_DISP_MIXER_FRAME_DONE_MM 708
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_0 709
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_1 710
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_2 711
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_3 712
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_4 713
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_5 714
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_6 715
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_7 716
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_8 717
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_9 718
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_10 719
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_11 720
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_12 721
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_13 722
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_14 723
+#define CMDQ_EVENT_VDO1_STREAM_DONE_ENG_15 724
+#define CMDQ_EVENT_VDO1_DISP_RDMA_0_UNDERRUN 725
+#define CMDQ_EVENT_VDO1_DISP_RDMA_1_UNDERRUN 726
+#define CMDQ_EVENT_VDO1_U_MERGE4_UNDERRUN 727
+#define CMDQ_EVENT_VDO1_U_VPP_SPLIT_VIDEO_0_OVERFLOW 728
+#define CMDQ_EVENT_VDO1_U_VPP_SPLIT_VIDEO_1_OVERFLOW 729
+#define CMDQ_EVENT_VDO1_DSI_0_UNDERRUN 730
+#define CMDQ_EVENT_VDO1_DSI_1_UNDERRUN 731
+#define CMDQ_EVENT_VDO1_DP_INTF_0 732
+#define CMDQ_EVENT_VDO1_DP_INTF_1 733
+#define CMDQ_EVENT_VDO1_DPI_0 734
+#define CMDQ_EVENT_VDO1_DPI_1 735
+#define CMDQ_EVENT_VDO1_MDP_RDMA0_SW_RST_DONE 741
+#define CMDQ_EVENT_VDO1_MDP_RDMA1_SW_RST_DONE 742
+#define CMDQ_EVENT_VDO1_MDP_RDMA2_SW_RST_DONE 743
+#define CMDQ_EVENT_VDO1_MDP_RDMA3_SW_RST_DONE 744
+#define CMDQ_EVENT_VDO1_MDP_RDMA4_SW_RST_DONE 745
+#define CMDQ_EVENT_VDO1_MDP_RDMA5_SW_RST_DONE 746
+#define CMDQ_EVENT_VDO1_MDP_RDMA6_SW_RST_DONE 747
+#define CMDQ_EVENT_VDO1_MDP_RDMA7_SW_RST_DONE 748
+#define CMDQ_EVENT_VDO1_DP0_VDE_END_ENG_EVENT_MM 749
+#define CMDQ_EVENT_VDO1_DP0_VDE_START_ENG_EVENT_MM 750
+#define CMDQ_EVENT_VDO1_DP0_VSYNC_END_ENG_EVENT_MM 751
+#define CMDQ_EVENT_VDO1_DP0_VSYNC_START_ENG_EVENT_MM 752
+#define CMDQ_EVENT_VDO1_DP0_TARGET_LINE_ENG_EVENT_MM 753
+#define CMDQ_EVENT_VDO1_VPP_MERGE0_EVENT 754
+#define CMDQ_EVENT_VDO1_VPP_MERGE1_EVENT 755
+#define CMDQ_EVENT_VDO1_VPP_MERGE2_EVENT 756
+#define CMDQ_EVENT_VDO1_VPP_MERGE3_EVENT 757
+#define CMDQ_EVENT_VDO1_VPP_MERGE4_EVENT 758
+#define CMDQ_EVENT_VDO1_HDMITX_EVENT 759
+#define CMDQ_EVENT_VDO1_HDR_VDO_BE0_ADL_TRIG_EVENT_MM 760
+#define CMDQ_EVENT_VDO1_HDR_GFX_FE1_THDR_ADL_TRIG_EVENT_MM 761
+#define CMDQ_EVENT_VDO1_HDR_GFX_FE1_DM_ADL_TRIG_EVENT_MM 762
+#define CMDQ_EVENT_VDO1_HDR_GFX_FE0_THDR_ADL_TRIG_EVENT_MM 763
+#define CMDQ_EVENT_VDO1_HDR_GFX_FE0_DM_ADL_TRIG_EVENT_MM 764
+#define CMDQ_EVENT_VDO1_HDR_VDO_FE1_ADL_TRIG_EVENT_MM 765
+#define CMDQ_EVENT_VDO1_HDR_VDO_FE1_AD0_TRIG_EVENT_MM 766
+#define CMDQ_EVENT_VDO1_DPI0_TARGET_LINE_1_EVENT_MM 767
+#define CMDQ_EVENT_HANDSHAKE_0 768
+#define CMDQ_EVENT_HANDSHAKE_1 769
+#define CMDQ_EVENT_HANDSHAKE_2 770
+#define CMDQ_EVENT_HANDSHAKE_3 771
+#define CMDQ_EVENT_HANDSHAKE_4 772
+#define CMDQ_EVENT_HANDSHAKE_5 773
+#define CMDQ_EVENT_HANDSHAKE_6 774
+#define CMDQ_EVENT_HANDSHAKE_7 775
+#define CMDQ_EVENT_HANDSHAKE_8 776
+#define CMDQ_EVENT_HANDSHAKE_9 777
+#define CMDQ_EVENT_HANDSHAKE_10 778
+#define CMDQ_EVENT_HANDSHAKE_11 779
+#define CMDQ_EVENT_HANDSHAKE_12 780
+#define CMDQ_EVENT_HANDSHAKE_13 781
+#define CMDQ_EVENT_HANDSHAKE_14 782
+#define CMDQ_EVENT_HANDSHAKE_15 783
+#define CMDQ_EVENT_VDEC_SOC_EVENT_0 800
+#define CMDQ_EVENT_VDEC_SOC_EVENT_1 801
+#define CMDQ_EVENT_VDEC_SOC_EVENT_2 802
+#define CMDQ_EVENT_VDEC_SOC_EVENT_3 803
+#define CMDQ_EVENT_VDEC_SOC_EVENT_4 804
+#define CMDQ_EVENT_VDEC_SOC_EVENT_5 805
+#define CMDQ_EVENT_VDEC_SOC_EVENT_6 806
+#define CMDQ_EVENT_VDEC_SOC_EVENT_7 807
+#define CMDQ_EVENT_VDEC_SOC_EVENT_8 808
+#define CMDQ_EVENT_VDEC_SOC_EVENT_9 809
+#define CMDQ_EVENT_VDEC_SOC_EVENT_10 810
+#define CMDQ_EVENT_VDEC_SOC_EVENT_11 811
+#define CMDQ_EVENT_VDEC_SOC_EVENT_12 812
+#define CMDQ_EVENT_VDEC_SOC_EVENT_13 813
+#define CMDQ_EVENT_VDEC_SOC_EVENT_14 814
+#define CMDQ_EVENT_VDEC_SOC_EVENT_15 815
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_0 832
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_1 833
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_2 834
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_3 835
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_4 836
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_5 837
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_6 838
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_7 839
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_8 840
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_9 841
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_10 842
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_11 843
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_12 844
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_13 845
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_14 846
+#define CMDQ_EVENT_VDEC_CORE0_EVENT_15 847
+#define CMDQ_EVENT_VENC_TOP_VENC_FRAME_DONE 865
+#define CMDQ_EVENT_VENC_TOP_VENC_PAUSE_DONE 866
+#define CMDQ_EVENT_VENC_TOP_JPGENC_DONE 867
+#define CMDQ_EVENT_VENC_TOP_VENC_MB_DONE 868
+#define CMDQ_EVENT_VENC_TOP_VENC_128BYTE_DONE 869
+#define CMDQ_EVENT_VENC_TOP_JPGDEC_DONE 870
+#define CMDQ_EVENT_VENC_TOP_VENC_SLICE_DONE 871
+#define CMDQ_EVENT_VENC_TOP_JPGDEC_INSUFF_DONE 872
+#define CMDQ_EVENT_VENC_TOP_WP_2ND_STAGE_DONE 874
+#define CMDQ_EVENT_VENC_TOP_WP_3RD_STAGE_DONE 875
+#define CMDQ_EVENT_VENC_TOP_PPS_HEADER_DONE 876
+#define CMDQ_EVENT_VENC_TOP_SPS_HEADER_DONE 877
+#define CMDQ_EVENT_VENC_TOP_VPS_HEADER_DONE 878
+#define CMDQ_EVENT_WPE_VPP0_WPE_GCE_FRAME_DONE 882
+#define CMDQ_EVENT_WPE_VPP0_WPE_DONE_SYNC_OUT 883
+#define CMDQ_EVENT_SVPP1_MDP_OVL_NEW_EVENT_2 896
+#define CMDQ_EVENT_SVPP1_MDP_OVL_NEW_EVENT_3 897
+#define CMDQ_EVENT_SVPP1_MDP_OVL_NEW_EVENT_4 898
+#define CMDQ_EVENT_SVPP1_MDP_OVL_NEW_EVENT_5 899
+#define CMDQ_EVENT_SVPP1_MDP_OVL_NEW_EVENT_6 900
+#define CMDQ_EVENT_VDO1_DPI0_TARGET_LINE_0_EVENT_MM 928
+#define CMDQ_EVENT_VDO1_DPI0_TRIGGER_LOOP_CLEAR_EVENT_MM 929
+#define CMDQ_EVENT_VDO1_DPI0_LAST_LINE_EVENT_MM 930
+#define CMDQ_EVENT_VDO1_DPI0_LAST_SAFE_BLANK_EVENT_MM 931
+#define CMDQ_EVENT_VDO1_DPI0_VSYNC_START_EVENT_MM 932
+#define CMDQ_EVENT_VDO1_DPI1_TARGET_LINE_1_EVENT_MM 933
+#define CMDQ_EVENT_VDO1_DPI1_TARGET_LINE_0_EVENT_MM 934
+#define CMDQ_EVENT_VDO1_DPI1_TRIGGER_LOOP_CLEAR_EVENT_MM 935
+#define CMDQ_EVENT_VDO1_DPI1_LAST_LINE_EVENT_MM 936
+#define CMDQ_EVENT_VDO1_DPI1_LAST_SAFE_BLANK_EVENT_MM 937
+#define CMDQ_EVENT_VDO1_DPI1_VSYNC_START_EVENT_MM 938
+#define CMDQ_EVENT_VDO1_DP_INTF_TARGET_LINE_1_EVENT_MM 939
+#define CMDQ_EVENT_VDO1_DP_INTF_TARGET_LINE_0_EVENT_MM 940
+#define CMDQ_EVENT_VDO1_DP_INTF_TRIGGER_LOOP_CLEAR_EVENT_MM 941
+#define CMDQ_EVENT_VDO1_DP_INTF_LAST_LINE_EVENT_MM 942
+#define CMDQ_EVENT_VDO1_DP_INTF_LAST_SAFE_BLANK_EVENT_MM 943
+#define CMDQ_EVENT_VBLANK_FALLING 946
+#define CMDQ_EVENT_VSC_FINISH 947
+#define CMDQ_EVENT_TPR_0 962
+#define CMDQ_EVENT_TPR_1 963
+#define CMDQ_EVENT_TPR_2 964
+#define CMDQ_EVENT_TPR_3 965
+#define CMDQ_EVENT_TPR_4 966
+#define CMDQ_EVENT_TPR_5 967
+#define CMDQ_EVENT_TPR_6 968
+#define CMDQ_EVENT_TPR_7 969
+#define CMDQ_EVENT_TPR_8 970
+#define CMDQ_EVENT_TPR_9 971
+#define CMDQ_EVENT_TPR_10 972
+#define CMDQ_EVENT_TPR_11 973
+#define CMDQ_EVENT_TPR_12 974
+#define CMDQ_EVENT_TPR_13 975
+#define CMDQ_EVENT_TPR_14 976
+#define CMDQ_EVENT_TPR_15 977
+#define CMDQ_EVENT_TPR_16 978
+#define CMDQ_EVENT_TPR_17 979
+#define CMDQ_EVENT_TPR_18 980
+#define CMDQ_EVENT_TPR_19 981
+#define CMDQ_EVENT_TPR_20 982
+#define CMDQ_EVENT_TPR_21 983
+#define CMDQ_EVENT_TPR_22 984
+#define CMDQ_EVENT_TPR_23 985
+#define CMDQ_EVENT_TPR_24 986
+#define CMDQ_EVENT_TPR_25 987
+#define CMDQ_EVENT_TPR_26 988
+#define CMDQ_EVENT_TPR_27 989
+#define CMDQ_EVENT_TPR_28 990
+#define CMDQ_EVENT_TPR_29 991
+#define CMDQ_EVENT_TPR_30 992
+#define CMDQ_EVENT_TPR_31 993
+#define CMDQ_EVENT_TPR_TIMEOUT_0 994
+#define CMDQ_EVENT_TPR_TIMEOUT_1 995
+#define CMDQ_EVENT_TPR_TIMEOUT_2 996
+#define CMDQ_EVENT_TPR_TIMEOUT_3 997
+#define CMDQ_EVENT_TPR_TIMEOUT_4 998
+#define CMDQ_EVENT_TPR_TIMEOUT_5 999
+#define CMDQ_EVENT_TPR_TIMEOUT_6 1000
+#define CMDQ_EVENT_TPR_TIMEOUT_7 1001
+#define CMDQ_EVENT_TPR_TIMEOUT_8 1002
+#define CMDQ_EVENT_TPR_TIMEOUT_9 1003
+#define CMDQ_EVENT_TPR_TIMEOUT_10 1004
+#define CMDQ_EVENT_TPR_TIMEOUT_11 1005
+#define CMDQ_EVENT_TPR_TIMEOUT_12 1006
+#define CMDQ_EVENT_TPR_TIMEOUT_13 1007
+#define CMDQ_EVENT_TPR_TIMEOUT_14 1008
+#define CMDQ_EVENT_TPR_TIMEOUT_15 1009
+#define CMDQ_EVENT_OUTPIN_0 1018
+#define CMDQ_EVENT_OUTPIN_1 1019
+
+#define CMDQ_SYNC_TOKEN_IMGSYS_WPE_EIS 124
+#define CMDQ_SYNC_TOKEN_IMGSYS_WPE_TNR 125
+#define CMDQ_SYNC_TOKEN_IMGSYS_WPE_LITE 126
+#define CMDQ_SYNC_TOKEN_IMGSYS_TRAW 127
+#define CMDQ_SYNC_TOKEN_IMGSYS_LTRAW 128
+#define CMDQ_SYNC_TOKEN_CAMSYS_POOL_1 223
+#define CMDQ_SYNC_TOKEN_CAMSYS_POOL_2 224
+#define CMDQ_SYNC_TOKEN_CAMSYS_POOL_3 225
+#define CMDQ_SYNC_TOKEN_CAMSYS_POOL_4 226
+#define CMDQ_SYNC_TOKEN_CAMSYS_POOL_5 227
+#define CMDQ_SYNC_TOKEN_CAMSYS_POOL_6 228
+#define CMDQ_SYNC_TOKEN_CAMSYS_POOL_7 229
+#define CMDQ_SYNC_TOKEN_CAMSYS_POOL_8 230
+#define CMDQ_SYNC_TOKEN_CAMSYS_POOL_9 231
+#define CMDQ_SYNC_TOKEN_CAMSYS_POOL_10 232
+#define CMDQ_SYNC_TOKEN_IMGSYS_XTRAW 233
+#define CMDQ_SYNC_TOKEN_IMGSYS_DIP 234
+#define CMDQ_SYNC_TOKEN_IMGSYS_PQDIP_A 235
+#define CMDQ_SYNC_TOKEN_IMGSYS_PQDIP_B 236
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_1 237
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_2 238
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_3 239
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_4 240
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_5 241
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_6 242
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_7 243
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_8 244
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_9 245
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_10 246
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_11 247
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_12 248
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_13 249
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_14 250
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_15 251
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_16 252
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_17 253
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_18 254
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_19 255
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_20 276
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_21 277
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_22 278
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_23 279
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_24 280
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_25 281
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_26 282
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_27 283
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_28 284
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_29 285
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_30 286
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_31 287
+#define CMDQ_SYNC_TOKEN_IPESYS_ME 300
+#define CMDQ_SYNC_TOKEN_IMGSYS_VSS_TRAW 301
+#define CMDQ_SYNC_TOKEN_IMGSYS_VSS_LTRAW 302
+#define CMDQ_SYNC_TOKEN_IMGSYS_VSS_XTRAW 303
+#define CMDQ_SYNC_TOKEN_IMGSYS_VSS_DIP 304
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_32 308
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_33 309
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_34 310
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_35 311
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_36 312
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_37 313
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_38 314
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_39 315
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_40 316
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_41 370
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_42 371
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_43 372
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_44 373
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_45 374
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_46 375
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_47 376
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_48 377
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_49 378
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_50 379
+#define CMDQ_SYNC_TOKEN_TZMP_ISP_WAIT 380
+#define CMDQ_SYNC_TOKEN_TZMP_ISP_SET 381
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_51 790
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_52 791
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_53 792
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_54 793
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_55 794
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_56 795
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_57 796
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_58 797
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_59 798
+#define CMDQ_SYNC_TOKEN_IMGSYS_POOL_60 799
+#define CMDQ_SYNC_TOKEN_PREBUILT_MDP_WAIT 816
+#define CMDQ_SYNC_TOKEN_PREBUILT_MDP_SET 817
+#define CMDQ_SYNC_TOKEN_PREBUILT_MDP_LOCK 818
+#define CMDQ_SYNC_TOKEN_PREBUILT_MML_WAIT 819
+#define CMDQ_SYNC_TOKEN_PREBUILT_MML_SET 820
+#define CMDQ_SYNC_TOKEN_PREBUILT_MML_LOCK 821
+#define CMDQ_SYNC_TOKEN_PREBUILT_VFMT_WAIT 822
+#define CMDQ_SYNC_TOKEN_PREBUILT_VFMT_SET 823
+#define CMDQ_SYNC_TOKEN_PREBUILT_VFMT_LOCK 824
+#define CMDQ_SYNC_TOKEN_PREBUILT_DISP_WAIT 825
+#define CMDQ_SYNC_TOKEN_PREBUILT_DISP_SET 826
+#define CMDQ_SYNC_TOKEN_PREBUILT_DISP_LOCK 827
+#define CMDQ_SYNC_TOKEN_CONFIG_DIRTY 848
+#define CMDQ_SYNC_TOKEN_STREAM_EOF 849
+#define CMDQ_SYNC_TOKEN_ESD_EOF 850
+#define CMDQ_SYNC_TOKEN_STREAM_BLOCK 851
+#define CMDQ_SYNC_TOKEN_CABC_EOF 852
+#define CMDQ_SYNC_TOKEN_VENC_INPUT_READY 853
+#define CMDQ_SYNC_TOKEN_VENC_EOF 854
+#define CMDQ_SYNC_TOKEN_SECURE_THR_EOF 855
+#define CMDQ_SYNC_TOKEN_USER_0 856
+#define CMDQ_SYNC_TOKEN_USER_1 857
+#define CMDQ_SYNC_TOKEN_POLL_MONITOR 858
+#define CMDQ_TOKEN_TPR_LOCK 859
+#define CMDQ_SYNC_TOKEN_MSS 860
+#define CMDQ_SYNC_TOKEN_MSF 861
+#define CMDQ_SYNC_TOKEN_GPR_SET_0 884
+#define CMDQ_SYNC_TOKEN_GPR_SET_1 885
+#define CMDQ_SYNC_TOKEN_GPR_SET_2 886
+#define CMDQ_SYNC_TOKEN_GPR_SET_3 887
+#define CMDQ_SYNC_TOKEN_GPR_SET_4 888
+#define CMDQ_SYNC_RESOURCE_WROT0 889
+#define CMDQ_SYNC_RESOURCE_WROT1 890
+#define CMDQ_SYNC_TOKEN_DISP_VA_START 1012
+#define CMDQ_SYNC_TOKEN_DISP_VA_END 1013
+
+#endif
--- /dev/null
+/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */
+/*
+ * Copyright (c) 2022 MediaTek Inc.
+ * Author: Yong Wu <yong.wu@mediatek.com>
+ */
+#ifndef _DT_BINDINGS_MEMORY_MT8365_LARB_PORT_H_
+#define _DT_BINDINGS_MEMORY_MT8365_LARB_PORT_H_
+
+#include <dt-bindings/memory/mtk-memory-port.h>
+
+#define M4U_LARB0_ID 0
+#define M4U_LARB1_ID 1
+#define M4U_LARB2_ID 2
+#define M4U_LARB3_ID 3
+
+/* larb0 */
+#define M4U_PORT_DISP_OVL0 MTK_M4U_ID(M4U_LARB0_ID, 0)
+#define M4U_PORT_DISP_OVL0_2L MTK_M4U_ID(M4U_LARB0_ID, 1)
+#define M4U_PORT_DISP_RDMA0 MTK_M4U_ID(M4U_LARB0_ID, 2)
+#define M4U_PORT_DISP_WDMA0 MTK_M4U_ID(M4U_LARB0_ID, 3)
+#define M4U_PORT_DISP_RDMA1 MTK_M4U_ID(M4U_LARB0_ID, 4)
+#define M4U_PORT_MDP_RDMA0 MTK_M4U_ID(M4U_LARB0_ID, 5)
+#define M4U_PORT_MDP_WROT1 MTK_M4U_ID(M4U_LARB0_ID, 6)
+#define M4U_PORT_MDP_WROT0 MTK_M4U_ID(M4U_LARB0_ID, 7)
+#define M4U_PORT_MDP_RDMA1 MTK_M4U_ID(M4U_LARB0_ID, 8)
+#define M4U_PORT_DISP_FAKE0 MTK_M4U_ID(M4U_LARB0_ID, 9)
+#define M4U_PORT_APU_READ MTK_M4U_ID(M4U_LARB0_ID, 10)
+#define M4U_PORT_APU_WRITE MTK_M4U_ID(M4U_LARB0_ID, 11)
+
+/* larb1 */
+#define M4U_PORT_VENC_RCPU MTK_M4U_ID(M4U_LARB1_ID, 0)
+#define M4U_PORT_VENC_REC MTK_M4U_ID(M4U_LARB1_ID, 1)
+#define M4U_PORT_VENC_BSDMA MTK_M4U_ID(M4U_LARB1_ID, 2)
+#define M4U_PORT_VENC_SV_COMV MTK_M4U_ID(M4U_LARB1_ID, 3)
+#define M4U_PORT_VENC_RD_COMV MTK_M4U_ID(M4U_LARB1_ID, 4)
+#define M4U_PORT_VENC_NBM_RDMA MTK_M4U_ID(M4U_LARB1_ID, 5)
+#define M4U_PORT_VENC_NBM_RDMA_LITE MTK_M4U_ID(M4U_LARB1_ID, 6)
+#define M4U_PORT_JPGENC_Y_RDMA MTK_M4U_ID(M4U_LARB1_ID, 7)
+#define M4U_PORT_JPGENC_C_RDMA MTK_M4U_ID(M4U_LARB1_ID, 8)
+#define M4U_PORT_JPGENC_Q_TABLE MTK_M4U_ID(M4U_LARB1_ID, 9)
+#define M4U_PORT_JPGENC_BSDMA MTK_M4U_ID(M4U_LARB1_ID, 10)
+#define M4U_PORT_JPGDEC_WDMA MTK_M4U_ID(M4U_LARB1_ID, 11)
+#define M4U_PORT_JPGDEC_BSDMA MTK_M4U_ID(M4U_LARB1_ID, 12)
+#define M4U_PORT_VENC_NBM_WDMA MTK_M4U_ID(M4U_LARB1_ID, 13)
+#define M4U_PORT_VENC_NBM_WDMA_LITE MTK_M4U_ID(M4U_LARB1_ID, 14)
+#define M4U_PORT_VENC_CUR_LUMA MTK_M4U_ID(M4U_LARB1_ID, 15)
+#define M4U_PORT_VENC_CUR_CHROMA MTK_M4U_ID(M4U_LARB1_ID, 16)
+#define M4U_PORT_VENC_REF_LUMA MTK_M4U_ID(M4U_LARB1_ID, 17)
+#define M4U_PORT_VENC_REF_CHROMA MTK_M4U_ID(M4U_LARB1_ID, 18)
+
+/* larb2 */
+#define M4U_PORT_CAM_IMGO MTK_M4U_ID(M4U_LARB2_ID, 0)
+#define M4U_PORT_CAM_RRZO MTK_M4U_ID(M4U_LARB2_ID, 1)
+#define M4U_PORT_CAM_AAO MTK_M4U_ID(M4U_LARB2_ID, 2)
+#define M4U_PORT_CAM_LCS MTK_M4U_ID(M4U_LARB2_ID, 3)
+#define M4U_PORT_CAM_ESFKO MTK_M4U_ID(M4U_LARB2_ID, 4)
+#define M4U_PORT_CAM_CAM_SV0 MTK_M4U_ID(M4U_LARB2_ID, 5)
+#define M4U_PORT_CAM_CAM_SV1 MTK_M4U_ID(M4U_LARB2_ID, 6)
+#define M4U_PORT_CAM_LSCI MTK_M4U_ID(M4U_LARB2_ID, 7)
+#define M4U_PORT_CAM_LSCI_D MTK_M4U_ID(M4U_LARB2_ID, 8)
+#define M4U_PORT_CAM_AFO MTK_M4U_ID(M4U_LARB2_ID, 9)
+#define M4U_PORT_CAM_SPARE MTK_M4U_ID(M4U_LARB2_ID, 10)
+#define M4U_PORT_CAM_BPCI MTK_M4U_ID(M4U_LARB2_ID, 11)
+#define M4U_PORT_CAM_BPCI_D MTK_M4U_ID(M4U_LARB2_ID, 12)
+#define M4U_PORT_CAM_UFDI MTK_M4U_ID(M4U_LARB2_ID, 13)
+#define M4U_PORT_CAM_IMGI MTK_M4U_ID(M4U_LARB2_ID, 14)
+#define M4U_PORT_CAM_IMG2O MTK_M4U_ID(M4U_LARB2_ID, 15)
+#define M4U_PORT_CAM_IMG3O MTK_M4U_ID(M4U_LARB2_ID, 16)
+#define M4U_PORT_CAM_WPE0_I MTK_M4U_ID(M4U_LARB2_ID, 17)
+#define M4U_PORT_CAM_WPE1_I MTK_M4U_ID(M4U_LARB2_ID, 18)
+#define M4U_PORT_CAM_WPE_O MTK_M4U_ID(M4U_LARB2_ID, 19)
+#define M4U_PORT_CAM_FD0_I MTK_M4U_ID(M4U_LARB2_ID, 20)
+#define M4U_PORT_CAM_FD1_I MTK_M4U_ID(M4U_LARB2_ID, 21)
+#define M4U_PORT_CAM_FD0_O MTK_M4U_ID(M4U_LARB2_ID, 22)
+#define M4U_PORT_CAM_FD1_O MTK_M4U_ID(M4U_LARB2_ID, 23)
+
+/* larb3 */
+#define M4U_PORT_HW_VDEC_MC_EXT MTK_M4U_ID(M4U_LARB3_ID, 0)
+#define M4U_PORT_HW_VDEC_UFO_EXT MTK_M4U_ID(M4U_LARB3_ID, 1)
+#define M4U_PORT_HW_VDEC_PP_EXT MTK_M4U_ID(M4U_LARB3_ID, 2)
+#define M4U_PORT_HW_VDEC_PRED_RD_EXT MTK_M4U_ID(M4U_LARB3_ID, 3)
+#define M4U_PORT_HW_VDEC_PRED_WR_EXT MTK_M4U_ID(M4U_LARB3_ID, 4)
+#define M4U_PORT_HW_VDEC_PPWRAP_EXT MTK_M4U_ID(M4U_LARB3_ID, 5)
+#define M4U_PORT_HW_VDEC_TILE_EXT MTK_M4U_ID(M4U_LARB3_ID, 6)
+#define M4U_PORT_HW_VDEC_VLD_EXT MTK_M4U_ID(M4U_LARB3_ID, 7)
+#define M4U_PORT_HW_VDEC_VLD2_EXT MTK_M4U_ID(M4U_LARB3_ID, 8)
+#define M4U_PORT_HW_VDEC_AVC_MV_EXT MTK_M4U_ID(M4U_LARB3_ID, 9)
+#define M4U_PORT_HW_VDEC_RG_CTRL_DMA_EXT MTK_M4U_ID(M4U_LARB3_ID, 10)
+
+#endif
--- /dev/null
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/*
+ * Qualcomm QMP PHY constants
+ *
+ * Copyright (C) 2022 Linaro Limited
+ */
+
+#ifndef _DT_BINDINGS_PHY_QMP
+#define _DT_BINDINGS_PHY_QMP
+
+/* QMP USB4-USB3-DP clocks */
+#define QMP_USB43DP_USB3_PIPE_CLK 0
+#define QMP_USB43DP_DP_LINK_CLK 1
+#define QMP_USB43DP_DP_VCO_DIV_CLK 2
+
+/* QMP USB4-USB3-DP PHYs */
+#define QMP_USB43DP_USB3_PHY 0
+#define QMP_USB43DP_DP_PHY 1
+
+#endif /* _DT_BINDINGS_PHY_QMP */
#ifndef _DT_BINDINGS_ZYNQMP_POWER_H
#define _DT_BINDINGS_ZYNQMP_POWER_H
+#define PD_RPU_0 7
+#define PD_RPU_1 8
+#define PD_R5_0_ATCM 15
+#define PD_R5_0_BTCM 16
+#define PD_R5_1_ATCM 17
+#define PD_R5_1_BTCM 18
#define PD_USB_0 22
#define PD_USB_1 23
#define PD_TTC_0 24
--- /dev/null
+/* SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause)*/
+/*
+ * Copyright (c) 2022 MediaTek Inc.
+ * Author: Runyang Chen <runyang.chen@mediatek.com>
+ */
+
+#ifndef _DT_BINDINGS_RESET_CONTROLLER_MT8188
+#define _DT_BINDINGS_RESET_CONTROLLER_MT8188
+
+#define MT8188_TOPRGU_CONN_MCU_SW_RST 0
+#define MT8188_TOPRGU_INFRA_GRST_SW_RST 1
+#define MT8188_TOPRGU_IPU0_SW_RST 2
+#define MT8188_TOPRGU_IPU1_SW_RST 3
+#define MT8188_TOPRGU_IPU2_SW_RST 4
+#define MT8188_TOPRGU_AUD_ASRC_SW_RST 5
+#define MT8188_TOPRGU_INFRA_SW_RST 6
+#define MT8188_TOPRGU_MMSYS_SW_RST 7
+#define MT8188_TOPRGU_MFG_SW_RST 8
+#define MT8188_TOPRGU_VENC_SW_RST 9
+#define MT8188_TOPRGU_VDEC_SW_RST 10
+#define MT8188_TOPRGU_CAM_VCORE_SW_RST 11
+#define MT8188_TOPRGU_SCP_SW_RST 12
+#define MT8188_TOPRGU_APMIXEDSYS_SW_RST 13
+#define MT8188_TOPRGU_AUDIO_SW_RST 14
+#define MT8188_TOPRGU_CAMSYS_SW_RST 15
+#define MT8188_TOPRGU_MJC_SW_RST 16
+#define MT8188_TOPRGU_PERI_SW_RST 17
+#define MT8188_TOPRGU_PERI_AO_SW_RST 18
+#define MT8188_TOPRGU_PCIE_SW_RST 19
+#define MT8188_TOPRGU_ADSPSYS_SW_RST 21
+#define MT8188_TOPRGU_DPTX_SW_RST 22
+#define MT8188_TOPRGU_SPMI_MST_SW_RST 23
+
+#define MT8188_TOPRGU_SW_RST_NUM 24
+
+#endif /* _DT_BINDINGS_RESET_CONTROLLER_MT8188 */
* @type: the type of the container struct this is embedded in.
* @member: the name of the member within the struct.
*
+ * WARNING: any const qualifier of @ptr is lost.
*/
#define container_of(ptr, type, member) ({ \
void *__mptr = (void *)(ptr); \
((type *)(__mptr - offsetof(type, member))); })
/**
- * container_of_safe - cast a member of a structure out to the containing structure
- * @ptr: the pointer to the member.
- * @type: the type of the container struct this is embedded in.
- * @member: the name of the member within the struct.
- *
- * If IS_ERR_OR_NULL(ptr), ptr is returned unchanged.
+ * container_of_const - cast a member of a structure out to the containing
+ * structure and preserve the const-ness of the pointer
+ * @ptr: the pointer to the member
+ * @type: the type of the container struct this is embedded in.
+ * @member: the name of the member within the struct.
*/
-#define container_of_safe(ptr, type, member) ({ \
- void *__mptr = (void *)(ptr); \
- static_assert(__same_type(*(ptr), ((type *)0)->member) || \
- __same_type(*(ptr), void), \
- "pointer type mismatch in container_of_safe()"); \
- IS_ERR_OR_NULL(__mptr) ? ERR_CAST(__mptr) : \
- ((type *)(__mptr - offsetof(type, member))); })
+#define container_of_const(ptr, type, member) \
+ _Generic(ptr, \
+ const typeof(*(ptr)) *: ((const type *)container_of(ptr, type, member)),\
+ default: ((type *)container_of(ptr, type, member)) \
+ )
#endif /* _LINUX_CONTAINER_OF_H */
bool supplier_preactivated; /* Owned by consumer probe. */
};
-static inline struct device *kobj_to_dev(struct kobject *kobj)
-{
- return container_of(kobj, struct device, kobj);
-}
+#define kobj_to_dev(__kobj) container_of_const(__kobj, struct device, kobj)
/**
* device_iommu_mapped - Returns true when the device DMA is translated
int __must_check devm_device_add_groups(struct device *dev,
const struct attribute_group **groups);
-void devm_device_remove_groups(struct device *dev,
- const struct attribute_group **groups);
int __must_check devm_device_add_group(struct device *dev,
const struct attribute_group *grp);
-void devm_device_remove_group(struct device *dev,
- const struct attribute_group *grp);
/*
* Platform "fixup" functions - allow the platform to have their say
const struct attribute_group **dev_groups;
struct kobject *dev_kobj;
- int (*dev_uevent)(struct device *dev, struct kobj_uevent_env *env);
- char *(*devnode)(struct device *dev, umode_t *mode);
+ int (*dev_uevent)(const struct device *dev, struct kobj_uevent_env *env);
+ char *(*devnode)(const struct device *dev, umode_t *mode);
void (*class_release)(struct class *class);
void (*dev_release)(struct device *dev);
int (*shutdown_pre)(struct device *dev);
const struct kobj_ns_type_operations *ns_type;
- const void *(*namespace)(struct device *dev);
+ const void *(*namespace)(const struct device *dev);
- void (*get_ownership)(struct device *dev, kuid_t *uid, kgid_t *gid);
+ void (*get_ownership)(const struct device *dev, kuid_t *uid, kgid_t *gid);
const struct dev_pm_ops *pm;
static inline void bpf_jit_binary_lock_ro(struct bpf_binary_header *hdr)
{
set_vm_flush_reset_perms(hdr);
- set_memory_ro((unsigned long)hdr, hdr->size >> PAGE_SHIFT);
- set_memory_x((unsigned long)hdr, hdr->size >> PAGE_SHIFT);
+ set_memory_rox((unsigned long)hdr, hdr->size >> PAGE_SHIFT);
}
int sk_filter_trim_cap(struct sock *sk, struct sk_buff *skb, unsigned int cap);
#ifndef __FIRMWARE_ZYNQMP_H__
#define __FIRMWARE_ZYNQMP_H__
+#include <linux/types.h>
#include <linux/err.h>
enum pm_api_id {
PM_GET_API_VERSION = 1,
PM_REGISTER_NOTIFIER = 5,
+ PM_FORCE_POWERDOWN = 8,
+ PM_REQUEST_WAKEUP = 10,
PM_SYSTEM_SHUTDOWN = 12,
PM_REQUEST_NODE = 13,
PM_RELEASE_NODE = 14,
};
enum pm_ioctl_id {
+ IOCTL_GET_RPU_OPER_MODE = 0,
+ IOCTL_SET_RPU_OPER_MODE = 1,
+ IOCTL_RPU_BOOT_ADDR_CONFIG = 2,
+ IOCTL_TCM_COMB_CONFIG = 3,
IOCTL_SET_TAPDELAY_BYPASS = 4,
IOCTL_SD_DLL_RESET = 6,
IOCTL_SET_SD_TAPDELAY = 7,
PM_QID_CLOCK_GET_MAX_DIVISOR = 13,
};
+enum rpu_oper_mode {
+ PM_RPU_MODE_LOCKSTEP = 0,
+ PM_RPU_MODE_SPLIT = 1,
+};
+
+enum rpu_boot_mem {
+ PM_RPU_BOOTMEM_LOVEC = 0,
+ PM_RPU_BOOTMEM_HIVEC = 1,
+};
+
+enum rpu_tcm_comb {
+ PM_RPU_TCM_SPLIT = 0,
+ PM_RPU_TCM_COMB = 1,
+};
+
enum zynqmp_pm_reset_action {
PM_RESET_ACTION_RELEASE = 0,
PM_RESET_ACTION_ASSERT = 1,
int zynqmp_pm_set_feature_config(enum pm_feature_config_id id, u32 value);
int zynqmp_pm_get_feature_config(enum pm_feature_config_id id, u32 *payload);
int zynqmp_pm_register_sgi(u32 sgi_num, u32 reset);
+int zynqmp_pm_force_pwrdwn(const u32 target,
+ const enum zynqmp_pm_request_ack ack);
+int zynqmp_pm_request_wake(const u32 node,
+ const bool set_addr,
+ const u64 address,
+ const enum zynqmp_pm_request_ack ack);
+int zynqmp_pm_get_rpu_mode(u32 node_id, enum rpu_oper_mode *rpu_mode);
+int zynqmp_pm_set_rpu_mode(u32 node_id, u32 arg1);
+int zynqmp_pm_set_tcm_config(u32 node_id, u32 arg1);
int zynqmp_pm_set_sd_config(u32 node, enum pm_sd_config_type config, u32 value);
int zynqmp_pm_set_gem_config(u32 node, enum pm_gem_config_type config,
u32 value);
return -ENODEV;
}
+static inline int zynqmp_pm_force_pwrdwn(const u32 target,
+ const enum zynqmp_pm_request_ack ack)
+{
+ return -ENODEV;
+}
+
+static inline int zynqmp_pm_request_wake(const u32 node,
+ const bool set_addr,
+ const u64 address,
+ const enum zynqmp_pm_request_ack ack)
+{
+ return -ENODEV;
+}
+
+static inline int zynqmp_pm_get_rpu_mode(u32 node_id, enum rpu_oper_mode *rpu_mode)
+{
+ return -ENODEV;
+}
+
+static inline int zynqmp_pm_set_rpu_mode(u32 node_id, u32 arg1)
+{
+ return -ENODEV;
+}
+
+static inline int zynqmp_pm_set_tcm_config(u32 node_id, u32 arg1)
+{
+ return -ENODEV;
+}
+
static inline int zynqmp_pm_set_sd_config(u32 node,
enum pm_sd_config_type config,
u32 value)
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef __LINUX_HTCPLD_H
-#define __LINUX_HTCPLD_H
-
-struct htcpld_chip_platform_data {
- unsigned int addr;
- unsigned int reset;
- unsigned int num_gpios;
- unsigned int gpio_out_base;
- unsigned int gpio_in_base;
- unsigned int irq_base;
- unsigned int num_irqs;
-};
-
-struct htcpld_core_platform_data {
- unsigned int i2c_adapter_id;
-
- struct htcpld_chip_platform_data *chip;
- unsigned int num_chip;
-};
-
-#endif /* __LINUX_HTCPLD_H */
-
struct attribute_group buffer_group;
/* @attrs: Standard attributes of the buffer. */
- const struct attribute **attrs;
+ const struct iio_dev_attr **attrs;
/* @demux_bounce: Buffer for doing gather from incoming scan. */
void *demux_bounce;
* @mount_matrix: The mounting matrix of the sensor.
* @sensor_settings: Pointer to the specific sensor settings in use.
* @current_fullscale: Maximum range of measure by the sensor.
- * @vdd: Pointer to sensor's Vdd power supply
- * @vdd_io: Pointer to sensor's Vdd-IO power supply
* @regmap: Pointer to specific sensor regmap configuration.
* @enabled: Status of the sensor (false->off, true->on).
* @odr: Output data rate of the sensor [Hz].
struct iio_mount_matrix mount_matrix;
struct st_sensor_settings *sensor_settings;
struct st_sensor_fullscale_avl *current_fullscale;
- struct regulator *vdd;
- struct regulator *vdd_io;
struct regmap *regmap;
bool enabled;
struct i2c_client *i2c;
struct iio_trigger *trig;
struct iio_mount_matrix orientation;
+ /* lock to protect against multiple access to the device */
+ struct mutex lock;
};
enum ITG3200_SCAN_INDEX {
* checked by device drivers but should be considered
* read-only as this is a core internal bit
* @driver_module: used to make it harder to undercut users
+ * @mlock: lock used to prevent simultaneous device state changes
* @mlock_key: lockdep class for iio_dev lock
* @info_exist_lock: lock to prevent use during removal
* @trig_readonly: mark the current trigger immutable
int currentmode;
int id;
struct module *driver_module;
+ struct mutex mlock;
struct lock_class_key mlock_key;
struct mutex info_exist_lock;
bool trig_readonly;
* and owner
* @buffer: [DRIVER] any buffer present
* @scan_bytes: [INTERN] num bytes captured to be fed to buffer demux
- * @mlock: [INTERN] lock used to prevent simultaneous device state
- * changes
* @available_scan_masks: [DRIVER] optional array of allowed bitmasks
* @masklength: [INTERN] the length of the mask established from
* channels
struct iio_buffer *buffer;
int scan_bytes;
- struct mutex mlock;
const unsigned long *available_scan_masks;
unsigned masklength;
int iio_push_event(struct iio_dev *indio_dev, u64 ev_code, s64 timestamp);
int iio_device_claim_direct_mode(struct iio_dev *indio_dev);
void iio_device_release_direct_mode(struct iio_dev *indio_dev);
+int iio_device_claim_buffer_mode(struct iio_dev *indio_dev);
+void iio_device_release_buffer_mode(struct iio_dev *indio_dev);
extern struct bus_type iio_bus_type;
__adis_update_bits_base(adis, reg, mask, val, sizeof(val)); \
})
-int adis_enable_irq(struct adis *adis, bool enable);
int __adis_check_status(struct adis *adis);
int __adis_initial_startup(struct adis *adis);
+int __adis_enable_irq(struct adis *adis, bool enable);
-static inline int adis_check_status(struct adis *adis)
+static inline int adis_enable_irq(struct adis *adis, bool enable)
{
int ret;
mutex_lock(&adis->state_lock);
- ret = __adis_check_status(adis);
+ ret = __adis_enable_irq(adis, enable);
mutex_unlock(&adis->state_lock);
return ret;
}
-/* locked version of __adis_initial_startup() */
-static inline int adis_initial_startup(struct adis *adis)
+static inline int adis_check_status(struct adis *adis)
{
int ret;
mutex_lock(&adis->state_lock);
- ret = __adis_initial_startup(adis);
+ ret = __adis_check_status(adis);
mutex_unlock(&adis->state_lock);
return ret;
struct iio_buffer;
struct iio_buffer_setup_ops;
struct iio_dev;
+struct iio_dev_attr;
struct device;
struct iio_buffer *iio_kfifo_allocate(void);
int devm_iio_kfifo_buffer_setup_ext(struct device *dev,
struct iio_dev *indio_dev,
const struct iio_buffer_setup_ops *setup_ops,
- const struct attribute **buffer_attrs);
+ const struct iio_dev_attr **buffer_attrs);
#define devm_iio_kfifo_buffer_setup(dev, indio_dev, setup_ops) \
devm_iio_kfifo_buffer_setup_ext((dev), (indio_dev), (setup_ops), NULL)
= { .string = _string, \
.dev_attr = __ATTR(_name, S_IRUGO, iio_read_const_attr, NULL)}
+#define IIO_STATIC_CONST_DEVICE_ATTR(_name, _string) \
+ static ssize_t iio_const_dev_attr_show_##_name( \
+ struct device *dev, \
+ struct device_attribute *attr, \
+ char *buf) \
+ { \
+ return sysfs_emit(buf, "%s\n", _string); \
+ } \
+ static IIO_DEVICE_ATTR(_name, 0444, \
+ iio_const_dev_attr_show_##_name, NULL, 0)
+
/* Generic attributes of onetype or another */
/**
#include <linux/iio/buffer.h>
#include <linux/interrupt.h>
-struct attribute;
struct iio_dev;
+struct iio_dev_attr;
struct iio_buffer_setup_ops;
int iio_triggered_buffer_setup_ext(struct iio_dev *indio_dev,
irqreturn_t (*thread)(int irq, void *p),
enum iio_buffer_direction direction,
const struct iio_buffer_setup_ops *setup_ops,
- const struct attribute **buffer_attrs);
+ const struct iio_dev_attr **buffer_attrs);
void iio_triggered_buffer_cleanup(struct iio_dev *indio_dev);
#define iio_triggered_buffer_setup(indio_dev, h, thread, setup_ops) \
irqreturn_t (*thread)(int irq, void *p),
enum iio_buffer_direction direction,
const struct iio_buffer_setup_ops *ops,
- const struct attribute **buffer_attrs);
+ const struct iio_dev_attr **buffer_attrs);
#define devm_iio_triggered_buffer_setup(dev, indio_dev, h, thread, setup_ops) \
devm_iio_triggered_buffer_setup_ext((dev), (indio_dev), (h), (thread), \
/**
* struct io_pgtable_ops - Page table manipulation API for IOMMU drivers.
*
- * @map: Map a physically contiguous memory region.
* @map_pages: Map a physically contiguous range of pages of the same size.
- * @unmap: Unmap a physically contiguous memory region.
* @unmap_pages: Unmap a range of virtually contiguous pages of the same size.
* @iova_to_phys: Translate iova to physical address.
*
* the same names.
*/
struct io_pgtable_ops {
- int (*map)(struct io_pgtable_ops *ops, unsigned long iova,
- phys_addr_t paddr, size_t size, int prot, gfp_t gfp);
int (*map_pages)(struct io_pgtable_ops *ops, unsigned long iova,
phys_addr_t paddr, size_t pgsize, size_t pgcount,
int prot, gfp_t gfp, size_t *mapped);
- size_t (*unmap)(struct io_pgtable_ops *ops, unsigned long iova,
- size_t size, struct iommu_iotlb_gather *gather);
size_t (*unmap_pages)(struct io_pgtable_ops *ops, unsigned long iova,
size_t pgsize, size_t pgcount,
struct iommu_iotlb_gather *gather);
/* helpers to define resources */
#define DEFINE_RES_NAMED(_start, _size, _name, _flags) \
- { \
+(struct resource) { \
.start = (_start), \
.end = (_start) + (_size) - 1, \
.name = (_name), \
struct kobject *kobj);
extern void kobject_put(struct kobject *kobj);
-extern const void *kobject_namespace(struct kobject *kobj);
-extern void kobject_get_ownership(struct kobject *kobj,
+extern const void *kobject_namespace(const struct kobject *kobj);
+extern void kobject_get_ownership(const struct kobject *kobj,
kuid_t *uid, kgid_t *gid);
-extern char *kobject_get_path(struct kobject *kobj, gfp_t flag);
+extern char *kobject_get_path(const struct kobject *kobj, gfp_t flag);
struct kobj_type {
void (*release)(struct kobject *kobj);
const struct sysfs_ops *sysfs_ops;
const struct attribute_group **default_groups;
- const struct kobj_ns_type_operations *(*child_ns_type)(struct kobject *kobj);
- const void *(*namespace)(struct kobject *kobj);
- void (*get_ownership)(struct kobject *kobj, kuid_t *uid, kgid_t *gid);
+ const struct kobj_ns_type_operations *(*child_ns_type)(const struct kobject *kobj);
+ const void *(*namespace)(const struct kobject *kobj);
+ void (*get_ownership)(const struct kobject *kobj, kuid_t *uid, kgid_t *gid);
};
struct kobj_uevent_env {
};
struct kset_uevent_ops {
- int (* const filter)(struct kobject *kobj);
- const char *(* const name)(struct kobject *kobj);
+ int (* const filter)(const struct kobject *kobj);
+ const char *(* const name)(const struct kobject *kobj);
int (* const uevent)(struct kobject *kobj, struct kobj_uevent_env *env);
};
kobject_put(&k->kobj);
}
-static inline const struct kobj_type *get_ktype(struct kobject *kobj)
+static inline const struct kobj_type *get_ktype(const struct kobject *kobj)
{
return kobj->ktype;
}
int kobj_ns_type_register(const struct kobj_ns_type_operations *ops);
int kobj_ns_type_registered(enum kobj_ns_type type);
-const struct kobj_ns_type_operations *kobj_child_ns_ops(struct kobject *parent);
-const struct kobj_ns_type_operations *kobj_ns_ops(struct kobject *kobj);
+const struct kobj_ns_type_operations *kobj_child_ns_ops(const struct kobject *parent);
+const struct kobj_ns_type_operations *kobj_ns_ops(const struct kobject *kobj);
bool kobj_ns_current_may_mount(enum kobj_ns_type type);
void *kobj_ns_grab_current(enum kobj_ns_type type);
void *);
extern void mISDN_unregister_clock(struct mISDNclock *);
-static inline struct mISDNdevice *dev_to_mISDN(struct device *dev)
+static inline struct mISDNdevice *dev_to_mISDN(const struct device *dev)
{
if (dev)
return dev_get_drvdata(dev);
+++ /dev/null
-/*
- * dm355evm_msp.h - support MSP430 microcontroller on DM355EVM board
- */
-#ifndef __LINUX_I2C_DM355EVM_MSP
-#define __LINUX_I2C_DM355EVM_MSP
-
-/*
- * Written against Spectrum's writeup for the A4 firmware revision,
- * and tweaked to match source and rev D2 schematics by removing CPLD
- * and NOR flash hooks (which were last appropriate in rev B boards).
- *
- * Note that the firmware supports a flavor of write posting ... to be
- * sure a write completes, issue another read or write.
- */
-
-/* utilities to access "registers" emulated by msp430 firmware */
-extern int dm355evm_msp_write(u8 value, u8 reg);
-extern int dm355evm_msp_read(u8 reg);
-
-
-/* command/control registers */
-#define DM355EVM_MSP_COMMAND 0x00
-# define MSP_COMMAND_NULL 0
-# define MSP_COMMAND_RESET_COLD 1
-# define MSP_COMMAND_RESET_WARM 2
-# define MSP_COMMAND_RESET_WARM_I 3
-# define MSP_COMMAND_POWEROFF 4
-# define MSP_COMMAND_IR_REINIT 5
-#define DM355EVM_MSP_STATUS 0x01
-# define MSP_STATUS_BAD_OFFSET BIT(0)
-# define MSP_STATUS_BAD_COMMAND BIT(1)
-# define MSP_STATUS_POWER_ERROR BIT(2)
-# define MSP_STATUS_RXBUF_OVERRUN BIT(3)
-#define DM355EVM_MSP_RESET 0x02 /* 0 bits == in reset */
-# define MSP_RESET_DC5 BIT(0)
-# define MSP_RESET_TVP5154 BIT(2)
-# define MSP_RESET_IMAGER BIT(3)
-# define MSP_RESET_ETHERNET BIT(4)
-# define MSP_RESET_SYS BIT(5)
-# define MSP_RESET_AIC33 BIT(7)
-
-/* GPIO registers ... bit patterns mostly match the source MSP ports */
-#define DM355EVM_MSP_LED 0x03 /* active low (MSP P4) */
-#define DM355EVM_MSP_SWITCH1 0x04 /* (MSP P5, masked) */
-# define MSP_SWITCH1_SW6_1 BIT(0)
-# define MSP_SWITCH1_SW6_2 BIT(1)
-# define MSP_SWITCH1_SW6_3 BIT(2)
-# define MSP_SWITCH1_SW6_4 BIT(3)
-# define MSP_SWITCH1_J1 BIT(4) /* NTSC/PAL */
-# define MSP_SWITCH1_MSP_INT BIT(5) /* active low */
-#define DM355EVM_MSP_SWITCH2 0x05 /* (MSP P6, masked) */
-# define MSP_SWITCH2_SW10 BIT(3)
-# define MSP_SWITCH2_SW11 BIT(4)
-# define MSP_SWITCH2_SW12 BIT(5)
-# define MSP_SWITCH2_SW13 BIT(6)
-# define MSP_SWITCH2_SW14 BIT(7)
-#define DM355EVM_MSP_SDMMC 0x06 /* (MSP P2, masked) */
-# define MSP_SDMMC_0_WP BIT(1)
-# define MSP_SDMMC_0_CD BIT(2) /* active low */
-# define MSP_SDMMC_1_WP BIT(3)
-# define MSP_SDMMC_1_CD BIT(4) /* active low */
-#define DM355EVM_MSP_FIRMREV 0x07 /* not a GPIO (out of order) */
-#define DM355EVM_MSP_VIDEO_IN 0x08 /* (MSP P3, masked) */
-# define MSP_VIDEO_IMAGER BIT(7) /* low == tvp5146 */
-
-/* power supply registers are currently omitted */
-
-/* RTC registers */
-#define DM355EVM_MSP_RTC_0 0x12 /* LSB */
-#define DM355EVM_MSP_RTC_1 0x13
-#define DM355EVM_MSP_RTC_2 0x14
-#define DM355EVM_MSP_RTC_3 0x15 /* MSB */
-
-/* input event queue registers; code == ((HIGH << 8) | LOW) */
-#define DM355EVM_MSP_INPUT_COUNT 0x16 /* decrement by reading LOW */
-#define DM355EVM_MSP_INPUT_HIGH 0x17
-#define DM355EVM_MSP_INPUT_LOW 0x18
-
-#endif /* __LINUX_I2C_DM355EVM_MSP */
#include <linux/regmap.h>
#include <linux/regulator/driver.h>
#include <linux/extcon-provider.h>
-#include <linux/of_gpio.h>
#include <linux/usb/phy_companion.h>
#define PALMAS_NUM_CLIENTS 3
#include <linux/workqueue.h>
#include <linux/regulator/driver.h>
#include <linux/regulator/machine.h>
+#include <linux/pm.h>
#include <linux/power_supply.h>
#include <linux/mfd/pcf50633/backlight.h>
int pcf50633_irq_init(struct pcf50633 *pcf, int irq);
void pcf50633_irq_free(struct pcf50633 *pcf);
-#ifdef CONFIG_PM
-int pcf50633_irq_suspend(struct pcf50633 *pcf);
-int pcf50633_irq_resume(struct pcf50633 *pcf);
-#endif
+extern const struct dev_pm_ops pcf50633_pm;
#endif
#define RN5T618_WATCHDOG_WDOGTIM_S 0
#define RN5T618_PWRIRQ_IR_WDOG BIT(6)
+#define RN5T618_POFFHIS_PWRON BIT(0)
+#define RN5T618_POFFHIS_TSHUT BIT(1)
+#define RN5T618_POFFHIS_VINDET BIT(2)
+#define RN5T618_POFFHIS_IODET BIT(3)
+#define RN5T618_POFFHIS_CPU BIT(4)
+#define RN5T618_POFFHIS_WDG BIT(5)
+#define RN5T618_POFFHIS_DCLIM BIT(6)
+#define RN5T618_POFFHIS_N_OE BIT(7)
+
enum {
RN5T618_DCDC1,
RN5T618_DCDC2,
struct irq_domain *irq_domain;
struct mutex lock; /* IRQ bus lock */
u8 irq_src;
-#ifdef CONFIG_PM
u8 bkp_sysctrl;
u8 bkp_irqoutpin;
-#endif
};
int stmfx_function_enable(struct stmfx *stmfx, u32 func);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Functions to access TPS65219 Power Management IC.
+ *
+ * Copyright (C) 2022 BayLibre Incorporated - https://www.baylibre.com/
+ */
+
+#ifndef MFD_TPS65219_H
+#define MFD_TPS65219_H
+
+#include <linux/bitops.h>
+#include <linux/notifier.h>
+#include <linux/regulator/driver.h>
+
+struct regmap;
+struct regmap_irq_chip_data;
+
+#define TPS65219_1V35 1350000
+#define TPS65219_1V8 1800000
+
+/* TPS chip id list */
+#define TPS65219 0xF0
+
+/* I2C ID for TPS65219 part */
+#define TPS65219_I2C_ID 0x24
+
+/* All register addresses */
+#define TPS65219_REG_TI_DEV_ID 0x00
+#define TPS65219_REG_NVM_ID 0x01
+#define TPS65219_REG_ENABLE_CTRL 0x02
+#define TPS65219_REG_BUCKS_CONFIG 0x03
+#define TPS65219_REG_LDO4_VOUT 0x04
+#define TPS65219_REG_LDO3_VOUT 0x05
+#define TPS65219_REG_LDO2_VOUT 0x06
+#define TPS65219_REG_LDO1_VOUT 0x07
+#define TPS65219_REG_BUCK3_VOUT 0x8
+#define TPS65219_REG_BUCK2_VOUT 0x9
+#define TPS65219_REG_BUCK1_VOUT 0xA
+#define TPS65219_REG_LDO4_SEQUENCE_SLOT 0xB
+#define TPS65219_REG_LDO3_SEQUENCE_SLOT 0xC
+#define TPS65219_REG_LDO2_SEQUENCE_SLOT 0xD
+#define TPS65219_REG_LDO1_SEQUENCE_SLOT 0xE
+#define TPS65219_REG_BUCK3_SEQUENCE_SLOT 0xF
+#define TPS65219_REG_BUCK2_SEQUENCE_SLOT 0x10
+#define TPS65219_REG_BUCK1_SEQUENCE_SLOT 0x11
+#define TPS65219_REG_nRST_SEQUENCE_SLOT 0x12
+#define TPS65219_REG_GPIO_SEQUENCE_SLOT 0x13
+#define TPS65219_REG_GPO2_SEQUENCE_SLOT 0x14
+#define TPS65219_REG_GPO1_SEQUENCE_SLOT 0x15
+#define TPS65219_REG_POWER_UP_SLOT_DURATION_1 0x16
+#define TPS65219_REG_POWER_UP_SLOT_DURATION_2 0x17
+#define TPS65219_REG_POWER_UP_SLOT_DURATION_3 0x18
+#define TPS65219_REG_POWER_UP_SLOT_DURATION_4 0x19
+#define TPS65219_REG_POWER_DOWN_SLOT_DURATION_1 0x1A
+#define TPS65219_REG_POWER_DOWN_SLOT_DURATION_2 0x1B
+#define TPS65219_REG_POWER_DOWN_SLOT_DURATION_3 0x1C
+#define TPS65219_REG_POWER_DOWN_SLOT_DURATION_4 0x1D
+#define TPS65219_REG_GENERAL_CONFIG 0x1E
+#define TPS65219_REG_MFP_1_CONFIG 0x1F
+#define TPS65219_REG_MFP_2_CONFIG 0x20
+#define TPS65219_REG_STBY_1_CONFIG 0x21
+#define TPS65219_REG_STBY_2_CONFIG 0x22
+#define TPS65219_REG_OC_DEGL_CONFIG 0x23
+/* 'sub irq' MASK registers */
+#define TPS65219_REG_INT_MASK_UV 0x24
+#define TPS65219_REG_MASK_CONFIG 0x25
+
+#define TPS65219_REG_I2C_ADDRESS_REG 0x26
+#define TPS65219_REG_USER_GENERAL_NVM_STORAGE 0x27
+#define TPS65219_REG_MANUFACTURING_VER 0x28
+#define TPS65219_REG_MFP_CTRL 0x29
+#define TPS65219_REG_DISCHARGE_CONFIG 0x2A
+/* main irq registers */
+#define TPS65219_REG_INT_SOURCE 0x2B
+/* 'sub irq' registers */
+#define TPS65219_REG_INT_LDO_3_4 0x2C
+#define TPS65219_REG_INT_LDO_1_2 0x2D
+#define TPS65219_REG_INT_BUCK_3 0x2E
+#define TPS65219_REG_INT_BUCK_1_2 0x2F
+#define TPS65219_REG_INT_SYSTEM 0x30
+#define TPS65219_REG_INT_RV 0x31
+#define TPS65219_REG_INT_TIMEOUT_RV_SD 0x32
+#define TPS65219_REG_INT_PB 0x33
+
+#define TPS65219_REG_INT_LDO_3_4_POS 0
+#define TPS65219_REG_INT_LDO_1_2_POS 1
+#define TPS65219_REG_INT_BUCK_3_POS 2
+#define TPS65219_REG_INT_BUCK_1_2_POS 3
+#define TPS65219_REG_INT_SYS_POS 4
+#define TPS65219_REG_INT_RV_POS 5
+#define TPS65219_REG_INT_TO_RV_POS 6
+#define TPS65219_REG_INT_PB_POS 7
+
+#define TPS65219_REG_USER_NVM_CMD 0x34
+#define TPS65219_REG_POWER_UP_STATUS 0x35
+#define TPS65219_REG_SPARE_2 0x36
+#define TPS65219_REG_SPARE_3 0x37
+#define TPS65219_REG_FACTORY_CONFIG_2 0x41
+
+/* Register field definitions */
+#define TPS65219_DEVID_REV_MASK GENMASK(7, 0)
+#define TPS65219_BUCKS_LDOS_VOUT_VSET_MASK GENMASK(5, 0)
+#define TPS65219_BUCKS_UV_THR_SEL_MASK BIT(6)
+#define TPS65219_BUCKS_BW_SEL_MASK BIT(7)
+#define LDO_BYP_SHIFT 6
+#define TPS65219_LDOS_BYP_CONFIG_MASK BIT(LDO_BYP_SHIFT)
+#define TPS65219_LDOS_LSW_CONFIG_MASK BIT(7)
+/* Regulators enable control */
+#define TPS65219_ENABLE_BUCK1_EN_MASK BIT(0)
+#define TPS65219_ENABLE_BUCK2_EN_MASK BIT(1)
+#define TPS65219_ENABLE_BUCK3_EN_MASK BIT(2)
+#define TPS65219_ENABLE_LDO1_EN_MASK BIT(3)
+#define TPS65219_ENABLE_LDO2_EN_MASK BIT(4)
+#define TPS65219_ENABLE_LDO3_EN_MASK BIT(5)
+#define TPS65219_ENABLE_LDO4_EN_MASK BIT(6)
+/* power ON-OFF sequence slot */
+#define TPS65219_BUCKS_LDOS_SEQUENCE_OFF_SLOT_MASK GENMASK(3, 0)
+#define TPS65219_BUCKS_LDOS_SEQUENCE_ON_SLOT_MASK GENMASK(7, 4)
+/* TODO: Not needed, same mapping as TPS65219_ENABLE_REGNAME_EN, factorize */
+#define TPS65219_STBY1_BUCK1_STBY_EN_MASK BIT(0)
+#define TPS65219_STBY1_BUCK2_STBY_EN_MASK BIT(1)
+#define TPS65219_STBY1_BUCK3_STBY_EN_MASK BIT(2)
+#define TPS65219_STBY1_LDO1_STBY_EN_MASK BIT(3)
+#define TPS65219_STBY1_LDO2_STBY_EN_MASK BIT(4)
+#define TPS65219_STBY1_LDO3_STBY_EN_MASK BIT(5)
+#define TPS65219_STBY1_LDO4_STBY_EN_MASK BIT(6)
+/* STBY_2 config */
+#define TPS65219_STBY2_GPO1_STBY_EN_MASK BIT(0)
+#define TPS65219_STBY2_GPO2_STBY_EN_MASK BIT(1)
+#define TPS65219_STBY2_GPIO_STBY_EN_MASK BIT(2)
+/* MFP Control */
+#define TPS65219_MFP_I2C_OFF_REQ_MASK BIT(0)
+#define TPS65219_MFP_STBY_I2C_CTRL_MASK BIT(1)
+#define TPS65219_MFP_COLD_RESET_I2C_CTRL_MASK BIT(2)
+#define TPS65219_MFP_WARM_RESET_I2C_CTRL_MASK BIT(3)
+#define TPS65219_MFP_GPIO_STATUS_MASK BIT(4)
+/* MFP_1 Config */
+#define TPS65219_MFP_1_VSEL_DDR_SEL_MASK BIT(0)
+#define TPS65219_MFP_1_VSEL_SD_POL_MASK BIT(1)
+#define TPS65219_MFP_1_VSEL_RAIL_MASK BIT(2)
+/* MFP_2 Config */
+#define TPS65219_MFP_2_MODE_STBY_MASK GENMASK(1, 0)
+#define TPS65219_MFP_2_MODE_RESET_MASK BIT(2)
+#define TPS65219_MFP_2_EN_PB_VSENSE_DEGL_MASK BIT(3)
+#define TPS65219_MFP_2_EN_PB_VSENSE_MASK GENMASK(5, 4)
+#define TPS65219_MFP_2_WARM_COLD_RESET_MASK BIT(6)
+#define TPS65219_MFP_2_PU_ON_FSD_MASK BIT(7)
+#define TPS65219_MFP_2_EN 0
+#define TPS65219_MFP_2_PB BIT(4)
+#define TPS65219_MFP_2_VSENSE BIT(5)
+/* MASK_UV Config */
+#define TPS65219_REG_MASK_UV_LDO1_UV_MASK BIT(0)
+#define TPS65219_REG_MASK_UV_LDO2_UV_MASK BIT(1)
+#define TPS65219_REG_MASK_UV_LDO3_UV_MASK BIT(2)
+#define TPS65219_REG_MASK_UV_LDO4_UV_MASK BIT(3)
+#define TPS65219_REG_MASK_UV_BUCK1_UV_MASK BIT(4)
+#define TPS65219_REG_MASK_UV_BUCK2_UV_MASK BIT(5)
+#define TPS65219_REG_MASK_UV_BUCK3_UV_MASK BIT(6)
+#define TPS65219_REG_MASK_UV_RETRY_MASK BIT(7)
+/* MASK Config */
+// SENSOR_N_WARM_MASK already defined in Thermal
+#define TPS65219_REG_MASK_INT_FOR_RV_MASK BIT(4)
+#define TPS65219_REG_MASK_EFFECT_MASK GENMASK(2, 1)
+#define TPS65219_REG_MASK_INT_FOR_PB_MASK BIT(7)
+/* UnderVoltage - Short to GND - OverCurrent*/
+/* LDO3-4 */
+#define TPS65219_INT_LDO3_SCG_MASK BIT(0)
+#define TPS65219_INT_LDO3_OC_MASK BIT(1)
+#define TPS65219_INT_LDO3_UV_MASK BIT(2)
+#define TPS65219_INT_LDO4_SCG_MASK BIT(3)
+#define TPS65219_INT_LDO4_OC_MASK BIT(4)
+#define TPS65219_INT_LDO4_UV_MASK BIT(5)
+/* LDO1-2 */
+#define TPS65219_INT_LDO1_SCG_MASK BIT(0)
+#define TPS65219_INT_LDO1_OC_MASK BIT(1)
+#define TPS65219_INT_LDO1_UV_MASK BIT(2)
+#define TPS65219_INT_LDO2_SCG_MASK BIT(3)
+#define TPS65219_INT_LDO2_OC_MASK BIT(4)
+#define TPS65219_INT_LDO2_UV_MASK BIT(5)
+/* BUCK3 */
+#define TPS65219_INT_BUCK3_SCG_MASK BIT(0)
+#define TPS65219_INT_BUCK3_OC_MASK BIT(1)
+#define TPS65219_INT_BUCK3_NEG_OC_MASK BIT(2)
+#define TPS65219_INT_BUCK3_UV_MASK BIT(3)
+/* BUCK1-2 */
+#define TPS65219_INT_BUCK1_SCG_MASK BIT(0)
+#define TPS65219_INT_BUCK1_OC_MASK BIT(1)
+#define TPS65219_INT_BUCK1_NEG_OC_MASK BIT(2)
+#define TPS65219_INT_BUCK1_UV_MASK BIT(3)
+#define TPS65219_INT_BUCK2_SCG_MASK BIT(4)
+#define TPS65219_INT_BUCK2_OC_MASK BIT(5)
+#define TPS65219_INT_BUCK2_NEG_OC_MASK BIT(6)
+#define TPS65219_INT_BUCK2_UV_MASK BIT(7)
+/* Thermal Sensor */
+#define TPS65219_INT_SENSOR_3_WARM_MASK BIT(0)
+#define TPS65219_INT_SENSOR_2_WARM_MASK BIT(1)
+#define TPS65219_INT_SENSOR_1_WARM_MASK BIT(2)
+#define TPS65219_INT_SENSOR_0_WARM_MASK BIT(3)
+#define TPS65219_INT_SENSOR_3_HOT_MASK BIT(4)
+#define TPS65219_INT_SENSOR_2_HOT_MASK BIT(5)
+#define TPS65219_INT_SENSOR_1_HOT_MASK BIT(6)
+#define TPS65219_INT_SENSOR_0_HOT_MASK BIT(7)
+/* Residual Voltage */
+#define TPS65219_INT_BUCK1_RV_MASK BIT(0)
+#define TPS65219_INT_BUCK2_RV_MASK BIT(1)
+#define TPS65219_INT_BUCK3_RV_MASK BIT(2)
+#define TPS65219_INT_LDO1_RV_MASK BIT(3)
+#define TPS65219_INT_LDO2_RV_MASK BIT(4)
+#define TPS65219_INT_LDO3_RV_MASK BIT(5)
+#define TPS65219_INT_LDO4_RV_MASK BIT(6)
+/* Residual Voltage ShutDown */
+#define TPS65219_INT_BUCK1_RV_SD_MASK BIT(0)
+#define TPS65219_INT_BUCK2_RV_SD_MASK BIT(1)
+#define TPS65219_INT_BUCK3_RV_SD_MASK BIT(2)
+#define TPS65219_INT_LDO1_RV_SD_MASK BIT(3)
+#define TPS65219_INT_LDO2_RV_SD_MASK BIT(4)
+#define TPS65219_INT_LDO3_RV_SD_MASK BIT(5)
+#define TPS65219_INT_LDO4_RV_SD_MASK BIT(6)
+#define TPS65219_INT_TIMEOUT_MASK BIT(7)
+/* Power Button */
+#define TPS65219_INT_PB_FALLING_EDGE_DETECT_MASK BIT(0)
+#define TPS65219_INT_PB_RISING_EDGE_DETECT_MASK BIT(1)
+#define TPS65219_INT_PB_REAL_TIME_STATUS_MASK BIT(2)
+
+#define TPS65219_PB_POS 7
+#define TPS65219_TO_RV_POS 6
+#define TPS65219_RV_POS 5
+#define TPS65219_SYS_POS 4
+#define TPS65219_BUCK_1_2_POS 3
+#define TPS65219_BUCK_3_POS 2
+#define TPS65219_LDO_1_2_POS 1
+#define TPS65219_LDO_3_4_POS 0
+
+/* IRQs */
+enum {
+ /* LDO3-4 register IRQs */
+ TPS65219_INT_LDO3_SCG,
+ TPS65219_INT_LDO3_OC,
+ TPS65219_INT_LDO3_UV,
+ TPS65219_INT_LDO4_SCG,
+ TPS65219_INT_LDO4_OC,
+ TPS65219_INT_LDO4_UV,
+ /* LDO1-2 */
+ TPS65219_INT_LDO1_SCG,
+ TPS65219_INT_LDO1_OC,
+ TPS65219_INT_LDO1_UV,
+ TPS65219_INT_LDO2_SCG,
+ TPS65219_INT_LDO2_OC,
+ TPS65219_INT_LDO2_UV,
+ /* BUCK3 */
+ TPS65219_INT_BUCK3_SCG,
+ TPS65219_INT_BUCK3_OC,
+ TPS65219_INT_BUCK3_NEG_OC,
+ TPS65219_INT_BUCK3_UV,
+ /* BUCK1-2 */
+ TPS65219_INT_BUCK1_SCG,
+ TPS65219_INT_BUCK1_OC,
+ TPS65219_INT_BUCK1_NEG_OC,
+ TPS65219_INT_BUCK1_UV,
+ TPS65219_INT_BUCK2_SCG,
+ TPS65219_INT_BUCK2_OC,
+ TPS65219_INT_BUCK2_NEG_OC,
+ TPS65219_INT_BUCK2_UV,
+ /* Thermal Sensor */
+ TPS65219_INT_SENSOR_3_WARM,
+ TPS65219_INT_SENSOR_2_WARM,
+ TPS65219_INT_SENSOR_1_WARM,
+ TPS65219_INT_SENSOR_0_WARM,
+ TPS65219_INT_SENSOR_3_HOT,
+ TPS65219_INT_SENSOR_2_HOT,
+ TPS65219_INT_SENSOR_1_HOT,
+ TPS65219_INT_SENSOR_0_HOT,
+ /* Residual Voltage */
+ TPS65219_INT_BUCK1_RV,
+ TPS65219_INT_BUCK2_RV,
+ TPS65219_INT_BUCK3_RV,
+ TPS65219_INT_LDO1_RV,
+ TPS65219_INT_LDO2_RV,
+ TPS65219_INT_LDO3_RV,
+ TPS65219_INT_LDO4_RV,
+ /* Residual Voltage ShutDown */
+ TPS65219_INT_BUCK1_RV_SD,
+ TPS65219_INT_BUCK2_RV_SD,
+ TPS65219_INT_BUCK3_RV_SD,
+ TPS65219_INT_LDO1_RV_SD,
+ TPS65219_INT_LDO2_RV_SD,
+ TPS65219_INT_LDO3_RV_SD,
+ TPS65219_INT_LDO4_RV_SD,
+ TPS65219_INT_TIMEOUT,
+ /* Power Button */
+ TPS65219_INT_PB_FALLING_EDGE_DETECT,
+ TPS65219_INT_PB_RISING_EDGE_DETECT,
+};
+
+enum tps65219_regulator_id {
+ /* DCDC's */
+ TPS65219_BUCK_1,
+ TPS65219_BUCK_2,
+ TPS65219_BUCK_3,
+ /* LDOs */
+ TPS65219_LDO_1,
+ TPS65219_LDO_2,
+ TPS65219_LDO_3,
+ TPS65219_LDO_4,
+};
+
+/* Number of step-down converters available */
+#define TPS65219_NUM_DCDC 3
+/* Number of LDO voltage regulators available */
+#define TPS65219_NUM_LDO 4
+/* Number of total regulators available */
+#define TPS65219_NUM_REGULATOR (TPS65219_NUM_DCDC + TPS65219_NUM_LDO)
+
+/* Define the TPS65219 IRQ numbers */
+enum tps65219_irqs {
+ /* INT source registers */
+ TPS65219_TO_RV_SD_SET_IRQ,
+ TPS65219_RV_SET_IRQ,
+ TPS65219_SYS_SET_IRQ,
+ TPS65219_BUCK_1_2_SET_IRQ,
+ TPS65219_BUCK_3_SET_IRQ,
+ TPS65219_LDO_1_2_SET_IRQ,
+ TPS65219_LDO_3_4_SET_IRQ,
+ TPS65219_PB_SET_IRQ,
+};
+
+/**
+ * struct tps65219 - tps65219 sub-driver chip access routines
+ *
+ * Device data may be used to access the TPS65219 chip
+ *
+ * @dev: MFD device
+ * @regmap: Regmap for accessing the device registers
+ * @irq_data: Regmap irq data used for the irq chip
+ * @nb: notifier block for the restart handler
+ */
+struct tps65219 {
+ struct device *dev;
+ struct regmap *regmap;
+
+ struct regmap_irq_chip_data *irq_data;
+ struct notifier_block nb;
+};
+
+#endif /* MFD_TPS65219_H */
#define TWL6040_GPO_MAX 3
-/* TODO: All platform data struct can be removed */
-struct twl6040_codec_data {
- u16 hs_left_step;
- u16 hs_right_step;
- u16 hf_left_step;
- u16 hf_right_step;
-};
-
-struct twl6040_vibra_data {
- unsigned int vibldrv_res; /* left driver resistance */
- unsigned int vibrdrv_res; /* right driver resistance */
- unsigned int viblmotor_res; /* left motor resistance */
- unsigned int vibrmotor_res; /* right motor resistance */
- int vddvibl_uV; /* VDDVIBL volt, set 0 for fixed reg */
- int vddvibr_uV; /* VDDVIBR volt, set 0 for fixed reg */
-};
-
-struct twl6040_gpo_data {
- int gpio_base;
-};
-
-struct twl6040_platform_data {
- int audpwron_gpio; /* audio power-on gpio */
-
- struct twl6040_codec_data *codec;
- struct twl6040_vibra_data *vibra;
- struct twl6040_gpo_data *gpo;
-};
-
+struct gpio_desc;
struct regmap;
struct regmap_irq_chips_data;
struct mfd_cell cells[TWL6040_CELLS];
struct completion ready;
- int audpwron;
+ struct gpio_desc *audpwron;
int power_count;
int rev;
return vfsgid_valid(vfsgid) && __vfsgid_val(vfsgid) == __kgid_val(kgid);
}
-static inline bool vfsuid_gt_kuid(vfsuid_t vfsuid, kuid_t kuid)
-{
- return __vfsuid_val(vfsuid) > __kuid_val(kuid);
-}
-
-static inline bool vfsgid_gt_kgid(vfsgid_t vfsgid, kgid_t kgid)
-{
- return __vfsgid_val(vfsgid) > __kgid_val(kgid);
-}
-
-static inline bool vfsuid_lt_kuid(vfsuid_t vfsuid, kuid_t kuid)
-{
- return __vfsuid_val(vfsuid) < __kuid_val(kuid);
-}
-
-static inline bool vfsgid_lt_kgid(vfsgid_t vfsgid, kgid_t kgid)
-{
- return __vfsgid_val(vfsgid) < __kgid_val(kgid);
-}
-
/*
* vfs{g,u}ids are created from k{g,u}ids.
* We don't allow them to be created from regular {u,g}id.
* must be implemented by each architecture.
*/
+/* arch may override to do additional checking of ELF header architecture */
+bool module_elf_check_arch(Elf_Ehdr *hdr);
+
/* Adjust arch-specific sections. Return 0 on success. */
int module_frob_arch_sections(Elf_Ehdr *hdr,
Elf_Shdr *sechdrs,
extern ssize_t of_device_modalias(struct device *dev, char *str, ssize_t len);
extern int of_device_request_module(struct device *dev);
-extern void of_device_uevent(struct device *dev, struct kobj_uevent_env *env);
+extern void of_device_uevent(const struct device *dev, struct kobj_uevent_env *env);
extern int of_device_uevent_modalias(struct device *dev, struct kobj_uevent_env *env);
static inline struct device_node *of_cpu_device_node_get(int cpu)
return 0;
}
-static inline void of_device_uevent(struct device *dev,
+static inline void of_device_uevent(const struct device *dev,
struct kobj_uevent_env *env) { }
static inline const void *of_device_get_match_data(const struct device *dev)
ptep_get_and_clear(mm, addr, ptep);
}
-#ifndef __HAVE_ARCH_PTEP_GET
+#ifndef ptep_get
static inline pte_t ptep_get(pte_t *ptep)
{
return READ_ONCE(*ptep);
}
#endif
-#ifdef CONFIG_GUP_GET_PTE_LOW_HIGH
+#ifndef pmdp_get
+static inline pmd_t pmdp_get(pmd_t *pmdp)
+{
+ return READ_ONCE(*pmdp);
+}
+#endif
+
+#ifdef CONFIG_GUP_GET_PXX_LOW_HIGH
/*
- * WARNING: only to be used in the get_user_pages_fast() implementation.
- *
- * With get_user_pages_fast(), we walk down the pagetables without taking any
- * locks. For this we would like to load the pointers atomically, but sometimes
- * that is not possible (e.g. without expensive cmpxchg8b on x86_32 PAE). What
- * we do have is the guarantee that a PTE will only either go from not present
- * to present, or present to not present or both -- it will not switch to a
- * completely different present page without a TLB flush in between; something
- * that we are blocking by holding interrupts off.
+ * For walking the pagetables without holding any locks. Some architectures
+ * (eg x86-32 PAE) cannot load the entries atomically without using expensive
+ * instructions. We are guaranteed that a PTE will only either go from not
+ * present to present, or present to not present -- it will not switch to a
+ * completely different present page without a TLB flush inbetween; which we
+ * are blocking by holding interrupts off.
*
* Setting ptes from not present to present goes:
*
return pte;
}
-#else /* CONFIG_GUP_GET_PTE_LOW_HIGH */
+#define ptep_get_lockless ptep_get_lockless
+
+#if CONFIG_PGTABLE_LEVELS > 2
+static inline pmd_t pmdp_get_lockless(pmd_t *pmdp)
+{
+ pmd_t pmd;
+
+ do {
+ pmd.pmd_low = pmdp->pmd_low;
+ smp_rmb();
+ pmd.pmd_high = pmdp->pmd_high;
+ smp_rmb();
+ } while (unlikely(pmd.pmd_low != pmdp->pmd_low));
+
+ return pmd;
+}
+#define pmdp_get_lockless pmdp_get_lockless
+#endif /* CONFIG_PGTABLE_LEVELS > 2 */
+#endif /* CONFIG_GUP_GET_PXX_LOW_HIGH */
+
/*
* We require that the PTE can be read atomically.
*/
+#ifndef ptep_get_lockless
static inline pte_t ptep_get_lockless(pte_t *ptep)
{
return ptep_get(ptep);
}
-#endif /* CONFIG_GUP_GET_PTE_LOW_HIGH */
+#endif
+
+#ifndef pmdp_get_lockless
+static inline pmd_t pmdp_get_lockless(pmd_t *pmdp)
+{
+ return pmdp_get(pmdp);
+}
+#endif
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
#ifndef __HAVE_ARCH_PMDP_HUGE_GET_AND_CLEAR
#endif
}
-#ifndef pmd_read_atomic
-static inline pmd_t pmd_read_atomic(pmd_t *pmdp)
-{
- /*
- * Depend on compiler for an atomic pmd read. NOTE: this is
- * only going to work, if the pmdval_t isn't larger than
- * an unsigned long.
- */
- return *pmdp;
-}
-#endif
-
#ifndef arch_needs_pgtable_deposit
#define arch_needs_pgtable_deposit() (false)
#endif
*/
static inline int pmd_none_or_trans_huge_or_clear_bad(pmd_t *pmd)
{
- pmd_t pmdval = pmd_read_atomic(pmd);
+ pmd_t pmdval = pmdp_get_lockless(pmd);
/*
* The barrier will stabilize the pmdval in a register or on
* the stack so that it will stop changing under the code.
*
* When CONFIG_TRANSPARENT_HUGEPAGE=y on x86 32bit PAE,
- * pmd_read_atomic is allowed to return a not atomic pmdval
+ * pmdp_get_lockless is allowed to return a not atomic pmdval
* (for example pointing to an hugepage that has never been
* mapped in the pmd). The below checks will only care about
* the low part of the pmd with 32bit PAE x86 anyway, with the
#define prandom_init_once(pcpu_state) \
DO_ONCE(prandom_seed_full_state, (pcpu_state))
-/* Deprecated: use get_random_u32_below() instead. */
-static inline u32 prandom_u32_max(u32 ep_ro)
-{
- return get_random_u32_below(ep_ro);
-}
-
/*
* Handle minimum values for seeds
*/
#include <linux/bits.h>
#include <linux/fwnode.h>
+#include <linux/stddef.h>
#include <linux/types.h>
struct device;
DEV_DMA_COHERENT,
};
-struct fwnode_handle *dev_fwnode(const struct device *dev);
+const struct fwnode_handle *__dev_fwnode_const(const struct device *dev);
+struct fwnode_handle *__dev_fwnode(struct device *dev);
+#define dev_fwnode(dev) \
+ _Generic((dev), \
+ const struct device *: __dev_fwnode_const, \
+ struct device *: __dev_fwnode)(dev)
bool device_property_present(struct device *dev, const char *propname);
int device_property_read_u8_array(struct device *dev, const char *propname,
for (child = fwnode_get_next_available_child_node(fwnode, NULL); child;\
child = fwnode_get_next_available_child_node(fwnode, child))
-struct fwnode_handle *device_get_next_child_node(
- struct device *dev, struct fwnode_handle *child);
+struct fwnode_handle *device_get_next_child_node(const struct device *dev,
+ struct fwnode_handle *child);
#define device_for_each_child_node(dev, child) \
for (child = device_get_next_child_node(dev, NULL); child; \
child = device_get_next_child_node(dev, child))
-struct fwnode_handle *fwnode_get_named_child_node(
- const struct fwnode_handle *fwnode, const char *childname);
-struct fwnode_handle *device_get_named_child_node(struct device *dev,
+struct fwnode_handle *fwnode_get_named_child_node(const struct fwnode_handle *fwnode,
+ const char *childname);
+struct fwnode_handle *device_get_named_child_node(const struct device *dev,
const char *childname);
struct fwnode_handle *fwnode_handle_get(struct fwnode_handle *fwnode);
int fwnode_irq_get(const struct fwnode_handle *fwnode, unsigned int index);
int fwnode_irq_get_byname(const struct fwnode_handle *fwnode, const char *name);
-unsigned int device_get_child_node_count(struct device *dev);
+unsigned int device_get_child_node_count(const struct device *dev);
static inline bool device_property_read_bool(struct device *dev,
const char *propname)
* crafted to avoid gcc-4.4.4's problems with initialization of anon unions
* and structs.
*/
-
-#define __PROPERTY_ENTRY_ELEMENT_SIZE(_elem_) \
- sizeof(((struct property_entry *)NULL)->value._elem_[0])
-
-#define __PROPERTY_ENTRY_ARRAY_ELSIZE_LEN(_name_, _elsize_, _Type_, \
- _val_, _len_) \
-(struct property_entry) { \
- .name = _name_, \
- .length = (_len_) * (_elsize_), \
- .type = DEV_PROP_##_Type_, \
- { .pointer = _val_ }, \
+#define __PROPERTY_ENTRY_ARRAY_LEN(_name_, _elem_, _Type_, _val_, _len_) \
+(struct property_entry) { \
+ .name = _name_, \
+ .length = (_len_) * sizeof_field(struct property_entry, value._elem_[0]), \
+ .type = DEV_PROP_##_Type_, \
+ { .pointer = _val_ }, \
}
-#define __PROPERTY_ENTRY_ARRAY_LEN(_name_, _elem_, _Type_, _val_, _len_)\
- __PROPERTY_ENTRY_ARRAY_ELSIZE_LEN(_name_, \
- __PROPERTY_ENTRY_ELEMENT_SIZE(_elem_), \
- _Type_, _val_, _len_)
-
#define PROPERTY_ENTRY_U8_ARRAY_LEN(_name_, _val_, _len_) \
__PROPERTY_ENTRY_ARRAY_LEN(_name_, u8_data, U8, _val_, _len_)
#define PROPERTY_ENTRY_U16_ARRAY_LEN(_name_, _val_, _len_) \
__PROPERTY_ENTRY_ARRAY_LEN(_name_, u64_data, U64, _val_, _len_)
#define PROPERTY_ENTRY_STRING_ARRAY_LEN(_name_, _val_, _len_) \
__PROPERTY_ENTRY_ARRAY_LEN(_name_, str, STRING, _val_, _len_)
+
#define PROPERTY_ENTRY_REF_ARRAY_LEN(_name_, _val_, _len_) \
- __PROPERTY_ENTRY_ARRAY_ELSIZE_LEN(_name_, \
- sizeof(struct software_node_ref_args), \
- REF, _val_, _len_)
+(struct property_entry) { \
+ .name = _name_, \
+ .length = (_len_) * sizeof(struct software_node_ref_args), \
+ .type = DEV_PROP_REF, \
+ { .pointer = _val_ }, \
+}
#define PROPERTY_ENTRY_U8_ARRAY(_name_, _val_) \
PROPERTY_ENTRY_U8_ARRAY_LEN(_name_, _val_, ARRAY_SIZE(_val_))
PROPERTY_ENTRY_U64_ARRAY_LEN(_name_, _val_, ARRAY_SIZE(_val_))
#define PROPERTY_ENTRY_STRING_ARRAY(_name_, _val_) \
PROPERTY_ENTRY_STRING_ARRAY_LEN(_name_, _val_, ARRAY_SIZE(_val_))
-#define PROPERTY_ENTRY_REF_ARRAY(_name_, _val_) \
+#define PROPERTY_ENTRY_REF_ARRAY(_name_, _val_) \
PROPERTY_ENTRY_REF_ARRAY_LEN(_name_, _val_, ARRAY_SIZE(_val_))
#define __PROPERTY_ENTRY_ELEMENT(_name_, _elem_, _Type_, _val_) \
(struct property_entry) { \
.name = _name_, \
- .length = __PROPERTY_ENTRY_ELEMENT_SIZE(_elem_), \
+ .length = sizeof_field(struct property_entry, value._elem_[0]), \
.is_inline = true, \
.type = DEV_PROP_##_Type_, \
{ .value = { ._elem_[0] = _val_ } }, \
#define PROPERTY_ENTRY_STRING(_name_, _val_) \
__PROPERTY_ENTRY_ELEMENT(_name_, str, STRING, _val_)
-#define PROPERTY_ENTRY_BOOL(_name_) \
-(struct property_entry) { \
- .name = _name_, \
- .is_inline = true, \
-}
-
#define PROPERTY_ENTRY_REF(_name_, _ref_, ...) \
(struct property_entry) { \
.name = _name_, \
{ .pointer = &SOFTWARE_NODE_REFERENCE(_ref_, ##__VA_ARGS__), }, \
}
+#define PROPERTY_ENTRY_BOOL(_name_) \
+(struct property_entry) { \
+ .name = _name_, \
+ .is_inline = true, \
+}
+
struct property_entry *
property_entries_dup(const struct property_entry *properties);
-
void property_entries_free(const struct property_entry *properties);
-bool device_dma_supported(struct device *dev);
-
-enum dev_dma_attr device_get_dma_attr(struct device *dev);
+bool device_dma_supported(const struct device *dev);
+enum dev_dma_attr device_get_dma_attr(const struct device *dev);
const void *device_get_match_data(const struct device *dev);
struct fwnode_handle *fwnode_graph_get_remote_endpoint(
const struct fwnode_handle *fwnode);
-static inline bool fwnode_graph_is_endpoint(struct fwnode_handle *fwnode)
+static inline bool fwnode_graph_is_endpoint(const struct fwnode_handle *fwnode)
{
return fwnode_property_present(fwnode, "remote-endpoint");
}
int fwnode_graph_parse_endpoint(const struct fwnode_handle *fwnode,
struct fwnode_endpoint *endpoint);
-typedef void *(*devcon_match_fn_t)(struct fwnode_handle *fwnode, const char *id,
+typedef void *(*devcon_match_fn_t)(const struct fwnode_handle *fwnode, const char *id,
void *data);
-void *fwnode_connection_find_match(struct fwnode_handle *fwnode,
+void *fwnode_connection_find_match(const struct fwnode_handle *fwnode,
const char *con_id, void *data,
devcon_match_fn_t match);
-static inline void *device_connection_find_match(struct device *dev,
+static inline void *device_connection_find_match(const struct device *dev,
const char *con_id, void *data,
devcon_match_fn_t match)
{
return fwnode_connection_find_match(dev_fwnode(dev), con_id, data, match);
}
-int fwnode_connection_find_matches(struct fwnode_handle *fwnode,
+int fwnode_connection_find_matches(const struct fwnode_handle *fwnode,
const char *con_id, void *data,
devcon_match_fn_t match,
void **matches, unsigned int matches_len);
struct pwm_capture *result, unsigned long timeout);
int (*apply)(struct pwm_chip *chip, struct pwm_device *pwm,
const struct pwm_state *state);
- void (*get_state)(struct pwm_chip *chip, struct pwm_device *pwm,
- struct pwm_state *state);
+ int (*get_state)(struct pwm_chip *chip, struct pwm_device *pwm,
+ struct pwm_state *state);
struct module *owner;
};
*/
#include <linux/prandom.h>
-#include <asm/archrandom.h>
-
#ifdef CONFIG_SMP
int random_prepare_cpu(unsigned int cpu);
int random_online_cpu(unsigned int cpu);
struct ring_buffer_event *ring_buffer_lock_reserve(struct trace_buffer *buffer,
unsigned long length);
-int ring_buffer_unlock_commit(struct trace_buffer *buffer,
- struct ring_buffer_event *event);
+int ring_buffer_unlock_commit(struct trace_buffer *buffer);
int ring_buffer_write(struct trace_buffer *buffer,
unsigned long length, void *data);
void __noreturn do_task_dead(void);
void __noreturn make_task_dead(int signr);
+extern void mm_cache_init(void);
extern void proc_caches_init(void);
extern void fork_init(void);
extern pid_t kernel_clone(struct kernel_clone_args *kargs);
struct task_struct *create_io_thread(int (*fn)(void *), void *arg, int node);
struct task_struct *fork_idle(int);
-struct mm_struct *copy_init_mm(void);
extern pid_t kernel_thread(int (*fn)(void *), void *arg, unsigned long flags);
extern pid_t user_mode_thread(int (*fn)(void *), void *arg, unsigned long flags);
extern long kernel_wait4(pid_t, int __user *, int, struct rusage *);
#include <linux/types.h>
#include <linux/device.h>
+#include <linux/iopoll.h>
#include <linux/uaccess.h>
#include <linux/termios.h>
#include <linux/delay.h>
static inline int serdev_device_wait_for_cts(struct serdev_device *serdev, bool state, int timeout_ms)
{
- unsigned long timeout;
bool signal;
- timeout = jiffies + msecs_to_jiffies(timeout_ms);
- while (time_is_after_jiffies(timeout)) {
- signal = serdev_device_get_cts(serdev);
- if (signal == state)
- return 0;
- usleep_range(1000, 2000);
- }
-
- return -ETIMEDOUT;
+ return readx_poll_timeout(serdev_device_get_cts, serdev, signal, signal == state,
+ 2000, timeout_ms * 1000);
}
static inline int serdev_device_set_rts(struct serdev_device *serdev, bool enable)
void uart_write_wakeup(struct uart_port *port);
+#define __uart_port_tx(uport, ch, tx_ready, put_char, tx_done, for_test, \
+ for_post) \
+({ \
+ struct uart_port *__port = (uport); \
+ struct circ_buf *xmit = &__port->state->xmit; \
+ unsigned int pending; \
+ \
+ for (; (for_test) && (tx_ready); (for_post), __port->icount.tx++) { \
+ if (__port->x_char) { \
+ (ch) = __port->x_char; \
+ (put_char); \
+ __port->x_char = 0; \
+ continue; \
+ } \
+ \
+ if (uart_circ_empty(xmit) || uart_tx_stopped(__port)) \
+ break; \
+ \
+ (ch) = xmit->buf[xmit->tail]; \
+ (put_char); \
+ xmit->tail = (xmit->tail + 1) % UART_XMIT_SIZE; \
+ } \
+ \
+ (tx_done); \
+ \
+ pending = uart_circ_chars_pending(xmit); \
+ if (pending < WAKEUP_CHARS) { \
+ uart_write_wakeup(__port); \
+ \
+ if (pending == 0) \
+ __port->ops->stop_tx(__port); \
+ } \
+ \
+ pending; \
+})
+
+/**
+ * uart_port_tx_limited -- transmit helper for uart_port with count limiting
+ * @port: uart port
+ * @ch: variable to store a character to be written to the HW
+ * @count: a limit of characters to send
+ * @tx_ready: can HW accept more data function
+ * @put_char: function to write a character
+ * @tx_done: function to call after the loop is done
+ *
+ * This helper transmits characters from the xmit buffer to the hardware using
+ * @put_char(). It does so until @count characters are sent and while @tx_ready
+ * evaluates to true.
+ *
+ * Returns: the number of characters in the xmit buffer when done.
+ *
+ * The expression in macro parameters shall be designed as follows:
+ * * **tx_ready:** should evaluate to true if the HW can accept more data to
+ * be sent. This parameter can be %true, which means the HW is always ready.
+ * * **put_char:** shall write @ch to the device of @port.
+ * * **tx_done:** when the write loop is done, this can perform arbitrary
+ * action before potential invocation of ops->stop_tx() happens. If the
+ * driver does not need to do anything, use e.g. ({}).
+ *
+ * For all of them, @port->lock is held, interrupts are locally disabled and
+ * the expressions must not sleep.
+ */
+#define uart_port_tx_limited(port, ch, count, tx_ready, put_char, tx_done) ({ \
+ unsigned int __count = (count); \
+ __uart_port_tx(port, ch, tx_ready, put_char, tx_done, __count, \
+ __count--); \
+})
+
+/**
+ * uart_port_tx -- transmit helper for uart_port
+ * @port: uart port
+ * @ch: variable to store a character to be written to the HW
+ * @tx_ready: can HW accept more data function
+ * @put_char: function to write a character
+ *
+ * See uart_port_tx_limited() for more details.
+ */
+#define uart_port_tx(port, ch, tx_ready, put_char) \
+ __uart_port_tx(port, ch, tx_ready, put_char, ({}), true, ({}))
+
/*
* Baud rate helpers.
*/
static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; }
#endif
+#ifndef set_memory_rox
+static inline int set_memory_rox(unsigned long addr, int numpages)
+{
+ int ret = set_memory_ro(addr, numpages);
+ if (ret)
+ return ret;
+ return set_memory_x(addr, numpages);
+}
+#endif
+
#ifndef CONFIG_ARCH_HAS_SET_DIRECT_MAP
static inline int set_direct_map_invalid_noflush(struct page *page)
{
* struct sdw_intel_res - Soundwire Intel global resource structure,
* typically populated by the DSP driver
*
+ * @hw_ops: abstraction for platform ops
* @count: link count
* @mmio_base: mmio base of SoundWire registers
* @irq: interrupt number
* @alh_base: sdw alh base.
*/
struct sdw_intel_res {
+ const struct sdw_intel_hw_ops *hw_ops;
int count;
void __iomem *mmio_base;
int irq;
#define SDW_INTEL_QUIRK_MASK_BUS_DISABLE BIT(1)
+struct sdw_intel;
+
+/* struct intel_sdw_hw_ops - SoundWire ops for Intel platforms.
+ * @debugfs_init: initialize all debugfs capabilities
+ * @debugfs_exit: close and cleanup debugfs capabilities
+ * @register_dai: read all PDI information and register DAIs
+ * @check_clock_stop: throw error message if clock is not stopped.
+ * @start_bus: normal start
+ * @start_bus_after_reset: start after reset
+ * @start_bus_after_clock_stop: start after mode0 clock stop
+ * @stop_bus: stop all bus
+ * @link_power_up: power-up using chip-specific helpers
+ * @link_power_down: power-down with chip-specific helpers
+ * @shim_check_wake: check if a wake was received
+ * @shim_wake: enable/disable in-band wake management
+ * @pre_bank_switch: helper for bus management
+ * @post_bank_switch: helper for bus management
+ */
+struct sdw_intel_hw_ops {
+ void (*debugfs_init)(struct sdw_intel *sdw);
+ void (*debugfs_exit)(struct sdw_intel *sdw);
+
+ int (*register_dai)(struct sdw_intel *sdw);
+
+ void (*check_clock_stop)(struct sdw_intel *sdw);
+ int (*start_bus)(struct sdw_intel *sdw);
+ int (*start_bus_after_reset)(struct sdw_intel *sdw);
+ int (*start_bus_after_clock_stop)(struct sdw_intel *sdw);
+ int (*stop_bus)(struct sdw_intel *sdw, bool clock_stop);
+
+ int (*link_power_up)(struct sdw_intel *sdw);
+ int (*link_power_down)(struct sdw_intel *sdw);
+
+ int (*shim_check_wake)(struct sdw_intel *sdw);
+ void (*shim_wake)(struct sdw_intel *sdw, bool wake_enable);
+
+ int (*pre_bank_switch)(struct sdw_intel *sdw);
+ int (*post_bank_switch)(struct sdw_intel *sdw);
+};
+
+extern const struct sdw_intel_hw_ops sdw_intel_cnl_hw_ops;
+
#endif
struct trace_event {
struct hlist_node node;
- struct list_head list;
int type;
struct trace_event_functions *funcs;
};
void tracing_record_cmdline(struct task_struct *task);
void tracing_record_tgid(struct task_struct *task);
-int trace_output_call(struct trace_iterator *iter, char *name, char *fmt, ...);
+int trace_output_call(struct trace_iterator *iter, char *name, char *fmt, ...)
+ __printf(3, 4);
struct event_filter;
const void *buf, size_t len, bool ascii);
#else /* CONFIG_TRACING */
-static inline void trace_seq_printf(struct trace_seq *s, const char *fmt, ...)
+static inline __printf(2, 3)
+void trace_seq_printf(struct trace_seq *s, const char *fmt, ...)
{
}
static inline void
int commit;
int lookahead; /* Lazy update on recv, can become less than "read" */
int read;
- int flags;
+ bool flags;
/* Data points here */
unsigned long data[];
};
-/* Values for .flags field of tty_buffer */
-#define TTYB_NORMAL 1 /* buffer has no flags buffer */
-
static inline unsigned char *char_buf_ptr(struct tty_buffer *b, int ofs)
{
return ((unsigned char *)b->data) + ofs;
struct tty_buffer *tb = port->buf.tail;
int change;
- change = (tb->flags & TTYB_NORMAL) && (flag != TTY_NORMAL);
+ change = !tb->flags && (flag != TTY_NORMAL);
if (!change && tb->used < tb->size) {
- if (~tb->flags & TTYB_NORMAL)
+ if (tb->flags)
*flag_buf_ptr(tb, tb->used) = flag;
*char_buf_ptr(tb, tb->used++) = ch;
return 1;
struct device *usb_dev;
struct work_struct reset_ws; /* for resets in atomic context */
};
-#define to_usb_interface(d) container_of(d, struct usb_interface, dev)
+
+#define to_usb_interface(__dev) container_of_const(__dev, struct usb_interface, dev)
static inline void *usb_get_intfdata(struct usb_interface *intf)
{
return dev_get_drvdata(&intf->dev);
}
+/**
+ * usb_set_intfdata() - associate driver-specific data with the interface
+ * @intf: the usb interface
+ * @data: pointer to the device priv structure or %NULL
+ *
+ * Drivers should use this function in their probe() to associate their
+ * driver-specific data with the usb interface.
+ *
+ * When disconnecting, the core will take care of setting @intf back to %NULL,
+ * so no actions are needed on the driver side. The interface should not be set
+ * to %NULL before all actions completed (e.g. no outsanding URB remaining).
+ */
static inline void usb_set_intfdata(struct usb_interface *intf, void *data)
{
dev_set_drvdata(&intf->dev, data);
u16 hub_delay;
unsigned use_generic_driver:1;
};
-#define to_usb_device(d) container_of(d, struct usb_device, dev)
-static inline struct usb_device *interface_to_usbdev(struct usb_interface *intf)
+#define to_usb_device(__dev) container_of_const(__dev, struct usb_device, dev)
+
+static inline struct usb_device *__intf_to_usbdev(struct usb_interface *intf)
{
return to_usb_device(intf->dev.parent);
}
+static inline const struct usb_device *__intf_to_usbdev_const(const struct usb_interface *intf)
+{
+ return to_usb_device((const struct device *)intf->dev.parent);
+}
+
+#define interface_to_usbdev(intf) \
+ _Generic((intf), \
+ const struct usb_interface *: __intf_to_usbdev_const, \
+ struct usb_interface *: __intf_to_usbdev)(intf)
extern struct usb_device *usb_get_dev(struct usb_device *dev);
extern void usb_put_dev(struct usb_device *dev);
*/
struct usb_class_driver {
char *name;
- char *(*devnode)(struct device *dev, umode_t *mode);
+ char *(*devnode)(const struct device *dev, umode_t *mode);
const struct file_operations *fops;
int minor_base;
};
/* called after entering D0 (etc), before resuming the hub */
int (*pci_resume)(struct usb_hcd *hcd, bool hibernated);
+ /* called just before hibernate final D3 state, allows host to poweroff parts */
+ int (*pci_poweroff_late)(struct usb_hcd *hcd, bool do_wakeup);
+
/* cleanly make HCD stop writing memory and doing I/O */
void (*stop) (struct usb_hcd *hcd);
struct device;
struct usb_power_delivery;
+struct usb_power_delivery_desc;
enum typec_port_type {
TYPEC_PORT_SRC,
enum usb_pd_svdm_ver svdm_version);
int typec_get_negotiated_svdm_version(struct typec_port *port);
+struct usb_power_delivery *typec_partner_usb_power_delivery_register(struct typec_partner *partner,
+ struct usb_power_delivery_desc *desc);
+
int typec_port_set_usb_power_delivery(struct typec_port *port, struct usb_power_delivery *pd);
int typec_partner_set_usb_power_delivery(struct typec_partner *partner,
struct usb_power_delivery *pd);
/* ===== ZSTDLIB_API : control library symbols visibility ===== */
-#define ZSTDLIB_VISIBILITY
-#define ZSTDLIB_API ZSTDLIB_VISIBILITY
+#ifndef ZSTDLIB_VISIBLE
+# if (__GNUC__ >= 4) && !defined(__MINGW32__)
+# define ZSTDLIB_VISIBLE __attribute__ ((visibility ("default")))
+# define ZSTDLIB_HIDDEN __attribute__ ((visibility ("hidden")))
+# else
+# define ZSTDLIB_VISIBLE
+# define ZSTDLIB_HIDDEN
+# endif
+#endif
+#define ZSTDLIB_API ZSTDLIB_VISIBLE
/* *****************************************************************************
/*------ Version ------*/
#define ZSTD_VERSION_MAJOR 1
-#define ZSTD_VERSION_MINOR 4
-#define ZSTD_VERSION_RELEASE 10
+#define ZSTD_VERSION_MINOR 5
+#define ZSTD_VERSION_RELEASE 2
#define ZSTD_VERSION_NUMBER (ZSTD_VERSION_MAJOR *100*100 + ZSTD_VERSION_MINOR *100 + ZSTD_VERSION_RELEASE)
/*! ZSTD_versionNumber() :
#define ZSTD_BLOCKSIZE_MAX (1<<ZSTD_BLOCKSIZELOG_MAX)
-
/* *************************************
* Simple API
***************************************/
* @return : decompressed size of `src` frame content _if known and not empty_, 0 otherwise. */
ZSTDLIB_API unsigned long long ZSTD_getDecompressedSize(const void* src, size_t srcSize);
-/*! ZSTD_findFrameCompressedSize() :
+/*! ZSTD_findFrameCompressedSize() : Requires v1.4.0+
* `src` should point to the start of a ZSTD frame or skippable frame.
* `srcSize` must be >= first frame size
* @return : the compressed size of the first frame starting at `src`,
ZSTDLIB_API size_t ZSTD_compressBound(size_t srcSize); /*!< maximum compressed size in worst case single-pass scenario */
ZSTDLIB_API unsigned ZSTD_isError(size_t code); /*!< tells if a `size_t` function result is an error code */
ZSTDLIB_API const char* ZSTD_getErrorName(size_t code); /*!< provides readable string from an error code */
-ZSTDLIB_API int ZSTD_minCLevel(void); /*!< minimum negative compression level allowed */
+ZSTDLIB_API int ZSTD_minCLevel(void); /*!< minimum negative compression level allowed, requires v1.4.0+ */
ZSTDLIB_API int ZSTD_maxCLevel(void); /*!< maximum compression level available */
+ZSTDLIB_API int ZSTD_defaultCLevel(void); /*!< default compression level, specified by ZSTD_CLEVEL_DEFAULT, requires v1.5.0+ */
/* *************************************
const void* src, size_t srcSize);
-/* *************************************
-* Advanced compression API
-***************************************/
+/* *******************************************
+* Advanced compression API (Requires v1.4.0+)
+**********************************************/
/* API design :
* Parameters are pushed one by one into an existing context,
*
* It's possible to reset all parameters to "default" using ZSTD_CCtx_reset().
*
- * This API supercedes all other "advanced" API entry points in the experimental section.
+ * This API supersedes all other "advanced" API entry points in the experimental section.
* In the future, we expect to remove from experimental API entry points which are redundant with this API.
*/
Only the order (from fast to strong) is guaranteed */
} ZSTD_strategy;
-
typedef enum {
/* compression parameters
* The higher the value of selected strategy, the more complex it is,
* resulting in stronger and slower compression.
* Special: value 0 means "use default strategy". */
-
/* LDM mode parameters */
ZSTD_c_enableLongDistanceMatching=160, /* Enable long distance matching.
* This parameter is designed to improve compression ratio
ZSTD_c_jobSize=401, /* Size of a compression job. This value is enforced only when nbWorkers >= 1.
* Each compression job is completed in parallel, so this value can indirectly impact the nb of active threads.
* 0 means default, which is dynamically determined based on compression parameters.
- * Job size must be a minimum of overlap size, or 1 MB, whichever is largest.
+ * Job size must be a minimum of overlap size, or ZSTDMT_JOBSIZE_MIN (= 512 KB), whichever is largest.
* The minimum size is automatically and transparently enforced. */
ZSTD_c_overlapLog=402, /* Control the overlap size, as a fraction of window size.
* The overlap size is an amount of data reloaded from previous job at the beginning of a new job.
* ZSTD_c_stableOutBuffer
* ZSTD_c_blockDelimiters
* ZSTD_c_validateSequences
+ * ZSTD_c_useBlockSplitter
+ * ZSTD_c_useRowMatchFinder
* Because they are not stable, it's necessary to define ZSTD_STATIC_LINKING_ONLY to access them.
* note : never ever use experimentalParam? names directly;
* also, the enums values themselves are unstable and can still change.
ZSTD_c_experimentalParam9=1006,
ZSTD_c_experimentalParam10=1007,
ZSTD_c_experimentalParam11=1008,
- ZSTD_c_experimentalParam12=1009
+ ZSTD_c_experimentalParam12=1009,
+ ZSTD_c_experimentalParam13=1010,
+ ZSTD_c_experimentalParam14=1011,
+ ZSTD_c_experimentalParam15=1012
} ZSTD_cParameter;
typedef struct {
const void* src, size_t srcSize);
-/* *************************************
-* Advanced decompression API
-***************************************/
+/* *********************************************
+* Advanced decompression API (Requires v1.4.0+)
+************************************************/
/* The advanced API pushes parameters one by one into an existing DCtx context.
* Parameters are sticky, and remain valid for all following frames
: note : multithreaded compression will block to flush as much output as possible. */
} ZSTD_EndDirective;
-/*! ZSTD_compressStream2() :
+/*! ZSTD_compressStream2() : Requires v1.4.0+
* Behaves about the same as ZSTD_compressStream, with additional control on end directive.
* - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*()
* - Compression parameters cannot be changed once compression is started (save a list of exceptions in multi-threading mode)
/* *****************************************************************************
- * This following is a legacy streaming API.
+ * This following is a legacy streaming API, available since v1.0+ .
* It can be replaced by ZSTD_CCtx_reset() and ZSTD_compressStream2().
* It is redundant, but remains fully supported.
- * Advanced parameters and dictionary compression can only be used through the
- * new API.
+ * Streaming in combination with advanced parameters and dictionary compression
+ * can only be used through the new API.
******************************************************************************/
/*!
/*! ZSTD_compress_usingDict() :
* Compression at an explicit compression level using a Dictionary.
* A dictionary can be any arbitrary data segment (also called a prefix),
- * or a buffer with specified information (see dictBuilder/zdict.h).
+ * or a buffer with specified information (see zdict.h).
* Note : This function loads the dictionary, resulting in significant startup delay.
* It's intended for a dictionary used only once.
* Note 2 : When `dict == NULL || dictSize < 8` no dictionary is used. */
* Dictionary helper functions
*******************************/
-/*! ZSTD_getDictID_fromDict() :
+/*! ZSTD_getDictID_fromDict() : Requires v1.4.0+
* Provides the dictID stored within dictionary.
* if @return == 0, the dictionary is not conformant with Zstandard specification.
* It can still be loaded, but as a content-only dictionary. */
ZSTDLIB_API unsigned ZSTD_getDictID_fromDict(const void* dict, size_t dictSize);
-/*! ZSTD_getDictID_fromDDict() :
+/*! ZSTD_getDictID_fromCDict() : Requires v1.5.0+
+ * Provides the dictID of the dictionary loaded into `cdict`.
+ * If @return == 0, the dictionary is not conformant to Zstandard specification, or empty.
+ * Non-conformant dictionaries can still be loaded, but as content-only dictionaries. */
+ZSTDLIB_API unsigned ZSTD_getDictID_fromCDict(const ZSTD_CDict* cdict);
+
+/*! ZSTD_getDictID_fromDDict() : Requires v1.4.0+
* Provides the dictID of the dictionary loaded into `ddict`.
* If @return == 0, the dictionary is not conformant to Zstandard specification, or empty.
* Non-conformant dictionaries can still be loaded, but as content-only dictionaries. */
ZSTDLIB_API unsigned ZSTD_getDictID_fromDDict(const ZSTD_DDict* ddict);
-/*! ZSTD_getDictID_fromFrame() :
+/*! ZSTD_getDictID_fromFrame() : Requires v1.4.0+
* Provides the dictID required to decompressed the frame stored within `src`.
* If @return == 0, the dictID could not be decoded.
* This could for one of the following reasons :
/* *****************************************************************************
- * Advanced dictionary and prefix API
+ * Advanced dictionary and prefix API (Requires v1.4.0+)
*
* This API allows dictionaries to be used with ZSTD_compress2(),
- * ZSTD_compressStream2(), and ZSTD_decompress(). Dictionaries are sticky, and
+ * ZSTD_compressStream2(), and ZSTD_decompressDCtx(). Dictionaries are sticky, and
* only reset with the context is reset with ZSTD_reset_parameters or
* ZSTD_reset_session_and_parameters. Prefixes are single-use.
******************************************************************************/
-/*! ZSTD_CCtx_loadDictionary() :
+/*! ZSTD_CCtx_loadDictionary() : Requires v1.4.0+
* Create an internal CDict from `dict` buffer.
* Decompression will have to use same dictionary.
* @result : 0, or an error code (which can be tested with ZSTD_isError()).
* to precisely select how dictionary content must be interpreted. */
ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary(ZSTD_CCtx* cctx, const void* dict, size_t dictSize);
-/*! ZSTD_CCtx_refCDict() :
+/*! ZSTD_CCtx_refCDict() : Requires v1.4.0+
* Reference a prepared dictionary, to be used for all next compressed frames.
* Note that compression parameters are enforced from within CDict,
* and supersede any compression parameter previously set within CCtx.
* Note 2 : CDict is just referenced, its lifetime must outlive its usage within CCtx. */
ZSTDLIB_API size_t ZSTD_CCtx_refCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict);
-/*! ZSTD_CCtx_refPrefix() :
+/*! ZSTD_CCtx_refPrefix() : Requires v1.4.0+
* Reference a prefix (single-usage dictionary) for next compressed frame.
* A prefix is **only used once**. Tables are discarded at end of frame (ZSTD_e_end).
* Decompression will need same prefix to properly regenerate data.
ZSTDLIB_API size_t ZSTD_CCtx_refPrefix(ZSTD_CCtx* cctx,
const void* prefix, size_t prefixSize);
-/*! ZSTD_DCtx_loadDictionary() :
+/*! ZSTD_DCtx_loadDictionary() : Requires v1.4.0+
* Create an internal DDict from dict buffer,
* to be used to decompress next frames.
* The dictionary remains valid for all future frames, until explicitly invalidated.
*/
ZSTDLIB_API size_t ZSTD_DCtx_loadDictionary(ZSTD_DCtx* dctx, const void* dict, size_t dictSize);
-/*! ZSTD_DCtx_refDDict() :
+/*! ZSTD_DCtx_refDDict() : Requires v1.4.0+
* Reference a prepared dictionary, to be used to decompress next frames.
* The dictionary remains active for decompression of future frames using same DCtx.
*
*/
ZSTDLIB_API size_t ZSTD_DCtx_refDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict);
-/*! ZSTD_DCtx_refPrefix() :
+/*! ZSTD_DCtx_refPrefix() : Requires v1.4.0+
* Reference a prefix (single-usage dictionary) to decompress next frame.
* This is the reverse operation of ZSTD_CCtx_refPrefix(),
* and must use the same prefix as the one used during compression.
/* === Memory management === */
-/*! ZSTD_sizeof_*() :
+/*! ZSTD_sizeof_*() : Requires v1.4.0+
* These functions give the _current_ memory usage of selected object.
* Note that object memory usage can evolve (increase or decrease) over time. */
ZSTDLIB_API size_t ZSTD_sizeof_CCtx(const ZSTD_CCtx* cctx);
#if !defined(ZSTD_H_ZSTD_STATIC_LINKING_ONLY)
#define ZSTD_H_ZSTD_STATIC_LINKING_ONLY
+/* This can be overridden externally to hide static symbols. */
+#ifndef ZSTDLIB_STATIC_API
+#define ZSTDLIB_STATIC_API ZSTDLIB_VISIBLE
+#endif
+
+/* Deprecation warnings :
+ * Should these warnings be a problem, it is generally possible to disable them,
+ * typically with -Wno-deprecated-declarations for gcc or _CRT_SECURE_NO_WARNINGS in Visual.
+ * Otherwise, it's also possible to define ZSTD_DISABLE_DEPRECATE_WARNINGS.
+ */
+#ifdef ZSTD_DISABLE_DEPRECATE_WARNINGS
+# define ZSTD_DEPRECATED(message) ZSTDLIB_STATIC_API /* disable deprecation warnings */
+#else
+# if (defined(GNUC) && (GNUC > 4 || (GNUC == 4 && GNUC_MINOR >= 5))) || defined(__clang__)
+# define ZSTD_DEPRECATED(message) ZSTDLIB_STATIC_API __attribute__((deprecated(message)))
+# elif (__GNUC__ >= 3)
+# define ZSTD_DEPRECATED(message) ZSTDLIB_STATIC_API __attribute__((deprecated))
+# else
+# pragma message("WARNING: You need to implement ZSTD_DEPRECATED for this compiler")
+# define ZSTD_DEPRECATED(message) ZSTDLIB_STATIC_API
+# endif
+#endif /* ZSTD_DISABLE_DEPRECATE_WARNINGS */
+
/* **************************************************************************************
* experimental API (static linking only)
****************************************************************************************
#define ZSTD_SRCSIZEHINT_MIN 0
#define ZSTD_SRCSIZEHINT_MAX INT_MAX
-/* internal */
-#define ZSTD_HASHLOG3_MAX 17
-
/* --- Advanced types --- */
ZSTD_lcm_uncompressed = 2 /*< Always emit uncompressed literals. */
} ZSTD_literalCompressionMode_e;
+typedef enum {
+ /* Note: This enum controls features which are conditionally beneficial. Zstd typically will make a final
+ * decision on whether or not to enable the feature (ZSTD_ps_auto), but setting the switch to ZSTD_ps_enable
+ * or ZSTD_ps_disable allow for a force enable/disable the feature.
+ */
+ ZSTD_ps_auto = 0, /* Let the library automatically determine whether the feature shall be enabled */
+ ZSTD_ps_enable = 1, /* Force-enable the feature */
+ ZSTD_ps_disable = 2 /* Do not use the feature */
+} ZSTD_paramSwitch_e;
/* *************************************
* Frame size functions
* note 5 : ZSTD_findDecompressedSize handles multiple frames, and so it must traverse the input to
* read each contained frame header. This is fast as most of the data is skipped,
* however it does mean that all frame data must be present and valid. */
-ZSTDLIB_API unsigned long long ZSTD_findDecompressedSize(const void* src, size_t srcSize);
+ZSTDLIB_STATIC_API unsigned long long ZSTD_findDecompressedSize(const void* src, size_t srcSize);
/*! ZSTD_decompressBound() :
* `src` should point to the start of a series of ZSTD encoded and/or skippable frames
* note 3 : when the decompressed size field isn't available, the upper-bound for that frame is calculated by:
* upper-bound = # blocks * min(128 KB, Window_Size)
*/
-ZSTDLIB_API unsigned long long ZSTD_decompressBound(const void* src, size_t srcSize);
+ZSTDLIB_STATIC_API unsigned long long ZSTD_decompressBound(const void* src, size_t srcSize);
/*! ZSTD_frameHeaderSize() :
* srcSize must be >= ZSTD_FRAMEHEADERSIZE_PREFIX.
* @return : size of the Frame Header,
* or an error code (if srcSize is too small) */
-ZSTDLIB_API size_t ZSTD_frameHeaderSize(const void* src, size_t srcSize);
+ZSTDLIB_STATIC_API size_t ZSTD_frameHeaderSize(const void* src, size_t srcSize);
typedef enum {
ZSTD_sf_noBlockDelimiters = 0, /* Representation of ZSTD_Sequence has no block delimiters, sequences only */
* @return : number of sequences generated
*/
-ZSTDLIB_API size_t ZSTD_generateSequences(ZSTD_CCtx* zc, ZSTD_Sequence* outSeqs,
+ZSTDLIB_STATIC_API size_t ZSTD_generateSequences(ZSTD_CCtx* zc, ZSTD_Sequence* outSeqs,
size_t outSeqsSize, const void* src, size_t srcSize);
/*! ZSTD_mergeBlockDelimiters() :
* Given an array of ZSTD_Sequence, remove all sequences that represent block delimiters/last literals
- * by merging them into into the literals of the next sequence.
+ * by merging them into the literals of the next sequence.
*
* As such, the final generated result has no explicit representation of block boundaries,
* and the final last literals segment is not represented in the sequences.
* setting of ZSTD_c_blockDelimiters as ZSTD_sf_noBlockDelimiters
* @return : number of sequences left after merging
*/
-ZSTDLIB_API size_t ZSTD_mergeBlockDelimiters(ZSTD_Sequence* sequences, size_t seqsSize);
+ZSTDLIB_STATIC_API size_t ZSTD_mergeBlockDelimiters(ZSTD_Sequence* sequences, size_t seqsSize);
/*! ZSTD_compressSequences() :
* Compress an array of ZSTD_Sequence, generated from the original source buffer, into dst.
* and cannot emit an RLE block that disagrees with the repcode history
* @return : final compressed size or a ZSTD error.
*/
-ZSTDLIB_API size_t ZSTD_compressSequences(ZSTD_CCtx* const cctx, void* dst, size_t dstSize,
+ZSTDLIB_STATIC_API size_t ZSTD_compressSequences(ZSTD_CCtx* const cctx, void* dst, size_t dstSize,
const ZSTD_Sequence* inSeqs, size_t inSeqsSize,
const void* src, size_t srcSize);
/*! ZSTD_writeSkippableFrame() :
* Generates a zstd skippable frame containing data given by src, and writes it to dst buffer.
*
- * Skippable frames begin with a a 4-byte magic number. There are 16 possible choices of magic number,
+ * Skippable frames begin with a 4-byte magic number. There are 16 possible choices of magic number,
* ranging from ZSTD_MAGIC_SKIPPABLE_START to ZSTD_MAGIC_SKIPPABLE_START+15.
* As such, the parameter magicVariant controls the exact skippable frame magic number variant used, so
* the magic number used will be ZSTD_MAGIC_SKIPPABLE_START + magicVariant.
*
* @return : number of bytes written or a ZSTD error.
*/
-ZSTDLIB_API size_t ZSTD_writeSkippableFrame(void* dst, size_t dstCapacity,
+ZSTDLIB_STATIC_API size_t ZSTD_writeSkippableFrame(void* dst, size_t dstCapacity,
const void* src, size_t srcSize, unsigned magicVariant);
+/*! ZSTD_readSkippableFrame() :
+ * Retrieves a zstd skippable frame containing data given by src, and writes it to dst buffer.
+ *
+ * The parameter magicVariant will receive the magicVariant that was supplied when the frame was written,
+ * i.e. magicNumber - ZSTD_MAGIC_SKIPPABLE_START. This can be NULL if the caller is not interested
+ * in the magicVariant.
+ *
+ * Returns an error if destination buffer is not large enough, or if the frame is not skippable.
+ *
+ * @return : number of bytes written or a ZSTD error.
+ */
+ZSTDLIB_API size_t ZSTD_readSkippableFrame(void* dst, size_t dstCapacity, unsigned* magicVariant,
+ const void* src, size_t srcSize);
+
+/*! ZSTD_isSkippableFrame() :
+ * Tells if the content of `buffer` starts with a valid Frame Identifier for a skippable frame.
+ */
+ZSTDLIB_API unsigned ZSTD_isSkippableFrame(const void* buffer, size_t size);
+
+
/* *************************************
* Memory management
* Note 2 : only single-threaded compression is supported.
* ZSTD_estimateCCtxSize_usingCCtxParams() will return an error code if ZSTD_c_nbWorkers is >= 1.
*/
-ZSTDLIB_API size_t ZSTD_estimateCCtxSize(int compressionLevel);
-ZSTDLIB_API size_t ZSTD_estimateCCtxSize_usingCParams(ZSTD_compressionParameters cParams);
-ZSTDLIB_API size_t ZSTD_estimateCCtxSize_usingCCtxParams(const ZSTD_CCtx_params* params);
-ZSTDLIB_API size_t ZSTD_estimateDCtxSize(void);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateCCtxSize(int compressionLevel);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateCCtxSize_usingCParams(ZSTD_compressionParameters cParams);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateCCtxSize_usingCCtxParams(const ZSTD_CCtx_params* params);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateDCtxSize(void);
/*! ZSTD_estimateCStreamSize() :
* ZSTD_estimateCStreamSize() will provide a budget large enough for any compression level up to selected one.
* Note : if streaming is init with function ZSTD_init?Stream_usingDict(),
* an internal ?Dict will be created, which additional size is not estimated here.
* In this case, get total size by adding ZSTD_estimate?DictSize */
-ZSTDLIB_API size_t ZSTD_estimateCStreamSize(int compressionLevel);
-ZSTDLIB_API size_t ZSTD_estimateCStreamSize_usingCParams(ZSTD_compressionParameters cParams);
-ZSTDLIB_API size_t ZSTD_estimateCStreamSize_usingCCtxParams(const ZSTD_CCtx_params* params);
-ZSTDLIB_API size_t ZSTD_estimateDStreamSize(size_t windowSize);
-ZSTDLIB_API size_t ZSTD_estimateDStreamSize_fromFrame(const void* src, size_t srcSize);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateCStreamSize(int compressionLevel);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateCStreamSize_usingCParams(ZSTD_compressionParameters cParams);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateCStreamSize_usingCCtxParams(const ZSTD_CCtx_params* params);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateDStreamSize(size_t windowSize);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateDStreamSize_fromFrame(const void* src, size_t srcSize);
/*! ZSTD_estimate?DictSize() :
* ZSTD_estimateCDictSize() will bet that src size is relatively "small", and content is copied, like ZSTD_createCDict().
* ZSTD_estimateCDictSize_advanced() makes it possible to control compression parameters precisely, like ZSTD_createCDict_advanced().
* Note : dictionaries created by reference (`ZSTD_dlm_byRef`) are logically smaller.
*/
-ZSTDLIB_API size_t ZSTD_estimateCDictSize(size_t dictSize, int compressionLevel);
-ZSTDLIB_API size_t ZSTD_estimateCDictSize_advanced(size_t dictSize, ZSTD_compressionParameters cParams, ZSTD_dictLoadMethod_e dictLoadMethod);
-ZSTDLIB_API size_t ZSTD_estimateDDictSize(size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateCDictSize(size_t dictSize, int compressionLevel);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateCDictSize_advanced(size_t dictSize, ZSTD_compressionParameters cParams, ZSTD_dictLoadMethod_e dictLoadMethod);
+ZSTDLIB_STATIC_API size_t ZSTD_estimateDDictSize(size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod);
/*! ZSTD_initStatic*() :
* Initialize an object using a pre-allocated fixed-size buffer.
* Limitation 2 : static cctx currently not compatible with multi-threading.
* Limitation 3 : static dctx is incompatible with legacy support.
*/
-ZSTDLIB_API ZSTD_CCtx* ZSTD_initStaticCCtx(void* workspace, size_t workspaceSize);
-ZSTDLIB_API ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspaceSize); /*< same as ZSTD_initStaticCCtx() */
+ZSTDLIB_STATIC_API ZSTD_CCtx* ZSTD_initStaticCCtx(void* workspace, size_t workspaceSize);
+ZSTDLIB_STATIC_API ZSTD_CStream* ZSTD_initStaticCStream(void* workspace, size_t workspaceSize); /*< same as ZSTD_initStaticCCtx() */
-ZSTDLIB_API ZSTD_DCtx* ZSTD_initStaticDCtx(void* workspace, size_t workspaceSize);
-ZSTDLIB_API ZSTD_DStream* ZSTD_initStaticDStream(void* workspace, size_t workspaceSize); /*< same as ZSTD_initStaticDCtx() */
+ZSTDLIB_STATIC_API ZSTD_DCtx* ZSTD_initStaticDCtx(void* workspace, size_t workspaceSize);
+ZSTDLIB_STATIC_API ZSTD_DStream* ZSTD_initStaticDStream(void* workspace, size_t workspaceSize); /*< same as ZSTD_initStaticDCtx() */
-ZSTDLIB_API const ZSTD_CDict* ZSTD_initStaticCDict(
+ZSTDLIB_STATIC_API const ZSTD_CDict* ZSTD_initStaticCDict(
void* workspace, size_t workspaceSize,
const void* dict, size_t dictSize,
ZSTD_dictLoadMethod_e dictLoadMethod,
ZSTD_dictContentType_e dictContentType,
ZSTD_compressionParameters cParams);
-ZSTDLIB_API const ZSTD_DDict* ZSTD_initStaticDDict(
+ZSTDLIB_STATIC_API const ZSTD_DDict* ZSTD_initStaticDDict(
void* workspace, size_t workspaceSize,
const void* dict, size_t dictSize,
ZSTD_dictLoadMethod_e dictLoadMethod,
__attribute__((__unused__))
ZSTD_customMem const ZSTD_defaultCMem = { NULL, NULL, NULL }; /*< this constant defers to stdlib's functions */
-ZSTDLIB_API ZSTD_CCtx* ZSTD_createCCtx_advanced(ZSTD_customMem customMem);
-ZSTDLIB_API ZSTD_CStream* ZSTD_createCStream_advanced(ZSTD_customMem customMem);
-ZSTDLIB_API ZSTD_DCtx* ZSTD_createDCtx_advanced(ZSTD_customMem customMem);
-ZSTDLIB_API ZSTD_DStream* ZSTD_createDStream_advanced(ZSTD_customMem customMem);
+ZSTDLIB_STATIC_API ZSTD_CCtx* ZSTD_createCCtx_advanced(ZSTD_customMem customMem);
+ZSTDLIB_STATIC_API ZSTD_CStream* ZSTD_createCStream_advanced(ZSTD_customMem customMem);
+ZSTDLIB_STATIC_API ZSTD_DCtx* ZSTD_createDCtx_advanced(ZSTD_customMem customMem);
+ZSTDLIB_STATIC_API ZSTD_DStream* ZSTD_createDStream_advanced(ZSTD_customMem customMem);
-ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict_advanced(const void* dict, size_t dictSize,
+ZSTDLIB_STATIC_API ZSTD_CDict* ZSTD_createCDict_advanced(const void* dict, size_t dictSize,
ZSTD_dictLoadMethod_e dictLoadMethod,
ZSTD_dictContentType_e dictContentType,
ZSTD_compressionParameters cParams,
ZSTD_customMem customMem);
-/* ! Thread pool :
- * These prototypes make it possible to share a thread pool among multiple compression contexts.
- * This can limit resources for applications with multiple threads where each one uses
- * a threaded compression mode (via ZSTD_c_nbWorkers parameter).
- * ZSTD_createThreadPool creates a new thread pool with a given number of threads.
- * Note that the lifetime of such pool must exist while being used.
- * ZSTD_CCtx_refThreadPool assigns a thread pool to a context (use NULL argument value
- * to use an internal thread pool).
- * ZSTD_freeThreadPool frees a thread pool, accepts NULL pointer.
+/*! Thread pool :
+ * These prototypes make it possible to share a thread pool among multiple compression contexts.
+ * This can limit resources for applications with multiple threads where each one uses
+ * a threaded compression mode (via ZSTD_c_nbWorkers parameter).
+ * ZSTD_createThreadPool creates a new thread pool with a given number of threads.
+ * Note that the lifetime of such pool must exist while being used.
+ * ZSTD_CCtx_refThreadPool assigns a thread pool to a context (use NULL argument value
+ * to use an internal thread pool).
+ * ZSTD_freeThreadPool frees a thread pool, accepts NULL pointer.
*/
typedef struct POOL_ctx_s ZSTD_threadPool;
-ZSTDLIB_API ZSTD_threadPool* ZSTD_createThreadPool(size_t numThreads);
-ZSTDLIB_API void ZSTD_freeThreadPool (ZSTD_threadPool* pool); /* accept NULL pointer */
-ZSTDLIB_API size_t ZSTD_CCtx_refThreadPool(ZSTD_CCtx* cctx, ZSTD_threadPool* pool);
+ZSTDLIB_STATIC_API ZSTD_threadPool* ZSTD_createThreadPool(size_t numThreads);
+ZSTDLIB_STATIC_API void ZSTD_freeThreadPool (ZSTD_threadPool* pool); /* accept NULL pointer */
+ZSTDLIB_STATIC_API size_t ZSTD_CCtx_refThreadPool(ZSTD_CCtx* cctx, ZSTD_threadPool* pool);
/*
* This API is temporary and is expected to change or disappear in the future!
*/
-ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict_advanced2(
+ZSTDLIB_STATIC_API ZSTD_CDict* ZSTD_createCDict_advanced2(
const void* dict, size_t dictSize,
ZSTD_dictLoadMethod_e dictLoadMethod,
ZSTD_dictContentType_e dictContentType,
const ZSTD_CCtx_params* cctxParams,
ZSTD_customMem customMem);
-ZSTDLIB_API ZSTD_DDict* ZSTD_createDDict_advanced(
+ZSTDLIB_STATIC_API ZSTD_DDict* ZSTD_createDDict_advanced(
const void* dict, size_t dictSize,
ZSTD_dictLoadMethod_e dictLoadMethod,
ZSTD_dictContentType_e dictContentType,
* As a consequence, `dictBuffer` **must** outlive CDict,
* and its content must remain unmodified throughout the lifetime of CDict.
* note: equivalent to ZSTD_createCDict_advanced(), with dictLoadMethod==ZSTD_dlm_byRef */
-ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict_byReference(const void* dictBuffer, size_t dictSize, int compressionLevel);
-
-/*! ZSTD_getDictID_fromCDict() :
- * Provides the dictID of the dictionary loaded into `cdict`.
- * If @return == 0, the dictionary is not conformant to Zstandard specification, or empty.
- * Non-conformant dictionaries can still be loaded, but as content-only dictionaries. */
-ZSTDLIB_API unsigned ZSTD_getDictID_fromCDict(const ZSTD_CDict* cdict);
+ZSTDLIB_STATIC_API ZSTD_CDict* ZSTD_createCDict_byReference(const void* dictBuffer, size_t dictSize, int compressionLevel);
/*! ZSTD_getCParams() :
* @return ZSTD_compressionParameters structure for a selected compression level and estimated srcSize.
* `estimatedSrcSize` value is optional, select 0 if not known */
-ZSTDLIB_API ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, unsigned long long estimatedSrcSize, size_t dictSize);
+ZSTDLIB_STATIC_API ZSTD_compressionParameters ZSTD_getCParams(int compressionLevel, unsigned long long estimatedSrcSize, size_t dictSize);
/*! ZSTD_getParams() :
* same as ZSTD_getCParams(), but @return a full `ZSTD_parameters` object instead of sub-component `ZSTD_compressionParameters`.
* All fields of `ZSTD_frameParameters` are set to default : contentSize=1, checksum=0, noDictID=0 */
-ZSTDLIB_API ZSTD_parameters ZSTD_getParams(int compressionLevel, unsigned long long estimatedSrcSize, size_t dictSize);
+ZSTDLIB_STATIC_API ZSTD_parameters ZSTD_getParams(int compressionLevel, unsigned long long estimatedSrcSize, size_t dictSize);
/*! ZSTD_checkCParams() :
* Ensure param values remain within authorized range.
* @return 0 on success, or an error code (can be checked with ZSTD_isError()) */
-ZSTDLIB_API size_t ZSTD_checkCParams(ZSTD_compressionParameters params);
+ZSTDLIB_STATIC_API size_t ZSTD_checkCParams(ZSTD_compressionParameters params);
/*! ZSTD_adjustCParams() :
* optimize params for a given `srcSize` and `dictSize`.
* `dictSize` must be `0` when there is no dictionary.
* cPar can be invalid : all parameters will be clamped within valid range in the @return struct.
* This function never fails (wide contract) */
-ZSTDLIB_API ZSTD_compressionParameters ZSTD_adjustCParams(ZSTD_compressionParameters cPar, unsigned long long srcSize, size_t dictSize);
+ZSTDLIB_STATIC_API ZSTD_compressionParameters ZSTD_adjustCParams(ZSTD_compressionParameters cPar, unsigned long long srcSize, size_t dictSize);
/*! ZSTD_compress_advanced() :
* Note : this function is now DEPRECATED.
* It can be replaced by ZSTD_compress2(), in combination with ZSTD_CCtx_setParameter() and other parameter setters.
- * This prototype will be marked as deprecated and generate compilation warning on reaching v1.5.x */
-ZSTDLIB_API size_t ZSTD_compress_advanced(ZSTD_CCtx* cctx,
+ * This prototype will generate compilation warnings. */
+ZSTD_DEPRECATED("use ZSTD_compress2")
+size_t ZSTD_compress_advanced(ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const void* dict,size_t dictSize,
ZSTD_parameters params);
/*! ZSTD_compress_usingCDict_advanced() :
- * Note : this function is now REDUNDANT.
+ * Note : this function is now DEPRECATED.
* It can be replaced by ZSTD_compress2(), in combination with ZSTD_CCtx_loadDictionary() and other parameter setters.
- * This prototype will be marked as deprecated and generate compilation warning in some future version */
-ZSTDLIB_API size_t ZSTD_compress_usingCDict_advanced(ZSTD_CCtx* cctx,
+ * This prototype will generate compilation warnings. */
+ZSTD_DEPRECATED("use ZSTD_compress2 with ZSTD_CCtx_loadDictionary")
+size_t ZSTD_compress_usingCDict_advanced(ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const ZSTD_CDict* cdict,
/*! ZSTD_CCtx_loadDictionary_byReference() :
* Same as ZSTD_CCtx_loadDictionary(), but dictionary content is referenced, instead of being copied into CCtx.
* It saves some memory, but also requires that `dict` outlives its usage within `cctx` */
-ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary_byReference(ZSTD_CCtx* cctx, const void* dict, size_t dictSize);
+ZSTDLIB_STATIC_API size_t ZSTD_CCtx_loadDictionary_byReference(ZSTD_CCtx* cctx, const void* dict, size_t dictSize);
/*! ZSTD_CCtx_loadDictionary_advanced() :
* Same as ZSTD_CCtx_loadDictionary(), but gives finer control over
* how to load the dictionary (by copy ? by reference ?)
* and how to interpret it (automatic ? force raw mode ? full mode only ?) */
-ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod, ZSTD_dictContentType_e dictContentType);
+ZSTDLIB_STATIC_API size_t ZSTD_CCtx_loadDictionary_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod, ZSTD_dictContentType_e dictContentType);
/*! ZSTD_CCtx_refPrefix_advanced() :
* Same as ZSTD_CCtx_refPrefix(), but gives finer control over
* how to interpret prefix content (automatic ? force raw mode (default) ? full mode only ?) */
-ZSTDLIB_API size_t ZSTD_CCtx_refPrefix_advanced(ZSTD_CCtx* cctx, const void* prefix, size_t prefixSize, ZSTD_dictContentType_e dictContentType);
+ZSTDLIB_STATIC_API size_t ZSTD_CCtx_refPrefix_advanced(ZSTD_CCtx* cctx, const void* prefix, size_t prefixSize, ZSTD_dictContentType_e dictContentType);
/* === experimental parameters === */
/* these parameters can be used with ZSTD_setParameter()
* See the comments on that enum for an explanation of the feature. */
#define ZSTD_c_forceAttachDict ZSTD_c_experimentalParam4
-/* Controls how the literals are compressed (default is auto).
- * The value must be of type ZSTD_literalCompressionMode_e.
- * See ZSTD_literalCompressionMode_t enum definition for details.
+/* Controlled with ZSTD_paramSwitch_e enum.
+ * Default is ZSTD_ps_auto.
+ * Set to ZSTD_ps_disable to never compress literals.
+ * Set to ZSTD_ps_enable to always compress literals. (Note: uncompressed literals
+ * may still be emitted if huffman is not beneficial to use.)
+ *
+ * By default, in ZSTD_ps_auto, the library will decide at runtime whether to use
+ * literals compression based on the compression parameters - specifically,
+ * negative compression levels do not use literal compression.
*/
#define ZSTD_c_literalCompressionMode ZSTD_c_experimentalParam5
*
* Note that this means that the CDict tables can no longer be copied into the
* CCtx, so the dict attachment mode ZSTD_dictForceCopy will no longer be
- * useable. The dictionary can only be attached or reloaded.
+ * usable. The dictionary can only be attached or reloaded.
*
* In general, you should expect compression to be faster--sometimes very much
* so--and CDict creation to be slightly slower. Eventually, we will probably
*/
#define ZSTD_c_validateSequences ZSTD_c_experimentalParam12
+/* ZSTD_c_useBlockSplitter
+ * Controlled with ZSTD_paramSwitch_e enum.
+ * Default is ZSTD_ps_auto.
+ * Set to ZSTD_ps_disable to never use block splitter.
+ * Set to ZSTD_ps_enable to always use block splitter.
+ *
+ * By default, in ZSTD_ps_auto, the library will decide at runtime whether to use
+ * block splitting based on the compression parameters.
+ */
+#define ZSTD_c_useBlockSplitter ZSTD_c_experimentalParam13
+
+/* ZSTD_c_useRowMatchFinder
+ * Controlled with ZSTD_paramSwitch_e enum.
+ * Default is ZSTD_ps_auto.
+ * Set to ZSTD_ps_disable to never use row-based matchfinder.
+ * Set to ZSTD_ps_enable to force usage of row-based matchfinder.
+ *
+ * By default, in ZSTD_ps_auto, the library will decide at runtime whether to use
+ * the row-based matchfinder based on support for SIMD instructions and the window log.
+ * Note that this only pertains to compression strategies: greedy, lazy, and lazy2
+ */
+#define ZSTD_c_useRowMatchFinder ZSTD_c_experimentalParam14
+
+/* ZSTD_c_deterministicRefPrefix
+ * Default is 0 == disabled. Set to 1 to enable.
+ *
+ * Zstd produces different results for prefix compression when the prefix is
+ * directly adjacent to the data about to be compressed vs. when it isn't.
+ * This is because zstd detects that the two buffers are contiguous and it can
+ * use a more efficient match finding algorithm. However, this produces different
+ * results than when the two buffers are non-contiguous. This flag forces zstd
+ * to always load the prefix in non-contiguous mode, even if it happens to be
+ * adjacent to the data, to guarantee determinism.
+ *
+ * If you really care about determinism when using a dictionary or prefix,
+ * like when doing delta compression, you should select this option. It comes
+ * at a speed penalty of about ~2.5% if the dictionary and data happened to be
+ * contiguous, and is free if they weren't contiguous. We don't expect that
+ * intentionally making the dictionary and data contiguous will be worth the
+ * cost to memcpy() the data.
+ */
+#define ZSTD_c_deterministicRefPrefix ZSTD_c_experimentalParam15
+
/*! ZSTD_CCtx_getParameter() :
* Get the requested compression parameter value, selected by enum ZSTD_cParameter,
* and store it into int* value.
* @return : 0, or an error code (which can be tested with ZSTD_isError()).
*/
-ZSTDLIB_API size_t ZSTD_CCtx_getParameter(const ZSTD_CCtx* cctx, ZSTD_cParameter param, int* value);
+ZSTDLIB_STATIC_API size_t ZSTD_CCtx_getParameter(const ZSTD_CCtx* cctx, ZSTD_cParameter param, int* value);
/*! ZSTD_CCtx_params :
* This can be used with ZSTD_estimateCCtxSize_advanced_usingCCtxParams()
* for static allocation of CCtx for single-threaded compression.
*/
-ZSTDLIB_API ZSTD_CCtx_params* ZSTD_createCCtxParams(void);
-ZSTDLIB_API size_t ZSTD_freeCCtxParams(ZSTD_CCtx_params* params); /* accept NULL pointer */
+ZSTDLIB_STATIC_API ZSTD_CCtx_params* ZSTD_createCCtxParams(void);
+ZSTDLIB_STATIC_API size_t ZSTD_freeCCtxParams(ZSTD_CCtx_params* params); /* accept NULL pointer */
/*! ZSTD_CCtxParams_reset() :
* Reset params to default values.
*/
-ZSTDLIB_API size_t ZSTD_CCtxParams_reset(ZSTD_CCtx_params* params);
+ZSTDLIB_STATIC_API size_t ZSTD_CCtxParams_reset(ZSTD_CCtx_params* params);
/*! ZSTD_CCtxParams_init() :
* Initializes the compression parameters of cctxParams according to
* compression level. All other parameters are reset to their default values.
*/
-ZSTDLIB_API size_t ZSTD_CCtxParams_init(ZSTD_CCtx_params* cctxParams, int compressionLevel);
+ZSTDLIB_STATIC_API size_t ZSTD_CCtxParams_init(ZSTD_CCtx_params* cctxParams, int compressionLevel);
/*! ZSTD_CCtxParams_init_advanced() :
* Initializes the compression and frame parameters of cctxParams according to
* params. All other parameters are reset to their default values.
*/
-ZSTDLIB_API size_t ZSTD_CCtxParams_init_advanced(ZSTD_CCtx_params* cctxParams, ZSTD_parameters params);
+ZSTDLIB_STATIC_API size_t ZSTD_CCtxParams_init_advanced(ZSTD_CCtx_params* cctxParams, ZSTD_parameters params);
-/*! ZSTD_CCtxParams_setParameter() :
+/*! ZSTD_CCtxParams_setParameter() : Requires v1.4.0+
* Similar to ZSTD_CCtx_setParameter.
* Set one compression parameter, selected by enum ZSTD_cParameter.
* Parameters must be applied to a ZSTD_CCtx using
* @result : a code representing success or failure (which can be tested with
* ZSTD_isError()).
*/
-ZSTDLIB_API size_t ZSTD_CCtxParams_setParameter(ZSTD_CCtx_params* params, ZSTD_cParameter param, int value);
+ZSTDLIB_STATIC_API size_t ZSTD_CCtxParams_setParameter(ZSTD_CCtx_params* params, ZSTD_cParameter param, int value);
/*! ZSTD_CCtxParams_getParameter() :
* Similar to ZSTD_CCtx_getParameter.
* Get the requested value of one compression parameter, selected by enum ZSTD_cParameter.
* @result : 0, or an error code (which can be tested with ZSTD_isError()).
*/
-ZSTDLIB_API size_t ZSTD_CCtxParams_getParameter(const ZSTD_CCtx_params* params, ZSTD_cParameter param, int* value);
+ZSTDLIB_STATIC_API size_t ZSTD_CCtxParams_getParameter(const ZSTD_CCtx_params* params, ZSTD_cParameter param, int* value);
/*! ZSTD_CCtx_setParametersUsingCCtxParams() :
* Apply a set of ZSTD_CCtx_params to the compression context.
* if nbWorkers>=1, new parameters will be picked up at next job,
* with a few restrictions (windowLog, pledgedSrcSize, nbWorkers, jobSize, and overlapLog are not updated).
*/
-ZSTDLIB_API size_t ZSTD_CCtx_setParametersUsingCCtxParams(
+ZSTDLIB_STATIC_API size_t ZSTD_CCtx_setParametersUsingCCtxParams(
ZSTD_CCtx* cctx, const ZSTD_CCtx_params* params);
/*! ZSTD_compressStream2_simpleArgs() :
* This variant might be helpful for binders from dynamic languages
* which have troubles handling structures containing memory pointers.
*/
-ZSTDLIB_API size_t ZSTD_compressStream2_simpleArgs (
+ZSTDLIB_STATIC_API size_t ZSTD_compressStream2_simpleArgs (
ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos,
* Note : Frame Identifier is 4 bytes. If `size < 4`, @return will always be 0.
* Note 2 : Legacy Frame Identifiers are considered valid only if Legacy Support is enabled.
* Note 3 : Skippable Frame Identifiers are considered valid. */
-ZSTDLIB_API unsigned ZSTD_isFrame(const void* buffer, size_t size);
+ZSTDLIB_STATIC_API unsigned ZSTD_isFrame(const void* buffer, size_t size);
/*! ZSTD_createDDict_byReference() :
* Create a digested dictionary, ready to start decompression operation without startup delay.
* Dictionary content is referenced, and therefore stays in dictBuffer.
* It is important that dictBuffer outlives DDict,
* it must remain read accessible throughout the lifetime of DDict */
-ZSTDLIB_API ZSTD_DDict* ZSTD_createDDict_byReference(const void* dictBuffer, size_t dictSize);
+ZSTDLIB_STATIC_API ZSTD_DDict* ZSTD_createDDict_byReference(const void* dictBuffer, size_t dictSize);
/*! ZSTD_DCtx_loadDictionary_byReference() :
* Same as ZSTD_DCtx_loadDictionary(),
* but references `dict` content instead of copying it into `dctx`.
* This saves memory if `dict` remains around.,
* However, it's imperative that `dict` remains accessible (and unmodified) while being used, so it must outlive decompression. */
-ZSTDLIB_API size_t ZSTD_DCtx_loadDictionary_byReference(ZSTD_DCtx* dctx, const void* dict, size_t dictSize);
+ZSTDLIB_STATIC_API size_t ZSTD_DCtx_loadDictionary_byReference(ZSTD_DCtx* dctx, const void* dict, size_t dictSize);
/*! ZSTD_DCtx_loadDictionary_advanced() :
* Same as ZSTD_DCtx_loadDictionary(),
* but gives direct control over
* how to load the dictionary (by copy ? by reference ?)
* and how to interpret it (automatic ? force raw mode ? full mode only ?). */
-ZSTDLIB_API size_t ZSTD_DCtx_loadDictionary_advanced(ZSTD_DCtx* dctx, const void* dict, size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod, ZSTD_dictContentType_e dictContentType);
+ZSTDLIB_STATIC_API size_t ZSTD_DCtx_loadDictionary_advanced(ZSTD_DCtx* dctx, const void* dict, size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod, ZSTD_dictContentType_e dictContentType);
/*! ZSTD_DCtx_refPrefix_advanced() :
* Same as ZSTD_DCtx_refPrefix(), but gives finer control over
* how to interpret prefix content (automatic ? force raw mode (default) ? full mode only ?) */
-ZSTDLIB_API size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize, ZSTD_dictContentType_e dictContentType);
+ZSTDLIB_STATIC_API size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize, ZSTD_dictContentType_e dictContentType);
/*! ZSTD_DCtx_setMaxWindowSize() :
* Refuses allocating internal buffers for frames requiring a window size larger than provided limit.
* By default, a decompression context accepts all window sizes <= (1 << ZSTD_WINDOWLOG_LIMIT_DEFAULT)
* @return : 0, or an error code (which can be tested using ZSTD_isError()).
*/
-ZSTDLIB_API size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowSize);
+ZSTDLIB_STATIC_API size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowSize);
/*! ZSTD_DCtx_getParameter() :
* Get the requested decompression parameter value, selected by enum ZSTD_dParameter,
* and store it into int* value.
* @return : 0, or an error code (which can be tested with ZSTD_isError()).
*/
-ZSTDLIB_API size_t ZSTD_DCtx_getParameter(ZSTD_DCtx* dctx, ZSTD_dParameter param, int* value);
+ZSTDLIB_STATIC_API size_t ZSTD_DCtx_getParameter(ZSTD_DCtx* dctx, ZSTD_dParameter param, int* value);
/* ZSTD_d_format
* experimental parameter,
/*! ZSTD_DCtx_setFormat() :
+ * This function is REDUNDANT. Prefer ZSTD_DCtx_setParameter().
* Instruct the decoder context about what kind of data to decode next.
* This instruction is mandatory to decode data without a fully-formed header,
* such ZSTD_f_zstd1_magicless for example.
* @return : 0, or an error code (which can be tested using ZSTD_isError()). */
-ZSTDLIB_API size_t ZSTD_DCtx_setFormat(ZSTD_DCtx* dctx, ZSTD_format_e format);
+ZSTD_DEPRECATED("use ZSTD_DCtx_setParameter() instead")
+size_t ZSTD_DCtx_setFormat(ZSTD_DCtx* dctx, ZSTD_format_e format);
/*! ZSTD_decompressStream_simpleArgs() :
* Same as ZSTD_decompressStream(),
* This can be helpful for binders from dynamic languages
* which have troubles handling structures containing memory pointers.
*/
-ZSTDLIB_API size_t ZSTD_decompressStream_simpleArgs (
+ZSTDLIB_STATIC_API size_t ZSTD_decompressStream_simpleArgs (
ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos);
/*===== Advanced Streaming compression functions =====*/
/*! ZSTD_initCStream_srcSize() :
- * This function is deprecated, and equivalent to:
+ * This function is DEPRECATED, and equivalent to:
* ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only);
* ZSTD_CCtx_refCDict(zcs, NULL); // clear the dictionary (if any)
* ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel);
* pledgedSrcSize must be correct. If it is not known at init time, use
* ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs,
* "0" also disables frame content size field. It may be enabled in the future.
- * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x
+ * This prototype will generate compilation warnings.
*/
-ZSTDLIB_API size_t
-ZSTD_initCStream_srcSize(ZSTD_CStream* zcs,
+ZSTD_DEPRECATED("use ZSTD_CCtx_reset, see zstd.h for detailed instructions")
+size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs,
int compressionLevel,
unsigned long long pledgedSrcSize);
/*! ZSTD_initCStream_usingDict() :
- * This function is deprecated, and is equivalent to:
+ * This function is DEPRECATED, and is equivalent to:
* ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only);
* ZSTD_CCtx_setParameter(zcs, ZSTD_c_compressionLevel, compressionLevel);
* ZSTD_CCtx_loadDictionary(zcs, dict, dictSize);
* dict == NULL or dictSize < 8, in which case no dict is used.
* Note: dict is loaded with ZSTD_dct_auto (treated as a full zstd dictionary if
* it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy.
- * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x
+ * This prototype will generate compilation warnings.
*/
-ZSTDLIB_API size_t
-ZSTD_initCStream_usingDict(ZSTD_CStream* zcs,
+ZSTD_DEPRECATED("use ZSTD_CCtx_reset, see zstd.h for detailed instructions")
+size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs,
const void* dict, size_t dictSize,
int compressionLevel);
/*! ZSTD_initCStream_advanced() :
- * This function is deprecated, and is approximately equivalent to:
+ * This function is DEPRECATED, and is approximately equivalent to:
* ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only);
* // Pseudocode: Set each zstd parameter and leave the rest as-is.
* for ((param, value) : params) {
* dict is loaded with ZSTD_dct_auto and ZSTD_dlm_byCopy.
* pledgedSrcSize must be correct.
* If srcSize is not known at init time, use value ZSTD_CONTENTSIZE_UNKNOWN.
- * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x
+ * This prototype will generate compilation warnings.
*/
-ZSTDLIB_API size_t
-ZSTD_initCStream_advanced(ZSTD_CStream* zcs,
+ZSTD_DEPRECATED("use ZSTD_CCtx_reset, see zstd.h for detailed instructions")
+size_t ZSTD_initCStream_advanced(ZSTD_CStream* zcs,
const void* dict, size_t dictSize,
ZSTD_parameters params,
unsigned long long pledgedSrcSize);
/*! ZSTD_initCStream_usingCDict() :
- * This function is deprecated, and equivalent to:
+ * This function is DEPRECATED, and equivalent to:
* ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only);
* ZSTD_CCtx_refCDict(zcs, cdict);
*
* note : cdict will just be referenced, and must outlive compression session
- * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x
+ * This prototype will generate compilation warnings.
*/
-ZSTDLIB_API size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);
+ZSTD_DEPRECATED("use ZSTD_CCtx_reset and ZSTD_CCtx_refCDict, see zstd.h for detailed instructions")
+size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);
/*! ZSTD_initCStream_usingCDict_advanced() :
* This function is DEPRECATED, and is approximately equivalent to:
* same as ZSTD_initCStream_usingCDict(), with control over frame parameters.
* pledgedSrcSize must be correct. If srcSize is not known at init time, use
* value ZSTD_CONTENTSIZE_UNKNOWN.
- * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x
+ * This prototype will generate compilation warnings.
*/
-ZSTDLIB_API size_t
-ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs,
+ZSTD_DEPRECATED("use ZSTD_CCtx_reset and ZSTD_CCtx_refCDict, see zstd.h for detailed instructions")
+size_t ZSTD_initCStream_usingCDict_advanced(ZSTD_CStream* zcs,
const ZSTD_CDict* cdict,
ZSTD_frameParameters fParams,
unsigned long long pledgedSrcSize);
/*! ZSTD_resetCStream() :
- * This function is deprecated, and is equivalent to:
+ * This function is DEPRECATED, and is equivalent to:
* ZSTD_CCtx_reset(zcs, ZSTD_reset_session_only);
* ZSTD_CCtx_setPledgedSrcSize(zcs, pledgedSrcSize);
+ * Note: ZSTD_resetCStream() interprets pledgedSrcSize == 0 as ZSTD_CONTENTSIZE_UNKNOWN, but
+ * ZSTD_CCtx_setPledgedSrcSize() does not do the same, so ZSTD_CONTENTSIZE_UNKNOWN must be
+ * explicitly specified.
*
* start a new frame, using same parameters from previous frame.
* This is typically useful to skip dictionary loading stage, since it will re-use it in-place.
* For the time being, pledgedSrcSize==0 is interpreted as "srcSize unknown" for compatibility with older programs,
* but it will change to mean "empty" in future version, so use macro ZSTD_CONTENTSIZE_UNKNOWN instead.
* @return : 0, or an error code (which can be tested using ZSTD_isError())
- * Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x
+ * This prototype will generate compilation warnings.
*/
-ZSTDLIB_API size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pledgedSrcSize);
+ZSTD_DEPRECATED("use ZSTD_CCtx_reset, see zstd.h for detailed instructions")
+size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pledgedSrcSize);
typedef struct {
* Note : (ingested - consumed) is amount of input data buffered internally, not yet compressed.
* Aggregates progression inside active worker threads.
*/
-ZSTDLIB_API ZSTD_frameProgression ZSTD_getFrameProgression(const ZSTD_CCtx* cctx);
+ZSTDLIB_STATIC_API ZSTD_frameProgression ZSTD_getFrameProgression(const ZSTD_CCtx* cctx);
/*! ZSTD_toFlushNow() :
* Tell how many bytes are ready to be flushed immediately.
* therefore flush speed is limited by production speed of oldest job
* irrespective of the speed of concurrent (and newer) jobs.
*/
-ZSTDLIB_API size_t ZSTD_toFlushNow(ZSTD_CCtx* cctx);
+ZSTDLIB_STATIC_API size_t ZSTD_toFlushNow(ZSTD_CCtx* cctx);
/*===== Advanced Streaming decompression functions =====*/
* note: no dictionary will be used if dict == NULL or dictSize < 8
* Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x
*/
-ZSTDLIB_API size_t ZSTD_initDStream_usingDict(ZSTD_DStream* zds, const void* dict, size_t dictSize);
+ZSTDLIB_STATIC_API size_t ZSTD_initDStream_usingDict(ZSTD_DStream* zds, const void* dict, size_t dictSize);
/*!
* This function is deprecated, and is equivalent to:
* note : ddict is referenced, it must outlive decompression session
* Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x
*/
-ZSTDLIB_API size_t ZSTD_initDStream_usingDDict(ZSTD_DStream* zds, const ZSTD_DDict* ddict);
+ZSTDLIB_STATIC_API size_t ZSTD_initDStream_usingDDict(ZSTD_DStream* zds, const ZSTD_DDict* ddict);
/*!
* This function is deprecated, and is equivalent to:
* re-use decompression parameters from previous init; saves dictionary loading
* Note : this prototype will be marked as deprecated and generate compilation warnings on reaching v1.5.x
*/
-ZSTDLIB_API size_t ZSTD_resetDStream(ZSTD_DStream* zds);
+ZSTDLIB_STATIC_API size_t ZSTD_resetDStream(ZSTD_DStream* zds);
/* *******************************************************************
ZSTD_CCtx object can be re-used multiple times within successive compression operations.
Start by initializing a context.
- Use ZSTD_compressBegin(), or ZSTD_compressBegin_usingDict() for dictionary compression,
- or ZSTD_compressBegin_advanced(), for finer parameter control.
+ Use ZSTD_compressBegin(), or ZSTD_compressBegin_usingDict() for dictionary compression.
It's also possible to duplicate a reference context which has already been initialized, using ZSTD_copyCCtx()
Then, consume your input using ZSTD_compressContinue().
*/
/*===== Buffer-less streaming compression functions =====*/
-ZSTDLIB_API size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel);
-ZSTDLIB_API size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, int compressionLevel);
-ZSTDLIB_API size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_parameters params, unsigned long long pledgedSrcSize); /*< pledgedSrcSize : If srcSize is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN */
-ZSTDLIB_API size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict); /*< note: fails if cdict==NULL */
-ZSTDLIB_API size_t ZSTD_compressBegin_usingCDict_advanced(ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize); /* compression parameters are already set within cdict. pledgedSrcSize must be correct. If srcSize is not known, use macro ZSTD_CONTENTSIZE_UNKNOWN */
-ZSTDLIB_API size_t ZSTD_copyCCtx(ZSTD_CCtx* cctx, const ZSTD_CCtx* preparedCCtx, unsigned long long pledgedSrcSize); /*< note: if pledgedSrcSize is not known, use ZSTD_CONTENTSIZE_UNKNOWN */
-
-ZSTDLIB_API size_t ZSTD_compressContinue(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
-ZSTDLIB_API size_t ZSTD_compressEnd(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
-
-
+ZSTDLIB_STATIC_API size_t ZSTD_compressBegin(ZSTD_CCtx* cctx, int compressionLevel);
+ZSTDLIB_STATIC_API size_t ZSTD_compressBegin_usingDict(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, int compressionLevel);
+ZSTDLIB_STATIC_API size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict); /*< note: fails if cdict==NULL */
+ZSTDLIB_STATIC_API size_t ZSTD_copyCCtx(ZSTD_CCtx* cctx, const ZSTD_CCtx* preparedCCtx, unsigned long long pledgedSrcSize); /*< note: if pledgedSrcSize is not known, use ZSTD_CONTENTSIZE_UNKNOWN */
+
+ZSTDLIB_STATIC_API size_t ZSTD_compressContinue(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
+ZSTDLIB_STATIC_API size_t ZSTD_compressEnd(ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
+
+/* The ZSTD_compressBegin_advanced() and ZSTD_compressBegin_usingCDict_advanced() are now DEPRECATED and will generate a compiler warning */
+ZSTD_DEPRECATED("use advanced API to access custom parameters")
+size_t ZSTD_compressBegin_advanced(ZSTD_CCtx* cctx, const void* dict, size_t dictSize, ZSTD_parameters params, unsigned long long pledgedSrcSize); /*< pledgedSrcSize : If srcSize is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN */
+ZSTD_DEPRECATED("use advanced API to access custom parameters")
+size_t ZSTD_compressBegin_usingCDict_advanced(ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict, ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize); /* compression parameters are already set within cdict. pledgedSrcSize must be correct. If srcSize is not known, use macro ZSTD_CONTENTSIZE_UNKNOWN */
/*
Buffer-less streaming decompression (synchronous mode)
* @return : 0, `zfhPtr` is correctly filled,
* >0, `srcSize` is too small, value is wanted `srcSize` amount,
* or an error code, which can be tested using ZSTD_isError() */
-ZSTDLIB_API size_t ZSTD_getFrameHeader(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize); /*< doesn't consume input */
+ZSTDLIB_STATIC_API size_t ZSTD_getFrameHeader(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize); /*< doesn't consume input */
/*! ZSTD_getFrameHeader_advanced() :
* same as ZSTD_getFrameHeader(),
* with added capability to select a format (like ZSTD_f_zstd1_magicless) */
-ZSTDLIB_API size_t ZSTD_getFrameHeader_advanced(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize, ZSTD_format_e format);
-ZSTDLIB_API size_t ZSTD_decodingBufferSize_min(unsigned long long windowSize, unsigned long long frameContentSize); /*< when frame content size is not known, pass in frameContentSize == ZSTD_CONTENTSIZE_UNKNOWN */
+ZSTDLIB_STATIC_API size_t ZSTD_getFrameHeader_advanced(ZSTD_frameHeader* zfhPtr, const void* src, size_t srcSize, ZSTD_format_e format);
+ZSTDLIB_STATIC_API size_t ZSTD_decodingBufferSize_min(unsigned long long windowSize, unsigned long long frameContentSize); /*< when frame content size is not known, pass in frameContentSize == ZSTD_CONTENTSIZE_UNKNOWN */
-ZSTDLIB_API size_t ZSTD_decompressBegin(ZSTD_DCtx* dctx);
-ZSTDLIB_API size_t ZSTD_decompressBegin_usingDict(ZSTD_DCtx* dctx, const void* dict, size_t dictSize);
-ZSTDLIB_API size_t ZSTD_decompressBegin_usingDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict);
+ZSTDLIB_STATIC_API size_t ZSTD_decompressBegin(ZSTD_DCtx* dctx);
+ZSTDLIB_STATIC_API size_t ZSTD_decompressBegin_usingDict(ZSTD_DCtx* dctx, const void* dict, size_t dictSize);
+ZSTDLIB_STATIC_API size_t ZSTD_decompressBegin_usingDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict);
-ZSTDLIB_API size_t ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx* dctx);
-ZSTDLIB_API size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
+ZSTDLIB_STATIC_API size_t ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx* dctx);
+ZSTDLIB_STATIC_API size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
/* misc */
-ZSTDLIB_API void ZSTD_copyDCtx(ZSTD_DCtx* dctx, const ZSTD_DCtx* preparedDCtx);
+ZSTDLIB_STATIC_API void ZSTD_copyDCtx(ZSTD_DCtx* dctx, const ZSTD_DCtx* preparedDCtx);
typedef enum { ZSTDnit_frameHeader, ZSTDnit_blockHeader, ZSTDnit_block, ZSTDnit_lastBlock, ZSTDnit_checksum, ZSTDnit_skippableFrame } ZSTD_nextInputType_e;
-ZSTDLIB_API ZSTD_nextInputType_e ZSTD_nextInputType(ZSTD_DCtx* dctx);
+ZSTDLIB_STATIC_API ZSTD_nextInputType_e ZSTD_nextInputType(ZSTD_DCtx* dctx);
*/
/*===== Raw zstd block functions =====*/
-ZSTDLIB_API size_t ZSTD_getBlockSize (const ZSTD_CCtx* cctx);
-ZSTDLIB_API size_t ZSTD_compressBlock (ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
-ZSTDLIB_API size_t ZSTD_decompressBlock(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
-ZSTDLIB_API size_t ZSTD_insertBlock (ZSTD_DCtx* dctx, const void* blockStart, size_t blockSize); /*< insert uncompressed block into `dctx` history. Useful for multi-blocks decompression. */
+ZSTDLIB_STATIC_API size_t ZSTD_getBlockSize (const ZSTD_CCtx* cctx);
+ZSTDLIB_STATIC_API size_t ZSTD_compressBlock (ZSTD_CCtx* cctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
+ZSTDLIB_STATIC_API size_t ZSTD_decompressBlock(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, const void* src, size_t srcSize);
+ZSTDLIB_STATIC_API size_t ZSTD_insertBlock (ZSTD_DCtx* dctx, const void* blockStart, size_t blockSize); /*< insert uncompressed block into `dctx` history. Useful for multi-blocks decompression. */
#endif /* ZSTD_H_ZSTD_STATIC_LINKING_ONLY */
/*
* Context lifetime overview:
*
- * An AFU context may be inited and then started and stoppped multiple times
+ * An AFU context may be inited and then started and stopped multiple times
* before it's released. ie.
* - cxl_dev_context_init()
* - cxl_start_context()
* @sk_stamp: time stamp of last packet received
* @sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only
* @sk_tsflags: SO_TIMESTAMPING flags
+ * @sk_use_task_frag: allow sk_page_frag() to use current->task_frag.
+ * Sockets that can be used under memory reclaim should
+ * set this to false.
* @sk_bind_phc: SO_TIMESTAMPING bind PHC index of PTP virtual clock
* for timestamping
* @sk_tskey: counter to disambiguate concurrent tstamp requests
u8 sk_txtime_deadline_mode : 1,
sk_txtime_report_errors : 1,
sk_txtime_unused : 6;
+ bool sk_use_task_frag;
struct socket *sk_socket;
void *sk_user_data;
* Both direct reclaim and page faults can nest inside other
* socket operations and end up recursing into sk_page_frag()
* while it's already in use: explicitly avoid task page_frag
- * usage if the caller is potentially doing any of them.
- * This assumes that page fault handlers use the GFP_NOFS flags.
+ * when users disable sk_use_task_frag.
*
* Return: a per task page_frag if context allows that,
* otherwise a per socket one.
*/
static inline struct page_frag *sk_page_frag(struct sock *sk)
{
- if ((sk->sk_allocation & (__GFP_DIRECT_RECLAIM | __GFP_MEMALLOC | __GFP_FS)) ==
- (__GFP_DIRECT_RECLAIM | __GFP_FS))
+ if (sk->sk_use_task_frag)
return ¤t->task_frag;
return &sk->sk_frag;
#undef __get_bitmask
#define __get_bitmask(field) (char *)__get_dynamic_array(field)
+#undef __get_cpumask
+#define __get_cpumask(field) (char *)__get_dynamic_array(field)
+
#undef __get_sockaddr
#define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field))
#undef __get_rel_bitmask
#define __get_rel_bitmask(field) (char *)__get_rel_dynamic_array(field)
+#undef __get_rel_cpumask
+#define __get_rel_cpumask(field) (char *)__get_rel_dynamic_array(field)
+
#undef __get_rel_sockaddr
#define __get_rel_sockaddr(field) ((struct sockaddr *)__get_rel_dynamic_array(field))
DECLARE_EVENT_CLASS(pwm,
- TP_PROTO(struct pwm_device *pwm, const struct pwm_state *state),
+ TP_PROTO(struct pwm_device *pwm, const struct pwm_state *state, int err),
- TP_ARGS(pwm, state),
+ TP_ARGS(pwm, state, err),
TP_STRUCT__entry(
__field(struct pwm_device *, pwm)
__field(u64, duty_cycle)
__field(enum pwm_polarity, polarity)
__field(bool, enabled)
+ __field(int, err)
),
TP_fast_assign(
__entry->duty_cycle = state->duty_cycle;
__entry->polarity = state->polarity;
__entry->enabled = state->enabled;
+ __entry->err = err;
),
- TP_printk("%p: period=%llu duty_cycle=%llu polarity=%d enabled=%d",
+ TP_printk("%p: period=%llu duty_cycle=%llu polarity=%d enabled=%d err=%d",
__entry->pwm, __entry->period, __entry->duty_cycle,
- __entry->polarity, __entry->enabled)
+ __entry->polarity, __entry->enabled, __entry->err)
);
DEFINE_EVENT(pwm, pwm_apply,
- TP_PROTO(struct pwm_device *pwm, const struct pwm_state *state),
-
- TP_ARGS(pwm, state)
+ TP_PROTO(struct pwm_device *pwm, const struct pwm_state *state, int err),
+ TP_ARGS(pwm, state, err)
);
DEFINE_EVENT(pwm, pwm_get,
- TP_PROTO(struct pwm_device *pwm, const struct pwm_state *state),
-
- TP_ARGS(pwm, state)
+ TP_PROTO(struct pwm_device *pwm, const struct pwm_state *state, int err),
+ TP_ARGS(pwm, state, err)
);
#endif /* _TRACE_PWM_H */
DECLARE_EVENT_CLASS(rwmmio_rw_template,
- TP_PROTO(unsigned long caller, u64 val, u8 width, volatile void __iomem *addr),
+ TP_PROTO(unsigned long caller, unsigned long caller0, u64 val, u8 width,
+ volatile void __iomem *addr),
- TP_ARGS(caller, val, width, addr),
+ TP_ARGS(caller, caller0, val, width, addr),
TP_STRUCT__entry(
__field(unsigned long, caller)
+ __field(unsigned long, caller0)
__field(unsigned long, addr)
__field(u64, val)
__field(u8, width)
TP_fast_assign(
__entry->caller = caller;
+ __entry->caller0 = caller0;
__entry->val = val;
__entry->addr = (unsigned long)addr;
__entry->width = width;
),
- TP_printk("%pS width=%d val=%#llx addr=%#lx",
- (void *)__entry->caller, __entry->width,
+ TP_printk("%pS -> %pS width=%d val=%#llx addr=%#lx",
+ (void *)__entry->caller0, (void *)__entry->caller, __entry->width,
__entry->val, __entry->addr)
);
DEFINE_EVENT(rwmmio_rw_template, rwmmio_write,
- TP_PROTO(unsigned long caller, u64 val, u8 width, volatile void __iomem *addr),
- TP_ARGS(caller, val, width, addr)
+ TP_PROTO(unsigned long caller, unsigned long caller0, u64 val, u8 width,
+ volatile void __iomem *addr),
+ TP_ARGS(caller, caller0, val, width, addr)
);
DEFINE_EVENT(rwmmio_rw_template, rwmmio_post_write,
- TP_PROTO(unsigned long caller, u64 val, u8 width, volatile void __iomem *addr),
- TP_ARGS(caller, val, width, addr)
+ TP_PROTO(unsigned long caller, unsigned long caller0, u64 val, u8 width,
+ volatile void __iomem *addr),
+ TP_ARGS(caller, caller0, val, width, addr)
);
TRACE_EVENT(rwmmio_read,
- TP_PROTO(unsigned long caller, u8 width, const volatile void __iomem *addr),
+ TP_PROTO(unsigned long caller, unsigned long caller0, u8 width,
+ const volatile void __iomem *addr),
- TP_ARGS(caller, width, addr),
+ TP_ARGS(caller, caller0, width, addr),
TP_STRUCT__entry(
__field(unsigned long, caller)
+ __field(unsigned long, caller0)
__field(unsigned long, addr)
__field(u8, width)
),
TP_fast_assign(
__entry->caller = caller;
+ __entry->caller0 = caller0;
__entry->addr = (unsigned long)addr;
__entry->width = width;
),
- TP_printk("%pS width=%d addr=%#lx",
- (void *)__entry->caller, __entry->width, __entry->addr)
+ TP_printk("%pS -> %pS width=%d addr=%#lx",
+ (void *)__entry->caller0, (void *)__entry->caller, __entry->width, __entry->addr)
);
TRACE_EVENT(rwmmio_post_read,
- TP_PROTO(unsigned long caller, u64 val, u8 width, const volatile void __iomem *addr),
+ TP_PROTO(unsigned long caller, unsigned long caller0, u64 val, u8 width,
+ const volatile void __iomem *addr),
- TP_ARGS(caller, val, width, addr),
+ TP_ARGS(caller, caller0, val, width, addr),
TP_STRUCT__entry(
__field(unsigned long, caller)
+ __field(unsigned long, caller0)
__field(unsigned long, addr)
__field(u64, val)
__field(u8, width)
TP_fast_assign(
__entry->caller = caller;
+ __entry->caller0 = caller0;
__entry->val = val;
__entry->addr = (unsigned long)addr;
__entry->width = width;
),
- TP_printk("%pS width=%d val=%#llx addr=%#lx",
- (void *)__entry->caller, __entry->width,
+ TP_printk("%pS -> %pS width=%d val=%#llx addr=%#lx",
+ (void *)__entry->caller0, (void *)__entry->caller, __entry->width,
__entry->val, __entry->addr)
);
TP_STRUCT__entry(
__field(unsigned int, peer )
__field(int, ref )
- __field(int, why )
+ __field(enum rxrpc_peer_trace, why )
),
TP_fast_assign(
#undef __get_bitmask
#define __get_bitmask(field) (char *)__get_dynamic_array(field)
+#undef __get_cpumask
+#define __get_cpumask(field) (char *)__get_dynamic_array(field)
+
#undef __get_sockaddr
#define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field))
#undef __get_rel_bitmask
#define __get_rel_bitmask(field) (char *)__get_rel_dynamic_array(field)
+#undef __get_rel_cpumask
+#define __get_rel_cpumask(field) (char *)__get_rel_dynamic_array(field)
+
#undef __get_rel_sockaddr
#define __get_rel_sockaddr(field) ((struct sockaddr *)__get_rel_dynamic_array(field))
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(char, item, -1)
+#undef __cpumask
+#define __cpumask(item) __dynamic_array(char, item, -1)
+
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(char, item, -1)
+#undef __rel_cpumask
+#define __rel_cpumask(item) __rel_dynamic_array(char, item, -1)
+
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
+#undef __cpumask
+#define __cpumask(item) __dynamic_array(unsigned long, item, -1)
+
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, -1)
+#undef __rel_cpumask
+#define __rel_cpumask(item) __rel_dynamic_array(unsigned long, item, -1)
+
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
trace_print_bitmask_seq(p, __bitmask, __bitmask_size); \
})
+#undef __get_cpumask
+#define __get_cpumask(field) __get_bitmask(field)
+
#undef __get_rel_bitmask
#define __get_rel_bitmask(field) \
({ \
trace_print_bitmask_seq(p, __bitmask, __bitmask_size); \
})
+#undef __get_rel_cpumask
+#define __get_rel_cpumask(field) __get_rel_bitmask(field)
+
#undef __get_sockaddr
#define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field))
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
+#undef __cpumask
+#define __cpumask(item) { \
+ .type = "__data_loc cpumask_t", .name = #item, \
+ .size = 4, .align = 4, \
+ .is_signed = 0, .filter_type = FILTER_OTHER },
+
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, -1)
+#undef __rel_cpumask
+#define __rel_cpumask(item) { \
+ .type = "__rel_loc cpumask_t", .name = #item, \
+ .size = 4, .align = 4, \
+ .is_signed = 0, .filter_type = FILTER_OTHER },
+
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, \
__bitmask_size_in_longs(nr_bits))
+#undef __cpumask
+#define __cpumask(item) __bitmask(item, nr_cpumask_bits)
+
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, \
__bitmask_size_in_longs(nr_bits))
+#undef __rel_cpumask
+#define __rel_cpumask(item) __rel_bitmask(item, nr_cpumask_bits)
+
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#define __assign_bitmask(dst, src, nr_bits) \
memcpy(__get_bitmask(dst), (src), __bitmask_size_in_bytes(nr_bits))
+#undef __cpumask
+#define __cpumask(item) __dynamic_array(unsigned long, item, -1)
+
+#undef __get_cpumask
+#define __get_cpumask(field) (char *)__get_dynamic_array(field)
+
+#undef __assign_cpumask
+#define __assign_cpumask(dst, src) \
+ memcpy(__get_cpumask(dst), (src), __bitmask_size_in_bytes(nr_cpumask_bits))
+
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#define __assign_rel_bitmask(dst, src, nr_bits) \
memcpy(__get_rel_bitmask(dst), (src), __bitmask_size_in_bytes(nr_bits))
+#undef __rel_cpumask
+#define __rel_cpumask(item) __rel_dynamic_array(unsigned long, item, -1)
+
+#undef __get_rel_cpumask
+#define __get_rel_cpumask(field) (char *)__get_rel_dynamic_array(field)
+
+#undef __assign_rel_cpumask
+#define __assign_rel_cpumask(dst, src) \
+ memcpy(__get_rel_cpumask(dst), (src), __bitmask_size_in_bytes(nr_cpumask_bits))
+
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
#undef __get_dynamic_array_len
#undef __get_str
#undef __get_bitmask
+#undef __get_cpumask
#undef __get_sockaddr
#undef __get_rel_dynamic_array
#undef __get_rel_dynamic_array_len
#undef __get_rel_str
#undef __get_rel_bitmask
+#undef __get_rel_cpumask
#undef __get_rel_sockaddr
#undef __print_array
#undef __print_hex_dump
/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
-#ifndef _ASM_GENERIC_TYPES_H
-#define _ASM_GENERIC_TYPES_H
+#ifndef _UAPI_ASM_GENERIC_TYPES_H
+#define _UAPI_ASM_GENERIC_TYPES_H
/*
* int-ll64 is used everywhere now.
*/
#include <asm-generic/int-ll64.h>
-#endif /* _ASM_GENERIC_TYPES_H */
+#endif /* _UAPI_ASM_GENERIC_TYPES_H */
#define _UAPI_ACRN_H
#include <linux/types.h>
-#include <linux/uuid.h>
#define ACRN_IO_REQUEST_MAX 16
* @reserved0: Reserved and must be 0
* @vcpu_num: Number of vCPU in the VM. Return from hypervisor.
* @reserved1: Reserved and must be 0
- * @uuid: UUID of the VM. Pass to hypervisor directly.
+ * @uuid: Empty space never to be used again (used to be UUID of the VM)
* @vm_flag: Flag of the VM creating. Pass to hypervisor directly.
* @ioreq_buf: Service VM GPA of I/O request buffer. Pass to
* hypervisor directly.
__u16 reserved0;
__u16 vcpu_num;
__u16 reserved1;
- guid_t uuid;
+ __u8 uuid[16];
__u64 vm_flag;
__u64 ioreq_buf;
__u64 cpu_affinity;
*
* Contact: Kai Vehmanen <kai.vehmanen@nokia.com>
* Original author: Peter Ujfalusi <peter.ujfalusi@nokia.com>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * version 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
- * 02110-1301 USA
*/
#ifndef _CS_PROTOCOL_H
* Copyright (C) 2010 Nokia Corporation. All rights reserved.
*
* Contact: Andras Domokos <andras.domokos at nokia.com>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * version 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
- * 02110-1301 USA
*/
#ifndef __HSI_CHAR_H
};
uint32_t delta_rec_size;
- uint32_t crc_val;
+ uint64_t crc_val;
/* DIF check & strip */
struct {
int reserved[9];
};
-/*
+/**
+ * struct serial_rs485 - serial interface for controlling RS485 settings.
+ * @flags: RS485 feature flags.
+ * @delay_rts_before_send: Delay before send (milliseconds).
+ * @delay_rts_after_send: Delay after send (milliseconds).
+ * @addr_recv: Receive filter for RS485 addressing mode
+ * (used only when %SER_RS485_ADDR_RECV is set).
+ * @addr_dest: Destination address for RS485 addressing mode
+ * (used only when %SER_RS485_ADDR_DEST is set).
+ * @padding0: Padding (set to zero).
+ * @padding1: Padding (set to zero).
+ * @padding: Deprecated, use @padding0 and @padding1 instead.
+ * Do not use with @addr_recv and @addr_dest (due to
+ * overlap).
+ *
* Serial interface for controlling RS485 settings on chips with suitable
* support. Set with TIOCSRS485 and get with TIOCGRS485 if supported by your
* platform. The set function returns the new state, with any unsupported bits
* reverted appropriately.
+ *
+ * The flag bits are:
+ *
+ * * %SER_RS485_ENABLED - RS485 enabled.
+ * * %SER_RS485_RTS_ON_SEND - Logical level for RTS pin when sending.
+ * * %SER_RS485_RTS_AFTER_SEND - Logical level for RTS pin after sent.
+ * * %SER_RS485_RX_DURING_TX - Full-duplex RS485 line.
+ * * %SER_RS485_TERMINATE_BUS - Enable bus termination (if supported).
+ * * %SER_RS485_ADDRB - Enable RS485 addressing mode.
+ * * %SER_RS485_ADDR_RECV - Receive address filter (enables @addr_recv). Requires %SER_RS485_ADDRB.
+ * * %SER_RS485_ADDR_DEST - Destination address (enables @addr_dest). Requires %SER_RS485_ADDRB.
*/
-
struct serial_rs485 {
- __u32 flags; /* RS485 feature flags */
-#define SER_RS485_ENABLED (1 << 0) /* If enabled */
-#define SER_RS485_RTS_ON_SEND (1 << 1) /* Logical level for
- RTS pin when
- sending */
-#define SER_RS485_RTS_AFTER_SEND (1 << 2) /* Logical level for
- RTS pin after sent*/
+ __u32 flags;
+#define SER_RS485_ENABLED (1 << 0)
+#define SER_RS485_RTS_ON_SEND (1 << 1)
+#define SER_RS485_RTS_AFTER_SEND (1 << 2)
#define SER_RS485_RX_DURING_TX (1 << 4)
-#define SER_RS485_TERMINATE_BUS (1 << 5) /* Enable bus
- termination
- (if supported) */
-
-/* RS-485 addressing mode */
-#define SER_RS485_ADDRB (1 << 6) /* Enable addressing mode */
-#define SER_RS485_ADDR_RECV (1 << 7) /* Receive address filter */
-#define SER_RS485_ADDR_DEST (1 << 8) /* Destination address */
+#define SER_RS485_TERMINATE_BUS (1 << 5)
+#define SER_RS485_ADDRB (1 << 6)
+#define SER_RS485_ADDR_RECV (1 << 7)
+#define SER_RS485_ADDR_DEST (1 << 8)
- __u32 delay_rts_before_send; /* Delay before send (milliseconds) */
- __u32 delay_rts_after_send; /* Delay after send (milliseconds) */
+ __u32 delay_rts_before_send;
+ __u32 delay_rts_after_send;
/* The fields below are defined by flags */
union {
#define _UAPI_LINUX_SWAB_H
#include <linux/types.h>
-#include <linux/compiler.h>
+#include <linux/stddef.h>
#include <asm/bitsperlong.h>
#include <asm/swab.h>
#define UVC_EVENT_DATA (V4L2_EVENT_PRIVATE_START + 5)
#define UVC_EVENT_LAST (V4L2_EVENT_PRIVATE_START + 5)
+#define UVC_STRING_CONTROL_IDX 0
+#define UVC_STRING_STREAMING_IDX 1
+
struct uvc_request_data {
__s32 length;
__u8 data[60];
__u8 bDefaultFrameIndex;
__u8 bAspectRatioX;
__u8 bAspectRatioY;
- __u8 bmInterfaceFlags;
+ __u8 bmInterlaceFlags;
__u8 bCopyProtect;
} __attribute__((__packed__));
__u8 bDefaultFrameIndex;
__u8 bAspectRatioX;
__u8 bAspectRatioY;
- __u8 bmInterfaceFlags;
+ __u8 bmInterlaceFlags;
__u8 bCopyProtect;
} __attribute__((__packed__));
#define FASTRPC_IOCTL_MMAP _IOWR('R', 6, struct fastrpc_req_mmap)
#define FASTRPC_IOCTL_MUNMAP _IOWR('R', 7, struct fastrpc_req_munmap)
#define FASTRPC_IOCTL_INIT_ATTACH_SNS _IO('R', 8)
+#define FASTRPC_IOCTL_INIT_CREATE_STATIC _IOWR('R', 9, struct fastrpc_init_create_static)
#define FASTRPC_IOCTL_MEM_MAP _IOWR('R', 10, struct fastrpc_mem_map)
#define FASTRPC_IOCTL_MEM_UNMAP _IOWR('R', 11, struct fastrpc_mem_unmap)
#define FASTRPC_IOCTL_GET_DSP_INFO _IOWR('R', 13, struct fastrpc_ioctl_capability)
__u64 file; /* pointer to elf file */
};
+struct fastrpc_init_create_static {
+ __u32 namelen; /* length of pd process name */
+ __u32 memlen;
+ __u64 name; /* pd process name */
+};
+
struct fastrpc_alloc_dma_buf {
__s32 fd; /* fd */
__u32 flags; /* flags to map with */
GAUDI2_ENGINE_ID_NIC10_1,
GAUDI2_ENGINE_ID_NIC11_0,
GAUDI2_ENGINE_ID_NIC11_1,
+ GAUDI2_ENGINE_ID_PCIE,
+ GAUDI2_ENGINE_ID_PSOC,
+ GAUDI2_ENGINE_ID_ARC_FARM,
+ GAUDI2_ENGINE_ID_KDMA,
GAUDI2_ENGINE_ID_SIZE
};
* HL_NOTIFIER_EVENT_DEVICE_UNAVAILABLE - Indicates device is unavailable
* HL_NOTIFIER_EVENT_USER_ENGINE_ERR - Indicates device engine in error state
* HL_NOTIFIER_EVENT_GENERAL_HW_ERR - Indicates device HW error
+ * HL_NOTIFIER_EVENT_RAZWI - Indicates razwi happened
+ * HL_NOTIFIER_EVENT_PAGE_FAULT - Indicates page fault happened
*/
#define HL_NOTIFIER_EVENT_TPC_ASSERT (1ULL << 0)
#define HL_NOTIFIER_EVENT_UNDEFINED_OPCODE (1ULL << 1)
#define HL_NOTIFIER_EVENT_DEVICE_UNAVAILABLE (1ULL << 4)
#define HL_NOTIFIER_EVENT_USER_ENGINE_ERR (1ULL << 5)
#define HL_NOTIFIER_EVENT_GENERAL_HW_ERR (1ULL << 6)
+#define HL_NOTIFIER_EVENT_RAZWI (1ULL << 7)
+#define HL_NOTIFIER_EVENT_PAGE_FAULT (1ULL << 8)
/* Opcode for management ioctl
*
* HL_INFO_UNREGISTER_EVENTFD - Unregister eventfd
* HL_INFO_GET_EVENTS - Retrieve the last occurred events
* HL_INFO_UNDEFINED_OPCODE_EVENT - Retrieve last undefined opcode error information.
+ * HL_INFO_ENGINE_STATUS - Retrieve the status of all the h/w engines in the asic.
+ * HL_INFO_PAGE_FAULT_EVENT - Retrieve parameters of captured page fault.
+ * HL_INFO_USER_MAPPINGS - Retrieve user mappings, captured after page fault event.
*/
#define HL_INFO_HW_IP_INFO 0
#define HL_INFO_HW_EVENTS 1
#define HL_INFO_GET_EVENTS 30
#define HL_INFO_UNDEFINED_OPCODE_EVENT 31
#define HL_INFO_ENGINE_STATUS 32
+#define HL_INFO_PAGE_FAULT_EVENT 33
+#define HL_INFO_USER_MAPPINGS 34
#define HL_INFO_VERSION_MAX_LEN 128
#define HL_INFO_CARD_NAME_MAX_LEN 16
* @number_of_user_interrupts: The number of interrupts that are available to the userspace
* application to use. Relevant for Gaudi2 and later.
* @device_mem_alloc_default_page_size: default page size used in device memory allocation.
+ * @revision_id: PCI revision ID of the ASIC.
*/
struct hl_info_hw_ip_info {
__u64 sram_base_address;
__u16 pad2;
__u64 reserved4;
__u64 device_mem_alloc_default_page_size;
+ __u64 reserved5;
+ __u64 reserved6;
+ __u32 reserved7;
+ __u8 reserved8;
+ __u8 revision_id;
+ __u8 pad[2];
};
struct hl_info_dram_usage {
__u64 ctx_dram_mem;
};
-#define HL_BUSY_ENGINES_MASK_EXT_SIZE 2
+#define HL_BUSY_ENGINES_MASK_EXT_SIZE 4
struct hl_info_hw_idle {
__u32 is_idle;
__u64 seq;
};
-#define HL_RAZWI_PAGE_FAULT 0
-#define HL_RAZWI_MMU_ACCESS_ERROR 1
+#define HL_RAZWI_NA_ENG_ID U16_MAX
+#define HL_RAZWI_MAX_NUM_OF_ENGINES_PER_RTR 128
+#define HL_RAZWI_READ BIT(0)
+#define HL_RAZWI_WRITE BIT(1)
+#define HL_RAZWI_LBW BIT(2)
+#define HL_RAZWI_HBW BIT(3)
+#define HL_RAZWI_RR BIT(4)
+#define HL_RAZWI_ADDR_DEC BIT(5)
/**
* struct hl_info_razwi_event - razwi information.
* @timestamp: timestamp of razwi.
* @addr: address which accessing it caused razwi.
- * @engine_id_1: engine id of the razwi initiator, if it was initiated by engine that does not
- * have engine id it will be set to U16_MAX.
- * @engine_id_2: second engine id of razwi initiator. Might happen that razwi have 2 possible
- * engines which one them caused the razwi. In that case, it will contain the
- * second possible engine id, otherwise it will be set to U16_MAX.
- * @no_engine_id: if razwi initiator does not have engine id, this field will be set to 1,
- * otherwise 0.
- * @error_type: cause of razwi, page fault or access error, otherwise it will be set to U8_MAX.
- * @pad: padding to 64 bit.
+ * @engine_id: engine id of the razwi initiator, if it was initiated by engine that does not
+ * have engine id it will be set to HL_RAZWI_NA_ENG_ID. If there are several possible
+ * engines which caused the razwi, it will hold all of them.
+ * @num_of_possible_engines: contains number of possible engine ids. In some asics, razwi indication
+ * might be common for several engines and there is no way to get the
+ * exact engine. In this way, engine_id array will be filled with all
+ * possible engines caused this razwi. Also, there might be possibility
+ * in gaudi, where we don't indication on specific engine, in that case
+ * the value of this parameter will be zero.
+ * @flags: bitmask for additional data: HL_RAZWI_READ - razwi caused by read operation
+ * HL_RAZWI_WRITE - razwi caused by write operation
+ * HL_RAZWI_LBW - razwi caused by lbw fabric transaction
+ * HL_RAZWI_HBW - razwi caused by hbw fabric transaction
+ * HL_RAZWI_RR - razwi caused by range register
+ * HL_RAZWI_ADDR_DEC - razwi caused by address decode error
+ * Note: this data is not supported by all asics, in that case the relevant bits will not
+ * be set.
*/
struct hl_info_razwi_event {
__s64 timestamp;
__u64 addr;
- __u16 engine_id_1;
- __u16 engine_id_2;
- __u8 no_engine_id;
- __u8 error_type;
- __u8 pad[2];
+ __u16 engine_id[HL_RAZWI_MAX_NUM_OF_ENGINES_PER_RTR];
+ __u16 num_of_possible_engines;
+ __u8 flags;
+ __u8 pad[5];
};
#define MAX_QMAN_STREAMS_INFO 4
__u8 pad0[2];
};
+/**
+ * struct hl_page_fault_info - page fault information.
+ * @timestamp: timestamp of page fault.
+ * @addr: address which accessing it caused page fault.
+ * @engine_id: engine id which caused the page fault, supported only in gaudi3.
+ */
+struct hl_page_fault_info {
+ __s64 timestamp;
+ __u64 addr;
+ __u16 engine_id;
+ __u8 pad[6];
+};
+
+/**
+ * struct hl_user_mapping - user mapping information.
+ * @dev_va: device virtual address.
+ * @size: virtual address mapping size.
+ */
+struct hl_user_mapping {
+ __u64 dev_va;
+ __u64 size;
+};
+
enum gaudi_dcores {
HL_GAUDI_WS_DCORE,
HL_GAUDI_WN_DCORE,
* needed, hence updating this variable so user will know the exact amount
* of bytes copied by the kernel to the buffer.
* @sec_attest_nonce: Nonce number used for attestation report.
+ * @array_size: Number of array members copied to user buffer.
+ * Relevant for HL_INFO_USER_MAPPINGS info ioctl.
* @pad: Padding to 64 bit.
*/
struct hl_info_args {
__u32 eventfd;
__u32 user_buffer_actual_size;
__u32 sec_attest_nonce;
+ __u32 array_size;
};
__u32 pad;
/* Should be run after espfix64 is set up. */
pti_init();
kmsan_init_runtime();
+ mm_cache_init();
}
#ifdef CONFIG_RANDOMIZE_KSTACK_OFFSET
sort_main_extable();
trap_init();
mm_init();
-
+ poking_init();
ftrace_init();
/* trace_printk can be enabled here */
taskstats_init_early();
delayacct_init();
- poking_init();
check_bugs();
acpi_subsystem_init();
#include <linux/printk.h>
#include <linux/uts.h>
#include <linux/utsname.h>
-#include <generated/utsrelease.h>
#include <linux/proc_ns.h>
static int __init early_hostname(char *arg)
refcount_set(&kvalue->refcnt, 1);
bpf_map_inc(map);
- set_memory_ro((long)st_map->image, 1);
- set_memory_x((long)st_map->image, 1);
+ set_memory_rox((long)st_map->image, 1);
err = st_ops->reg(kdata);
if (likely(!err)) {
/* Pair with smp_load_acquire() during lookup_elem().
list_add_tail(&pack->list, &pack_list);
set_vm_flush_reset_perms(pack->ptr);
- set_memory_ro((unsigned long)pack->ptr, BPF_PROG_PACK_SIZE / PAGE_SIZE);
- set_memory_x((unsigned long)pack->ptr, BPF_PROG_PACK_SIZE / PAGE_SIZE);
+ set_memory_rox((unsigned long)pack->ptr, BPF_PROG_PACK_SIZE / PAGE_SIZE);
return pack;
}
if (ptr) {
bpf_fill_ill_insns(ptr, size);
set_vm_flush_reset_perms(ptr);
- set_memory_ro((unsigned long)ptr, size / PAGE_SIZE);
- set_memory_x((unsigned long)ptr, size / PAGE_SIZE);
+ set_memory_rox((unsigned long)ptr, size / PAGE_SIZE);
}
goto out;
}
bool bpf_prog_map_compatible(struct bpf_map *map,
const struct bpf_prog *fp)
{
+ enum bpf_prog_type prog_type = resolve_prog_type(fp);
bool ret;
if (fp->kprobe_override)
/* There's no owner yet where we could check for
* compatibility.
*/
- map->owner.type = fp->type;
+ map->owner.type = prog_type;
map->owner.jited = fp->jited;
map->owner.xdp_has_frags = fp->aux->xdp_has_frags;
ret = true;
} else {
- ret = map->owner.type == fp->type &&
+ ret = map->owner.type == prog_type &&
map->owner.jited == fp->jited &&
map->owner.xdp_has_frags == fp->aux->xdp_has_frags;
}
__BPF_DISPATCHER_UPDATE(d, new ?: (void *)&bpf_dispatcher_nop_func);
+ /* Make sure all the callers executing the previous/old half of the
+ * image leave it, so following update call can modify it safely.
+ */
+ synchronize_rcu();
+
if (new)
d->image_off = noff;
}
case BPF_PROG_TYPE_LSM:
if (ptype == BPF_PROG_TYPE_LSM &&
prog->expected_attach_type != BPF_LSM_CGROUP)
- return -EINVAL;
-
- ret = cgroup_bpf_prog_attach(attr, ptype, prog);
+ ret = -EINVAL;
+ else
+ ret = cgroup_bpf_prog_attach(attr, ptype, prog);
break;
default:
ret = -EINVAL;
if (err < 0)
goto out;
- set_memory_ro((long)im->image, 1);
- set_memory_x((long)im->image, 1);
+ set_memory_rox((long)im->image, 1);
WARN_ON(tr->cur_image && tr->selector == 0);
WARN_ON(!tr->cur_image && tr->selector);
return pud_leaf_size(pud);
pmdp = pmd_offset_lockless(pudp, pud, addr);
- pmd = READ_ONCE(*pmdp);
+ pmd = pmdp_get_lockless(pmdp);
if (!pmd_present(pmd))
return 0;
return task;
}
-struct mm_struct *copy_init_mm(void)
-{
- return dup_mm(NULL, &init_mm);
-}
-
/*
* This is like kernel_clone(), but shaved down and tailored to just
* creating io_uring workers. It returns a created task, or an error pointer.
init_waitqueue_head(&sighand->signalfd_wqh);
}
-void __init proc_caches_init(void)
+void __init mm_cache_init(void)
{
unsigned int mm_size;
+ /*
+ * The mm_cpumask is located at the end of mm_struct, and is
+ * dynamically sized based on the maximum CPU number this system
+ * can have, taking hotplug into account (nr_cpu_ids).
+ */
+ mm_size = sizeof(struct mm_struct) + cpumask_size();
+
+ mm_cachep = kmem_cache_create_usercopy("mm_struct",
+ mm_size, ARCH_MIN_MMSTRUCT_ALIGN,
+ SLAB_HWCACHE_ALIGN|SLAB_PANIC|SLAB_ACCOUNT,
+ offsetof(struct mm_struct, saved_auxv),
+ sizeof_field(struct mm_struct, saved_auxv),
+ NULL);
+}
+
+void __init proc_caches_init(void)
+{
sighand_cachep = kmem_cache_create("sighand_cache",
sizeof(struct sighand_struct), 0,
SLAB_HWCACHE_ALIGN|SLAB_PANIC|SLAB_TYPESAFE_BY_RCU|
SLAB_HWCACHE_ALIGN|SLAB_PANIC|SLAB_ACCOUNT,
NULL);
- /*
- * The mm_cpumask is located at the end of mm_struct, and is
- * dynamically sized based on the maximum CPU number this system
- * can have, taking hotplug into account (nr_cpu_ids).
- */
- mm_size = sizeof(struct mm_struct) + cpumask_size();
-
- mm_cachep = kmem_cache_create_usercopy("mm_struct",
- mm_size, ARCH_MIN_MMSTRUCT_ALIGN,
- SLAB_HWCACHE_ALIGN|SLAB_PANIC|SLAB_ACCOUNT,
- offsetof(struct mm_struct, saved_auxv),
- sizeof_field(struct mm_struct, saved_auxv),
- NULL);
vm_area_cachep = KMEM_CACHE(vm_area_struct, SLAB_PANIC|SLAB_ACCOUNT);
mmap_init();
nsproxy_cache_init();
unsigned int hwsize;
if (WARN_ON_ONCE(ctrl->domid >= MSI_MAX_DEVICE_IRQDOMAINS ||
- !dev->msi.data->__domains[ctrl->domid].domain))
+ (dev->msi.domain &&
+ !dev->msi.data->__domains[ctrl->domid].domain)))
return false;
hwsize = msi_domain_get_hwsize(dev, ctrl->domid);
info = domain->host_data;
return info->hwsize;
}
- /* No domain, no size... */
- return 0;
+ /* No domain, default to MSI_XA_DOMAIN_SIZE */
+ return MSI_XA_DOMAIN_SIZE;
}
static inline void irq_chip_write_msi_msg(struct irq_data *data,
* Copyright (C) 2004 Kay Sievers <kay.sievers@vrfy.org>
*/
+#include <asm/byteorder.h>
#include <linux/kobject.h>
#include <linux/string.h>
#include <linux/sysfs.h>
#include <linux/rcupdate.h> /* rcu_expedited and rcu_normal */
+#if defined(__LITTLE_ENDIAN)
+#define CPU_BYTEORDER_STRING "little"
+#elif defined(__BIG_ENDIAN)
+#define CPU_BYTEORDER_STRING "big"
+#else
+#error Unknown byteorder
+#endif
+
#define KERNEL_ATTR_RO(_name) \
static struct kobj_attribute _name##_attr = __ATTR_RO(_name)
}
KERNEL_ATTR_RO(uevent_seqnum);
+/* cpu byteorder */
+static ssize_t cpu_byteorder_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return sysfs_emit(buf, "%s\n", CPU_BYTEORDER_STRING);
+}
+KERNEL_ATTR_RO(cpu_byteorder);
+
#ifdef CONFIG_UEVENT_HELPER
/* uevent helper program, used during early boot */
static ssize_t uevent_helper_show(struct kobject *kobj,
static struct attribute * kernel_attrs[] = {
&fscaps_attr.attr,
&uevent_seqnum_attr.attr,
+ &cpu_byteorder_attr.attr,
#ifdef CONFIG_UEVENT_HELPER
&uevent_helper_attr.attr,
#endif
info->hdr->e_machine);
goto no_exec;
}
+ if (!module_elf_check_arch(info->hdr)) {
+ pr_err("Invalid module architecture in ELF header: %u\n",
+ info->hdr->e_machine);
+ goto no_exec;
+ }
if (info->hdr->e_shentsize != sizeof(Elf_Shdr)) {
pr_err("Invalid ELF section header size\n");
goto no_exec;
(unsigned long)mod->core_layout.base + mod->core_layout.size);
}
+bool __weak module_elf_check_arch(Elf_Ehdr *hdr)
+{
+ return true;
+}
+
int __weak module_frob_arch_sections(Elf_Ehdr *hdr,
Elf_Shdr *sechdrs,
char *secstrings,
return pw;
}
-static void padata_work_init(struct padata_work *pw, work_func_t work_fn,
- void *data, int flags)
+/*
+ * This function is marked __ref because this function may be optimized in such
+ * a way that it directly refers to work_fn's address, which causes modpost to
+ * complain when work_fn is marked __init. This scenario was observed with clang
+ * LTO, where padata_work_init() was optimized to refer directly to
+ * padata_mt_helper() because the calls to padata_work_init() with other work_fn
+ * values were eliminated or inlined.
+ */
+static void __ref padata_work_init(struct padata_work *pw, work_func_t work_fn,
+ void *data, int flags)
{
if (flags & PADATA_WORK_ONSTACK)
INIT_WORK_ONSTACK(&pw->pw_work, work_fn);
.store = module_attr_store,
};
-static int uevent_filter(struct kobject *kobj)
+static int uevent_filter(const struct kobject *kobj)
{
const struct kobj_type *ktype = get_ktype(kobj);
{
struct rcu_node *rnp = rcu_get_root();
- if (rcu_init_invoked())
+ if (rcu_scheduler_active != RCU_SCHEDULER_INACTIVE)
raw_lockdep_assert_held_rcu_node(rnp);
// If RCU was idle, note beginning of GP.
{
struct rcu_node *rnp = rcu_get_root();
- if (rcu_init_invoked())
+ if (rcu_scheduler_active != RCU_SCHEDULER_INACTIVE)
raw_lockdep_assert_held_rcu_node(rnp);
// If the previously noted GP is still in effect, record the
struct rcu_node *rnp = rcu_get_root();
if (rcu_init_invoked()) {
- lockdep_assert_irqs_enabled();
+ if (rcu_scheduler_active != RCU_SCHEDULER_INACTIVE)
+ lockdep_assert_irqs_enabled();
raw_spin_lock_irqsave_rcu_node(rnp, flags);
}
rcu_poll_gp_seq_start(snap);
struct rcu_node *rnp = rcu_get_root();
if (rcu_init_invoked()) {
- lockdep_assert_irqs_enabled();
+ if (rcu_scheduler_active != RCU_SCHEDULER_INACTIVE)
+ lockdep_assert_irqs_enabled();
raw_spin_lock_irqsave_rcu_node(rnp, flags);
}
rcu_poll_gp_seq_end(snap);
if (conflict->end > new->end)
new->end = conflict->end;
- printk("Expanded resource %s due to conflict with %s\n", new->name, conflict->name);
+ pr_info("Expanded resource %s due to conflict with %s\n", new->name, conflict->name);
}
write_unlock(&resource_lock);
}
write_unlock(&resource_lock);
- printk(KERN_WARNING "Trying to free nonexistent resource "
- "<%016llx-%016llx>\n", (unsigned long long)start,
- (unsigned long long)end);
+ pr_warn("Trying to free nonexistent resource <%pa-%pa>\n", &start, &end);
}
EXPORT_SYMBOL(__release_region);
int iomem_map_sanity_check(resource_size_t addr, unsigned long size)
{
struct resource *p = &iomem_resource;
+ resource_size_t end = addr + size - 1;
int err = 0;
loff_t l;
* We can probably skip the resources without
* IORESOURCE_IO attribute?
*/
- if (p->start >= addr + size)
+ if (p->start > end)
continue;
if (p->end < addr)
continue;
if (PFN_DOWN(p->start) <= PFN_DOWN(addr) &&
- PFN_DOWN(p->end) >= PFN_DOWN(addr + size - 1))
+ PFN_DOWN(p->end) >= PFN_DOWN(end))
continue;
/*
* if a resource is "BUSY", it's not a hardware resource
if (p->flags & IORESOURCE_BUSY)
continue;
- printk(KERN_WARNING "resource sanity check: requesting [mem %#010llx-%#010llx], which spans more than %s %pR\n",
- (unsigned long long)addr,
- (unsigned long long)(addr + size - 1),
- p->name, p);
+ pr_warn("resource sanity check: requesting [mem %pa-%pa], which spans more than %s %pR\n",
+ &addr, &end, p->name, p);
err = -1;
break;
}
help
Arch supports objtool --mcount
+config HAVE_OBJTOOL_NOP_MCOUNT
+ bool
+ help
+ Arch supports the objtool options --mcount with --mnop.
+ An architecture can select this if it wants to enable nop'ing
+ of ftrace locations.
+
config HAVE_C_RECORDMCOUNT
bool
help
config HWLAT_TRACER
bool "Tracer to detect hardware latencies (like SMIs)"
select GENERIC_TRACER
+ select TRACER_MAX_TRACE
help
This tracer, when enabled will create one or more kernel threads,
depending on what the cpumask file is set to, which each thread
config OSNOISE_TRACER
bool "OS Noise tracer"
select GENERIC_TRACER
+ select TRACER_MAX_TRACE
help
In the context of high-performance computing (HPC), the Operating
System Noise (osnoise) refers to the interference experienced by an
static ftrace_func_t ftrace_ops_get_list_func(struct ftrace_ops *ops)
{
/*
- * If this is a dynamic, RCU, or per CPU ops, or we force list func,
+ * If this is a dynamic or RCU ops, or we force list func,
* then it needs to call the list anyway.
*/
if (ops->flags & (FTRACE_OPS_FL_DYNAMIC | FTRACE_OPS_FL_RCU) ||
{
}
+static int update_ftrace_func(ftrace_func_t func)
+{
+ static ftrace_func_t save_func;
+
+ /* Avoid updating if it hasn't changed */
+ if (func == save_func)
+ return 0;
+
+ save_func = func;
+
+ return ftrace_update_ftrace_func(func);
+}
+
void ftrace_modify_all_code(int command)
{
int update = command & FTRACE_UPDATE_TRACE_FUNC;
* traced.
*/
if (update) {
- err = ftrace_update_ftrace_func(ftrace_ops_list_func);
+ err = update_ftrace_func(ftrace_ops_list_func);
if (FTRACE_WARN_ON(err))
return;
}
/* If irqs are disabled, we are in stop machine */
if (!irqs_disabled())
smp_call_function(ftrace_sync_ipi, NULL, 1);
- err = ftrace_update_ftrace_func(ftrace_trace_function);
+ err = update_ftrace_func(ftrace_trace_function);
if (FTRACE_WARN_ON(err))
return;
}
/*
* Dynamic ops may be freed, we must make sure that all
* callers are done before leaving this function.
- * The same goes for freeing the per_cpu data of the per_cpu
- * ops.
*/
if (ops->flags & FTRACE_OPS_FL_DYNAMIC) {
/*
}
found = 1;
}
+ cond_resched();
} while_for_each_ftrace_rec();
out_unlock:
mutex_unlock(&ftrace_lock);
/*
* Check the following for each ops before calling their func:
* if RCU flag is set, then rcu_is_watching() must be true
- * if PER_CPU is set, then ftrace_function_local_disable()
- * must be false
* Otherwise test if the ip matches the ops filter
*
* If any of the above fails then the op->func() is not executed.
/*
* If there's only one function registered but it does not support
- * recursion, needs RCU protection and/or requires per cpu handling, then
- * this function will be called by the mcount trampoline.
+ * recursion, needs RCU protection, then this function will be called
+ * by the mcount trampoline.
*/
static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
struct ftrace_ops *op, struct ftrace_regs *fregs)
{
struct list_head *pages = &cpu_buffer->new_pages;
int retries, success;
+ unsigned long flags;
- raw_spin_lock_irq(&cpu_buffer->reader_lock);
+ /* Can be called at early boot up, where interrupts must not been enabled */
+ raw_spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
/*
* We are holding the reader lock, so the reader page won't be swapped
* in the ring buffer. Now we are racing with the writer trying to
* tracing
*/
RB_WARN_ON(cpu_buffer, !success);
- raw_spin_unlock_irq(&cpu_buffer->reader_lock);
+ raw_spin_unlock_irqrestore(&cpu_buffer->reader_lock, flags);
/* free pages if they weren't inserted */
if (!success) {
rb_update_pages(cpu_buffer);
cpu_buffer->nr_pages_to_update = 0;
} else {
- schedule_work_on(cpu,
- &cpu_buffer->update_pages_work);
+ /* Run directly if possible. */
+ migrate_disable();
+ if (cpu != smp_processor_id()) {
+ migrate_enable();
+ schedule_work_on(cpu,
+ &cpu_buffer->update_pages_work);
+ } else {
+ update_pages_handler(&cpu_buffer->update_pages_work);
+ migrate_enable();
+ }
}
}
if (!cpu_online(cpu_id))
rb_update_pages(cpu_buffer);
else {
- schedule_work_on(cpu_id,
- &cpu_buffer->update_pages_work);
- wait_for_completion(&cpu_buffer->update_done);
+ /* Run directly if possible. */
+ migrate_disable();
+ if (cpu_id == smp_processor_id()) {
+ rb_update_pages(cpu_buffer);
+ migrate_enable();
+ } else {
+ migrate_enable();
+ schedule_work_on(cpu_id,
+ &cpu_buffer->update_pages_work);
+ wait_for_completion(&cpu_buffer->update_done);
+ }
}
cpu_buffer->nr_pages_to_update = 0;
event->time_delta = 1;
}
-static void rb_commit(struct ring_buffer_per_cpu *cpu_buffer,
- struct ring_buffer_event *event)
+static void rb_commit(struct ring_buffer_per_cpu *cpu_buffer)
{
local_inc(&cpu_buffer->entries);
rb_end_commit(cpu_buffer);
*
* Must be paired with ring_buffer_lock_reserve.
*/
-int ring_buffer_unlock_commit(struct trace_buffer *buffer,
- struct ring_buffer_event *event)
+int ring_buffer_unlock_commit(struct trace_buffer *buffer)
{
struct ring_buffer_per_cpu *cpu_buffer;
int cpu = raw_smp_processor_id();
cpu_buffer = buffer->buffers[cpu];
- rb_commit(cpu_buffer, event);
+ rb_commit(cpu_buffer);
rb_wakeups(buffer, cpu_buffer);
memcpy(body, data, length);
- rb_commit(cpu_buffer, event);
+ rb_commit(cpu_buffer);
rb_wakeups(buffer, cpu_buffer);
}
out:
- ring_buffer_unlock_commit(data->buffer, event);
+ ring_buffer_unlock_commit(data->buffer);
return 0;
}
hit++;
entry = ring_buffer_event_data(event);
*entry = smp_processor_id();
- ring_buffer_unlock_commit(buffer, event);
+ ring_buffer_unlock_commit(buffer);
}
}
end_time = ktime_get();
#include <linux/kallsyms.h>
#include <linux/security.h>
#include <linux/seq_file.h>
-#include <linux/notifier.h>
#include <linux/irqflags.h>
#include <linux/debugfs.h>
#include <linux/tracefs.h>
#endif
/* Pipe tracepoints to printk */
-struct trace_iterator *tracepoint_print_iter;
+static struct trace_iterator *tracepoint_print_iter;
int tracepoint_printk;
static bool tracepoint_printk_stop_on_boot __initdata;
static DEFINE_STATIC_KEY_FALSE(tracepoint_printk_key);
/* ring_buffer_unlock_commit() enables preemption */
preempt_enable_notrace();
} else
- ring_buffer_unlock_commit(buffer, event);
+ ring_buffer_unlock_commit(buffer);
}
/**
return false;
}
EXPORT_SYMBOL_GPL(tracing_snapshot_cond_disable);
+#define free_snapshot(tr) do { } while (0)
#endif /* CONFIG_TRACER_SNAPSHOT */
void tracer_tracing_off(struct trace_array *tr)
}
unsigned long __read_mostly tracing_thresh;
+
+#ifdef CONFIG_TRACER_MAX_TRACE
static const struct file_operations tracing_max_lat_fops;
#ifdef LATENCY_FS_NOTIFY
irq_work_queue(&tr->fsnotify_irqwork);
}
-#elif defined(CONFIG_TRACER_MAX_TRACE) || defined(CONFIG_HWLAT_TRACER) \
- || defined(CONFIG_OSNOISE_TRACER)
+#else /* !LATENCY_FS_NOTIFY */
#define trace_create_maxlat_file(tr, d_tracer) \
trace_create_file("tracing_max_latency", TRACE_MODE_WRITE, \
d_tracer, &tr->max_latency, &tracing_max_lat_fops)
-#else
-#define trace_create_maxlat_file(tr, d_tracer) do { } while (0)
#endif
-#ifdef CONFIG_TRACER_MAX_TRACE
/*
* Copy the new maximum trace into the separate maximum-trace
* structure. (this way the maximum trace is permanently saved,
ring_buffer_record_off(tr->max_buffer.buffer);
#ifdef CONFIG_TRACER_SNAPSHOT
- if (tr->cond_snapshot && !tr->cond_snapshot->update(tr, cond_data))
- goto out_unlock;
+ if (tr->cond_snapshot && !tr->cond_snapshot->update(tr, cond_data)) {
+ arch_spin_unlock(&tr->max_lock);
+ return;
+ }
#endif
swap(tr->array_buffer.buffer, tr->max_buffer.buffer);
__update_max_tr(tr, tsk, cpu);
- out_unlock:
arch_spin_unlock(&tr->max_lock);
}
__update_max_tr(tr, tsk, cpu);
arch_spin_unlock(&tr->max_lock);
}
+
#endif /* CONFIG_TRACER_MAX_TRACE */
static int wait_on_pipe(struct trace_iterator *iter, int full)
"\t [:size=#entries]\n"
"\t [:pause][:continue][:clear]\n"
"\t [:name=histname1]\n"
+ "\t [:nohitcount]\n"
"\t [:<handler>.<action>]\n"
"\t [if <filter>]\n\n"
"\t Note, special fields can be used as well:\n"
"\t .syscall display a syscall id as a syscall name\n"
"\t .log2 display log2 value rather than raw number\n"
"\t .buckets=size display values in groups of size rather than raw number\n"
- "\t .usecs display a common_timestamp in microseconds\n\n"
+ "\t .usecs display a common_timestamp in microseconds\n"
+ "\t .percent display a number of percentage value\n"
+ "\t .graph display a bar-graph of a value\n\n"
"\t The 'pause' parameter can be used to pause an existing hist\n"
"\t trigger or to start a hist trigger but not log any events\n"
"\t until told to do so. 'continue' can be used to start or\n"
"\t The 'clear' parameter will clear the contents of a running\n"
"\t hist trigger and leave its current paused/active state\n"
"\t unchanged.\n\n"
+ "\t The 'nohitcount' (or NOHC) parameter will suppress display of\n"
+ "\t raw hitcount in the histogram.\n\n"
"\t The enable_hist and disable_hist triggers can be used to\n"
"\t have one event conditionally start and stop another event's\n"
"\t already-attached hist trigger. The syntax is analogous to\n"
return ret;
}
-#if defined(CONFIG_TRACER_MAX_TRACE) || defined(CONFIG_HWLAT_TRACER)
+#ifdef CONFIG_TRACER_MAX_TRACE
static ssize_t
tracing_max_lat_read(struct file *filp, char __user *ubuf,
ret = print_trace_line(iter);
if (ret == TRACE_TYPE_PARTIAL_LINE) {
- /* don't print partial lines */
+ /*
+ * If one print_trace_line() fills entire trace_seq in one shot,
+ * trace_seq_to_user() will returns -EBUSY because save_len == 0,
+ * In this case, we need to consume it, otherwise, loop will peek
+ * this event next time, resulting in an infinite loop.
+ */
+ if (save_len == 0) {
+ iter->seq.full = 0;
+ trace_seq_puts(&iter->seq, "[LINE TOO BIG]\n");
+ trace_consume(iter);
+ break;
+ }
+
+ /* In other cases, don't print partial lines */
iter->seq.seq.len = save_len;
break;
}
.llseek = generic_file_llseek,
};
-#if defined(CONFIG_TRACER_MAX_TRACE) || defined(CONFIG_HWLAT_TRACER)
+#ifdef CONFIG_TRACER_MAX_TRACE
static const struct file_operations tracing_max_lat_fops = {
.open = tracing_open_generic,
.read = tracing_max_lat_read,
create_trace_options_dir(tr);
+#ifdef CONFIG_TRACER_MAX_TRACE
trace_create_maxlat_file(tr, d_tracer);
+#endif
if (ftrace_create_function_files(tr, d_tracer))
MEM_FAIL(1, "Could not allocate function filter files");
fs_initcall(tracer_init_tracefs);
-static int trace_panic_handler(struct notifier_block *this,
- unsigned long event, void *unused)
-{
- if (ftrace_dump_on_oops)
- ftrace_dump(ftrace_dump_on_oops);
- return NOTIFY_OK;
-}
+static int trace_die_panic_handler(struct notifier_block *self,
+ unsigned long ev, void *unused);
static struct notifier_block trace_panic_notifier = {
- .notifier_call = trace_panic_handler,
- .next = NULL,
- .priority = 150 /* priority: INT_MAX >= x >= 0 */
+ .notifier_call = trace_die_panic_handler,
+ .priority = INT_MAX - 1,
};
-static int trace_die_handler(struct notifier_block *self,
- unsigned long val,
- void *data)
-{
- switch (val) {
- case DIE_OOPS:
- if (ftrace_dump_on_oops)
- ftrace_dump(ftrace_dump_on_oops);
- break;
- default:
- break;
- }
- return NOTIFY_OK;
-}
-
static struct notifier_block trace_die_notifier = {
- .notifier_call = trace_die_handler,
- .priority = 200
+ .notifier_call = trace_die_panic_handler,
+ .priority = INT_MAX - 1,
};
/*
+ * The idea is to execute the following die/panic callback early, in order
+ * to avoid showing irrelevant information in the trace (like other panic
+ * notifier functions); we are the 2nd to run, after hung_task/rcu_stall
+ * warnings get disabled (to prevent potential log flooding).
+ */
+static int trace_die_panic_handler(struct notifier_block *self,
+ unsigned long ev, void *unused)
+{
+ if (!ftrace_dump_on_oops)
+ return NOTIFY_DONE;
+
+ /* The die notifier requires DIE_OOPS to trigger */
+ if (self == &trace_die_notifier && ev != DIE_OOPS)
+ return NOTIFY_DONE;
+
+ ftrace_dump(ftrace_dump_on_oops);
+
+ return NOTIFY_DONE;
+}
+
+/*
* printk is set to max of 1024, we really don't need it that big.
* Nothing should be printing 1000 characters anyway.
*/
struct array_buffer max_buffer;
bool allocated_snapshot;
#endif
-#if defined(CONFIG_TRACER_MAX_TRACE) || defined(CONFIG_HWLAT_TRACER) \
- || defined(CONFIG_OSNOISE_TRACER)
+#ifdef CONFIG_TRACER_MAX_TRACE
unsigned long max_latency;
#ifdef CONFIG_FSNOTIFY
struct dentry *d_max_latency;
bool trace_is_tracepoint_string(const char *str);
const char *trace_event_format(struct trace_iterator *iter, const char *fmt);
void trace_check_vprintf(struct trace_iterator *iter, const char *fmt,
- va_list ap);
+ va_list ap) __printf(2, 0);
int trace_empty(struct trace_iterator *iter);
void *cond_data);
void update_max_tr_single(struct trace_array *tr,
struct task_struct *tsk, int cpu);
-#endif /* CONFIG_TRACER_MAX_TRACE */
-#if (defined(CONFIG_TRACER_MAX_TRACE) || defined(CONFIG_HWLAT_TRACER) \
- || defined(CONFIG_OSNOISE_TRACER)) && defined(CONFIG_FSNOTIFY)
+#ifdef CONFIG_FSNOTIFY
#define LATENCY_FS_NOTIFY
#endif
+#endif /* CONFIG_TRACER_MAX_TRACE */
#ifdef LATENCY_FS_NOTIFY
void latency_fsnotify(struct trace_array *tr);
static inline void tracer_hardirqs_off(unsigned long a0, unsigned long a1) { }
#endif
-extern struct trace_iterator *tracepoint_print_iter;
-
/*
* Reset the state of the trace_iterator so that it can read consumed data.
* Normally, the trace_iterator is used for reading the data when it is not
}
/* Check the name is good for event/group/fields */
-static inline bool is_good_name(const char *name)
+static inline bool __is_good_name(const char *name, bool hash_ok)
{
- if (!isalpha(*name) && *name != '_')
+ if (!isalpha(*name) && *name != '_' && (!hash_ok || *name != '-'))
return false;
while (*++name != '\0') {
- if (!isalpha(*name) && !isdigit(*name) && *name != '_')
+ if (!isalpha(*name) && !isdigit(*name) && *name != '_' &&
+ (!hash_ok || *name != '-'))
return false;
}
return true;
}
+/* Check the name is good for event/group/fields */
+static inline bool is_good_name(const char *name)
+{
+ return __is_good_name(name, false);
+}
+
+/* Check the name is good for system */
+static inline bool is_good_system_name(const char *name)
+{
+ return __is_good_name(name, true);
+}
+
/* Convert certain expected symbols into '_' when generating event names */
static inline void sanitize_event_name(char *name)
{
struct trace_event_call *tp_event;
if (p_event->attr.kprobe_func) {
- func = kzalloc(KSYM_NAME_LEN, GFP_KERNEL);
- if (!func)
- return -ENOMEM;
- ret = strncpy_from_user(
- func, u64_to_user_ptr(p_event->attr.kprobe_func),
- KSYM_NAME_LEN);
- if (ret == KSYM_NAME_LEN)
- ret = -E2BIG;
- if (ret < 0)
- goto out;
+ func = strndup_user(u64_to_user_ptr(p_event->attr.kprobe_func),
+ KSYM_NAME_LEN);
+ if (IS_ERR(func)) {
+ ret = PTR_ERR(func);
+ return (ret == -EINVAL) ? -E2BIG : ret;
+ }
if (func[0] == '\0') {
kfree(func);
return file;
}
+#define MAX_BOOT_TRIGGERS 32
+
+static struct boot_triggers {
+ const char *event;
+ char *trigger;
+} bootup_triggers[MAX_BOOT_TRIGGERS];
+
+static char bootup_trigger_buf[COMMAND_LINE_SIZE];
+static int nr_boot_triggers;
+
+static __init int setup_trace_triggers(char *str)
+{
+ char *trigger;
+ char *buf;
+ int i;
+
+ strlcpy(bootup_trigger_buf, str, COMMAND_LINE_SIZE);
+ ring_buffer_expanded = true;
+ disable_tracing_selftest("running event triggers");
+
+ buf = bootup_trigger_buf;
+ for (i = 0; i < MAX_BOOT_TRIGGERS; i++) {
+ trigger = strsep(&buf, ",");
+ if (!trigger)
+ break;
+ bootup_triggers[i].event = strsep(&trigger, ".");
+ bootup_triggers[i].trigger = strsep(&trigger, ".");
+ if (!bootup_triggers[i].trigger)
+ break;
+ }
+
+ nr_boot_triggers = i;
+ return 1;
+}
+__setup("trace_trigger=", setup_trace_triggers);
+
/* Add an event to a trace directory */
static int
__trace_add_new_event(struct trace_event_call *call, struct trace_array *tr)
return event_define_fields(call);
}
+static void trace_early_triggers(struct trace_event_file *file, const char *name)
+{
+ int ret;
+ int i;
+
+ for (i = 0; i < nr_boot_triggers; i++) {
+ if (strcmp(name, bootup_triggers[i].event))
+ continue;
+ mutex_lock(&event_mutex);
+ ret = trigger_process_regex(file, bootup_triggers[i].trigger);
+ mutex_unlock(&event_mutex);
+ if (ret)
+ pr_err("Failed to register trigger '%s' on event %s\n",
+ bootup_triggers[i].trigger,
+ bootup_triggers[i].event);
+ }
+}
+
/*
* Just create a descriptor for early init. A descriptor is required
* for enabling events at boot. We want to enable events before
struct trace_array *tr)
{
struct trace_event_file *file;
+ int ret;
file = trace_create_new_event(call, tr);
if (!file)
return -ENOMEM;
- return event_define_fields(call);
+ ret = event_define_fields(call);
+ if (ret)
+ return ret;
+
+ trace_early_triggers(file, trace_event_name(call));
+
+ return 0;
}
struct ftrace_module_file_ops;
list_add(&call->list, &ftrace_events);
}
+ register_trigger_cmds();
+
/*
* We need the top trace array to have a working set of trace
* points at early init, before the debug files and directories
register_event_cmds();
- register_trigger_cmds();
return 0;
}
C(INVALID_STR_OPERAND, "String type can not be an operand in expression"), \
C(EXPECT_NUMBER, "Expecting numeric literal"), \
C(UNARY_MINUS_SUBEXPR, "Unary minus not supported in sub-expressions"), \
- C(DIVISION_BY_ZERO, "Division by zero"),
+ C(DIVISION_BY_ZERO, "Division by zero"), \
+ C(NEED_NOHC_VAL, "Non-hitcount value is required for 'nohitcount'"),
#undef C
#define C(a, b) HIST_ERR_##a
HIST_FIELD_FL_ALIAS = 1 << 16,
HIST_FIELD_FL_BUCKET = 1 << 17,
HIST_FIELD_FL_CONST = 1 << 18,
+ HIST_FIELD_FL_PERCENT = 1 << 19,
+ HIST_FIELD_FL_GRAPH = 1 << 20,
};
struct var_defs {
bool cont;
bool clear;
bool ts_in_usecs;
+ bool no_hitcount;
unsigned int map_bits;
char *assignment_str[TRACING_MAP_VARS_MAX];
* event param, and is passed to the synthetic event
* invocation.
*/
- unsigned int var_ref_idx[TRACING_MAP_VARS_MAX];
+ unsigned int var_ref_idx[SYNTH_FIELDS_MAX];
struct synth_event *synth_event;
bool use_trace_keyword;
char *synth_event_name;
field_name = field->name;
} else if (field->flags & HIST_FIELD_FL_TIMESTAMP)
field_name = "common_timestamp";
+ else if (field->flags & HIST_FIELD_FL_HITCOUNT)
+ field_name = "hitcount";
if (field_name == NULL)
field_name = "";
ret = parse_assignment(tr, str, attrs);
if (ret)
goto free;
- } else if (strcmp(str, "pause") == 0)
+ } else if (strcmp(str, "nohitcount") == 0 ||
+ strcmp(str, "NOHC") == 0)
+ attrs->no_hitcount = true;
+ else if (strcmp(str, "pause") == 0)
attrs->pause = true;
else if ((strcmp(str, "cont") == 0) ||
(strcmp(str, "continue") == 0))
flags_str = "buckets";
else if (hist_field->flags & HIST_FIELD_FL_TIMESTAMP_USECS)
flags_str = "usecs";
+ else if (hist_field->flags & HIST_FIELD_FL_PERCENT)
+ flags_str = "percent";
+ else if (hist_field->flags & HIST_FIELD_FL_GRAPH)
+ flags_str = "graph";
return flags_str;
}
return ref_field;
}
}
-
+ /* Sanity check to avoid out-of-bound write on 'hist_data->var_refs' */
+ if (hist_data->n_var_refs >= TRACING_MAP_VARS_MAX)
+ return NULL;
ref_field = create_hist_field(var_field->hist_data, NULL, flags, NULL);
if (ref_field) {
if (init_var_ref(ref_field, var_field, system, event_name)) {
if (ret || !(*buckets))
goto error;
*flags |= HIST_FIELD_FL_BUCKET;
+ } else if (strncmp(modifier, "percent", 7) == 0) {
+ if (*flags & (HIST_FIELD_FL_VAR | HIST_FIELD_FL_KEY))
+ goto error;
+ *flags |= HIST_FIELD_FL_PERCENT;
+ } else if (strncmp(modifier, "graph", 5) == 0) {
+ if (*flags & (HIST_FIELD_FL_VAR | HIST_FIELD_FL_KEY))
+ goto error;
+ *flags |= HIST_FIELD_FL_GRAPH;
} else {
error:
hist_err(tr, HIST_ERR_BAD_FIELD_MODIFIER, errpos(modifier));
hist_data->attrs->ts_in_usecs = true;
} else if (strcmp(field_name, "common_cpu") == 0)
*flags |= HIST_FIELD_FL_CPU;
+ else if (strcmp(field_name, "hitcount") == 0)
+ *flags |= HIST_FIELD_FL_HITCOUNT;
else {
field = trace_find_event_field(file->event_call, field_name);
if (!field || !field->size) {
while (params) {
if (data->n_params >= SYNTH_FIELDS_MAX) {
hist_err(tr, HIST_ERR_TOO_MANY_PARAMS, 0);
+ ret = -EINVAL;
goto out;
}
lockdep_assert_held(&event_mutex);
+ /* Sanity check to avoid out-of-bound write on 'data->var_ref_idx' */
+ if (data->n_params > SYNTH_FIELDS_MAX)
+ return -EINVAL;
+
if (data->use_trace_keyword)
synth_event_name = data->synth_event_name;
else
static int create_val_fields(struct hist_trigger_data *hist_data,
struct trace_event_file *file)
{
+ unsigned int i, j = 1, n_hitcount = 0;
char *fields_str, *field_str;
- unsigned int i, j = 1;
int ret;
ret = create_hitcount_val(hist_data);
if (!field_str)
break;
- if (strcmp(field_str, "hitcount") == 0)
- continue;
+ if (strcmp(field_str, "hitcount") == 0) {
+ if (!n_hitcount++)
+ continue;
+ }
ret = create_val_field(hist_data, j++, file, field_str);
if (ret)
if (fields_str && (strcmp(fields_str, "hitcount") != 0))
ret = -EINVAL;
out:
+ /* There is only raw hitcount but nohitcount suppresses it. */
+ if (j == 1 && hist_data->attrs->no_hitcount) {
+ hist_err(hist_data->event_file->tr, HIST_ERR_NEED_NOHC_VAL, 0);
+ ret = -ENOENT;
+ }
+
return ret;
}
seq_puts(m, "}");
}
+/* Get the 100 times of the percentage of @val in @total */
+static inline unsigned int __get_percentage(u64 val, u64 total)
+{
+ if (!total)
+ goto div0;
+
+ if (val < (U64_MAX / 10000))
+ return (unsigned int)div64_ul(val * 10000, total);
+
+ total = div64_u64(total, 10000);
+ if (!total)
+ goto div0;
+
+ return (unsigned int)div64_ul(val, total);
+div0:
+ return val ? UINT_MAX : 0;
+}
+
+#define BAR_CHAR '#'
+
+static inline const char *__fill_bar_str(char *buf, int size, u64 val, u64 max)
+{
+ unsigned int len = __get_percentage(val, max);
+ int i;
+
+ if (len == UINT_MAX) {
+ snprintf(buf, size, "[ERROR]");
+ return buf;
+ }
+
+ len = len * size / 10000;
+ for (i = 0; i < len && i < size; i++)
+ buf[i] = BAR_CHAR;
+ while (i < size)
+ buf[i++] = ' ';
+ buf[size] = '\0';
+
+ return buf;
+}
+
+struct hist_val_stat {
+ u64 max;
+ u64 total;
+};
+
+static void hist_trigger_print_val(struct seq_file *m, unsigned int idx,
+ const char *field_name, unsigned long flags,
+ struct hist_val_stat *stats,
+ struct tracing_map_elt *elt)
+{
+ u64 val = tracing_map_read_sum(elt, idx);
+ unsigned int pc;
+ char bar[21];
+
+ if (flags & HIST_FIELD_FL_PERCENT) {
+ pc = __get_percentage(val, stats[idx].total);
+ if (pc == UINT_MAX)
+ seq_printf(m, " %s (%%):[ERROR]", field_name);
+ else
+ seq_printf(m, " %s (%%): %3u.%02u", field_name,
+ pc / 100, pc % 100);
+ } else if (flags & HIST_FIELD_FL_GRAPH) {
+ seq_printf(m, " %s: %20s", field_name,
+ __fill_bar_str(bar, 20, val, stats[idx].max));
+ } else if (flags & HIST_FIELD_FL_HEX) {
+ seq_printf(m, " %s: %10llx", field_name, val);
+ } else {
+ seq_printf(m, " %s: %10llu", field_name, val);
+ }
+}
+
static void hist_trigger_entry_print(struct seq_file *m,
struct hist_trigger_data *hist_data,
+ struct hist_val_stat *stats,
void *key,
struct tracing_map_elt *elt)
{
const char *field_name;
- unsigned int i;
+ unsigned int i = HITCOUNT_IDX;
+ unsigned long flags;
hist_trigger_print_key(m, hist_data, key, elt);
- seq_printf(m, " hitcount: %10llu",
- tracing_map_read_sum(elt, HITCOUNT_IDX));
+ /* At first, show the raw hitcount if !nohitcount */
+ if (!hist_data->attrs->no_hitcount)
+ hist_trigger_print_val(m, i, "hitcount", 0, stats, elt);
for (i = 1; i < hist_data->n_vals; i++) {
field_name = hist_field_name(hist_data->fields[i], 0);
-
- if (hist_data->fields[i]->flags & HIST_FIELD_FL_VAR ||
- hist_data->fields[i]->flags & HIST_FIELD_FL_EXPR)
+ flags = hist_data->fields[i]->flags;
+ if (flags & HIST_FIELD_FL_VAR || flags & HIST_FIELD_FL_EXPR)
continue;
- if (hist_data->fields[i]->flags & HIST_FIELD_FL_HEX) {
- seq_printf(m, " %s: %10llx", field_name,
- tracing_map_read_sum(elt, i));
- } else {
- seq_printf(m, " %s: %10llu", field_name,
- tracing_map_read_sum(elt, i));
- }
+ seq_puts(m, " ");
+ hist_trigger_print_val(m, i, field_name, flags, stats, elt);
}
print_actions(m, hist_data, elt);
{
struct tracing_map_sort_entry **sort_entries = NULL;
struct tracing_map *map = hist_data->map;
- int i, n_entries;
+ int i, j, n_entries;
+ struct hist_val_stat *stats = NULL;
+ u64 val;
n_entries = tracing_map_sort_entries(map, hist_data->sort_keys,
hist_data->n_sort_keys,
if (n_entries < 0)
return n_entries;
+ /* Calculate the max and the total for each field if needed. */
+ for (j = 0; j < hist_data->n_vals; j++) {
+ if (!(hist_data->fields[j]->flags &
+ (HIST_FIELD_FL_PERCENT | HIST_FIELD_FL_GRAPH)))
+ continue;
+ if (!stats) {
+ stats = kcalloc(hist_data->n_vals, sizeof(*stats),
+ GFP_KERNEL);
+ if (!stats) {
+ n_entries = -ENOMEM;
+ goto out;
+ }
+ }
+ for (i = 0; i < n_entries; i++) {
+ val = tracing_map_read_sum(sort_entries[i]->elt, j);
+ stats[j].total += val;
+ if (stats[j].max < val)
+ stats[j].max = val;
+ }
+ }
+
for (i = 0; i < n_entries; i++)
- hist_trigger_entry_print(m, hist_data,
+ hist_trigger_entry_print(m, hist_data, stats,
sort_entries[i]->key,
sort_entries[i]->elt);
+ kfree(stats);
+out:
tracing_map_destroy_sort_entries(sort_entries, n_entries);
return n_entries;
struct hist_trigger_data *hist_data = data->private_data;
struct hist_field *field;
bool have_var = false;
+ bool show_val = false;
unsigned int i;
seq_puts(m, HIST_PREFIX);
continue;
}
- if (i == HITCOUNT_IDX)
+ if (i == HITCOUNT_IDX) {
+ if (hist_data->attrs->no_hitcount)
+ continue;
seq_puts(m, "hitcount");
- else {
- seq_puts(m, ",");
+ } else {
+ if (show_val)
+ seq_puts(m, ",");
hist_field_print(m, field);
}
+ show_val = true;
}
if (have_var) {
seq_printf(m, ":size=%u", (1 << hist_data->map->map_bits));
if (hist_data->enable_timestamps)
seq_printf(m, ":clock=%s", hist_data->attrs->clock);
+ if (hist_data->attrs->no_hitcount)
+ seq_puts(m, ":nohitcount");
print_actions_spec(m, hist_data);
if (se)
se->ref++;
out:
- if (ret == 0)
+ if (ret == 0 && glob[0])
hist_err_clear();
return ret;
goto err_free_arg;
}
- fields[n_fields++] = field;
if (n_fields == SYNTH_FIELDS_MAX) {
synth_err(SYNTH_ERR_TOO_MANY_FIELDS, 0);
ret = -EINVAL;
goto err_free_arg;
}
+ fields[n_fields++] = field;
n_fields_this_loop++;
}
/* The filter is for the 'trigger' event, not the triggered event */
ret = create_event_filter(file->tr, file->event_call,
- filter_str, false, &filter);
+ filter_str, true, &filter);
+
+ /* Only enabled set_str for error handling */
+ if (filter) {
+ kfree(filter->filter_string);
+ filter->filter_string = NULL;
+ }
+
/*
* If create_event_filter() fails, filter still needs to be freed.
* Which the calling code will do with data->filter.
rcu_assign_pointer(data->filter, filter);
if (tmp) {
- /* Make sure the call is done with the filter */
- tracepoint_synchronize_unregister();
+ /*
+ * Make sure the call is done with the filter.
+ * It is possible that a filter could fail at boot up,
+ * and then this path will be called. Avoid the synchronization
+ * in that case.
+ */
+ if (system_state != SYSTEM_BOOTING)
+ tracepoint_synchronize_unregister();
free_event_filter(tmp);
}
put_user:
user_event_destroy_fields(user);
user_event_destroy_validators(user);
+ kfree(user->call.print_fmt);
kfree(user);
return ret;
}
return;
fbuffer.regs = regs;
- entry = fbuffer.entry = ring_buffer_event_data(fbuffer.event);
entry->ip = (unsigned long)tk->rp.kp.addr;
store_trace_args(&entry[1], &tk->tp, regs, sizeof(*entry), dsize);
return;
fbuffer.regs = regs;
- entry = fbuffer.entry = ring_buffer_event_data(fbuffer.event);
entry->func = (unsigned long)tk->rp.kp.addr;
entry->ret_ip = get_kretprobe_retaddr(ri);
store_trace_args(&entry[1], &tk->tp, regs, sizeof(*entry), dsize);
#define DEFAULT_TIMERLAT_PRIO 95 /* FIFO 95 */
/*
+ * osnoise/options entries.
+ */
+enum osnoise_options_index {
+ OSN_DEFAULTS = 0,
+ OSN_WORKLOAD,
+ OSN_PANIC_ON_STOP,
+ OSN_PREEMPT_DISABLE,
+ OSN_IRQ_DISABLE,
+ OSN_MAX
+};
+
+static const char * const osnoise_options_str[OSN_MAX] = {
+ "DEFAULTS",
+ "OSNOISE_WORKLOAD",
+ "PANIC_ON_STOP",
+ "OSNOISE_PREEMPT_DISABLE",
+ "OSNOISE_IRQ_DISABLE" };
+
+#define OSN_DEFAULT_OPTIONS 0x2
+static unsigned long osnoise_options = OSN_DEFAULT_OPTIONS;
+
+/*
* trace_array of the enabled osnoise/timerlat instances.
*/
struct osnoise_instance {
unsigned int prev_state)
{
struct osnoise_variables *osn_var = this_cpu_osn_var();
+ int workload = test_bit(OSN_WORKLOAD, &osnoise_options);
- if (p->pid != osn_var->pid)
+ if ((p->pid != osn_var->pid) || !workload)
thread_exit(osn_var, p);
- if (n->pid != osn_var->pid)
+ if ((n->pid != osn_var->pid) || !workload)
thread_entry(osn_var, n);
}
trace_array_printk_buf(tr->array_buffer.buffer, _THIS_IP_,
"stop tracing hit on cpu %d\n", smp_processor_id());
+ if (test_bit(OSN_PANIC_ON_STOP, &osnoise_options))
+ panic("tracer hit stop condition on CPU %d\n", smp_processor_id());
+
tracer_tracing_off(tr);
}
rcu_read_unlock();
*/
static int run_osnoise(void)
{
+ bool disable_irq = test_bit(OSN_IRQ_DISABLE, &osnoise_options);
struct osnoise_variables *osn_var = this_cpu_osn_var();
u64 start, sample, last_sample;
u64 last_int_count, int_count;
s64 noise = 0, max_noise = 0;
s64 total, last_total = 0;
struct osnoise_sample s;
+ bool disable_preemption;
unsigned int threshold;
u64 runtime, stop_in;
u64 sum_noise = 0;
int ret = -1;
/*
+ * Disabling preemption is only required if IRQs are enabled,
+ * and the options is set on.
+ */
+ disable_preemption = !disable_irq && test_bit(OSN_PREEMPT_DISABLE, &osnoise_options);
+
+ /*
* Considers the current thread as the workload.
*/
osn_var->pid = current->pid;
threshold = tracing_thresh ? : 5000;
/*
+ * Apply PREEMPT and IRQ disabled options.
+ */
+ if (disable_irq)
+ local_irq_disable();
+
+ if (disable_preemption)
+ preempt_disable();
+
+ /*
* Make sure NMIs see sampling first
*/
osn_var->sampling = true;
* cond_resched()
*/
if (IS_ENABLED(CONFIG_PREEMPT_RCU)) {
- local_irq_disable();
+ if (!disable_irq)
+ local_irq_disable();
+
rcu_momentary_dyntick_idle();
- local_irq_enable();
+
+ if (!disable_irq)
+ local_irq_enable();
}
/*
* For the non-preemptive kernel config: let threads runs, if
- * they so wish.
+ * they so wish, unless set not do to so.
*/
- cond_resched();
+ if (!disable_irq && !disable_preemption)
+ cond_resched();
last_sample = sample;
last_int_count = int_count;
barrier();
/*
+ * Return to the preemptive state.
+ */
+ if (disable_preemption)
+ preempt_enable();
+
+ if (disable_irq)
+ local_irq_enable();
+
+ /*
* Save noise info.
*/
s.noise = time_to_us(sum_noise);
struct task_struct *kthread;
kthread = per_cpu(per_cpu_osnoise_var, cpu).kthread;
- if (kthread)
+ if (kthread) {
kthread_stop(kthread);
- per_cpu(per_cpu_osnoise_var, cpu).kthread = NULL;
+ per_cpu(per_cpu_osnoise_var, cpu).kthread = NULL;
+ } else {
+ if (!test_bit(OSN_WORKLOAD, &osnoise_options)) {
+ per_cpu(per_cpu_osnoise_var, cpu).sampling = false;
+ barrier();
+ return;
+ }
+ }
}
/*
snprintf(comm, 24, "timerlat/%d", cpu);
main = timerlat_main;
} else {
+ /* if no workload, just return */
+ if (!test_bit(OSN_WORKLOAD, &osnoise_options)) {
+ per_cpu(per_cpu_osnoise_var, cpu).sampling = true;
+ barrier();
+ return 0;
+ }
+
snprintf(comm, 24, "osnoise/%d", cpu);
}
#endif /* CONFIG_HOTPLUG_CPU */
/*
+ * seq file functions for the osnoise/options file.
+ */
+static void *s_options_start(struct seq_file *s, loff_t *pos)
+{
+ int option = *pos;
+
+ mutex_lock(&interface_lock);
+
+ if (option >= OSN_MAX)
+ return NULL;
+
+ return pos;
+}
+
+static void *s_options_next(struct seq_file *s, void *v, loff_t *pos)
+{
+ int option = ++(*pos);
+
+ if (option >= OSN_MAX)
+ return NULL;
+
+ return pos;
+}
+
+static int s_options_show(struct seq_file *s, void *v)
+{
+ loff_t *pos = v;
+ int option = *pos;
+
+ if (option == OSN_DEFAULTS) {
+ if (osnoise_options == OSN_DEFAULT_OPTIONS)
+ seq_printf(s, "%s", osnoise_options_str[option]);
+ else
+ seq_printf(s, "NO_%s", osnoise_options_str[option]);
+ goto out;
+ }
+
+ if (test_bit(option, &osnoise_options))
+ seq_printf(s, "%s", osnoise_options_str[option]);
+ else
+ seq_printf(s, "NO_%s", osnoise_options_str[option]);
+
+out:
+ if (option != OSN_MAX)
+ seq_puts(s, " ");
+
+ return 0;
+}
+
+static void s_options_stop(struct seq_file *s, void *v)
+{
+ seq_puts(s, "\n");
+ mutex_unlock(&interface_lock);
+}
+
+static const struct seq_operations osnoise_options_seq_ops = {
+ .start = s_options_start,
+ .next = s_options_next,
+ .show = s_options_show,
+ .stop = s_options_stop
+};
+
+static int osnoise_options_open(struct inode *inode, struct file *file)
+{
+ return seq_open(file, &osnoise_options_seq_ops);
+};
+
+/**
+ * osnoise_options_write - Write function for "options" entry
+ * @filp: The active open file structure
+ * @ubuf: The user buffer that contains the value to write
+ * @cnt: The maximum number of bytes to write to "file"
+ * @ppos: The current position in @file
+ *
+ * Writing the option name sets the option, writing the "NO_"
+ * prefix in front of the option name disables it.
+ *
+ * Writing "DEFAULTS" resets the option values to the default ones.
+ */
+static ssize_t osnoise_options_write(struct file *filp, const char __user *ubuf,
+ size_t cnt, loff_t *ppos)
+{
+ int running, option, enable, retval;
+ char buf[256], *option_str;
+
+ if (cnt >= 256)
+ return -EINVAL;
+
+ if (copy_from_user(buf, ubuf, cnt))
+ return -EFAULT;
+
+ buf[cnt] = 0;
+
+ if (strncmp(buf, "NO_", 3)) {
+ option_str = strstrip(buf);
+ enable = true;
+ } else {
+ option_str = strstrip(&buf[3]);
+ enable = false;
+ }
+
+ option = match_string(osnoise_options_str, OSN_MAX, option_str);
+ if (option < 0)
+ return -EINVAL;
+
+ /*
+ * trace_types_lock is taken to avoid concurrency on start/stop.
+ */
+ mutex_lock(&trace_types_lock);
+ running = osnoise_has_registered_instances();
+ if (running)
+ stop_per_cpu_kthreads();
+
+ mutex_lock(&interface_lock);
+ /*
+ * avoid CPU hotplug operations that might read options.
+ */
+ cpus_read_lock();
+
+ retval = cnt;
+
+ if (enable) {
+ if (option == OSN_DEFAULTS)
+ osnoise_options = OSN_DEFAULT_OPTIONS;
+ else
+ set_bit(option, &osnoise_options);
+ } else {
+ if (option == OSN_DEFAULTS)
+ retval = -EINVAL;
+ else
+ clear_bit(option, &osnoise_options);
+ }
+
+ cpus_read_unlock();
+ mutex_unlock(&interface_lock);
+
+ if (running)
+ start_per_cpu_kthreads();
+ mutex_unlock(&trace_types_lock);
+
+ return retval;
+}
+
+/*
* osnoise_cpus_read - Read function for reading the "cpus" file
* @filp: The active open file structure
* @ubuf: The userspace provided buffer to read value into
.llseek = generic_file_llseek,
};
+static const struct file_operations osnoise_options_fops = {
+ .open = osnoise_options_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = seq_release,
+ .write = osnoise_options_write
+};
+
#ifdef CONFIG_TIMERLAT_TRACER
#ifdef CONFIG_STACKTRACE
static int init_timerlat_stack_tracefs(struct dentry *top_dir)
if (!tmp)
goto err;
+ tmp = trace_create_file("options", TRACE_MODE_WRITE, top_dir, NULL,
+ &osnoise_options_fops);
+ if (!tmp)
+ goto err;
+
ret = init_timerlat_tracefs(top_dir);
if (ret)
goto err;
#include <linux/kprobes.h>
#include <linux/sched/clock.h>
#include <linux/sched/mm.h>
+#include <linux/idr.h>
#include "trace_output.h"
static struct hlist_head event_hash[EVENT_HASHSIZE] __read_mostly;
-static int next_event_type = __TRACE_LAST_TYPE;
-
enum print_line_t trace_print_bputs_msg_only(struct trace_iterator *iter)
{
struct trace_seq *s = &iter->seq;
}
EXPORT_SYMBOL(trace_event_printf);
-static int trace_output_raw(struct trace_iterator *iter, char *name,
- char *fmt, va_list ap)
+static __printf(3, 0)
+int trace_output_raw(struct trace_iterator *iter, char *name,
+ char *fmt, va_list ap)
{
struct trace_seq *s = &iter->seq;
return NULL;
}
-static LIST_HEAD(ftrace_event_list);
+static DEFINE_IDA(trace_event_ida);
-static int trace_search_list(struct list_head **list)
+static void free_trace_event_type(int type)
{
- struct trace_event *e = NULL, *iter;
- int next = __TRACE_LAST_TYPE;
-
- if (list_empty(&ftrace_event_list)) {
- *list = &ftrace_event_list;
- return next;
- }
+ if (type >= __TRACE_LAST_TYPE)
+ ida_free(&trace_event_ida, type);
+}
- /*
- * We used up all possible max events,
- * lets see if somebody freed one.
- */
- list_for_each_entry(iter, &ftrace_event_list, list) {
- if (iter->type != next) {
- e = iter;
- break;
- }
- next++;
- }
+static int alloc_trace_event_type(void)
+{
+ int next;
- /* Did we used up all 65 thousand events??? */
- if (next > TRACE_EVENT_TYPE_MAX)
+ /* Skip static defined type numbers */
+ next = ida_alloc_range(&trace_event_ida, __TRACE_LAST_TYPE,
+ TRACE_EVENT_TYPE_MAX, GFP_KERNEL);
+ if (next < 0)
return 0;
-
- if (e)
- *list = &e->list;
- else
- *list = &ftrace_event_list;
return next;
}
if (WARN_ON(!event->funcs))
goto out;
- INIT_LIST_HEAD(&event->list);
-
if (!event->type) {
- struct list_head *list = NULL;
-
- if (next_event_type > TRACE_EVENT_TYPE_MAX) {
-
- event->type = trace_search_list(&list);
- if (!event->type)
- goto out;
-
- } else {
-
- event->type = next_event_type++;
- list = &ftrace_event_list;
- }
-
- if (WARN_ON(ftrace_find_event(event->type)))
+ event->type = alloc_trace_event_type();
+ if (!event->type)
goto out;
-
- list_add_tail(&event->list, list);
-
} else if (WARN(event->type > __TRACE_LAST_TYPE,
"Need to add type to trace.h")) {
goto out;
int __unregister_trace_event(struct trace_event *event)
{
hlist_del(&event->node);
- list_del(&event->list);
+ free_trace_event_type(event->type);
return 0;
}
return -EINVAL;
}
strlcpy(buf, event, slash - event + 1);
- if (!is_good_name(buf)) {
+ if (!is_good_system_name(buf)) {
trace_probe_log_err(offset, BAD_GROUP_NAME);
return -EINVAL;
}
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.
-config DEBUG_INFO_COMPRESSED
- bool "Compressed debugging information"
+choice
+ prompt "Compressed Debug information"
+ help
+ Compress the resulting debug info. Results in smaller debug info sections,
+ but requires that consumers are able to decompress the results.
+
+ If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
+
+config DEBUG_INFO_COMPRESSED_NONE
+ bool "Don't compress debug information"
+ help
+ Don't compress debug info sections.
+
+config DEBUG_INFO_COMPRESSED_ZLIB
+ bool "Compress debugging information with zlib"
depends on $(cc-option,-gz=zlib)
depends on $(ld-option,--compress-debug-sections=zlib)
help
preferable to setting $KDEB_COMPRESS to "none" which would be even
larger.
+config DEBUG_INFO_COMPRESSED_ZSTD
+ bool "Compress debugging information with zstd"
+ depends on $(cc-option,-gz=zstd)
+ depends on $(ld-option,--compress-debug-sections=zstd)
+ help
+ Compress the debug information using zstd. This may provide better
+ compression than zlib, for about the same time costs, but requires newer
+ toolchain support. Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
+ zstd.
+
+endchoice # "Compressed Debug information"
+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on $(cc-option,-gsplit-dwarf)
select STACKTRACE if STACKTRACE_SUPPORT
select KALLSYMS
select CRC32
+ select STACKDEPOT
help
Say Y here if you want to enable the memory leak
detector. The memory allocation/freeing is traced in a way
config FAULT_INJECTION_STACKTRACE_FILTER
bool "stacktrace filter for fault-injection capabilities"
depends on FAULT_INJECTION_DEBUG_FS && STACKTRACE_SUPPORT
- depends on !X86_64
select STACKTRACE
depends on FRAME_POINTER || MIPS || PPC || S390 || MICROBLAZE || ARM || ARC || X86
help
int n, nr_entries;
bool found = (attr->require_start == 0 && attr->require_end == ULONG_MAX);
- if (depth == 0)
+ if (depth == 0 || (found && !attr->reject_start && !attr->reject_end))
return found;
nr_entries = stack_trace_save(entries, depth, 1);
bool should_fail_ex(struct fault_attr *attr, ssize_t size, int flags)
{
+ bool stack_checked = false;
+
if (in_task()) {
unsigned int fail_nth = READ_ONCE(current->fail_nth);
if (fail_nth) {
+ if (!fail_stacktrace(attr))
+ return false;
+
+ stack_checked = true;
fail_nth--;
WRITE_ONCE(current->fail_nth, fail_nth);
if (!fail_nth)
if (atomic_read(&attr->times) == 0)
return false;
+ if (!stack_checked && !fail_stacktrace(attr))
+ return false;
+
if (atomic_read(&attr->space) > size) {
atomic_sub(size, &attr->space);
return false;
if (attr->probability <= get_random_u32_below(100))
return false;
- if (!fail_stacktrace(attr))
- return false;
-
fail:
if (!(flags & FAULT_NOWARN))
fail_dump(attr);
#ifdef CONFIG_FAULT_INJECTION_STACKTRACE_FILTER
debugfs_create_stacktrace_depth("stacktrace-depth", mode, dir,
&attr->stacktrace_depth);
- debugfs_create_ul("require-start", mode, dir, &attr->require_start);
- debugfs_create_ul("require-end", mode, dir, &attr->require_end);
- debugfs_create_ul("reject-start", mode, dir, &attr->reject_start);
- debugfs_create_ul("reject-end", mode, dir, &attr->reject_end);
+ debugfs_create_xul("require-start", mode, dir, &attr->require_start);
+ debugfs_create_xul("require-end", mode, dir, &attr->require_end);
+ debugfs_create_xul("reject-start", mode, dir, &attr->reject_start);
+ debugfs_create_xul("reject-end", mode, dir, &attr->reject_end);
#endif /* CONFIG_FAULT_INJECTION_STACKTRACE_FILTER */
attr->dname = dget(dir);
* and thus @kobj should have a namespace tag associated with it. Returns
* %NULL otherwise.
*/
-const void *kobject_namespace(struct kobject *kobj)
+const void *kobject_namespace(const struct kobject *kobj)
{
const struct kobj_ns_type_operations *ns_ops = kobj_ns_ops(kobj);
* representation of given kobject. Normally used to adjust ownership of
* objects in a container.
*/
-void kobject_get_ownership(struct kobject *kobj, kuid_t *uid, kgid_t *gid)
+void kobject_get_ownership(const struct kobject *kobj, kuid_t *uid, kgid_t *gid)
{
*uid = GLOBAL_ROOT_UID;
*gid = GLOBAL_ROOT_GID;
return 0;
}
-static int get_kobj_path_length(struct kobject *kobj)
+static int get_kobj_path_length(const struct kobject *kobj)
{
int length = 1;
- struct kobject *parent = kobj;
+ const struct kobject *parent = kobj;
/* walk up the ancestors until we hit the one pointing to the
* root.
return length;
}
-static void fill_kobj_path(struct kobject *kobj, char *path, int length)
+static void fill_kobj_path(const struct kobject *kobj, char *path, int length)
{
- struct kobject *parent;
+ const struct kobject *parent;
--length;
for (parent = kobj; parent; parent = parent->parent) {
*
* Return: The newly allocated memory, caller must free with kfree().
*/
-char *kobject_get_path(struct kobject *kobj, gfp_t gfp_mask)
+char *kobject_get_path(const struct kobject *kobj, gfp_t gfp_mask)
{
char *path;
int len;
/**
* kset_register() - Initialize and add a kset.
* @k: kset.
+ *
+ * NOTE: On error, the kset.kobj.name allocated by() kobj_set_name()
+ * is freed, it can not be used any more.
*/
int kset_register(struct kset *k)
{
kset_init(k);
err = kobject_add_internal(&k->kobj);
- if (err)
+ if (err) {
+ kfree_const(k->kobj.name);
+ /* Set it to NULL to avoid accessing bad pointer in callers. */
+ k->kobj.name = NULL;
return err;
+ }
kobject_uevent(&k->kobj, KOBJ_ADD);
return 0;
}
kfree(kset);
}
-static void kset_get_ownership(struct kobject *kobj, kuid_t *uid, kgid_t *gid)
+static void kset_get_ownership(const struct kobject *kobj, kuid_t *uid, kgid_t *gid)
{
if (kobj->parent)
kobject_get_ownership(kobj->parent, uid, gid);
return registered;
}
-const struct kobj_ns_type_operations *kobj_child_ns_ops(struct kobject *parent)
+const struct kobj_ns_type_operations *kobj_child_ns_ops(const struct kobject *parent)
{
const struct kobj_ns_type_operations *ops = NULL;
return ops;
}
-const struct kobj_ns_type_operations *kobj_ns_ops(struct kobject *kobj)
+const struct kobj_ns_type_operations *kobj_ns_ops(const struct kobject *kobj)
{
return kobj_child_ns_ops(kobj->parent);
}
if (mas->index < min)
return NULL;
- /* Retries on dead nodes handled by mas_next_entry */
+ /* Retries on dead nodes handled by mas_prev_entry */
return mas_prev_entry(mas, min);
}
EXPORT_SYMBOL_GPL(mas_find_rev);
test_fw_config->name);
else
len += scnprintf(buf + len, PAGE_SIZE - len,
- "name:\tEMTPY\n");
+ "name:\tEMPTY\n");
len += scnprintf(buf + len, PAGE_SIZE - len,
"num_requests:\t%u\n", test_fw_config->num_requests);
test_fw_config->upload_name);
else
len += scnprintf(buf + len, PAGE_SIZE - len,
- "upload_name:\tEMTPY\n");
+ "upload_name:\tEMPTY\n");
mutex_unlock(&test_fw_mutex);
rc = misc_register(&test_fw_misc_device);
if (rc) {
+ __test_firmware_config_free();
kfree(test_fw_config);
pr_err("could not register misc device: %d\n", rc);
return rc;
+// SPDX-License-Identifier: GPL-2.0-or-later OR copyleft-next-0.3.1
/*
* kmod stress test driver
*
* Copyright (C) 2017 Luis R. Rodriguez <mcgrof@kernel.org>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the Free
- * Software Foundation; either version 2 of the License, or at your option any
- * later version; or, when distributed separately from the Linux kernel or
- * when incorporated into other software packages, subject to the following
- * license:
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of copyleft-next (version 0.3.1 or later) as published
- * at http://copyleft-next.org/.
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+// SPDX-License-Identifier: GPL-2.0-or-later OR copyleft-next-0.3.1
/*
* proc sysctl test driver
*
* Copyright (C) 2017 Luis R. Rodriguez <mcgrof@kernel.org>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the Free
- * Software Foundation; either version 2 of the License, or at your option any
- * later version; or, when distributed separately from the Linux kernel or
- * when incorporated into other software packages, subject to the following
- * license:
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of copyleft-next (version 0.3.1 or later) as published
- * at http://copyleft-next.org/.
*/
/*
#ifdef CONFIG_TRACE_MMIO_ACCESS
void log_write_mmio(u64 val, u8 width, volatile void __iomem *addr,
- unsigned long caller_addr)
+ unsigned long caller_addr, unsigned long caller_addr0)
{
- trace_rwmmio_write(caller_addr, val, width, addr);
+ trace_rwmmio_write(caller_addr, caller_addr0, val, width, addr);
}
EXPORT_SYMBOL_GPL(log_write_mmio);
EXPORT_TRACEPOINT_SYMBOL_GPL(rwmmio_write);
void log_post_write_mmio(u64 val, u8 width, volatile void __iomem *addr,
- unsigned long caller_addr)
+ unsigned long caller_addr, unsigned long caller_addr0)
{
- trace_rwmmio_post_write(caller_addr, val, width, addr);
+ trace_rwmmio_post_write(caller_addr, caller_addr0, val, width, addr);
}
EXPORT_SYMBOL_GPL(log_post_write_mmio);
EXPORT_TRACEPOINT_SYMBOL_GPL(rwmmio_post_write);
void log_read_mmio(u8 width, const volatile void __iomem *addr,
- unsigned long caller_addr)
+ unsigned long caller_addr, unsigned long caller_addr0)
{
- trace_rwmmio_read(caller_addr, width, addr);
+ trace_rwmmio_read(caller_addr, caller_addr0, width, addr);
}
EXPORT_SYMBOL_GPL(log_read_mmio);
EXPORT_TRACEPOINT_SYMBOL_GPL(rwmmio_read);
void log_post_read_mmio(u64 val, u8 width, const volatile void __iomem *addr,
- unsigned long caller_addr)
+ unsigned long caller_addr, unsigned long caller_addr0)
{
- trace_rwmmio_post_read(caller_addr, val, width, addr);
+ trace_rwmmio_post_read(caller_addr, caller_addr0, val, width, addr);
}
EXPORT_SYMBOL_GPL(log_post_read_mmio);
EXPORT_TRACEPOINT_SYMBOL_GPL(rwmmio_post_read);
decompress/zstd_decompress_block.o \
zstd_common-y := \
+ zstd_common_module.o \
common/debug.o \
common/entropy_common.o \
common/error_private.o \
U32 const regMask = sizeof(bitContainer)*8 - 1;
/* if start > regMask, bitstream is corrupted, and result is undefined */
assert(nbBits < BIT_MASK_SIZE);
+ /* x86 transform & ((1 << nbBits) - 1) to bzhi instruction, it is better
+ * than accessing memory. When bmi2 instruction is not present, we consider
+ * such cpus old (pre-Haswell, 2013) and their performance is not of that
+ * importance.
+ */
+#if defined(__x86_64__) || defined(_M_X86)
+ return (bitContainer >> (start & regMask)) & ((((U64)1) << nbBits) - 1);
+#else
return (bitContainer >> (start & regMask)) & BIT_mask[nbBits];
+#endif
}
MEM_STATIC FORCE_INLINE_ATTR size_t BIT_getLowerBits(size_t bitContainer, U32 const nbBits)
#ifndef ZSTD_COMPILER_H
#define ZSTD_COMPILER_H
+#include "portability_macros.h"
+
/*-*******************************************************
* Compiler specifics
*********************************************************/
/*
On MSVC qsort requires that functions passed into it use the __cdecl calling conversion(CC).
- This explictly marks such functions as __cdecl so that the code will still compile
+ This explicitly marks such functions as __cdecl so that the code will still compile
if a CC other than __cdecl has been made the default.
*/
#define WIN_CDECL
/* target attribute */
-#ifndef __has_attribute
- #define __has_attribute(x) 0 /* Compatibility with non-clang compilers. */
-#endif
#define TARGET_ATTRIBUTE(target) __attribute__((__target__(target)))
-/* Enable runtime BMI2 dispatch based on the CPU.
- * Enabled for clang & gcc >=4.8 on x86 when BMI2 isn't enabled by default.
+/* Target attribute for BMI2 dynamic dispatch.
+ * Enable lzcnt, bmi, and bmi2.
+ * We test for bmi1 & bmi2. lzcnt is included in bmi1.
*/
-#ifndef DYNAMIC_BMI2
- #if ((defined(__clang__) && __has_attribute(__target__)) \
- || (defined(__GNUC__) \
- && (__GNUC__ >= 5 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8)))) \
- && (defined(__x86_64__) || defined(_M_X86)) \
- && !defined(__BMI2__)
- # define DYNAMIC_BMI2 1
- #else
- # define DYNAMIC_BMI2 0
- #endif
-#endif
+#define BMI2_TARGET_ATTRIBUTE TARGET_ATTRIBUTE("lzcnt,bmi,bmi2")
/* prefetch
* can be disabled, by declaring NO_PREFETCH build macro */
}
/* vectorization
- * older GCC (pre gcc-4.3 picked as the cutoff) uses a different syntax */
-#if !defined(__INTEL_COMPILER) && !defined(__clang__) && defined(__GNUC__)
+ * older GCC (pre gcc-4.3 picked as the cutoff) uses a different syntax,
+ * and some compilers, like Intel ICC and MCST LCC, do not support it at all. */
+#if !defined(__INTEL_COMPILER) && !defined(__clang__) && defined(__GNUC__) && !defined(__LCC__)
# if (__GNUC__ == 4 && __GNUC_MINOR__ > 3) || (__GNUC__ >= 5)
# define DONT_VECTORIZE __attribute__((optimize("no-tree-vectorize")))
# else
#define LIKELY(x) (__builtin_expect((x), 1))
#define UNLIKELY(x) (__builtin_expect((x), 0))
+#if __has_builtin(__builtin_unreachable) || (defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 5)))
+# define ZSTD_UNREACHABLE { assert(0), __builtin_unreachable(); }
+#else
+# define ZSTD_UNREACHABLE { assert(0); }
+#endif
+
/* disable warnings */
/*Like DYNAMIC_BMI2 but for compile time determination of BMI2 support*/
-/* compat. with non-clang compilers */
-#ifndef __has_builtin
-# define __has_builtin(x) 0
-#endif
-
-/* compat. with non-clang compilers */
-#ifndef __has_feature
-# define __has_feature(x) 0
-#endif
+/* compile time determination of SIMD support */
/* C-language Attributes are added in C23. */
#if defined(__STDC_VERSION__) && (__STDC_VERSION__ > 201710L) && defined(__has_c_attribute)
*/
#define ZSTD_FALLTHROUGH fallthrough
-/* detects whether we are being compiled under msan */
+/*-**************************************************************
+* Alignment check
+*****************************************************************/
+
+/* this test was initially positioned in mem.h,
+ * but this file is removed (or replaced) for linux kernel
+ * so it's now hosted in compiler.h,
+ * which remains valid for both user & kernel spaces.
+ */
+
+#ifndef ZSTD_ALIGNOF
+/* covers gcc, clang & MSVC */
+/* note : this section must come first, before C11,
+ * due to a limitation in the kernel source generator */
+# define ZSTD_ALIGNOF(T) __alignof(T)
+
+#endif /* ZSTD_ALIGNOF */
+/*-**************************************************************
+* Sanitizer
+*****************************************************************/
-/* detects whether we are being compiled under asan */
#endif /* ZSTD_COMPILER_H */
/* *************************************
* Dependencies
***************************************/
-#include <linux/module.h>
#include "mem.h"
#include "error_private.h" /* ERR_*, ERROR */
#define FSE_STATIC_LINKING_ONLY /* FSE_MIN_TABLELOG */
}
#if DYNAMIC_BMI2
-TARGET_ATTRIBUTE("bmi2") static size_t FSE_readNCount_body_bmi2(
+BMI2_TARGET_ATTRIBUTE static size_t FSE_readNCount_body_bmi2(
short* normalizedCounter, unsigned* maxSVPtr, unsigned* tableLogPtr,
const void* headerBuffer, size_t hbSize)
{
{
return FSE_readNCount_bmi2(normalizedCounter, maxSVPtr, tableLogPtr, headerBuffer, hbSize, /* bmi2 */ 0);
}
-EXPORT_SYMBOL_GPL(FSE_readNCount);
+
/*! HUF_readStats() :
Read compact Huffman tree, saved by HUF_writeCTable().
U32 wksp[HUF_READ_STATS_WORKSPACE_SIZE_U32];
return HUF_readStats_wksp(huffWeight, hwSize, rankStats, nbSymbolsPtr, tableLogPtr, src, srcSize, wksp, sizeof(wksp), /* bmi2 */ 0);
}
-EXPORT_SYMBOL_GPL(HUF_readStats);
FORCE_INLINE_TEMPLATE size_t
HUF_readStats_body(BYTE* huffWeight, size_t hwSize, U32* rankStats,
ZSTD_memset(rankStats, 0, (HUF_TABLELOG_MAX + 1) * sizeof(U32));
weightTotal = 0;
{ U32 n; for (n=0; n<oSize; n++) {
- if (huffWeight[n] >= HUF_TABLELOG_MAX) return ERROR(corruption_detected);
+ if (huffWeight[n] > HUF_TABLELOG_MAX) return ERROR(corruption_detected);
rankStats[huffWeight[n]]++;
weightTotal += (1 << huffWeight[n]) >> 1;
} }
}
#if DYNAMIC_BMI2
-static TARGET_ATTRIBUTE("bmi2") size_t HUF_readStats_body_bmi2(BYTE* huffWeight, size_t hwSize, U32* rankStats,
+static BMI2_TARGET_ATTRIBUTE size_t HUF_readStats_body_bmi2(BYTE* huffWeight, size_t hwSize, U32* rankStats,
U32* nbSymbolsPtr, U32* tableLogPtr,
const void* src, size_t srcSize,
void* workSpace, size_t wkspSize)
(void)bmi2;
return HUF_readStats_body_default(huffWeight, hwSize, rankStats, nbSymbolsPtr, tableLogPtr, src, srcSize, workSpace, wkspSize);
}
-EXPORT_SYMBOL_GPL(HUF_readStats_wksp);
/* ****************************************
* Dependencies
******************************************/
-#include "zstd_deps.h" /* size_t */
#include <linux/zstd_errors.h> /* enum list */
+#include "compiler.h"
+#include "debug.h"
+#include "zstd_deps.h" /* size_t */
/* ****************************************
return ERR_getErrorString(ERR_getErrorCode(code));
}
+/*
+ * Ignore: this is an internal helper.
+ *
+ * This is a helper function to help force C99-correctness during compilation.
+ * Under strict compilation modes, variadic macro arguments can't be empty.
+ * However, variadic function arguments can be. Using a function therefore lets
+ * us statically check that at least one (string) argument was passed,
+ * independent of the compilation flags.
+ */
+static INLINE_KEYWORD UNUSED_ATTR
+void _force_has_format_string(const char *format, ...) {
+ (void)format;
+}
+
+/*
+ * Ignore: this is an internal helper.
+ *
+ * We want to force this function invocation to be syntactically correct, but
+ * we don't want to force runtime evaluation of its arguments.
+ */
+#define _FORCE_HAS_FORMAT_STRING(...) \
+ if (0) { \
+ _force_has_format_string(__VA_ARGS__); \
+ }
+
+#define ERR_QUOTE(str) #str
+
+/*
+ * Return the specified error if the condition evaluates to true.
+ *
+ * In debug modes, prints additional information.
+ * In order to do that (particularly, printing the conditional that failed),
+ * this can't just wrap RETURN_ERROR().
+ */
+#define RETURN_ERROR_IF(cond, err, ...) \
+ if (cond) { \
+ RAWLOG(3, "%s:%d: ERROR!: check %s failed, returning %s", \
+ __FILE__, __LINE__, ERR_QUOTE(cond), ERR_QUOTE(ERROR(err))); \
+ _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \
+ RAWLOG(3, ": " __VA_ARGS__); \
+ RAWLOG(3, "\n"); \
+ return ERROR(err); \
+ }
+
+/*
+ * Unconditionally return the specified error.
+ *
+ * In debug modes, prints additional information.
+ */
+#define RETURN_ERROR(err, ...) \
+ do { \
+ RAWLOG(3, "%s:%d: ERROR!: unconditional check failed, returning %s", \
+ __FILE__, __LINE__, ERR_QUOTE(ERROR(err))); \
+ _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \
+ RAWLOG(3, ": " __VA_ARGS__); \
+ RAWLOG(3, "\n"); \
+ return ERROR(err); \
+ } while(0);
+
+/*
+ * If the provided expression evaluates to an error code, returns that error code.
+ *
+ * In debug modes, prints additional information.
+ */
+#define FORWARD_IF_ERROR(err, ...) \
+ do { \
+ size_t const err_code = (err); \
+ if (ERR_isError(err_code)) { \
+ RAWLOG(3, "%s:%d: ERROR!: forwarding error in %s: %s", \
+ __FILE__, __LINE__, ERR_QUOTE(err), ERR_getErrorName(err_code)); \
+ _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \
+ RAWLOG(3, ": " __VA_ARGS__); \
+ RAWLOG(3, "\n"); \
+ return err_code; \
+ } \
+ } while(0);
+
#endif /* ERROR_H_MODULE */
/* FSE_buildCTable_wksp() :
* Same as FSE_buildCTable(), but using an externally allocated scratch buffer (`workSpace`).
* `wkspSize` must be >= `FSE_BUILD_CTABLE_WORKSPACE_SIZE_U32(maxSymbolValue, tableLog)` of `unsigned`.
+ * See FSE_buildCTable_wksp() for breakdown of workspace usage.
*/
-#define FSE_BUILD_CTABLE_WORKSPACE_SIZE_U32(maxSymbolValue, tableLog) (maxSymbolValue + 2 + (1ull << (tableLog - 2)))
+#define FSE_BUILD_CTABLE_WORKSPACE_SIZE_U32(maxSymbolValue, tableLog) (((maxSymbolValue + 2) + (1ull << (tableLog)))/2 + sizeof(U64)/sizeof(U32) /* additional 8 bytes for potential table overwrite */)
#define FSE_BUILD_CTABLE_WORKSPACE_SIZE(maxSymbolValue, tableLog) (sizeof(unsigned) * FSE_BUILD_CTABLE_WORKSPACE_SIZE_U32(maxSymbolValue, tableLog))
size_t FSE_buildCTable_wksp(FSE_CTable* ct, const short* normalizedCounter, unsigned maxSymbolValue, unsigned tableLog, void* workSpace, size_t wkspSize);
}
#if DYNAMIC_BMI2
-TARGET_ATTRIBUTE("bmi2") static size_t FSE_decompress_wksp_body_bmi2(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize, unsigned maxLog, void* workSpace, size_t wkspSize)
+BMI2_TARGET_ATTRIBUTE static size_t FSE_decompress_wksp_body_bmi2(void* dst, size_t dstCapacity, const void* cSrc, size_t cSrcSize, unsigned maxLog, void* workSpace, size_t wkspSize)
{
return FSE_decompress_wksp_body(dst, dstCapacity, cSrc, cSrcSize, maxLog, workSpace, wkspSize, 1);
}
/* HUF_compress4X_wksp() :
* Same as HUF_compress2(), but uses externally allocated `workSpace`.
- * `workspace` must have minimum alignment of 4, and be at least as large as HUF_WORKSPACE_SIZE */
-#define HUF_WORKSPACE_SIZE ((6 << 10) + 256)
-#define HUF_WORKSPACE_SIZE_U32 (HUF_WORKSPACE_SIZE / sizeof(U32))
+ * `workspace` must be at least as large as HUF_WORKSPACE_SIZE */
+#define HUF_WORKSPACE_SIZE ((8 << 10) + 512 /* sorting scratch space */)
+#define HUF_WORKSPACE_SIZE_U64 (HUF_WORKSPACE_SIZE / sizeof(U64))
HUF_PUBLIC_API size_t HUF_compress4X_wksp (void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned tableLog,
/* *** Constants *** */
-#define HUF_TABLELOG_MAX 12 /* max runtime value of tableLog (due to static allocation); can be modified up to HUF_ABSOLUTEMAX_TABLELOG */
+#define HUF_TABLELOG_MAX 12 /* max runtime value of tableLog (due to static allocation); can be modified up to HUF_TABLELOG_ABSOLUTEMAX */
#define HUF_TABLELOG_DEFAULT 11 /* default tableLog value when none specified */
#define HUF_SYMBOLVALUE_MAX 255
-#define HUF_TABLELOG_ABSOLUTEMAX 15 /* absolute limit of HUF_MAX_TABLELOG. Beyond that value, code does not work */
+#define HUF_TABLELOG_ABSOLUTEMAX 12 /* absolute limit of HUF_MAX_TABLELOG. Beyond that value, code does not work */
#if (HUF_TABLELOG_MAX > HUF_TABLELOG_ABSOLUTEMAX)
# error "HUF_TABLELOG_MAX is too large !"
#endif
/* static allocation of HUF's Compression Table */
/* this is a private definition, just exposed for allocation and strict aliasing purpose. never EVER access its members directly */
-struct HUF_CElt_s {
- U16 val;
- BYTE nbBits;
-}; /* typedef'd to HUF_CElt */
-typedef struct HUF_CElt_s HUF_CElt; /* consider it an incomplete type */
-#define HUF_CTABLE_SIZE_U32(maxSymbolValue) ((maxSymbolValue)+1) /* Use tables of U32, for proper alignment */
-#define HUF_CTABLE_SIZE(maxSymbolValue) (HUF_CTABLE_SIZE_U32(maxSymbolValue) * sizeof(U32))
+typedef size_t HUF_CElt; /* consider it an incomplete type */
+#define HUF_CTABLE_SIZE_ST(maxSymbolValue) ((maxSymbolValue)+2) /* Use tables of size_t, for proper alignment */
+#define HUF_CTABLE_SIZE(maxSymbolValue) (HUF_CTABLE_SIZE_ST(maxSymbolValue) * sizeof(size_t))
#define HUF_CREATE_STATIC_CTABLE(name, maxSymbolValue) \
- HUF_CElt name[HUF_CTABLE_SIZE_U32(maxSymbolValue)] /* no final ; */
+ HUF_CElt name[HUF_CTABLE_SIZE_ST(maxSymbolValue)] /* no final ; */
/* static allocation of HUF's DTable */
typedef U32 HUF_DTable;
size_t HUF_writeCTable (void* dst, size_t maxDstSize, const HUF_CElt* CTable, unsigned maxSymbolValue, unsigned huffLog);
size_t HUF_writeCTable_wksp(void* dst, size_t maxDstSize, const HUF_CElt* CTable, unsigned maxSymbolValue, unsigned huffLog, void* workspace, size_t workspaceSize);
size_t HUF_compress4X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable);
+size_t HUF_compress4X_usingCTable_bmi2(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable, int bmi2);
size_t HUF_estimateCompressedSize(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue);
int HUF_validateCTable(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue);
* Same as HUF_compress4X_wksp(), but considers using hufTable if *repeat != HUF_repeat_none.
* If it uses hufTable it does not modify hufTable or repeat.
* If it doesn't, it sets *repeat = HUF_repeat_none, and it sets hufTable to the table used.
- * If preferRepeat then the old table will always be used if valid. */
+ * If preferRepeat then the old table will always be used if valid.
+ * If suspectUncompressible then some sampling checks will be run to potentially skip huffman coding */
size_t HUF_compress4X_repeat(void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned tableLog,
void* workSpace, size_t wkspSize, /*< `workSpace` must be aligned on 4-bytes boundaries, `wkspSize` must be >= HUF_WORKSPACE_SIZE */
- HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2);
+ HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2, unsigned suspectUncompressible);
/* HUF_buildCTable_wksp() :
* Same as HUF_buildCTable(), but using externally allocated scratch buffer.
* Loading a CTable saved with HUF_writeCTable() */
size_t HUF_readCTable (HUF_CElt* CTable, unsigned* maxSymbolValuePtr, const void* src, size_t srcSize, unsigned *hasZeroWeights);
-/* HUF_getNbBits() :
+/* HUF_getNbBitsFromCTable() :
* Read nbBits from CTable symbolTable, for symbol `symbolValue` presumed <= HUF_SYMBOLVALUE_MAX
- * Note 1 : is not inlined, as HUF_CElt definition is private
- * Note 2 : const void* used, so that it can provide a statically allocated table as argument (which uses type U32) */
-U32 HUF_getNbBits(const void* symbolTable, U32 symbolValue);
+ * Note 1 : is not inlined, as HUF_CElt definition is private */
+U32 HUF_getNbBitsFromCTable(const HUF_CElt* symbolTable, U32 symbolValue);
/*
* HUF_decompress() does the following:
/* ====================== */
size_t HUF_compress1X (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog);
-size_t HUF_compress1X_wksp (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog, void* workSpace, size_t wkspSize); /*< `workSpace` must be a table of at least HUF_WORKSPACE_SIZE_U32 unsigned */
+size_t HUF_compress1X_wksp (void* dst, size_t dstSize, const void* src, size_t srcSize, unsigned maxSymbolValue, unsigned tableLog, void* workSpace, size_t wkspSize); /*< `workSpace` must be a table of at least HUF_WORKSPACE_SIZE_U64 U64 */
size_t HUF_compress1X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable);
+size_t HUF_compress1X_usingCTable_bmi2(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable, int bmi2);
/* HUF_compress1X_repeat() :
* Same as HUF_compress1X_wksp(), but considers using hufTable if *repeat != HUF_repeat_none.
* If it uses hufTable it does not modify hufTable or repeat.
* If it doesn't, it sets *repeat = HUF_repeat_none, and it sets hufTable to the table used.
- * If preferRepeat then the old table will always be used if valid. */
+ * If preferRepeat then the old table will always be used if valid.
+ * If suspectUncompressible then some sampling checks will be run to potentially skip huffman coding */
size_t HUF_compress1X_repeat(void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned tableLog,
void* workSpace, size_t wkspSize, /*< `workSpace` must be aligned on 4-bytes boundaries, `wkspSize` must be >= HUF_WORKSPACE_SIZE */
- HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2);
+ HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2, unsigned suspectUncompressible);
size_t HUF_decompress1X1 (void* dst, size_t dstSize, const void* cSrc, size_t cSrcSize); /* single-symbol decoder */
#ifndef HUF_FORCE_DECOMPRESS_X1
#ifndef HUF_FORCE_DECOMPRESS_X2
size_t HUF_readDTableX1_wksp_bmi2(HUF_DTable* DTable, const void* src, size_t srcSize, void* workSpace, size_t wkspSize, int bmi2);
#endif
+#ifndef HUF_FORCE_DECOMPRESS_X1
+size_t HUF_readDTableX2_wksp_bmi2(HUF_DTable* DTable, const void* src, size_t srcSize, void* workSpace, size_t wkspSize, int bmi2);
+#endif
#endif /* HUF_STATIC_LINKING_ONLY */
* Basic Types
*****************************************************************/
typedef uint8_t BYTE;
+typedef uint8_t U8;
+typedef int8_t S8;
typedef uint16_t U16;
typedef int16_t S16;
typedef uint32_t U32;
--- /dev/null
+/*
+ * Copyright (c) Facebook, Inc.
+ * All rights reserved.
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+ */
+
+#ifndef ZSTD_PORTABILITY_MACROS_H
+#define ZSTD_PORTABILITY_MACROS_H
+
+/*
+ * This header file contains macro defintions to support portability.
+ * This header is shared between C and ASM code, so it MUST only
+ * contain macro definitions. It MUST not contain any C code.
+ *
+ * This header ONLY defines macros to detect platforms/feature support.
+ *
+ */
+
+
+/* compat. with non-clang compilers */
+#ifndef __has_attribute
+ #define __has_attribute(x) 0
+#endif
+
+/* compat. with non-clang compilers */
+#ifndef __has_builtin
+# define __has_builtin(x) 0
+#endif
+
+/* compat. with non-clang compilers */
+#ifndef __has_feature
+# define __has_feature(x) 0
+#endif
+
+/* detects whether we are being compiled under msan */
+
+/* detects whether we are being compiled under asan */
+
+/* detects whether we are being compiled under dfsan */
+
+/* Mark the internal assembly functions as hidden */
+#ifdef __ELF__
+# define ZSTD_HIDE_ASM_FUNCTION(func) .hidden func
+#else
+# define ZSTD_HIDE_ASM_FUNCTION(func)
+#endif
+
+/* Enable runtime BMI2 dispatch based on the CPU.
+ * Enabled for clang & gcc >=4.8 on x86 when BMI2 isn't enabled by default.
+ */
+#ifndef DYNAMIC_BMI2
+ #if ((defined(__clang__) && __has_attribute(__target__)) \
+ || (defined(__GNUC__) \
+ && (__GNUC__ >= 5 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8)))) \
+ && (defined(__x86_64__) || defined(_M_X64)) \
+ && !defined(__BMI2__)
+ # define DYNAMIC_BMI2 1
+ #else
+ # define DYNAMIC_BMI2 0
+ #endif
+#endif
+
+/*
+ * Only enable assembly for GNUC comptabile compilers,
+ * because other platforms may not support GAS assembly syntax.
+ *
+ * Only enable assembly for Linux / MacOS, other platforms may
+ * work, but they haven't been tested. This could likely be
+ * extended to BSD systems.
+ *
+ * Disable assembly when MSAN is enabled, because MSAN requires
+ * 100% of code to be instrumented to work.
+ */
+#define ZSTD_ASM_SUPPORTED 1
+
+/*
+ * Determines whether we should enable assembly for x86-64
+ * with BMI2.
+ *
+ * Enable if all of the following conditions hold:
+ * - ASM hasn't been explicitly disabled by defining ZSTD_DISABLE_ASM
+ * - Assembly is supported
+ * - We are compiling for x86-64 and either:
+ * - DYNAMIC_BMI2 is enabled
+ * - BMI2 is supported at compile time
+ */
+#define ZSTD_ENABLE_ASM_X86_64_BMI2 0
+
+#endif /* ZSTD_PORTABILITY_MACROS_H */
/*-*************************************
* Dependencies
***************************************/
-#include <linux/module.h>
#define ZSTD_DEPS_NEED_MALLOC
#include "zstd_deps.h" /* ZSTD_malloc, ZSTD_calloc, ZSTD_free, ZSTD_memset */
#include "error_private.h"
* tells if a return value is an error code
* symbol is required for external callers */
unsigned ZSTD_isError(size_t code) { return ERR_isError(code); }
-EXPORT_SYMBOL_GPL(ZSTD_isError);
/*! ZSTD_getErrorName() :
* provides error code string from function result (useful for debugging) */
const char* ZSTD_getErrorName(size_t code) { return ERR_getErrorName(code); }
-EXPORT_SYMBOL_GPL(ZSTD_getErrorName);
/*! ZSTD_getError() :
* convert a `size_t` function result into a proper ZSTD_errorCode enum */
ZSTD_ErrorCode ZSTD_getErrorCode(size_t code) { return ERR_getErrorCode(code); }
-EXPORT_SYMBOL_GPL(ZSTD_getErrorCode);
/*! ZSTD_getErrorString() :
* provides error code string from enum */
return customMem.customAlloc(customMem.opaque, size);
return ZSTD_malloc(size);
}
-EXPORT_SYMBOL_GPL(ZSTD_customMalloc);
void* ZSTD_customCalloc(size_t size, ZSTD_customMem customMem)
{
}
return ZSTD_calloc(1, size);
}
-EXPORT_SYMBOL_GPL(ZSTD_customCalloc);
void ZSTD_customFree(void* ptr, ZSTD_customMem customMem)
{
ZSTD_free(ptr);
}
}
-EXPORT_SYMBOL_GPL(ZSTD_customFree);
-
-MODULE_LICENSE("Dual BSD/GPL");
-MODULE_DESCRIPTION("Zstd Common");
* Dependencies
***************************************/
#include "compiler.h"
+#include "cpu.h"
#include "mem.h"
#include "debug.h" /* assert, DEBUGLOG, RAWLOG, g_debuglevel */
#include "error_private.h"
#undef MAX
#define MIN(a,b) ((a)<(b) ? (a) : (b))
#define MAX(a,b) ((a)>(b) ? (a) : (b))
-
-/*
- * Ignore: this is an internal helper.
- *
- * This is a helper function to help force C99-correctness during compilation.
- * Under strict compilation modes, variadic macro arguments can't be empty.
- * However, variadic function arguments can be. Using a function therefore lets
- * us statically check that at least one (string) argument was passed,
- * independent of the compilation flags.
- */
-static INLINE_KEYWORD UNUSED_ATTR
-void _force_has_format_string(const char *format, ...) {
- (void)format;
-}
-
-/*
- * Ignore: this is an internal helper.
- *
- * We want to force this function invocation to be syntactically correct, but
- * we don't want to force runtime evaluation of its arguments.
- */
-#define _FORCE_HAS_FORMAT_STRING(...) \
- if (0) { \
- _force_has_format_string(__VA_ARGS__); \
- }
-
-/*
- * Return the specified error if the condition evaluates to true.
- *
- * In debug modes, prints additional information.
- * In order to do that (particularly, printing the conditional that failed),
- * this can't just wrap RETURN_ERROR().
- */
-#define RETURN_ERROR_IF(cond, err, ...) \
- if (cond) { \
- RAWLOG(3, "%s:%d: ERROR!: check %s failed, returning %s", \
- __FILE__, __LINE__, ZSTD_QUOTE(cond), ZSTD_QUOTE(ERROR(err))); \
- _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \
- RAWLOG(3, ": " __VA_ARGS__); \
- RAWLOG(3, "\n"); \
- return ERROR(err); \
- }
-
-/*
- * Unconditionally return the specified error.
- *
- * In debug modes, prints additional information.
- */
-#define RETURN_ERROR(err, ...) \
- do { \
- RAWLOG(3, "%s:%d: ERROR!: unconditional check failed, returning %s", \
- __FILE__, __LINE__, ZSTD_QUOTE(ERROR(err))); \
- _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \
- RAWLOG(3, ": " __VA_ARGS__); \
- RAWLOG(3, "\n"); \
- return ERROR(err); \
- } while(0);
-
-/*
- * If the provided expression evaluates to an error code, returns that error code.
- *
- * In debug modes, prints additional information.
- */
-#define FORWARD_IF_ERROR(err, ...) \
- do { \
- size_t const err_code = (err); \
- if (ERR_isError(err_code)) { \
- RAWLOG(3, "%s:%d: ERROR!: forwarding error in %s: %s", \
- __FILE__, __LINE__, ZSTD_QUOTE(err), ERR_getErrorName(err_code)); \
- _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \
- RAWLOG(3, ": " __VA_ARGS__); \
- RAWLOG(3, "\n"); \
- return err_code; \
- } \
- } while(0);
+#define BOUNDED(min,val,max) (MAX(min,MIN(val,max)))
/*-*************************************
#define ZSTD_OPT_NUM (1<<12)
#define ZSTD_REP_NUM 3 /* number of repcodes */
-#define ZSTD_REP_MOVE (ZSTD_REP_NUM-1)
static UNUSED_ATTR const U32 repStartValue[ZSTD_REP_NUM] = { 1, 4, 8 };
#define KB *(1 <<10)
/* Each table cannot take more than #symbols * FSELog bits */
#define ZSTD_MAX_FSE_HEADERS_SIZE (((MaxML + 1) * MLFSELog + (MaxLL + 1) * LLFSELog + (MaxOff + 1) * OffFSELog + 7) / 8)
-static UNUSED_ATTR const U32 LL_bits[MaxLL+1] = {
+static UNUSED_ATTR const U8 LL_bits[MaxLL+1] = {
0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
1, 1, 1, 1, 2, 2, 3, 3,
#define LL_DEFAULTNORMLOG 6 /* for static allocation */
static UNUSED_ATTR const U32 LL_defaultNormLog = LL_DEFAULTNORMLOG;
-static UNUSED_ATTR const U32 ML_bits[MaxML+1] = {
+static UNUSED_ATTR const U8 ML_bits[MaxML+1] = {
0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
* Shared functions to include for inlining
*********************************************/
static void ZSTD_copy8(void* dst, const void* src) {
+#if defined(ZSTD_ARCH_ARM_NEON)
+ vst1_u8((uint8_t*)dst, vld1_u8((const uint8_t*)src));
+#else
ZSTD_memcpy(dst, src, 8);
+#endif
}
-
#define COPY8(d,s) { ZSTD_copy8(d,s); d+=8; s+=8; }
+
+/* Need to use memmove here since the literal buffer can now be located within
+ the dst buffer. In circumstances where the op "catches up" to where the
+ literal buffer is, there can be partial overlaps in this call on the final
+ copy if the literal is being shifted by less than 16 bytes. */
static void ZSTD_copy16(void* dst, const void* src) {
- ZSTD_memcpy(dst, src, 16);
+#if defined(ZSTD_ARCH_ARM_NEON)
+ vst1q_u8((uint8_t*)dst, vld1q_u8((const uint8_t*)src));
+#elif defined(ZSTD_ARCH_X86_SSE2)
+ _mm_storeu_si128((__m128i*)dst, _mm_loadu_si128((const __m128i*)src));
+#elif defined(__clang__)
+ ZSTD_memmove(dst, src, 16);
+#else
+ /* ZSTD_memmove is not inlined properly by gcc */
+ BYTE copy16_buf[16];
+ ZSTD_memcpy(copy16_buf, src, 16);
+ ZSTD_memcpy(dst, copy16_buf, 16);
+#endif
}
#define COPY16(d,s) { ZSTD_copy16(d,s); d+=16; s+=16; }
BYTE* op = (BYTE*)dst;
BYTE* const oend = op + length;
- assert(diff >= 8 || (ovtype == ZSTD_no_overlap && diff <= -WILDCOPY_VECLEN));
-
if (ovtype == ZSTD_overlap_src_before_dst && diff < WILDCOPY_VECLEN) {
/* Handle short offset copies. */
do {
* Private declarations
*********************************************/
typedef struct seqDef_s {
- U32 offset; /* Offset code of the sequence */
+ U32 offBase; /* offBase == Offset + ZSTD_REP_NUM, or repcode 1,2,3 */
U16 litLength;
- U16 matchLength;
+ U16 mlBase; /* mlBase == matchLength - MINMATCH */
} seqDef;
+/* Controls whether seqStore has a single "long" litLength or matchLength. See seqStore_t. */
+typedef enum {
+ ZSTD_llt_none = 0, /* no longLengthType */
+ ZSTD_llt_literalLength = 1, /* represents a long literal */
+ ZSTD_llt_matchLength = 2 /* represents a long match */
+} ZSTD_longLengthType_e;
+
typedef struct {
seqDef* sequencesStart;
seqDef* sequences; /* ptr to end of sequences */
size_t maxNbSeq;
size_t maxNbLit;
- /* longLengthPos and longLengthID to allow us to represent either a single litLength or matchLength
+ /* longLengthPos and longLengthType to allow us to represent either a single litLength or matchLength
* in the seqStore that has a value larger than U16 (if it exists). To do so, we increment
* the existing value of the litLength or matchLength by 0x10000.
*/
- U32 longLengthID; /* 0 == no longLength; 1 == Represent the long literal; 2 == Represent the long match; */
- U32 longLengthPos; /* Index of the sequence to apply long length modification to */
+ ZSTD_longLengthType_e longLengthType;
+ U32 longLengthPos; /* Index of the sequence to apply long length modification to */
} seqStore_t;
typedef struct {
/*
* Returns the ZSTD_sequenceLength for the given sequences. It handles the decoding of long sequences
- * indicated by longLengthPos and longLengthID, and adds MINMATCH back to matchLength.
+ * indicated by longLengthPos and longLengthType, and adds MINMATCH back to matchLength.
*/
MEM_STATIC ZSTD_sequenceLength ZSTD_getSequenceLength(seqStore_t const* seqStore, seqDef const* seq)
{
ZSTD_sequenceLength seqLen;
seqLen.litLength = seq->litLength;
- seqLen.matchLength = seq->matchLength + MINMATCH;
+ seqLen.matchLength = seq->mlBase + MINMATCH;
if (seqStore->longLengthPos == (U32)(seq - seqStore->sequencesStart)) {
- if (seqStore->longLengthID == 1) {
+ if (seqStore->longLengthType == ZSTD_llt_literalLength) {
seqLen.litLength += 0xFFFF;
}
- if (seqStore->longLengthID == 2) {
+ if (seqStore->longLengthType == ZSTD_llt_matchLength) {
seqLen.matchLength += 0xFFFF;
}
}
}
}
+/*
+ * Counts the number of trailing zeros of a `size_t`.
+ * Most compilers should support CTZ as a builtin. A backup
+ * implementation is provided if the builtin isn't supported, but
+ * it may not be terribly efficient.
+ */
+MEM_STATIC unsigned ZSTD_countTrailingZeros(size_t val)
+{
+ if (MEM_64bits()) {
+# if (__GNUC__ >= 4)
+ return __builtin_ctzll((U64)val);
+# else
+ static const int DeBruijnBytePos[64] = { 0, 1, 2, 7, 3, 13, 8, 19,
+ 4, 25, 14, 28, 9, 34, 20, 56,
+ 5, 17, 26, 54, 15, 41, 29, 43,
+ 10, 31, 38, 35, 21, 45, 49, 57,
+ 63, 6, 12, 18, 24, 27, 33, 55,
+ 16, 53, 40, 42, 30, 37, 44, 48,
+ 62, 11, 23, 32, 52, 39, 36, 47,
+ 61, 22, 51, 46, 60, 50, 59, 58 };
+ return DeBruijnBytePos[((U64)((val & -(long long)val) * 0x0218A392CDABBD3FULL)) >> 58];
+# endif
+ } else { /* 32 bits */
+# if (__GNUC__ >= 3)
+ return __builtin_ctz((U32)val);
+# else
+ static const int DeBruijnBytePos[32] = { 0, 1, 28, 2, 29, 14, 24, 3,
+ 30, 22, 20, 15, 25, 17, 4, 8,
+ 31, 27, 13, 23, 21, 19, 16, 7,
+ 26, 12, 18, 6, 11, 5, 10, 9 };
+ return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27];
+# endif
+ }
+}
+
/* ZSTD_invalidateRepCodes() :
* ensures next compression will not use repcodes from previous block.
size_t ZSTD_decodeSeqHeaders(ZSTD_DCtx* dctx, int* nbSeqPtr,
const void* src, size_t srcSize);
+/*
+ * @returns true iff the CPU supports dynamic BMI2 dispatch.
+ */
+MEM_STATIC int ZSTD_cpuSupportsBmi2(void)
+{
+ ZSTD_cpuid_t cpuid = ZSTD_cpuid();
+ return ZSTD_cpuid_bmi1(cpuid) && ZSTD_cpuid_bmi2(cpuid);
+}
#endif /* ZSTD_CCOMMON_H_MODULE */
--- /dev/null
+/*
+ * Copyright (c) Yann Collet, Facebook, Inc.
+ * All rights reserved.
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+ */
+
+#ifndef ZSTD_CLEVELS_H
+#define ZSTD_CLEVELS_H
+
+#define ZSTD_STATIC_LINKING_ONLY /* ZSTD_compressionParameters */
+#include <linux/zstd.h>
+
+/*-===== Pre-defined compression levels =====-*/
+
+#define ZSTD_MAX_CLEVEL 22
+
+__attribute__((__unused__))
+
+static const ZSTD_compressionParameters ZSTD_defaultCParameters[4][ZSTD_MAX_CLEVEL+1] = {
+{ /* "default" - for any srcSize > 256 KB */
+ /* W, C, H, S, L, TL, strat */
+ { 19, 12, 13, 1, 6, 1, ZSTD_fast }, /* base for negative levels */
+ { 19, 13, 14, 1, 7, 0, ZSTD_fast }, /* level 1 */
+ { 20, 15, 16, 1, 6, 0, ZSTD_fast }, /* level 2 */
+ { 21, 16, 17, 1, 5, 0, ZSTD_dfast }, /* level 3 */
+ { 21, 18, 18, 1, 5, 0, ZSTD_dfast }, /* level 4 */
+ { 21, 18, 19, 3, 5, 2, ZSTD_greedy }, /* level 5 */
+ { 21, 18, 19, 3, 5, 4, ZSTD_lazy }, /* level 6 */
+ { 21, 19, 20, 4, 5, 8, ZSTD_lazy }, /* level 7 */
+ { 21, 19, 20, 4, 5, 16, ZSTD_lazy2 }, /* level 8 */
+ { 22, 20, 21, 4, 5, 16, ZSTD_lazy2 }, /* level 9 */
+ { 22, 21, 22, 5, 5, 16, ZSTD_lazy2 }, /* level 10 */
+ { 22, 21, 22, 6, 5, 16, ZSTD_lazy2 }, /* level 11 */
+ { 22, 22, 23, 6, 5, 32, ZSTD_lazy2 }, /* level 12 */
+ { 22, 22, 22, 4, 5, 32, ZSTD_btlazy2 }, /* level 13 */
+ { 22, 22, 23, 5, 5, 32, ZSTD_btlazy2 }, /* level 14 */
+ { 22, 23, 23, 6, 5, 32, ZSTD_btlazy2 }, /* level 15 */
+ { 22, 22, 22, 5, 5, 48, ZSTD_btopt }, /* level 16 */
+ { 23, 23, 22, 5, 4, 64, ZSTD_btopt }, /* level 17 */
+ { 23, 23, 22, 6, 3, 64, ZSTD_btultra }, /* level 18 */
+ { 23, 24, 22, 7, 3,256, ZSTD_btultra2}, /* level 19 */
+ { 25, 25, 23, 7, 3,256, ZSTD_btultra2}, /* level 20 */
+ { 26, 26, 24, 7, 3,512, ZSTD_btultra2}, /* level 21 */
+ { 27, 27, 25, 9, 3,999, ZSTD_btultra2}, /* level 22 */
+},
+{ /* for srcSize <= 256 KB */
+ /* W, C, H, S, L, T, strat */
+ { 18, 12, 13, 1, 5, 1, ZSTD_fast }, /* base for negative levels */
+ { 18, 13, 14, 1, 6, 0, ZSTD_fast }, /* level 1 */
+ { 18, 14, 14, 1, 5, 0, ZSTD_dfast }, /* level 2 */
+ { 18, 16, 16, 1, 4, 0, ZSTD_dfast }, /* level 3 */
+ { 18, 16, 17, 3, 5, 2, ZSTD_greedy }, /* level 4.*/
+ { 18, 17, 18, 5, 5, 2, ZSTD_greedy }, /* level 5.*/
+ { 18, 18, 19, 3, 5, 4, ZSTD_lazy }, /* level 6.*/
+ { 18, 18, 19, 4, 4, 4, ZSTD_lazy }, /* level 7 */
+ { 18, 18, 19, 4, 4, 8, ZSTD_lazy2 }, /* level 8 */
+ { 18, 18, 19, 5, 4, 8, ZSTD_lazy2 }, /* level 9 */
+ { 18, 18, 19, 6, 4, 8, ZSTD_lazy2 }, /* level 10 */
+ { 18, 18, 19, 5, 4, 12, ZSTD_btlazy2 }, /* level 11.*/
+ { 18, 19, 19, 7, 4, 12, ZSTD_btlazy2 }, /* level 12.*/
+ { 18, 18, 19, 4, 4, 16, ZSTD_btopt }, /* level 13 */
+ { 18, 18, 19, 4, 3, 32, ZSTD_btopt }, /* level 14.*/
+ { 18, 18, 19, 6, 3,128, ZSTD_btopt }, /* level 15.*/
+ { 18, 19, 19, 6, 3,128, ZSTD_btultra }, /* level 16.*/
+ { 18, 19, 19, 8, 3,256, ZSTD_btultra }, /* level 17.*/
+ { 18, 19, 19, 6, 3,128, ZSTD_btultra2}, /* level 18.*/
+ { 18, 19, 19, 8, 3,256, ZSTD_btultra2}, /* level 19.*/
+ { 18, 19, 19, 10, 3,512, ZSTD_btultra2}, /* level 20.*/
+ { 18, 19, 19, 12, 3,512, ZSTD_btultra2}, /* level 21.*/
+ { 18, 19, 19, 13, 3,999, ZSTD_btultra2}, /* level 22.*/
+},
+{ /* for srcSize <= 128 KB */
+ /* W, C, H, S, L, T, strat */
+ { 17, 12, 12, 1, 5, 1, ZSTD_fast }, /* base for negative levels */
+ { 17, 12, 13, 1, 6, 0, ZSTD_fast }, /* level 1 */
+ { 17, 13, 15, 1, 5, 0, ZSTD_fast }, /* level 2 */
+ { 17, 15, 16, 2, 5, 0, ZSTD_dfast }, /* level 3 */
+ { 17, 17, 17, 2, 4, 0, ZSTD_dfast }, /* level 4 */
+ { 17, 16, 17, 3, 4, 2, ZSTD_greedy }, /* level 5 */
+ { 17, 16, 17, 3, 4, 4, ZSTD_lazy }, /* level 6 */
+ { 17, 16, 17, 3, 4, 8, ZSTD_lazy2 }, /* level 7 */
+ { 17, 16, 17, 4, 4, 8, ZSTD_lazy2 }, /* level 8 */
+ { 17, 16, 17, 5, 4, 8, ZSTD_lazy2 }, /* level 9 */
+ { 17, 16, 17, 6, 4, 8, ZSTD_lazy2 }, /* level 10 */
+ { 17, 17, 17, 5, 4, 8, ZSTD_btlazy2 }, /* level 11 */
+ { 17, 18, 17, 7, 4, 12, ZSTD_btlazy2 }, /* level 12 */
+ { 17, 18, 17, 3, 4, 12, ZSTD_btopt }, /* level 13.*/
+ { 17, 18, 17, 4, 3, 32, ZSTD_btopt }, /* level 14.*/
+ { 17, 18, 17, 6, 3,256, ZSTD_btopt }, /* level 15.*/
+ { 17, 18, 17, 6, 3,128, ZSTD_btultra }, /* level 16.*/
+ { 17, 18, 17, 8, 3,256, ZSTD_btultra }, /* level 17.*/
+ { 17, 18, 17, 10, 3,512, ZSTD_btultra }, /* level 18.*/
+ { 17, 18, 17, 5, 3,256, ZSTD_btultra2}, /* level 19.*/
+ { 17, 18, 17, 7, 3,512, ZSTD_btultra2}, /* level 20.*/
+ { 17, 18, 17, 9, 3,512, ZSTD_btultra2}, /* level 21.*/
+ { 17, 18, 17, 11, 3,999, ZSTD_btultra2}, /* level 22.*/
+},
+{ /* for srcSize <= 16 KB */
+ /* W, C, H, S, L, T, strat */
+ { 14, 12, 13, 1, 5, 1, ZSTD_fast }, /* base for negative levels */
+ { 14, 14, 15, 1, 5, 0, ZSTD_fast }, /* level 1 */
+ { 14, 14, 15, 1, 4, 0, ZSTD_fast }, /* level 2 */
+ { 14, 14, 15, 2, 4, 0, ZSTD_dfast }, /* level 3 */
+ { 14, 14, 14, 4, 4, 2, ZSTD_greedy }, /* level 4 */
+ { 14, 14, 14, 3, 4, 4, ZSTD_lazy }, /* level 5.*/
+ { 14, 14, 14, 4, 4, 8, ZSTD_lazy2 }, /* level 6 */
+ { 14, 14, 14, 6, 4, 8, ZSTD_lazy2 }, /* level 7 */
+ { 14, 14, 14, 8, 4, 8, ZSTD_lazy2 }, /* level 8.*/
+ { 14, 15, 14, 5, 4, 8, ZSTD_btlazy2 }, /* level 9.*/
+ { 14, 15, 14, 9, 4, 8, ZSTD_btlazy2 }, /* level 10.*/
+ { 14, 15, 14, 3, 4, 12, ZSTD_btopt }, /* level 11.*/
+ { 14, 15, 14, 4, 3, 24, ZSTD_btopt }, /* level 12.*/
+ { 14, 15, 14, 5, 3, 32, ZSTD_btultra }, /* level 13.*/
+ { 14, 15, 15, 6, 3, 64, ZSTD_btultra }, /* level 14.*/
+ { 14, 15, 15, 7, 3,256, ZSTD_btultra }, /* level 15.*/
+ { 14, 15, 15, 5, 3, 48, ZSTD_btultra2}, /* level 16.*/
+ { 14, 15, 15, 6, 3,128, ZSTD_btultra2}, /* level 17.*/
+ { 14, 15, 15, 7, 3,256, ZSTD_btultra2}, /* level 18.*/
+ { 14, 15, 15, 8, 3,256, ZSTD_btultra2}, /* level 19.*/
+ { 14, 15, 15, 8, 3,512, ZSTD_btultra2}, /* level 20.*/
+ { 14, 15, 15, 9, 3,512, ZSTD_btultra2}, /* level 21.*/
+ { 14, 15, 15, 10, 3,999, ZSTD_btultra2}, /* level 22.*/
+},
+};
+
+
+
+#endif /* ZSTD_CLEVELS_H */
void* const FSCT = ((U32*)ptr) + 1 /* header */ + (tableLog ? tableSize>>1 : 1) ;
FSE_symbolCompressionTransform* const symbolTT = (FSE_symbolCompressionTransform*) (FSCT);
U32 const step = FSE_TABLESTEP(tableSize);
+ U32 const maxSV1 = maxSymbolValue+1;
- U32* cumul = (U32*)workSpace;
- FSE_FUNCTION_TYPE* tableSymbol = (FSE_FUNCTION_TYPE*)(cumul + (maxSymbolValue + 2));
+ U16* cumul = (U16*)workSpace; /* size = maxSV1 */
+ FSE_FUNCTION_TYPE* const tableSymbol = (FSE_FUNCTION_TYPE*)(cumul + (maxSV1+1)); /* size = tableSize */
U32 highThreshold = tableSize-1;
- if ((size_t)workSpace & 3) return ERROR(GENERIC); /* Must be 4 byte aligned */
+ assert(((size_t)workSpace & 1) == 0); /* Must be 2 bytes-aligned */
if (FSE_BUILD_CTABLE_WORKSPACE_SIZE(maxSymbolValue, tableLog) > wkspSize) return ERROR(tableLog_tooLarge);
/* CTable header */
tableU16[-2] = (U16) tableLog;
/* symbol start positions */
{ U32 u;
cumul[0] = 0;
- for (u=1; u <= maxSymbolValue+1; u++) {
+ for (u=1; u <= maxSV1; u++) {
if (normalizedCounter[u-1]==-1) { /* Low proba symbol */
cumul[u] = cumul[u-1] + 1;
tableSymbol[highThreshold--] = (FSE_FUNCTION_TYPE)(u-1);
} else {
- cumul[u] = cumul[u-1] + normalizedCounter[u-1];
+ assert(normalizedCounter[u-1] >= 0);
+ cumul[u] = cumul[u-1] + (U16)normalizedCounter[u-1];
+ assert(cumul[u] >= cumul[u-1]); /* no overflow */
} }
- cumul[maxSymbolValue+1] = tableSize+1;
+ cumul[maxSV1] = (U16)(tableSize+1);
}
/* Spread symbols */
- { U32 position = 0;
+ if (highThreshold == tableSize - 1) {
+ /* Case for no low prob count symbols. Lay down 8 bytes at a time
+ * to reduce branch misses since we are operating on a small block
+ */
+ BYTE* const spread = tableSymbol + tableSize; /* size = tableSize + 8 (may write beyond tableSize) */
+ { U64 const add = 0x0101010101010101ull;
+ size_t pos = 0;
+ U64 sv = 0;
+ U32 s;
+ for (s=0; s<maxSV1; ++s, sv += add) {
+ int i;
+ int const n = normalizedCounter[s];
+ MEM_write64(spread + pos, sv);
+ for (i = 8; i < n; i += 8) {
+ MEM_write64(spread + pos + i, sv);
+ }
+ assert(n>=0);
+ pos += (size_t)n;
+ }
+ }
+ /* Spread symbols across the table. Lack of lowprob symbols means that
+ * we don't need variable sized inner loop, so we can unroll the loop and
+ * reduce branch misses.
+ */
+ { size_t position = 0;
+ size_t s;
+ size_t const unroll = 2; /* Experimentally determined optimal unroll */
+ assert(tableSize % unroll == 0); /* FSE_MIN_TABLELOG is 5 */
+ for (s = 0; s < (size_t)tableSize; s += unroll) {
+ size_t u;
+ for (u = 0; u < unroll; ++u) {
+ size_t const uPosition = (position + (u * step)) & tableMask;
+ tableSymbol[uPosition] = spread[s + u];
+ }
+ position = (position + (unroll * step)) & tableMask;
+ }
+ assert(position == 0); /* Must have initialized all positions */
+ }
+ } else {
+ U32 position = 0;
U32 symbol;
- for (symbol=0; symbol<=maxSymbolValue; symbol++) {
+ for (symbol=0; symbol<maxSV1; symbol++) {
int nbOccurrences;
int const freq = normalizedCounter[symbol];
for (nbOccurrences=0; nbOccurrences<freq; nbOccurrences++) {
while (position > highThreshold)
position = (position + step) & tableMask; /* Low proba area */
} }
-
assert(position==0); /* Must have initialized all positions */
}
case -1:
case 1:
symbolTT[s].deltaNbBits = (tableLog << 16) - (1<<tableLog);
- symbolTT[s].deltaFindState = total - 1;
+ assert(total <= INT_MAX);
+ symbolTT[s].deltaFindState = (int)(total - 1);
total ++;
break;
default :
- {
- U32 const maxBitsOut = tableLog - BIT_highbit32 (normalizedCounter[s]-1);
- U32 const minStatePlus = normalizedCounter[s] << maxBitsOut;
+ assert(normalizedCounter[s] > 1);
+ { U32 const maxBitsOut = tableLog - BIT_highbit32 ((U32)normalizedCounter[s]-1);
+ U32 const minStatePlus = (U32)normalizedCounter[s] << maxBitsOut;
symbolTT[s].deltaNbBits = (maxBitsOut << 16) - minStatePlus;
- symbolTT[s].deltaFindState = total - normalizedCounter[s];
- total += normalizedCounter[s];
+ symbolTT[s].deltaFindState = (int)(total - (unsigned)normalizedCounter[s]);
+ total += (unsigned)normalizedCounter[s];
} } } }
#if 0 /* debug : symbol costs */
symbol, normalizedCounter[symbol],
FSE_getMaxNbBits(symbolTT, symbol),
(double)FSE_bitCost(symbolTT, tableLog, symbol, 8) / 256);
- }
- }
+ } }
#endif
return 0;
-
#ifndef FSE_COMMONDEFS_ONLY
-
/*-**************************************************************
* FSE NCount encoding
****************************************************************/
size_t FSE_NCountWriteBound(unsigned maxSymbolValue, unsigned tableLog)
{
- size_t const maxHeaderSize = (((maxSymbolValue+1) * tableLog) >> 3) + 3;
+ size_t const maxHeaderSize = (((maxSymbolValue+1) * tableLog
+ + 4 /* bitCount initialized at 4 */
+ + 2 /* first two symbols may use one additional bit each */) / 8)
+ + 1 /* round up to whole nb bytes */
+ + 2 /* additional two bytes for bitstream flush */;
return maxSymbolValue ? maxHeaderSize : FSE_NCOUNTBOUND; /* maxSymbolValue==0 ? use default */
}
/* *******************************************************
* HUF : Huffman block compression
*********************************************************/
+#define HUF_WORKSPACE_MAX_ALIGNMENT 8
+
+static void* HUF_alignUpWorkspace(void* workspace, size_t* workspaceSizePtr, size_t align)
+{
+ size_t const mask = align - 1;
+ size_t const rem = (size_t)workspace & mask;
+ size_t const add = (align - rem) & mask;
+ BYTE* const aligned = (BYTE*)workspace + add;
+ assert((align & (align - 1)) == 0); /* pow 2 */
+ assert(align <= HUF_WORKSPACE_MAX_ALIGNMENT);
+ if (*workspaceSizePtr >= add) {
+ assert(add < align);
+ assert(((size_t)aligned & mask) == 0);
+ *workspaceSizePtr -= add;
+ return aligned;
+ } else {
+ *workspaceSizePtr = 0;
+ return NULL;
+ }
+}
+
+
/* HUF_compressWeights() :
* Same as FSE_compress(), but dedicated to huff0's weights compression.
* The use case needs much less stack memory.
unsigned maxSymbolValue = HUF_TABLELOG_MAX;
U32 tableLog = MAX_FSE_TABLELOG_FOR_HUFF_HEADER;
- HUF_CompressWeightsWksp* wksp = (HUF_CompressWeightsWksp*)workspace;
+ HUF_CompressWeightsWksp* wksp = (HUF_CompressWeightsWksp*)HUF_alignUpWorkspace(workspace, &workspaceSize, ZSTD_ALIGNOF(U32));
if (workspaceSize < sizeof(HUF_CompressWeightsWksp)) return ERROR(GENERIC);
return (size_t)(op-ostart);
}
+static size_t HUF_getNbBits(HUF_CElt elt)
+{
+ return elt & 0xFF;
+}
+
+static size_t HUF_getNbBitsFast(HUF_CElt elt)
+{
+ return elt;
+}
+
+static size_t HUF_getValue(HUF_CElt elt)
+{
+ return elt & ~0xFF;
+}
+
+static size_t HUF_getValueFast(HUF_CElt elt)
+{
+ return elt;
+}
+
+static void HUF_setNbBits(HUF_CElt* elt, size_t nbBits)
+{
+ assert(nbBits <= HUF_TABLELOG_ABSOLUTEMAX);
+ *elt = nbBits;
+}
+
+static void HUF_setValue(HUF_CElt* elt, size_t value)
+{
+ size_t const nbBits = HUF_getNbBits(*elt);
+ if (nbBits > 0) {
+ assert((value >> nbBits) == 0);
+ *elt |= value << (sizeof(HUF_CElt) * 8 - nbBits);
+ }
+}
typedef struct {
HUF_CompressWeightsWksp wksp;
const HUF_CElt* CTable, unsigned maxSymbolValue, unsigned huffLog,
void* workspace, size_t workspaceSize)
{
+ HUF_CElt const* const ct = CTable + 1;
BYTE* op = (BYTE*)dst;
U32 n;
- HUF_WriteCTableWksp* wksp = (HUF_WriteCTableWksp*)workspace;
+ HUF_WriteCTableWksp* wksp = (HUF_WriteCTableWksp*)HUF_alignUpWorkspace(workspace, &workspaceSize, ZSTD_ALIGNOF(U32));
/* check conditions */
if (workspaceSize < sizeof(HUF_WriteCTableWksp)) return ERROR(GENERIC);
for (n=1; n<huffLog+1; n++)
wksp->bitsToWeight[n] = (BYTE)(huffLog + 1 - n);
for (n=0; n<maxSymbolValue; n++)
- wksp->huffWeight[n] = wksp->bitsToWeight[CTable[n].nbBits];
+ wksp->huffWeight[n] = wksp->bitsToWeight[HUF_getNbBits(ct[n])];
/* attempt weights compression by FSE */
+ if (maxDstSize < 1) return ERROR(dstSize_tooSmall);
{ CHECK_V_F(hSize, HUF_compressWeights(op+1, maxDstSize-1, wksp->huffWeight, maxSymbolValue, &wksp->wksp, sizeof(wksp->wksp)) );
if ((hSize>1) & (hSize < maxSymbolValue/2)) { /* FSE compressed */
op[0] = (BYTE)hSize;
U32 rankVal[HUF_TABLELOG_ABSOLUTEMAX + 1]; /* large enough for values from 0 to 16 */
U32 tableLog = 0;
U32 nbSymbols = 0;
+ HUF_CElt* const ct = CTable + 1;
/* get symbol weights */
CHECK_V_F(readSize, HUF_readStats(huffWeight, HUF_SYMBOLVALUE_MAX+1, rankVal, &nbSymbols, &tableLog, src, srcSize));
if (tableLog > HUF_TABLELOG_MAX) return ERROR(tableLog_tooLarge);
if (nbSymbols > *maxSymbolValuePtr+1) return ERROR(maxSymbolValue_tooSmall);
+ CTable[0] = tableLog;
+
/* Prepare base value per rank */
{ U32 n, nextRankStart = 0;
for (n=1; n<=tableLog; n++) {
/* fill nbBits */
{ U32 n; for (n=0; n<nbSymbols; n++) {
const U32 w = huffWeight[n];
- CTable[n].nbBits = (BYTE)(tableLog + 1 - w) & -(w != 0);
+ HUF_setNbBits(ct + n, (BYTE)(tableLog + 1 - w) & -(w != 0));
} }
/* fill val */
{ U16 nbPerRank[HUF_TABLELOG_MAX+2] = {0}; /* support w=0=>n=tableLog+1 */
U16 valPerRank[HUF_TABLELOG_MAX+2] = {0};
- { U32 n; for (n=0; n<nbSymbols; n++) nbPerRank[CTable[n].nbBits]++; }
+ { U32 n; for (n=0; n<nbSymbols; n++) nbPerRank[HUF_getNbBits(ct[n])]++; }
/* determine stating value per rank */
valPerRank[tableLog+1] = 0; /* for w==0 */
{ U16 min = 0;
min >>= 1;
} }
/* assign value within rank, symbol order */
- { U32 n; for (n=0; n<nbSymbols; n++) CTable[n].val = valPerRank[CTable[n].nbBits]++; }
+ { U32 n; for (n=0; n<nbSymbols; n++) HUF_setValue(ct + n, valPerRank[HUF_getNbBits(ct[n])]++); }
}
*maxSymbolValuePtr = nbSymbols - 1;
return readSize;
}
-U32 HUF_getNbBits(const void* symbolTable, U32 symbolValue)
+U32 HUF_getNbBitsFromCTable(HUF_CElt const* CTable, U32 symbolValue)
{
- const HUF_CElt* table = (const HUF_CElt*)symbolTable;
+ const HUF_CElt* ct = CTable + 1;
assert(symbolValue <= HUF_SYMBOLVALUE_MAX);
- return table[symbolValue].nbBits;
+ return (U32)HUF_getNbBits(ct[symbolValue]);
}
}
typedef struct {
- U32 base;
- U32 curr;
+ U16 base;
+ U16 curr;
} rankPos;
typedef nodeElt huffNodeTable[HUF_CTABLE_WORKSPACE_SIZE_U32];
-#define RANK_POSITION_TABLE_SIZE 32
+/* Number of buckets available for HUF_sort() */
+#define RANK_POSITION_TABLE_SIZE 192
typedef struct {
huffNodeTable huffNodeTbl;
rankPos rankPosition[RANK_POSITION_TABLE_SIZE];
} HUF_buildCTable_wksp_tables;
+/* RANK_POSITION_DISTINCT_COUNT_CUTOFF == Cutoff point in HUF_sort() buckets for which we use log2 bucketing.
+ * Strategy is to use as many buckets as possible for representing distinct
+ * counts while using the remainder to represent all "large" counts.
+ *
+ * To satisfy this requirement for 192 buckets, we can do the following:
+ * Let buckets 0-166 represent distinct counts of [0, 166]
+ * Let buckets 166 to 192 represent all remaining counts up to RANK_POSITION_MAX_COUNT_LOG using log2 bucketing.
+ */
+#define RANK_POSITION_MAX_COUNT_LOG 32
+#define RANK_POSITION_LOG_BUCKETS_BEGIN (RANK_POSITION_TABLE_SIZE - 1) - RANK_POSITION_MAX_COUNT_LOG - 1 /* == 158 */
+#define RANK_POSITION_DISTINCT_COUNT_CUTOFF RANK_POSITION_LOG_BUCKETS_BEGIN + BIT_highbit32(RANK_POSITION_LOG_BUCKETS_BEGIN) /* == 166 */
+
+/* Return the appropriate bucket index for a given count. See definition of
+ * RANK_POSITION_DISTINCT_COUNT_CUTOFF for explanation of bucketing strategy.
+ */
+static U32 HUF_getIndex(U32 const count) {
+ return (count < RANK_POSITION_DISTINCT_COUNT_CUTOFF)
+ ? count
+ : BIT_highbit32(count) + RANK_POSITION_LOG_BUCKETS_BEGIN;
+}
+
+/* Helper swap function for HUF_quickSortPartition() */
+static void HUF_swapNodes(nodeElt* a, nodeElt* b) {
+ nodeElt tmp = *a;
+ *a = *b;
+ *b = tmp;
+}
+
+/* Returns 0 if the huffNode array is not sorted by descending count */
+MEM_STATIC int HUF_isSorted(nodeElt huffNode[], U32 const maxSymbolValue1) {
+ U32 i;
+ for (i = 1; i < maxSymbolValue1; ++i) {
+ if (huffNode[i].count > huffNode[i-1].count) {
+ return 0;
+ }
+ }
+ return 1;
+}
+
+/* Insertion sort by descending order */
+HINT_INLINE void HUF_insertionSort(nodeElt huffNode[], int const low, int const high) {
+ int i;
+ int const size = high-low+1;
+ huffNode += low;
+ for (i = 1; i < size; ++i) {
+ nodeElt const key = huffNode[i];
+ int j = i - 1;
+ while (j >= 0 && huffNode[j].count < key.count) {
+ huffNode[j + 1] = huffNode[j];
+ j--;
+ }
+ huffNode[j + 1] = key;
+ }
+}
+
+/* Pivot helper function for quicksort. */
+static int HUF_quickSortPartition(nodeElt arr[], int const low, int const high) {
+ /* Simply select rightmost element as pivot. "Better" selectors like
+ * median-of-three don't experimentally appear to have any benefit.
+ */
+ U32 const pivot = arr[high].count;
+ int i = low - 1;
+ int j = low;
+ for ( ; j < high; j++) {
+ if (arr[j].count > pivot) {
+ i++;
+ HUF_swapNodes(&arr[i], &arr[j]);
+ }
+ }
+ HUF_swapNodes(&arr[i + 1], &arr[high]);
+ return i + 1;
+}
+
+/* Classic quicksort by descending with partially iterative calls
+ * to reduce worst case callstack size.
+ */
+static void HUF_simpleQuickSort(nodeElt arr[], int low, int high) {
+ int const kInsertionSortThreshold = 8;
+ if (high - low < kInsertionSortThreshold) {
+ HUF_insertionSort(arr, low, high);
+ return;
+ }
+ while (low < high) {
+ int const idx = HUF_quickSortPartition(arr, low, high);
+ if (idx - low < high - idx) {
+ HUF_simpleQuickSort(arr, low, idx - 1);
+ low = idx + 1;
+ } else {
+ HUF_simpleQuickSort(arr, idx + 1, high);
+ high = idx - 1;
+ }
+ }
+}
+
/*
* HUF_sort():
* Sorts the symbols [0, maxSymbolValue] by count[symbol] in decreasing order.
+ * This is a typical bucket sorting strategy that uses either quicksort or insertion sort to sort each bucket.
*
* @param[out] huffNode Sorted symbols by decreasing count. Only members `.count` and `.byte` are filled.
* Must have (maxSymbolValue + 1) entries.
* @param[in] maxSymbolValue Maximum symbol value.
* @param rankPosition This is a scratch workspace. Must have RANK_POSITION_TABLE_SIZE entries.
*/
-static void HUF_sort(nodeElt* huffNode, const unsigned* count, U32 maxSymbolValue, rankPos* rankPosition)
-{
- int n;
- int const maxSymbolValue1 = (int)maxSymbolValue + 1;
+static void HUF_sort(nodeElt huffNode[], const unsigned count[], U32 const maxSymbolValue, rankPos rankPosition[]) {
+ U32 n;
+ U32 const maxSymbolValue1 = maxSymbolValue+1;
/* Compute base and set curr to base.
- * For symbol s let lowerRank = BIT_highbit32(count[n]+1) and rank = lowerRank + 1.
- * Then 2^lowerRank <= count[n]+1 <= 2^rank.
+ * For symbol s let lowerRank = HUF_getIndex(count[n]) and rank = lowerRank + 1.
+ * See HUF_getIndex to see bucketing strategy.
* We attribute each symbol to lowerRank's base value, because we want to know where
* each rank begins in the output, so for rank R we want to count ranks R+1 and above.
*/
ZSTD_memset(rankPosition, 0, sizeof(*rankPosition) * RANK_POSITION_TABLE_SIZE);
for (n = 0; n < maxSymbolValue1; ++n) {
- U32 lowerRank = BIT_highbit32(count[n] + 1);
+ U32 lowerRank = HUF_getIndex(count[n]);
+ assert(lowerRank < RANK_POSITION_TABLE_SIZE - 1);
rankPosition[lowerRank].base++;
}
+
assert(rankPosition[RANK_POSITION_TABLE_SIZE - 1].base == 0);
+ /* Set up the rankPosition table */
for (n = RANK_POSITION_TABLE_SIZE - 1; n > 0; --n) {
rankPosition[n-1].base += rankPosition[n].base;
rankPosition[n-1].curr = rankPosition[n-1].base;
}
- /* Sort */
+
+ /* Insert each symbol into their appropriate bucket, setting up rankPosition table. */
for (n = 0; n < maxSymbolValue1; ++n) {
U32 const c = count[n];
- U32 const r = BIT_highbit32(c+1) + 1;
- U32 pos = rankPosition[r].curr++;
- /* Insert into the correct position in the rank.
- * We have at most 256 symbols, so this insertion should be fine.
- */
- while ((pos > rankPosition[r].base) && (c > huffNode[pos-1].count)) {
- huffNode[pos] = huffNode[pos-1];
- pos--;
- }
+ U32 const r = HUF_getIndex(c) + 1;
+ U32 const pos = rankPosition[r].curr++;
+ assert(pos < maxSymbolValue1);
huffNode[pos].count = c;
huffNode[pos].byte = (BYTE)n;
}
-}
+ /* Sort each bucket. */
+ for (n = RANK_POSITION_DISTINCT_COUNT_CUTOFF; n < RANK_POSITION_TABLE_SIZE - 1; ++n) {
+ U32 const bucketSize = rankPosition[n].curr-rankPosition[n].base;
+ U32 const bucketStartIdx = rankPosition[n].base;
+ if (bucketSize > 1) {
+ assert(bucketStartIdx < maxSymbolValue1);
+ HUF_simpleQuickSort(huffNode + bucketStartIdx, 0, bucketSize-1);
+ }
+ }
+
+ assert(HUF_isSorted(huffNode, maxSymbolValue1));
+}
/* HUF_buildCTable_wksp() :
* Same as HUF_buildCTable(), but using externally allocated scratch buffer.
*/
static void HUF_buildCTableFromTree(HUF_CElt* CTable, nodeElt const* huffNode, int nonNullRank, U32 maxSymbolValue, U32 maxNbBits)
{
+ HUF_CElt* const ct = CTable + 1;
/* fill result into ctable (val, nbBits) */
int n;
U16 nbPerRank[HUF_TABLELOG_MAX+1] = {0};
min >>= 1;
} }
for (n=0; n<alphabetSize; n++)
- CTable[huffNode[n].byte].nbBits = huffNode[n].nbBits; /* push nbBits per symbol, symbol order */
+ HUF_setNbBits(ct + huffNode[n].byte, huffNode[n].nbBits); /* push nbBits per symbol, symbol order */
for (n=0; n<alphabetSize; n++)
- CTable[n].val = valPerRank[CTable[n].nbBits]++; /* assign value within rank, symbol order */
+ HUF_setValue(ct + n, valPerRank[HUF_getNbBits(ct[n])]++); /* assign value within rank, symbol order */
+ CTable[0] = maxNbBits;
}
-size_t HUF_buildCTable_wksp (HUF_CElt* tree, const unsigned* count, U32 maxSymbolValue, U32 maxNbBits, void* workSpace, size_t wkspSize)
+size_t HUF_buildCTable_wksp (HUF_CElt* CTable, const unsigned* count, U32 maxSymbolValue, U32 maxNbBits, void* workSpace, size_t wkspSize)
{
- HUF_buildCTable_wksp_tables* const wksp_tables = (HUF_buildCTable_wksp_tables*)workSpace;
+ HUF_buildCTable_wksp_tables* const wksp_tables = (HUF_buildCTable_wksp_tables*)HUF_alignUpWorkspace(workSpace, &wkspSize, ZSTD_ALIGNOF(U32));
nodeElt* const huffNode0 = wksp_tables->huffNodeTbl;
nodeElt* const huffNode = huffNode0+1;
int nonNullRank;
/* safety checks */
- if (((size_t)workSpace & 3) != 0) return ERROR(GENERIC); /* must be aligned on 4-bytes boundaries */
if (wkspSize < sizeof(HUF_buildCTable_wksp_tables))
return ERROR(workSpace_tooSmall);
if (maxNbBits == 0) maxNbBits = HUF_TABLELOG_DEFAULT;
maxNbBits = HUF_setMaxHeight(huffNode, (U32)nonNullRank, maxNbBits);
if (maxNbBits > HUF_TABLELOG_MAX) return ERROR(GENERIC); /* check fit into table */
- HUF_buildCTableFromTree(tree, huffNode, nonNullRank, maxSymbolValue, maxNbBits);
+ HUF_buildCTableFromTree(CTable, huffNode, nonNullRank, maxSymbolValue, maxNbBits);
return maxNbBits;
}
size_t HUF_estimateCompressedSize(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue)
{
+ HUF_CElt const* ct = CTable + 1;
size_t nbBits = 0;
int s;
for (s = 0; s <= (int)maxSymbolValue; ++s) {
- nbBits += CTable[s].nbBits * count[s];
+ nbBits += HUF_getNbBits(ct[s]) * count[s];
}
return nbBits >> 3;
}
int HUF_validateCTable(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue) {
+ HUF_CElt const* ct = CTable + 1;
int bad = 0;
int s;
for (s = 0; s <= (int)maxSymbolValue; ++s) {
- bad |= (count[s] != 0) & (CTable[s].nbBits == 0);
+ bad |= (count[s] != 0) & (HUF_getNbBits(ct[s]) == 0);
}
return !bad;
}
size_t HUF_compressBound(size_t size) { return HUF_COMPRESSBOUND(size); }
+/* HUF_CStream_t:
+ * Huffman uses its own BIT_CStream_t implementation.
+ * There are three major differences from BIT_CStream_t:
+ * 1. HUF_addBits() takes a HUF_CElt (size_t) which is
+ * the pair (nbBits, value) in the format:
+ * format:
+ * - Bits [0, 4) = nbBits
+ * - Bits [4, 64 - nbBits) = 0
+ * - Bits [64 - nbBits, 64) = value
+ * 2. The bitContainer is built from the upper bits and
+ * right shifted. E.g. to add a new value of N bits
+ * you right shift the bitContainer by N, then or in
+ * the new value into the N upper bits.
+ * 3. The bitstream has two bit containers. You can add
+ * bits to the second container and merge them into
+ * the first container.
+ */
+
+#define HUF_BITS_IN_CONTAINER (sizeof(size_t) * 8)
+
+typedef struct {
+ size_t bitContainer[2];
+ size_t bitPos[2];
+
+ BYTE* startPtr;
+ BYTE* ptr;
+ BYTE* endPtr;
+} HUF_CStream_t;
+
+/*! HUF_initCStream():
+ * Initializes the bitstream.
+ * @returns 0 or an error code.
+ */
+static size_t HUF_initCStream(HUF_CStream_t* bitC,
+ void* startPtr, size_t dstCapacity)
+{
+ ZSTD_memset(bitC, 0, sizeof(*bitC));
+ bitC->startPtr = (BYTE*)startPtr;
+ bitC->ptr = bitC->startPtr;
+ bitC->endPtr = bitC->startPtr + dstCapacity - sizeof(bitC->bitContainer[0]);
+ if (dstCapacity <= sizeof(bitC->bitContainer[0])) return ERROR(dstSize_tooSmall);
+ return 0;
+}
+
+/*! HUF_addBits():
+ * Adds the symbol stored in HUF_CElt elt to the bitstream.
+ *
+ * @param elt The element we're adding. This is a (nbBits, value) pair.
+ * See the HUF_CStream_t docs for the format.
+ * @param idx Insert into the bitstream at this idx.
+ * @param kFast This is a template parameter. If the bitstream is guaranteed
+ * to have at least 4 unused bits after this call it may be 1,
+ * otherwise it must be 0. HUF_addBits() is faster when fast is set.
+ */
+FORCE_INLINE_TEMPLATE void HUF_addBits(HUF_CStream_t* bitC, HUF_CElt elt, int idx, int kFast)
+{
+ assert(idx <= 1);
+ assert(HUF_getNbBits(elt) <= HUF_TABLELOG_ABSOLUTEMAX);
+ /* This is efficient on x86-64 with BMI2 because shrx
+ * only reads the low 6 bits of the register. The compiler
+ * knows this and elides the mask. When fast is set,
+ * every operation can use the same value loaded from elt.
+ */
+ bitC->bitContainer[idx] >>= HUF_getNbBits(elt);
+ bitC->bitContainer[idx] |= kFast ? HUF_getValueFast(elt) : HUF_getValue(elt);
+ /* We only read the low 8 bits of bitC->bitPos[idx] so it
+ * doesn't matter that the high bits have noise from the value.
+ */
+ bitC->bitPos[idx] += HUF_getNbBitsFast(elt);
+ assert((bitC->bitPos[idx] & 0xFF) <= HUF_BITS_IN_CONTAINER);
+ /* The last 4-bits of elt are dirty if fast is set,
+ * so we must not be overwriting bits that have already been
+ * inserted into the bit container.
+ */
+#if DEBUGLEVEL >= 1
+ {
+ size_t const nbBits = HUF_getNbBits(elt);
+ size_t const dirtyBits = nbBits == 0 ? 0 : BIT_highbit32((U32)nbBits) + 1;
+ (void)dirtyBits;
+ /* Middle bits are 0. */
+ assert(((elt >> dirtyBits) << (dirtyBits + nbBits)) == 0);
+ /* We didn't overwrite any bits in the bit container. */
+ assert(!kFast || (bitC->bitPos[idx] & 0xFF) <= HUF_BITS_IN_CONTAINER);
+ (void)dirtyBits;
+ }
+#endif
+}
+
+FORCE_INLINE_TEMPLATE void HUF_zeroIndex1(HUF_CStream_t* bitC)
+{
+ bitC->bitContainer[1] = 0;
+ bitC->bitPos[1] = 0;
+}
+
+/*! HUF_mergeIndex1() :
+ * Merges the bit container @ index 1 into the bit container @ index 0
+ * and zeros the bit container @ index 1.
+ */
+FORCE_INLINE_TEMPLATE void HUF_mergeIndex1(HUF_CStream_t* bitC)
+{
+ assert((bitC->bitPos[1] & 0xFF) < HUF_BITS_IN_CONTAINER);
+ bitC->bitContainer[0] >>= (bitC->bitPos[1] & 0xFF);
+ bitC->bitContainer[0] |= bitC->bitContainer[1];
+ bitC->bitPos[0] += bitC->bitPos[1];
+ assert((bitC->bitPos[0] & 0xFF) <= HUF_BITS_IN_CONTAINER);
+}
+
+/*! HUF_flushBits() :
+* Flushes the bits in the bit container @ index 0.
+*
+* @post bitPos will be < 8.
+* @param kFast If kFast is set then we must know a-priori that
+* the bit container will not overflow.
+*/
+FORCE_INLINE_TEMPLATE void HUF_flushBits(HUF_CStream_t* bitC, int kFast)
+{
+ /* The upper bits of bitPos are noisy, so we must mask by 0xFF. */
+ size_t const nbBits = bitC->bitPos[0] & 0xFF;
+ size_t const nbBytes = nbBits >> 3;
+ /* The top nbBits bits of bitContainer are the ones we need. */
+ size_t const bitContainer = bitC->bitContainer[0] >> (HUF_BITS_IN_CONTAINER - nbBits);
+ /* Mask bitPos to account for the bytes we consumed. */
+ bitC->bitPos[0] &= 7;
+ assert(nbBits > 0);
+ assert(nbBits <= sizeof(bitC->bitContainer[0]) * 8);
+ assert(bitC->ptr <= bitC->endPtr);
+ MEM_writeLEST(bitC->ptr, bitContainer);
+ bitC->ptr += nbBytes;
+ assert(!kFast || bitC->ptr <= bitC->endPtr);
+ if (!kFast && bitC->ptr > bitC->endPtr) bitC->ptr = bitC->endPtr;
+ /* bitContainer doesn't need to be modified because the leftover
+ * bits are already the top bitPos bits. And we don't care about
+ * noise in the lower values.
+ */
+}
+
+/*! HUF_endMark()
+ * @returns The Huffman stream end mark: A 1-bit value = 1.
+ */
+static HUF_CElt HUF_endMark(void)
+{
+ HUF_CElt endMark;
+ HUF_setNbBits(&endMark, 1);
+ HUF_setValue(&endMark, 1);
+ return endMark;
+}
+
+/*! HUF_closeCStream() :
+ * @return Size of CStream, in bytes,
+ * or 0 if it could not fit into dstBuffer */
+static size_t HUF_closeCStream(HUF_CStream_t* bitC)
+{
+ HUF_addBits(bitC, HUF_endMark(), /* idx */ 0, /* kFast */ 0);
+ HUF_flushBits(bitC, /* kFast */ 0);
+ {
+ size_t const nbBits = bitC->bitPos[0] & 0xFF;
+ if (bitC->ptr >= bitC->endPtr) return 0; /* overflow detected */
+ return (bitC->ptr - bitC->startPtr) + (nbBits > 0);
+ }
+}
+
FORCE_INLINE_TEMPLATE void
-HUF_encodeSymbol(BIT_CStream_t* bitCPtr, U32 symbol, const HUF_CElt* CTable)
+HUF_encodeSymbol(HUF_CStream_t* bitCPtr, U32 symbol, const HUF_CElt* CTable, int idx, int fast)
{
- BIT_addBitsFast(bitCPtr, CTable[symbol].val, CTable[symbol].nbBits);
+ HUF_addBits(bitCPtr, CTable[symbol], idx, fast);
}
-#define HUF_FLUSHBITS(s) BIT_flushBits(s)
+FORCE_INLINE_TEMPLATE void
+HUF_compress1X_usingCTable_internal_body_loop(HUF_CStream_t* bitC,
+ const BYTE* ip, size_t srcSize,
+ const HUF_CElt* ct,
+ int kUnroll, int kFastFlush, int kLastFast)
+{
+ /* Join to kUnroll */
+ int n = (int)srcSize;
+ int rem = n % kUnroll;
+ if (rem > 0) {
+ for (; rem > 0; --rem) {
+ HUF_encodeSymbol(bitC, ip[--n], ct, 0, /* fast */ 0);
+ }
+ HUF_flushBits(bitC, kFastFlush);
+ }
+ assert(n % kUnroll == 0);
+
+ /* Join to 2 * kUnroll */
+ if (n % (2 * kUnroll)) {
+ int u;
+ for (u = 1; u < kUnroll; ++u) {
+ HUF_encodeSymbol(bitC, ip[n - u], ct, 0, 1);
+ }
+ HUF_encodeSymbol(bitC, ip[n - kUnroll], ct, 0, kLastFast);
+ HUF_flushBits(bitC, kFastFlush);
+ n -= kUnroll;
+ }
+ assert(n % (2 * kUnroll) == 0);
+
+ for (; n>0; n-= 2 * kUnroll) {
+ /* Encode kUnroll symbols into the bitstream @ index 0. */
+ int u;
+ for (u = 1; u < kUnroll; ++u) {
+ HUF_encodeSymbol(bitC, ip[n - u], ct, /* idx */ 0, /* fast */ 1);
+ }
+ HUF_encodeSymbol(bitC, ip[n - kUnroll], ct, /* idx */ 0, /* fast */ kLastFast);
+ HUF_flushBits(bitC, kFastFlush);
+ /* Encode kUnroll symbols into the bitstream @ index 1.
+ * This allows us to start filling the bit container
+ * without any data dependencies.
+ */
+ HUF_zeroIndex1(bitC);
+ for (u = 1; u < kUnroll; ++u) {
+ HUF_encodeSymbol(bitC, ip[n - kUnroll - u], ct, /* idx */ 1, /* fast */ 1);
+ }
+ HUF_encodeSymbol(bitC, ip[n - kUnroll - kUnroll], ct, /* idx */ 1, /* fast */ kLastFast);
+ /* Merge bitstream @ index 1 into the bitstream @ index 0 */
+ HUF_mergeIndex1(bitC);
+ HUF_flushBits(bitC, kFastFlush);
+ }
+ assert(n == 0);
+
+}
-#define HUF_FLUSHBITS_1(stream) \
- if (sizeof((stream)->bitContainer)*8 < HUF_TABLELOG_MAX*2+7) HUF_FLUSHBITS(stream)
+/*
+ * Returns a tight upper bound on the output space needed by Huffman
+ * with 8 bytes buffer to handle over-writes. If the output is at least
+ * this large we don't need to do bounds checks during Huffman encoding.
+ */
+static size_t HUF_tightCompressBound(size_t srcSize, size_t tableLog)
+{
+ return ((srcSize * tableLog) >> 3) + 8;
+}
-#define HUF_FLUSHBITS_2(stream) \
- if (sizeof((stream)->bitContainer)*8 < HUF_TABLELOG_MAX*4+7) HUF_FLUSHBITS(stream)
FORCE_INLINE_TEMPLATE size_t
HUF_compress1X_usingCTable_internal_body(void* dst, size_t dstSize,
const void* src, size_t srcSize,
const HUF_CElt* CTable)
{
+ U32 const tableLog = (U32)CTable[0];
+ HUF_CElt const* ct = CTable + 1;
const BYTE* ip = (const BYTE*) src;
BYTE* const ostart = (BYTE*)dst;
BYTE* const oend = ostart + dstSize;
BYTE* op = ostart;
- size_t n;
- BIT_CStream_t bitC;
+ HUF_CStream_t bitC;
/* init */
if (dstSize < 8) return 0; /* not enough space to compress */
- { size_t const initErr = BIT_initCStream(&bitC, op, (size_t)(oend-op));
+ { size_t const initErr = HUF_initCStream(&bitC, op, (size_t)(oend-op));
if (HUF_isError(initErr)) return 0; }
- n = srcSize & ~3; /* join to mod 4 */
- switch (srcSize & 3)
- {
- case 3:
- HUF_encodeSymbol(&bitC, ip[n+ 2], CTable);
- HUF_FLUSHBITS_2(&bitC);
- ZSTD_FALLTHROUGH;
- case 2:
- HUF_encodeSymbol(&bitC, ip[n+ 1], CTable);
- HUF_FLUSHBITS_1(&bitC);
- ZSTD_FALLTHROUGH;
- case 1:
- HUF_encodeSymbol(&bitC, ip[n+ 0], CTable);
- HUF_FLUSHBITS(&bitC);
- ZSTD_FALLTHROUGH;
- case 0: ZSTD_FALLTHROUGH;
- default: break;
- }
-
- for (; n>0; n-=4) { /* note : n&3==0 at this stage */
- HUF_encodeSymbol(&bitC, ip[n- 1], CTable);
- HUF_FLUSHBITS_1(&bitC);
- HUF_encodeSymbol(&bitC, ip[n- 2], CTable);
- HUF_FLUSHBITS_2(&bitC);
- HUF_encodeSymbol(&bitC, ip[n- 3], CTable);
- HUF_FLUSHBITS_1(&bitC);
- HUF_encodeSymbol(&bitC, ip[n- 4], CTable);
- HUF_FLUSHBITS(&bitC);
- }
-
- return BIT_closeCStream(&bitC);
+ if (dstSize < HUF_tightCompressBound(srcSize, (size_t)tableLog) || tableLog > 11)
+ HUF_compress1X_usingCTable_internal_body_loop(&bitC, ip, srcSize, ct, /* kUnroll */ MEM_32bits() ? 2 : 4, /* kFast */ 0, /* kLastFast */ 0);
+ else {
+ if (MEM_32bits()) {
+ switch (tableLog) {
+ case 11:
+ HUF_compress1X_usingCTable_internal_body_loop(&bitC, ip, srcSize, ct, /* kUnroll */ 2, /* kFastFlush */ 1, /* kLastFast */ 0);
+ break;
+ case 10: ZSTD_FALLTHROUGH;
+ case 9: ZSTD_FALLTHROUGH;
+ case 8:
+ HUF_compress1X_usingCTable_internal_body_loop(&bitC, ip, srcSize, ct, /* kUnroll */ 2, /* kFastFlush */ 1, /* kLastFast */ 1);
+ break;
+ case 7: ZSTD_FALLTHROUGH;
+ default:
+ HUF_compress1X_usingCTable_internal_body_loop(&bitC, ip, srcSize, ct, /* kUnroll */ 3, /* kFastFlush */ 1, /* kLastFast */ 1);
+ break;
+ }
+ } else {
+ switch (tableLog) {
+ case 11:
+ HUF_compress1X_usingCTable_internal_body_loop(&bitC, ip, srcSize, ct, /* kUnroll */ 5, /* kFastFlush */ 1, /* kLastFast */ 0);
+ break;
+ case 10:
+ HUF_compress1X_usingCTable_internal_body_loop(&bitC, ip, srcSize, ct, /* kUnroll */ 5, /* kFastFlush */ 1, /* kLastFast */ 1);
+ break;
+ case 9:
+ HUF_compress1X_usingCTable_internal_body_loop(&bitC, ip, srcSize, ct, /* kUnroll */ 6, /* kFastFlush */ 1, /* kLastFast */ 0);
+ break;
+ case 8:
+ HUF_compress1X_usingCTable_internal_body_loop(&bitC, ip, srcSize, ct, /* kUnroll */ 7, /* kFastFlush */ 1, /* kLastFast */ 0);
+ break;
+ case 7:
+ HUF_compress1X_usingCTable_internal_body_loop(&bitC, ip, srcSize, ct, /* kUnroll */ 8, /* kFastFlush */ 1, /* kLastFast */ 0);
+ break;
+ case 6: ZSTD_FALLTHROUGH;
+ default:
+ HUF_compress1X_usingCTable_internal_body_loop(&bitC, ip, srcSize, ct, /* kUnroll */ 9, /* kFastFlush */ 1, /* kLastFast */ 1);
+ break;
+ }
+ }
+ }
+ assert(bitC.ptr <= bitC.endPtr);
+
+ return HUF_closeCStream(&bitC);
}
#if DYNAMIC_BMI2
-static TARGET_ATTRIBUTE("bmi2") size_t
+static BMI2_TARGET_ATTRIBUTE size_t
HUF_compress1X_usingCTable_internal_bmi2(void* dst, size_t dstSize,
const void* src, size_t srcSize,
const HUF_CElt* CTable)
size_t HUF_compress1X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable)
{
- return HUF_compress1X_usingCTable_internal(dst, dstSize, src, srcSize, CTable, /* bmi2 */ 0);
+ return HUF_compress1X_usingCTable_bmi2(dst, dstSize, src, srcSize, CTable, /* bmi2 */ 0);
}
+size_t HUF_compress1X_usingCTable_bmi2(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable, int bmi2)
+{
+ return HUF_compress1X_usingCTable_internal(dst, dstSize, src, srcSize, CTable, bmi2);
+}
static size_t
HUF_compress4X_usingCTable_internal(void* dst, size_t dstSize,
assert(op <= oend);
{ CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, segmentSize, CTable, bmi2) );
- if (cSize==0) return 0;
- assert(cSize <= 65535);
+ if (cSize == 0 || cSize > 65535) return 0;
MEM_writeLE16(ostart, (U16)cSize);
op += cSize;
}
ip += segmentSize;
assert(op <= oend);
{ CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, segmentSize, CTable, bmi2) );
- if (cSize==0) return 0;
- assert(cSize <= 65535);
+ if (cSize == 0 || cSize > 65535) return 0;
MEM_writeLE16(ostart+2, (U16)cSize);
op += cSize;
}
ip += segmentSize;
assert(op <= oend);
{ CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, segmentSize, CTable, bmi2) );
- if (cSize==0) return 0;
- assert(cSize <= 65535);
+ if (cSize == 0 || cSize > 65535) return 0;
MEM_writeLE16(ostart+4, (U16)cSize);
op += cSize;
}
assert(op <= oend);
assert(ip <= iend);
{ CHECK_V_F(cSize, HUF_compress1X_usingCTable_internal(op, (size_t)(oend-op), ip, (size_t)(iend-ip), CTable, bmi2) );
- if (cSize==0) return 0;
+ if (cSize == 0 || cSize > 65535) return 0;
op += cSize;
}
size_t HUF_compress4X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable)
{
- return HUF_compress4X_usingCTable_internal(dst, dstSize, src, srcSize, CTable, /* bmi2 */ 0);
+ return HUF_compress4X_usingCTable_bmi2(dst, dstSize, src, srcSize, CTable, /* bmi2 */ 0);
+}
+
+size_t HUF_compress4X_usingCTable_bmi2(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable, int bmi2)
+{
+ return HUF_compress4X_usingCTable_internal(dst, dstSize, src, srcSize, CTable, bmi2);
}
typedef enum { HUF_singleStream, HUF_fourStreams } HUF_nbStreams_e;
typedef struct {
unsigned count[HUF_SYMBOLVALUE_MAX + 1];
- HUF_CElt CTable[HUF_SYMBOLVALUE_MAX + 1];
+ HUF_CElt CTable[HUF_CTABLE_SIZE_ST(HUF_SYMBOLVALUE_MAX)];
union {
HUF_buildCTable_wksp_tables buildCTable_wksp;
HUF_WriteCTableWksp writeCTable_wksp;
+ U32 hist_wksp[HIST_WKSP_SIZE_U32];
} wksps;
} HUF_compress_tables_t;
+#define SUSPECT_INCOMPRESSIBLE_SAMPLE_SIZE 4096
+#define SUSPECT_INCOMPRESSIBLE_SAMPLE_RATIO 10 /* Must be >= 2 */
+
/* HUF_compress_internal() :
* `workSpace_align4` must be aligned on 4-bytes boundaries,
- * and occupies the same space as a table of HUF_WORKSPACE_SIZE_U32 unsigned */
+ * and occupies the same space as a table of HUF_WORKSPACE_SIZE_U64 unsigned */
static size_t
HUF_compress_internal (void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned huffLog,
HUF_nbStreams_e nbStreams,
- void* workSpace_align4, size_t wkspSize,
+ void* workSpace, size_t wkspSize,
HUF_CElt* oldHufTable, HUF_repeat* repeat, int preferRepeat,
- const int bmi2)
+ const int bmi2, unsigned suspectUncompressible)
{
- HUF_compress_tables_t* const table = (HUF_compress_tables_t*)workSpace_align4;
+ HUF_compress_tables_t* const table = (HUF_compress_tables_t*)HUF_alignUpWorkspace(workSpace, &wkspSize, ZSTD_ALIGNOF(size_t));
BYTE* const ostart = (BYTE*)dst;
BYTE* const oend = ostart + dstSize;
BYTE* op = ostart;
- HUF_STATIC_ASSERT(sizeof(*table) <= HUF_WORKSPACE_SIZE);
- assert(((size_t)workSpace_align4 & 3) == 0); /* must be aligned on 4-bytes boundaries */
+ HUF_STATIC_ASSERT(sizeof(*table) + HUF_WORKSPACE_MAX_ALIGNMENT <= HUF_WORKSPACE_SIZE);
/* checks & inits */
- if (wkspSize < HUF_WORKSPACE_SIZE) return ERROR(workSpace_tooSmall);
+ if (wkspSize < sizeof(*table)) return ERROR(workSpace_tooSmall);
if (!srcSize) return 0; /* Uncompressed */
if (!dstSize) return 0; /* cannot fit anything within dst budget */
if (srcSize > HUF_BLOCKSIZE_MAX) return ERROR(srcSize_wrong); /* current block size limit */
nbStreams, oldHufTable, bmi2);
}
+ /* If uncompressible data is suspected, do a smaller sampling first */
+ DEBUG_STATIC_ASSERT(SUSPECT_INCOMPRESSIBLE_SAMPLE_RATIO >= 2);
+ if (suspectUncompressible && srcSize >= (SUSPECT_INCOMPRESSIBLE_SAMPLE_SIZE * SUSPECT_INCOMPRESSIBLE_SAMPLE_RATIO)) {
+ size_t largestTotal = 0;
+ { unsigned maxSymbolValueBegin = maxSymbolValue;
+ CHECK_V_F(largestBegin, HIST_count_simple (table->count, &maxSymbolValueBegin, (const BYTE*)src, SUSPECT_INCOMPRESSIBLE_SAMPLE_SIZE) );
+ largestTotal += largestBegin;
+ }
+ { unsigned maxSymbolValueEnd = maxSymbolValue;
+ CHECK_V_F(largestEnd, HIST_count_simple (table->count, &maxSymbolValueEnd, (const BYTE*)src + srcSize - SUSPECT_INCOMPRESSIBLE_SAMPLE_SIZE, SUSPECT_INCOMPRESSIBLE_SAMPLE_SIZE) );
+ largestTotal += largestEnd;
+ }
+ if (largestTotal <= ((2 * SUSPECT_INCOMPRESSIBLE_SAMPLE_SIZE) >> 7)+4) return 0; /* heuristic : probably not compressible enough */
+ }
+
/* Scan input and build symbol stats */
- { CHECK_V_F(largest, HIST_count_wksp (table->count, &maxSymbolValue, (const BYTE*)src, srcSize, workSpace_align4, wkspSize) );
+ { CHECK_V_F(largest, HIST_count_wksp (table->count, &maxSymbolValue, (const BYTE*)src, srcSize, table->wksps.hist_wksp, sizeof(table->wksps.hist_wksp)) );
if (largest == srcSize) { *ostart = ((const BYTE*)src)[0]; return 1; } /* single symbol, rle */
if (largest <= (srcSize >> 7)+4) return 0; /* heuristic : probably not compressible enough */
}
&table->wksps.buildCTable_wksp, sizeof(table->wksps.buildCTable_wksp));
CHECK_F(maxBits);
huffLog = (U32)maxBits;
- /* Zero unused symbols in CTable, so we can check it for validity */
- ZSTD_memset(table->CTable + (maxSymbolValue + 1), 0,
- sizeof(table->CTable) - ((maxSymbolValue + 1) * sizeof(HUF_CElt)));
+ }
+ /* Zero unused symbols in CTable, so we can check it for validity */
+ {
+ size_t const ctableSize = HUF_CTABLE_SIZE_ST(maxSymbolValue);
+ size_t const unusedSize = sizeof(table->CTable) - ctableSize * sizeof(HUF_CElt);
+ ZSTD_memset(table->CTable + ctableSize, 0, unusedSize);
}
/* Write table description header */
return HUF_compress_internal(dst, dstSize, src, srcSize,
maxSymbolValue, huffLog, HUF_singleStream,
workSpace, wkspSize,
- NULL, NULL, 0, 0 /*bmi2*/);
+ NULL, NULL, 0, 0 /*bmi2*/, 0);
}
size_t HUF_compress1X_repeat (void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned huffLog,
void* workSpace, size_t wkspSize,
- HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2)
+ HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat,
+ int bmi2, unsigned suspectUncompressible)
{
return HUF_compress_internal(dst, dstSize, src, srcSize,
maxSymbolValue, huffLog, HUF_singleStream,
workSpace, wkspSize, hufTable,
- repeat, preferRepeat, bmi2);
+ repeat, preferRepeat, bmi2, suspectUncompressible);
}
/* HUF_compress4X_repeat():
return HUF_compress_internal(dst, dstSize, src, srcSize,
maxSymbolValue, huffLog, HUF_fourStreams,
workSpace, wkspSize,
- NULL, NULL, 0, 0 /*bmi2*/);
+ NULL, NULL, 0, 0 /*bmi2*/, 0);
}
/* HUF_compress4X_repeat():
* compress input using 4 streams.
+ * consider skipping quickly
* re-use an existing huffman compression table */
size_t HUF_compress4X_repeat (void* dst, size_t dstSize,
const void* src, size_t srcSize,
unsigned maxSymbolValue, unsigned huffLog,
void* workSpace, size_t wkspSize,
- HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2)
+ HUF_CElt* hufTable, HUF_repeat* repeat, int preferRepeat, int bmi2, unsigned suspectUncompressible)
{
return HUF_compress_internal(dst, dstSize, src, srcSize,
maxSymbolValue, huffLog, HUF_fourStreams,
workSpace, wkspSize,
- hufTable, repeat, preferRepeat, bmi2);
+ hufTable, repeat, preferRepeat, bmi2, suspectUncompressible);
}
* Dependencies
***************************************/
#include "../common/zstd_deps.h" /* INT_MAX, ZSTD_memset, ZSTD_memcpy */
-#include "../common/cpu.h"
#include "../common/mem.h"
#include "hist.h" /* HIST_countFast_wksp */
#define FSE_STATIC_LINKING_ONLY /* FSE_encodeSymbol */
* Note that functions with explicit context such as ZSTD_compressCCtx() are unaffected.
*/
+/*!
+ * ZSTD_HASHLOG3_MAX :
+ * Maximum size of the hash table dedicated to find 3-bytes matches,
+ * in log format, aka 17 => 1 << 17 == 128Ki positions.
+ * This structure is only used in zstd_opt.
+ * Since allocation is centralized for all strategies, it has to be known here.
+ * The actual (selected) size of the hash table is then stored in ZSTD_matchState_t.hashLog3,
+ * so that zstd_opt.c doesn't need to know about this constant.
+ */
+#ifndef ZSTD_HASHLOG3_MAX
+# define ZSTD_HASHLOG3_MAX 17
+#endif
/*-*************************************
* Helper functions
ZSTD_customMem customMem;
U32 dictID;
int compressionLevel; /* 0 indicates that advanced API was used to select CDict params */
+ ZSTD_paramSwitch_e useRowMatchFinder; /* Indicates whether the CDict was created with params that would use
+ * row-based matchfinder. Unless the cdict is reloaded, we will use
+ * the same greedy/lazy matchfinder at compression time.
+ */
}; /* typedef'd to ZSTD_CDict within "zstd.h" */
ZSTD_CCtx* ZSTD_createCCtx(void)
assert(cctx != NULL);
ZSTD_memset(cctx, 0, sizeof(*cctx));
cctx->customMem = memManager;
- cctx->bmi2 = ZSTD_cpuid_bmi2(ZSTD_cpuid());
+ cctx->bmi2 = ZSTD_cpuSupportsBmi2();
{ size_t const err = ZSTD_CCtx_reset(cctx, ZSTD_reset_parameters);
assert(!ZSTD_isError(err));
(void)err;
/* private API call, for dictBuilder only */
const seqStore_t* ZSTD_getSeqStore(const ZSTD_CCtx* ctx) { return &(ctx->seqStore); }
+/* Returns true if the strategy supports using a row based matchfinder */
+static int ZSTD_rowMatchFinderSupported(const ZSTD_strategy strategy) {
+ return (strategy >= ZSTD_greedy && strategy <= ZSTD_lazy2);
+}
+
+/* Returns true if the strategy and useRowMatchFinder mode indicate that we will use the row based matchfinder
+ * for this compression.
+ */
+static int ZSTD_rowMatchFinderUsed(const ZSTD_strategy strategy, const ZSTD_paramSwitch_e mode) {
+ assert(mode != ZSTD_ps_auto);
+ return ZSTD_rowMatchFinderSupported(strategy) && (mode == ZSTD_ps_enable);
+}
+
+/* Returns row matchfinder usage given an initial mode and cParams */
+static ZSTD_paramSwitch_e ZSTD_resolveRowMatchFinderMode(ZSTD_paramSwitch_e mode,
+ const ZSTD_compressionParameters* const cParams) {
+#if defined(ZSTD_ARCH_X86_SSE2) || defined(ZSTD_ARCH_ARM_NEON)
+ int const kHasSIMD128 = 1;
+#else
+ int const kHasSIMD128 = 0;
+#endif
+ if (mode != ZSTD_ps_auto) return mode; /* if requested enabled, but no SIMD, we still will use row matchfinder */
+ mode = ZSTD_ps_disable;
+ if (!ZSTD_rowMatchFinderSupported(cParams->strategy)) return mode;
+ if (kHasSIMD128) {
+ if (cParams->windowLog > 14) mode = ZSTD_ps_enable;
+ } else {
+ if (cParams->windowLog > 17) mode = ZSTD_ps_enable;
+ }
+ return mode;
+}
+
+/* Returns block splitter usage (generally speaking, when using slower/stronger compression modes) */
+static ZSTD_paramSwitch_e ZSTD_resolveBlockSplitterMode(ZSTD_paramSwitch_e mode,
+ const ZSTD_compressionParameters* const cParams) {
+ if (mode != ZSTD_ps_auto) return mode;
+ return (cParams->strategy >= ZSTD_btopt && cParams->windowLog >= 17) ? ZSTD_ps_enable : ZSTD_ps_disable;
+}
+
+/* Returns 1 if the arguments indicate that we should allocate a chainTable, 0 otherwise */
+static int ZSTD_allocateChainTable(const ZSTD_strategy strategy,
+ const ZSTD_paramSwitch_e useRowMatchFinder,
+ const U32 forDDSDict) {
+ assert(useRowMatchFinder != ZSTD_ps_auto);
+ /* We always should allocate a chaintable if we are allocating a matchstate for a DDS dictionary matchstate.
+ * We do not allocate a chaintable if we are using ZSTD_fast, or are using the row-based matchfinder.
+ */
+ return forDDSDict || ((strategy != ZSTD_fast) && !ZSTD_rowMatchFinderUsed(strategy, useRowMatchFinder));
+}
+
/* Returns 1 if compression parameters are such that we should
* enable long distance matching (wlog >= 27, strategy >= btopt).
* Returns 0 otherwise.
*/
-static U32 ZSTD_CParams_shouldEnableLdm(const ZSTD_compressionParameters* const cParams) {
- return cParams->strategy >= ZSTD_btopt && cParams->windowLog >= 27;
+static ZSTD_paramSwitch_e ZSTD_resolveEnableLdm(ZSTD_paramSwitch_e mode,
+ const ZSTD_compressionParameters* const cParams) {
+ if (mode != ZSTD_ps_auto) return mode;
+ return (cParams->strategy >= ZSTD_btopt && cParams->windowLog >= 27) ? ZSTD_ps_enable : ZSTD_ps_disable;
}
static ZSTD_CCtx_params ZSTD_makeCCtxParamsFromCParams(
ZSTD_CCtxParams_init(&cctxParams, ZSTD_CLEVEL_DEFAULT);
cctxParams.cParams = cParams;
- if (ZSTD_CParams_shouldEnableLdm(&cParams)) {
- DEBUGLOG(4, "ZSTD_makeCCtxParamsFromCParams(): Including LDM into cctx params");
- cctxParams.ldmParams.enableLdm = 1;
- /* LDM is enabled by default for optimal parser and window size >= 128MB */
+ /* Adjust advanced params according to cParams */
+ cctxParams.ldmParams.enableLdm = ZSTD_resolveEnableLdm(cctxParams.ldmParams.enableLdm, &cParams);
+ if (cctxParams.ldmParams.enableLdm == ZSTD_ps_enable) {
ZSTD_ldm_adjustParameters(&cctxParams.ldmParams, &cParams);
assert(cctxParams.ldmParams.hashLog >= cctxParams.ldmParams.bucketSizeLog);
assert(cctxParams.ldmParams.hashRateLog < 32);
}
-
+ cctxParams.useBlockSplitter = ZSTD_resolveBlockSplitterMode(cctxParams.useBlockSplitter, &cParams);
+ cctxParams.useRowMatchFinder = ZSTD_resolveRowMatchFinderMode(cctxParams.useRowMatchFinder, &cParams);
assert(!ZSTD_checkCParams(cParams));
return cctxParams;
}
* But, set it for tracing anyway.
*/
cctxParams->compressionLevel = compressionLevel;
+ cctxParams->useRowMatchFinder = ZSTD_resolveRowMatchFinderMode(cctxParams->useRowMatchFinder, ¶ms->cParams);
+ cctxParams->useBlockSplitter = ZSTD_resolveBlockSplitterMode(cctxParams->useBlockSplitter, ¶ms->cParams);
+ cctxParams->ldmParams.enableLdm = ZSTD_resolveEnableLdm(cctxParams->ldmParams.enableLdm, ¶ms->cParams);
+ DEBUGLOG(4, "ZSTD_CCtxParams_init_internal: useRowMatchFinder=%d, useBlockSplitter=%d ldm=%d",
+ cctxParams->useRowMatchFinder, cctxParams->useBlockSplitter, cctxParams->ldmParams.enableLdm);
}
size_t ZSTD_CCtxParams_init_advanced(ZSTD_CCtx_params* cctxParams, ZSTD_parameters params)
return bounds;
case ZSTD_c_literalCompressionMode:
- ZSTD_STATIC_ASSERT(ZSTD_lcm_auto < ZSTD_lcm_huffman && ZSTD_lcm_huffman < ZSTD_lcm_uncompressed);
- bounds.lowerBound = ZSTD_lcm_auto;
- bounds.upperBound = ZSTD_lcm_uncompressed;
+ ZSTD_STATIC_ASSERT(ZSTD_ps_auto < ZSTD_ps_enable && ZSTD_ps_enable < ZSTD_ps_disable);
+ bounds.lowerBound = (int)ZSTD_ps_auto;
+ bounds.upperBound = (int)ZSTD_ps_disable;
return bounds;
case ZSTD_c_targetCBlockSize:
bounds.upperBound = 1;
return bounds;
+ case ZSTD_c_useBlockSplitter:
+ bounds.lowerBound = (int)ZSTD_ps_auto;
+ bounds.upperBound = (int)ZSTD_ps_disable;
+ return bounds;
+
+ case ZSTD_c_useRowMatchFinder:
+ bounds.lowerBound = (int)ZSTD_ps_auto;
+ bounds.upperBound = (int)ZSTD_ps_disable;
+ return bounds;
+
+ case ZSTD_c_deterministicRefPrefix:
+ bounds.lowerBound = 0;
+ bounds.upperBound = 1;
+ return bounds;
+
default:
bounds.error = ERROR(parameter_unsupported);
return bounds;
case ZSTD_c_stableOutBuffer:
case ZSTD_c_blockDelimiters:
case ZSTD_c_validateSequences:
+ case ZSTD_c_useBlockSplitter:
+ case ZSTD_c_useRowMatchFinder:
+ case ZSTD_c_deterministicRefPrefix:
default:
return 0;
}
case ZSTD_c_stableOutBuffer:
case ZSTD_c_blockDelimiters:
case ZSTD_c_validateSequences:
+ case ZSTD_c_useBlockSplitter:
+ case ZSTD_c_useRowMatchFinder:
+ case ZSTD_c_deterministicRefPrefix:
break;
default: RETURN_ERROR(parameter_unsupported, "unknown parameter");
}
case ZSTD_c_literalCompressionMode : {
- const ZSTD_literalCompressionMode_e lcm = (ZSTD_literalCompressionMode_e)value;
+ const ZSTD_paramSwitch_e lcm = (ZSTD_paramSwitch_e)value;
BOUNDCHECK(ZSTD_c_literalCompressionMode, lcm);
CCtxParams->literalCompressionMode = lcm;
return CCtxParams->literalCompressionMode;
return CCtxParams->enableDedicatedDictSearch;
case ZSTD_c_enableLongDistanceMatching :
- CCtxParams->ldmParams.enableLdm = (value!=0);
+ CCtxParams->ldmParams.enableLdm = (ZSTD_paramSwitch_e)value;
return CCtxParams->ldmParams.enableLdm;
case ZSTD_c_ldmHashLog :
CCtxParams->validateSequences = value;
return CCtxParams->validateSequences;
+ case ZSTD_c_useBlockSplitter:
+ BOUNDCHECK(ZSTD_c_useBlockSplitter, value);
+ CCtxParams->useBlockSplitter = (ZSTD_paramSwitch_e)value;
+ return CCtxParams->useBlockSplitter;
+
+ case ZSTD_c_useRowMatchFinder:
+ BOUNDCHECK(ZSTD_c_useRowMatchFinder, value);
+ CCtxParams->useRowMatchFinder = (ZSTD_paramSwitch_e)value;
+ return CCtxParams->useRowMatchFinder;
+
+ case ZSTD_c_deterministicRefPrefix:
+ BOUNDCHECK(ZSTD_c_deterministicRefPrefix, value);
+ CCtxParams->deterministicRefPrefix = !!value;
+ return CCtxParams->deterministicRefPrefix;
+
default: RETURN_ERROR(parameter_unsupported, "unknown parameter");
}
}
case ZSTD_c_validateSequences :
*value = (int)CCtxParams->validateSequences;
break;
+ case ZSTD_c_useBlockSplitter :
+ *value = (int)CCtxParams->useBlockSplitter;
+ break;
+ case ZSTD_c_useRowMatchFinder :
+ *value = (int)CCtxParams->useRowMatchFinder;
+ break;
+ case ZSTD_c_deterministicRefPrefix:
+ *value = (int)CCtxParams->deterministicRefPrefix;
+ break;
default: RETURN_ERROR(parameter_unsupported, "unknown parameter");
}
return 0;
return 0;
}
-ZSTDLIB_API size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize)
+size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize)
{
DEBUGLOG(4, "ZSTD_CCtx_setPledgedSrcSize to %u bytes", (U32)pledgedSrcSize);
RETURN_ERROR_IF(cctx->streamStage != zcss_init, stage_wrong,
return 0;
}
-ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary_byReference(
+size_t ZSTD_CCtx_loadDictionary_byReference(
ZSTD_CCtx* cctx, const void* dict, size_t dictSize)
{
return ZSTD_CCtx_loadDictionary_advanced(
cctx, dict, dictSize, ZSTD_dlm_byRef, ZSTD_dct_auto);
}
-ZSTDLIB_API size_t ZSTD_CCtx_loadDictionary(ZSTD_CCtx* cctx, const void* dict, size_t dictSize)
+size_t ZSTD_CCtx_loadDictionary(ZSTD_CCtx* cctx, const void* dict, size_t dictSize)
{
return ZSTD_CCtx_loadDictionary_advanced(
cctx, dict, dictSize, ZSTD_dlm_byCopy, ZSTD_dct_auto);
break;
case ZSTD_cpm_createCDict:
/* Assume a small source size when creating a dictionary
- * with an unkown source size.
+ * with an unknown source size.
*/
if (dictSize && srcSize == ZSTD_CONTENTSIZE_UNKNOWN)
srcSize = minSrcSize;
srcSizeHint = CCtxParams->srcSizeHint;
}
cParams = ZSTD_getCParams_internal(CCtxParams->compressionLevel, srcSizeHint, dictSize, mode);
- if (CCtxParams->ldmParams.enableLdm) cParams.windowLog = ZSTD_LDM_DEFAULT_WINDOW_LOG;
+ if (CCtxParams->ldmParams.enableLdm == ZSTD_ps_enable) cParams.windowLog = ZSTD_LDM_DEFAULT_WINDOW_LOG;
ZSTD_overrideCParams(&cParams, &CCtxParams->cParams);
assert(!ZSTD_checkCParams(cParams));
/* srcSizeHint == 0 means 0 */
static size_t
ZSTD_sizeof_matchState(const ZSTD_compressionParameters* const cParams,
+ const ZSTD_paramSwitch_e useRowMatchFinder,
+ const U32 enableDedicatedDictSearch,
const U32 forCCtx)
{
- size_t const chainSize = (cParams->strategy == ZSTD_fast) ? 0 : ((size_t)1 << cParams->chainLog);
+ /* chain table size should be 0 for fast or row-hash strategies */
+ size_t const chainSize = ZSTD_allocateChainTable(cParams->strategy, useRowMatchFinder, enableDedicatedDictSearch && !forCCtx)
+ ? ((size_t)1 << cParams->chainLog)
+ : 0;
size_t const hSize = ((size_t)1) << cParams->hashLog;
U32 const hashLog3 = (forCCtx && cParams->minMatch==3) ? MIN(ZSTD_HASHLOG3_MAX, cParams->windowLog) : 0;
size_t const h3Size = hashLog3 ? ((size_t)1) << hashLog3 : 0;
+ hSize * sizeof(U32)
+ h3Size * sizeof(U32);
size_t const optPotentialSpace =
- ZSTD_cwksp_alloc_size((MaxML+1) * sizeof(U32))
- + ZSTD_cwksp_alloc_size((MaxLL+1) * sizeof(U32))
- + ZSTD_cwksp_alloc_size((MaxOff+1) * sizeof(U32))
- + ZSTD_cwksp_alloc_size((1<<Litbits) * sizeof(U32))
- + ZSTD_cwksp_alloc_size((ZSTD_OPT_NUM+1) * sizeof(ZSTD_match_t))
- + ZSTD_cwksp_alloc_size((ZSTD_OPT_NUM+1) * sizeof(ZSTD_optimal_t));
+ ZSTD_cwksp_aligned_alloc_size((MaxML+1) * sizeof(U32))
+ + ZSTD_cwksp_aligned_alloc_size((MaxLL+1) * sizeof(U32))
+ + ZSTD_cwksp_aligned_alloc_size((MaxOff+1) * sizeof(U32))
+ + ZSTD_cwksp_aligned_alloc_size((1<<Litbits) * sizeof(U32))
+ + ZSTD_cwksp_aligned_alloc_size((ZSTD_OPT_NUM+1) * sizeof(ZSTD_match_t))
+ + ZSTD_cwksp_aligned_alloc_size((ZSTD_OPT_NUM+1) * sizeof(ZSTD_optimal_t));
+ size_t const lazyAdditionalSpace = ZSTD_rowMatchFinderUsed(cParams->strategy, useRowMatchFinder)
+ ? ZSTD_cwksp_aligned_alloc_size(hSize*sizeof(U16))
+ : 0;
size_t const optSpace = (forCCtx && (cParams->strategy >= ZSTD_btopt))
? optPotentialSpace
: 0;
+ size_t const slackSpace = ZSTD_cwksp_slack_space_required();
+
+ /* tables are guaranteed to be sized in multiples of 64 bytes (or 16 uint32_t) */
+ ZSTD_STATIC_ASSERT(ZSTD_HASHLOG_MIN >= 4 && ZSTD_WINDOWLOG_MIN >= 4 && ZSTD_CHAINLOG_MIN >= 4);
+ assert(useRowMatchFinder != ZSTD_ps_auto);
+
DEBUGLOG(4, "chainSize: %u - hSize: %u - h3Size: %u",
(U32)chainSize, (U32)hSize, (U32)h3Size);
- return tableSpace + optSpace;
+ return tableSpace + optSpace + slackSpace + lazyAdditionalSpace;
}
static size_t ZSTD_estimateCCtxSize_usingCCtxParams_internal(
const ZSTD_compressionParameters* cParams,
const ldmParams_t* ldmParams,
const int isStatic,
+ const ZSTD_paramSwitch_e useRowMatchFinder,
const size_t buffInSize,
const size_t buffOutSize,
const U64 pledgedSrcSize)
{
- size_t const windowSize = MAX(1, (size_t)MIN(((U64)1 << cParams->windowLog), pledgedSrcSize));
+ size_t const windowSize = (size_t) BOUNDED(1ULL, 1ULL << cParams->windowLog, pledgedSrcSize);
size_t const blockSize = MIN(ZSTD_BLOCKSIZE_MAX, windowSize);
U32 const divider = (cParams->minMatch==3) ? 3 : 4;
size_t const maxNbSeq = blockSize / divider;
size_t const tokenSpace = ZSTD_cwksp_alloc_size(WILDCOPY_OVERLENGTH + blockSize)
- + ZSTD_cwksp_alloc_size(maxNbSeq * sizeof(seqDef))
+ + ZSTD_cwksp_aligned_alloc_size(maxNbSeq * sizeof(seqDef))
+ 3 * ZSTD_cwksp_alloc_size(maxNbSeq * sizeof(BYTE));
size_t const entropySpace = ZSTD_cwksp_alloc_size(ENTROPY_WORKSPACE_SIZE);
size_t const blockStateSpace = 2 * ZSTD_cwksp_alloc_size(sizeof(ZSTD_compressedBlockState_t));
- size_t const matchStateSize = ZSTD_sizeof_matchState(cParams, /* forCCtx */ 1);
+ size_t const matchStateSize = ZSTD_sizeof_matchState(cParams, useRowMatchFinder, /* enableDedicatedDictSearch */ 0, /* forCCtx */ 1);
size_t const ldmSpace = ZSTD_ldm_getTableSize(*ldmParams);
size_t const maxNbLdmSeq = ZSTD_ldm_getMaxNbSeq(*ldmParams, blockSize);
- size_t const ldmSeqSpace = ldmParams->enableLdm ?
- ZSTD_cwksp_alloc_size(maxNbLdmSeq * sizeof(rawSeq)) : 0;
+ size_t const ldmSeqSpace = ldmParams->enableLdm == ZSTD_ps_enable ?
+ ZSTD_cwksp_aligned_alloc_size(maxNbLdmSeq * sizeof(rawSeq)) : 0;
size_t const bufferSpace = ZSTD_cwksp_alloc_size(buffInSize)
{
ZSTD_compressionParameters const cParams =
ZSTD_getCParamsFromCCtxParams(params, ZSTD_CONTENTSIZE_UNKNOWN, 0, ZSTD_cpm_noAttachDict);
+ ZSTD_paramSwitch_e const useRowMatchFinder = ZSTD_resolveRowMatchFinderMode(params->useRowMatchFinder,
+ &cParams);
RETURN_ERROR_IF(params->nbWorkers > 0, GENERIC, "Estimate CCtx size is supported for single-threaded compression only.");
/* estimateCCtxSize is for one-shot compression. So no buffers should
* be needed. However, we still allocate two 0-sized buffers, which can
* take space under ASAN. */
return ZSTD_estimateCCtxSize_usingCCtxParams_internal(
- &cParams, ¶ms->ldmParams, 1, 0, 0, ZSTD_CONTENTSIZE_UNKNOWN);
+ &cParams, ¶ms->ldmParams, 1, useRowMatchFinder, 0, 0, ZSTD_CONTENTSIZE_UNKNOWN);
}
size_t ZSTD_estimateCCtxSize_usingCParams(ZSTD_compressionParameters cParams)
{
- ZSTD_CCtx_params const params = ZSTD_makeCCtxParamsFromCParams(cParams);
- return ZSTD_estimateCCtxSize_usingCCtxParams(¶ms);
+ ZSTD_CCtx_params initialParams = ZSTD_makeCCtxParamsFromCParams(cParams);
+ if (ZSTD_rowMatchFinderSupported(cParams.strategy)) {
+ /* Pick bigger of not using and using row-based matchfinder for greedy and lazy strategies */
+ size_t noRowCCtxSize;
+ size_t rowCCtxSize;
+ initialParams.useRowMatchFinder = ZSTD_ps_disable;
+ noRowCCtxSize = ZSTD_estimateCCtxSize_usingCCtxParams(&initialParams);
+ initialParams.useRowMatchFinder = ZSTD_ps_enable;
+ rowCCtxSize = ZSTD_estimateCCtxSize_usingCCtxParams(&initialParams);
+ return MAX(noRowCCtxSize, rowCCtxSize);
+ } else {
+ return ZSTD_estimateCCtxSize_usingCCtxParams(&initialParams);
+ }
}
static size_t ZSTD_estimateCCtxSize_internal(int compressionLevel)
size_t const outBuffSize = (params->outBufferMode == ZSTD_bm_buffered)
? ZSTD_compressBound(blockSize) + 1
: 0;
+ ZSTD_paramSwitch_e const useRowMatchFinder = ZSTD_resolveRowMatchFinderMode(params->useRowMatchFinder, ¶ms->cParams);
return ZSTD_estimateCCtxSize_usingCCtxParams_internal(
- &cParams, ¶ms->ldmParams, 1, inBuffSize, outBuffSize,
+ &cParams, ¶ms->ldmParams, 1, useRowMatchFinder, inBuffSize, outBuffSize,
ZSTD_CONTENTSIZE_UNKNOWN);
}
}
size_t ZSTD_estimateCStreamSize_usingCParams(ZSTD_compressionParameters cParams)
{
- ZSTD_CCtx_params const params = ZSTD_makeCCtxParamsFromCParams(cParams);
- return ZSTD_estimateCStreamSize_usingCCtxParams(¶ms);
+ ZSTD_CCtx_params initialParams = ZSTD_makeCCtxParamsFromCParams(cParams);
+ if (ZSTD_rowMatchFinderSupported(cParams.strategy)) {
+ /* Pick bigger of not using and using row-based matchfinder for greedy and lazy strategies */
+ size_t noRowCCtxSize;
+ size_t rowCCtxSize;
+ initialParams.useRowMatchFinder = ZSTD_ps_disable;
+ noRowCCtxSize = ZSTD_estimateCStreamSize_usingCCtxParams(&initialParams);
+ initialParams.useRowMatchFinder = ZSTD_ps_enable;
+ rowCCtxSize = ZSTD_estimateCStreamSize_usingCCtxParams(&initialParams);
+ return MAX(noRowCCtxSize, rowCCtxSize);
+ } else {
+ return ZSTD_estimateCStreamSize_usingCCtxParams(&initialParams);
+ }
}
static size_t ZSTD_estimateCStreamSize_internal(int compressionLevel)
ZSTD_resetTarget_CCtx
} ZSTD_resetTarget_e;
+
static size_t
ZSTD_reset_matchState(ZSTD_matchState_t* ms,
ZSTD_cwksp* ws,
const ZSTD_compressionParameters* cParams,
+ const ZSTD_paramSwitch_e useRowMatchFinder,
const ZSTD_compResetPolicy_e crp,
const ZSTD_indexResetPolicy_e forceResetIndex,
const ZSTD_resetTarget_e forWho)
{
- size_t const chainSize = (cParams->strategy == ZSTD_fast) ? 0 : ((size_t)1 << cParams->chainLog);
+ /* disable chain table allocation for fast or row-based strategies */
+ size_t const chainSize = ZSTD_allocateChainTable(cParams->strategy, useRowMatchFinder,
+ ms->dedicatedDictSearch && (forWho == ZSTD_resetTarget_CDict))
+ ? ((size_t)1 << cParams->chainLog)
+ : 0;
size_t const hSize = ((size_t)1) << cParams->hashLog;
U32 const hashLog3 = ((forWho == ZSTD_resetTarget_CCtx) && cParams->minMatch==3) ? MIN(ZSTD_HASHLOG3_MAX, cParams->windowLog) : 0;
size_t const h3Size = hashLog3 ? ((size_t)1) << hashLog3 : 0;
DEBUGLOG(4, "reset indices : %u", forceResetIndex == ZSTDirp_reset);
+ assert(useRowMatchFinder != ZSTD_ps_auto);
if (forceResetIndex == ZSTDirp_reset) {
ZSTD_window_init(&ms->window);
ZSTD_cwksp_mark_tables_dirty(ws);
ms->opt.priceTable = (ZSTD_optimal_t*)ZSTD_cwksp_reserve_aligned(ws, (ZSTD_OPT_NUM+1) * sizeof(ZSTD_optimal_t));
}
+ if (ZSTD_rowMatchFinderUsed(cParams->strategy, useRowMatchFinder)) {
+ { /* Row match finder needs an additional table of hashes ("tags") */
+ size_t const tagTableSize = hSize*sizeof(U16);
+ ms->tagTable = (U16*)ZSTD_cwksp_reserve_aligned(ws, tagTableSize);
+ if (ms->tagTable) ZSTD_memset(ms->tagTable, 0, tagTableSize);
+ }
+ { /* Switch to 32-entry rows if searchLog is 5 (or more) */
+ U32 const rowLog = BOUNDED(4, cParams->searchLog, 6);
+ assert(cParams->hashLog >= rowLog);
+ ms->rowHashLog = cParams->hashLog - rowLog;
+ }
+ }
+
ms->cParams = *cParams;
RETURN_ERROR_IF(ZSTD_cwksp_reserve_failed(ws), memory_allocation,
"failed a workspace allocation in ZSTD_reset_matchState");
-
return 0;
}
return (size_t)(w.nextSrc - w.base) > (ZSTD_CURRENT_MAX - ZSTD_INDEXOVERFLOW_MARGIN);
}
+/* ZSTD_dictTooBig():
+ * When dictionaries are larger than ZSTD_CHUNKSIZE_MAX they can't be loaded in
+ * one go generically. So we ensure that in that case we reset the tables to zero,
+ * so that we can load as much of the dictionary as possible.
+ */
+static int ZSTD_dictTooBig(size_t const loadedDictSize)
+{
+ return loadedDictSize > ZSTD_CHUNKSIZE_MAX;
+}
+
/*! ZSTD_resetCCtx_internal() :
- note : `params` are assumed fully validated at this stage */
+ * @param loadedDictSize The size of the dictionary to be loaded
+ * into the context, if any. If no dictionary is used, or the
+ * dictionary is being attached / copied, then pass 0.
+ * note : `params` are assumed fully validated at this stage.
+ */
static size_t ZSTD_resetCCtx_internal(ZSTD_CCtx* zc,
- ZSTD_CCtx_params params,
+ ZSTD_CCtx_params const* params,
U64 const pledgedSrcSize,
+ size_t const loadedDictSize,
ZSTD_compResetPolicy_e const crp,
ZSTD_buffered_policy_e const zbuff)
{
ZSTD_cwksp* const ws = &zc->workspace;
- DEBUGLOG(4, "ZSTD_resetCCtx_internal: pledgedSrcSize=%u, wlog=%u",
- (U32)pledgedSrcSize, params.cParams.windowLog);
- assert(!ZSTD_isError(ZSTD_checkCParams(params.cParams)));
+ DEBUGLOG(4, "ZSTD_resetCCtx_internal: pledgedSrcSize=%u, wlog=%u, useRowMatchFinder=%d useBlockSplitter=%d",
+ (U32)pledgedSrcSize, params->cParams.windowLog, (int)params->useRowMatchFinder, (int)params->useBlockSplitter);
+ assert(!ZSTD_isError(ZSTD_checkCParams(params->cParams)));
zc->isFirstBlock = 1;
- if (params.ldmParams.enableLdm) {
+ /* Set applied params early so we can modify them for LDM,
+ * and point params at the applied params.
+ */
+ zc->appliedParams = *params;
+ params = &zc->appliedParams;
+
+ assert(params->useRowMatchFinder != ZSTD_ps_auto);
+ assert(params->useBlockSplitter != ZSTD_ps_auto);
+ assert(params->ldmParams.enableLdm != ZSTD_ps_auto);
+ if (params->ldmParams.enableLdm == ZSTD_ps_enable) {
/* Adjust long distance matching parameters */
- ZSTD_ldm_adjustParameters(¶ms.ldmParams, ¶ms.cParams);
- assert(params.ldmParams.hashLog >= params.ldmParams.bucketSizeLog);
- assert(params.ldmParams.hashRateLog < 32);
+ ZSTD_ldm_adjustParameters(&zc->appliedParams.ldmParams, ¶ms->cParams);
+ assert(params->ldmParams.hashLog >= params->ldmParams.bucketSizeLog);
+ assert(params->ldmParams.hashRateLog < 32);
}
- { size_t const windowSize = MAX(1, (size_t)MIN(((U64)1 << params.cParams.windowLog), pledgedSrcSize));
+ { size_t const windowSize = MAX(1, (size_t)MIN(((U64)1 << params->cParams.windowLog), pledgedSrcSize));
size_t const blockSize = MIN(ZSTD_BLOCKSIZE_MAX, windowSize);
- U32 const divider = (params.cParams.minMatch==3) ? 3 : 4;
+ U32 const divider = (params->cParams.minMatch==3) ? 3 : 4;
size_t const maxNbSeq = blockSize / divider;
- size_t const buffOutSize = (zbuff == ZSTDb_buffered && params.outBufferMode == ZSTD_bm_buffered)
+ size_t const buffOutSize = (zbuff == ZSTDb_buffered && params->outBufferMode == ZSTD_bm_buffered)
? ZSTD_compressBound(blockSize) + 1
: 0;
- size_t const buffInSize = (zbuff == ZSTDb_buffered && params.inBufferMode == ZSTD_bm_buffered)
+ size_t const buffInSize = (zbuff == ZSTDb_buffered && params->inBufferMode == ZSTD_bm_buffered)
? windowSize + blockSize
: 0;
- size_t const maxNbLdmSeq = ZSTD_ldm_getMaxNbSeq(params.ldmParams, blockSize);
+ size_t const maxNbLdmSeq = ZSTD_ldm_getMaxNbSeq(params->ldmParams, blockSize);
int const indexTooClose = ZSTD_indexTooCloseToMax(zc->blockState.matchState.window);
+ int const dictTooBig = ZSTD_dictTooBig(loadedDictSize);
ZSTD_indexResetPolicy_e needsIndexReset =
- (!indexTooClose && zc->initialized) ? ZSTDirp_continue : ZSTDirp_reset;
+ (indexTooClose || dictTooBig || !zc->initialized) ? ZSTDirp_reset : ZSTDirp_continue;
size_t const neededSpace =
ZSTD_estimateCCtxSize_usingCCtxParams_internal(
- ¶ms.cParams, ¶ms.ldmParams, zc->staticSize != 0,
+ ¶ms->cParams, ¶ms->ldmParams, zc->staticSize != 0, params->useRowMatchFinder,
buffInSize, buffOutSize, pledgedSrcSize);
+ int resizeWorkspace;
+
FORWARD_IF_ERROR(neededSpace, "cctx size estimate failed!");
if (!zc->staticSize) ZSTD_cwksp_bump_oversized_duration(ws, 0);
- /* Check if workspace is large enough, alloc a new one if needed */
- {
+ { /* Check if workspace is large enough, alloc a new one if needed */
int const workspaceTooSmall = ZSTD_cwksp_sizeof(ws) < neededSpace;
int const workspaceWasteful = ZSTD_cwksp_check_wasteful(ws, neededSpace);
-
+ resizeWorkspace = workspaceTooSmall || workspaceWasteful;
DEBUGLOG(4, "Need %zu B workspace", neededSpace);
DEBUGLOG(4, "windowSize: %zu - blockSize: %zu", windowSize, blockSize);
- if (workspaceTooSmall || workspaceWasteful) {
+ if (resizeWorkspace) {
DEBUGLOG(4, "Resize workspaceSize from %zuKB to %zuKB",
ZSTD_cwksp_sizeof(ws) >> 10,
neededSpace >> 10);
zc->blockState.nextCBlock = (ZSTD_compressedBlockState_t*) ZSTD_cwksp_reserve_object(ws, sizeof(ZSTD_compressedBlockState_t));
RETURN_ERROR_IF(zc->blockState.nextCBlock == NULL, memory_allocation, "couldn't allocate nextCBlock");
zc->entropyWorkspace = (U32*) ZSTD_cwksp_reserve_object(ws, ENTROPY_WORKSPACE_SIZE);
- RETURN_ERROR_IF(zc->blockState.nextCBlock == NULL, memory_allocation, "couldn't allocate entropyWorkspace");
+ RETURN_ERROR_IF(zc->entropyWorkspace == NULL, memory_allocation, "couldn't allocate entropyWorkspace");
} }
ZSTD_cwksp_clear(ws);
/* init params */
- zc->appliedParams = params;
- zc->blockState.matchState.cParams = params.cParams;
+ zc->blockState.matchState.cParams = params->cParams;
zc->pledgedSrcSizePlusOne = pledgedSrcSize+1;
zc->consumedSrcSize = 0;
zc->producedCSize = 0;
zc->outBuff = (char*)ZSTD_cwksp_reserve_buffer(ws, buffOutSize);
/* ldm bucketOffsets table */
- if (params.ldmParams.enableLdm) {
+ if (params->ldmParams.enableLdm == ZSTD_ps_enable) {
/* TODO: avoid memset? */
size_t const numBuckets =
- ((size_t)1) << (params.ldmParams.hashLog -
- params.ldmParams.bucketSizeLog);
+ ((size_t)1) << (params->ldmParams.hashLog -
+ params->ldmParams.bucketSizeLog);
zc->ldmState.bucketOffsets = ZSTD_cwksp_reserve_buffer(ws, numBuckets);
ZSTD_memset(zc->ldmState.bucketOffsets, 0, numBuckets);
}
FORWARD_IF_ERROR(ZSTD_reset_matchState(
&zc->blockState.matchState,
ws,
- ¶ms.cParams,
+ ¶ms->cParams,
+ params->useRowMatchFinder,
crp,
needsIndexReset,
ZSTD_resetTarget_CCtx), "");
/* ldm hash table */
- if (params.ldmParams.enableLdm) {
+ if (params->ldmParams.enableLdm == ZSTD_ps_enable) {
/* TODO: avoid memset? */
- size_t const ldmHSize = ((size_t)1) << params.ldmParams.hashLog;
+ size_t const ldmHSize = ((size_t)1) << params->ldmParams.hashLog;
zc->ldmState.hashTable = (ldmEntry_t*)ZSTD_cwksp_reserve_aligned(ws, ldmHSize * sizeof(ldmEntry_t));
ZSTD_memset(zc->ldmState.hashTable, 0, ldmHSize * sizeof(ldmEntry_t));
zc->ldmSequences = (rawSeq*)ZSTD_cwksp_reserve_aligned(ws, maxNbLdmSeq * sizeof(rawSeq));
zc->maxNbLdmSequences = maxNbLdmSeq;
ZSTD_window_init(&zc->ldmState.window);
- ZSTD_window_clear(&zc->ldmState.window);
zc->ldmState.loadedDictEnd = 0;
}
- /* Due to alignment, when reusing a workspace, we can actually consume
- * up to 3 extra bytes for alignment. See the comments in zstd_cwksp.h
- */
- assert(ZSTD_cwksp_used(ws) >= neededSpace &&
- ZSTD_cwksp_used(ws) <= neededSpace + 3);
-
DEBUGLOG(3, "wksp: finished allocating, %zd bytes remain available", ZSTD_cwksp_available_space(ws));
+ assert(ZSTD_cwksp_estimated_space_within_bounds(ws, neededSpace, resizeWorkspace));
+
zc->initialized = 1;
return 0;
U64 pledgedSrcSize,
ZSTD_buffered_policy_e zbuff)
{
+ DEBUGLOG(4, "ZSTD_resetCCtx_byAttachingCDict() pledgedSrcSize=%llu",
+ (unsigned long long)pledgedSrcSize);
{
ZSTD_compressionParameters adjusted_cdict_cParams = cdict->matchState.cParams;
unsigned const windowLog = params.cParams.windowLog;
params.cParams = ZSTD_adjustCParams_internal(adjusted_cdict_cParams, pledgedSrcSize,
cdict->dictContentSize, ZSTD_cpm_attachDict);
params.cParams.windowLog = windowLog;
- FORWARD_IF_ERROR(ZSTD_resetCCtx_internal(cctx, params, pledgedSrcSize,
+ params.useRowMatchFinder = cdict->useRowMatchFinder; /* cdict overrides */
+ FORWARD_IF_ERROR(ZSTD_resetCCtx_internal(cctx, ¶ms, pledgedSrcSize,
+ /* loadedDictSize */ 0,
ZSTDcrp_makeClean, zbuff), "");
assert(cctx->appliedParams.cParams.strategy == adjusted_cdict_cParams.strategy);
}
const ZSTD_compressionParameters *cdict_cParams = &cdict->matchState.cParams;
assert(!cdict->matchState.dedicatedDictSearch);
-
- DEBUGLOG(4, "copying dictionary into context");
+ DEBUGLOG(4, "ZSTD_resetCCtx_byCopyingCDict() pledgedSrcSize=%llu",
+ (unsigned long long)pledgedSrcSize);
{ unsigned const windowLog = params.cParams.windowLog;
assert(windowLog != 0);
/* Copy only compression parameters related to tables. */
params.cParams = *cdict_cParams;
params.cParams.windowLog = windowLog;
- FORWARD_IF_ERROR(ZSTD_resetCCtx_internal(cctx, params, pledgedSrcSize,
+ params.useRowMatchFinder = cdict->useRowMatchFinder;
+ FORWARD_IF_ERROR(ZSTD_resetCCtx_internal(cctx, ¶ms, pledgedSrcSize,
+ /* loadedDictSize */ 0,
ZSTDcrp_leaveDirty, zbuff), "");
assert(cctx->appliedParams.cParams.strategy == cdict_cParams->strategy);
assert(cctx->appliedParams.cParams.hashLog == cdict_cParams->hashLog);
}
ZSTD_cwksp_mark_tables_dirty(&cctx->workspace);
+ assert(params.useRowMatchFinder != ZSTD_ps_auto);
/* copy tables */
- { size_t const chainSize = (cdict_cParams->strategy == ZSTD_fast) ? 0 : ((size_t)1 << cdict_cParams->chainLog);
+ { size_t const chainSize = ZSTD_allocateChainTable(cdict_cParams->strategy, cdict->useRowMatchFinder, 0 /* DDS guaranteed disabled */)
+ ? ((size_t)1 << cdict_cParams->chainLog)
+ : 0;
size_t const hSize = (size_t)1 << cdict_cParams->hashLog;
ZSTD_memcpy(cctx->blockState.matchState.hashTable,
cdict->matchState.hashTable,
hSize * sizeof(U32));
- ZSTD_memcpy(cctx->blockState.matchState.chainTable,
+ /* Do not copy cdict's chainTable if cctx has parameters such that it would not use chainTable */
+ if (ZSTD_allocateChainTable(cctx->appliedParams.cParams.strategy, cctx->appliedParams.useRowMatchFinder, 0 /* forDDSDict */)) {
+ ZSTD_memcpy(cctx->blockState.matchState.chainTable,
cdict->matchState.chainTable,
chainSize * sizeof(U32));
+ }
+ /* copy tag table */
+ if (ZSTD_rowMatchFinderUsed(cdict_cParams->strategy, cdict->useRowMatchFinder)) {
+ size_t const tagTableSize = hSize*sizeof(U16);
+ ZSTD_memcpy(cctx->blockState.matchState.tagTable,
+ cdict->matchState.tagTable,
+ tagTableSize);
+ }
}
/* Zero the hashTable3, since the cdict never fills it */
U64 pledgedSrcSize,
ZSTD_buffered_policy_e zbuff)
{
- DEBUGLOG(5, "ZSTD_copyCCtx_internal");
RETURN_ERROR_IF(srcCCtx->stage!=ZSTDcs_init, stage_wrong,
"Can't copy a ctx that's not in init stage.");
-
+ DEBUGLOG(5, "ZSTD_copyCCtx_internal");
ZSTD_memcpy(&dstCCtx->customMem, &srcCCtx->customMem, sizeof(ZSTD_customMem));
{ ZSTD_CCtx_params params = dstCCtx->requestedParams;
/* Copy only compression parameters related to tables. */
params.cParams = srcCCtx->appliedParams.cParams;
+ assert(srcCCtx->appliedParams.useRowMatchFinder != ZSTD_ps_auto);
+ assert(srcCCtx->appliedParams.useBlockSplitter != ZSTD_ps_auto);
+ assert(srcCCtx->appliedParams.ldmParams.enableLdm != ZSTD_ps_auto);
+ params.useRowMatchFinder = srcCCtx->appliedParams.useRowMatchFinder;
+ params.useBlockSplitter = srcCCtx->appliedParams.useBlockSplitter;
+ params.ldmParams = srcCCtx->appliedParams.ldmParams;
params.fParams = fParams;
- ZSTD_resetCCtx_internal(dstCCtx, params, pledgedSrcSize,
+ ZSTD_resetCCtx_internal(dstCCtx, ¶ms, pledgedSrcSize,
+ /* loadedDictSize */ 0,
ZSTDcrp_leaveDirty, zbuff);
assert(dstCCtx->appliedParams.cParams.windowLog == srcCCtx->appliedParams.cParams.windowLog);
assert(dstCCtx->appliedParams.cParams.strategy == srcCCtx->appliedParams.cParams.strategy);
ZSTD_cwksp_mark_tables_dirty(&dstCCtx->workspace);
/* copy tables */
- { size_t const chainSize = (srcCCtx->appliedParams.cParams.strategy == ZSTD_fast) ? 0 : ((size_t)1 << srcCCtx->appliedParams.cParams.chainLog);
+ { size_t const chainSize = ZSTD_allocateChainTable(srcCCtx->appliedParams.cParams.strategy,
+ srcCCtx->appliedParams.useRowMatchFinder,
+ 0 /* forDDSDict */)
+ ? ((size_t)1 << srcCCtx->appliedParams.cParams.chainLog)
+ : 0;
size_t const hSize = (size_t)1 << srcCCtx->appliedParams.cParams.hashLog;
int const h3log = srcCCtx->blockState.matchState.hashLog3;
size_t const h3Size = h3log ? ((size_t)1 << h3log) : 0;
int const nbRows = (int)size / ZSTD_ROWSIZE;
int cellNb = 0;
int rowNb;
+ /* Protect special index values < ZSTD_WINDOW_START_INDEX. */
+ U32 const reducerThreshold = reducerValue + ZSTD_WINDOW_START_INDEX;
assert((size & (ZSTD_ROWSIZE-1)) == 0); /* multiple of ZSTD_ROWSIZE */
assert(size < (1U<<31)); /* can be casted to int */
for (rowNb=0 ; rowNb < nbRows ; rowNb++) {
int column;
for (column=0; column<ZSTD_ROWSIZE; column++) {
- if (preserveMark) {
- U32 const adder = (table[cellNb] == ZSTD_DUBT_UNSORTED_MARK) ? reducerValue : 0;
- table[cellNb] += adder;
+ U32 newVal;
+ if (preserveMark && table[cellNb] == ZSTD_DUBT_UNSORTED_MARK) {
+ /* This write is pointless, but is required(?) for the compiler
+ * to auto-vectorize the loop. */
+ newVal = ZSTD_DUBT_UNSORTED_MARK;
+ } else if (table[cellNb] < reducerThreshold) {
+ newVal = 0;
+ } else {
+ newVal = table[cellNb] - reducerValue;
}
- if (table[cellNb] < reducerValue) table[cellNb] = 0;
- else table[cellNb] -= reducerValue;
+ table[cellNb] = newVal;
cellNb++;
} }
}
ZSTD_reduceTable(ms->hashTable, hSize, reducerValue);
}
- if (params->cParams.strategy != ZSTD_fast) {
+ if (ZSTD_allocateChainTable(params->cParams.strategy, params->useRowMatchFinder, (U32)ms->dedicatedDictSearch)) {
U32 const chainSize = (U32)1 << params->cParams.chainLog;
if (params->cParams.strategy == ZSTD_btlazy2)
ZSTD_reduceTable_btlazy2(ms->chainTable, chainSize, reducerValue);
assert(nbSeq <= seqStorePtr->maxNbSeq);
for (u=0; u<nbSeq; u++) {
U32 const llv = sequences[u].litLength;
- U32 const mlv = sequences[u].matchLength;
+ U32 const mlv = sequences[u].mlBase;
llCodeTable[u] = (BYTE)ZSTD_LLcode(llv);
- ofCodeTable[u] = (BYTE)ZSTD_highbit32(sequences[u].offset);
+ ofCodeTable[u] = (BYTE)ZSTD_highbit32(sequences[u].offBase);
mlCodeTable[u] = (BYTE)ZSTD_MLcode(mlv);
}
- if (seqStorePtr->longLengthID==1)
+ if (seqStorePtr->longLengthType==ZSTD_llt_literalLength)
llCodeTable[seqStorePtr->longLengthPos] = MaxLL;
- if (seqStorePtr->longLengthID==2)
+ if (seqStorePtr->longLengthType==ZSTD_llt_matchLength)
mlCodeTable[seqStorePtr->longLengthPos] = MaxML;
}
return (cctxParams->targetCBlockSize != 0);
}
-/* ZSTD_entropyCompressSequences_internal():
- * actually compresses both literals and sequences */
+/* ZSTD_blockSplitterEnabled():
+ * Returns if block splitting param is being used
+ * If used, compression will do best effort to split a block in order to improve compression ratio.
+ * At the time this function is called, the parameter must be finalized.
+ * Returns 1 if true, 0 otherwise. */
+static int ZSTD_blockSplitterEnabled(ZSTD_CCtx_params* cctxParams)
+{
+ DEBUGLOG(5, "ZSTD_blockSplitterEnabled (useBlockSplitter=%d)", cctxParams->useBlockSplitter);
+ assert(cctxParams->useBlockSplitter != ZSTD_ps_auto);
+ return (cctxParams->useBlockSplitter == ZSTD_ps_enable);
+}
+
+/* Type returned by ZSTD_buildSequencesStatistics containing finalized symbol encoding types
+ * and size of the sequences statistics
+ */
+typedef struct {
+ U32 LLtype;
+ U32 Offtype;
+ U32 MLtype;
+ size_t size;
+ size_t lastCountSize; /* Accounts for bug in 1.3.4. More detail in ZSTD_entropyCompressSeqStore_internal() */
+} ZSTD_symbolEncodingTypeStats_t;
+
+/* ZSTD_buildSequencesStatistics():
+ * Returns a ZSTD_symbolEncodingTypeStats_t, or a zstd error code in the `size` field.
+ * Modifies `nextEntropy` to have the appropriate values as a side effect.
+ * nbSeq must be greater than 0.
+ *
+ * entropyWkspSize must be of size at least ENTROPY_WORKSPACE_SIZE - (MaxSeq + 1)*sizeof(U32)
+ */
+static ZSTD_symbolEncodingTypeStats_t
+ZSTD_buildSequencesStatistics(seqStore_t* seqStorePtr, size_t nbSeq,
+ const ZSTD_fseCTables_t* prevEntropy, ZSTD_fseCTables_t* nextEntropy,
+ BYTE* dst, const BYTE* const dstEnd,
+ ZSTD_strategy strategy, unsigned* countWorkspace,
+ void* entropyWorkspace, size_t entropyWkspSize) {
+ BYTE* const ostart = dst;
+ const BYTE* const oend = dstEnd;
+ BYTE* op = ostart;
+ FSE_CTable* CTable_LitLength = nextEntropy->litlengthCTable;
+ FSE_CTable* CTable_OffsetBits = nextEntropy->offcodeCTable;
+ FSE_CTable* CTable_MatchLength = nextEntropy->matchlengthCTable;
+ const BYTE* const ofCodeTable = seqStorePtr->ofCode;
+ const BYTE* const llCodeTable = seqStorePtr->llCode;
+ const BYTE* const mlCodeTable = seqStorePtr->mlCode;
+ ZSTD_symbolEncodingTypeStats_t stats;
+
+ stats.lastCountSize = 0;
+ /* convert length/distances into codes */
+ ZSTD_seqToCodes(seqStorePtr);
+ assert(op <= oend);
+ assert(nbSeq != 0); /* ZSTD_selectEncodingType() divides by nbSeq */
+ /* build CTable for Literal Lengths */
+ { unsigned max = MaxLL;
+ size_t const mostFrequent = HIST_countFast_wksp(countWorkspace, &max, llCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */
+ DEBUGLOG(5, "Building LL table");
+ nextEntropy->litlength_repeatMode = prevEntropy->litlength_repeatMode;
+ stats.LLtype = ZSTD_selectEncodingType(&nextEntropy->litlength_repeatMode,
+ countWorkspace, max, mostFrequent, nbSeq,
+ LLFSELog, prevEntropy->litlengthCTable,
+ LL_defaultNorm, LL_defaultNormLog,
+ ZSTD_defaultAllowed, strategy);
+ assert(set_basic < set_compressed && set_rle < set_compressed);
+ assert(!(stats.LLtype < set_compressed && nextEntropy->litlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */
+ { size_t const countSize = ZSTD_buildCTable(
+ op, (size_t)(oend - op),
+ CTable_LitLength, LLFSELog, (symbolEncodingType_e)stats.LLtype,
+ countWorkspace, max, llCodeTable, nbSeq,
+ LL_defaultNorm, LL_defaultNormLog, MaxLL,
+ prevEntropy->litlengthCTable,
+ sizeof(prevEntropy->litlengthCTable),
+ entropyWorkspace, entropyWkspSize);
+ if (ZSTD_isError(countSize)) {
+ DEBUGLOG(3, "ZSTD_buildCTable for LitLens failed");
+ stats.size = countSize;
+ return stats;
+ }
+ if (stats.LLtype == set_compressed)
+ stats.lastCountSize = countSize;
+ op += countSize;
+ assert(op <= oend);
+ } }
+ /* build CTable for Offsets */
+ { unsigned max = MaxOff;
+ size_t const mostFrequent = HIST_countFast_wksp(
+ countWorkspace, &max, ofCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */
+ /* We can only use the basic table if max <= DefaultMaxOff, otherwise the offsets are too large */
+ ZSTD_defaultPolicy_e const defaultPolicy = (max <= DefaultMaxOff) ? ZSTD_defaultAllowed : ZSTD_defaultDisallowed;
+ DEBUGLOG(5, "Building OF table");
+ nextEntropy->offcode_repeatMode = prevEntropy->offcode_repeatMode;
+ stats.Offtype = ZSTD_selectEncodingType(&nextEntropy->offcode_repeatMode,
+ countWorkspace, max, mostFrequent, nbSeq,
+ OffFSELog, prevEntropy->offcodeCTable,
+ OF_defaultNorm, OF_defaultNormLog,
+ defaultPolicy, strategy);
+ assert(!(stats.Offtype < set_compressed && nextEntropy->offcode_repeatMode != FSE_repeat_none)); /* We don't copy tables */
+ { size_t const countSize = ZSTD_buildCTable(
+ op, (size_t)(oend - op),
+ CTable_OffsetBits, OffFSELog, (symbolEncodingType_e)stats.Offtype,
+ countWorkspace, max, ofCodeTable, nbSeq,
+ OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff,
+ prevEntropy->offcodeCTable,
+ sizeof(prevEntropy->offcodeCTable),
+ entropyWorkspace, entropyWkspSize);
+ if (ZSTD_isError(countSize)) {
+ DEBUGLOG(3, "ZSTD_buildCTable for Offsets failed");
+ stats.size = countSize;
+ return stats;
+ }
+ if (stats.Offtype == set_compressed)
+ stats.lastCountSize = countSize;
+ op += countSize;
+ assert(op <= oend);
+ } }
+ /* build CTable for MatchLengths */
+ { unsigned max = MaxML;
+ size_t const mostFrequent = HIST_countFast_wksp(
+ countWorkspace, &max, mlCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */
+ DEBUGLOG(5, "Building ML table (remaining space : %i)", (int)(oend-op));
+ nextEntropy->matchlength_repeatMode = prevEntropy->matchlength_repeatMode;
+ stats.MLtype = ZSTD_selectEncodingType(&nextEntropy->matchlength_repeatMode,
+ countWorkspace, max, mostFrequent, nbSeq,
+ MLFSELog, prevEntropy->matchlengthCTable,
+ ML_defaultNorm, ML_defaultNormLog,
+ ZSTD_defaultAllowed, strategy);
+ assert(!(stats.MLtype < set_compressed && nextEntropy->matchlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */
+ { size_t const countSize = ZSTD_buildCTable(
+ op, (size_t)(oend - op),
+ CTable_MatchLength, MLFSELog, (symbolEncodingType_e)stats.MLtype,
+ countWorkspace, max, mlCodeTable, nbSeq,
+ ML_defaultNorm, ML_defaultNormLog, MaxML,
+ prevEntropy->matchlengthCTable,
+ sizeof(prevEntropy->matchlengthCTable),
+ entropyWorkspace, entropyWkspSize);
+ if (ZSTD_isError(countSize)) {
+ DEBUGLOG(3, "ZSTD_buildCTable for MatchLengths failed");
+ stats.size = countSize;
+ return stats;
+ }
+ if (stats.MLtype == set_compressed)
+ stats.lastCountSize = countSize;
+ op += countSize;
+ assert(op <= oend);
+ } }
+ stats.size = (size_t)(op-ostart);
+ return stats;
+}
+
+/* ZSTD_entropyCompressSeqStore_internal():
+ * compresses both literals and sequences
+ * Returns compressed size of block, or a zstd error.
+ */
+#define SUSPECT_UNCOMPRESSIBLE_LITERAL_RATIO 20
MEM_STATIC size_t
-ZSTD_entropyCompressSequences_internal(seqStore_t* seqStorePtr,
+ZSTD_entropyCompressSeqStore_internal(seqStore_t* seqStorePtr,
const ZSTD_entropyCTables_t* prevEntropy,
ZSTD_entropyCTables_t* nextEntropy,
const ZSTD_CCtx_params* cctxParams,
FSE_CTable* CTable_LitLength = nextEntropy->fse.litlengthCTable;
FSE_CTable* CTable_OffsetBits = nextEntropy->fse.offcodeCTable;
FSE_CTable* CTable_MatchLength = nextEntropy->fse.matchlengthCTable;
- U32 LLtype, Offtype, MLtype; /* compressed, raw or rle */
const seqDef* const sequences = seqStorePtr->sequencesStart;
+ const size_t nbSeq = seqStorePtr->sequences - seqStorePtr->sequencesStart;
const BYTE* const ofCodeTable = seqStorePtr->ofCode;
const BYTE* const llCodeTable = seqStorePtr->llCode;
const BYTE* const mlCodeTable = seqStorePtr->mlCode;
BYTE* const ostart = (BYTE*)dst;
BYTE* const oend = ostart + dstCapacity;
BYTE* op = ostart;
- size_t const nbSeq = (size_t)(seqStorePtr->sequences - seqStorePtr->sequencesStart);
- BYTE* seqHead;
- BYTE* lastNCount = NULL;
+ size_t lastCountSize;
entropyWorkspace = count + (MaxSeq + 1);
entropyWkspSize -= (MaxSeq + 1) * sizeof(*count);
- DEBUGLOG(4, "ZSTD_entropyCompressSequences_internal (nbSeq=%zu)", nbSeq);
+ DEBUGLOG(4, "ZSTD_entropyCompressSeqStore_internal (nbSeq=%zu)", nbSeq);
ZSTD_STATIC_ASSERT(HUF_WORKSPACE_SIZE >= (1<<MAX(MLFSELog,LLFSELog)));
assert(entropyWkspSize >= HUF_WORKSPACE_SIZE);
/* Compress literals */
{ const BYTE* const literals = seqStorePtr->litStart;
+ size_t const numSequences = seqStorePtr->sequences - seqStorePtr->sequencesStart;
+ size_t const numLiterals = seqStorePtr->lit - seqStorePtr->litStart;
+ /* Base suspicion of uncompressibility on ratio of literals to sequences */
+ unsigned const suspectUncompressible = (numSequences == 0) || (numLiterals / numSequences >= SUSPECT_UNCOMPRESSIBLE_LITERAL_RATIO);
size_t const litSize = (size_t)(seqStorePtr->lit - literals);
size_t const cSize = ZSTD_compressLiterals(
&prevEntropy->huf, &nextEntropy->huf,
cctxParams->cParams.strategy,
- ZSTD_disableLiteralsCompression(cctxParams),
+ ZSTD_literalsCompressionIsDisabled(cctxParams),
op, dstCapacity,
literals, litSize,
entropyWorkspace, entropyWkspSize,
- bmi2);
+ bmi2, suspectUncompressible);
FORWARD_IF_ERROR(cSize, "ZSTD_compressLiterals failed");
assert(cSize <= dstCapacity);
op += cSize;
ZSTD_memcpy(&nextEntropy->fse, &prevEntropy->fse, sizeof(prevEntropy->fse));
return (size_t)(op - ostart);
}
-
- /* seqHead : flags for FSE encoding type */
- seqHead = op++;
- assert(op <= oend);
-
- /* convert length/distances into codes */
- ZSTD_seqToCodes(seqStorePtr);
- /* build CTable for Literal Lengths */
- { unsigned max = MaxLL;
- size_t const mostFrequent = HIST_countFast_wksp(count, &max, llCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */
- DEBUGLOG(5, "Building LL table");
- nextEntropy->fse.litlength_repeatMode = prevEntropy->fse.litlength_repeatMode;
- LLtype = ZSTD_selectEncodingType(&nextEntropy->fse.litlength_repeatMode,
- count, max, mostFrequent, nbSeq,
- LLFSELog, prevEntropy->fse.litlengthCTable,
- LL_defaultNorm, LL_defaultNormLog,
- ZSTD_defaultAllowed, strategy);
- assert(set_basic < set_compressed && set_rle < set_compressed);
- assert(!(LLtype < set_compressed && nextEntropy->fse.litlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */
- { size_t const countSize = ZSTD_buildCTable(
- op, (size_t)(oend - op),
- CTable_LitLength, LLFSELog, (symbolEncodingType_e)LLtype,
- count, max, llCodeTable, nbSeq,
- LL_defaultNorm, LL_defaultNormLog, MaxLL,
- prevEntropy->fse.litlengthCTable,
- sizeof(prevEntropy->fse.litlengthCTable),
- entropyWorkspace, entropyWkspSize);
- FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for LitLens failed");
- if (LLtype == set_compressed)
- lastNCount = op;
- op += countSize;
- assert(op <= oend);
- } }
- /* build CTable for Offsets */
- { unsigned max = MaxOff;
- size_t const mostFrequent = HIST_countFast_wksp(
- count, &max, ofCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */
- /* We can only use the basic table if max <= DefaultMaxOff, otherwise the offsets are too large */
- ZSTD_defaultPolicy_e const defaultPolicy = (max <= DefaultMaxOff) ? ZSTD_defaultAllowed : ZSTD_defaultDisallowed;
- DEBUGLOG(5, "Building OF table");
- nextEntropy->fse.offcode_repeatMode = prevEntropy->fse.offcode_repeatMode;
- Offtype = ZSTD_selectEncodingType(&nextEntropy->fse.offcode_repeatMode,
- count, max, mostFrequent, nbSeq,
- OffFSELog, prevEntropy->fse.offcodeCTable,
- OF_defaultNorm, OF_defaultNormLog,
- defaultPolicy, strategy);
- assert(!(Offtype < set_compressed && nextEntropy->fse.offcode_repeatMode != FSE_repeat_none)); /* We don't copy tables */
- { size_t const countSize = ZSTD_buildCTable(
- op, (size_t)(oend - op),
- CTable_OffsetBits, OffFSELog, (symbolEncodingType_e)Offtype,
- count, max, ofCodeTable, nbSeq,
- OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff,
- prevEntropy->fse.offcodeCTable,
- sizeof(prevEntropy->fse.offcodeCTable),
- entropyWorkspace, entropyWkspSize);
- FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for Offsets failed");
- if (Offtype == set_compressed)
- lastNCount = op;
- op += countSize;
- assert(op <= oend);
- } }
- /* build CTable for MatchLengths */
- { unsigned max = MaxML;
- size_t const mostFrequent = HIST_countFast_wksp(
- count, &max, mlCodeTable, nbSeq, entropyWorkspace, entropyWkspSize); /* can't fail */
- DEBUGLOG(5, "Building ML table (remaining space : %i)", (int)(oend-op));
- nextEntropy->fse.matchlength_repeatMode = prevEntropy->fse.matchlength_repeatMode;
- MLtype = ZSTD_selectEncodingType(&nextEntropy->fse.matchlength_repeatMode,
- count, max, mostFrequent, nbSeq,
- MLFSELog, prevEntropy->fse.matchlengthCTable,
- ML_defaultNorm, ML_defaultNormLog,
- ZSTD_defaultAllowed, strategy);
- assert(!(MLtype < set_compressed && nextEntropy->fse.matchlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */
- { size_t const countSize = ZSTD_buildCTable(
- op, (size_t)(oend - op),
- CTable_MatchLength, MLFSELog, (symbolEncodingType_e)MLtype,
- count, max, mlCodeTable, nbSeq,
- ML_defaultNorm, ML_defaultNormLog, MaxML,
- prevEntropy->fse.matchlengthCTable,
- sizeof(prevEntropy->fse.matchlengthCTable),
- entropyWorkspace, entropyWkspSize);
- FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for MatchLengths failed");
- if (MLtype == set_compressed)
- lastNCount = op;
- op += countSize;
- assert(op <= oend);
- } }
-
- *seqHead = (BYTE)((LLtype<<6) + (Offtype<<4) + (MLtype<<2));
+ {
+ ZSTD_symbolEncodingTypeStats_t stats;
+ BYTE* seqHead = op++;
+ /* build stats for sequences */
+ stats = ZSTD_buildSequencesStatistics(seqStorePtr, nbSeq,
+ &prevEntropy->fse, &nextEntropy->fse,
+ op, oend,
+ strategy, count,
+ entropyWorkspace, entropyWkspSize);
+ FORWARD_IF_ERROR(stats.size, "ZSTD_buildSequencesStatistics failed!");
+ *seqHead = (BYTE)((stats.LLtype<<6) + (stats.Offtype<<4) + (stats.MLtype<<2));
+ lastCountSize = stats.lastCountSize;
+ op += stats.size;
+ }
{ size_t const bitstreamSize = ZSTD_encodeSequences(
op, (size_t)(oend - op),
* In this exceedingly rare case, we will simply emit an uncompressed
* block, since it isn't worth optimizing.
*/
- if (lastNCount && (op - lastNCount) < 4) {
- /* NCountSize >= 2 && bitstreamSize > 0 ==> lastCountSize == 3 */
- assert(op - lastNCount == 3);
+ if (lastCountSize && (lastCountSize + bitstreamSize) < 4) {
+ /* lastCountSize >= 2 && bitstreamSize > 0 ==> lastCountSize == 3 */
+ assert(lastCountSize + bitstreamSize == 3);
DEBUGLOG(5, "Avoiding bug in zstd decoder in versions <= 1.3.4 by "
"emitting an uncompressed block.");
return 0;
}
MEM_STATIC size_t
-ZSTD_entropyCompressSequences(seqStore_t* seqStorePtr,
+ZSTD_entropyCompressSeqStore(seqStore_t* seqStorePtr,
const ZSTD_entropyCTables_t* prevEntropy,
ZSTD_entropyCTables_t* nextEntropy,
const ZSTD_CCtx_params* cctxParams,
void* entropyWorkspace, size_t entropyWkspSize,
int bmi2)
{
- size_t const cSize = ZSTD_entropyCompressSequences_internal(
+ size_t const cSize = ZSTD_entropyCompressSeqStore_internal(
seqStorePtr, prevEntropy, nextEntropy, cctxParams,
dst, dstCapacity,
entropyWorkspace, entropyWkspSize, bmi2);
*/
if ((cSize == ERROR(dstSize_tooSmall)) & (srcSize <= dstCapacity))
return 0; /* block not compressed */
- FORWARD_IF_ERROR(cSize, "ZSTD_entropyCompressSequences_internal failed");
+ FORWARD_IF_ERROR(cSize, "ZSTD_entropyCompressSeqStore_internal failed");
/* Check compressibility */
{ size_t const maxCSize = srcSize - ZSTD_minGain(srcSize, cctxParams->cParams.strategy);
if (cSize >= maxCSize) return 0; /* block not compressed */
}
- DEBUGLOG(4, "ZSTD_entropyCompressSequences() cSize: %zu\n", cSize);
+ DEBUGLOG(4, "ZSTD_entropyCompressSeqStore() cSize: %zu", cSize);
return cSize;
}
/* ZSTD_selectBlockCompressor() :
* Not static, but internal use only (used by long distance matcher)
* assumption : strat is a valid strategy */
-ZSTD_blockCompressor ZSTD_selectBlockCompressor(ZSTD_strategy strat, ZSTD_dictMode_e dictMode)
+ZSTD_blockCompressor ZSTD_selectBlockCompressor(ZSTD_strategy strat, ZSTD_paramSwitch_e useRowMatchFinder, ZSTD_dictMode_e dictMode)
{
static const ZSTD_blockCompressor blockCompressor[4][ZSTD_STRATEGY_MAX+1] = {
{ ZSTD_compressBlock_fast /* default for 0 */,
ZSTD_STATIC_ASSERT((unsigned)ZSTD_fast == 1);
assert(ZSTD_cParam_withinBounds(ZSTD_c_strategy, strat));
- selectedCompressor = blockCompressor[(int)dictMode][(int)strat];
+ DEBUGLOG(4, "Selected block compressor: dictMode=%d strat=%d rowMatchfinder=%d", (int)dictMode, (int)strat, (int)useRowMatchFinder);
+ if (ZSTD_rowMatchFinderUsed(strat, useRowMatchFinder)) {
+ static const ZSTD_blockCompressor rowBasedBlockCompressors[4][3] = {
+ { ZSTD_compressBlock_greedy_row,
+ ZSTD_compressBlock_lazy_row,
+ ZSTD_compressBlock_lazy2_row },
+ { ZSTD_compressBlock_greedy_extDict_row,
+ ZSTD_compressBlock_lazy_extDict_row,
+ ZSTD_compressBlock_lazy2_extDict_row },
+ { ZSTD_compressBlock_greedy_dictMatchState_row,
+ ZSTD_compressBlock_lazy_dictMatchState_row,
+ ZSTD_compressBlock_lazy2_dictMatchState_row },
+ { ZSTD_compressBlock_greedy_dedicatedDictSearch_row,
+ ZSTD_compressBlock_lazy_dedicatedDictSearch_row,
+ ZSTD_compressBlock_lazy2_dedicatedDictSearch_row }
+ };
+ DEBUGLOG(4, "Selecting a row-based matchfinder");
+ assert(useRowMatchFinder != ZSTD_ps_auto);
+ selectedCompressor = rowBasedBlockCompressors[(int)dictMode][(int)strat - (int)ZSTD_greedy];
+ } else {
+ selectedCompressor = blockCompressor[(int)dictMode][(int)strat];
+ }
assert(selectedCompressor != NULL);
return selectedCompressor;
}
{
ssPtr->lit = ssPtr->litStart;
ssPtr->sequences = ssPtr->sequencesStart;
- ssPtr->longLengthID = 0;
+ ssPtr->longLengthType = ZSTD_llt_none;
}
typedef enum { ZSTDbss_compress, ZSTDbss_noCompress } ZSTD_buildSeqStore_e;
zc->blockState.nextCBlock->rep[i] = zc->blockState.prevCBlock->rep[i];
}
if (zc->externSeqStore.pos < zc->externSeqStore.size) {
- assert(!zc->appliedParams.ldmParams.enableLdm);
+ assert(zc->appliedParams.ldmParams.enableLdm == ZSTD_ps_disable);
/* Updates ldmSeqStore.pos */
lastLLSize =
ZSTD_ldm_blockCompress(&zc->externSeqStore,
ms, &zc->seqStore,
zc->blockState.nextCBlock->rep,
+ zc->appliedParams.useRowMatchFinder,
src, srcSize);
assert(zc->externSeqStore.pos <= zc->externSeqStore.size);
- } else if (zc->appliedParams.ldmParams.enableLdm) {
+ } else if (zc->appliedParams.ldmParams.enableLdm == ZSTD_ps_enable) {
rawSeqStore_t ldmSeqStore = kNullRawSeqStore;
ldmSeqStore.seq = zc->ldmSequences;
ZSTD_ldm_blockCompress(&ldmSeqStore,
ms, &zc->seqStore,
zc->blockState.nextCBlock->rep,
+ zc->appliedParams.useRowMatchFinder,
src, srcSize);
assert(ldmSeqStore.pos == ldmSeqStore.size);
} else { /* not long range mode */
- ZSTD_blockCompressor const blockCompressor = ZSTD_selectBlockCompressor(zc->appliedParams.cParams.strategy, dictMode);
+ ZSTD_blockCompressor const blockCompressor = ZSTD_selectBlockCompressor(zc->appliedParams.cParams.strategy,
+ zc->appliedParams.useRowMatchFinder,
+ dictMode);
ms->ldmSeqStore = NULL;
lastLLSize = blockCompressor(ms, &zc->seqStore, zc->blockState.nextCBlock->rep, src, srcSize);
}
assert(zc->seqCollector.maxSequences >= seqStoreSeqSize + 1);
ZSTD_memcpy(updatedRepcodes.rep, zc->blockState.prevCBlock->rep, sizeof(repcodes_t));
for (i = 0; i < seqStoreSeqSize; ++i) {
- U32 rawOffset = seqStoreSeqs[i].offset - ZSTD_REP_NUM;
+ U32 rawOffset = seqStoreSeqs[i].offBase - ZSTD_REP_NUM;
outSeqs[i].litLength = seqStoreSeqs[i].litLength;
- outSeqs[i].matchLength = seqStoreSeqs[i].matchLength + MINMATCH;
+ outSeqs[i].matchLength = seqStoreSeqs[i].mlBase + MINMATCH;
outSeqs[i].rep = 0;
if (i == seqStore->longLengthPos) {
- if (seqStore->longLengthID == 1) {
+ if (seqStore->longLengthType == ZSTD_llt_literalLength) {
outSeqs[i].litLength += 0x10000;
- } else if (seqStore->longLengthID == 2) {
+ } else if (seqStore->longLengthType == ZSTD_llt_matchLength) {
outSeqs[i].matchLength += 0x10000;
}
}
- if (seqStoreSeqs[i].offset <= ZSTD_REP_NUM) {
+ if (seqStoreSeqs[i].offBase <= ZSTD_REP_NUM) {
/* Derive the correct offset corresponding to a repcode */
- outSeqs[i].rep = seqStoreSeqs[i].offset;
+ outSeqs[i].rep = seqStoreSeqs[i].offBase;
if (outSeqs[i].litLength != 0) {
rawOffset = updatedRepcodes.rep[outSeqs[i].rep - 1];
} else {
outSeqs[i].offset = rawOffset;
/* seqStoreSeqs[i].offset == offCode+1, and ZSTD_updateRep() expects offCode
so we provide seqStoreSeqs[i].offset - 1 */
- updatedRepcodes = ZSTD_updateRep(updatedRepcodes.rep,
- seqStoreSeqs[i].offset - 1,
- seqStoreSeqs[i].litLength == 0);
+ ZSTD_updateRep(updatedRepcodes.rep,
+ seqStoreSeqs[i].offBase - 1,
+ seqStoreSeqs[i].litLength == 0);
literalsRead += outSeqs[i].litLength;
}
/* Insert last literals (if any exist) in the block as a sequence with ml == off == 0.
return nbSeqs < 4 && nbLits < 10;
}
-static void ZSTD_confirmRepcodesAndEntropyTables(ZSTD_CCtx* zc)
+static void ZSTD_blockState_confirmRepcodesAndEntropyTables(ZSTD_blockState_t* const bs)
+{
+ ZSTD_compressedBlockState_t* const tmp = bs->prevCBlock;
+ bs->prevCBlock = bs->nextCBlock;
+ bs->nextCBlock = tmp;
+}
+
+/* Writes the block header */
+static void writeBlockHeader(void* op, size_t cSize, size_t blockSize, U32 lastBlock) {
+ U32 const cBlockHeader = cSize == 1 ?
+ lastBlock + (((U32)bt_rle)<<1) + (U32)(blockSize << 3) :
+ lastBlock + (((U32)bt_compressed)<<1) + (U32)(cSize << 3);
+ MEM_writeLE24(op, cBlockHeader);
+ DEBUGLOG(3, "writeBlockHeader: cSize: %zu blockSize: %zu lastBlock: %u", cSize, blockSize, lastBlock);
+}
+
+/* ZSTD_buildBlockEntropyStats_literals() :
+ * Builds entropy for the literals.
+ * Stores literals block type (raw, rle, compressed, repeat) and
+ * huffman description table to hufMetadata.
+ * Requires ENTROPY_WORKSPACE_SIZE workspace
+ * @return : size of huffman description table or error code */
+static size_t ZSTD_buildBlockEntropyStats_literals(void* const src, size_t srcSize,
+ const ZSTD_hufCTables_t* prevHuf,
+ ZSTD_hufCTables_t* nextHuf,
+ ZSTD_hufCTablesMetadata_t* hufMetadata,
+ const int literalsCompressionIsDisabled,
+ void* workspace, size_t wkspSize)
+{
+ BYTE* const wkspStart = (BYTE*)workspace;
+ BYTE* const wkspEnd = wkspStart + wkspSize;
+ BYTE* const countWkspStart = wkspStart;
+ unsigned* const countWksp = (unsigned*)workspace;
+ const size_t countWkspSize = (HUF_SYMBOLVALUE_MAX + 1) * sizeof(unsigned);
+ BYTE* const nodeWksp = countWkspStart + countWkspSize;
+ const size_t nodeWkspSize = wkspEnd-nodeWksp;
+ unsigned maxSymbolValue = HUF_SYMBOLVALUE_MAX;
+ unsigned huffLog = HUF_TABLELOG_DEFAULT;
+ HUF_repeat repeat = prevHuf->repeatMode;
+ DEBUGLOG(5, "ZSTD_buildBlockEntropyStats_literals (srcSize=%zu)", srcSize);
+
+ /* Prepare nextEntropy assuming reusing the existing table */
+ ZSTD_memcpy(nextHuf, prevHuf, sizeof(*prevHuf));
+
+ if (literalsCompressionIsDisabled) {
+ DEBUGLOG(5, "set_basic - disabled");
+ hufMetadata->hType = set_basic;
+ return 0;
+ }
+
+ /* small ? don't even attempt compression (speed opt) */
+#ifndef COMPRESS_LITERALS_SIZE_MIN
+#define COMPRESS_LITERALS_SIZE_MIN 63
+#endif
+ { size_t const minLitSize = (prevHuf->repeatMode == HUF_repeat_valid) ? 6 : COMPRESS_LITERALS_SIZE_MIN;
+ if (srcSize <= minLitSize) {
+ DEBUGLOG(5, "set_basic - too small");
+ hufMetadata->hType = set_basic;
+ return 0;
+ }
+ }
+
+ /* Scan input and build symbol stats */
+ { size_t const largest = HIST_count_wksp (countWksp, &maxSymbolValue, (const BYTE*)src, srcSize, workspace, wkspSize);
+ FORWARD_IF_ERROR(largest, "HIST_count_wksp failed");
+ if (largest == srcSize) {
+ DEBUGLOG(5, "set_rle");
+ hufMetadata->hType = set_rle;
+ return 0;
+ }
+ if (largest <= (srcSize >> 7)+4) {
+ DEBUGLOG(5, "set_basic - no gain");
+ hufMetadata->hType = set_basic;
+ return 0;
+ }
+ }
+
+ /* Validate the previous Huffman table */
+ if (repeat == HUF_repeat_check && !HUF_validateCTable((HUF_CElt const*)prevHuf->CTable, countWksp, maxSymbolValue)) {
+ repeat = HUF_repeat_none;
+ }
+
+ /* Build Huffman Tree */
+ ZSTD_memset(nextHuf->CTable, 0, sizeof(nextHuf->CTable));
+ huffLog = HUF_optimalTableLog(huffLog, srcSize, maxSymbolValue);
+ { size_t const maxBits = HUF_buildCTable_wksp((HUF_CElt*)nextHuf->CTable, countWksp,
+ maxSymbolValue, huffLog,
+ nodeWksp, nodeWkspSize);
+ FORWARD_IF_ERROR(maxBits, "HUF_buildCTable_wksp");
+ huffLog = (U32)maxBits;
+ { /* Build and write the CTable */
+ size_t const newCSize = HUF_estimateCompressedSize(
+ (HUF_CElt*)nextHuf->CTable, countWksp, maxSymbolValue);
+ size_t const hSize = HUF_writeCTable_wksp(
+ hufMetadata->hufDesBuffer, sizeof(hufMetadata->hufDesBuffer),
+ (HUF_CElt*)nextHuf->CTable, maxSymbolValue, huffLog,
+ nodeWksp, nodeWkspSize);
+ /* Check against repeating the previous CTable */
+ if (repeat != HUF_repeat_none) {
+ size_t const oldCSize = HUF_estimateCompressedSize(
+ (HUF_CElt const*)prevHuf->CTable, countWksp, maxSymbolValue);
+ if (oldCSize < srcSize && (oldCSize <= hSize + newCSize || hSize + 12 >= srcSize)) {
+ DEBUGLOG(5, "set_repeat - smaller");
+ ZSTD_memcpy(nextHuf, prevHuf, sizeof(*prevHuf));
+ hufMetadata->hType = set_repeat;
+ return 0;
+ }
+ }
+ if (newCSize + hSize >= srcSize) {
+ DEBUGLOG(5, "set_basic - no gains");
+ ZSTD_memcpy(nextHuf, prevHuf, sizeof(*prevHuf));
+ hufMetadata->hType = set_basic;
+ return 0;
+ }
+ DEBUGLOG(5, "set_compressed (hSize=%u)", (U32)hSize);
+ hufMetadata->hType = set_compressed;
+ nextHuf->repeatMode = HUF_repeat_check;
+ return hSize;
+ }
+ }
+}
+
+
+/* ZSTD_buildDummySequencesStatistics():
+ * Returns a ZSTD_symbolEncodingTypeStats_t with all encoding types as set_basic,
+ * and updates nextEntropy to the appropriate repeatMode.
+ */
+static ZSTD_symbolEncodingTypeStats_t
+ZSTD_buildDummySequencesStatistics(ZSTD_fseCTables_t* nextEntropy) {
+ ZSTD_symbolEncodingTypeStats_t stats = {set_basic, set_basic, set_basic, 0, 0};
+ nextEntropy->litlength_repeatMode = FSE_repeat_none;
+ nextEntropy->offcode_repeatMode = FSE_repeat_none;
+ nextEntropy->matchlength_repeatMode = FSE_repeat_none;
+ return stats;
+}
+
+/* ZSTD_buildBlockEntropyStats_sequences() :
+ * Builds entropy for the sequences.
+ * Stores symbol compression modes and fse table to fseMetadata.
+ * Requires ENTROPY_WORKSPACE_SIZE wksp.
+ * @return : size of fse tables or error code */
+static size_t ZSTD_buildBlockEntropyStats_sequences(seqStore_t* seqStorePtr,
+ const ZSTD_fseCTables_t* prevEntropy,
+ ZSTD_fseCTables_t* nextEntropy,
+ const ZSTD_CCtx_params* cctxParams,
+ ZSTD_fseCTablesMetadata_t* fseMetadata,
+ void* workspace, size_t wkspSize)
+{
+ ZSTD_strategy const strategy = cctxParams->cParams.strategy;
+ size_t const nbSeq = seqStorePtr->sequences - seqStorePtr->sequencesStart;
+ BYTE* const ostart = fseMetadata->fseTablesBuffer;
+ BYTE* const oend = ostart + sizeof(fseMetadata->fseTablesBuffer);
+ BYTE* op = ostart;
+ unsigned* countWorkspace = (unsigned*)workspace;
+ unsigned* entropyWorkspace = countWorkspace + (MaxSeq + 1);
+ size_t entropyWorkspaceSize = wkspSize - (MaxSeq + 1) * sizeof(*countWorkspace);
+ ZSTD_symbolEncodingTypeStats_t stats;
+
+ DEBUGLOG(5, "ZSTD_buildBlockEntropyStats_sequences (nbSeq=%zu)", nbSeq);
+ stats = nbSeq != 0 ? ZSTD_buildSequencesStatistics(seqStorePtr, nbSeq,
+ prevEntropy, nextEntropy, op, oend,
+ strategy, countWorkspace,
+ entropyWorkspace, entropyWorkspaceSize)
+ : ZSTD_buildDummySequencesStatistics(nextEntropy);
+ FORWARD_IF_ERROR(stats.size, "ZSTD_buildSequencesStatistics failed!");
+ fseMetadata->llType = (symbolEncodingType_e) stats.LLtype;
+ fseMetadata->ofType = (symbolEncodingType_e) stats.Offtype;
+ fseMetadata->mlType = (symbolEncodingType_e) stats.MLtype;
+ fseMetadata->lastCountSize = stats.lastCountSize;
+ return stats.size;
+}
+
+
+/* ZSTD_buildBlockEntropyStats() :
+ * Builds entropy for the block.
+ * Requires workspace size ENTROPY_WORKSPACE_SIZE
+ *
+ * @return : 0 on success or error code
+ */
+size_t ZSTD_buildBlockEntropyStats(seqStore_t* seqStorePtr,
+ const ZSTD_entropyCTables_t* prevEntropy,
+ ZSTD_entropyCTables_t* nextEntropy,
+ const ZSTD_CCtx_params* cctxParams,
+ ZSTD_entropyCTablesMetadata_t* entropyMetadata,
+ void* workspace, size_t wkspSize)
+{
+ size_t const litSize = seqStorePtr->lit - seqStorePtr->litStart;
+ entropyMetadata->hufMetadata.hufDesSize =
+ ZSTD_buildBlockEntropyStats_literals(seqStorePtr->litStart, litSize,
+ &prevEntropy->huf, &nextEntropy->huf,
+ &entropyMetadata->hufMetadata,
+ ZSTD_literalsCompressionIsDisabled(cctxParams),
+ workspace, wkspSize);
+ FORWARD_IF_ERROR(entropyMetadata->hufMetadata.hufDesSize, "ZSTD_buildBlockEntropyStats_literals failed");
+ entropyMetadata->fseMetadata.fseTablesSize =
+ ZSTD_buildBlockEntropyStats_sequences(seqStorePtr,
+ &prevEntropy->fse, &nextEntropy->fse,
+ cctxParams,
+ &entropyMetadata->fseMetadata,
+ workspace, wkspSize);
+ FORWARD_IF_ERROR(entropyMetadata->fseMetadata.fseTablesSize, "ZSTD_buildBlockEntropyStats_sequences failed");
+ return 0;
+}
+
+/* Returns the size estimate for the literals section (header + content) of a block */
+static size_t ZSTD_estimateBlockSize_literal(const BYTE* literals, size_t litSize,
+ const ZSTD_hufCTables_t* huf,
+ const ZSTD_hufCTablesMetadata_t* hufMetadata,
+ void* workspace, size_t wkspSize,
+ int writeEntropy)
+{
+ unsigned* const countWksp = (unsigned*)workspace;
+ unsigned maxSymbolValue = HUF_SYMBOLVALUE_MAX;
+ size_t literalSectionHeaderSize = 3 + (litSize >= 1 KB) + (litSize >= 16 KB);
+ U32 singleStream = litSize < 256;
+
+ if (hufMetadata->hType == set_basic) return litSize;
+ else if (hufMetadata->hType == set_rle) return 1;
+ else if (hufMetadata->hType == set_compressed || hufMetadata->hType == set_repeat) {
+ size_t const largest = HIST_count_wksp (countWksp, &maxSymbolValue, (const BYTE*)literals, litSize, workspace, wkspSize);
+ if (ZSTD_isError(largest)) return litSize;
+ { size_t cLitSizeEstimate = HUF_estimateCompressedSize((const HUF_CElt*)huf->CTable, countWksp, maxSymbolValue);
+ if (writeEntropy) cLitSizeEstimate += hufMetadata->hufDesSize;
+ if (!singleStream) cLitSizeEstimate += 6; /* multi-stream huffman uses 6-byte jump table */
+ return cLitSizeEstimate + literalSectionHeaderSize;
+ } }
+ assert(0); /* impossible */
+ return 0;
+}
+
+/* Returns the size estimate for the FSE-compressed symbols (of, ml, ll) of a block */
+static size_t ZSTD_estimateBlockSize_symbolType(symbolEncodingType_e type,
+ const BYTE* codeTable, size_t nbSeq, unsigned maxCode,
+ const FSE_CTable* fseCTable,
+ const U8* additionalBits,
+ short const* defaultNorm, U32 defaultNormLog, U32 defaultMax,
+ void* workspace, size_t wkspSize)
+{
+ unsigned* const countWksp = (unsigned*)workspace;
+ const BYTE* ctp = codeTable;
+ const BYTE* const ctStart = ctp;
+ const BYTE* const ctEnd = ctStart + nbSeq;
+ size_t cSymbolTypeSizeEstimateInBits = 0;
+ unsigned max = maxCode;
+
+ HIST_countFast_wksp(countWksp, &max, codeTable, nbSeq, workspace, wkspSize); /* can't fail */
+ if (type == set_basic) {
+ /* We selected this encoding type, so it must be valid. */
+ assert(max <= defaultMax);
+ (void)defaultMax;
+ cSymbolTypeSizeEstimateInBits = ZSTD_crossEntropyCost(defaultNorm, defaultNormLog, countWksp, max);
+ } else if (type == set_rle) {
+ cSymbolTypeSizeEstimateInBits = 0;
+ } else if (type == set_compressed || type == set_repeat) {
+ cSymbolTypeSizeEstimateInBits = ZSTD_fseBitCost(fseCTable, countWksp, max);
+ }
+ if (ZSTD_isError(cSymbolTypeSizeEstimateInBits)) {
+ return nbSeq * 10;
+ }
+ while (ctp < ctEnd) {
+ if (additionalBits) cSymbolTypeSizeEstimateInBits += additionalBits[*ctp];
+ else cSymbolTypeSizeEstimateInBits += *ctp; /* for offset, offset code is also the number of additional bits */
+ ctp++;
+ }
+ return cSymbolTypeSizeEstimateInBits >> 3;
+}
+
+/* Returns the size estimate for the sequences section (header + content) of a block */
+static size_t ZSTD_estimateBlockSize_sequences(const BYTE* ofCodeTable,
+ const BYTE* llCodeTable,
+ const BYTE* mlCodeTable,
+ size_t nbSeq,
+ const ZSTD_fseCTables_t* fseTables,
+ const ZSTD_fseCTablesMetadata_t* fseMetadata,
+ void* workspace, size_t wkspSize,
+ int writeEntropy)
+{
+ size_t sequencesSectionHeaderSize = 1 /* seqHead */ + 1 /* min seqSize size */ + (nbSeq >= 128) + (nbSeq >= LONGNBSEQ);
+ size_t cSeqSizeEstimate = 0;
+ cSeqSizeEstimate += ZSTD_estimateBlockSize_symbolType(fseMetadata->ofType, ofCodeTable, nbSeq, MaxOff,
+ fseTables->offcodeCTable, NULL,
+ OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff,
+ workspace, wkspSize);
+ cSeqSizeEstimate += ZSTD_estimateBlockSize_symbolType(fseMetadata->llType, llCodeTable, nbSeq, MaxLL,
+ fseTables->litlengthCTable, LL_bits,
+ LL_defaultNorm, LL_defaultNormLog, MaxLL,
+ workspace, wkspSize);
+ cSeqSizeEstimate += ZSTD_estimateBlockSize_symbolType(fseMetadata->mlType, mlCodeTable, nbSeq, MaxML,
+ fseTables->matchlengthCTable, ML_bits,
+ ML_defaultNorm, ML_defaultNormLog, MaxML,
+ workspace, wkspSize);
+ if (writeEntropy) cSeqSizeEstimate += fseMetadata->fseTablesSize;
+ return cSeqSizeEstimate + sequencesSectionHeaderSize;
+}
+
+/* Returns the size estimate for a given stream of literals, of, ll, ml */
+static size_t ZSTD_estimateBlockSize(const BYTE* literals, size_t litSize,
+ const BYTE* ofCodeTable,
+ const BYTE* llCodeTable,
+ const BYTE* mlCodeTable,
+ size_t nbSeq,
+ const ZSTD_entropyCTables_t* entropy,
+ const ZSTD_entropyCTablesMetadata_t* entropyMetadata,
+ void* workspace, size_t wkspSize,
+ int writeLitEntropy, int writeSeqEntropy) {
+ size_t const literalsSize = ZSTD_estimateBlockSize_literal(literals, litSize,
+ &entropy->huf, &entropyMetadata->hufMetadata,
+ workspace, wkspSize, writeLitEntropy);
+ size_t const seqSize = ZSTD_estimateBlockSize_sequences(ofCodeTable, llCodeTable, mlCodeTable,
+ nbSeq, &entropy->fse, &entropyMetadata->fseMetadata,
+ workspace, wkspSize, writeSeqEntropy);
+ return seqSize + literalsSize + ZSTD_blockHeaderSize;
+}
+
+/* Builds entropy statistics and uses them for blocksize estimation.
+ *
+ * Returns the estimated compressed size of the seqStore, or a zstd error.
+ */
+static size_t ZSTD_buildEntropyStatisticsAndEstimateSubBlockSize(seqStore_t* seqStore, ZSTD_CCtx* zc) {
+ ZSTD_entropyCTablesMetadata_t* entropyMetadata = &zc->blockSplitCtx.entropyMetadata;
+ DEBUGLOG(6, "ZSTD_buildEntropyStatisticsAndEstimateSubBlockSize()");
+ FORWARD_IF_ERROR(ZSTD_buildBlockEntropyStats(seqStore,
+ &zc->blockState.prevCBlock->entropy,
+ &zc->blockState.nextCBlock->entropy,
+ &zc->appliedParams,
+ entropyMetadata,
+ zc->entropyWorkspace, ENTROPY_WORKSPACE_SIZE /* statically allocated in resetCCtx */), "");
+ return ZSTD_estimateBlockSize(seqStore->litStart, (size_t)(seqStore->lit - seqStore->litStart),
+ seqStore->ofCode, seqStore->llCode, seqStore->mlCode,
+ (size_t)(seqStore->sequences - seqStore->sequencesStart),
+ &zc->blockState.nextCBlock->entropy, entropyMetadata, zc->entropyWorkspace, ENTROPY_WORKSPACE_SIZE,
+ (int)(entropyMetadata->hufMetadata.hType == set_compressed), 1);
+}
+
+/* Returns literals bytes represented in a seqStore */
+static size_t ZSTD_countSeqStoreLiteralsBytes(const seqStore_t* const seqStore) {
+ size_t literalsBytes = 0;
+ size_t const nbSeqs = seqStore->sequences - seqStore->sequencesStart;
+ size_t i;
+ for (i = 0; i < nbSeqs; ++i) {
+ seqDef seq = seqStore->sequencesStart[i];
+ literalsBytes += seq.litLength;
+ if (i == seqStore->longLengthPos && seqStore->longLengthType == ZSTD_llt_literalLength) {
+ literalsBytes += 0x10000;
+ }
+ }
+ return literalsBytes;
+}
+
+/* Returns match bytes represented in a seqStore */
+static size_t ZSTD_countSeqStoreMatchBytes(const seqStore_t* const seqStore) {
+ size_t matchBytes = 0;
+ size_t const nbSeqs = seqStore->sequences - seqStore->sequencesStart;
+ size_t i;
+ for (i = 0; i < nbSeqs; ++i) {
+ seqDef seq = seqStore->sequencesStart[i];
+ matchBytes += seq.mlBase + MINMATCH;
+ if (i == seqStore->longLengthPos && seqStore->longLengthType == ZSTD_llt_matchLength) {
+ matchBytes += 0x10000;
+ }
+ }
+ return matchBytes;
+}
+
+/* Derives the seqStore that is a chunk of the originalSeqStore from [startIdx, endIdx).
+ * Stores the result in resultSeqStore.
+ */
+static void ZSTD_deriveSeqStoreChunk(seqStore_t* resultSeqStore,
+ const seqStore_t* originalSeqStore,
+ size_t startIdx, size_t endIdx) {
+ BYTE* const litEnd = originalSeqStore->lit;
+ size_t literalsBytes;
+ size_t literalsBytesPreceding = 0;
+
+ *resultSeqStore = *originalSeqStore;
+ if (startIdx > 0) {
+ resultSeqStore->sequences = originalSeqStore->sequencesStart + startIdx;
+ literalsBytesPreceding = ZSTD_countSeqStoreLiteralsBytes(resultSeqStore);
+ }
+
+ /* Move longLengthPos into the correct position if necessary */
+ if (originalSeqStore->longLengthType != ZSTD_llt_none) {
+ if (originalSeqStore->longLengthPos < startIdx || originalSeqStore->longLengthPos > endIdx) {
+ resultSeqStore->longLengthType = ZSTD_llt_none;
+ } else {
+ resultSeqStore->longLengthPos -= (U32)startIdx;
+ }
+ }
+ resultSeqStore->sequencesStart = originalSeqStore->sequencesStart + startIdx;
+ resultSeqStore->sequences = originalSeqStore->sequencesStart + endIdx;
+ literalsBytes = ZSTD_countSeqStoreLiteralsBytes(resultSeqStore);
+ resultSeqStore->litStart += literalsBytesPreceding;
+ if (endIdx == (size_t)(originalSeqStore->sequences - originalSeqStore->sequencesStart)) {
+ /* This accounts for possible last literals if the derived chunk reaches the end of the block */
+ resultSeqStore->lit = litEnd;
+ } else {
+ resultSeqStore->lit = resultSeqStore->litStart+literalsBytes;
+ }
+ resultSeqStore->llCode += startIdx;
+ resultSeqStore->mlCode += startIdx;
+ resultSeqStore->ofCode += startIdx;
+}
+
+/*
+ * Returns the raw offset represented by the combination of offCode, ll0, and repcode history.
+ * offCode must represent a repcode in the numeric representation of ZSTD_storeSeq().
+ */
+static U32
+ZSTD_resolveRepcodeToRawOffset(const U32 rep[ZSTD_REP_NUM], const U32 offCode, const U32 ll0)
+{
+ U32 const adjustedOffCode = STORED_REPCODE(offCode) - 1 + ll0; /* [ 0 - 3 ] */
+ assert(STORED_IS_REPCODE(offCode));
+ if (adjustedOffCode == ZSTD_REP_NUM) {
+ /* litlength == 0 and offCode == 2 implies selection of first repcode - 1 */
+ assert(rep[0] > 0);
+ return rep[0] - 1;
+ }
+ return rep[adjustedOffCode];
+}
+
+/*
+ * ZSTD_seqStore_resolveOffCodes() reconciles any possible divergences in offset history that may arise
+ * due to emission of RLE/raw blocks that disturb the offset history,
+ * and replaces any repcodes within the seqStore that may be invalid.
+ *
+ * dRepcodes are updated as would be on the decompression side.
+ * cRepcodes are updated exactly in accordance with the seqStore.
+ *
+ * Note : this function assumes seq->offBase respects the following numbering scheme :
+ * 0 : invalid
+ * 1-3 : repcode 1-3
+ * 4+ : real_offset+3
+ */
+static void ZSTD_seqStore_resolveOffCodes(repcodes_t* const dRepcodes, repcodes_t* const cRepcodes,
+ seqStore_t* const seqStore, U32 const nbSeq) {
+ U32 idx = 0;
+ for (; idx < nbSeq; ++idx) {
+ seqDef* const seq = seqStore->sequencesStart + idx;
+ U32 const ll0 = (seq->litLength == 0);
+ U32 const offCode = OFFBASE_TO_STORED(seq->offBase);
+ assert(seq->offBase > 0);
+ if (STORED_IS_REPCODE(offCode)) {
+ U32 const dRawOffset = ZSTD_resolveRepcodeToRawOffset(dRepcodes->rep, offCode, ll0);
+ U32 const cRawOffset = ZSTD_resolveRepcodeToRawOffset(cRepcodes->rep, offCode, ll0);
+ /* Adjust simulated decompression repcode history if we come across a mismatch. Replace
+ * the repcode with the offset it actually references, determined by the compression
+ * repcode history.
+ */
+ if (dRawOffset != cRawOffset) {
+ seq->offBase = cRawOffset + ZSTD_REP_NUM;
+ }
+ }
+ /* Compression repcode history is always updated with values directly from the unmodified seqStore.
+ * Decompression repcode history may use modified seq->offset value taken from compression repcode history.
+ */
+ ZSTD_updateRep(dRepcodes->rep, OFFBASE_TO_STORED(seq->offBase), ll0);
+ ZSTD_updateRep(cRepcodes->rep, offCode, ll0);
+ }
+}
+
+/* ZSTD_compressSeqStore_singleBlock():
+ * Compresses a seqStore into a block with a block header, into the buffer dst.
+ *
+ * Returns the total size of that block (including header) or a ZSTD error code.
+ */
+static size_t
+ZSTD_compressSeqStore_singleBlock(ZSTD_CCtx* zc, seqStore_t* const seqStore,
+ repcodes_t* const dRep, repcodes_t* const cRep,
+ void* dst, size_t dstCapacity,
+ const void* src, size_t srcSize,
+ U32 lastBlock, U32 isPartition)
{
- ZSTD_compressedBlockState_t* const tmp = zc->blockState.prevCBlock;
- zc->blockState.prevCBlock = zc->blockState.nextCBlock;
- zc->blockState.nextCBlock = tmp;
+ const U32 rleMaxLength = 25;
+ BYTE* op = (BYTE*)dst;
+ const BYTE* ip = (const BYTE*)src;
+ size_t cSize;
+ size_t cSeqsSize;
+
+ /* In case of an RLE or raw block, the simulated decompression repcode history must be reset */
+ repcodes_t const dRepOriginal = *dRep;
+ DEBUGLOG(5, "ZSTD_compressSeqStore_singleBlock");
+ if (isPartition)
+ ZSTD_seqStore_resolveOffCodes(dRep, cRep, seqStore, (U32)(seqStore->sequences - seqStore->sequencesStart));
+
+ RETURN_ERROR_IF(dstCapacity < ZSTD_blockHeaderSize, dstSize_tooSmall, "Block header doesn't fit");
+ cSeqsSize = ZSTD_entropyCompressSeqStore(seqStore,
+ &zc->blockState.prevCBlock->entropy, &zc->blockState.nextCBlock->entropy,
+ &zc->appliedParams,
+ op + ZSTD_blockHeaderSize, dstCapacity - ZSTD_blockHeaderSize,
+ srcSize,
+ zc->entropyWorkspace, ENTROPY_WORKSPACE_SIZE /* statically allocated in resetCCtx */,
+ zc->bmi2);
+ FORWARD_IF_ERROR(cSeqsSize, "ZSTD_entropyCompressSeqStore failed!");
+
+ if (!zc->isFirstBlock &&
+ cSeqsSize < rleMaxLength &&
+ ZSTD_isRLE((BYTE const*)src, srcSize)) {
+ /* We don't want to emit our first block as a RLE even if it qualifies because
+ * doing so will cause the decoder (cli only) to throw a "should consume all input error."
+ * This is only an issue for zstd <= v1.4.3
+ */
+ cSeqsSize = 1;
+ }
+
+ if (zc->seqCollector.collectSequences) {
+ ZSTD_copyBlockSequences(zc);
+ ZSTD_blockState_confirmRepcodesAndEntropyTables(&zc->blockState);
+ return 0;
+ }
+
+ if (cSeqsSize == 0) {
+ cSize = ZSTD_noCompressBlock(op, dstCapacity, ip, srcSize, lastBlock);
+ FORWARD_IF_ERROR(cSize, "Nocompress block failed");
+ DEBUGLOG(4, "Writing out nocompress block, size: %zu", cSize);
+ *dRep = dRepOriginal; /* reset simulated decompression repcode history */
+ } else if (cSeqsSize == 1) {
+ cSize = ZSTD_rleCompressBlock(op, dstCapacity, *ip, srcSize, lastBlock);
+ FORWARD_IF_ERROR(cSize, "RLE compress block failed");
+ DEBUGLOG(4, "Writing out RLE block, size: %zu", cSize);
+ *dRep = dRepOriginal; /* reset simulated decompression repcode history */
+ } else {
+ ZSTD_blockState_confirmRepcodesAndEntropyTables(&zc->blockState);
+ writeBlockHeader(op, cSeqsSize, srcSize, lastBlock);
+ cSize = ZSTD_blockHeaderSize + cSeqsSize;
+ DEBUGLOG(4, "Writing out compressed block, size: %zu", cSize);
+ }
+
+ if (zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode == FSE_repeat_valid)
+ zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode = FSE_repeat_check;
+
+ return cSize;
+}
+
+/* Struct to keep track of where we are in our recursive calls. */
+typedef struct {
+ U32* splitLocations; /* Array of split indices */
+ size_t idx; /* The current index within splitLocations being worked on */
+} seqStoreSplits;
+
+#define MIN_SEQUENCES_BLOCK_SPLITTING 300
+
+/* Helper function to perform the recursive search for block splits.
+ * Estimates the cost of seqStore prior to split, and estimates the cost of splitting the sequences in half.
+ * If advantageous to split, then we recurse down the two sub-blocks. If not, or if an error occurred in estimation, then
+ * we do not recurse.
+ *
+ * Note: The recursion depth is capped by a heuristic minimum number of sequences, defined by MIN_SEQUENCES_BLOCK_SPLITTING.
+ * In theory, this means the absolute largest recursion depth is 10 == log2(maxNbSeqInBlock/MIN_SEQUENCES_BLOCK_SPLITTING).
+ * In practice, recursion depth usually doesn't go beyond 4.
+ *
+ * Furthermore, the number of splits is capped by ZSTD_MAX_NB_BLOCK_SPLITS. At ZSTD_MAX_NB_BLOCK_SPLITS == 196 with the current existing blockSize
+ * maximum of 128 KB, this value is actually impossible to reach.
+ */
+static void
+ZSTD_deriveBlockSplitsHelper(seqStoreSplits* splits, size_t startIdx, size_t endIdx,
+ ZSTD_CCtx* zc, const seqStore_t* origSeqStore)
+{
+ seqStore_t* fullSeqStoreChunk = &zc->blockSplitCtx.fullSeqStoreChunk;
+ seqStore_t* firstHalfSeqStore = &zc->blockSplitCtx.firstHalfSeqStore;
+ seqStore_t* secondHalfSeqStore = &zc->blockSplitCtx.secondHalfSeqStore;
+ size_t estimatedOriginalSize;
+ size_t estimatedFirstHalfSize;
+ size_t estimatedSecondHalfSize;
+ size_t midIdx = (startIdx + endIdx)/2;
+
+ if (endIdx - startIdx < MIN_SEQUENCES_BLOCK_SPLITTING || splits->idx >= ZSTD_MAX_NB_BLOCK_SPLITS) {
+ DEBUGLOG(6, "ZSTD_deriveBlockSplitsHelper: Too few sequences");
+ return;
+ }
+ DEBUGLOG(4, "ZSTD_deriveBlockSplitsHelper: startIdx=%zu endIdx=%zu", startIdx, endIdx);
+ ZSTD_deriveSeqStoreChunk(fullSeqStoreChunk, origSeqStore, startIdx, endIdx);
+ ZSTD_deriveSeqStoreChunk(firstHalfSeqStore, origSeqStore, startIdx, midIdx);
+ ZSTD_deriveSeqStoreChunk(secondHalfSeqStore, origSeqStore, midIdx, endIdx);
+ estimatedOriginalSize = ZSTD_buildEntropyStatisticsAndEstimateSubBlockSize(fullSeqStoreChunk, zc);
+ estimatedFirstHalfSize = ZSTD_buildEntropyStatisticsAndEstimateSubBlockSize(firstHalfSeqStore, zc);
+ estimatedSecondHalfSize = ZSTD_buildEntropyStatisticsAndEstimateSubBlockSize(secondHalfSeqStore, zc);
+ DEBUGLOG(4, "Estimated original block size: %zu -- First half split: %zu -- Second half split: %zu",
+ estimatedOriginalSize, estimatedFirstHalfSize, estimatedSecondHalfSize);
+ if (ZSTD_isError(estimatedOriginalSize) || ZSTD_isError(estimatedFirstHalfSize) || ZSTD_isError(estimatedSecondHalfSize)) {
+ return;
+ }
+ if (estimatedFirstHalfSize + estimatedSecondHalfSize < estimatedOriginalSize) {
+ ZSTD_deriveBlockSplitsHelper(splits, startIdx, midIdx, zc, origSeqStore);
+ splits->splitLocations[splits->idx] = (U32)midIdx;
+ splits->idx++;
+ ZSTD_deriveBlockSplitsHelper(splits, midIdx, endIdx, zc, origSeqStore);
+ }
+}
+
+/* Base recursive function. Populates a table with intra-block partition indices that can improve compression ratio.
+ *
+ * Returns the number of splits made (which equals the size of the partition table - 1).
+ */
+static size_t ZSTD_deriveBlockSplits(ZSTD_CCtx* zc, U32 partitions[], U32 nbSeq) {
+ seqStoreSplits splits = {partitions, 0};
+ if (nbSeq <= 4) {
+ DEBUGLOG(4, "ZSTD_deriveBlockSplits: Too few sequences to split");
+ /* Refuse to try and split anything with less than 4 sequences */
+ return 0;
+ }
+ ZSTD_deriveBlockSplitsHelper(&splits, 0, nbSeq, zc, &zc->seqStore);
+ splits.splitLocations[splits.idx] = nbSeq;
+ DEBUGLOG(5, "ZSTD_deriveBlockSplits: final nb partitions: %zu", splits.idx+1);
+ return splits.idx;
+}
+
+/* ZSTD_compressBlock_splitBlock():
+ * Attempts to split a given block into multiple blocks to improve compression ratio.
+ *
+ * Returns combined size of all blocks (which includes headers), or a ZSTD error code.
+ */
+static size_t
+ZSTD_compressBlock_splitBlock_internal(ZSTD_CCtx* zc, void* dst, size_t dstCapacity,
+ const void* src, size_t blockSize, U32 lastBlock, U32 nbSeq)
+{
+ size_t cSize = 0;
+ const BYTE* ip = (const BYTE*)src;
+ BYTE* op = (BYTE*)dst;
+ size_t i = 0;
+ size_t srcBytesTotal = 0;
+ U32* partitions = zc->blockSplitCtx.partitions; /* size == ZSTD_MAX_NB_BLOCK_SPLITS */
+ seqStore_t* nextSeqStore = &zc->blockSplitCtx.nextSeqStore;
+ seqStore_t* currSeqStore = &zc->blockSplitCtx.currSeqStore;
+ size_t numSplits = ZSTD_deriveBlockSplits(zc, partitions, nbSeq);
+
+ /* If a block is split and some partitions are emitted as RLE/uncompressed, then repcode history
+ * may become invalid. In order to reconcile potentially invalid repcodes, we keep track of two
+ * separate repcode histories that simulate repcode history on compression and decompression side,
+ * and use the histories to determine whether we must replace a particular repcode with its raw offset.
+ *
+ * 1) cRep gets updated for each partition, regardless of whether the block was emitted as uncompressed
+ * or RLE. This allows us to retrieve the offset value that an invalid repcode references within
+ * a nocompress/RLE block.
+ * 2) dRep gets updated only for compressed partitions, and when a repcode gets replaced, will use
+ * the replacement offset value rather than the original repcode to update the repcode history.
+ * dRep also will be the final repcode history sent to the next block.
+ *
+ * See ZSTD_seqStore_resolveOffCodes() for more details.
+ */
+ repcodes_t dRep;
+ repcodes_t cRep;
+ ZSTD_memcpy(dRep.rep, zc->blockState.prevCBlock->rep, sizeof(repcodes_t));
+ ZSTD_memcpy(cRep.rep, zc->blockState.prevCBlock->rep, sizeof(repcodes_t));
+ ZSTD_memset(nextSeqStore, 0, sizeof(seqStore_t));
+
+ DEBUGLOG(4, "ZSTD_compressBlock_splitBlock_internal (dstCapacity=%u, dictLimit=%u, nextToUpdate=%u)",
+ (unsigned)dstCapacity, (unsigned)zc->blockState.matchState.window.dictLimit,
+ (unsigned)zc->blockState.matchState.nextToUpdate);
+
+ if (numSplits == 0) {
+ size_t cSizeSingleBlock = ZSTD_compressSeqStore_singleBlock(zc, &zc->seqStore,
+ &dRep, &cRep,
+ op, dstCapacity,
+ ip, blockSize,
+ lastBlock, 0 /* isPartition */);
+ FORWARD_IF_ERROR(cSizeSingleBlock, "Compressing single block from splitBlock_internal() failed!");
+ DEBUGLOG(5, "ZSTD_compressBlock_splitBlock_internal: No splits");
+ assert(cSizeSingleBlock <= ZSTD_BLOCKSIZE_MAX + ZSTD_blockHeaderSize);
+ return cSizeSingleBlock;
+ }
+
+ ZSTD_deriveSeqStoreChunk(currSeqStore, &zc->seqStore, 0, partitions[0]);
+ for (i = 0; i <= numSplits; ++i) {
+ size_t srcBytes;
+ size_t cSizeChunk;
+ U32 const lastPartition = (i == numSplits);
+ U32 lastBlockEntireSrc = 0;
+
+ srcBytes = ZSTD_countSeqStoreLiteralsBytes(currSeqStore) + ZSTD_countSeqStoreMatchBytes(currSeqStore);
+ srcBytesTotal += srcBytes;
+ if (lastPartition) {
+ /* This is the final partition, need to account for possible last literals */
+ srcBytes += blockSize - srcBytesTotal;
+ lastBlockEntireSrc = lastBlock;
+ } else {
+ ZSTD_deriveSeqStoreChunk(nextSeqStore, &zc->seqStore, partitions[i], partitions[i+1]);
+ }
+
+ cSizeChunk = ZSTD_compressSeqStore_singleBlock(zc, currSeqStore,
+ &dRep, &cRep,
+ op, dstCapacity,
+ ip, srcBytes,
+ lastBlockEntireSrc, 1 /* isPartition */);
+ DEBUGLOG(5, "Estimated size: %zu actual size: %zu", ZSTD_buildEntropyStatisticsAndEstimateSubBlockSize(currSeqStore, zc), cSizeChunk);
+ FORWARD_IF_ERROR(cSizeChunk, "Compressing chunk failed!");
+
+ ip += srcBytes;
+ op += cSizeChunk;
+ dstCapacity -= cSizeChunk;
+ cSize += cSizeChunk;
+ *currSeqStore = *nextSeqStore;
+ assert(cSizeChunk <= ZSTD_BLOCKSIZE_MAX + ZSTD_blockHeaderSize);
+ }
+ /* cRep and dRep may have diverged during the compression. If so, we use the dRep repcodes
+ * for the next block.
+ */
+ ZSTD_memcpy(zc->blockState.prevCBlock->rep, dRep.rep, sizeof(repcodes_t));
+ return cSize;
+}
+
+static size_t
+ZSTD_compressBlock_splitBlock(ZSTD_CCtx* zc,
+ void* dst, size_t dstCapacity,
+ const void* src, size_t srcSize, U32 lastBlock)
+{
+ const BYTE* ip = (const BYTE*)src;
+ BYTE* op = (BYTE*)dst;
+ U32 nbSeq;
+ size_t cSize;
+ DEBUGLOG(4, "ZSTD_compressBlock_splitBlock");
+ assert(zc->appliedParams.useBlockSplitter == ZSTD_ps_enable);
+
+ { const size_t bss = ZSTD_buildSeqStore(zc, src, srcSize);
+ FORWARD_IF_ERROR(bss, "ZSTD_buildSeqStore failed");
+ if (bss == ZSTDbss_noCompress) {
+ if (zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode == FSE_repeat_valid)
+ zc->blockState.prevCBlock->entropy.fse.offcode_repeatMode = FSE_repeat_check;
+ cSize = ZSTD_noCompressBlock(op, dstCapacity, ip, srcSize, lastBlock);
+ FORWARD_IF_ERROR(cSize, "ZSTD_noCompressBlock failed");
+ DEBUGLOG(4, "ZSTD_compressBlock_splitBlock: Nocompress block");
+ return cSize;
+ }
+ nbSeq = (U32)(zc->seqStore.sequences - zc->seqStore.sequencesStart);
+ }
+
+ cSize = ZSTD_compressBlock_splitBlock_internal(zc, dst, dstCapacity, src, srcSize, lastBlock, nbSeq);
+ FORWARD_IF_ERROR(cSize, "Splitting blocks failed!");
+ return cSize;
}
-static size_t ZSTD_compressBlock_internal(ZSTD_CCtx* zc,
- void* dst, size_t dstCapacity,
- const void* src, size_t srcSize, U32 frame)
+static size_t
+ZSTD_compressBlock_internal(ZSTD_CCtx* zc,
+ void* dst, size_t dstCapacity,
+ const void* src, size_t srcSize, U32 frame)
{
/* This the upper bound for the length of an rle block.
* This isn't the actual upper bound. Finding the real threshold
if (zc->seqCollector.collectSequences) {
ZSTD_copyBlockSequences(zc);
- ZSTD_confirmRepcodesAndEntropyTables(zc);
+ ZSTD_blockState_confirmRepcodesAndEntropyTables(&zc->blockState);
return 0;
}
/* encode sequences and literals */
- cSize = ZSTD_entropyCompressSequences(&zc->seqStore,
+ cSize = ZSTD_entropyCompressSeqStore(&zc->seqStore,
&zc->blockState.prevCBlock->entropy, &zc->blockState.nextCBlock->entropy,
&zc->appliedParams,
dst, dstCapacity,
zc->entropyWorkspace, ENTROPY_WORKSPACE_SIZE /* statically allocated in resetCCtx */,
zc->bmi2);
- if (zc->seqCollector.collectSequences) {
- ZSTD_copyBlockSequences(zc);
- return 0;
- }
-
-
if (frame &&
/* We don't want to emit our first block as a RLE even if it qualifies because
* doing so will cause the decoder (cli only) to throw a "should consume all input error."
out:
if (!ZSTD_isError(cSize) && cSize > 1) {
- ZSTD_confirmRepcodesAndEntropyTables(zc);
+ ZSTD_blockState_confirmRepcodesAndEntropyTables(&zc->blockState);
}
/* We check that dictionaries have offset codes available for the first
* block. After the first block, the offcode table might not have large
size_t const maxCSize = srcSize - ZSTD_minGain(srcSize, zc->appliedParams.cParams.strategy);
FORWARD_IF_ERROR(cSize, "ZSTD_compressSuperBlock failed");
if (cSize != 0 && cSize < maxCSize + ZSTD_blockHeaderSize) {
- ZSTD_confirmRepcodesAndEntropyTables(zc);
+ ZSTD_blockState_confirmRepcodesAndEntropyTables(&zc->blockState);
return cSize;
}
}
void const* ip,
void const* iend)
{
- if (ZSTD_window_needOverflowCorrection(ms->window, iend)) {
- U32 const maxDist = (U32)1 << params->cParams.windowLog;
- U32 const cycleLog = ZSTD_cycleLog(params->cParams.chainLog, params->cParams.strategy);
+ U32 const cycleLog = ZSTD_cycleLog(params->cParams.chainLog, params->cParams.strategy);
+ U32 const maxDist = (U32)1 << params->cParams.windowLog;
+ if (ZSTD_window_needOverflowCorrection(ms->window, cycleLog, maxDist, ms->loadedDictEnd, ip, iend)) {
U32 const correction = ZSTD_window_correctOverflow(&ms->window, cycleLog, maxDist, ip);
ZSTD_STATIC_ASSERT(ZSTD_CHAINLOG_MAX <= 30);
ZSTD_STATIC_ASSERT(ZSTD_WINDOWLOG_MAX_32 <= 30);
* Frame is supposed already started (header already produced)
* @return : compressed size, or an error code
*/
-static size_t ZSTD_compress_frameChunk (ZSTD_CCtx* cctx,
+static size_t ZSTD_compress_frameChunk(ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
U32 lastFrameChunk)
ZSTD_overflowCorrectIfNeeded(
ms, &cctx->workspace, &cctx->appliedParams, ip, ip + blockSize);
ZSTD_checkDictValidity(&ms->window, ip + blockSize, maxDist, &ms->loadedDictEnd, &ms->dictMatchState);
+ ZSTD_window_enforceMaxDist(&ms->window, ip, maxDist, &ms->loadedDictEnd, &ms->dictMatchState);
/* Ensure hash/chain table insertion resumes no sooner than lowlimit */
if (ms->nextToUpdate < ms->window.lowLimit) ms->nextToUpdate = ms->window.lowLimit;
FORWARD_IF_ERROR(cSize, "ZSTD_compressBlock_targetCBlockSize failed");
assert(cSize > 0);
assert(cSize <= blockSize + ZSTD_blockHeaderSize);
+ } else if (ZSTD_blockSplitterEnabled(&cctx->appliedParams)) {
+ cSize = ZSTD_compressBlock_splitBlock(cctx, op, dstCapacity, ip, blockSize, lastBlock);
+ FORWARD_IF_ERROR(cSize, "ZSTD_compressBlock_splitBlock failed");
+ assert(cSize > 0 || cctx->seqCollector.collectSequences == 1);
} else {
cSize = ZSTD_compressBlock_internal(cctx,
op+ZSTD_blockHeaderSize, dstCapacity-ZSTD_blockHeaderSize,
{
RETURN_ERROR_IF(cctx->stage != ZSTDcs_init, stage_wrong,
"wrong cctx stage");
- RETURN_ERROR_IF(cctx->appliedParams.ldmParams.enableLdm,
+ RETURN_ERROR_IF(cctx->appliedParams.ldmParams.enableLdm == ZSTD_ps_enable,
parameter_unsupported,
"incompatible with ldm");
cctx->externSeqStore.seq = seq;
if (!srcSize) return fhSize; /* do not generate an empty block if no input */
- if (!ZSTD_window_update(&ms->window, src, srcSize)) {
+ if (!ZSTD_window_update(&ms->window, src, srcSize, ms->forceNonContiguous)) {
+ ms->forceNonContiguous = 0;
ms->nextToUpdate = ms->window.dictLimit;
}
- if (cctx->appliedParams.ldmParams.enableLdm) {
- ZSTD_window_update(&cctx->ldmState.window, src, srcSize);
+ if (cctx->appliedParams.ldmParams.enableLdm == ZSTD_ps_enable) {
+ ZSTD_window_update(&cctx->ldmState.window, src, srcSize, /* forceNonContiguous */ 0);
}
if (!frame) {
{
const BYTE* ip = (const BYTE*) src;
const BYTE* const iend = ip + srcSize;
+ int const loadLdmDict = params->ldmParams.enableLdm == ZSTD_ps_enable && ls != NULL;
- ZSTD_window_update(&ms->window, src, srcSize);
+ /* Assert that we the ms params match the params we're being given */
+ ZSTD_assertEqualCParams(params->cParams, ms->cParams);
+
+ if (srcSize > ZSTD_CHUNKSIZE_MAX) {
+ /* Allow the dictionary to set indices up to exactly ZSTD_CURRENT_MAX.
+ * Dictionaries right at the edge will immediately trigger overflow
+ * correction, but I don't want to insert extra constraints here.
+ */
+ U32 const maxDictSize = ZSTD_CURRENT_MAX - 1;
+ /* We must have cleared our windows when our source is this large. */
+ assert(ZSTD_window_isEmpty(ms->window));
+ if (loadLdmDict)
+ assert(ZSTD_window_isEmpty(ls->window));
+ /* If the dictionary is too large, only load the suffix of the dictionary. */
+ if (srcSize > maxDictSize) {
+ ip = iend - maxDictSize;
+ src = ip;
+ srcSize = maxDictSize;
+ }
+ }
+
+ DEBUGLOG(4, "ZSTD_loadDictionaryContent(): useRowMatchFinder=%d", (int)params->useRowMatchFinder);
+ ZSTD_window_update(&ms->window, src, srcSize, /* forceNonContiguous */ 0);
ms->loadedDictEnd = params->forceWindow ? 0 : (U32)(iend - ms->window.base);
+ ms->forceNonContiguous = params->deterministicRefPrefix;
- if (params->ldmParams.enableLdm && ls != NULL) {
- ZSTD_window_update(&ls->window, src, srcSize);
+ if (loadLdmDict) {
+ ZSTD_window_update(&ls->window, src, srcSize, /* forceNonContiguous */ 0);
ls->loadedDictEnd = params->forceWindow ? 0 : (U32)(iend - ls->window.base);
}
- /* Assert that we the ms params match the params we're being given */
- ZSTD_assertEqualCParams(params->cParams, ms->cParams);
-
if (srcSize <= HASH_READ_SIZE) return 0;
- while (iend - ip > HASH_READ_SIZE) {
- size_t const remaining = (size_t)(iend - ip);
- size_t const chunk = MIN(remaining, ZSTD_CHUNKSIZE_MAX);
- const BYTE* const ichunk = ip + chunk;
-
- ZSTD_overflowCorrectIfNeeded(ms, ws, params, ip, ichunk);
+ ZSTD_overflowCorrectIfNeeded(ms, ws, params, ip, iend);
- if (params->ldmParams.enableLdm && ls != NULL)
- ZSTD_ldm_fillHashTable(ls, (const BYTE*)src, (const BYTE*)src + srcSize, ¶ms->ldmParams);
+ if (loadLdmDict)
+ ZSTD_ldm_fillHashTable(ls, ip, iend, ¶ms->ldmParams);
- switch(params->cParams.strategy)
- {
- case ZSTD_fast:
- ZSTD_fillHashTable(ms, ichunk, dtlm);
- break;
- case ZSTD_dfast:
- ZSTD_fillDoubleHashTable(ms, ichunk, dtlm);
- break;
+ switch(params->cParams.strategy)
+ {
+ case ZSTD_fast:
+ ZSTD_fillHashTable(ms, iend, dtlm);
+ break;
+ case ZSTD_dfast:
+ ZSTD_fillDoubleHashTable(ms, iend, dtlm);
+ break;
- case ZSTD_greedy:
- case ZSTD_lazy:
- case ZSTD_lazy2:
- if (chunk >= HASH_READ_SIZE && ms->dedicatedDictSearch) {
- assert(chunk == remaining); /* must load everything in one go */
- ZSTD_dedicatedDictSearch_lazy_loadDictionary(ms, ichunk-HASH_READ_SIZE);
- } else if (chunk >= HASH_READ_SIZE) {
- ZSTD_insertAndFindFirstIndex(ms, ichunk-HASH_READ_SIZE);
+ case ZSTD_greedy:
+ case ZSTD_lazy:
+ case ZSTD_lazy2:
+ assert(srcSize >= HASH_READ_SIZE);
+ if (ms->dedicatedDictSearch) {
+ assert(ms->chainTable != NULL);
+ ZSTD_dedicatedDictSearch_lazy_loadDictionary(ms, iend-HASH_READ_SIZE);
+ } else {
+ assert(params->useRowMatchFinder != ZSTD_ps_auto);
+ if (params->useRowMatchFinder == ZSTD_ps_enable) {
+ size_t const tagTableSize = ((size_t)1 << params->cParams.hashLog) * sizeof(U16);
+ ZSTD_memset(ms->tagTable, 0, tagTableSize);
+ ZSTD_row_update(ms, iend-HASH_READ_SIZE);
+ DEBUGLOG(4, "Using row-based hash table for lazy dict");
+ } else {
+ ZSTD_insertAndFindFirstIndex(ms, iend-HASH_READ_SIZE);
+ DEBUGLOG(4, "Using chain-based hash table for lazy dict");
}
- break;
-
- case ZSTD_btlazy2: /* we want the dictionary table fully sorted */
- case ZSTD_btopt:
- case ZSTD_btultra:
- case ZSTD_btultra2:
- if (chunk >= HASH_READ_SIZE)
- ZSTD_updateTree(ms, ichunk-HASH_READ_SIZE, ichunk);
- break;
-
- default:
- assert(0); /* not possible : not a valid strategy id */
}
+ break;
+
+ case ZSTD_btlazy2: /* we want the dictionary table fully sorted */
+ case ZSTD_btopt:
+ case ZSTD_btultra:
+ case ZSTD_btultra2:
+ assert(srcSize >= HASH_READ_SIZE);
+ ZSTD_updateTree(ms, iend-HASH_READ_SIZE, iend);
+ break;
- ip = ichunk;
+ default:
+ assert(0); /* not possible : not a valid strategy id */
}
ms->nextToUpdate = (U32)(iend - ms->window.base);
const BYTE* const dictEnd = dictPtr + dictSize;
size_t dictID;
size_t eSize;
-
ZSTD_STATIC_ASSERT(HUF_WORKSPACE_SIZE >= (1<<MAX(MLFSELog,LLFSELog)));
assert(dictSize >= 8);
assert(MEM_readLE32(dictPtr) == ZSTD_MAGIC_DICTIONARY);
const ZSTD_CCtx_params* params, U64 pledgedSrcSize,
ZSTD_buffered_policy_e zbuff)
{
+ size_t const dictContentSize = cdict ? cdict->dictContentSize : dictSize;
DEBUGLOG(4, "ZSTD_compressBegin_internal: wlog=%u", params->cParams.windowLog);
/* params are supposed to be fully validated at this point */
assert(!ZSTD_isError(ZSTD_checkCParams(params->cParams)));
return ZSTD_resetCCtx_usingCDict(cctx, cdict, params, pledgedSrcSize, zbuff);
}
- FORWARD_IF_ERROR( ZSTD_resetCCtx_internal(cctx, *params, pledgedSrcSize,
+ FORWARD_IF_ERROR( ZSTD_resetCCtx_internal(cctx, params, pledgedSrcSize,
+ dictContentSize,
ZSTDcrp_makeClean, zbuff) , "");
{ size_t const dictID = cdict ?
ZSTD_compress_insertDictionary(
FORWARD_IF_ERROR(dictID, "ZSTD_compress_insertDictionary failed");
assert(dictID <= UINT_MAX);
cctx->dictID = (U32)dictID;
- cctx->dictContentSize = cdict ? cdict->dictContentSize : dictSize;
+ cctx->dictContentSize = dictContentSize;
}
return 0;
}
const void* dict,size_t dictSize,
ZSTD_parameters params)
{
- ZSTD_CCtx_params cctxParams;
DEBUGLOG(4, "ZSTD_compress_advanced");
FORWARD_IF_ERROR(ZSTD_checkCParams(params.cParams), "");
- ZSTD_CCtxParams_init_internal(&cctxParams, ¶ms, ZSTD_NO_CLEVEL);
+ ZSTD_CCtxParams_init_internal(&cctx->simpleApiParams, ¶ms, ZSTD_NO_CLEVEL);
return ZSTD_compress_advanced_internal(cctx,
dst, dstCapacity,
src, srcSize,
dict, dictSize,
- &cctxParams);
+ &cctx->simpleApiParams);
}
/* Internal */
const void* dict, size_t dictSize,
int compressionLevel)
{
- ZSTD_CCtx_params cctxParams;
{
ZSTD_parameters const params = ZSTD_getParams_internal(compressionLevel, srcSize, dict ? dictSize : 0, ZSTD_cpm_noAttachDict);
assert(params.fParams.contentSizeFlag == 1);
- ZSTD_CCtxParams_init_internal(&cctxParams, ¶ms, (compressionLevel == 0) ? ZSTD_CLEVEL_DEFAULT: compressionLevel);
+ ZSTD_CCtxParams_init_internal(&cctx->simpleApiParams, ¶ms, (compressionLevel == 0) ? ZSTD_CLEVEL_DEFAULT: compressionLevel);
}
DEBUGLOG(4, "ZSTD_compress_usingDict (srcSize=%u)", (unsigned)srcSize);
- return ZSTD_compress_advanced_internal(cctx, dst, dstCapacity, src, srcSize, dict, dictSize, &cctxParams);
+ return ZSTD_compress_advanced_internal(cctx, dst, dstCapacity, src, srcSize, dict, dictSize, &cctx->simpleApiParams);
}
size_t ZSTD_compressCCtx(ZSTD_CCtx* cctx,
DEBUGLOG(5, "sizeof(ZSTD_CDict) : %u", (unsigned)sizeof(ZSTD_CDict));
return ZSTD_cwksp_alloc_size(sizeof(ZSTD_CDict))
+ ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE)
- + ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 0)
+ /* enableDedicatedDictSearch == 1 ensures that CDict estimation will not be too small
+ * in case we are using DDS with row-hash. */
+ + ZSTD_sizeof_matchState(&cParams, ZSTD_resolveRowMatchFinderMode(ZSTD_ps_auto, &cParams),
+ /* enableDedicatedDictSearch */ 1, /* forCCtx */ 0)
+ (dictLoadMethod == ZSTD_dlm_byRef ? 0
: ZSTD_cwksp_alloc_size(ZSTD_cwksp_align(dictSize, sizeof(void *))));
}
assert(!ZSTD_checkCParams(params.cParams));
cdict->matchState.cParams = params.cParams;
cdict->matchState.dedicatedDictSearch = params.enableDedicatedDictSearch;
- if (cdict->matchState.dedicatedDictSearch && dictSize > ZSTD_CHUNKSIZE_MAX) {
- cdict->matchState.dedicatedDictSearch = 0;
- }
if ((dictLoadMethod == ZSTD_dlm_byRef) || (!dictBuffer) || (!dictSize)) {
cdict->dictContent = dictBuffer;
} else {
&cdict->matchState,
&cdict->workspace,
¶ms.cParams,
+ params.useRowMatchFinder,
ZSTDcrp_makeClean,
ZSTDirp_reset,
ZSTD_resetTarget_CDict), "");
static ZSTD_CDict* ZSTD_createCDict_advanced_internal(size_t dictSize,
ZSTD_dictLoadMethod_e dictLoadMethod,
- ZSTD_compressionParameters cParams, ZSTD_customMem customMem)
+ ZSTD_compressionParameters cParams,
+ ZSTD_paramSwitch_e useRowMatchFinder,
+ U32 enableDedicatedDictSearch,
+ ZSTD_customMem customMem)
{
if ((!customMem.customAlloc) ^ (!customMem.customFree)) return NULL;
{ size_t const workspaceSize =
ZSTD_cwksp_alloc_size(sizeof(ZSTD_CDict)) +
ZSTD_cwksp_alloc_size(HUF_WORKSPACE_SIZE) +
- ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 0) +
+ ZSTD_sizeof_matchState(&cParams, useRowMatchFinder, enableDedicatedDictSearch, /* forCCtx */ 0) +
(dictLoadMethod == ZSTD_dlm_byRef ? 0
: ZSTD_cwksp_alloc_size(ZSTD_cwksp_align(dictSize, sizeof(void*))));
void* const workspace = ZSTD_customMalloc(workspaceSize, customMem);
ZSTD_cwksp_move(&cdict->workspace, &ws);
cdict->customMem = customMem;
cdict->compressionLevel = ZSTD_NO_CLEVEL; /* signals advanced API usage */
-
+ cdict->useRowMatchFinder = useRowMatchFinder;
return cdict;
}
}
&cctxParams, customMem);
}
-ZSTDLIB_API ZSTD_CDict* ZSTD_createCDict_advanced2(
+ZSTD_CDict* ZSTD_createCDict_advanced2(
const void* dict, size_t dictSize,
ZSTD_dictLoadMethod_e dictLoadMethod,
ZSTD_dictContentType_e dictContentType,
&cctxParams, ZSTD_CONTENTSIZE_UNKNOWN, dictSize, ZSTD_cpm_createCDict);
}
+ DEBUGLOG(3, "ZSTD_createCDict_advanced2: DDS: %u", cctxParams.enableDedicatedDictSearch);
cctxParams.cParams = cParams;
+ cctxParams.useRowMatchFinder = ZSTD_resolveRowMatchFinderMode(cctxParams.useRowMatchFinder, &cParams);
cdict = ZSTD_createCDict_advanced_internal(dictSize,
dictLoadMethod, cctxParams.cParams,
+ cctxParams.useRowMatchFinder, cctxParams.enableDedicatedDictSearch,
customMem);
if (ZSTD_isError( ZSTD_initCDict_internal(cdict,
ZSTD_dictContentType_e dictContentType,
ZSTD_compressionParameters cParams)
{
- size_t const matchStateSize = ZSTD_sizeof_matchState(&cParams, /* forCCtx */ 0);
+ ZSTD_paramSwitch_e const useRowMatchFinder = ZSTD_resolveRowMatchFinderMode(ZSTD_ps_auto, &cParams);
+ /* enableDedicatedDictSearch == 1 ensures matchstate is not too small in case this CDict will be used for DDS + row hash */
+ size_t const matchStateSize = ZSTD_sizeof_matchState(&cParams, useRowMatchFinder, /* enableDedicatedDictSearch */ 1, /* forCCtx */ 0);
size_t const neededSize = ZSTD_cwksp_alloc_size(sizeof(ZSTD_CDict))
+ (dictLoadMethod == ZSTD_dlm_byRef ? 0
: ZSTD_cwksp_alloc_size(ZSTD_cwksp_align(dictSize, sizeof(void*))))
ZSTD_CCtxParams_init(¶ms, 0);
params.cParams = cParams;
+ params.useRowMatchFinder = useRowMatchFinder;
+ cdict->useRowMatchFinder = useRowMatchFinder;
if (ZSTD_isError( ZSTD_initCDict_internal(cdict,
dict, dictSize,
return cdict->dictID;
}
-
-/* ZSTD_compressBegin_usingCDict_advanced() :
- * cdict must be != NULL */
-size_t ZSTD_compressBegin_usingCDict_advanced(
+/* ZSTD_compressBegin_usingCDict_internal() :
+ * Implementation of various ZSTD_compressBegin_usingCDict* functions.
+ */
+static size_t ZSTD_compressBegin_usingCDict_internal(
ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict,
ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize)
{
ZSTD_CCtx_params cctxParams;
- DEBUGLOG(4, "ZSTD_compressBegin_usingCDict_advanced");
+ DEBUGLOG(4, "ZSTD_compressBegin_usingCDict_internal");
RETURN_ERROR_IF(cdict==NULL, dictionary_wrong, "NULL pointer!");
/* Initialize the cctxParams from the cdict */
{
ZSTDb_not_buffered);
}
+
+/* ZSTD_compressBegin_usingCDict_advanced() :
+ * This function is DEPRECATED.
+ * cdict must be != NULL */
+size_t ZSTD_compressBegin_usingCDict_advanced(
+ ZSTD_CCtx* const cctx, const ZSTD_CDict* const cdict,
+ ZSTD_frameParameters const fParams, unsigned long long const pledgedSrcSize)
+{
+ return ZSTD_compressBegin_usingCDict_internal(cctx, cdict, fParams, pledgedSrcSize);
+}
+
/* ZSTD_compressBegin_usingCDict() :
- * pledgedSrcSize=0 means "unknown"
- * if pledgedSrcSize>0, it will enable contentSizeFlag */
+ * cdict must be != NULL */
size_t ZSTD_compressBegin_usingCDict(ZSTD_CCtx* cctx, const ZSTD_CDict* cdict)
{
ZSTD_frameParameters const fParams = { 0 /*content*/, 0 /*checksum*/, 0 /*noDictID*/ };
- DEBUGLOG(4, "ZSTD_compressBegin_usingCDict : dictIDFlag == %u", !fParams.noDictIDFlag);
- return ZSTD_compressBegin_usingCDict_advanced(cctx, cdict, fParams, ZSTD_CONTENTSIZE_UNKNOWN);
+ return ZSTD_compressBegin_usingCDict_internal(cctx, cdict, fParams, ZSTD_CONTENTSIZE_UNKNOWN);
}
-size_t ZSTD_compress_usingCDict_advanced(ZSTD_CCtx* cctx,
+/*! ZSTD_compress_usingCDict_internal():
+ * Implementation of various ZSTD_compress_usingCDict* functions.
+ */
+static size_t ZSTD_compress_usingCDict_internal(ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const ZSTD_CDict* cdict, ZSTD_frameParameters fParams)
{
- FORWARD_IF_ERROR(ZSTD_compressBegin_usingCDict_advanced(cctx, cdict, fParams, srcSize), ""); /* will check if cdict != NULL */
+ FORWARD_IF_ERROR(ZSTD_compressBegin_usingCDict_internal(cctx, cdict, fParams, srcSize), ""); /* will check if cdict != NULL */
return ZSTD_compressEnd(cctx, dst, dstCapacity, src, srcSize);
}
+/*! ZSTD_compress_usingCDict_advanced():
+ * This function is DEPRECATED.
+ */
+size_t ZSTD_compress_usingCDict_advanced(ZSTD_CCtx* cctx,
+ void* dst, size_t dstCapacity,
+ const void* src, size_t srcSize,
+ const ZSTD_CDict* cdict, ZSTD_frameParameters fParams)
+{
+ return ZSTD_compress_usingCDict_internal(cctx, dst, dstCapacity, src, srcSize, cdict, fParams);
+}
+
/*! ZSTD_compress_usingCDict() :
* Compression using a digested Dictionary.
* Faster startup than ZSTD_compress_usingDict(), recommended when same dictionary is used multiple times.
const ZSTD_CDict* cdict)
{
ZSTD_frameParameters const fParams = { 1 /*content*/, 0 /*checksum*/, 0 /*noDictID*/ };
- return ZSTD_compress_usingCDict_advanced(cctx, dst, dstCapacity, src, srcSize, cdict, fParams);
+ return ZSTD_compress_usingCDict_internal(cctx, dst, dstCapacity, src, srcSize, cdict, fParams);
}
FORWARD_IF_ERROR( ZSTD_initLocalDict(cctx) , ""); /* Init the local dict if present. */
ZSTD_memset(&cctx->prefixDict, 0, sizeof(cctx->prefixDict)); /* single usage */
assert(prefixDict.dict==NULL || cctx->cdict==NULL); /* only one can be set */
- if (cctx->cdict)
- params.compressionLevel = cctx->cdict->compressionLevel; /* let cdict take priority in terms of compression level */
+ if (cctx->cdict && !cctx->localDict.cdict) {
+ /* Let the cdict's compression level take priority over the requested params.
+ * But do not take the cdict's compression level if the "cdict" is actually a localDict
+ * generated from ZSTD_initLocalDict().
+ */
+ params.compressionLevel = cctx->cdict->compressionLevel;
+ }
DEBUGLOG(4, "ZSTD_compressStream2 : transparent init stage");
if (endOp == ZSTD_e_end) cctx->pledgedSrcSizePlusOne = inSize + 1; /* auto-fix pledgedSrcSize */
{
dictSize, mode);
}
- if (ZSTD_CParams_shouldEnableLdm(¶ms.cParams)) {
- /* Enable LDM by default for optimal parser and window size >= 128MB */
- DEBUGLOG(4, "LDM enabled by default (window size >= 128MB, strategy >= btopt)");
- params.ldmParams.enableLdm = 1;
- }
+ params.useBlockSplitter = ZSTD_resolveBlockSplitterMode(params.useBlockSplitter, ¶ms.cParams);
+ params.ldmParams.enableLdm = ZSTD_resolveEnableLdm(params.ldmParams.enableLdm, ¶ms.cParams);
+ params.useRowMatchFinder = ZSTD_resolveRowMatchFinderMode(params.useRowMatchFinder, ¶ms.cParams);
{ U64 const pledgedSrcSize = cctx->pledgedSrcSizePlusOne - 1;
assert(!ZSTD_isError(ZSTD_checkCParams(params.cParams)));
size_t posInSrc; /* Number of bytes given by sequences provided so far */
} ZSTD_sequencePosition;
-/* Returns a ZSTD error code if sequence is not valid */
-static size_t ZSTD_validateSequence(U32 offCode, U32 matchLength,
- size_t posInSrc, U32 windowLog, size_t dictSize, U32 minMatch) {
- size_t offsetBound;
- U32 windowSize = 1 << windowLog;
- /* posInSrc represents the amount of data the the decoder would decode up to this point.
+/* ZSTD_validateSequence() :
+ * @offCode : is presumed to follow format required by ZSTD_storeSeq()
+ * @returns a ZSTD error code if sequence is not valid
+ */
+static size_t
+ZSTD_validateSequence(U32 offCode, U32 matchLength,
+ size_t posInSrc, U32 windowLog, size_t dictSize)
+{
+ U32 const windowSize = 1 << windowLog;
+ /* posInSrc represents the amount of data the decoder would decode up to this point.
* As long as the amount of data decoded is less than or equal to window size, offsets may be
* larger than the total length of output decoded in order to reference the dict, even larger than
* window size. After output surpasses windowSize, we're limited to windowSize offsets again.
*/
- offsetBound = posInSrc > windowSize ? (size_t)windowSize : posInSrc + (size_t)dictSize;
- RETURN_ERROR_IF(offCode > offsetBound + ZSTD_REP_MOVE, corruption_detected, "Offset too large!");
- RETURN_ERROR_IF(matchLength < minMatch, corruption_detected, "Matchlength too small");
+ size_t const offsetBound = posInSrc > windowSize ? (size_t)windowSize : posInSrc + (size_t)dictSize;
+ RETURN_ERROR_IF(offCode > STORE_OFFSET(offsetBound), corruption_detected, "Offset too large!");
+ RETURN_ERROR_IF(matchLength < MINMATCH, corruption_detected, "Matchlength too small");
return 0;
}
/* Returns an offset code, given a sequence's raw offset, the ongoing repcode array, and whether litLength == 0 */
-static U32 ZSTD_finalizeOffCode(U32 rawOffset, const U32 rep[ZSTD_REP_NUM], U32 ll0) {
- U32 offCode = rawOffset + ZSTD_REP_MOVE;
- U32 repCode = 0;
+static U32 ZSTD_finalizeOffCode(U32 rawOffset, const U32 rep[ZSTD_REP_NUM], U32 ll0)
+{
+ U32 offCode = STORE_OFFSET(rawOffset);
if (!ll0 && rawOffset == rep[0]) {
- repCode = 1;
+ offCode = STORE_REPCODE_1;
} else if (rawOffset == rep[1]) {
- repCode = 2 - ll0;
+ offCode = STORE_REPCODE(2 - ll0);
} else if (rawOffset == rep[2]) {
- repCode = 3 - ll0;
+ offCode = STORE_REPCODE(3 - ll0);
} else if (ll0 && rawOffset == rep[0] - 1) {
- repCode = 3;
- }
- if (repCode) {
- /* ZSTD_storeSeq expects a number in the range [0, 2] to represent a repcode */
- offCode = repCode - 1;
+ offCode = STORE_REPCODE_3;
}
return offCode;
}
/* Returns 0 on success, and a ZSTD_error otherwise. This function scans through an array of
* ZSTD_Sequence, storing the sequences it finds, until it reaches a block delimiter.
*/
-static size_t ZSTD_copySequencesToSeqStoreExplicitBlockDelim(ZSTD_CCtx* cctx, ZSTD_sequencePosition* seqPos,
- const ZSTD_Sequence* const inSeqs, size_t inSeqsSize,
- const void* src, size_t blockSize) {
+static size_t
+ZSTD_copySequencesToSeqStoreExplicitBlockDelim(ZSTD_CCtx* cctx,
+ ZSTD_sequencePosition* seqPos,
+ const ZSTD_Sequence* const inSeqs, size_t inSeqsSize,
+ const void* src, size_t blockSize)
+{
U32 idx = seqPos->idx;
BYTE const* ip = (BYTE const*)(src);
const BYTE* const iend = ip + blockSize;
repcodes_t updatedRepcodes;
U32 dictSize;
- U32 litLength;
- U32 matchLength;
- U32 ll0;
- U32 offCode;
if (cctx->cdict) {
dictSize = (U32)cctx->cdict->dictContentSize;
}
ZSTD_memcpy(updatedRepcodes.rep, cctx->blockState.prevCBlock->rep, sizeof(repcodes_t));
for (; (inSeqs[idx].matchLength != 0 || inSeqs[idx].offset != 0) && idx < inSeqsSize; ++idx) {
- litLength = inSeqs[idx].litLength;
- matchLength = inSeqs[idx].matchLength;
- ll0 = litLength == 0;
- offCode = ZSTD_finalizeOffCode(inSeqs[idx].offset, updatedRepcodes.rep, ll0);
- updatedRepcodes = ZSTD_updateRep(updatedRepcodes.rep, offCode, ll0);
+ U32 const litLength = inSeqs[idx].litLength;
+ U32 const ll0 = (litLength == 0);
+ U32 const matchLength = inSeqs[idx].matchLength;
+ U32 const offCode = ZSTD_finalizeOffCode(inSeqs[idx].offset, updatedRepcodes.rep, ll0);
+ ZSTD_updateRep(updatedRepcodes.rep, offCode, ll0);
DEBUGLOG(6, "Storing sequence: (of: %u, ml: %u, ll: %u)", offCode, matchLength, litLength);
if (cctx->appliedParams.validateSequences) {
seqPos->posInSrc += litLength + matchLength;
FORWARD_IF_ERROR(ZSTD_validateSequence(offCode, matchLength, seqPos->posInSrc,
- cctx->appliedParams.cParams.windowLog, dictSize,
- cctx->appliedParams.cParams.minMatch),
+ cctx->appliedParams.cParams.windowLog, dictSize),
"Sequence validation failed");
}
RETURN_ERROR_IF(idx - seqPos->idx > cctx->seqStore.maxNbSeq, memory_allocation,
"Not enough memory allocated. Try adjusting ZSTD_c_minMatch.");
- ZSTD_storeSeq(&cctx->seqStore, litLength, ip, iend, offCode, matchLength - MINMATCH);
+ ZSTD_storeSeq(&cctx->seqStore, litLength, ip, iend, offCode, matchLength);
ip += matchLength + litLength;
}
ZSTD_memcpy(cctx->blockState.nextCBlock->rep, updatedRepcodes.rep, sizeof(repcodes_t));
* avoid splitting a match, or to avoid splitting a match such that it would produce a match
* smaller than MINMATCH. In this case, we return the number of bytes that we didn't read from this block.
*/
-static size_t ZSTD_copySequencesToSeqStoreNoBlockDelim(ZSTD_CCtx* cctx, ZSTD_sequencePosition* seqPos,
- const ZSTD_Sequence* const inSeqs, size_t inSeqsSize,
- const void* src, size_t blockSize) {
+static size_t
+ZSTD_copySequencesToSeqStoreNoBlockDelim(ZSTD_CCtx* cctx, ZSTD_sequencePosition* seqPos,
+ const ZSTD_Sequence* const inSeqs, size_t inSeqsSize,
+ const void* src, size_t blockSize)
+{
U32 idx = seqPos->idx;
U32 startPosInSequence = seqPos->posInSequence;
U32 endPosInSequence = seqPos->posInSequence + (U32)blockSize;
repcodes_t updatedRepcodes;
U32 bytesAdjustment = 0;
U32 finalMatchSplit = 0;
- U32 litLength;
- U32 matchLength;
- U32 rawOffset;
- U32 offCode;
if (cctx->cdict) {
dictSize = cctx->cdict->dictContentSize;
ZSTD_memcpy(updatedRepcodes.rep, cctx->blockState.prevCBlock->rep, sizeof(repcodes_t));
while (endPosInSequence && idx < inSeqsSize && !finalMatchSplit) {
const ZSTD_Sequence currSeq = inSeqs[idx];
- litLength = currSeq.litLength;
- matchLength = currSeq.matchLength;
- rawOffset = currSeq.offset;
+ U32 litLength = currSeq.litLength;
+ U32 matchLength = currSeq.matchLength;
+ U32 const rawOffset = currSeq.offset;
+ U32 offCode;
/* Modify the sequence depending on where endPosInSequence lies */
if (endPosInSequence >= currSeq.litLength + currSeq.matchLength) {
}
}
/* Check if this offset can be represented with a repcode */
- { U32 ll0 = (litLength == 0);
+ { U32 const ll0 = (litLength == 0);
offCode = ZSTD_finalizeOffCode(rawOffset, updatedRepcodes.rep, ll0);
- updatedRepcodes = ZSTD_updateRep(updatedRepcodes.rep, offCode, ll0);
+ ZSTD_updateRep(updatedRepcodes.rep, offCode, ll0);
}
if (cctx->appliedParams.validateSequences) {
seqPos->posInSrc += litLength + matchLength;
FORWARD_IF_ERROR(ZSTD_validateSequence(offCode, matchLength, seqPos->posInSrc,
- cctx->appliedParams.cParams.windowLog, dictSize,
- cctx->appliedParams.cParams.minMatch),
+ cctx->appliedParams.cParams.windowLog, dictSize),
"Sequence validation failed");
}
DEBUGLOG(6, "Storing sequence: (of: %u, ml: %u, ll: %u)", offCode, matchLength, litLength);
RETURN_ERROR_IF(idx - seqPos->idx > cctx->seqStore.maxNbSeq, memory_allocation,
"Not enough memory allocated. Try adjusting ZSTD_c_minMatch.");
- ZSTD_storeSeq(&cctx->seqStore, litLength, ip, iend, offCode, matchLength - MINMATCH);
+ ZSTD_storeSeq(&cctx->seqStore, litLength, ip, iend, offCode, matchLength);
ip += matchLength + litLength;
}
DEBUGLOG(5, "Ending seq: idx: %u (of: %u ml: %u ll: %u)", idx, inSeqs[idx].offset, inSeqs[idx].matchLength, inSeqs[idx].litLength);
typedef size_t (*ZSTD_sequenceCopier) (ZSTD_CCtx* cctx, ZSTD_sequencePosition* seqPos,
const ZSTD_Sequence* const inSeqs, size_t inSeqsSize,
const void* src, size_t blockSize);
-static ZSTD_sequenceCopier ZSTD_selectSequenceCopier(ZSTD_sequenceFormat_e mode) {
+static ZSTD_sequenceCopier ZSTD_selectSequenceCopier(ZSTD_sequenceFormat_e mode)
+{
ZSTD_sequenceCopier sequenceCopier = NULL;
assert(ZSTD_cParam_withinBounds(ZSTD_c_blockDelimiters, mode));
if (mode == ZSTD_sf_explicitBlockDelimiters) {
/* Compress, block-by-block, all of the sequences given.
*
- * Returns the cumulative size of all compressed blocks (including their headers), otherwise a ZSTD error.
+ * Returns the cumulative size of all compressed blocks (including their headers),
+ * otherwise a ZSTD error.
*/
-static size_t ZSTD_compressSequences_internal(ZSTD_CCtx* cctx,
- void* dst, size_t dstCapacity,
- const ZSTD_Sequence* inSeqs, size_t inSeqsSize,
- const void* src, size_t srcSize) {
+static size_t
+ZSTD_compressSequences_internal(ZSTD_CCtx* cctx,
+ void* dst, size_t dstCapacity,
+ const ZSTD_Sequence* inSeqs, size_t inSeqsSize,
+ const void* src, size_t srcSize)
+{
size_t cSize = 0;
U32 lastBlock;
size_t blockSize;
BYTE const* ip = (BYTE const*)src;
BYTE* op = (BYTE*)dst;
- ZSTD_sequenceCopier sequenceCopier = ZSTD_selectSequenceCopier(cctx->appliedParams.blockDelimiters);
+ ZSTD_sequenceCopier const sequenceCopier = ZSTD_selectSequenceCopier(cctx->appliedParams.blockDelimiters);
DEBUGLOG(4, "ZSTD_compressSequences_internal srcSize: %zu, inSeqsSize: %zu", srcSize, inSeqsSize);
/* Special case: empty frame */
continue;
}
- compressedSeqsSize = ZSTD_entropyCompressSequences(&cctx->seqStore,
+ compressedSeqsSize = ZSTD_entropyCompressSeqStore(&cctx->seqStore,
&cctx->blockState.prevCBlock->entropy, &cctx->blockState.nextCBlock->entropy,
&cctx->appliedParams,
op + ZSTD_blockHeaderSize /* Leave space for block header */, dstCapacity - ZSTD_blockHeaderSize,
} else {
U32 cBlockHeader;
/* Error checking and repcodes update */
- ZSTD_confirmRepcodesAndEntropyTables(cctx);
+ ZSTD_blockState_confirmRepcodesAndEntropyTables(&cctx->blockState);
if (cctx->blockState.prevCBlock->entropy.fse.offcode_repeatMode == FSE_repeat_valid)
cctx->blockState.prevCBlock->entropy.fse.offcode_repeatMode = FSE_repeat_check;
size_t ZSTD_compressSequences(ZSTD_CCtx* const cctx, void* dst, size_t dstCapacity,
const ZSTD_Sequence* inSeqs, size_t inSeqsSize,
- const void* src, size_t srcSize) {
+ const void* src, size_t srcSize)
+{
BYTE* op = (BYTE*)dst;
size_t cSize = 0;
size_t compressedBlocksSize = 0;
/*-===== Pre-defined compression levels =====-*/
+#include "clevels.h"
-#define ZSTD_MAX_CLEVEL 22
int ZSTD_maxCLevel(void) { return ZSTD_MAX_CLEVEL; }
int ZSTD_minCLevel(void) { return (int)-ZSTD_TARGETLENGTH_MAX; }
-
-static const ZSTD_compressionParameters ZSTD_defaultCParameters[4][ZSTD_MAX_CLEVEL+1] = {
-{ /* "default" - for any srcSize > 256 KB */
- /* W, C, H, S, L, TL, strat */
- { 19, 12, 13, 1, 6, 1, ZSTD_fast }, /* base for negative levels */
- { 19, 13, 14, 1, 7, 0, ZSTD_fast }, /* level 1 */
- { 20, 15, 16, 1, 6, 0, ZSTD_fast }, /* level 2 */
- { 21, 16, 17, 1, 5, 0, ZSTD_dfast }, /* level 3 */
- { 21, 18, 18, 1, 5, 0, ZSTD_dfast }, /* level 4 */
- { 21, 18, 19, 2, 5, 2, ZSTD_greedy }, /* level 5 */
- { 21, 19, 19, 3, 5, 4, ZSTD_greedy }, /* level 6 */
- { 21, 19, 19, 3, 5, 8, ZSTD_lazy }, /* level 7 */
- { 21, 19, 19, 3, 5, 16, ZSTD_lazy2 }, /* level 8 */
- { 21, 19, 20, 4, 5, 16, ZSTD_lazy2 }, /* level 9 */
- { 22, 20, 21, 4, 5, 16, ZSTD_lazy2 }, /* level 10 */
- { 22, 21, 22, 4, 5, 16, ZSTD_lazy2 }, /* level 11 */
- { 22, 21, 22, 5, 5, 16, ZSTD_lazy2 }, /* level 12 */
- { 22, 21, 22, 5, 5, 32, ZSTD_btlazy2 }, /* level 13 */
- { 22, 22, 23, 5, 5, 32, ZSTD_btlazy2 }, /* level 14 */
- { 22, 23, 23, 6, 5, 32, ZSTD_btlazy2 }, /* level 15 */
- { 22, 22, 22, 5, 5, 48, ZSTD_btopt }, /* level 16 */
- { 23, 23, 22, 5, 4, 64, ZSTD_btopt }, /* level 17 */
- { 23, 23, 22, 6, 3, 64, ZSTD_btultra }, /* level 18 */
- { 23, 24, 22, 7, 3,256, ZSTD_btultra2}, /* level 19 */
- { 25, 25, 23, 7, 3,256, ZSTD_btultra2}, /* level 20 */
- { 26, 26, 24, 7, 3,512, ZSTD_btultra2}, /* level 21 */
- { 27, 27, 25, 9, 3,999, ZSTD_btultra2}, /* level 22 */
-},
-{ /* for srcSize <= 256 KB */
- /* W, C, H, S, L, T, strat */
- { 18, 12, 13, 1, 5, 1, ZSTD_fast }, /* base for negative levels */
- { 18, 13, 14, 1, 6, 0, ZSTD_fast }, /* level 1 */
- { 18, 14, 14, 1, 5, 0, ZSTD_dfast }, /* level 2 */
- { 18, 16, 16, 1, 4, 0, ZSTD_dfast }, /* level 3 */
- { 18, 16, 17, 2, 5, 2, ZSTD_greedy }, /* level 4.*/
- { 18, 18, 18, 3, 5, 2, ZSTD_greedy }, /* level 5.*/
- { 18, 18, 19, 3, 5, 4, ZSTD_lazy }, /* level 6.*/
- { 18, 18, 19, 4, 4, 4, ZSTD_lazy }, /* level 7 */
- { 18, 18, 19, 4, 4, 8, ZSTD_lazy2 }, /* level 8 */
- { 18, 18, 19, 5, 4, 8, ZSTD_lazy2 }, /* level 9 */
- { 18, 18, 19, 6, 4, 8, ZSTD_lazy2 }, /* level 10 */
- { 18, 18, 19, 5, 4, 12, ZSTD_btlazy2 }, /* level 11.*/
- { 18, 19, 19, 7, 4, 12, ZSTD_btlazy2 }, /* level 12.*/
- { 18, 18, 19, 4, 4, 16, ZSTD_btopt }, /* level 13 */
- { 18, 18, 19, 4, 3, 32, ZSTD_btopt }, /* level 14.*/
- { 18, 18, 19, 6, 3,128, ZSTD_btopt }, /* level 15.*/
- { 18, 19, 19, 6, 3,128, ZSTD_btultra }, /* level 16.*/
- { 18, 19, 19, 8, 3,256, ZSTD_btultra }, /* level 17.*/
- { 18, 19, 19, 6, 3,128, ZSTD_btultra2}, /* level 18.*/
- { 18, 19, 19, 8, 3,256, ZSTD_btultra2}, /* level 19.*/
- { 18, 19, 19, 10, 3,512, ZSTD_btultra2}, /* level 20.*/
- { 18, 19, 19, 12, 3,512, ZSTD_btultra2}, /* level 21.*/
- { 18, 19, 19, 13, 3,999, ZSTD_btultra2}, /* level 22.*/
-},
-{ /* for srcSize <= 128 KB */
- /* W, C, H, S, L, T, strat */
- { 17, 12, 12, 1, 5, 1, ZSTD_fast }, /* base for negative levels */
- { 17, 12, 13, 1, 6, 0, ZSTD_fast }, /* level 1 */
- { 17, 13, 15, 1, 5, 0, ZSTD_fast }, /* level 2 */
- { 17, 15, 16, 2, 5, 0, ZSTD_dfast }, /* level 3 */
- { 17, 17, 17, 2, 4, 0, ZSTD_dfast }, /* level 4 */
- { 17, 16, 17, 3, 4, 2, ZSTD_greedy }, /* level 5 */
- { 17, 17, 17, 3, 4, 4, ZSTD_lazy }, /* level 6 */
- { 17, 17, 17, 3, 4, 8, ZSTD_lazy2 }, /* level 7 */
- { 17, 17, 17, 4, 4, 8, ZSTD_lazy2 }, /* level 8 */
- { 17, 17, 17, 5, 4, 8, ZSTD_lazy2 }, /* level 9 */
- { 17, 17, 17, 6, 4, 8, ZSTD_lazy2 }, /* level 10 */
- { 17, 17, 17, 5, 4, 8, ZSTD_btlazy2 }, /* level 11 */
- { 17, 18, 17, 7, 4, 12, ZSTD_btlazy2 }, /* level 12 */
- { 17, 18, 17, 3, 4, 12, ZSTD_btopt }, /* level 13.*/
- { 17, 18, 17, 4, 3, 32, ZSTD_btopt }, /* level 14.*/
- { 17, 18, 17, 6, 3,256, ZSTD_btopt }, /* level 15.*/
- { 17, 18, 17, 6, 3,128, ZSTD_btultra }, /* level 16.*/
- { 17, 18, 17, 8, 3,256, ZSTD_btultra }, /* level 17.*/
- { 17, 18, 17, 10, 3,512, ZSTD_btultra }, /* level 18.*/
- { 17, 18, 17, 5, 3,256, ZSTD_btultra2}, /* level 19.*/
- { 17, 18, 17, 7, 3,512, ZSTD_btultra2}, /* level 20.*/
- { 17, 18, 17, 9, 3,512, ZSTD_btultra2}, /* level 21.*/
- { 17, 18, 17, 11, 3,999, ZSTD_btultra2}, /* level 22.*/
-},
-{ /* for srcSize <= 16 KB */
- /* W, C, H, S, L, T, strat */
- { 14, 12, 13, 1, 5, 1, ZSTD_fast }, /* base for negative levels */
- { 14, 14, 15, 1, 5, 0, ZSTD_fast }, /* level 1 */
- { 14, 14, 15, 1, 4, 0, ZSTD_fast }, /* level 2 */
- { 14, 14, 15, 2, 4, 0, ZSTD_dfast }, /* level 3 */
- { 14, 14, 14, 4, 4, 2, ZSTD_greedy }, /* level 4 */
- { 14, 14, 14, 3, 4, 4, ZSTD_lazy }, /* level 5.*/
- { 14, 14, 14, 4, 4, 8, ZSTD_lazy2 }, /* level 6 */
- { 14, 14, 14, 6, 4, 8, ZSTD_lazy2 }, /* level 7 */
- { 14, 14, 14, 8, 4, 8, ZSTD_lazy2 }, /* level 8.*/
- { 14, 15, 14, 5, 4, 8, ZSTD_btlazy2 }, /* level 9.*/
- { 14, 15, 14, 9, 4, 8, ZSTD_btlazy2 }, /* level 10.*/
- { 14, 15, 14, 3, 4, 12, ZSTD_btopt }, /* level 11.*/
- { 14, 15, 14, 4, 3, 24, ZSTD_btopt }, /* level 12.*/
- { 14, 15, 14, 5, 3, 32, ZSTD_btultra }, /* level 13.*/
- { 14, 15, 15, 6, 3, 64, ZSTD_btultra }, /* level 14.*/
- { 14, 15, 15, 7, 3,256, ZSTD_btultra }, /* level 15.*/
- { 14, 15, 15, 5, 3, 48, ZSTD_btultra2}, /* level 16.*/
- { 14, 15, 15, 6, 3,128, ZSTD_btultra2}, /* level 17.*/
- { 14, 15, 15, 7, 3,256, ZSTD_btultra2}, /* level 18.*/
- { 14, 15, 15, 8, 3,256, ZSTD_btultra2}, /* level 19.*/
- { 14, 15, 15, 8, 3,512, ZSTD_btultra2}, /* level 20.*/
- { 14, 15, 15, 9, 3,512, ZSTD_btultra2}, /* level 21.*/
- { 14, 15, 15, 10, 3,999, ZSTD_btultra2}, /* level 22.*/
-},
-};
+int ZSTD_defaultCLevel(void) { return ZSTD_CLEVEL_DEFAULT; }
static ZSTD_compressionParameters ZSTD_dedicatedDictSearch_getCParams(int const compressionLevel, size_t const dictSize)
{
{
return (cParams->strategy >= ZSTD_greedy)
&& (cParams->strategy <= ZSTD_lazy2)
- && (cParams->hashLog >= cParams->chainLog)
+ && (cParams->hashLog > cParams->chainLog)
&& (cParams->chainLog <= 24);
}
case ZSTD_lazy:
case ZSTD_lazy2:
cParams->hashLog -= ZSTD_LAZY_DDSS_BUCKET_LOG;
+ if (cParams->hashLog < ZSTD_HASHLOG_MIN) {
+ cParams->hashLog = ZSTD_HASHLOG_MIN;
+ }
break;
case ZSTD_btlazy2:
case ZSTD_btopt:
else row = compressionLevel;
{ ZSTD_compressionParameters cp = ZSTD_defaultCParameters[tableID][row];
+ DEBUGLOG(5, "ZSTD_getCParams_internal selected tableID: %u row: %u strat: %u", tableID, row, (U32)cp.strategy);
/* acceleration factor */
if (compressionLevel < 0) {
int const clampedCompressionLevel = MAX(ZSTD_minCLevel(), compressionLevel);
} ZSTD_localDict;
typedef struct {
- HUF_CElt CTable[HUF_CTABLE_SIZE_U32(255)];
+ HUF_CElt CTable[HUF_CTABLE_SIZE_ST(255)];
HUF_repeat repeatMode;
} ZSTD_hufCTables_t;
ZSTD_fseCTables_t fse;
} ZSTD_entropyCTables_t;
+/* *********************************************
+* Entropy buffer statistics structs and funcs *
+***********************************************/
+/* ZSTD_hufCTablesMetadata_t :
+ * Stores Literals Block Type for a super-block in hType, and
+ * huffman tree description in hufDesBuffer.
+ * hufDesSize refers to the size of huffman tree description in bytes.
+ * This metadata is populated in ZSTD_buildBlockEntropyStats_literals() */
typedef struct {
- U32 off; /* Offset code (offset + ZSTD_REP_MOVE) for the match */
+ symbolEncodingType_e hType;
+ BYTE hufDesBuffer[ZSTD_MAX_HUF_HEADER_SIZE];
+ size_t hufDesSize;
+} ZSTD_hufCTablesMetadata_t;
+
+/* ZSTD_fseCTablesMetadata_t :
+ * Stores symbol compression modes for a super-block in {ll, ol, ml}Type, and
+ * fse tables in fseTablesBuffer.
+ * fseTablesSize refers to the size of fse tables in bytes.
+ * This metadata is populated in ZSTD_buildBlockEntropyStats_sequences() */
+typedef struct {
+ symbolEncodingType_e llType;
+ symbolEncodingType_e ofType;
+ symbolEncodingType_e mlType;
+ BYTE fseTablesBuffer[ZSTD_MAX_FSE_HEADERS_SIZE];
+ size_t fseTablesSize;
+ size_t lastCountSize; /* This is to account for bug in 1.3.4. More detail in ZSTD_entropyCompressSeqStore_internal() */
+} ZSTD_fseCTablesMetadata_t;
+
+typedef struct {
+ ZSTD_hufCTablesMetadata_t hufMetadata;
+ ZSTD_fseCTablesMetadata_t fseMetadata;
+} ZSTD_entropyCTablesMetadata_t;
+
+/* ZSTD_buildBlockEntropyStats() :
+ * Builds entropy for the block.
+ * @return : 0 on success or error code */
+size_t ZSTD_buildBlockEntropyStats(seqStore_t* seqStorePtr,
+ const ZSTD_entropyCTables_t* prevEntropy,
+ ZSTD_entropyCTables_t* nextEntropy,
+ const ZSTD_CCtx_params* cctxParams,
+ ZSTD_entropyCTablesMetadata_t* entropyMetadata,
+ void* workspace, size_t wkspSize);
+
+/* *******************************
+* Compression internals structs *
+*********************************/
+
+typedef struct {
+ U32 off; /* Offset sumtype code for the match, using ZSTD_storeSeq() format */
U32 len; /* Raw length of match */
} ZSTD_match_t;
U32 offCodeSumBasePrice; /* to compare to log2(offreq) */
ZSTD_OptPrice_e priceType; /* prices can be determined dynamically, or follow a pre-defined cost structure */
const ZSTD_entropyCTables_t* symbolCosts; /* pre-calculated dictionary statistics */
- ZSTD_literalCompressionMode_e literalCompressionMode;
+ ZSTD_paramSwitch_e literalCompressionMode;
} optState_t;
typedef struct {
} ZSTD_compressedBlockState_t;
typedef struct {
- BYTE const* nextSrc; /* next block here to continue on current prefix */
- BYTE const* base; /* All regular indexes relative to this position */
- BYTE const* dictBase; /* extDict indexes relative to this position */
- U32 dictLimit; /* below that point, need extDict */
- U32 lowLimit; /* below that point, no more valid data */
+ BYTE const* nextSrc; /* next block here to continue on current prefix */
+ BYTE const* base; /* All regular indexes relative to this position */
+ BYTE const* dictBase; /* extDict indexes relative to this position */
+ U32 dictLimit; /* below that point, need extDict */
+ U32 lowLimit; /* below that point, no more valid data */
+ U32 nbOverflowCorrections; /* Number of times overflow correction has run since
+ * ZSTD_window_init(). Useful for debugging coredumps
+ * and for ZSTD_WINDOW_OVERFLOW_CORRECT_FREQUENTLY.
+ */
} ZSTD_window_t;
+#define ZSTD_WINDOW_START_INDEX 2
+
typedef struct ZSTD_matchState_t ZSTD_matchState_t;
+
+#define ZSTD_ROW_HASH_CACHE_SIZE 8 /* Size of prefetching hash cache for row-based matchfinder */
+
struct ZSTD_matchState_t {
ZSTD_window_t window; /* State for window round buffer management */
U32 loadedDictEnd; /* index of end of dictionary, within context's referential.
*/
U32 nextToUpdate; /* index from which to continue table update */
U32 hashLog3; /* dispatch table for matches of len==3 : larger == faster, more memory */
+
+ U32 rowHashLog; /* For row-based matchfinder: Hashlog based on nb of rows in the hashTable.*/
+ U16* tagTable; /* For row-based matchFinder: A row-based table containing the hashes and head index. */
+ U32 hashCache[ZSTD_ROW_HASH_CACHE_SIZE]; /* For row-based matchFinder: a cache of hashes to improve speed */
+
U32* hashTable;
U32* hashTable3;
U32* chainTable;
+
+ U32 forceNonContiguous; /* Non-zero if we should force non-contiguous load for the next window update. */
+
int dedicatedDictSearch; /* Indicates whether this matchState is using the
* dedicated dictionary search structure.
*/
} ldmState_t;
typedef struct {
- U32 enableLdm; /* 1 if enable long distance matching */
+ ZSTD_paramSwitch_e enableLdm; /* ZSTD_ps_enable to enable LDM. ZSTD_ps_auto by default */
U32 hashLog; /* Log size of hashTable */
U32 bucketSizeLog; /* Log bucket size for collision resolution, at most 8 */
U32 minMatchLength; /* Minimum match length */
* There is no guarantee that hint is close to actual source size */
ZSTD_dictAttachPref_e attachDictPref;
- ZSTD_literalCompressionMode_e literalCompressionMode;
+ ZSTD_paramSwitch_e literalCompressionMode;
/* Multithreading: used to pass parameters to mtctx */
int nbWorkers;
ZSTD_sequenceFormat_e blockDelimiters;
int validateSequences;
+ /* Block splitting */
+ ZSTD_paramSwitch_e useBlockSplitter;
+
+ /* Param for deciding whether to use row-based matchfinder */
+ ZSTD_paramSwitch_e useRowMatchFinder;
+
+ /* Always load a dictionary in ext-dict mode (not prefix mode)? */
+ int deterministicRefPrefix;
+
/* Internal use, for createCCtxParams() and freeCCtxParams() only */
ZSTD_customMem customMem;
}; /* typedef'd to ZSTD_CCtx_params within "zstd.h" */
ZSTDb_buffered
} ZSTD_buffered_policy_e;
+/*
+ * Struct that contains all elements of block splitter that should be allocated
+ * in a wksp.
+ */
+#define ZSTD_MAX_NB_BLOCK_SPLITS 196
+typedef struct {
+ seqStore_t fullSeqStoreChunk;
+ seqStore_t firstHalfSeqStore;
+ seqStore_t secondHalfSeqStore;
+ seqStore_t currSeqStore;
+ seqStore_t nextSeqStore;
+
+ U32 partitions[ZSTD_MAX_NB_BLOCK_SPLITS];
+ ZSTD_entropyCTablesMetadata_t entropyMetadata;
+} ZSTD_blockSplitCtx;
+
struct ZSTD_CCtx_s {
ZSTD_compressionStage_e stage;
int cParamsChanged; /* == 1 if cParams(except wlog) or compression level are changed in requestedParams. Triggers transmission of new params to ZSTDMT (if available) then reset to 0. */
int bmi2; /* == 1 if the CPU supports BMI2 and 0 otherwise. CPU support is determined dynamically once per context lifetime. */
ZSTD_CCtx_params requestedParams;
ZSTD_CCtx_params appliedParams;
+ ZSTD_CCtx_params simpleApiParams; /* Param storage used by the simple API - not sticky. Must only be used in top-level simple API functions for storage. */
U32 dictID;
size_t dictContentSize;
ZSTD_blockState_t blockState;
U32* entropyWorkspace; /* entropy workspace of ENTROPY_WORKSPACE_SIZE bytes */
- /* Wether we are streaming or not */
+ /* Whether we are streaming or not */
ZSTD_buffered_policy_e bufferedPolicy;
/* streaming */
/* Multi-threading */
/* Tracing */
+
+ /* Workspace for block splitter */
+ ZSTD_blockSplitCtx blockSplitCtx;
};
typedef enum { ZSTD_dtlm_fast, ZSTD_dtlm_full } ZSTD_dictTableLoadMethod_e;
typedef size_t (*ZSTD_blockCompressor) (
ZSTD_matchState_t* bs, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
void const* src, size_t srcSize);
-ZSTD_blockCompressor ZSTD_selectBlockCompressor(ZSTD_strategy strat, ZSTD_dictMode_e dictMode);
+ZSTD_blockCompressor ZSTD_selectBlockCompressor(ZSTD_strategy strat, ZSTD_paramSwitch_e rowMatchfinderMode, ZSTD_dictMode_e dictMode);
MEM_STATIC U32 ZSTD_LLcode(U32 litLength)
return (mlBase > 127) ? ZSTD_highbit32(mlBase) + ML_deltaCode : ML_Code[mlBase];
}
-typedef struct repcodes_s {
- U32 rep[3];
-} repcodes_t;
-
-MEM_STATIC repcodes_t ZSTD_updateRep(U32 const rep[3], U32 const offset, U32 const ll0)
-{
- repcodes_t newReps;
- if (offset >= ZSTD_REP_NUM) { /* full offset */
- newReps.rep[2] = rep[1];
- newReps.rep[1] = rep[0];
- newReps.rep[0] = offset - ZSTD_REP_MOVE;
- } else { /* repcode */
- U32 const repCode = offset + ll0;
- if (repCode > 0) { /* note : if repCode==0, no change */
- U32 const currentOffset = (repCode==ZSTD_REP_NUM) ? (rep[0] - 1) : rep[repCode];
- newReps.rep[2] = (repCode >= 2) ? rep[1] : rep[2];
- newReps.rep[1] = rep[0];
- newReps.rep[0] = currentOffset;
- } else { /* repCode == 0 */
- ZSTD_memcpy(&newReps, rep, sizeof(newReps));
- }
- }
- return newReps;
-}
-
/* ZSTD_cParam_withinBounds:
* @return 1 if value is within cParam bounds,
* 0 otherwise */
return (srcSize >> minlog) + 2;
}
-MEM_STATIC int ZSTD_disableLiteralsCompression(const ZSTD_CCtx_params* cctxParams)
+MEM_STATIC int ZSTD_literalsCompressionIsDisabled(const ZSTD_CCtx_params* cctxParams)
{
switch (cctxParams->literalCompressionMode) {
- case ZSTD_lcm_huffman:
+ case ZSTD_ps_enable:
return 0;
- case ZSTD_lcm_uncompressed:
+ case ZSTD_ps_disable:
return 1;
default:
assert(0 /* impossible: pre-validated */);
ZSTD_FALLTHROUGH;
- case ZSTD_lcm_auto:
+ case ZSTD_ps_auto:
return (cctxParams->cParams.strategy == ZSTD_fast) && (cctxParams->cParams.targetLength > 0);
}
}
* Only called when the sequence ends past ilimit_w, so it only needs to be optimized for single
* large copies.
*/
-static void ZSTD_safecopyLiterals(BYTE* op, BYTE const* ip, BYTE const* const iend, BYTE const* ilimit_w) {
+static void
+ZSTD_safecopyLiterals(BYTE* op, BYTE const* ip, BYTE const* const iend, BYTE const* ilimit_w)
+{
assert(iend > ilimit_w);
if (ip <= ilimit_w) {
ZSTD_wildcopy(op, ip, ilimit_w - ip, ZSTD_no_overlap);
while (ip < iend) *op++ = *ip++;
}
+#define ZSTD_REP_MOVE (ZSTD_REP_NUM-1)
+#define STORE_REPCODE_1 STORE_REPCODE(1)
+#define STORE_REPCODE_2 STORE_REPCODE(2)
+#define STORE_REPCODE_3 STORE_REPCODE(3)
+#define STORE_REPCODE(r) (assert((r)>=1), assert((r)<=3), (r)-1)
+#define STORE_OFFSET(o) (assert((o)>0), o + ZSTD_REP_MOVE)
+#define STORED_IS_OFFSET(o) ((o) > ZSTD_REP_MOVE)
+#define STORED_IS_REPCODE(o) ((o) <= ZSTD_REP_MOVE)
+#define STORED_OFFSET(o) (assert(STORED_IS_OFFSET(o)), (o)-ZSTD_REP_MOVE)
+#define STORED_REPCODE(o) (assert(STORED_IS_REPCODE(o)), (o)+1) /* returns ID 1,2,3 */
+#define STORED_TO_OFFBASE(o) ((o)+1)
+#define OFFBASE_TO_STORED(o) ((o)-1)
+
/*! ZSTD_storeSeq() :
- * Store a sequence (litlen, litPtr, offCode and mlBase) into seqStore_t.
- * `offCode` : distance to match + ZSTD_REP_MOVE (values <= ZSTD_REP_MOVE are repCodes).
- * `mlBase` : matchLength - MINMATCH
+ * Store a sequence (litlen, litPtr, offCode and matchLength) into seqStore_t.
+ * @offBase_minus1 : Users should use employ macros STORE_REPCODE_X and STORE_OFFSET().
+ * @matchLength : must be >= MINMATCH
* Allowed to overread literals up to litLimit.
*/
-HINT_INLINE UNUSED_ATTR
-void ZSTD_storeSeq(seqStore_t* seqStorePtr, size_t litLength, const BYTE* literals, const BYTE* litLimit, U32 offCode, size_t mlBase)
+HINT_INLINE UNUSED_ATTR void
+ZSTD_storeSeq(seqStore_t* seqStorePtr,
+ size_t litLength, const BYTE* literals, const BYTE* litLimit,
+ U32 offBase_minus1,
+ size_t matchLength)
{
BYTE const* const litLimit_w = litLimit - WILDCOPY_OVERLENGTH;
BYTE const* const litEnd = literals + litLength;
if (g_start==NULL) g_start = (const BYTE*)literals; /* note : index only works for compression within a single segment */
{ U32 const pos = (U32)((const BYTE*)literals - g_start);
DEBUGLOG(6, "Cpos%7u :%3u literals, match%4u bytes at offCode%7u",
- pos, (U32)litLength, (U32)mlBase+MINMATCH, (U32)offCode);
+ pos, (U32)litLength, (U32)matchLength, (U32)offBase_minus1);
}
#endif
assert((size_t)(seqStorePtr->sequences - seqStorePtr->sequencesStart) < seqStorePtr->maxNbSeq);
/* literal Length */
if (litLength>0xFFFF) {
- assert(seqStorePtr->longLengthID == 0); /* there can only be a single long length */
- seqStorePtr->longLengthID = 1;
+ assert(seqStorePtr->longLengthType == ZSTD_llt_none); /* there can only be a single long length */
+ seqStorePtr->longLengthType = ZSTD_llt_literalLength;
seqStorePtr->longLengthPos = (U32)(seqStorePtr->sequences - seqStorePtr->sequencesStart);
}
seqStorePtr->sequences[0].litLength = (U16)litLength;
/* match offset */
- seqStorePtr->sequences[0].offset = offCode + 1;
+ seqStorePtr->sequences[0].offBase = STORED_TO_OFFBASE(offBase_minus1);
/* match Length */
- if (mlBase>0xFFFF) {
- assert(seqStorePtr->longLengthID == 0); /* there can only be a single long length */
- seqStorePtr->longLengthID = 2;
- seqStorePtr->longLengthPos = (U32)(seqStorePtr->sequences - seqStorePtr->sequencesStart);
+ assert(matchLength >= MINMATCH);
+ { size_t const mlBase = matchLength - MINMATCH;
+ if (mlBase>0xFFFF) {
+ assert(seqStorePtr->longLengthType == ZSTD_llt_none); /* there can only be a single long length */
+ seqStorePtr->longLengthType = ZSTD_llt_matchLength;
+ seqStorePtr->longLengthPos = (U32)(seqStorePtr->sequences - seqStorePtr->sequencesStart);
+ }
+ seqStorePtr->sequences[0].mlBase = (U16)mlBase;
}
- seqStorePtr->sequences[0].matchLength = (U16)mlBase;
seqStorePtr->sequences++;
}
+/* ZSTD_updateRep() :
+ * updates in-place @rep (array of repeat offsets)
+ * @offBase_minus1 : sum-type, with same numeric representation as ZSTD_storeSeq()
+ */
+MEM_STATIC void
+ZSTD_updateRep(U32 rep[ZSTD_REP_NUM], U32 const offBase_minus1, U32 const ll0)
+{
+ if (STORED_IS_OFFSET(offBase_minus1)) { /* full offset */
+ rep[2] = rep[1];
+ rep[1] = rep[0];
+ rep[0] = STORED_OFFSET(offBase_minus1);
+ } else { /* repcode */
+ U32 const repCode = STORED_REPCODE(offBase_minus1) - 1 + ll0;
+ if (repCode > 0) { /* note : if repCode==0, no change */
+ U32 const currentOffset = (repCode==ZSTD_REP_NUM) ? (rep[0] - 1) : rep[repCode];
+ rep[2] = (repCode >= 2) ? rep[1] : rep[2];
+ rep[1] = rep[0];
+ rep[0] = currentOffset;
+ } else { /* repCode == 0 */
+ /* nothing to do */
+ }
+ }
+}
+
+typedef struct repcodes_s {
+ U32 rep[3];
+} repcodes_t;
+
+MEM_STATIC repcodes_t
+ZSTD_newRep(U32 const rep[ZSTD_REP_NUM], U32 const offBase_minus1, U32 const ll0)
+{
+ repcodes_t newReps;
+ ZSTD_memcpy(&newReps, rep, sizeof(newReps));
+ ZSTD_updateRep(newReps.rep, offBase_minus1, ll0);
+ return newReps;
+}
+
/*-*************************************
* Match length counter
window->dictLimit = end;
}
+MEM_STATIC U32 ZSTD_window_isEmpty(ZSTD_window_t const window)
+{
+ return window.dictLimit == ZSTD_WINDOW_START_INDEX &&
+ window.lowLimit == ZSTD_WINDOW_START_INDEX &&
+ (window.nextSrc - window.base) == ZSTD_WINDOW_START_INDEX;
+}
+
/*
* ZSTD_window_hasExtDict():
* Returns non-zero if the window has a non-empty extDict.
ZSTD_noDict;
}
+/* Defining this macro to non-zero tells zstd to run the overflow correction
+ * code much more frequently. This is very inefficient, and should only be
+ * used for tests and fuzzers.
+ */
+#ifndef ZSTD_WINDOW_OVERFLOW_CORRECT_FREQUENTLY
+# ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
+# define ZSTD_WINDOW_OVERFLOW_CORRECT_FREQUENTLY 1
+# else
+# define ZSTD_WINDOW_OVERFLOW_CORRECT_FREQUENTLY 0
+# endif
+#endif
+
+/*
+ * ZSTD_window_canOverflowCorrect():
+ * Returns non-zero if the indices are large enough for overflow correction
+ * to work correctly without impacting compression ratio.
+ */
+MEM_STATIC U32 ZSTD_window_canOverflowCorrect(ZSTD_window_t const window,
+ U32 cycleLog,
+ U32 maxDist,
+ U32 loadedDictEnd,
+ void const* src)
+{
+ U32 const cycleSize = 1u << cycleLog;
+ U32 const curr = (U32)((BYTE const*)src - window.base);
+ U32 const minIndexToOverflowCorrect = cycleSize
+ + MAX(maxDist, cycleSize)
+ + ZSTD_WINDOW_START_INDEX;
+
+ /* Adjust the min index to backoff the overflow correction frequency,
+ * so we don't waste too much CPU in overflow correction. If this
+ * computation overflows we don't really care, we just need to make
+ * sure it is at least minIndexToOverflowCorrect.
+ */
+ U32 const adjustment = window.nbOverflowCorrections + 1;
+ U32 const adjustedIndex = MAX(minIndexToOverflowCorrect * adjustment,
+ minIndexToOverflowCorrect);
+ U32 const indexLargeEnough = curr > adjustedIndex;
+
+ /* Only overflow correct early if the dictionary is invalidated already,
+ * so we don't hurt compression ratio.
+ */
+ U32 const dictionaryInvalidated = curr > maxDist + loadedDictEnd;
+
+ return indexLargeEnough && dictionaryInvalidated;
+}
+
/*
* ZSTD_window_needOverflowCorrection():
* Returns non-zero if the indices are getting too large and need overflow
* protection.
*/
MEM_STATIC U32 ZSTD_window_needOverflowCorrection(ZSTD_window_t const window,
+ U32 cycleLog,
+ U32 maxDist,
+ U32 loadedDictEnd,
+ void const* src,
void const* srcEnd)
{
U32 const curr = (U32)((BYTE const*)srcEnd - window.base);
+ if (ZSTD_WINDOW_OVERFLOW_CORRECT_FREQUENTLY) {
+ if (ZSTD_window_canOverflowCorrect(window, cycleLog, maxDist, loadedDictEnd, src)) {
+ return 1;
+ }
+ }
return curr > ZSTD_CURRENT_MAX;
}
*
* The least significant cycleLog bits of the indices must remain the same,
* which may be 0. Every index up to maxDist in the past must be valid.
- * NOTE: (maxDist & cycleMask) must be zero.
*/
MEM_STATIC U32 ZSTD_window_correctOverflow(ZSTD_window_t* window, U32 cycleLog,
U32 maxDist, void const* src)
* 3. (cctx->lowLimit + 1<<windowLog) < 1<<32:
* windowLog <= 31 ==> 3<<29 + 1<<windowLog < 7<<29 < 1<<32.
*/
- U32 const cycleMask = (1U << cycleLog) - 1;
+ U32 const cycleSize = 1u << cycleLog;
+ U32 const cycleMask = cycleSize - 1;
U32 const curr = (U32)((BYTE const*)src - window->base);
- U32 const currentCycle0 = curr & cycleMask;
- /* Exclude zero so that newCurrent - maxDist >= 1. */
- U32 const currentCycle1 = currentCycle0 == 0 ? (1U << cycleLog) : currentCycle0;
- U32 const newCurrent = currentCycle1 + maxDist;
+ U32 const currentCycle = curr & cycleMask;
+ /* Ensure newCurrent - maxDist >= ZSTD_WINDOW_START_INDEX. */
+ U32 const currentCycleCorrection = currentCycle < ZSTD_WINDOW_START_INDEX
+ ? MAX(cycleSize, ZSTD_WINDOW_START_INDEX)
+ : 0;
+ U32 const newCurrent = currentCycle
+ + currentCycleCorrection
+ + MAX(maxDist, cycleSize);
U32 const correction = curr - newCurrent;
- assert((maxDist & cycleMask) == 0);
+ /* maxDist must be a power of two so that:
+ * (newCurrent & cycleMask) == (curr & cycleMask)
+ * This is required to not corrupt the chains / binary tree.
+ */
+ assert((maxDist & (maxDist - 1)) == 0);
+ assert((curr & cycleMask) == (newCurrent & cycleMask));
assert(curr > newCurrent);
- /* Loose bound, should be around 1<<29 (see above) */
- assert(correction > 1<<28);
+ if (!ZSTD_WINDOW_OVERFLOW_CORRECT_FREQUENTLY) {
+ /* Loose bound, should be around 1<<29 (see above) */
+ assert(correction > 1<<28);
+ }
window->base += correction;
window->dictBase += correction;
- if (window->lowLimit <= correction) window->lowLimit = 1;
- else window->lowLimit -= correction;
- if (window->dictLimit <= correction) window->dictLimit = 1;
- else window->dictLimit -= correction;
+ if (window->lowLimit < correction + ZSTD_WINDOW_START_INDEX) {
+ window->lowLimit = ZSTD_WINDOW_START_INDEX;
+ } else {
+ window->lowLimit -= correction;
+ }
+ if (window->dictLimit < correction + ZSTD_WINDOW_START_INDEX) {
+ window->dictLimit = ZSTD_WINDOW_START_INDEX;
+ } else {
+ window->dictLimit -= correction;
+ }
/* Ensure we can still reference the full window. */
assert(newCurrent >= maxDist);
- assert(newCurrent - maxDist >= 1);
+ assert(newCurrent - maxDist >= ZSTD_WINDOW_START_INDEX);
/* Ensure that lowLimit and dictLimit didn't underflow. */
assert(window->lowLimit <= newCurrent);
assert(window->dictLimit <= newCurrent);
+ ++window->nbOverflowCorrections;
+
DEBUGLOG(4, "Correction of 0x%x bytes to lowLimit=0x%x", correction,
window->lowLimit);
return correction;
MEM_STATIC void ZSTD_window_init(ZSTD_window_t* window) {
ZSTD_memset(window, 0, sizeof(*window));
- window->base = (BYTE const*)"";
- window->dictBase = (BYTE const*)"";
- window->dictLimit = 1; /* start from 1, so that 1st position is valid */
- window->lowLimit = 1; /* it ensures first and later CCtx usages compress the same */
- window->nextSrc = window->base + 1; /* see issue #1241 */
+ window->base = (BYTE const*)" ";
+ window->dictBase = (BYTE const*)" ";
+ ZSTD_STATIC_ASSERT(ZSTD_DUBT_UNSORTED_MARK < ZSTD_WINDOW_START_INDEX); /* Start above ZSTD_DUBT_UNSORTED_MARK */
+ window->dictLimit = ZSTD_WINDOW_START_INDEX; /* start from >0, so that 1st position is valid */
+ window->lowLimit = ZSTD_WINDOW_START_INDEX; /* it ensures first and later CCtx usages compress the same */
+ window->nextSrc = window->base + ZSTD_WINDOW_START_INDEX; /* see issue #1241 */
+ window->nbOverflowCorrections = 0;
}
/*
* Returns non-zero if the segment is contiguous.
*/
MEM_STATIC U32 ZSTD_window_update(ZSTD_window_t* window,
- void const* src, size_t srcSize)
+ void const* src, size_t srcSize,
+ int forceNonContiguous)
{
BYTE const* const ip = (BYTE const*)src;
U32 contiguous = 1;
assert(window->base != NULL);
assert(window->dictBase != NULL);
/* Check if blocks follow each other */
- if (src != window->nextSrc) {
+ if (src != window->nextSrc || forceNonContiguous) {
/* not contiguous */
size_t const distanceFromBase = (size_t)(window->nextSrc - window->base);
DEBUGLOG(5, "Non contiguous blocks, new segment starts at %u", window->dictLimit);
*/
MEM_STATIC U32 ZSTD_getLowestMatchIndex(const ZSTD_matchState_t* ms, U32 curr, unsigned windowLog)
{
- U32 const maxDistance = 1U << windowLog;
- U32 const lowestValid = ms->window.lowLimit;
- U32 const withinWindow = (curr - lowestValid > maxDistance) ? curr - maxDistance : lowestValid;
- U32 const isDictionary = (ms->loadedDictEnd != 0);
+ U32 const maxDistance = 1U << windowLog;
+ U32 const lowestValid = ms->window.lowLimit;
+ U32 const withinWindow = (curr - lowestValid > maxDistance) ? curr - maxDistance : lowestValid;
+ U32 const isDictionary = (ms->loadedDictEnd != 0);
/* When using a dictionary the entire dictionary is valid if a single byte of the dictionary
* is within the window. We invalidate the dictionary (and set loadedDictEnd to 0) when it isn't
* valid for the entire block. So this check is sufficient to find the lowest valid match index.
*/
- U32 const matchLowest = isDictionary ? lowestValid : withinWindow;
+ U32 const matchLowest = isDictionary ? lowestValid : withinWindow;
return matchLowest;
}
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
void* entropyWorkspace, size_t entropyWorkspaceSize,
- const int bmi2)
+ const int bmi2,
+ unsigned suspectUncompressible)
{
size_t const minGain = ZSTD_minGain(srcSize, strategy);
size_t const lhSize = 3 + (srcSize >= 1 KB) + (srcSize >= 16 KB);
HUF_compress1X_repeat(
ostart+lhSize, dstCapacity-lhSize, src, srcSize,
HUF_SYMBOLVALUE_MAX, HUF_TABLELOG_DEFAULT, entropyWorkspace, entropyWorkspaceSize,
- (HUF_CElt*)nextHuf->CTable, &repeat, preferRepeat, bmi2) :
+ (HUF_CElt*)nextHuf->CTable, &repeat, preferRepeat, bmi2, suspectUncompressible) :
HUF_compress4X_repeat(
ostart+lhSize, dstCapacity-lhSize, src, srcSize,
HUF_SYMBOLVALUE_MAX, HUF_TABLELOG_DEFAULT, entropyWorkspace, entropyWorkspaceSize,
- (HUF_CElt*)nextHuf->CTable, &repeat, preferRepeat, bmi2);
+ (HUF_CElt*)nextHuf->CTable, &repeat, preferRepeat, bmi2, suspectUncompressible);
if (repeat != HUF_repeat_none) {
/* reused the existing table */
DEBUGLOG(5, "Reusing previous huffman table");
}
}
- if ((cLitSize==0) | (cLitSize >= srcSize - minGain) | ERR_isError(cLitSize)) {
+ if ((cLitSize==0) || (cLitSize >= srcSize - minGain) || ERR_isError(cLitSize)) {
ZSTD_memcpy(nextHuf, prevHuf, sizeof(*prevHuf));
return ZSTD_noCompressLiterals(dst, dstCapacity, src, srcSize);
}
size_t ZSTD_compressRleLiteralsBlock (void* dst, size_t dstCapacity, const void* src, size_t srcSize);
+/* If suspectUncompressible then some sampling checks will be run to potentially skip huffman coding */
size_t ZSTD_compressLiterals (ZSTD_hufCTables_t const* prevHuf,
ZSTD_hufCTables_t* nextHuf,
ZSTD_strategy strategy, int disableLiteralCompression,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
void* entropyWorkspace, size_t entropyWorkspaceSize,
- const int bmi2);
+ const int bmi2,
+ unsigned suspectUncompressible);
#endif /* ZSTD_COMPRESS_LITERALS_H */
{
unsigned cost = 0;
unsigned s;
+
+ assert(total > 0);
for (s = 0; s <= max; ++s) {
unsigned norm = (unsigned)((256 * count[s]) / total);
if (count[s] != 0 && norm == 0)
assert(nbSeq_1 > 1);
assert(entropyWorkspaceSize >= sizeof(ZSTD_BuildCTableWksp));
(void)entropyWorkspaceSize;
- FORWARD_IF_ERROR(FSE_normalizeCount(wksp->norm, tableLog, count, nbSeq_1, max, ZSTD_useLowProbCount(nbSeq_1)), "");
- { size_t const NCountSize = FSE_writeNCount(op, oend - op, wksp->norm, max, tableLog); /* overflow protected */
+ FORWARD_IF_ERROR(FSE_normalizeCount(wksp->norm, tableLog, count, nbSeq_1, max, ZSTD_useLowProbCount(nbSeq_1)), "FSE_normalizeCount failed");
+ assert(oend >= op);
+ { size_t const NCountSize = FSE_writeNCount(op, (size_t)(oend - op), wksp->norm, max, tableLog); /* overflow protected */
FORWARD_IF_ERROR(NCountSize, "FSE_writeNCount failed");
- FORWARD_IF_ERROR(FSE_buildCTable_wksp(nextCTable, wksp->norm, max, tableLog, wksp->wksp, sizeof(wksp->wksp)), "");
+ FORWARD_IF_ERROR(FSE_buildCTable_wksp(nextCTable, wksp->norm, max, tableLog, wksp->wksp, sizeof(wksp->wksp)), "FSE_buildCTable_wksp failed");
return NCountSize;
}
}
FSE_initCState2(&stateLitLength, CTable_LitLength, llCodeTable[nbSeq-1]);
BIT_addBits(&blockStream, sequences[nbSeq-1].litLength, LL_bits[llCodeTable[nbSeq-1]]);
if (MEM_32bits()) BIT_flushBits(&blockStream);
- BIT_addBits(&blockStream, sequences[nbSeq-1].matchLength, ML_bits[mlCodeTable[nbSeq-1]]);
+ BIT_addBits(&blockStream, sequences[nbSeq-1].mlBase, ML_bits[mlCodeTable[nbSeq-1]]);
if (MEM_32bits()) BIT_flushBits(&blockStream);
if (longOffsets) {
U32 const ofBits = ofCodeTable[nbSeq-1];
unsigned const extraBits = ofBits - MIN(ofBits, STREAM_ACCUMULATOR_MIN-1);
if (extraBits) {
- BIT_addBits(&blockStream, sequences[nbSeq-1].offset, extraBits);
+ BIT_addBits(&blockStream, sequences[nbSeq-1].offBase, extraBits);
BIT_flushBits(&blockStream);
}
- BIT_addBits(&blockStream, sequences[nbSeq-1].offset >> extraBits,
+ BIT_addBits(&blockStream, sequences[nbSeq-1].offBase >> extraBits,
ofBits - extraBits);
} else {
- BIT_addBits(&blockStream, sequences[nbSeq-1].offset, ofCodeTable[nbSeq-1]);
+ BIT_addBits(&blockStream, sequences[nbSeq-1].offBase, ofCodeTable[nbSeq-1]);
}
BIT_flushBits(&blockStream);
U32 const mlBits = ML_bits[mlCode];
DEBUGLOG(6, "encoding: litlen:%2u - matchlen:%2u - offCode:%7u",
(unsigned)sequences[n].litLength,
- (unsigned)sequences[n].matchLength + MINMATCH,
- (unsigned)sequences[n].offset);
+ (unsigned)sequences[n].mlBase + MINMATCH,
+ (unsigned)sequences[n].offBase);
/* 32b*/ /* 64b*/
/* (7)*/ /* (7)*/
FSE_encodeSymbol(&blockStream, &stateOffsetBits, ofCode); /* 15 */ /* 15 */
BIT_flushBits(&blockStream); /* (7)*/
BIT_addBits(&blockStream, sequences[n].litLength, llBits);
if (MEM_32bits() && ((llBits+mlBits)>24)) BIT_flushBits(&blockStream);
- BIT_addBits(&blockStream, sequences[n].matchLength, mlBits);
+ BIT_addBits(&blockStream, sequences[n].mlBase, mlBits);
if (MEM_32bits() || (ofBits+mlBits+llBits > 56)) BIT_flushBits(&blockStream);
if (longOffsets) {
unsigned const extraBits = ofBits - MIN(ofBits, STREAM_ACCUMULATOR_MIN-1);
if (extraBits) {
- BIT_addBits(&blockStream, sequences[n].offset, extraBits);
+ BIT_addBits(&blockStream, sequences[n].offBase, extraBits);
BIT_flushBits(&blockStream); /* (7)*/
}
- BIT_addBits(&blockStream, sequences[n].offset >> extraBits,
+ BIT_addBits(&blockStream, sequences[n].offBase >> extraBits,
ofBits - extraBits); /* 31 */
} else {
- BIT_addBits(&blockStream, sequences[n].offset, ofBits); /* 31 */
+ BIT_addBits(&blockStream, sequences[n].offBase, ofBits); /* 31 */
}
BIT_flushBits(&blockStream); /* (7)*/
DEBUGLOG(7, "remaining space : %i", (int)(blockStream.endPtr - blockStream.ptr));
#if DYNAMIC_BMI2
-static TARGET_ATTRIBUTE("bmi2") size_t
+static BMI2_TARGET_ATTRIBUTE size_t
ZSTD_encodeSequences_bmi2(
void* dst, size_t dstCapacity,
FSE_CTable const* CTable_MatchLength, BYTE const* mlCodeTable,
#include "../common/zstd_internal.h" /* ZSTD_getSequenceLength */
#include "hist.h" /* HIST_countFast_wksp */
-#include "zstd_compress_internal.h"
+#include "zstd_compress_internal.h" /* ZSTD_[huf|fse|entropy]CTablesMetadata_t */
#include "zstd_compress_sequences.h"
#include "zstd_compress_literals.h"
-/*-*************************************
-* Superblock entropy buffer structs
-***************************************/
-/* ZSTD_hufCTablesMetadata_t :
- * Stores Literals Block Type for a super-block in hType, and
- * huffman tree description in hufDesBuffer.
- * hufDesSize refers to the size of huffman tree description in bytes.
- * This metadata is populated in ZSTD_buildSuperBlockEntropy_literal() */
-typedef struct {
- symbolEncodingType_e hType;
- BYTE hufDesBuffer[ZSTD_MAX_HUF_HEADER_SIZE];
- size_t hufDesSize;
-} ZSTD_hufCTablesMetadata_t;
-
-/* ZSTD_fseCTablesMetadata_t :
- * Stores symbol compression modes for a super-block in {ll, ol, ml}Type, and
- * fse tables in fseTablesBuffer.
- * fseTablesSize refers to the size of fse tables in bytes.
- * This metadata is populated in ZSTD_buildSuperBlockEntropy_sequences() */
-typedef struct {
- symbolEncodingType_e llType;
- symbolEncodingType_e ofType;
- symbolEncodingType_e mlType;
- BYTE fseTablesBuffer[ZSTD_MAX_FSE_HEADERS_SIZE];
- size_t fseTablesSize;
- size_t lastCountSize; /* This is to account for bug in 1.3.4. More detail in ZSTD_compressSubBlock_sequences() */
-} ZSTD_fseCTablesMetadata_t;
-
-typedef struct {
- ZSTD_hufCTablesMetadata_t hufMetadata;
- ZSTD_fseCTablesMetadata_t fseMetadata;
-} ZSTD_entropyCTablesMetadata_t;
-
-
-/* ZSTD_buildSuperBlockEntropy_literal() :
- * Builds entropy for the super-block literals.
- * Stores literals block type (raw, rle, compressed, repeat) and
- * huffman description table to hufMetadata.
- * @return : size of huffman description table or error code */
-static size_t ZSTD_buildSuperBlockEntropy_literal(void* const src, size_t srcSize,
- const ZSTD_hufCTables_t* prevHuf,
- ZSTD_hufCTables_t* nextHuf,
- ZSTD_hufCTablesMetadata_t* hufMetadata,
- const int disableLiteralsCompression,
- void* workspace, size_t wkspSize)
-{
- BYTE* const wkspStart = (BYTE*)workspace;
- BYTE* const wkspEnd = wkspStart + wkspSize;
- BYTE* const countWkspStart = wkspStart;
- unsigned* const countWksp = (unsigned*)workspace;
- const size_t countWkspSize = (HUF_SYMBOLVALUE_MAX + 1) * sizeof(unsigned);
- BYTE* const nodeWksp = countWkspStart + countWkspSize;
- const size_t nodeWkspSize = wkspEnd-nodeWksp;
- unsigned maxSymbolValue = 255;
- unsigned huffLog = HUF_TABLELOG_DEFAULT;
- HUF_repeat repeat = prevHuf->repeatMode;
-
- DEBUGLOG(5, "ZSTD_buildSuperBlockEntropy_literal (srcSize=%zu)", srcSize);
-
- /* Prepare nextEntropy assuming reusing the existing table */
- ZSTD_memcpy(nextHuf, prevHuf, sizeof(*prevHuf));
-
- if (disableLiteralsCompression) {
- DEBUGLOG(5, "set_basic - disabled");
- hufMetadata->hType = set_basic;
- return 0;
- }
-
- /* small ? don't even attempt compression (speed opt) */
-# define COMPRESS_LITERALS_SIZE_MIN 63
- { size_t const minLitSize = (prevHuf->repeatMode == HUF_repeat_valid) ? 6 : COMPRESS_LITERALS_SIZE_MIN;
- if (srcSize <= minLitSize) {
- DEBUGLOG(5, "set_basic - too small");
- hufMetadata->hType = set_basic;
- return 0;
- }
- }
-
- /* Scan input and build symbol stats */
- { size_t const largest = HIST_count_wksp (countWksp, &maxSymbolValue, (const BYTE*)src, srcSize, workspace, wkspSize);
- FORWARD_IF_ERROR(largest, "HIST_count_wksp failed");
- if (largest == srcSize) {
- DEBUGLOG(5, "set_rle");
- hufMetadata->hType = set_rle;
- return 0;
- }
- if (largest <= (srcSize >> 7)+4) {
- DEBUGLOG(5, "set_basic - no gain");
- hufMetadata->hType = set_basic;
- return 0;
- }
- }
-
- /* Validate the previous Huffman table */
- if (repeat == HUF_repeat_check && !HUF_validateCTable((HUF_CElt const*)prevHuf->CTable, countWksp, maxSymbolValue)) {
- repeat = HUF_repeat_none;
- }
-
- /* Build Huffman Tree */
- ZSTD_memset(nextHuf->CTable, 0, sizeof(nextHuf->CTable));
- huffLog = HUF_optimalTableLog(huffLog, srcSize, maxSymbolValue);
- { size_t const maxBits = HUF_buildCTable_wksp((HUF_CElt*)nextHuf->CTable, countWksp,
- maxSymbolValue, huffLog,
- nodeWksp, nodeWkspSize);
- FORWARD_IF_ERROR(maxBits, "HUF_buildCTable_wksp");
- huffLog = (U32)maxBits;
- { /* Build and write the CTable */
- size_t const newCSize = HUF_estimateCompressedSize(
- (HUF_CElt*)nextHuf->CTable, countWksp, maxSymbolValue);
- size_t const hSize = HUF_writeCTable_wksp(
- hufMetadata->hufDesBuffer, sizeof(hufMetadata->hufDesBuffer),
- (HUF_CElt*)nextHuf->CTable, maxSymbolValue, huffLog,
- nodeWksp, nodeWkspSize);
- /* Check against repeating the previous CTable */
- if (repeat != HUF_repeat_none) {
- size_t const oldCSize = HUF_estimateCompressedSize(
- (HUF_CElt const*)prevHuf->CTable, countWksp, maxSymbolValue);
- if (oldCSize < srcSize && (oldCSize <= hSize + newCSize || hSize + 12 >= srcSize)) {
- DEBUGLOG(5, "set_repeat - smaller");
- ZSTD_memcpy(nextHuf, prevHuf, sizeof(*prevHuf));
- hufMetadata->hType = set_repeat;
- return 0;
- }
- }
- if (newCSize + hSize >= srcSize) {
- DEBUGLOG(5, "set_basic - no gains");
- ZSTD_memcpy(nextHuf, prevHuf, sizeof(*prevHuf));
- hufMetadata->hType = set_basic;
- return 0;
- }
- DEBUGLOG(5, "set_compressed (hSize=%u)", (U32)hSize);
- hufMetadata->hType = set_compressed;
- nextHuf->repeatMode = HUF_repeat_check;
- return hSize;
- }
- }
-}
-
-/* ZSTD_buildSuperBlockEntropy_sequences() :
- * Builds entropy for the super-block sequences.
- * Stores symbol compression modes and fse table to fseMetadata.
- * @return : size of fse tables or error code */
-static size_t ZSTD_buildSuperBlockEntropy_sequences(seqStore_t* seqStorePtr,
- const ZSTD_fseCTables_t* prevEntropy,
- ZSTD_fseCTables_t* nextEntropy,
- const ZSTD_CCtx_params* cctxParams,
- ZSTD_fseCTablesMetadata_t* fseMetadata,
- void* workspace, size_t wkspSize)
-{
- BYTE* const wkspStart = (BYTE*)workspace;
- BYTE* const wkspEnd = wkspStart + wkspSize;
- BYTE* const countWkspStart = wkspStart;
- unsigned* const countWksp = (unsigned*)workspace;
- const size_t countWkspSize = (MaxSeq + 1) * sizeof(unsigned);
- BYTE* const cTableWksp = countWkspStart + countWkspSize;
- const size_t cTableWkspSize = wkspEnd-cTableWksp;
- ZSTD_strategy const strategy = cctxParams->cParams.strategy;
- FSE_CTable* CTable_LitLength = nextEntropy->litlengthCTable;
- FSE_CTable* CTable_OffsetBits = nextEntropy->offcodeCTable;
- FSE_CTable* CTable_MatchLength = nextEntropy->matchlengthCTable;
- const BYTE* const ofCodeTable = seqStorePtr->ofCode;
- const BYTE* const llCodeTable = seqStorePtr->llCode;
- const BYTE* const mlCodeTable = seqStorePtr->mlCode;
- size_t const nbSeq = seqStorePtr->sequences - seqStorePtr->sequencesStart;
- BYTE* const ostart = fseMetadata->fseTablesBuffer;
- BYTE* const oend = ostart + sizeof(fseMetadata->fseTablesBuffer);
- BYTE* op = ostart;
-
- assert(cTableWkspSize >= (1 << MaxFSELog) * sizeof(FSE_FUNCTION_TYPE));
- DEBUGLOG(5, "ZSTD_buildSuperBlockEntropy_sequences (nbSeq=%zu)", nbSeq);
- ZSTD_memset(workspace, 0, wkspSize);
-
- fseMetadata->lastCountSize = 0;
- /* convert length/distances into codes */
- ZSTD_seqToCodes(seqStorePtr);
- /* build CTable for Literal Lengths */
- { U32 LLtype;
- unsigned max = MaxLL;
- size_t const mostFrequent = HIST_countFast_wksp(countWksp, &max, llCodeTable, nbSeq, workspace, wkspSize); /* can't fail */
- DEBUGLOG(5, "Building LL table");
- nextEntropy->litlength_repeatMode = prevEntropy->litlength_repeatMode;
- LLtype = ZSTD_selectEncodingType(&nextEntropy->litlength_repeatMode,
- countWksp, max, mostFrequent, nbSeq,
- LLFSELog, prevEntropy->litlengthCTable,
- LL_defaultNorm, LL_defaultNormLog,
- ZSTD_defaultAllowed, strategy);
- assert(set_basic < set_compressed && set_rle < set_compressed);
- assert(!(LLtype < set_compressed && nextEntropy->litlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */
- { size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_LitLength, LLFSELog, (symbolEncodingType_e)LLtype,
- countWksp, max, llCodeTable, nbSeq, LL_defaultNorm, LL_defaultNormLog, MaxLL,
- prevEntropy->litlengthCTable, sizeof(prevEntropy->litlengthCTable),
- cTableWksp, cTableWkspSize);
- FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for LitLens failed");
- if (LLtype == set_compressed)
- fseMetadata->lastCountSize = countSize;
- op += countSize;
- fseMetadata->llType = (symbolEncodingType_e) LLtype;
- } }
- /* build CTable for Offsets */
- { U32 Offtype;
- unsigned max = MaxOff;
- size_t const mostFrequent = HIST_countFast_wksp(countWksp, &max, ofCodeTable, nbSeq, workspace, wkspSize); /* can't fail */
- /* We can only use the basic table if max <= DefaultMaxOff, otherwise the offsets are too large */
- ZSTD_defaultPolicy_e const defaultPolicy = (max <= DefaultMaxOff) ? ZSTD_defaultAllowed : ZSTD_defaultDisallowed;
- DEBUGLOG(5, "Building OF table");
- nextEntropy->offcode_repeatMode = prevEntropy->offcode_repeatMode;
- Offtype = ZSTD_selectEncodingType(&nextEntropy->offcode_repeatMode,
- countWksp, max, mostFrequent, nbSeq,
- OffFSELog, prevEntropy->offcodeCTable,
- OF_defaultNorm, OF_defaultNormLog,
- defaultPolicy, strategy);
- assert(!(Offtype < set_compressed && nextEntropy->offcode_repeatMode != FSE_repeat_none)); /* We don't copy tables */
- { size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_OffsetBits, OffFSELog, (symbolEncodingType_e)Offtype,
- countWksp, max, ofCodeTable, nbSeq, OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff,
- prevEntropy->offcodeCTable, sizeof(prevEntropy->offcodeCTable),
- cTableWksp, cTableWkspSize);
- FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for Offsets failed");
- if (Offtype == set_compressed)
- fseMetadata->lastCountSize = countSize;
- op += countSize;
- fseMetadata->ofType = (symbolEncodingType_e) Offtype;
- } }
- /* build CTable for MatchLengths */
- { U32 MLtype;
- unsigned max = MaxML;
- size_t const mostFrequent = HIST_countFast_wksp(countWksp, &max, mlCodeTable, nbSeq, workspace, wkspSize); /* can't fail */
- DEBUGLOG(5, "Building ML table (remaining space : %i)", (int)(oend-op));
- nextEntropy->matchlength_repeatMode = prevEntropy->matchlength_repeatMode;
- MLtype = ZSTD_selectEncodingType(&nextEntropy->matchlength_repeatMode,
- countWksp, max, mostFrequent, nbSeq,
- MLFSELog, prevEntropy->matchlengthCTable,
- ML_defaultNorm, ML_defaultNormLog,
- ZSTD_defaultAllowed, strategy);
- assert(!(MLtype < set_compressed && nextEntropy->matchlength_repeatMode != FSE_repeat_none)); /* We don't copy tables */
- { size_t const countSize = ZSTD_buildCTable(op, oend - op, CTable_MatchLength, MLFSELog, (symbolEncodingType_e)MLtype,
- countWksp, max, mlCodeTable, nbSeq, ML_defaultNorm, ML_defaultNormLog, MaxML,
- prevEntropy->matchlengthCTable, sizeof(prevEntropy->matchlengthCTable),
- cTableWksp, cTableWkspSize);
- FORWARD_IF_ERROR(countSize, "ZSTD_buildCTable for MatchLengths failed");
- if (MLtype == set_compressed)
- fseMetadata->lastCountSize = countSize;
- op += countSize;
- fseMetadata->mlType = (symbolEncodingType_e) MLtype;
- } }
- assert((size_t) (op-ostart) <= sizeof(fseMetadata->fseTablesBuffer));
- return op-ostart;
-}
-
-
-/* ZSTD_buildSuperBlockEntropy() :
- * Builds entropy for the super-block.
- * @return : 0 on success or error code */
-static size_t
-ZSTD_buildSuperBlockEntropy(seqStore_t* seqStorePtr,
- const ZSTD_entropyCTables_t* prevEntropy,
- ZSTD_entropyCTables_t* nextEntropy,
- const ZSTD_CCtx_params* cctxParams,
- ZSTD_entropyCTablesMetadata_t* entropyMetadata,
- void* workspace, size_t wkspSize)
-{
- size_t const litSize = seqStorePtr->lit - seqStorePtr->litStart;
- DEBUGLOG(5, "ZSTD_buildSuperBlockEntropy");
- entropyMetadata->hufMetadata.hufDesSize =
- ZSTD_buildSuperBlockEntropy_literal(seqStorePtr->litStart, litSize,
- &prevEntropy->huf, &nextEntropy->huf,
- &entropyMetadata->hufMetadata,
- ZSTD_disableLiteralsCompression(cctxParams),
- workspace, wkspSize);
- FORWARD_IF_ERROR(entropyMetadata->hufMetadata.hufDesSize, "ZSTD_buildSuperBlockEntropy_literal failed");
- entropyMetadata->fseMetadata.fseTablesSize =
- ZSTD_buildSuperBlockEntropy_sequences(seqStorePtr,
- &prevEntropy->fse, &nextEntropy->fse,
- cctxParams,
- &entropyMetadata->fseMetadata,
- workspace, wkspSize);
- FORWARD_IF_ERROR(entropyMetadata->fseMetadata.fseTablesSize, "ZSTD_buildSuperBlockEntropy_sequences failed");
- return 0;
-}
-
/* ZSTD_compressSubBlock_literal() :
* Compresses literals section for a sub-block.
* When we have to write the Huffman table we will sometimes choose a header
const seqDef* sp = sstart;
size_t matchLengthSum = 0;
size_t litLengthSum = 0;
- /* Only used by assert(), suppress unused variable warnings in production. */
- (void)litLengthSum;
+ (void)(litLengthSum); /* suppress unused variable warning on some environments */
while (send-sp > 0) {
ZSTD_sequenceLength const seqLen = ZSTD_getSequenceLength(seqStore, sp);
litLengthSum += seqLen.litLength;
static size_t ZSTD_estimateSubBlockSize_symbolType(symbolEncodingType_e type,
const BYTE* codeTable, unsigned maxCode,
size_t nbSeq, const FSE_CTable* fseCTable,
- const U32* additionalBits,
+ const U8* additionalBits,
short const* defaultNorm, U32 defaultNormLog, U32 defaultMax,
void* workspace, size_t wkspSize)
{
void* workspace, size_t wkspSize,
int writeEntropy)
{
- size_t sequencesSectionHeaderSize = 3; /* Use hard coded size of 3 bytes */
+ size_t const sequencesSectionHeaderSize = 3; /* Use hard coded size of 3 bytes */
size_t cSeqSizeEstimate = 0;
+ if (nbSeq == 0) return sequencesSectionHeaderSize;
cSeqSizeEstimate += ZSTD_estimateSubBlockSize_symbolType(fseMetadata->ofType, ofCodeTable, MaxOff,
nbSeq, fseTables->offcodeCTable, NULL,
OF_defaultNorm, OF_defaultNormLog, DefaultMaxOff,
/* I think there is an optimization opportunity here.
* Calling ZSTD_estimateSubBlockSize for every sequence can be wasteful
* since it recalculates estimate from scratch.
- * For example, it would recount literal distribution and symbol codes everytime.
+ * For example, it would recount literal distribution and symbol codes every time.
*/
cBlockSizeEstimate = ZSTD_estimateSubBlockSize(lp, litSize, ofCodePtr, llCodePtr, mlCodePtr, seqCount,
&nextCBlock->entropy, entropyMetadata,
repcodes_t rep;
ZSTD_memcpy(&rep, prevCBlock->rep, sizeof(rep));
for (seq = sstart; seq < sp; ++seq) {
- rep = ZSTD_updateRep(rep.rep, seq->offset - 1, ZSTD_getSequenceLength(seqStorePtr, seq).litLength == 0);
+ ZSTD_updateRep(rep.rep, seq->offBase - 1, ZSTD_getSequenceLength(seqStorePtr, seq).litLength == 0);
}
ZSTD_memcpy(nextCBlock->rep, &rep, sizeof(rep));
}
unsigned lastBlock) {
ZSTD_entropyCTablesMetadata_t entropyMetadata;
- FORWARD_IF_ERROR(ZSTD_buildSuperBlockEntropy(&zc->seqStore,
+ FORWARD_IF_ERROR(ZSTD_buildBlockEntropyStats(&zc->seqStore,
&zc->blockState.prevCBlock->entropy,
&zc->blockState.nextCBlock->entropy,
&zc->appliedParams,
#define ZSTD_CWKSP_ASAN_REDZONE_SIZE 128
#endif
+
+/* Set our tables and aligneds to align by 64 bytes */
+#define ZSTD_CWKSP_ALIGNMENT_BYTES 64
+
/*-*************************************
* Structures
***************************************/
* - Tables: these are any of several different datastructures (hash tables,
* chain tables, binary trees) that all respect a common format: they are
* uint32_t arrays, all of whose values are between 0 and (nextSrc - base).
- * Their sizes depend on the cparams.
+ * Their sizes depend on the cparams. These tables are 64-byte aligned.
*
* - Aligned: these buffers are used for various purposes that require 4 byte
- * alignment, but don't require any initialization before they're used.
+ * alignment, but don't require any initialization before they're used. These
+ * buffers are each aligned to 64 bytes.
*
* - Buffers: these buffers are used for various purposes that don't require
* any alignment or initialization before they're used. This means they can
*
* 1. Objects
* 2. Buffers
- * 3. Aligned
- * 4. Tables
+ * 3. Aligned/Tables
*
* Attempts to reserve objects of different types out of order will fail.
*/
* Since tables aren't currently redzoned, you don't need to call through this
* to figure out how much space you need for the matchState tables. Everything
* else is though.
+ *
+ * Do not use for sizing aligned buffers. Instead, use ZSTD_cwksp_aligned_alloc_size().
*/
MEM_STATIC size_t ZSTD_cwksp_alloc_size(size_t size) {
if (size == 0)
return size;
}
-MEM_STATIC void ZSTD_cwksp_internal_advance_phase(
- ZSTD_cwksp* ws, ZSTD_cwksp_alloc_phase_e phase) {
+/*
+ * Returns an adjusted alloc size that is the nearest larger multiple of 64 bytes.
+ * Used to determine the number of bytes required for a given "aligned".
+ */
+MEM_STATIC size_t ZSTD_cwksp_aligned_alloc_size(size_t size) {
+ return ZSTD_cwksp_alloc_size(ZSTD_cwksp_align(size, ZSTD_CWKSP_ALIGNMENT_BYTES));
+}
+
+/*
+ * Returns the amount of additional space the cwksp must allocate
+ * for internal purposes (currently only alignment).
+ */
+MEM_STATIC size_t ZSTD_cwksp_slack_space_required(void) {
+ /* For alignment, the wksp will always allocate an additional n_1=[1, 64] bytes
+ * to align the beginning of tables section, as well as another n_2=[0, 63] bytes
+ * to align the beginning of the aligned section.
+ *
+ * n_1 + n_2 == 64 bytes if the cwksp is freshly allocated, due to tables and
+ * aligneds being sized in multiples of 64 bytes.
+ */
+ size_t const slackSpace = ZSTD_CWKSP_ALIGNMENT_BYTES;
+ return slackSpace;
+}
+
+
+/*
+ * Return the number of additional bytes required to align a pointer to the given number of bytes.
+ * alignBytes must be a power of two.
+ */
+MEM_STATIC size_t ZSTD_cwksp_bytes_to_align_ptr(void* ptr, const size_t alignBytes) {
+ size_t const alignBytesMask = alignBytes - 1;
+ size_t const bytes = (alignBytes - ((size_t)ptr & (alignBytesMask))) & alignBytesMask;
+ assert((alignBytes & alignBytesMask) == 0);
+ assert(bytes != ZSTD_CWKSP_ALIGNMENT_BYTES);
+ return bytes;
+}
+
+/*
+ * Internal function. Do not use directly.
+ * Reserves the given number of bytes within the aligned/buffer segment of the wksp,
+ * which counts from the end of the wksp (as opposed to the object/table segment).
+ *
+ * Returns a pointer to the beginning of that space.
+ */
+MEM_STATIC void*
+ZSTD_cwksp_reserve_internal_buffer_space(ZSTD_cwksp* ws, size_t const bytes)
+{
+ void* const alloc = (BYTE*)ws->allocStart - bytes;
+ void* const bottom = ws->tableEnd;
+ DEBUGLOG(5, "cwksp: reserving %p %zd bytes, %zd bytes remaining",
+ alloc, bytes, ZSTD_cwksp_available_space(ws) - bytes);
+ ZSTD_cwksp_assert_internal_consistency(ws);
+ assert(alloc >= bottom);
+ if (alloc < bottom) {
+ DEBUGLOG(4, "cwksp: alloc failed!");
+ ws->allocFailed = 1;
+ return NULL;
+ }
+ /* the area is reserved from the end of wksp.
+ * If it overlaps with tableValidEnd, it voids guarantees on values' range */
+ if (alloc < ws->tableValidEnd) {
+ ws->tableValidEnd = alloc;
+ }
+ ws->allocStart = alloc;
+ return alloc;
+}
+
+/*
+ * Moves the cwksp to the next phase, and does any necessary allocations.
+ * cwksp initialization must necessarily go through each phase in order.
+ * Returns a 0 on success, or zstd error
+ */
+MEM_STATIC size_t
+ZSTD_cwksp_internal_advance_phase(ZSTD_cwksp* ws, ZSTD_cwksp_alloc_phase_e phase)
+{
assert(phase >= ws->phase);
if (phase > ws->phase) {
+ /* Going from allocating objects to allocating buffers */
if (ws->phase < ZSTD_cwksp_alloc_buffers &&
phase >= ZSTD_cwksp_alloc_buffers) {
ws->tableValidEnd = ws->objectEnd;
}
+
+ /* Going from allocating buffers to allocating aligneds/tables */
if (ws->phase < ZSTD_cwksp_alloc_aligned &&
phase >= ZSTD_cwksp_alloc_aligned) {
- /* If unaligned allocations down from a too-large top have left us
- * unaligned, we need to realign our alloc ptr. Technically, this
- * can consume space that is unaccounted for in the neededSpace
- * calculation. However, I believe this can only happen when the
- * workspace is too large, and specifically when it is too large
- * by a larger margin than the space that will be consumed. */
- /* TODO: cleaner, compiler warning friendly way to do this??? */
- ws->allocStart = (BYTE*)ws->allocStart - ((size_t)ws->allocStart & (sizeof(U32)-1));
- if (ws->allocStart < ws->tableValidEnd) {
- ws->tableValidEnd = ws->allocStart;
+ { /* Align the start of the "aligned" to 64 bytes. Use [1, 64] bytes. */
+ size_t const bytesToAlign =
+ ZSTD_CWKSP_ALIGNMENT_BYTES - ZSTD_cwksp_bytes_to_align_ptr(ws->allocStart, ZSTD_CWKSP_ALIGNMENT_BYTES);
+ DEBUGLOG(5, "reserving aligned alignment addtl space: %zu", bytesToAlign);
+ ZSTD_STATIC_ASSERT((ZSTD_CWKSP_ALIGNMENT_BYTES & (ZSTD_CWKSP_ALIGNMENT_BYTES - 1)) == 0); /* power of 2 */
+ RETURN_ERROR_IF(!ZSTD_cwksp_reserve_internal_buffer_space(ws, bytesToAlign),
+ memory_allocation, "aligned phase - alignment initial allocation failed!");
}
- }
+ { /* Align the start of the tables to 64 bytes. Use [0, 63] bytes */
+ void* const alloc = ws->objectEnd;
+ size_t const bytesToAlign = ZSTD_cwksp_bytes_to_align_ptr(alloc, ZSTD_CWKSP_ALIGNMENT_BYTES);
+ void* const objectEnd = (BYTE*)alloc + bytesToAlign;
+ DEBUGLOG(5, "reserving table alignment addtl space: %zu", bytesToAlign);
+ RETURN_ERROR_IF(objectEnd > ws->workspaceEnd, memory_allocation,
+ "table phase - alignment initial allocation failed!");
+ ws->objectEnd = objectEnd;
+ ws->tableEnd = objectEnd; /* table area starts being empty */
+ if (ws->tableValidEnd < ws->tableEnd) {
+ ws->tableValidEnd = ws->tableEnd;
+ } } }
ws->phase = phase;
+ ZSTD_cwksp_assert_internal_consistency(ws);
}
+ return 0;
}
/*
* Returns whether this object/buffer/etc was allocated in this workspace.
*/
-MEM_STATIC int ZSTD_cwksp_owns_buffer(const ZSTD_cwksp* ws, const void* ptr) {
+MEM_STATIC int ZSTD_cwksp_owns_buffer(const ZSTD_cwksp* ws, const void* ptr)
+{
return (ptr != NULL) && (ws->workspace <= ptr) && (ptr <= ws->workspaceEnd);
}
/*
* Internal function. Do not use directly.
*/
-MEM_STATIC void* ZSTD_cwksp_reserve_internal(
- ZSTD_cwksp* ws, size_t bytes, ZSTD_cwksp_alloc_phase_e phase) {
+MEM_STATIC void*
+ZSTD_cwksp_reserve_internal(ZSTD_cwksp* ws, size_t bytes, ZSTD_cwksp_alloc_phase_e phase)
+{
void* alloc;
- void* bottom = ws->tableEnd;
- ZSTD_cwksp_internal_advance_phase(ws, phase);
- alloc = (BYTE *)ws->allocStart - bytes;
-
- if (bytes == 0)
+ if (ZSTD_isError(ZSTD_cwksp_internal_advance_phase(ws, phase)) || bytes == 0) {
return NULL;
+ }
- DEBUGLOG(5, "cwksp: reserving %p %zd bytes, %zd bytes remaining",
- alloc, bytes, ZSTD_cwksp_available_space(ws) - bytes);
- ZSTD_cwksp_assert_internal_consistency(ws);
- assert(alloc >= bottom);
- if (alloc < bottom) {
- DEBUGLOG(4, "cwksp: alloc failed!");
- ws->allocFailed = 1;
- return NULL;
- }
- if (alloc < ws->tableValidEnd) {
- ws->tableValidEnd = alloc;
- }
- ws->allocStart = alloc;
+ alloc = ZSTD_cwksp_reserve_internal_buffer_space(ws, bytes);
return alloc;
/*
* Reserves and returns unaligned memory.
*/
-MEM_STATIC BYTE* ZSTD_cwksp_reserve_buffer(ZSTD_cwksp* ws, size_t bytes) {
+MEM_STATIC BYTE* ZSTD_cwksp_reserve_buffer(ZSTD_cwksp* ws, size_t bytes)
+{
return (BYTE*)ZSTD_cwksp_reserve_internal(ws, bytes, ZSTD_cwksp_alloc_buffers);
}
/*
- * Reserves and returns memory sized on and aligned on sizeof(unsigned).
+ * Reserves and returns memory sized on and aligned on ZSTD_CWKSP_ALIGNMENT_BYTES (64 bytes).
*/
-MEM_STATIC void* ZSTD_cwksp_reserve_aligned(ZSTD_cwksp* ws, size_t bytes) {
- assert((bytes & (sizeof(U32)-1)) == 0);
- return ZSTD_cwksp_reserve_internal(ws, ZSTD_cwksp_align(bytes, sizeof(U32)), ZSTD_cwksp_alloc_aligned);
+MEM_STATIC void* ZSTD_cwksp_reserve_aligned(ZSTD_cwksp* ws, size_t bytes)
+{
+ void* ptr = ZSTD_cwksp_reserve_internal(ws, ZSTD_cwksp_align(bytes, ZSTD_CWKSP_ALIGNMENT_BYTES),
+ ZSTD_cwksp_alloc_aligned);
+ assert(((size_t)ptr & (ZSTD_CWKSP_ALIGNMENT_BYTES-1))== 0);
+ return ptr;
}
/*
- * Aligned on sizeof(unsigned). These buffers have the special property that
+ * Aligned on 64 bytes. These buffers have the special property that
* their values remain constrained, allowing us to re-use them without
* memset()-ing them.
*/
-MEM_STATIC void* ZSTD_cwksp_reserve_table(ZSTD_cwksp* ws, size_t bytes) {
+MEM_STATIC void* ZSTD_cwksp_reserve_table(ZSTD_cwksp* ws, size_t bytes)
+{
const ZSTD_cwksp_alloc_phase_e phase = ZSTD_cwksp_alloc_aligned;
- void* alloc = ws->tableEnd;
- void* end = (BYTE *)alloc + bytes;
- void* top = ws->allocStart;
+ void* alloc;
+ void* end;
+ void* top;
+
+ if (ZSTD_isError(ZSTD_cwksp_internal_advance_phase(ws, phase))) {
+ return NULL;
+ }
+ alloc = ws->tableEnd;
+ end = (BYTE *)alloc + bytes;
+ top = ws->allocStart;
DEBUGLOG(5, "cwksp: reserving %p table %zd bytes, %zd bytes remaining",
alloc, bytes, ZSTD_cwksp_available_space(ws) - bytes);
assert((bytes & (sizeof(U32)-1)) == 0);
- ZSTD_cwksp_internal_advance_phase(ws, phase);
ZSTD_cwksp_assert_internal_consistency(ws);
assert(end <= top);
if (end > top) {
ws->tableEnd = end;
+ assert((bytes & (ZSTD_CWKSP_ALIGNMENT_BYTES-1)) == 0);
+ assert(((size_t)alloc & (ZSTD_CWKSP_ALIGNMENT_BYTES-1))== 0);
return alloc;
}
/*
* Aligned on sizeof(void*).
+ * Note : should happen only once, at workspace first initialization
*/
-MEM_STATIC void* ZSTD_cwksp_reserve_object(ZSTD_cwksp* ws, size_t bytes) {
- size_t roundedBytes = ZSTD_cwksp_align(bytes, sizeof(void*));
+MEM_STATIC void* ZSTD_cwksp_reserve_object(ZSTD_cwksp* ws, size_t bytes)
+{
+ size_t const roundedBytes = ZSTD_cwksp_align(bytes, sizeof(void*));
void* alloc = ws->objectEnd;
void* end = (BYTE*)alloc + roundedBytes;
- DEBUGLOG(5,
+ DEBUGLOG(4,
"cwksp: reserving %p object %zd bytes (rounded to %zd), %zd bytes remaining",
alloc, bytes, roundedBytes, ZSTD_cwksp_available_space(ws) - roundedBytes);
- assert(((size_t)alloc & (sizeof(void*)-1)) == 0);
- assert((bytes & (sizeof(void*)-1)) == 0);
+ assert((size_t)alloc % ZSTD_ALIGNOF(void*) == 0);
+ assert(bytes % ZSTD_ALIGNOF(void*) == 0);
ZSTD_cwksp_assert_internal_consistency(ws);
/* we must be in the first phase, no advance is possible */
if (ws->phase != ZSTD_cwksp_alloc_objects || end > ws->workspaceEnd) {
- DEBUGLOG(4, "cwksp: object alloc failed!");
+ DEBUGLOG(3, "cwksp: object alloc failed!");
ws->allocFailed = 1;
return NULL;
}
return alloc;
}
-MEM_STATIC void ZSTD_cwksp_mark_tables_dirty(ZSTD_cwksp* ws) {
+MEM_STATIC void ZSTD_cwksp_mark_tables_dirty(ZSTD_cwksp* ws)
+{
DEBUGLOG(4, "cwksp: ZSTD_cwksp_mark_tables_dirty");
* Functions Checking Free Space
***************************************/
+/* ZSTD_alignmentSpaceWithinBounds() :
+ * Returns if the estimated space needed for a wksp is within an acceptable limit of the
+ * actual amount of space used.
+ */
+MEM_STATIC int ZSTD_cwksp_estimated_space_within_bounds(const ZSTD_cwksp* const ws,
+ size_t const estimatedSpace, int resizedWorkspace) {
+ if (resizedWorkspace) {
+ /* Resized/newly allocated wksp should have exact bounds */
+ return ZSTD_cwksp_used(ws) == estimatedSpace;
+ } else {
+ /* Due to alignment, when reusing a workspace, we can actually consume 63 fewer or more bytes
+ * than estimatedSpace. See the comments in zstd_cwksp.h for details.
+ */
+ return (ZSTD_cwksp_used(ws) >= estimatedSpace - 63) && (ZSTD_cwksp_used(ws) <= estimatedSpace + 63);
+ }
+}
+
+
MEM_STATIC size_t ZSTD_cwksp_available_space(ZSTD_cwksp* ws) {
return (size_t)((BYTE*)ws->allocStart - (BYTE*)ws->tableEnd);
}
FORCE_INLINE_TEMPLATE
-size_t ZSTD_compressBlock_doubleFast_generic(
+size_t ZSTD_compressBlock_doubleFast_noDict_generic(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize, U32 const mls /* template */)
+{
+ ZSTD_compressionParameters const* cParams = &ms->cParams;
+ U32* const hashLong = ms->hashTable;
+ const U32 hBitsL = cParams->hashLog;
+ U32* const hashSmall = ms->chainTable;
+ const U32 hBitsS = cParams->chainLog;
+ const BYTE* const base = ms->window.base;
+ const BYTE* const istart = (const BYTE*)src;
+ const BYTE* anchor = istart;
+ const U32 endIndex = (U32)((size_t)(istart - base) + srcSize);
+ /* presumes that, if there is a dictionary, it must be using Attach mode */
+ const U32 prefixLowestIndex = ZSTD_getLowestPrefixIndex(ms, endIndex, cParams->windowLog);
+ const BYTE* const prefixLowest = base + prefixLowestIndex;
+ const BYTE* const iend = istart + srcSize;
+ const BYTE* const ilimit = iend - HASH_READ_SIZE;
+ U32 offset_1=rep[0], offset_2=rep[1];
+ U32 offsetSaved = 0;
+
+ size_t mLength;
+ U32 offset;
+ U32 curr;
+
+ /* how many positions to search before increasing step size */
+ const size_t kStepIncr = 1 << kSearchStrength;
+ /* the position at which to increment the step size if no match is found */
+ const BYTE* nextStep;
+ size_t step; /* the current step size */
+
+ size_t hl0; /* the long hash at ip */
+ size_t hl1; /* the long hash at ip1 */
+
+ U32 idxl0; /* the long match index for ip */
+ U32 idxl1; /* the long match index for ip1 */
+
+ const BYTE* matchl0; /* the long match for ip */
+ const BYTE* matchs0; /* the short match for ip */
+ const BYTE* matchl1; /* the long match for ip1 */
+
+ const BYTE* ip = istart; /* the current position */
+ const BYTE* ip1; /* the next position */
+
+ DEBUGLOG(5, "ZSTD_compressBlock_doubleFast_noDict_generic");
+
+ /* init */
+ ip += ((ip - prefixLowest) == 0);
+ {
+ U32 const current = (U32)(ip - base);
+ U32 const windowLow = ZSTD_getLowestPrefixIndex(ms, current, cParams->windowLog);
+ U32 const maxRep = current - windowLow;
+ if (offset_2 > maxRep) offsetSaved = offset_2, offset_2 = 0;
+ if (offset_1 > maxRep) offsetSaved = offset_1, offset_1 = 0;
+ }
+
+ /* Outer Loop: one iteration per match found and stored */
+ while (1) {
+ step = 1;
+ nextStep = ip + kStepIncr;
+ ip1 = ip + step;
+
+ if (ip1 > ilimit) {
+ goto _cleanup;
+ }
+
+ hl0 = ZSTD_hashPtr(ip, hBitsL, 8);
+ idxl0 = hashLong[hl0];
+ matchl0 = base + idxl0;
+
+ /* Inner Loop: one iteration per search / position */
+ do {
+ const size_t hs0 = ZSTD_hashPtr(ip, hBitsS, mls);
+ const U32 idxs0 = hashSmall[hs0];
+ curr = (U32)(ip-base);
+ matchs0 = base + idxs0;
+
+ hashLong[hl0] = hashSmall[hs0] = curr; /* update hash tables */
+
+ /* check noDict repcode */
+ if ((offset_1 > 0) & (MEM_read32(ip+1-offset_1) == MEM_read32(ip+1))) {
+ mLength = ZSTD_count(ip+1+4, ip+1+4-offset_1, iend) + 4;
+ ip++;
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_REPCODE_1, mLength);
+ goto _match_stored;
+ }
+
+ hl1 = ZSTD_hashPtr(ip1, hBitsL, 8);
+
+ if (idxl0 > prefixLowestIndex) {
+ /* check prefix long match */
+ if (MEM_read64(matchl0) == MEM_read64(ip)) {
+ mLength = ZSTD_count(ip+8, matchl0+8, iend) + 8;
+ offset = (U32)(ip-matchl0);
+ while (((ip>anchor) & (matchl0>prefixLowest)) && (ip[-1] == matchl0[-1])) { ip--; matchl0--; mLength++; } /* catch up */
+ goto _match_found;
+ }
+ }
+
+ idxl1 = hashLong[hl1];
+ matchl1 = base + idxl1;
+
+ if (idxs0 > prefixLowestIndex) {
+ /* check prefix short match */
+ if (MEM_read32(matchs0) == MEM_read32(ip)) {
+ goto _search_next_long;
+ }
+ }
+
+ if (ip1 >= nextStep) {
+ PREFETCH_L1(ip1 + 64);
+ PREFETCH_L1(ip1 + 128);
+ step++;
+ nextStep += kStepIncr;
+ }
+ ip = ip1;
+ ip1 += step;
+
+ hl0 = hl1;
+ idxl0 = idxl1;
+ matchl0 = matchl1;
+ #if defined(__aarch64__)
+ PREFETCH_L1(ip+256);
+ #endif
+ } while (ip1 <= ilimit);
+
+_cleanup:
+ /* save reps for next block */
+ rep[0] = offset_1 ? offset_1 : offsetSaved;
+ rep[1] = offset_2 ? offset_2 : offsetSaved;
+
+ /* Return the last literals size */
+ return (size_t)(iend - anchor);
+
+_search_next_long:
+
+ /* check prefix long +1 match */
+ if (idxl1 > prefixLowestIndex) {
+ if (MEM_read64(matchl1) == MEM_read64(ip1)) {
+ ip = ip1;
+ mLength = ZSTD_count(ip+8, matchl1+8, iend) + 8;
+ offset = (U32)(ip-matchl1);
+ while (((ip>anchor) & (matchl1>prefixLowest)) && (ip[-1] == matchl1[-1])) { ip--; matchl1--; mLength++; } /* catch up */
+ goto _match_found;
+ }
+ }
+
+ /* if no long +1 match, explore the short match we found */
+ mLength = ZSTD_count(ip+4, matchs0+4, iend) + 4;
+ offset = (U32)(ip - matchs0);
+ while (((ip>anchor) & (matchs0>prefixLowest)) && (ip[-1] == matchs0[-1])) { ip--; matchs0--; mLength++; } /* catch up */
+
+ /* fall-through */
+
+_match_found: /* requires ip, offset, mLength */
+ offset_2 = offset_1;
+ offset_1 = offset;
+
+ if (step < 4) {
+ /* It is unsafe to write this value back to the hashtable when ip1 is
+ * greater than or equal to the new ip we will have after we're done
+ * processing this match. Rather than perform that test directly
+ * (ip1 >= ip + mLength), which costs speed in practice, we do a simpler
+ * more predictable test. The minmatch even if we take a short match is
+ * 4 bytes, so as long as step, the distance between ip and ip1
+ * (initially) is less than 4, we know ip1 < new ip. */
+ hashLong[hl1] = (U32)(ip1 - base);
+ }
+
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_OFFSET(offset), mLength);
+
+_match_stored:
+ /* match found */
+ ip += mLength;
+ anchor = ip;
+
+ if (ip <= ilimit) {
+ /* Complementary insertion */
+ /* done after iLimit test, as candidates could be > iend-8 */
+ { U32 const indexToInsert = curr+2;
+ hashLong[ZSTD_hashPtr(base+indexToInsert, hBitsL, 8)] = indexToInsert;
+ hashLong[ZSTD_hashPtr(ip-2, hBitsL, 8)] = (U32)(ip-2-base);
+ hashSmall[ZSTD_hashPtr(base+indexToInsert, hBitsS, mls)] = indexToInsert;
+ hashSmall[ZSTD_hashPtr(ip-1, hBitsS, mls)] = (U32)(ip-1-base);
+ }
+
+ /* check immediate repcode */
+ while ( (ip <= ilimit)
+ && ( (offset_2>0)
+ & (MEM_read32(ip) == MEM_read32(ip - offset_2)) )) {
+ /* store sequence */
+ size_t const rLength = ZSTD_count(ip+4, ip+4-offset_2, iend) + 4;
+ U32 const tmpOff = offset_2; offset_2 = offset_1; offset_1 = tmpOff; /* swap offset_2 <=> offset_1 */
+ hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = (U32)(ip-base);
+ hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = (U32)(ip-base);
+ ZSTD_storeSeq(seqStore, 0, anchor, iend, STORE_REPCODE_1, rLength);
+ ip += rLength;
+ anchor = ip;
+ continue; /* faster when present ... (?) */
+ }
+ }
+ }
+}
+
+
+FORCE_INLINE_TEMPLATE
+size_t ZSTD_compressBlock_doubleFast_dictMatchState_generic(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
void const* src, size_t srcSize,
- U32 const mls /* template */, ZSTD_dictMode_e const dictMode)
+ U32 const mls /* template */)
{
ZSTD_compressionParameters const* cParams = &ms->cParams;
U32* const hashLong = ms->hashTable;
U32 offsetSaved = 0;
const ZSTD_matchState_t* const dms = ms->dictMatchState;
- const ZSTD_compressionParameters* const dictCParams =
- dictMode == ZSTD_dictMatchState ?
- &dms->cParams : NULL;
- const U32* const dictHashLong = dictMode == ZSTD_dictMatchState ?
- dms->hashTable : NULL;
- const U32* const dictHashSmall = dictMode == ZSTD_dictMatchState ?
- dms->chainTable : NULL;
- const U32 dictStartIndex = dictMode == ZSTD_dictMatchState ?
- dms->window.dictLimit : 0;
- const BYTE* const dictBase = dictMode == ZSTD_dictMatchState ?
- dms->window.base : NULL;
- const BYTE* const dictStart = dictMode == ZSTD_dictMatchState ?
- dictBase + dictStartIndex : NULL;
- const BYTE* const dictEnd = dictMode == ZSTD_dictMatchState ?
- dms->window.nextSrc : NULL;
- const U32 dictIndexDelta = dictMode == ZSTD_dictMatchState ?
- prefixLowestIndex - (U32)(dictEnd - dictBase) :
- 0;
- const U32 dictHBitsL = dictMode == ZSTD_dictMatchState ?
- dictCParams->hashLog : hBitsL;
- const U32 dictHBitsS = dictMode == ZSTD_dictMatchState ?
- dictCParams->chainLog : hBitsS;
+ const ZSTD_compressionParameters* const dictCParams = &dms->cParams;
+ const U32* const dictHashLong = dms->hashTable;
+ const U32* const dictHashSmall = dms->chainTable;
+ const U32 dictStartIndex = dms->window.dictLimit;
+ const BYTE* const dictBase = dms->window.base;
+ const BYTE* const dictStart = dictBase + dictStartIndex;
+ const BYTE* const dictEnd = dms->window.nextSrc;
+ const U32 dictIndexDelta = prefixLowestIndex - (U32)(dictEnd - dictBase);
+ const U32 dictHBitsL = dictCParams->hashLog;
+ const U32 dictHBitsS = dictCParams->chainLog;
const U32 dictAndPrefixLength = (U32)((ip - prefixLowest) + (dictEnd - dictStart));
- DEBUGLOG(5, "ZSTD_compressBlock_doubleFast_generic");
-
- assert(dictMode == ZSTD_noDict || dictMode == ZSTD_dictMatchState);
+ DEBUGLOG(5, "ZSTD_compressBlock_doubleFast_dictMatchState_generic");
/* if a dictionary is attached, it must be within window range */
- if (dictMode == ZSTD_dictMatchState) {
- assert(ms->window.dictLimit + (1U << cParams->windowLog) >= endIndex);
- }
+ assert(ms->window.dictLimit + (1U << cParams->windowLog) >= endIndex);
/* init */
ip += (dictAndPrefixLength == 0);
- if (dictMode == ZSTD_noDict) {
- U32 const curr = (U32)(ip - base);
- U32 const windowLow = ZSTD_getLowestPrefixIndex(ms, curr, cParams->windowLog);
- U32 const maxRep = curr - windowLow;
- if (offset_2 > maxRep) offsetSaved = offset_2, offset_2 = 0;
- if (offset_1 > maxRep) offsetSaved = offset_1, offset_1 = 0;
- }
- if (dictMode == ZSTD_dictMatchState) {
- /* dictMatchState repCode checks don't currently handle repCode == 0
- * disabling. */
- assert(offset_1 <= dictAndPrefixLength);
- assert(offset_2 <= dictAndPrefixLength);
- }
+
+ /* dictMatchState repCode checks don't currently handle repCode == 0
+ * disabling. */
+ assert(offset_1 <= dictAndPrefixLength);
+ assert(offset_2 <= dictAndPrefixLength);
/* Main Search Loop */
while (ip < ilimit) { /* < instead of <=, because repcode check at (ip+1) */
const BYTE* matchLong = base + matchIndexL;
const BYTE* match = base + matchIndexS;
const U32 repIndex = curr + 1 - offset_1;
- const BYTE* repMatch = (dictMode == ZSTD_dictMatchState
- && repIndex < prefixLowestIndex) ?
+ const BYTE* repMatch = (repIndex < prefixLowestIndex) ?
dictBase + (repIndex - dictIndexDelta) :
base + repIndex;
hashLong[h2] = hashSmall[h] = curr; /* update hash tables */
- /* check dictMatchState repcode */
- if (dictMode == ZSTD_dictMatchState
- && ((U32)((prefixLowestIndex-1) - repIndex) >= 3 /* intentional underflow */)
+ /* check repcode */
+ if (((U32)((prefixLowestIndex-1) - repIndex) >= 3 /* intentional underflow */)
&& (MEM_read32(repMatch) == MEM_read32(ip+1)) ) {
const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend;
mLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4;
ip++;
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH);
- goto _match_stored;
- }
-
- /* check noDict repcode */
- if ( dictMode == ZSTD_noDict
- && ((offset_1 > 0) & (MEM_read32(ip+1-offset_1) == MEM_read32(ip+1)))) {
- mLength = ZSTD_count(ip+1+4, ip+1+4-offset_1, iend) + 4;
- ip++;
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH);
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_REPCODE_1, mLength);
goto _match_stored;
}
while (((ip>anchor) & (matchLong>prefixLowest)) && (ip[-1] == matchLong[-1])) { ip--; matchLong--; mLength++; } /* catch up */
goto _match_found;
}
- } else if (dictMode == ZSTD_dictMatchState) {
+ } else {
/* check dictMatchState long match */
U32 const dictMatchIndexL = dictHashLong[dictHL];
const BYTE* dictMatchL = dictBase + dictMatchIndexL;
if (MEM_read32(match) == MEM_read32(ip)) {
goto _search_next_long;
}
- } else if (dictMode == ZSTD_dictMatchState) {
+ } else {
/* check dictMatchState short match */
U32 const dictMatchIndexS = dictHashSmall[dictHS];
match = dictBase + dictMatchIndexS;
while (((ip>anchor) & (matchL3>prefixLowest)) && (ip[-1] == matchL3[-1])) { ip--; matchL3--; mLength++; } /* catch up */
goto _match_found;
}
- } else if (dictMode == ZSTD_dictMatchState) {
+ } else {
/* check dict long +1 match */
U32 const dictMatchIndexL3 = dictHashLong[dictHLNext];
const BYTE* dictMatchL3 = dictBase + dictMatchIndexL3;
} } }
/* if no long +1 match, explore the short match we found */
- if (dictMode == ZSTD_dictMatchState && matchIndexS < prefixLowestIndex) {
+ if (matchIndexS < prefixLowestIndex) {
mLength = ZSTD_count_2segments(ip+4, match+4, iend, dictEnd, prefixLowest) + 4;
offset = (U32)(curr - matchIndexS);
while (((ip>anchor) & (match>dictStart)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */
offset_2 = offset_1;
offset_1 = offset;
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH);
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_OFFSET(offset), mLength);
_match_stored:
/* match found */
}
/* check immediate repcode */
- if (dictMode == ZSTD_dictMatchState) {
- while (ip <= ilimit) {
- U32 const current2 = (U32)(ip-base);
- U32 const repIndex2 = current2 - offset_2;
- const BYTE* repMatch2 = dictMode == ZSTD_dictMatchState
- && repIndex2 < prefixLowestIndex ?
- dictBase + repIndex2 - dictIndexDelta :
- base + repIndex2;
- if ( ((U32)((prefixLowestIndex-1) - (U32)repIndex2) >= 3 /* intentional overflow */)
- && (MEM_read32(repMatch2) == MEM_read32(ip)) ) {
- const BYTE* const repEnd2 = repIndex2 < prefixLowestIndex ? dictEnd : iend;
- size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixLowest) + 4;
- U32 tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */
- ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, repLength2-MINMATCH);
- hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = current2;
- hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = current2;
- ip += repLength2;
- anchor = ip;
- continue;
- }
- break;
- } }
-
- if (dictMode == ZSTD_noDict) {
- while ( (ip <= ilimit)
- && ( (offset_2>0)
- & (MEM_read32(ip) == MEM_read32(ip - offset_2)) )) {
- /* store sequence */
- size_t const rLength = ZSTD_count(ip+4, ip+4-offset_2, iend) + 4;
- U32 const tmpOff = offset_2; offset_2 = offset_1; offset_1 = tmpOff; /* swap offset_2 <=> offset_1 */
- hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = (U32)(ip-base);
- hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = (U32)(ip-base);
- ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, rLength-MINMATCH);
- ip += rLength;
+ while (ip <= ilimit) {
+ U32 const current2 = (U32)(ip-base);
+ U32 const repIndex2 = current2 - offset_2;
+ const BYTE* repMatch2 = repIndex2 < prefixLowestIndex ?
+ dictBase + repIndex2 - dictIndexDelta :
+ base + repIndex2;
+ if ( ((U32)((prefixLowestIndex-1) - (U32)repIndex2) >= 3 /* intentional overflow */)
+ && (MEM_read32(repMatch2) == MEM_read32(ip)) ) {
+ const BYTE* const repEnd2 = repIndex2 < prefixLowestIndex ? dictEnd : iend;
+ size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixLowest) + 4;
+ U32 tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */
+ ZSTD_storeSeq(seqStore, 0, anchor, iend, STORE_REPCODE_1, repLength2);
+ hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = current2;
+ hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = current2;
+ ip += repLength2;
anchor = ip;
- continue; /* faster when present ... (?) */
- } } }
+ continue;
+ }
+ break;
+ }
+ }
} /* while (ip < ilimit) */
/* save reps for next block */
return (size_t)(iend - anchor);
}
+#define ZSTD_GEN_DFAST_FN(dictMode, mls) \
+ static size_t ZSTD_compressBlock_doubleFast_##dictMode##_##mls( \
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], \
+ void const* src, size_t srcSize) \
+ { \
+ return ZSTD_compressBlock_doubleFast_##dictMode##_generic(ms, seqStore, rep, src, srcSize, mls); \
+ }
+
+ZSTD_GEN_DFAST_FN(noDict, 4)
+ZSTD_GEN_DFAST_FN(noDict, 5)
+ZSTD_GEN_DFAST_FN(noDict, 6)
+ZSTD_GEN_DFAST_FN(noDict, 7)
+
+ZSTD_GEN_DFAST_FN(dictMatchState, 4)
+ZSTD_GEN_DFAST_FN(dictMatchState, 5)
+ZSTD_GEN_DFAST_FN(dictMatchState, 6)
+ZSTD_GEN_DFAST_FN(dictMatchState, 7)
+
size_t ZSTD_compressBlock_doubleFast(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
{
default: /* includes case 3 */
case 4 :
- return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 4, ZSTD_noDict);
+ return ZSTD_compressBlock_doubleFast_noDict_4(ms, seqStore, rep, src, srcSize);
case 5 :
- return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 5, ZSTD_noDict);
+ return ZSTD_compressBlock_doubleFast_noDict_5(ms, seqStore, rep, src, srcSize);
case 6 :
- return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 6, ZSTD_noDict);
+ return ZSTD_compressBlock_doubleFast_noDict_6(ms, seqStore, rep, src, srcSize);
case 7 :
- return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 7, ZSTD_noDict);
+ return ZSTD_compressBlock_doubleFast_noDict_7(ms, seqStore, rep, src, srcSize);
}
}
{
default: /* includes case 3 */
case 4 :
- return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 4, ZSTD_dictMatchState);
+ return ZSTD_compressBlock_doubleFast_dictMatchState_4(ms, seqStore, rep, src, srcSize);
case 5 :
- return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 5, ZSTD_dictMatchState);
+ return ZSTD_compressBlock_doubleFast_dictMatchState_5(ms, seqStore, rep, src, srcSize);
case 6 :
- return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 6, ZSTD_dictMatchState);
+ return ZSTD_compressBlock_doubleFast_dictMatchState_6(ms, seqStore, rep, src, srcSize);
case 7 :
- return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, 7, ZSTD_dictMatchState);
+ return ZSTD_compressBlock_doubleFast_dictMatchState_7(ms, seqStore, rep, src, srcSize);
}
}
/* if extDict is invalidated due to maxDistance, switch to "regular" variant */
if (prefixStartIndex == dictStartIndex)
- return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, mls, ZSTD_noDict);
+ return ZSTD_compressBlock_doubleFast(ms, seqStore, rep, src, srcSize);
/* Search Loop */
while (ip < ilimit) { /* < instead of <=, because (ip+1) */
hashSmall[hSmall] = hashLong[hLong] = curr; /* update hash table */
if ((((U32)((prefixStartIndex-1) - repIndex) >= 3) /* intentional underflow : ensure repIndex doesn't overlap dict + prefix */
- & (repIndex > dictStartIndex))
+ & (offset_1 <= curr+1 - dictStartIndex)) /* note: we are searching at curr+1 */
&& (MEM_read32(repMatch) == MEM_read32(ip+1)) ) {
const BYTE* repMatchEnd = repIndex < prefixStartIndex ? dictEnd : iend;
mLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixStart) + 4;
ip++;
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH);
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_REPCODE_1, mLength);
} else {
if ((matchLongIndex > dictStartIndex) && (MEM_read64(matchLong) == MEM_read64(ip))) {
const BYTE* const matchEnd = matchLongIndex < prefixStartIndex ? dictEnd : iend;
while (((ip>anchor) & (matchLong>lowMatchPtr)) && (ip[-1] == matchLong[-1])) { ip--; matchLong--; mLength++; } /* catch up */
offset_2 = offset_1;
offset_1 = offset;
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH);
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_OFFSET(offset), mLength);
} else if ((matchIndex > dictStartIndex) && (MEM_read32(match) == MEM_read32(ip))) {
size_t const h3 = ZSTD_hashPtr(ip+1, hBitsL, 8);
}
offset_2 = offset_1;
offset_1 = offset;
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH);
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_OFFSET(offset), mLength);
} else {
ip += ((ip-anchor) >> kSearchStrength) + 1;
U32 const repIndex2 = current2 - offset_2;
const BYTE* repMatch2 = repIndex2 < prefixStartIndex ? dictBase + repIndex2 : base + repIndex2;
if ( (((U32)((prefixStartIndex-1) - repIndex2) >= 3) /* intentional overflow : ensure repIndex2 doesn't overlap dict + prefix */
- & (repIndex2 > dictStartIndex))
+ & (offset_2 <= current2 - dictStartIndex))
&& (MEM_read32(repMatch2) == MEM_read32(ip)) ) {
const BYTE* const repEnd2 = repIndex2 < prefixStartIndex ? dictEnd : iend;
size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixStart) + 4;
U32 const tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */
- ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, repLength2-MINMATCH);
+ ZSTD_storeSeq(seqStore, 0, anchor, iend, STORE_REPCODE_1, repLength2);
hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = current2;
hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = current2;
ip += repLength2;
return (size_t)(iend - anchor);
}
+ZSTD_GEN_DFAST_FN(extDict, 4)
+ZSTD_GEN_DFAST_FN(extDict, 5)
+ZSTD_GEN_DFAST_FN(extDict, 6)
+ZSTD_GEN_DFAST_FN(extDict, 7)
size_t ZSTD_compressBlock_doubleFast_extDict(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
{
default: /* includes case 3 */
case 4 :
- return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 4);
+ return ZSTD_compressBlock_doubleFast_extDict_4(ms, seqStore, rep, src, srcSize);
case 5 :
- return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 5);
+ return ZSTD_compressBlock_doubleFast_extDict_5(ms, seqStore, rep, src, srcSize);
case 6 :
- return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 6);
+ return ZSTD_compressBlock_doubleFast_extDict_6(ms, seqStore, rep, src, srcSize);
case 7 :
- return ZSTD_compressBlock_doubleFast_extDict_generic(ms, seqStore, rep, src, srcSize, 7);
+ return ZSTD_compressBlock_doubleFast_extDict_7(ms, seqStore, rep, src, srcSize);
}
}
}
+/*
+ * If you squint hard enough (and ignore repcodes), the search operation at any
+ * given position is broken into 4 stages:
+ *
+ * 1. Hash (map position to hash value via input read)
+ * 2. Lookup (map hash val to index via hashtable read)
+ * 3. Load (map index to value at that position via input read)
+ * 4. Compare
+ *
+ * Each of these steps involves a memory read at an address which is computed
+ * from the previous step. This means these steps must be sequenced and their
+ * latencies are cumulative.
+ *
+ * Rather than do 1->2->3->4 sequentially for a single position before moving
+ * onto the next, this implementation interleaves these operations across the
+ * next few positions:
+ *
+ * R = Repcode Read & Compare
+ * H = Hash
+ * T = Table Lookup
+ * M = Match Read & Compare
+ *
+ * Pos | Time -->
+ * ----+-------------------
+ * N | ... M
+ * N+1 | ... TM
+ * N+2 | R H T M
+ * N+3 | H TM
+ * N+4 | R H T M
+ * N+5 | H ...
+ * N+6 | R ...
+ *
+ * This is very much analogous to the pipelining of execution in a CPU. And just
+ * like a CPU, we have to dump the pipeline when we find a match (i.e., take a
+ * branch).
+ *
+ * When this happens, we throw away our current state, and do the following prep
+ * to re-enter the loop:
+ *
+ * Pos | Time -->
+ * ----+-------------------
+ * N | H T
+ * N+1 | H
+ *
+ * This is also the work we do at the beginning to enter the loop initially.
+ */
FORCE_INLINE_TEMPLATE size_t
-ZSTD_compressBlock_fast_generic(
+ZSTD_compressBlock_fast_noDict_generic(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
void const* src, size_t srcSize,
- U32 const mls)
+ U32 const mls, U32 const hasStep)
{
const ZSTD_compressionParameters* const cParams = &ms->cParams;
U32* const hashTable = ms->hashTable;
U32 const hlog = cParams->hashLog;
/* support stepSize of 0 */
- size_t const stepSize = cParams->targetLength + !(cParams->targetLength) + 1;
+ size_t const stepSize = hasStep ? (cParams->targetLength + !(cParams->targetLength) + 1) : 2;
const BYTE* const base = ms->window.base;
const BYTE* const istart = (const BYTE*)src;
- /* We check ip0 (ip + 0) and ip1 (ip + 1) each loop */
- const BYTE* ip0 = istart;
- const BYTE* ip1;
- const BYTE* anchor = istart;
const U32 endIndex = (U32)((size_t)(istart - base) + srcSize);
const U32 prefixStartIndex = ZSTD_getLowestPrefixIndex(ms, endIndex, cParams->windowLog);
const BYTE* const prefixStart = base + prefixStartIndex;
const BYTE* const iend = istart + srcSize;
const BYTE* const ilimit = iend - HASH_READ_SIZE;
- U32 offset_1=rep[0], offset_2=rep[1];
+
+ const BYTE* anchor = istart;
+ const BYTE* ip0 = istart;
+ const BYTE* ip1;
+ const BYTE* ip2;
+ const BYTE* ip3;
+ U32 current0;
+
+ U32 rep_offset1 = rep[0];
+ U32 rep_offset2 = rep[1];
U32 offsetSaved = 0;
- /* init */
+ size_t hash0; /* hash for ip0 */
+ size_t hash1; /* hash for ip1 */
+ U32 idx; /* match idx for ip0 */
+ U32 mval; /* src value at match idx */
+
+ U32 offcode;
+ const BYTE* match0;
+ size_t mLength;
+
+ /* ip0 and ip1 are always adjacent. The targetLength skipping and
+ * uncompressibility acceleration is applied to every other position,
+ * matching the behavior of #1562. step therefore represents the gap
+ * between pairs of positions, from ip0 to ip2 or ip1 to ip3. */
+ size_t step;
+ const BYTE* nextStep;
+ const size_t kStepIncr = (1 << (kSearchStrength - 1));
+
DEBUGLOG(5, "ZSTD_compressBlock_fast_generic");
ip0 += (ip0 == prefixStart);
- ip1 = ip0 + 1;
{ U32 const curr = (U32)(ip0 - base);
U32 const windowLow = ZSTD_getLowestPrefixIndex(ms, curr, cParams->windowLog);
U32 const maxRep = curr - windowLow;
- if (offset_2 > maxRep) offsetSaved = offset_2, offset_2 = 0;
- if (offset_1 > maxRep) offsetSaved = offset_1, offset_1 = 0;
+ if (rep_offset2 > maxRep) offsetSaved = rep_offset2, rep_offset2 = 0;
+ if (rep_offset1 > maxRep) offsetSaved = rep_offset1, rep_offset1 = 0;
}
- /* Main Search Loop */
-#ifdef __INTEL_COMPILER
- /* From intel 'The vector pragma indicates that the loop should be
- * vectorized if it is legal to do so'. Can be used together with
- * #pragma ivdep (but have opted to exclude that because intel
- * warns against using it).*/
- #pragma vector always
-#endif
- while (ip1 < ilimit) { /* < instead of <=, because check at ip0+2 */
- size_t mLength;
- BYTE const* ip2 = ip0 + 2;
- size_t const h0 = ZSTD_hashPtr(ip0, hlog, mls);
- U32 const val0 = MEM_read32(ip0);
- size_t const h1 = ZSTD_hashPtr(ip1, hlog, mls);
- U32 const val1 = MEM_read32(ip1);
- U32 const current0 = (U32)(ip0-base);
- U32 const current1 = (U32)(ip1-base);
- U32 const matchIndex0 = hashTable[h0];
- U32 const matchIndex1 = hashTable[h1];
- BYTE const* repMatch = ip2 - offset_1;
- const BYTE* match0 = base + matchIndex0;
- const BYTE* match1 = base + matchIndex1;
- U32 offcode;
-
-#if defined(__aarch64__)
- PREFETCH_L1(ip0+256);
-#endif
-
- hashTable[h0] = current0; /* update hash table */
- hashTable[h1] = current1; /* update hash table */
-
- assert(ip0 + 1 == ip1);
-
- if ((offset_1 > 0) & (MEM_read32(repMatch) == MEM_read32(ip2))) {
- mLength = (ip2[-1] == repMatch[-1]) ? 1 : 0;
- ip0 = ip2 - mLength;
- match0 = repMatch - mLength;
+ /* start each op */
+_start: /* Requires: ip0 */
+
+ step = stepSize;
+ nextStep = ip0 + kStepIncr;
+
+ /* calculate positions, ip0 - anchor == 0, so we skip step calc */
+ ip1 = ip0 + 1;
+ ip2 = ip0 + step;
+ ip3 = ip2 + 1;
+
+ if (ip3 >= ilimit) {
+ goto _cleanup;
+ }
+
+ hash0 = ZSTD_hashPtr(ip0, hlog, mls);
+ hash1 = ZSTD_hashPtr(ip1, hlog, mls);
+
+ idx = hashTable[hash0];
+
+ do {
+ /* load repcode match for ip[2]*/
+ const U32 rval = MEM_read32(ip2 - rep_offset1);
+
+ /* write back hash table entry */
+ current0 = (U32)(ip0 - base);
+ hashTable[hash0] = current0;
+
+ /* check repcode at ip[2] */
+ if ((MEM_read32(ip2) == rval) & (rep_offset1 > 0)) {
+ ip0 = ip2;
+ match0 = ip0 - rep_offset1;
+ mLength = ip0[-1] == match0[-1];
+ ip0 -= mLength;
+ match0 -= mLength;
+ offcode = STORE_REPCODE_1;
mLength += 4;
- offcode = 0;
goto _match;
}
- if ((matchIndex0 > prefixStartIndex) && MEM_read32(match0) == val0) {
- /* found a regular match */
- goto _offset;
+
+ /* load match for ip[0] */
+ if (idx >= prefixStartIndex) {
+ mval = MEM_read32(base + idx);
+ } else {
+ mval = MEM_read32(ip0) ^ 1; /* guaranteed to not match. */
}
- if ((matchIndex1 > prefixStartIndex) && MEM_read32(match1) == val1) {
- /* found a regular match after one literal */
- ip0 = ip1;
- match0 = match1;
+
+ /* check match at ip[0] */
+ if (MEM_read32(ip0) == mval) {
+ /* found a match! */
goto _offset;
}
- { size_t const step = ((size_t)(ip0-anchor) >> (kSearchStrength - 1)) + stepSize;
- assert(step >= 2);
- ip0 += step;
- ip1 += step;
- continue;
+
+ /* lookup ip[1] */
+ idx = hashTable[hash1];
+
+ /* hash ip[2] */
+ hash0 = hash1;
+ hash1 = ZSTD_hashPtr(ip2, hlog, mls);
+
+ /* advance to next positions */
+ ip0 = ip1;
+ ip1 = ip2;
+ ip2 = ip3;
+
+ /* write back hash table entry */
+ current0 = (U32)(ip0 - base);
+ hashTable[hash0] = current0;
+
+ /* load match for ip[0] */
+ if (idx >= prefixStartIndex) {
+ mval = MEM_read32(base + idx);
+ } else {
+ mval = MEM_read32(ip0) ^ 1; /* guaranteed to not match. */
}
-_offset: /* Requires: ip0, match0 */
- /* Compute the offset code */
- offset_2 = offset_1;
- offset_1 = (U32)(ip0-match0);
- offcode = offset_1 + ZSTD_REP_MOVE;
- mLength = 4;
- /* Count the backwards match length */
- while (((ip0>anchor) & (match0>prefixStart))
- && (ip0[-1] == match0[-1])) { ip0--; match0--; mLength++; } /* catch up */
-_match: /* Requires: ip0, match0, offcode */
- /* Count the forward length */
- mLength += ZSTD_count(ip0+mLength, match0+mLength, iend);
- ZSTD_storeSeq(seqStore, (size_t)(ip0-anchor), anchor, iend, offcode, mLength-MINMATCH);
- /* match found */
- ip0 += mLength;
- anchor = ip0;
+ /* check match at ip[0] */
+ if (MEM_read32(ip0) == mval) {
+ /* found a match! */
+ goto _offset;
+ }
- if (ip0 <= ilimit) {
- /* Fill Table */
- assert(base+current0+2 > istart); /* check base overflow */
- hashTable[ZSTD_hashPtr(base+current0+2, hlog, mls)] = current0+2; /* here because current+2 could be > iend-8 */
- hashTable[ZSTD_hashPtr(ip0-2, hlog, mls)] = (U32)(ip0-2-base);
-
- if (offset_2 > 0) { /* offset_2==0 means offset_2 is invalidated */
- while ( (ip0 <= ilimit) && (MEM_read32(ip0) == MEM_read32(ip0 - offset_2)) ) {
- /* store sequence */
- size_t const rLength = ZSTD_count(ip0+4, ip0+4-offset_2, iend) + 4;
- { U32 const tmpOff = offset_2; offset_2 = offset_1; offset_1 = tmpOff; } /* swap offset_2 <=> offset_1 */
- hashTable[ZSTD_hashPtr(ip0, hlog, mls)] = (U32)(ip0-base);
- ip0 += rLength;
- ZSTD_storeSeq(seqStore, 0 /*litLen*/, anchor, iend, 0 /*offCode*/, rLength-MINMATCH);
- anchor = ip0;
- continue; /* faster when present (confirmed on gcc-8) ... (?) */
- } } }
- ip1 = ip0 + 1;
- }
+ /* lookup ip[1] */
+ idx = hashTable[hash1];
+
+ /* hash ip[2] */
+ hash0 = hash1;
+ hash1 = ZSTD_hashPtr(ip2, hlog, mls);
+
+ /* advance to next positions */
+ ip0 = ip1;
+ ip1 = ip2;
+ ip2 = ip0 + step;
+ ip3 = ip1 + step;
+
+ /* calculate step */
+ if (ip2 >= nextStep) {
+ step++;
+ PREFETCH_L1(ip1 + 64);
+ PREFETCH_L1(ip1 + 128);
+ nextStep += kStepIncr;
+ }
+ } while (ip3 < ilimit);
+
+_cleanup:
+ /* Note that there are probably still a couple positions we could search.
+ * However, it seems to be a meaningful performance hit to try to search
+ * them. So let's not. */
/* save reps for next block */
- rep[0] = offset_1 ? offset_1 : offsetSaved;
- rep[1] = offset_2 ? offset_2 : offsetSaved;
+ rep[0] = rep_offset1 ? rep_offset1 : offsetSaved;
+ rep[1] = rep_offset2 ? rep_offset2 : offsetSaved;
/* Return the last literals size */
return (size_t)(iend - anchor);
+
+_offset: /* Requires: ip0, idx */
+
+ /* Compute the offset code. */
+ match0 = base + idx;
+ rep_offset2 = rep_offset1;
+ rep_offset1 = (U32)(ip0-match0);
+ offcode = STORE_OFFSET(rep_offset1);
+ mLength = 4;
+
+ /* Count the backwards match length. */
+ while (((ip0>anchor) & (match0>prefixStart)) && (ip0[-1] == match0[-1])) {
+ ip0--;
+ match0--;
+ mLength++;
+ }
+
+_match: /* Requires: ip0, match0, offcode */
+
+ /* Count the forward length. */
+ mLength += ZSTD_count(ip0 + mLength, match0 + mLength, iend);
+
+ ZSTD_storeSeq(seqStore, (size_t)(ip0 - anchor), anchor, iend, offcode, mLength);
+
+ ip0 += mLength;
+ anchor = ip0;
+
+ /* write next hash table entry */
+ if (ip1 < ip0) {
+ hashTable[hash1] = (U32)(ip1 - base);
+ }
+
+ /* Fill table and check for immediate repcode. */
+ if (ip0 <= ilimit) {
+ /* Fill Table */
+ assert(base+current0+2 > istart); /* check base overflow */
+ hashTable[ZSTD_hashPtr(base+current0+2, hlog, mls)] = current0+2; /* here because current+2 could be > iend-8 */
+ hashTable[ZSTD_hashPtr(ip0-2, hlog, mls)] = (U32)(ip0-2-base);
+
+ if (rep_offset2 > 0) { /* rep_offset2==0 means rep_offset2 is invalidated */
+ while ( (ip0 <= ilimit) && (MEM_read32(ip0) == MEM_read32(ip0 - rep_offset2)) ) {
+ /* store sequence */
+ size_t const rLength = ZSTD_count(ip0+4, ip0+4-rep_offset2, iend) + 4;
+ { U32 const tmpOff = rep_offset2; rep_offset2 = rep_offset1; rep_offset1 = tmpOff; } /* swap rep_offset2 <=> rep_offset1 */
+ hashTable[ZSTD_hashPtr(ip0, hlog, mls)] = (U32)(ip0-base);
+ ip0 += rLength;
+ ZSTD_storeSeq(seqStore, 0 /*litLen*/, anchor, iend, STORE_REPCODE_1, rLength);
+ anchor = ip0;
+ continue; /* faster when present (confirmed on gcc-8) ... (?) */
+ } } }
+
+ goto _start;
}
+#define ZSTD_GEN_FAST_FN(dictMode, mls, step) \
+ static size_t ZSTD_compressBlock_fast_##dictMode##_##mls##_##step( \
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM], \
+ void const* src, size_t srcSize) \
+ { \
+ return ZSTD_compressBlock_fast_##dictMode##_generic(ms, seqStore, rep, src, srcSize, mls, step); \
+ }
+
+ZSTD_GEN_FAST_FN(noDict, 4, 1)
+ZSTD_GEN_FAST_FN(noDict, 5, 1)
+ZSTD_GEN_FAST_FN(noDict, 6, 1)
+ZSTD_GEN_FAST_FN(noDict, 7, 1)
+
+ZSTD_GEN_FAST_FN(noDict, 4, 0)
+ZSTD_GEN_FAST_FN(noDict, 5, 0)
+ZSTD_GEN_FAST_FN(noDict, 6, 0)
+ZSTD_GEN_FAST_FN(noDict, 7, 0)
size_t ZSTD_compressBlock_fast(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
{
U32 const mls = ms->cParams.minMatch;
assert(ms->dictMatchState == NULL);
- switch(mls)
- {
- default: /* includes case 3 */
- case 4 :
- return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 4);
- case 5 :
- return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 5);
- case 6 :
- return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 6);
- case 7 :
- return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, 7);
+ if (ms->cParams.targetLength > 1) {
+ switch(mls)
+ {
+ default: /* includes case 3 */
+ case 4 :
+ return ZSTD_compressBlock_fast_noDict_4_1(ms, seqStore, rep, src, srcSize);
+ case 5 :
+ return ZSTD_compressBlock_fast_noDict_5_1(ms, seqStore, rep, src, srcSize);
+ case 6 :
+ return ZSTD_compressBlock_fast_noDict_6_1(ms, seqStore, rep, src, srcSize);
+ case 7 :
+ return ZSTD_compressBlock_fast_noDict_7_1(ms, seqStore, rep, src, srcSize);
+ }
+ } else {
+ switch(mls)
+ {
+ default: /* includes case 3 */
+ case 4 :
+ return ZSTD_compressBlock_fast_noDict_4_0(ms, seqStore, rep, src, srcSize);
+ case 5 :
+ return ZSTD_compressBlock_fast_noDict_5_0(ms, seqStore, rep, src, srcSize);
+ case 6 :
+ return ZSTD_compressBlock_fast_noDict_6_0(ms, seqStore, rep, src, srcSize);
+ case 7 :
+ return ZSTD_compressBlock_fast_noDict_7_0(ms, seqStore, rep, src, srcSize);
+ }
+
}
}
FORCE_INLINE_TEMPLATE
size_t ZSTD_compressBlock_fast_dictMatchState_generic(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
- void const* src, size_t srcSize, U32 const mls)
+ void const* src, size_t srcSize, U32 const mls, U32 const hasStep)
{
const ZSTD_compressionParameters* const cParams = &ms->cParams;
U32* const hashTable = ms->hashTable;
assert(endIndex - prefixStartIndex <= maxDistance);
(void)maxDistance; (void)endIndex; /* these variables are not used when assert() is disabled */
+ (void)hasStep; /* not currently specialized on whether it's accelerated */
+
/* ensure there will be no underflow
* when translating a dict index into a local index */
assert(prefixStartIndex >= (U32)(dictEnd - dictBase));
const BYTE* const repMatchEnd = repIndex < prefixStartIndex ? dictEnd : iend;
mLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixStart) + 4;
ip++;
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH);
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_REPCODE_1, mLength);
} else if ( (matchIndex <= prefixStartIndex) ) {
size_t const dictHash = ZSTD_hashPtr(ip, dictHLog, mls);
U32 const dictMatchIndex = dictHashTable[dictHash];
} /* catch up */
offset_2 = offset_1;
offset_1 = offset;
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH);
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_OFFSET(offset), mLength);
}
} else if (MEM_read32(match) != MEM_read32(ip)) {
/* it's not a match, and we're not going to check the dictionary */
&& (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */
offset_2 = offset_1;
offset_1 = offset;
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH);
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_OFFSET(offset), mLength);
}
/* match found */
const BYTE* const repEnd2 = repIndex2 < prefixStartIndex ? dictEnd : iend;
size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixStart) + 4;
U32 tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */
- ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, repLength2-MINMATCH);
+ ZSTD_storeSeq(seqStore, 0, anchor, iend, STORE_REPCODE_1, repLength2);
hashTable[ZSTD_hashPtr(ip, hlog, mls)] = current2;
ip += repLength2;
anchor = ip;
return (size_t)(iend - anchor);
}
+
+ZSTD_GEN_FAST_FN(dictMatchState, 4, 0)
+ZSTD_GEN_FAST_FN(dictMatchState, 5, 0)
+ZSTD_GEN_FAST_FN(dictMatchState, 6, 0)
+ZSTD_GEN_FAST_FN(dictMatchState, 7, 0)
+
size_t ZSTD_compressBlock_fast_dictMatchState(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
void const* src, size_t srcSize)
{
default: /* includes case 3 */
case 4 :
- return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 4);
+ return ZSTD_compressBlock_fast_dictMatchState_4_0(ms, seqStore, rep, src, srcSize);
case 5 :
- return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 5);
+ return ZSTD_compressBlock_fast_dictMatchState_5_0(ms, seqStore, rep, src, srcSize);
case 6 :
- return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 6);
+ return ZSTD_compressBlock_fast_dictMatchState_6_0(ms, seqStore, rep, src, srcSize);
case 7 :
- return ZSTD_compressBlock_fast_dictMatchState_generic(ms, seqStore, rep, src, srcSize, 7);
+ return ZSTD_compressBlock_fast_dictMatchState_7_0(ms, seqStore, rep, src, srcSize);
}
}
static size_t ZSTD_compressBlock_fast_extDict_generic(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
- void const* src, size_t srcSize, U32 const mls)
+ void const* src, size_t srcSize, U32 const mls, U32 const hasStep)
{
const ZSTD_compressionParameters* const cParams = &ms->cParams;
U32* const hashTable = ms->hashTable;
const BYTE* const ilimit = iend - 8;
U32 offset_1=rep[0], offset_2=rep[1];
+ (void)hasStep; /* not currently specialized on whether it's accelerated */
+
DEBUGLOG(5, "ZSTD_compressBlock_fast_extDict_generic (offset_1=%u)", offset_1);
/* switch to "regular" variant if extDict is invalidated due to maxDistance */
if (prefixStartIndex == dictStartIndex)
- return ZSTD_compressBlock_fast_generic(ms, seqStore, rep, src, srcSize, mls);
+ return ZSTD_compressBlock_fast(ms, seqStore, rep, src, srcSize);
/* Search Loop */
while (ip < ilimit) { /* < instead of <=, because (ip+1) */
const BYTE* const repMatch = repBase + repIndex;
hashTable[h] = curr; /* update hash table */
DEBUGLOG(7, "offset_1 = %u , curr = %u", offset_1, curr);
- assert(offset_1 <= curr +1); /* check repIndex */
- if ( (((U32)((prefixStartIndex-1) - repIndex) >= 3) /* intentional underflow */ & (repIndex > dictStartIndex))
+ if ( ( ((U32)((prefixStartIndex-1) - repIndex) >= 3) /* intentional underflow */
+ & (offset_1 <= curr+1 - dictStartIndex) ) /* note: we are searching at curr+1 */
&& (MEM_read32(repMatch) == MEM_read32(ip+1)) ) {
const BYTE* const repMatchEnd = repIndex < prefixStartIndex ? dictEnd : iend;
size_t const rLength = ZSTD_count_2segments(ip+1 +4, repMatch +4, iend, repMatchEnd, prefixStart) + 4;
ip++;
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, rLength-MINMATCH);
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_REPCODE_1, rLength);
ip += rLength;
anchor = ip;
} else {
size_t mLength = ZSTD_count_2segments(ip+4, match+4, iend, matchEnd, prefixStart) + 4;
while (((ip>anchor) & (match>lowMatchPtr)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */
offset_2 = offset_1; offset_1 = offset; /* update offset history */
- ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH);
+ ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, STORE_OFFSET(offset), mLength);
ip += mLength;
anchor = ip;
} }
U32 const current2 = (U32)(ip-base);
U32 const repIndex2 = current2 - offset_2;
const BYTE* const repMatch2 = repIndex2 < prefixStartIndex ? dictBase + repIndex2 : base + repIndex2;
- if ( (((U32)((prefixStartIndex-1) - repIndex2) >= 3) & (repIndex2 > dictStartIndex)) /* intentional overflow */
+ if ( (((U32)((prefixStartIndex-1) - repIndex2) >= 3) & (offset_2 <= curr - dictStartIndex)) /* intentional overflow */
&& (MEM_read32(repMatch2) == MEM_read32(ip)) ) {
const BYTE* const repEnd2 = repIndex2 < prefixStartIndex ? dictEnd : iend;
size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixStart) + 4;
{ U32 const tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; } /* swap offset_2 <=> offset_1 */
- ZSTD_storeSeq(seqStore, 0 /*litlen*/, anchor, iend, 0 /*offcode*/, repLength2-MINMATCH);
+ ZSTD_storeSeq(seqStore, 0 /*litlen*/, anchor, iend, STORE_REPCODE_1, repLength2);
hashTable[ZSTD_hashPtr(ip, hlog, mls)] = current2;
ip += repLength2;
anchor = ip;
return (size_t)(iend - anchor);
}
+ZSTD_GEN_FAST_FN(extDict, 4, 0)
+ZSTD_GEN_FAST_FN(extDict, 5, 0)
+ZSTD_GEN_FAST_FN(extDict, 6, 0)
+ZSTD_GEN_FAST_FN(extDict, 7, 0)
size_t ZSTD_compressBlock_fast_extDict(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
{
default: /* includes case 3 */
case 4 :
- return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 4);
+ return ZSTD_compressBlock_fast_extDict_4_0(ms, seqStore, rep, src, srcSize);
case 5 :
- return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 5);
+ return ZSTD_compressBlock_fast_extDict_5_0(ms, seqStore, rep, src, srcSize);
case 6 :
- return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 6);
+ return ZSTD_compressBlock_fast_extDict_6_0(ms, seqStore, rep, src, srcSize);
case 7 :
- return ZSTD_compressBlock_fast_extDict_generic(ms, seqStore, rep, src, srcSize, 7);
+ return ZSTD_compressBlock_fast_extDict_7_0(ms, seqStore, rep, src, srcSize);
}
}
* assumption : curr >= btlow == (curr - btmask)
* doesn't fail */
static void
-ZSTD_insertDUBT1(ZSTD_matchState_t* ms,
+ZSTD_insertDUBT1(const ZSTD_matchState_t* ms,
U32 curr, const BYTE* inputEnd,
U32 nbCompares, U32 btLow,
const ZSTD_dictMode_e dictMode)
static size_t
ZSTD_DUBT_findBetterDictMatch (
- ZSTD_matchState_t* ms,
+ const ZSTD_matchState_t* ms,
const BYTE* const ip, const BYTE* const iend,
size_t* offsetPtr,
size_t bestLength,
U32 matchIndex = dictMatchIndex + dictIndexDelta;
if ( (4*(int)(matchLength-bestLength)) > (int)(ZSTD_highbit32(curr-matchIndex+1) - ZSTD_highbit32((U32)offsetPtr[0]+1)) ) {
DEBUGLOG(9, "ZSTD_DUBT_findBetterDictMatch(%u) : found better match length %u -> %u and offsetCode %u -> %u (dictMatchIndex %u, matchIndex %u)",
- curr, (U32)bestLength, (U32)matchLength, (U32)*offsetPtr, ZSTD_REP_MOVE + curr - matchIndex, dictMatchIndex, matchIndex);
- bestLength = matchLength, *offsetPtr = ZSTD_REP_MOVE + curr - matchIndex;
+ curr, (U32)bestLength, (U32)matchLength, (U32)*offsetPtr, STORE_OFFSET(curr - matchIndex), dictMatchIndex, matchIndex);
+ bestLength = matchLength, *offsetPtr = STORE_OFFSET(curr - matchIndex);
}
if (ip+matchLength == iend) { /* reached end of input : ip[matchLength] is not valid, no way to know if it's larger or smaller than match */
break; /* drop, to guarantee consistency (miss a little bit of compression) */
}
if (bestLength >= MINMATCH) {
- U32 const mIndex = curr - ((U32)*offsetPtr - ZSTD_REP_MOVE); (void)mIndex;
+ U32 const mIndex = curr - (U32)STORED_OFFSET(*offsetPtr); (void)mIndex;
DEBUGLOG(8, "ZSTD_DUBT_findBetterDictMatch(%u) : found match of length %u and offsetCode %u (pos %u)",
curr, (U32)bestLength, (U32)*offsetPtr, mIndex);
}
if (matchLength > matchEndIdx - matchIndex)
matchEndIdx = matchIndex + (U32)matchLength;
if ( (4*(int)(matchLength-bestLength)) > (int)(ZSTD_highbit32(curr-matchIndex+1) - ZSTD_highbit32((U32)offsetPtr[0]+1)) )
- bestLength = matchLength, *offsetPtr = ZSTD_REP_MOVE + curr - matchIndex;
+ bestLength = matchLength, *offsetPtr = STORE_OFFSET(curr - matchIndex);
if (ip+matchLength == iend) { /* equal : no way to know if inf or sup */
if (dictMode == ZSTD_dictMatchState) {
nbCompares = 0; /* in addition to avoiding checking any
assert(matchEndIdx > curr+8); /* ensure nextToUpdate is increased */
ms->nextToUpdate = matchEndIdx - 8; /* skip repetitive patterns */
if (bestLength >= MINMATCH) {
- U32 const mIndex = curr - ((U32)*offsetPtr - ZSTD_REP_MOVE); (void)mIndex;
+ U32 const mIndex = curr - (U32)STORED_OFFSET(*offsetPtr); (void)mIndex;
DEBUGLOG(8, "ZSTD_DUBT_findBestMatch(%u) : found match of length %u and offsetCode %u (pos %u)",
curr, (U32)bestLength, (U32)*offsetPtr, mIndex);
}
return ZSTD_DUBT_findBestMatch(ms, ip, iLimit, offsetPtr, mls, dictMode);
}
-
-static size_t
-ZSTD_BtFindBestMatch_selectMLS ( ZSTD_matchState_t* ms,
- const BYTE* ip, const BYTE* const iLimit,
- size_t* offsetPtr)
-{
- switch(ms->cParams.minMatch)
- {
- default : /* includes case 3 */
- case 4 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 4, ZSTD_noDict);
- case 5 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 5, ZSTD_noDict);
- case 7 :
- case 6 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 6, ZSTD_noDict);
- }
-}
-
-
-static size_t ZSTD_BtFindBestMatch_dictMatchState_selectMLS (
- ZSTD_matchState_t* ms,
- const BYTE* ip, const BYTE* const iLimit,
- size_t* offsetPtr)
-{
- switch(ms->cParams.minMatch)
- {
- default : /* includes case 3 */
- case 4 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 4, ZSTD_dictMatchState);
- case 5 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 5, ZSTD_dictMatchState);
- case 7 :
- case 6 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 6, ZSTD_dictMatchState);
- }
-}
-
-
-static size_t ZSTD_BtFindBestMatch_extDict_selectMLS (
- ZSTD_matchState_t* ms,
- const BYTE* ip, const BYTE* const iLimit,
- size_t* offsetPtr)
-{
- switch(ms->cParams.minMatch)
- {
- default : /* includes case 3 */
- case 4 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 4, ZSTD_extDict);
- case 5 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 5, ZSTD_extDict);
- case 7 :
- case 6 : return ZSTD_BtFindBestMatch(ms, ip, iLimit, offsetPtr, 6, ZSTD_extDict);
- }
-}
-
-
-
/* *********************************
-* Hash Chain
+* Dedicated dict search
***********************************/
-#define NEXT_IN_CHAIN(d, mask) chainTable[(d) & (mask)]
-
-/* Update chains up to ip (excluded)
- Assumption : always within prefix (i.e. not within extDict) */
-FORCE_INLINE_TEMPLATE U32 ZSTD_insertAndFindFirstIndex_internal(
- ZSTD_matchState_t* ms,
- const ZSTD_compressionParameters* const cParams,
- const BYTE* ip, U32 const mls)
-{
- U32* const hashTable = ms->hashTable;
- const U32 hashLog = cParams->hashLog;
- U32* const chainTable = ms->chainTable;
- const U32 chainMask = (1 << cParams->chainLog) - 1;
- const BYTE* const base = ms->window.base;
- const U32 target = (U32)(ip - base);
- U32 idx = ms->nextToUpdate;
-
- while(idx < target) { /* catch up */
- size_t const h = ZSTD_hashPtr(base+idx, hashLog, mls);
- NEXT_IN_CHAIN(idx, chainMask) = hashTable[h];
- hashTable[h] = idx;
- idx++;
- }
-
- ms->nextToUpdate = target;
- return hashTable[ZSTD_hashPtr(ip, hashLog, mls)];
-}
-
-U32 ZSTD_insertAndFindFirstIndex(ZSTD_matchState_t* ms, const BYTE* ip) {
- const ZSTD_compressionParameters* const cParams = &ms->cParams;
- return ZSTD_insertAndFindFirstIndex_internal(ms, cParams, ip, ms->cParams.minMatch);
-}
void ZSTD_dedicatedDictSearch_lazy_loadDictionary(ZSTD_matchState_t* ms, const BYTE* const ip)
{
U32* const chainTable = ms->chainTable;
U32 const chainSize = 1 << ms->cParams.chainLog;
U32 idx = ms->nextToUpdate;
- U32 const minChain = chainSize < target ? target - chainSize : idx;
+ U32 const minChain = chainSize < target - idx ? target - chainSize : idx;
U32 const bucketSize = 1 << ZSTD_LAZY_DDSS_BUCKET_LOG;
U32 const cacheSize = bucketSize - 1;
U32 const chainAttempts = (1 << ms->cParams.searchLog) - cacheSize;
U32 const hashLog = ms->cParams.hashLog - ZSTD_LAZY_DDSS_BUCKET_LOG;
U32* const tmpHashTable = hashTable;
U32* const tmpChainTable = hashTable + ((size_t)1 << hashLog);
- U32 const tmpChainSize = ((1 << ZSTD_LAZY_DDSS_BUCKET_LOG) - 1) << hashLog;
+ U32 const tmpChainSize = (U32)((1 << ZSTD_LAZY_DDSS_BUCKET_LOG) - 1) << hashLog;
U32 const tmpMinChain = tmpChainSize < target ? target - tmpChainSize : idx;
-
U32 hashIdx;
assert(ms->cParams.chainLog <= 24);
- assert(ms->cParams.hashLog >= ms->cParams.chainLog);
+ assert(ms->cParams.hashLog > ms->cParams.chainLog);
assert(idx != 0);
assert(tmpMinChain <= minChain);
if (count == cacheSize) {
for (count = 0; count < chainLimit;) {
if (i < minChain) {
- if (!i || countBeyondMinChain++ > cacheSize) {
+ if (!i || ++countBeyondMinChain > cacheSize) {
/* only allow pulling `cacheSize` number of entries
* into the cache or chainTable beyond `minChain`,
* to replace the entries pulled out of the
ms->nextToUpdate = target;
}
+/* Returns the longest match length found in the dedicated dict search structure.
+ * If none are longer than the argument ml, then ml will be returned.
+ */
+FORCE_INLINE_TEMPLATE
+size_t ZSTD_dedicatedDictSearch_lazy_search(size_t* offsetPtr, size_t ml, U32 nbAttempts,
+ const ZSTD_matchState_t* const dms,
+ const BYTE* const ip, const BYTE* const iLimit,
+ const BYTE* const prefixStart, const U32 curr,
+ const U32 dictLimit, const size_t ddsIdx) {
+ const U32 ddsLowestIndex = dms->window.dictLimit;
+ const BYTE* const ddsBase = dms->window.base;
+ const BYTE* const ddsEnd = dms->window.nextSrc;
+ const U32 ddsSize = (U32)(ddsEnd - ddsBase);
+ const U32 ddsIndexDelta = dictLimit - ddsSize;
+ const U32 bucketSize = (1 << ZSTD_LAZY_DDSS_BUCKET_LOG);
+ const U32 bucketLimit = nbAttempts < bucketSize - 1 ? nbAttempts : bucketSize - 1;
+ U32 ddsAttempt;
+ U32 matchIndex;
+
+ for (ddsAttempt = 0; ddsAttempt < bucketSize - 1; ddsAttempt++) {
+ PREFETCH_L1(ddsBase + dms->hashTable[ddsIdx + ddsAttempt]);
+ }
+
+ {
+ U32 const chainPackedPointer = dms->hashTable[ddsIdx + bucketSize - 1];
+ U32 const chainIndex = chainPackedPointer >> 8;
+
+ PREFETCH_L1(&dms->chainTable[chainIndex]);
+ }
+
+ for (ddsAttempt = 0; ddsAttempt < bucketLimit; ddsAttempt++) {
+ size_t currentMl=0;
+ const BYTE* match;
+ matchIndex = dms->hashTable[ddsIdx + ddsAttempt];
+ match = ddsBase + matchIndex;
+
+ if (!matchIndex) {
+ return ml;
+ }
+
+ /* guaranteed by table construction */
+ (void)ddsLowestIndex;
+ assert(matchIndex >= ddsLowestIndex);
+ assert(match+4 <= ddsEnd);
+ if (MEM_read32(match) == MEM_read32(ip)) {
+ /* assumption : matchIndex <= dictLimit-4 (by table construction) */
+ currentMl = ZSTD_count_2segments(ip+4, match+4, iLimit, ddsEnd, prefixStart) + 4;
+ }
+
+ /* save best solution */
+ if (currentMl > ml) {
+ ml = currentMl;
+ *offsetPtr = STORE_OFFSET(curr - (matchIndex + ddsIndexDelta));
+ if (ip+currentMl == iLimit) {
+ /* best possible, avoids read overflow on next attempt */
+ return ml;
+ }
+ }
+ }
+
+ {
+ U32 const chainPackedPointer = dms->hashTable[ddsIdx + bucketSize - 1];
+ U32 chainIndex = chainPackedPointer >> 8;
+ U32 const chainLength = chainPackedPointer & 0xFF;
+ U32 const chainAttempts = nbAttempts - ddsAttempt;
+ U32 const chainLimit = chainAttempts > chainLength ? chainLength : chainAttempts;
+ U32 chainAttempt;
+
+ for (chainAttempt = 0 ; chainAttempt < chainLimit; chainAttempt++) {
+ PREFETCH_L1(ddsBase + dms->chainTable[chainIndex + chainAttempt]);
+ }
+
+ for (chainAttempt = 0 ; chainAttempt < chainLimit; chainAttempt++, chainIndex++) {
+ size_t currentMl=0;
+ const BYTE* match;
+ matchIndex = dms->chainTable[chainIndex];
+ match = ddsBase + matchIndex;
+
+ /* guaranteed by table construction */
+ assert(matchIndex >= ddsLowestIndex);
+ assert(match+4 <= ddsEnd);
+ if (MEM_read32(match) == MEM_read32(ip)) {
+ /* assumption : matchIndex <= dictLimit-4 (by table construction) */
+ currentMl = ZSTD_count_2segments(ip+4, match+4, iLimit, ddsEnd, prefixStart) + 4;
+ }
+
+ /* save best solution */
+ if (currentMl > ml) {
+ ml = currentMl;
+ *offsetPtr = STORE_OFFSET(curr - (matchIndex + ddsIndexDelta));
+ if (ip+currentMl == iLimit) break; /* best possible, avoids read overflow on next attempt */
+ }
+ }
+ }
+ return ml;
+}
+
+
+/* *********************************
+* Hash Chain
+***********************************/
+#define NEXT_IN_CHAIN(d, mask) chainTable[(d) & (mask)]
+
+/* Update chains up to ip (excluded)
+ Assumption : always within prefix (i.e. not within extDict) */
+FORCE_INLINE_TEMPLATE U32 ZSTD_insertAndFindFirstIndex_internal(
+ ZSTD_matchState_t* ms,
+ const ZSTD_compressionParameters* const cParams,
+ const BYTE* ip, U32 const mls)
+{
+ U32* const hashTable = ms->hashTable;
+ const U32 hashLog = cParams->hashLog;
+ U32* const chainTable = ms->chainTable;
+ const U32 chainMask = (1 << cParams->chainLog) - 1;
+ const BYTE* const base = ms->window.base;
+ const U32 target = (U32)(ip - base);
+ U32 idx = ms->nextToUpdate;
+
+ while(idx < target) { /* catch up */
+ size_t const h = ZSTD_hashPtr(base+idx, hashLog, mls);
+ NEXT_IN_CHAIN(idx, chainMask) = hashTable[h];
+ hashTable[h] = idx;
+ idx++;
+ }
+
+ ms->nextToUpdate = target;
+ return hashTable[ZSTD_hashPtr(ip, hashLog, mls)];
+}
+
+U32 ZSTD_insertAndFindFirstIndex(ZSTD_matchState_t* ms, const BYTE* ip) {
+ const ZSTD_compressionParameters* const cParams = &ms->cParams;
+ return ZSTD_insertAndFindFirstIndex_internal(ms, cParams, ip, ms->cParams.minMatch);
+}
/* inlining is important to hardwire a hot branch (template emulation) */
FORCE_INLINE_TEMPLATE
-size_t ZSTD_HcFindBestMatch_generic (
+size_t ZSTD_HcFindBestMatch(
ZSTD_matchState_t* ms,
const BYTE* const ip, const BYTE* const iLimit,
size_t* offsetPtr,
/* save best solution */
if (currentMl > ml) {
ml = currentMl;
- *offsetPtr = curr - matchIndex + ZSTD_REP_MOVE;
+ *offsetPtr = STORE_OFFSET(curr - matchIndex);
if (ip+currentMl == iLimit) break; /* best possible, avoids read overflow on next attempt */
}
assert(nbAttempts <= (1U << ZSTD_SEARCHLOG_MAX)); /* Check we haven't underflowed. */
if (dictMode == ZSTD_dedicatedDictSearch) {
- const U32 ddsLowestIndex = dms->window.dictLimit;
- const BYTE* const ddsBase = dms->window.base;
- const BYTE* const ddsEnd = dms->window.nextSrc;
- const U32 ddsSize = (U32)(ddsEnd - ddsBase);
- const U32 ddsIndexDelta = dictLimit - ddsSize;
- const U32 bucketSize = (1 << ZSTD_LAZY_DDSS_BUCKET_LOG);
- const U32 bucketLimit = nbAttempts < bucketSize - 1 ? nbAttempts : bucketSize - 1;
- U32 ddsAttempt;
-
- for (ddsAttempt = 0; ddsAttempt < bucketSize - 1; ddsAttempt++) {
- PREFETCH_L1(ddsBase + dms->hashTable[ddsIdx + ddsAttempt]);
- }
-
- {
- U32 const chainPackedPointer = dms->hashTable[ddsIdx + bucketSize - 1];
- U32 const chainIndex = chainPackedPointer >> 8;
-
- PREFETCH_L1(&dms->chainTable[chainIndex]);
- }
-
- for (ddsAttempt = 0; ddsAttempt < bucketLimit; ddsAttempt++) {
- size_t currentMl=0;
- const BYTE* match;
- matchIndex = dms->hashTable[ddsIdx + ddsAttempt];
- match = ddsBase + matchIndex;
-
- if (!matchIndex) {
- return ml;
- }
-
- /* guaranteed by table construction */
- (void)ddsLowestIndex;
- assert(matchIndex >= ddsLowestIndex);
- assert(match+4 <= ddsEnd);
- if (MEM_read32(match) == MEM_read32(ip)) {
- /* assumption : matchIndex <= dictLimit-4 (by table construction) */
- currentMl = ZSTD_count_2segments(ip+4, match+4, iLimit, ddsEnd, prefixStart) + 4;
- }
-
- /* save best solution */
- if (currentMl > ml) {
- ml = currentMl;
- *offsetPtr = curr - (matchIndex + ddsIndexDelta) + ZSTD_REP_MOVE;
- if (ip+currentMl == iLimit) {
- /* best possible, avoids read overflow on next attempt */
- return ml;
- }
- }
- }
-
- {
- U32 const chainPackedPointer = dms->hashTable[ddsIdx + bucketSize - 1];
- U32 chainIndex = chainPackedPointer >> 8;
- U32 const chainLength = chainPackedPointer & 0xFF;
- U32 const chainAttempts = nbAttempts - ddsAttempt;
- U32 const chainLimit = chainAttempts > chainLength ? chainLength : chainAttempts;
- U32 chainAttempt;
-
- for (chainAttempt = 0 ; chainAttempt < chainLimit; chainAttempt++) {
- PREFETCH_L1(ddsBase + dms->chainTable[chainIndex + chainAttempt]);
- }
-
- for (chainAttempt = 0 ; chainAttempt < chainLimit; chainAttempt++, chainIndex++) {
- size_t currentMl=0;
- const BYTE* match;
- matchIndex = dms->chainTable[chainIndex];
- match = ddsBase + matchIndex;
-
- /* guaranteed by table construction */
- assert(matchIndex >= ddsLowestIndex);
- assert(match+4 <= ddsEnd);
- if (MEM_read32(match) == MEM_read32(ip)) {
- /* assumption : matchIndex <= dictLimit-4 (by table construction) */
- currentMl = ZSTD_count_2segments(ip+4, match+4, iLimit, ddsEnd, prefixStart) + 4;
- }
-
- /* save best solution */
- if (currentMl > ml) {
- ml = currentMl;
- *offsetPtr = curr - (matchIndex + ddsIndexDelta) + ZSTD_REP_MOVE;
- if (ip+currentMl == iLimit) break; /* best possible, avoids read overflow on next attempt */
- }
- }
- }
+ ml = ZSTD_dedicatedDictSearch_lazy_search(offsetPtr, ml, nbAttempts, dms,
+ ip, iLimit, prefixStart, curr, dictLimit, ddsIdx);
} else if (dictMode == ZSTD_dictMatchState) {
const U32* const dmsChainTable = dms->chainTable;
const U32 dmsChainSize = (1 << dms->cParams.chainLog);
/* save best solution */
if (currentMl > ml) {
ml = currentMl;
- *offsetPtr = curr - (matchIndex + dmsIndexDelta) + ZSTD_REP_MOVE;
+ assert(curr > matchIndex + dmsIndexDelta);
+ *offsetPtr = STORE_OFFSET(curr - (matchIndex + dmsIndexDelta));
if (ip+currentMl == iLimit) break; /* best possible, avoids read overflow on next attempt */
}
return ml;
}
+/* *********************************
+* (SIMD) Row-based matchfinder
+***********************************/
+/* Constants for row-based hash */
+#define ZSTD_ROW_HASH_TAG_OFFSET 16 /* byte offset of hashes in the match state's tagTable from the beginning of a row */
+#define ZSTD_ROW_HASH_TAG_BITS 8 /* nb bits to use for the tag */
+#define ZSTD_ROW_HASH_TAG_MASK ((1u << ZSTD_ROW_HASH_TAG_BITS) - 1)
+#define ZSTD_ROW_HASH_MAX_ENTRIES 64 /* absolute maximum number of entries per row, for all configurations */
+
+#define ZSTD_ROW_HASH_CACHE_MASK (ZSTD_ROW_HASH_CACHE_SIZE - 1)
-FORCE_INLINE_TEMPLATE size_t ZSTD_HcFindBestMatch_selectMLS (
- ZSTD_matchState_t* ms,
- const BYTE* ip, const BYTE* const iLimit,
- size_t* offsetPtr)
+typedef U64 ZSTD_VecMask; /* Clarifies when we are interacting with a U64 representing a mask of matches */
+
+/* ZSTD_VecMask_next():
+ * Starting from the LSB, returns the idx of the next non-zero bit.
+ * Basically counting the nb of trailing zeroes.
+ */
+static U32 ZSTD_VecMask_next(ZSTD_VecMask val) {
+ assert(val != 0);
+# if (defined(__GNUC__) && ((__GNUC__ > 3) || ((__GNUC__ == 3) && (__GNUC_MINOR__ >= 4))))
+ if (sizeof(size_t) == 4) {
+ U32 mostSignificantWord = (U32)(val >> 32);
+ U32 leastSignificantWord = (U32)val;
+ if (leastSignificantWord == 0) {
+ return 32 + (U32)__builtin_ctz(mostSignificantWord);
+ } else {
+ return (U32)__builtin_ctz(leastSignificantWord);
+ }
+ } else {
+ return (U32)__builtin_ctzll(val);
+ }
+# else
+ /* Software ctz version: http://aggregate.org/MAGIC/#Trailing%20Zero%20Count
+ * and: https://stackoverflow.com/questions/2709430/count-number-of-bits-in-a-64-bit-long-big-integer
+ */
+ val = ~val & (val - 1ULL); /* Lowest set bit mask */
+ val = val - ((val >> 1) & 0x5555555555555555);
+ val = (val & 0x3333333333333333ULL) + ((val >> 2) & 0x3333333333333333ULL);
+ return (U32)((((val + (val >> 4)) & 0xF0F0F0F0F0F0F0FULL) * 0x101010101010101ULL) >> 56);
+# endif
+}
+
+/* ZSTD_rotateRight_*():
+ * Rotates a bitfield to the right by "count" bits.
+ * https://en.wikipedia.org/w/index.php?title=Circular_shift&oldid=991635599#Implementing_circular_shifts
+ */
+FORCE_INLINE_TEMPLATE
+U64 ZSTD_rotateRight_U64(U64 const value, U32 count) {
+ assert(count < 64);
+ count &= 0x3F; /* for fickle pattern recognition */
+ return (value >> count) | (U64)(value << ((0U - count) & 0x3F));
+}
+
+FORCE_INLINE_TEMPLATE
+U32 ZSTD_rotateRight_U32(U32 const value, U32 count) {
+ assert(count < 32);
+ count &= 0x1F; /* for fickle pattern recognition */
+ return (value >> count) | (U32)(value << ((0U - count) & 0x1F));
+}
+
+FORCE_INLINE_TEMPLATE
+U16 ZSTD_rotateRight_U16(U16 const value, U32 count) {
+ assert(count < 16);
+ count &= 0x0F; /* for fickle pattern recognition */
+ return (value >> count) | (U16)(value << ((0U - count) & 0x0F));
+}
+
+/* ZSTD_row_nextIndex():
+ * Returns the next index to insert at within a tagTable row, and updates the "head"
+ * value to reflect the update. Essentially cycles backwards from [0, {entries per row})
+ */
+FORCE_INLINE_TEMPLATE U32 ZSTD_row_nextIndex(BYTE* const tagRow, U32 const rowMask) {
+ U32 const next = (*tagRow - 1) & rowMask;
+ *tagRow = (BYTE)next;
+ return next;
+}
+
+/* ZSTD_isAligned():
+ * Checks that a pointer is aligned to "align" bytes which must be a power of 2.
+ */
+MEM_STATIC int ZSTD_isAligned(void const* ptr, size_t align) {
+ assert((align & (align - 1)) == 0);
+ return (((size_t)ptr) & (align - 1)) == 0;
+}
+
+/* ZSTD_row_prefetch():
+ * Performs prefetching for the hashTable and tagTable at a given row.
+ */
+FORCE_INLINE_TEMPLATE void ZSTD_row_prefetch(U32 const* hashTable, U16 const* tagTable, U32 const relRow, U32 const rowLog) {
+ PREFETCH_L1(hashTable + relRow);
+ if (rowLog >= 5) {
+ PREFETCH_L1(hashTable + relRow + 16);
+ /* Note: prefetching more of the hash table does not appear to be beneficial for 128-entry rows */
+ }
+ PREFETCH_L1(tagTable + relRow);
+ if (rowLog == 6) {
+ PREFETCH_L1(tagTable + relRow + 32);
+ }
+ assert(rowLog == 4 || rowLog == 5 || rowLog == 6);
+ assert(ZSTD_isAligned(hashTable + relRow, 64)); /* prefetched hash row always 64-byte aligned */
+ assert(ZSTD_isAligned(tagTable + relRow, (size_t)1 << rowLog)); /* prefetched tagRow sits on correct multiple of bytes (32,64,128) */
+}
+
+/* ZSTD_row_fillHashCache():
+ * Fill up the hash cache starting at idx, prefetching up to ZSTD_ROW_HASH_CACHE_SIZE entries,
+ * but not beyond iLimit.
+ */
+FORCE_INLINE_TEMPLATE void ZSTD_row_fillHashCache(ZSTD_matchState_t* ms, const BYTE* base,
+ U32 const rowLog, U32 const mls,
+ U32 idx, const BYTE* const iLimit)
{
- switch(ms->cParams.minMatch)
- {
- default : /* includes case 3 */
- case 4 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 4, ZSTD_noDict);
- case 5 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 5, ZSTD_noDict);
- case 7 :
- case 6 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 6, ZSTD_noDict);
+ U32 const* const hashTable = ms->hashTable;
+ U16 const* const tagTable = ms->tagTable;
+ U32 const hashLog = ms->rowHashLog;
+ U32 const maxElemsToPrefetch = (base + idx) > iLimit ? 0 : (U32)(iLimit - (base + idx) + 1);
+ U32 const lim = idx + MIN(ZSTD_ROW_HASH_CACHE_SIZE, maxElemsToPrefetch);
+
+ for (; idx < lim; ++idx) {
+ U32 const hash = (U32)ZSTD_hashPtr(base + idx, hashLog + ZSTD_ROW_HASH_TAG_BITS, mls);
+ U32 const row = (hash >> ZSTD_ROW_HASH_TAG_BITS) << rowLog;
+ ZSTD_row_prefetch(hashTable, tagTable, row, rowLog);
+ ms->hashCache[idx & ZSTD_ROW_HASH_CACHE_MASK] = hash;
}
+
+ DEBUGLOG(6, "ZSTD_row_fillHashCache(): [%u %u %u %u %u %u %u %u]", ms->hashCache[0], ms->hashCache[1],
+ ms->hashCache[2], ms->hashCache[3], ms->hashCache[4],
+ ms->hashCache[5], ms->hashCache[6], ms->hashCache[7]);
}
+/* ZSTD_row_nextCachedHash():
+ * Returns the hash of base + idx, and replaces the hash in the hash cache with the byte at
+ * base + idx + ZSTD_ROW_HASH_CACHE_SIZE. Also prefetches the appropriate rows from hashTable and tagTable.
+ */
+FORCE_INLINE_TEMPLATE U32 ZSTD_row_nextCachedHash(U32* cache, U32 const* hashTable,
+ U16 const* tagTable, BYTE const* base,
+ U32 idx, U32 const hashLog,
+ U32 const rowLog, U32 const mls)
+{
+ U32 const newHash = (U32)ZSTD_hashPtr(base+idx+ZSTD_ROW_HASH_CACHE_SIZE, hashLog + ZSTD_ROW_HASH_TAG_BITS, mls);
+ U32 const row = (newHash >> ZSTD_ROW_HASH_TAG_BITS) << rowLog;
+ ZSTD_row_prefetch(hashTable, tagTable, row, rowLog);
+ { U32 const hash = cache[idx & ZSTD_ROW_HASH_CACHE_MASK];
+ cache[idx & ZSTD_ROW_HASH_CACHE_MASK] = newHash;
+ return hash;
+ }
+}
-static size_t ZSTD_HcFindBestMatch_dictMatchState_selectMLS (
- ZSTD_matchState_t* ms,
- const BYTE* ip, const BYTE* const iLimit,
- size_t* offsetPtr)
+/* ZSTD_row_update_internalImpl():
+ * Updates the hash table with positions starting from updateStartIdx until updateEndIdx.
+ */
+FORCE_INLINE_TEMPLATE void ZSTD_row_update_internalImpl(ZSTD_matchState_t* ms,
+ U32 updateStartIdx, U32 const updateEndIdx,
+ U32 const mls, U32 const rowLog,
+ U32 const rowMask, U32 const useCache)
{
- switch(ms->cParams.minMatch)
- {
- default : /* includes case 3 */
- case 4 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 4, ZSTD_dictMatchState);
- case 5 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 5, ZSTD_dictMatchState);
- case 7 :
- case 6 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 6, ZSTD_dictMatchState);
+ U32* const hashTable = ms->hashTable;
+ U16* const tagTable = ms->tagTable;
+ U32 const hashLog = ms->rowHashLog;
+ const BYTE* const base = ms->window.base;
+
+ DEBUGLOG(6, "ZSTD_row_update_internalImpl(): updateStartIdx=%u, updateEndIdx=%u", updateStartIdx, updateEndIdx);
+ for (; updateStartIdx < updateEndIdx; ++updateStartIdx) {
+ U32 const hash = useCache ? ZSTD_row_nextCachedHash(ms->hashCache, hashTable, tagTable, base, updateStartIdx, hashLog, rowLog, mls)
+ : (U32)ZSTD_hashPtr(base + updateStartIdx, hashLog + ZSTD_ROW_HASH_TAG_BITS, mls);
+ U32 const relRow = (hash >> ZSTD_ROW_HASH_TAG_BITS) << rowLog;
+ U32* const row = hashTable + relRow;
+ BYTE* tagRow = (BYTE*)(tagTable + relRow); /* Though tagTable is laid out as a table of U16, each tag is only 1 byte.
+ Explicit cast allows us to get exact desired position within each row */
+ U32 const pos = ZSTD_row_nextIndex(tagRow, rowMask);
+
+ assert(hash == ZSTD_hashPtr(base + updateStartIdx, hashLog + ZSTD_ROW_HASH_TAG_BITS, mls));
+ ((BYTE*)tagRow)[pos + ZSTD_ROW_HASH_TAG_OFFSET] = hash & ZSTD_ROW_HASH_TAG_MASK;
+ row[pos] = updateStartIdx;
}
}
+/* ZSTD_row_update_internal():
+ * Inserts the byte at ip into the appropriate position in the hash table, and updates ms->nextToUpdate.
+ * Skips sections of long matches as is necessary.
+ */
+FORCE_INLINE_TEMPLATE void ZSTD_row_update_internal(ZSTD_matchState_t* ms, const BYTE* ip,
+ U32 const mls, U32 const rowLog,
+ U32 const rowMask, U32 const useCache)
+{
+ U32 idx = ms->nextToUpdate;
+ const BYTE* const base = ms->window.base;
+ const U32 target = (U32)(ip - base);
+ const U32 kSkipThreshold = 384;
+ const U32 kMaxMatchStartPositionsToUpdate = 96;
+ const U32 kMaxMatchEndPositionsToUpdate = 32;
+
+ if (useCache) {
+ /* Only skip positions when using hash cache, i.e.
+ * if we are loading a dict, don't skip anything.
+ * If we decide to skip, then we only update a set number
+ * of positions at the beginning and end of the match.
+ */
+ if (UNLIKELY(target - idx > kSkipThreshold)) {
+ U32 const bound = idx + kMaxMatchStartPositionsToUpdate;
+ ZSTD_row_update_internalImpl(ms, idx, bound, mls, rowLog, rowMask, useCache);
+ idx = target - kMaxMatchEndPositionsToUpdate;
+ ZSTD_row_fillHashCache(ms, base, rowLog, mls, idx, ip+1);
+ }
+ }
+ assert(target >= idx);
+ ZSTD_row_update_internalImpl(ms, idx, target, mls, rowLog, rowMask, useCache);
+ ms->nextToUpdate = target;
+}
-static size_t ZSTD_HcFindBestMatch_dedicatedDictSearch_selectMLS (
- ZSTD_matchState_t* ms,
- const BYTE* ip, const BYTE* const iLimit,
- size_t* offsetPtr)
+/* ZSTD_row_update():
+ * External wrapper for ZSTD_row_update_internal(). Used for filling the hashtable during dictionary
+ * processing.
+ */
+void ZSTD_row_update(ZSTD_matchState_t* const ms, const BYTE* ip) {
+ const U32 rowLog = BOUNDED(4, ms->cParams.searchLog, 6);
+ const U32 rowMask = (1u << rowLog) - 1;
+ const U32 mls = MIN(ms->cParams.minMatch, 6 /* mls caps out at 6 */);
+
+ DEBUGLOG(5, "ZSTD_row_update(), rowLog=%u", rowLog);
+ ZSTD_row_update_internal(ms, ip, mls, rowLog, rowMask, 0 /* dont use cache */);
+}
+
+#if defined(ZSTD_ARCH_X86_SSE2)
+FORCE_INLINE_TEMPLATE ZSTD_VecMask
+ZSTD_row_getSSEMask(int nbChunks, const BYTE* const src, const BYTE tag, const U32 head)
{
- switch(ms->cParams.minMatch)
- {
- default : /* includes case 3 */
- case 4 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 4, ZSTD_dedicatedDictSearch);
- case 5 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 5, ZSTD_dedicatedDictSearch);
- case 7 :
- case 6 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 6, ZSTD_dedicatedDictSearch);
+ const __m128i comparisonMask = _mm_set1_epi8((char)tag);
+ int matches[4] = {0};
+ int i;
+ assert(nbChunks == 1 || nbChunks == 2 || nbChunks == 4);
+ for (i=0; i<nbChunks; i++) {
+ const __m128i chunk = _mm_loadu_si128((const __m128i*)(const void*)(src + 16*i));
+ const __m128i equalMask = _mm_cmpeq_epi8(chunk, comparisonMask);
+ matches[i] = _mm_movemask_epi8(equalMask);
}
+ if (nbChunks == 1) return ZSTD_rotateRight_U16((U16)matches[0], head);
+ if (nbChunks == 2) return ZSTD_rotateRight_U32((U32)matches[1] << 16 | (U32)matches[0], head);
+ assert(nbChunks == 4);
+ return ZSTD_rotateRight_U64((U64)matches[3] << 48 | (U64)matches[2] << 32 | (U64)matches[1] << 16 | (U64)matches[0], head);
}
+#endif
+/* Returns a ZSTD_VecMask (U32) that has the nth bit set to 1 if the newly-computed "tag" matches
+ * the hash at the nth position in a row of the tagTable.
+ * Each row is a circular buffer beginning at the value of "head". So we must rotate the "matches" bitfield
+ * to match up with the actual layout of the entries within the hashTable */
+FORCE_INLINE_TEMPLATE ZSTD_VecMask
+ZSTD_row_getMatchMask(const BYTE* const tagRow, const BYTE tag, const U32 head, const U32 rowEntries)
+{
+ const BYTE* const src = tagRow + ZSTD_ROW_HASH_TAG_OFFSET;
+ assert((rowEntries == 16) || (rowEntries == 32) || rowEntries == 64);
+ assert(rowEntries <= ZSTD_ROW_HASH_MAX_ENTRIES);
+
+#if defined(ZSTD_ARCH_X86_SSE2)
+
+ return ZSTD_row_getSSEMask(rowEntries / 16, src, tag, head);
+
+#else /* SW or NEON-LE */
+
+# if defined(ZSTD_ARCH_ARM_NEON)
+ /* This NEON path only works for little endian - otherwise use SWAR below */
+ if (MEM_isLittleEndian()) {
+ if (rowEntries == 16) {
+ const uint8x16_t chunk = vld1q_u8(src);
+ const uint16x8_t equalMask = vreinterpretq_u16_u8(vceqq_u8(chunk, vdupq_n_u8(tag)));
+ const uint16x8_t t0 = vshlq_n_u16(equalMask, 7);
+ const uint32x4_t t1 = vreinterpretq_u32_u16(vsriq_n_u16(t0, t0, 14));
+ const uint64x2_t t2 = vreinterpretq_u64_u32(vshrq_n_u32(t1, 14));
+ const uint8x16_t t3 = vreinterpretq_u8_u64(vsraq_n_u64(t2, t2, 28));
+ const U16 hi = (U16)vgetq_lane_u8(t3, 8);
+ const U16 lo = (U16)vgetq_lane_u8(t3, 0);
+ return ZSTD_rotateRight_U16((hi << 8) | lo, head);
+ } else if (rowEntries == 32) {
+ const uint16x8x2_t chunk = vld2q_u16((const U16*)(const void*)src);
+ const uint8x16_t chunk0 = vreinterpretq_u8_u16(chunk.val[0]);
+ const uint8x16_t chunk1 = vreinterpretq_u8_u16(chunk.val[1]);
+ const uint8x16_t equalMask0 = vceqq_u8(chunk0, vdupq_n_u8(tag));
+ const uint8x16_t equalMask1 = vceqq_u8(chunk1, vdupq_n_u8(tag));
+ const int8x8_t pack0 = vqmovn_s16(vreinterpretq_s16_u8(equalMask0));
+ const int8x8_t pack1 = vqmovn_s16(vreinterpretq_s16_u8(equalMask1));
+ const uint8x8_t t0 = vreinterpret_u8_s8(pack0);
+ const uint8x8_t t1 = vreinterpret_u8_s8(pack1);
+ const uint8x8_t t2 = vsri_n_u8(t1, t0, 2);
+ const uint8x8x2_t t3 = vuzp_u8(t2, t0);
+ const uint8x8_t t4 = vsri_n_u8(t3.val[1], t3.val[0], 4);
+ const U32 matches = vget_lane_u32(vreinterpret_u32_u8(t4), 0);
+ return ZSTD_rotateRight_U32(matches, head);
+ } else { /* rowEntries == 64 */
+ const uint8x16x4_t chunk = vld4q_u8(src);
+ const uint8x16_t dup = vdupq_n_u8(tag);
+ const uint8x16_t cmp0 = vceqq_u8(chunk.val[0], dup);
+ const uint8x16_t cmp1 = vceqq_u8(chunk.val[1], dup);
+ const uint8x16_t cmp2 = vceqq_u8(chunk.val[2], dup);
+ const uint8x16_t cmp3 = vceqq_u8(chunk.val[3], dup);
+
+ const uint8x16_t t0 = vsriq_n_u8(cmp1, cmp0, 1);
+ const uint8x16_t t1 = vsriq_n_u8(cmp3, cmp2, 1);
+ const uint8x16_t t2 = vsriq_n_u8(t1, t0, 2);
+ const uint8x16_t t3 = vsriq_n_u8(t2, t2, 4);
+ const uint8x8_t t4 = vshrn_n_u16(vreinterpretq_u16_u8(t3), 4);
+ const U64 matches = vget_lane_u64(vreinterpret_u64_u8(t4), 0);
+ return ZSTD_rotateRight_U64(matches, head);
+ }
+ }
+# endif /* ZSTD_ARCH_ARM_NEON */
+ /* SWAR */
+ { const size_t chunkSize = sizeof(size_t);
+ const size_t shiftAmount = ((chunkSize * 8) - chunkSize);
+ const size_t xFF = ~((size_t)0);
+ const size_t x01 = xFF / 0xFF;
+ const size_t x80 = x01 << 7;
+ const size_t splatChar = tag * x01;
+ ZSTD_VecMask matches = 0;
+ int i = rowEntries - chunkSize;
+ assert((sizeof(size_t) == 4) || (sizeof(size_t) == 8));
+ if (MEM_isLittleEndian()) { /* runtime check so have two loops */
+ const size_t extractMagic = (xFF / 0x7F) >> chunkSize;
+ do {
+ size_t chunk = MEM_readST(&src[i]);
+ chunk ^= splatChar;
+ chunk = (((chunk | x80) - x01) | chunk) & x80;
+ matches <<= chunkSize;
+ matches |= (chunk * extractMagic) >> shiftAmount;
+ i -= chunkSize;
+ } while (i >= 0);
+ } else { /* big endian: reverse bits during extraction */
+ const size_t msb = xFF ^ (xFF >> 1);
+ const size_t extractMagic = (msb / 0x1FF) | msb;
+ do {
+ size_t chunk = MEM_readST(&src[i]);
+ chunk ^= splatChar;
+ chunk = (((chunk | x80) - x01) | chunk) & x80;
+ matches <<= chunkSize;
+ matches |= ((chunk >> 7) * extractMagic) >> shiftAmount;
+ i -= chunkSize;
+ } while (i >= 0);
+ }
+ matches = ~matches;
+ if (rowEntries == 16) {
+ return ZSTD_rotateRight_U16((U16)matches, head);
+ } else if (rowEntries == 32) {
+ return ZSTD_rotateRight_U32((U32)matches, head);
+ } else {
+ return ZSTD_rotateRight_U64((U64)matches, head);
+ }
+ }
+#endif
+}
-FORCE_INLINE_TEMPLATE size_t ZSTD_HcFindBestMatch_extDict_selectMLS (
+/* The high-level approach of the SIMD row based match finder is as follows:
+ * - Figure out where to insert the new entry:
+ * - Generate a hash from a byte along with an additional 1-byte "short hash". The additional byte is our "tag"
+ * - The hashTable is effectively split into groups or "rows" of 16 or 32 entries of U32, and the hash determines
+ * which row to insert into.
+ * - Determine the correct position within the row to insert the entry into. Each row of 16 or 32 can
+ * be considered as a circular buffer with a "head" index that resides in the tagTable.
+ * - Also insert the "tag" into the equivalent row and position in the tagTable.
+ * - Note: The tagTable has 17 or 33 1-byte entries per row, due to 16 or 32 tags, and 1 "head" entry.
+ * The 17 or 33 entry rows are spaced out to occur every 32 or 64 bytes, respectively,
+ * for alignment/performance reasons, leaving some bytes unused.
+ * - Use SIMD to efficiently compare the tags in the tagTable to the 1-byte "short hash" and
+ * generate a bitfield that we can cycle through to check the collisions in the hash table.
+ * - Pick the longest match.
+ */
+FORCE_INLINE_TEMPLATE
+size_t ZSTD_RowFindBestMatch(
ZSTD_matchState_t* ms,
- const BYTE* ip, const BYTE* const iLimit,
- size_t* offsetPtr)
+ const BYTE* const ip, const BYTE* const iLimit,
+ size_t* offsetPtr,
+ const U32 mls, const ZSTD_dictMode_e dictMode,
+ const U32 rowLog)
{
- switch(ms->cParams.minMatch)
- {
- default : /* includes case 3 */
- case 4 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 4, ZSTD_extDict);
- case 5 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 5, ZSTD_extDict);
- case 7 :
- case 6 : return ZSTD_HcFindBestMatch_generic(ms, ip, iLimit, offsetPtr, 6, ZSTD_extDict);
+ U32* const hashTable = ms->hashTable;
+ U16* const tagTable = ms->tagTable;
+ U32* const hashCache = ms->hashCache;
+ const U32 hashLog = ms->rowHashLog;
+ const ZSTD_compressionParameters* const cParams = &ms->cParams;
+ const BYTE* const base = ms->window.base;
+ const BYTE* const dictBase = ms->window.dictBase;
+ const U32 dictLimit = ms->window.dictLimit;
+ const BYTE* const prefixStart = base + dictLimit;
+ const BYTE* const dictEnd = dictBase + dictLimit;
+ const U32 curr = (U32)(ip-base);
+ const U32 maxDistance = 1U << cParams->windowLog;
+ const U32 lowestValid = ms->window.lowLimit;
+ const U32 withinMaxDistance = (curr - lowestValid > maxDistance) ? curr - maxDistance : lowestValid;
+ const U32 isDictionary = (ms->loadedDictEnd != 0);
+ const U32 lowLimit = isDictionary ? lowestValid : withinMaxDistance;
+ const U32 rowEntries = (1U << rowLog);
+ const U32 rowMask = rowEntries - 1;
+ const U32 cappedSearchLog = MIN(cParams->searchLog, rowLog); /* nb of searches is capped at nb entries per row */
+ U32 nbAttempts = 1U << cappedSearchLog;
+ size_t ml=4-1;
+
+ /* DMS/DDS variables that may be referenced laster */
+ const ZSTD_matchState_t* const dms = ms->dictMatchState;
+
+ /* Initialize the following variables to satisfy static analyzer */
+ size_t ddsIdx = 0;
+ U32 ddsExtraAttempts = 0; /* cctx hash tables are limited in searches, but allow extra searches into DDS */
+ U32 dmsTag = 0;
+ U32* dmsRow = NULL;
+ BYTE* dmsTagRow = NULL;
+
+ if (dictMode == ZSTD_dedicatedDictSearch) {
+ const U32 ddsHashLog = dms->cParams.hashLog - ZSTD_LAZY_DDSS_BUCKET_LOG;
+ { /* Prefetch DDS hashtable entry */
+ ddsIdx = ZSTD_hashPtr(ip, ddsHashLog, mls) << ZSTD_LAZY_DDSS_BUCKET_LOG;
+ PREFETCH_L1(&dms->hashTable[ddsIdx]);
+ }
+ ddsExtraAttempts = cParams->searchLog > rowLog ? 1U << (cParams->searchLog - rowLog) : 0;
+ }
+
+ if (dictMode == ZSTD_dictMatchState) {
+ /* Prefetch DMS rows */
+ U32* const dmsHashTable = dms->hashTable;
+ U16* const dmsTagTable = dms->tagTable;
+ U32 const dmsHash = (U32)ZSTD_hashPtr(ip, dms->rowHashLog + ZSTD_ROW_HASH_TAG_BITS, mls);
+ U32 const dmsRelRow = (dmsHash >> ZSTD_ROW_HASH_TAG_BITS) << rowLog;
+ dmsTag = dmsHash & ZSTD_ROW_HASH_TAG_MASK;
+ dmsTagRow = (BYTE*)(dmsTagTable + dmsRelRow);
+ dmsRow = dmsHashTable + dmsRelRow;
+ ZSTD_row_prefetch(dmsHashTable, dmsTagTable, dmsRelRow, rowLog);
+ }
+
+ /* Update the hashTable and tagTable up to (but not including) ip */
+ ZSTD_row_update_internal(ms, ip, mls, rowLog, rowMask, 1 /* useCache */);
+ { /* Get the hash for ip, compute the appropriate row */
+ U32 const hash = ZSTD_row_nextCachedHash(hashCache, hashTable, tagTable, base, curr, hashLog, rowLog, mls);
+ U32 const relRow = (hash >> ZSTD_ROW_HASH_TAG_BITS) << rowLog;
+ U32 const tag = hash & ZSTD_ROW_HASH_TAG_MASK;
+ U32* const row = hashTable + relRow;
+ BYTE* tagRow = (BYTE*)(tagTable + relRow);
+ U32 const head = *tagRow & rowMask;
+ U32 matchBuffer[ZSTD_ROW_HASH_MAX_ENTRIES];
+ size_t numMatches = 0;
+ size_t currMatch = 0;
+ ZSTD_VecMask matches = ZSTD_row_getMatchMask(tagRow, (BYTE)tag, head, rowEntries);
+
+ /* Cycle through the matches and prefetch */
+ for (; (matches > 0) && (nbAttempts > 0); --nbAttempts, matches &= (matches - 1)) {
+ U32 const matchPos = (head + ZSTD_VecMask_next(matches)) & rowMask;
+ U32 const matchIndex = row[matchPos];
+ assert(numMatches < rowEntries);
+ if (matchIndex < lowLimit)
+ break;
+ if ((dictMode != ZSTD_extDict) || matchIndex >= dictLimit) {
+ PREFETCH_L1(base + matchIndex);
+ } else {
+ PREFETCH_L1(dictBase + matchIndex);
+ }
+ matchBuffer[numMatches++] = matchIndex;
+ }
+
+ /* Speed opt: insert current byte into hashtable too. This allows us to avoid one iteration of the loop
+ in ZSTD_row_update_internal() at the next search. */
+ {
+ U32 const pos = ZSTD_row_nextIndex(tagRow, rowMask);
+ tagRow[pos + ZSTD_ROW_HASH_TAG_OFFSET] = (BYTE)tag;
+ row[pos] = ms->nextToUpdate++;
+ }
+
+ /* Return the longest match */
+ for (; currMatch < numMatches; ++currMatch) {
+ U32 const matchIndex = matchBuffer[currMatch];
+ size_t currentMl=0;
+ assert(matchIndex < curr);
+ assert(matchIndex >= lowLimit);
+
+ if ((dictMode != ZSTD_extDict) || matchIndex >= dictLimit) {
+ const BYTE* const match = base + matchIndex;
+ assert(matchIndex >= dictLimit); /* ensures this is true if dictMode != ZSTD_extDict */
+ if (match[ml] == ip[ml]) /* potentially better */
+ currentMl = ZSTD_count(ip, match, iLimit);
+ } else {
+ const BYTE* const match = dictBase + matchIndex;
+ assert(match+4 <= dictEnd);
+ if (MEM_read32(match) == MEM_read32(ip)) /* assumption : matchIndex <= dictLimit-4 (by table construction) */
+ currentMl = ZSTD_count_2segments(ip+4, match+4, iLimit, dictEnd, prefixStart) + 4;
+ }
+
+ /* Save best solution */
+ if (currentMl > ml) {
+ ml = currentMl;
+ *offsetPtr = STORE_OFFSET(curr - matchIndex);
+ if (ip+currentMl == iLimit) break; /* best possible, avoids read overflow on next attempt */
+ }
+ }
+ }
+
+ assert(nbAttempts <= (1U << ZSTD_SEARCHLOG_MAX)); /* Check we haven't underflowed. */
+ if (dictMode == ZSTD_dedicatedDictSearch) {
+ ml = ZSTD_dedicatedDictSearch_lazy_search(offsetPtr, ml, nbAttempts + ddsExtraAttempts, dms,
+ ip, iLimit, prefixStart, curr, dictLimit, ddsIdx);
+ } else if (dictMode == ZSTD_dictMatchState) {
+ /* TODO: Measure and potentially add prefetching to DMS */
+ const U32 dmsLowestIndex = dms->window.dictLimit;
+ const BYTE* const dmsBase = dms->window.base;
+ const BYTE* const dmsEnd = dms->window.nextSrc;
+ const U32 dmsSize = (U32)(dmsEnd - dmsBase);
+ const U32 dmsIndexDelta = dictLimit - dmsSize;
+
+ { U32 const head = *dmsTagRow & rowMask;
+ U32 matchBuffer[ZSTD_ROW_HASH_MAX_ENTRIES];
+ size_t numMatches = 0;
+ size_t currMatch = 0;
+ ZSTD_VecMask matches = ZSTD_row_getMatchMask(dmsTagRow, (BYTE)dmsTag, head, rowEntries);
+
+ for (; (matches > 0) && (nbAttempts > 0); --nbAttempts, matches &= (matches - 1)) {
+ U32 const matchPos = (head + ZSTD_VecMask_next(matches)) & rowMask;
+ U32 const matchIndex = dmsRow[matchPos];
+ if (matchIndex < dmsLowestIndex)
+ break;
+ PREFETCH_L1(dmsBase + matchIndex);
+ matchBuffer[numMatches++] = matchIndex;
+ }
+
+ /* Return the longest match */
+ for (; currMatch < numMatches; ++currMatch) {
+ U32 const matchIndex = matchBuffer[currMatch];
+ size_t currentMl=0;
+ assert(matchIndex >= dmsLowestIndex);
+ assert(matchIndex < curr);
+
+ { const BYTE* const match = dmsBase + matchIndex;
+ assert(match+4 <= dmsEnd);
+ if (MEM_read32(match) == MEM_read32(ip))
+ currentMl = ZSTD_count_2segments(ip+4, match+4, iLimit, dmsEnd, prefixStart) + 4;
+ }
+
+ if (currentMl > ml) {
+ ml = currentMl;
+ assert(curr > matchIndex + dmsIndexDelta);
+ *offsetPtr = STORE_OFFSET(curr - (matchIndex + dmsIndexDelta));
+ if (ip+currentMl == iLimit) break;
+ }
+ }
+ }
}
+ return ml;
}
+/*
+ * Generate search functions templated on (dictMode, mls, rowLog).
+ * These functions are outlined for code size & compilation time.
+ * ZSTD_searchMax() dispatches to the correct implementation function.
+ *
+ * TODO: The start of the search function involves loading and calculating a
+ * bunch of constants from the ZSTD_matchState_t. These computations could be
+ * done in an initialization function, and saved somewhere in the match state.
+ * Then we could pass a pointer to the saved state instead of the match state,
+ * and avoid duplicate computations.
+ *
+ * TODO: Move the match re-winding into searchMax. This improves compression
+ * ratio, and unlocks further simplifications with the next TODO.
+ *
+ * TODO: Try moving the repcode search into searchMax. After the re-winding
+ * and repcode search are in searchMax, there is no more logic in the match
+ * finder loop that requires knowledge about the dictMode. So we should be
+ * able to avoid force inlining it, and we can join the extDict loop with
+ * the single segment loop. It should go in searchMax instead of its own
+ * function to avoid having multiple virtual function calls per search.
+ */
+
+#define ZSTD_BT_SEARCH_FN(dictMode, mls) ZSTD_BtFindBestMatch_##dictMode##_##mls
+#define ZSTD_HC_SEARCH_FN(dictMode, mls) ZSTD_HcFindBestMatch_##dictMode##_##mls
+#define ZSTD_ROW_SEARCH_FN(dictMode, mls, rowLog) ZSTD_RowFindBestMatch_##dictMode##_##mls##_##rowLog
+
+#define ZSTD_SEARCH_FN_ATTRS FORCE_NOINLINE
+
+#define GEN_ZSTD_BT_SEARCH_FN(dictMode, mls) \
+ ZSTD_SEARCH_FN_ATTRS size_t ZSTD_BT_SEARCH_FN(dictMode, mls)( \
+ ZSTD_matchState_t* ms, \
+ const BYTE* ip, const BYTE* const iLimit, \
+ size_t* offBasePtr) \
+ { \
+ assert(MAX(4, MIN(6, ms->cParams.minMatch)) == mls); \
+ return ZSTD_BtFindBestMatch(ms, ip, iLimit, offBasePtr, mls, ZSTD_##dictMode); \
+ } \
+
+#define GEN_ZSTD_HC_SEARCH_FN(dictMode, mls) \
+ ZSTD_SEARCH_FN_ATTRS size_t ZSTD_HC_SEARCH_FN(dictMode, mls)( \
+ ZSTD_matchState_t* ms, \
+ const BYTE* ip, const BYTE* const iLimit, \
+ size_t* offsetPtr) \
+ { \
+ assert(MAX(4, MIN(6, ms->cParams.minMatch)) == mls); \
+ return ZSTD_HcFindBestMatch(ms, ip, iLimit, offsetPtr, mls, ZSTD_##dictMode); \
+ } \
+
+#define GEN_ZSTD_ROW_SEARCH_FN(dictMode, mls, rowLog) \
+ ZSTD_SEARCH_FN_ATTRS size_t ZSTD_ROW_SEARCH_FN(dictMode, mls, rowLog)( \
+ ZSTD_matchState_t* ms, \
+ const BYTE* ip, const BYTE* const iLimit, \
+ size_t* offsetPtr) \
+ { \
+ assert(MAX(4, MIN(6, ms->cParams.minMatch)) == mls); \
+ assert(MAX(4, MIN(6, ms->cParams.searchLog)) == rowLog); \
+ return ZSTD_RowFindBestMatch(ms, ip, iLimit, offsetPtr, mls, ZSTD_##dictMode, rowLog); \
+ } \
+
+#define ZSTD_FOR_EACH_ROWLOG(X, dictMode, mls) \
+ X(dictMode, mls, 4) \
+ X(dictMode, mls, 5) \
+ X(dictMode, mls, 6)
+
+#define ZSTD_FOR_EACH_MLS_ROWLOG(X, dictMode) \
+ ZSTD_FOR_EACH_ROWLOG(X, dictMode, 4) \
+ ZSTD_FOR_EACH_ROWLOG(X, dictMode, 5) \
+ ZSTD_FOR_EACH_ROWLOG(X, dictMode, 6)
+
+#define ZSTD_FOR_EACH_MLS(X, dictMode) \
+ X(dictMode, 4) \
+ X(dictMode, 5) \
+ X(dictMode, 6)
+
+#define ZSTD_FOR_EACH_DICT_MODE(X, ...) \
+ X(__VA_ARGS__, noDict) \
+ X(__VA_ARGS__, extDict) \
+ X(__VA_ARGS__, dictMatchState) \
+ X(__VA_ARGS__, dedicatedDictSearch)
+
+/* Generate row search fns for each combination of (dictMode, mls, rowLog) */
+ZSTD_FOR_EACH_DICT_MODE(ZSTD_FOR_EACH_MLS_ROWLOG, GEN_ZSTD_ROW_SEARCH_FN)
+/* Generate binary Tree search fns for each combination of (dictMode, mls) */
+ZSTD_FOR_EACH_DICT_MODE(ZSTD_FOR_EACH_MLS, GEN_ZSTD_BT_SEARCH_FN)
+/* Generate hash chain search fns for each combination of (dictMode, mls) */
+ZSTD_FOR_EACH_DICT_MODE(ZSTD_FOR_EACH_MLS, GEN_ZSTD_HC_SEARCH_FN)
+
+typedef enum { search_hashChain=0, search_binaryTree=1, search_rowHash=2 } searchMethod_e;
+
+#define GEN_ZSTD_CALL_BT_SEARCH_FN(dictMode, mls) \
+ case mls: \
+ return ZSTD_BT_SEARCH_FN(dictMode, mls)(ms, ip, iend, offsetPtr);
+#define GEN_ZSTD_CALL_HC_SEARCH_FN(dictMode, mls) \
+ case mls: \
+ return ZSTD_HC_SEARCH_FN(dictMode, mls)(ms, ip, iend, offsetPtr);
+#define GEN_ZSTD_CALL_ROW_SEARCH_FN(dictMode, mls, rowLog) \
+ case rowLog: \
+ return ZSTD_ROW_SEARCH_FN(dictMode, mls, rowLog)(ms, ip, iend, offsetPtr);
+
+#define ZSTD_SWITCH_MLS(X, dictMode) \
+ switch (mls) { \
+ ZSTD_FOR_EACH_MLS(X, dictMode) \
+ }
+
+#define ZSTD_SWITCH_ROWLOG(dictMode, mls) \
+ case mls: \
+ switch (rowLog) { \
+ ZSTD_FOR_EACH_ROWLOG(GEN_ZSTD_CALL_ROW_SEARCH_FN, dictMode, mls) \
+ } \
+ ZSTD_UNREACHABLE; \
+ break;
+
+#define ZSTD_SWITCH_SEARCH_METHOD(dictMode) \
+ switch (searchMethod) { \
+ case search_hashChain: \
+ ZSTD_SWITCH_MLS(GEN_ZSTD_CALL_HC_SEARCH_FN, dictMode) \
+ break; \
+ case search_binaryTree: \
+ ZSTD_SWITCH_MLS(GEN_ZSTD_CALL_BT_SEARCH_FN, dictMode) \
+ break; \
+ case search_rowHash: \
+ ZSTD_SWITCH_MLS(ZSTD_SWITCH_ROWLOG, dictMode) \
+ break; \
+ } \
+ ZSTD_UNREACHABLE;
+
+/*
+ * Searches for the longest match at @p ip.
+ * Dispatches to the correct implementation function based on the
+ * (searchMethod, dictMode, mls, rowLog). We use switch statements
+ * here instead of using an indirect function call through a function
+ * pointer because after Spectre and Meltdown mitigations, indirect
+ * function calls can be very costly, especially in the kernel.
+ *
+ * NOTE: dictMode and searchMethod should be templated, so those switch
+ * statements should be optimized out. Only the mls & rowLog switches
+ * should be left.
+ *
+ * @param ms The match state.
+ * @param ip The position to search at.
+ * @param iend The end of the input data.
+ * @param[out] offsetPtr Stores the match offset into this pointer.
+ * @param mls The minimum search length, in the range [4, 6].
+ * @param rowLog The row log (if applicable), in the range [4, 6].
+ * @param searchMethod The search method to use (templated).
+ * @param dictMode The dictMode (templated).
+ *
+ * @returns The length of the longest match found, or < mls if no match is found.
+ * If a match is found its offset is stored in @p offsetPtr.
+ */
+FORCE_INLINE_TEMPLATE size_t ZSTD_searchMax(
+ ZSTD_matchState_t* ms,
+ const BYTE* ip,
+ const BYTE* iend,
+ size_t* offsetPtr,
+ U32 const mls,
+ U32 const rowLog,
+ searchMethod_e const searchMethod,
+ ZSTD_dictMode_e const dictMode)
+{
+ if (dictMode == ZSTD_noDict) {
+ ZSTD_SWITCH_SEARCH_METHOD(noDict)
+ } else if (dictMode == ZSTD_extDict) {
+ ZSTD_SWITCH_SEARCH_METHOD(extDict)
+ } else if (dictMode == ZSTD_dictMatchState) {
+ ZSTD_SWITCH_SEARCH_METHOD(dictMatchState)
+ } else if (dictMode == ZSTD_dedicatedDictSearch) {
+ ZSTD_SWITCH_SEARCH_METHOD(dedicatedDictSearch)
+ }
+ ZSTD_UNREACHABLE;
+ return 0;
+}
+
/* *******************************
* Common parser - lazy strategy
*********************************/
-typedef enum { search_hashChain, search_binaryTree } searchMethod_e;
FORCE_INLINE_TEMPLATE size_t
ZSTD_compressBlock_lazy_generic(
const BYTE* ip = istart;
const BYTE* anchor = istart;
const BYTE* const iend = istart + srcSize;
- const BYTE* const ilimit = iend - 8;
+ const BYTE* const ilimit = (searchMethod == search_rowHash) ? iend - 8 - ZSTD_ROW_HASH_CACHE_SIZE : iend - 8;
const BYTE* const base = ms->window.base;
const U32 prefixLowestIndex = ms->window.dictLimit;
const BYTE* const prefixLowest = base + prefixLowestIndex;
+ const U32 mls = BOUNDED(4, ms->cParams.minMatch, 6);
+ const U32 rowLog = BOUNDED(4, ms->cParams.searchLog, 6);
- typedef size_t (*searchMax_f)(
- ZSTD_matchState_t* ms,
- const BYTE* ip, const BYTE* iLimit, size_t* offsetPtr);
-
- /*
- * This table is indexed first by the four ZSTD_dictMode_e values, and then
- * by the two searchMethod_e values. NULLs are placed for configurations
- * that should never occur (extDict modes go to the other implementation
- * below and there is no DDSS for binary tree search yet).
- */
- const searchMax_f searchFuncs[4][2] = {
- {
- ZSTD_HcFindBestMatch_selectMLS,
- ZSTD_BtFindBestMatch_selectMLS
- },
- {
- NULL,
- NULL
- },
- {
- ZSTD_HcFindBestMatch_dictMatchState_selectMLS,
- ZSTD_BtFindBestMatch_dictMatchState_selectMLS
- },
- {
- ZSTD_HcFindBestMatch_dedicatedDictSearch_selectMLS,
- NULL
- }
- };
-
- searchMax_f const searchMax = searchFuncs[dictMode][searchMethod == search_binaryTree];
U32 offset_1 = rep[0], offset_2 = rep[1], savedOffset=0;
const int isDMS = dictMode == ZSTD_dictMatchState;
0;
const U32 dictAndPrefixLength = (U32)((ip - prefixLowest) + (dictEnd - dictLowest));
- assert(searchMax != NULL);
-
- DEBUGLOG(5, "ZSTD_compressBlock_lazy_generic (dictMode=%u)", (U32)dictMode);
-
- /* init */
+ DEBUGLOG(5, "ZSTD_compressBlock_lazy_generic (dictMode=%u) (searchFunc=%u)", (U32)dictMode, (U32)searchMethod);
ip += (dictAndPrefixLength == 0);
if (dictMode == ZSTD_noDict) {
U32 const curr = (U32)(ip - base);
assert(offset_2 <= dictAndPrefixLength);
}
+ if (searchMethod == search_rowHash) {
+ ZSTD_row_fillHashCache(ms, base, rowLog,
+ MIN(ms->cParams.minMatch, 6 /* mls caps out at 6 */),
+ ms->nextToUpdate, ilimit);
+ }
+
/* Match Loop */
#if defined(__x86_64__)
/* I've measured random a 5% speed loss on levels 5 & 6 (greedy) when the
#endif
while (ip < ilimit) {
size_t matchLength=0;
- size_t offset=0;
+ size_t offcode=STORE_REPCODE_1;
const BYTE* start=ip+1;
+ DEBUGLOG(7, "search baseline (depth 0)");
/* check repCode */
if (isDxS) {
/* first search (depth 0) */
{ size_t offsetFound = 999999999;
- size_t const ml2 = searchMax(ms, ip, iend, &offsetFound);
+ size_t const ml2 = ZSTD_searchMax(ms, ip, iend, &offsetFound, mls, rowLog, searchMethod, dictMode);
if (ml2 > matchLength)
- matchLength = ml2, start = ip, offset=offsetFound;
+ matchLength = ml2, start = ip, offcode=offsetFound;
}
if (matchLength < 4) {
/* let's try to find a better solution */
if (depth>=1)
while (ip<ilimit) {
+ DEBUGLOG(7, "search depth 1");
ip ++;
if ( (dictMode == ZSTD_noDict)
- && (offset) && ((offset_1>0) & (MEM_read32(ip) == MEM_read32(ip - offset_1)))) {
+ && (offcode) && ((offset_1>0) & (MEM_read32(ip) == MEM_read32(ip - offset_1)))) {
size_t const mlRep = ZSTD_count(ip+4, ip+4-offset_1, iend) + 4;
int const gain2 = (int)(mlRep * 3);
- int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)offset+1) + 1);
+ int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offcode)) + 1);
if ((mlRep >= 4) && (gain2 > gain1))
- matchLength = mlRep, offset = 0, start = ip;
+ matchLength = mlRep, offcode = STORE_REPCODE_1, start = ip;
}
if (isDxS) {
const U32 repIndex = (U32)(ip - base) - offset_1;
const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend;
size_t const mlRep = ZSTD_count_2segments(ip+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4;
int const gain2 = (int)(mlRep * 3);
- int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)offset+1) + 1);
+ int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offcode)) + 1);
if ((mlRep >= 4) && (gain2 > gain1))
- matchLength = mlRep, offset = 0, start = ip;
+ matchLength = mlRep, offcode = STORE_REPCODE_1, start = ip;
}
}
{ size_t offset2=999999999;
- size_t const ml2 = searchMax(ms, ip, iend, &offset2);
- int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */
- int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 4);
+ size_t const ml2 = ZSTD_searchMax(ms, ip, iend, &offset2, mls, rowLog, searchMethod, dictMode);
+ int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offset2))); /* raw approx */
+ int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offcode)) + 4);
if ((ml2 >= 4) && (gain2 > gain1)) {
- matchLength = ml2, offset = offset2, start = ip;
+ matchLength = ml2, offcode = offset2, start = ip;
continue; /* search a better one */
} }
/* let's find an even better one */
if ((depth==2) && (ip<ilimit)) {
+ DEBUGLOG(7, "search depth 2");
ip ++;
if ( (dictMode == ZSTD_noDict)
- && (offset) && ((offset_1>0) & (MEM_read32(ip) == MEM_read32(ip - offset_1)))) {
+ && (offcode) && ((offset_1>0) & (MEM_read32(ip) == MEM_read32(ip - offset_1)))) {
size_t const mlRep = ZSTD_count(ip+4, ip+4-offset_1, iend) + 4;
int const gain2 = (int)(mlRep * 4);
- int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 1);
+ int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offcode)) + 1);
if ((mlRep >= 4) && (gain2 > gain1))
- matchLength = mlRep, offset = 0, start = ip;
+ matchLength = mlRep, offcode = STORE_REPCODE_1, start = ip;
}
if (isDxS) {
const U32 repIndex = (U32)(ip - base) - offset_1;
const BYTE* repMatchEnd = repIndex < prefixLowestIndex ? dictEnd : iend;
size_t const mlRep = ZSTD_count_2segments(ip+4, repMatch+4, iend, repMatchEnd, prefixLowest) + 4;
int const gain2 = (int)(mlRep * 4);
- int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 1);
+ int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offcode)) + 1);
if ((mlRep >= 4) && (gain2 > gain1))
- matchLength = mlRep, offset = 0, start = ip;
+ matchLength = mlRep, offcode = STORE_REPCODE_1, start = ip;
}
}
{ size_t offset2=999999999;
- size_t const ml2 = searchMax(ms, ip, iend, &offset2);
- int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */
- int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 7);
+ size_t const ml2 = ZSTD_searchMax(ms, ip, iend, &offset2, mls, rowLog, searchMethod, dictMode);
+ int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offset2))); /* raw approx */
+ int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offcode)) + 7);
if ((ml2 >= 4) && (gain2 > gain1)) {
- matchLength = ml2, offset = offset2, start = ip;
+ matchLength = ml2, offcode = offset2, start = ip;
continue;
} } }
break; /* nothing found : store previous solution */
}
/* NOTE:
- * start[-offset+ZSTD_REP_MOVE-1] is undefined behavior.
- * (-offset+ZSTD_REP_MOVE-1) is unsigned, and is added to start, which
- * overflows the pointer, which is undefined behavior.
+ * Pay attention that `start[-value]` can lead to strange undefined behavior
+ * notably if `value` is unsigned, resulting in a large positive `-value`.
*/
/* catch up */
- if (offset) {
+ if (STORED_IS_OFFSET(offcode)) {
if (dictMode == ZSTD_noDict) {
- while ( ((start > anchor) & (start - (offset-ZSTD_REP_MOVE) > prefixLowest))
- && (start[-1] == (start-(offset-ZSTD_REP_MOVE))[-1]) ) /* only search for offset within prefix */
+ while ( ((start > anchor) & (start - STORED_OFFSET(offcode) > prefixLowest))
+ && (start[-1] == (start-STORED_OFFSET(offcode))[-1]) ) /* only search for offset within prefix */
{ start--; matchLength++; }
}
if (isDxS) {
- U32 const matchIndex = (U32)((start-base) - (offset - ZSTD_REP_MOVE));
+ U32 const matchIndex = (U32)((size_t)(start-base) - STORED_OFFSET(offcode));
const BYTE* match = (matchIndex < prefixLowestIndex) ? dictBase + matchIndex - dictIndexDelta : base + matchIndex;
const BYTE* const mStart = (matchIndex < prefixLowestIndex) ? dictLowest : prefixLowest;
while ((start>anchor) && (match>mStart) && (start[-1] == match[-1])) { start--; match--; matchLength++; } /* catch up */
}
- offset_2 = offset_1; offset_1 = (U32)(offset - ZSTD_REP_MOVE);
+ offset_2 = offset_1; offset_1 = (U32)STORED_OFFSET(offcode);
}
/* store sequence */
_storeSequence:
- { size_t const litLength = start - anchor;
- ZSTD_storeSeq(seqStore, litLength, anchor, iend, (U32)offset, matchLength-MINMATCH);
+ { size_t const litLength = (size_t)(start - anchor);
+ ZSTD_storeSeq(seqStore, litLength, anchor, iend, (U32)offcode, matchLength);
anchor = ip = start + matchLength;
}
&& (MEM_read32(repMatch) == MEM_read32(ip)) ) {
const BYTE* const repEnd2 = repIndex < prefixLowestIndex ? dictEnd : iend;
matchLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd2, prefixLowest) + 4;
- offset = offset_2; offset_2 = offset_1; offset_1 = (U32)offset; /* swap offset_2 <=> offset_1 */
- ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, matchLength-MINMATCH);
+ offcode = offset_2; offset_2 = offset_1; offset_1 = (U32)offcode; /* swap offset_2 <=> offset_1 */
+ ZSTD_storeSeq(seqStore, 0, anchor, iend, STORE_REPCODE_1, matchLength);
ip += matchLength;
anchor = ip;
continue;
&& (MEM_read32(ip) == MEM_read32(ip - offset_2)) ) {
/* store sequence */
matchLength = ZSTD_count(ip+4, ip+4-offset_2, iend) + 4;
- offset = offset_2; offset_2 = offset_1; offset_1 = (U32)offset; /* swap repcodes */
- ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, matchLength-MINMATCH);
+ offcode = offset_2; offset_2 = offset_1; offset_1 = (U32)offcode; /* swap repcodes */
+ ZSTD_storeSeq(seqStore, 0, anchor, iend, STORE_REPCODE_1, matchLength);
ip += matchLength;
anchor = ip;
continue; /* faster when present ... (?) */
return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_hashChain, 0, ZSTD_dedicatedDictSearch);
}
+/* Row-based matchfinder */
+size_t ZSTD_compressBlock_lazy2_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+{
+ return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 2, ZSTD_noDict);
+}
+
+size_t ZSTD_compressBlock_lazy_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+{
+ return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 1, ZSTD_noDict);
+}
+
+size_t ZSTD_compressBlock_greedy_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+{
+ return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 0, ZSTD_noDict);
+}
+
+size_t ZSTD_compressBlock_lazy2_dictMatchState_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+{
+ return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 2, ZSTD_dictMatchState);
+}
+
+size_t ZSTD_compressBlock_lazy_dictMatchState_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+{
+ return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 1, ZSTD_dictMatchState);
+}
+
+size_t ZSTD_compressBlock_greedy_dictMatchState_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+{
+ return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 0, ZSTD_dictMatchState);
+}
+
+
+size_t ZSTD_compressBlock_lazy2_dedicatedDictSearch_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+{
+ return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 2, ZSTD_dedicatedDictSearch);
+}
+
+size_t ZSTD_compressBlock_lazy_dedicatedDictSearch_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+{
+ return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 1, ZSTD_dedicatedDictSearch);
+}
+
+size_t ZSTD_compressBlock_greedy_dedicatedDictSearch_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+{
+ return ZSTD_compressBlock_lazy_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 0, ZSTD_dedicatedDictSearch);
+}
FORCE_INLINE_TEMPLATE
size_t ZSTD_compressBlock_lazy_extDict_generic(
const BYTE* ip = istart;
const BYTE* anchor = istart;
const BYTE* const iend = istart + srcSize;
- const BYTE* const ilimit = iend - 8;
+ const BYTE* const ilimit = searchMethod == search_rowHash ? iend - 8 - ZSTD_ROW_HASH_CACHE_SIZE : iend - 8;
const BYTE* const base = ms->window.base;
const U32 dictLimit = ms->window.dictLimit;
const BYTE* const prefixStart = base + dictLimit;
const BYTE* const dictEnd = dictBase + dictLimit;
const BYTE* const dictStart = dictBase + ms->window.lowLimit;
const U32 windowLog = ms->cParams.windowLog;
-
- typedef size_t (*searchMax_f)(
- ZSTD_matchState_t* ms,
- const BYTE* ip, const BYTE* iLimit, size_t* offsetPtr);
- searchMax_f searchMax = searchMethod==search_binaryTree ? ZSTD_BtFindBestMatch_extDict_selectMLS : ZSTD_HcFindBestMatch_extDict_selectMLS;
+ const U32 mls = BOUNDED(4, ms->cParams.minMatch, 6);
+ const U32 rowLog = BOUNDED(4, ms->cParams.searchLog, 6);
U32 offset_1 = rep[0], offset_2 = rep[1];
- DEBUGLOG(5, "ZSTD_compressBlock_lazy_extDict_generic");
+ DEBUGLOG(5, "ZSTD_compressBlock_lazy_extDict_generic (searchFunc=%u)", (U32)searchMethod);
/* init */
ip += (ip == prefixStart);
+ if (searchMethod == search_rowHash) {
+ ZSTD_row_fillHashCache(ms, base, rowLog,
+ MIN(ms->cParams.minMatch, 6 /* mls caps out at 6 */),
+ ms->nextToUpdate, ilimit);
+ }
/* Match Loop */
#if defined(__x86_64__)
#endif
while (ip < ilimit) {
size_t matchLength=0;
- size_t offset=0;
+ size_t offcode=STORE_REPCODE_1;
const BYTE* start=ip+1;
U32 curr = (U32)(ip-base);
const U32 repIndex = (U32)(curr+1 - offset_1);
const BYTE* const repBase = repIndex < dictLimit ? dictBase : base;
const BYTE* const repMatch = repBase + repIndex;
- if (((U32)((dictLimit-1) - repIndex) >= 3) & (repIndex > windowLow)) /* intentional overflow */
+ if ( ((U32)((dictLimit-1) - repIndex) >= 3) /* intentional overflow */
+ & (offset_1 <= curr+1 - windowLow) ) /* note: we are searching at curr+1 */
if (MEM_read32(ip+1) == MEM_read32(repMatch)) {
/* repcode detected we should take it */
const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend;
/* first search (depth 0) */
{ size_t offsetFound = 999999999;
- size_t const ml2 = searchMax(ms, ip, iend, &offsetFound);
+ size_t const ml2 = ZSTD_searchMax(ms, ip, iend, &offsetFound, mls, rowLog, searchMethod, ZSTD_extDict);
if (ml2 > matchLength)
- matchLength = ml2, start = ip, offset=offsetFound;
+ matchLength = ml2, start = ip, offcode=offsetFound;
}
if (matchLength < 4) {
ip ++;
curr++;
/* check repCode */
- if (offset) {
+ if (offcode) {
const U32 windowLow = ZSTD_getLowestMatchIndex(ms, curr, windowLog);
const U32 repIndex = (U32)(curr - offset_1);
const BYTE* const repBase = repIndex < dictLimit ? dictBase : base;
const BYTE* const repMatch = repBase + repIndex;
- if (((U32)((dictLimit-1) - repIndex) >= 3) & (repIndex > windowLow)) /* intentional overflow */
+ if ( ((U32)((dictLimit-1) - repIndex) >= 3) /* intentional overflow : do not test positions overlapping 2 memory segments */
+ & (offset_1 <= curr - windowLow) ) /* equivalent to `curr > repIndex >= windowLow` */
if (MEM_read32(ip) == MEM_read32(repMatch)) {
/* repcode detected */
const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend;
size_t const repLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd, prefixStart) + 4;
int const gain2 = (int)(repLength * 3);
- int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)offset+1) + 1);
+ int const gain1 = (int)(matchLength*3 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offcode)) + 1);
if ((repLength >= 4) && (gain2 > gain1))
- matchLength = repLength, offset = 0, start = ip;
+ matchLength = repLength, offcode = STORE_REPCODE_1, start = ip;
} }
/* search match, depth 1 */
{ size_t offset2=999999999;
- size_t const ml2 = searchMax(ms, ip, iend, &offset2);
- int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */
- int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 4);
+ size_t const ml2 = ZSTD_searchMax(ms, ip, iend, &offset2, mls, rowLog, searchMethod, ZSTD_extDict);
+ int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offset2))); /* raw approx */
+ int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offcode)) + 4);
if ((ml2 >= 4) && (gain2 > gain1)) {
- matchLength = ml2, offset = offset2, start = ip;
+ matchLength = ml2, offcode = offset2, start = ip;
continue; /* search a better one */
} }
ip ++;
curr++;
/* check repCode */
- if (offset) {
+ if (offcode) {
const U32 windowLow = ZSTD_getLowestMatchIndex(ms, curr, windowLog);
const U32 repIndex = (U32)(curr - offset_1);
const BYTE* const repBase = repIndex < dictLimit ? dictBase : base;
const BYTE* const repMatch = repBase + repIndex;
- if (((U32)((dictLimit-1) - repIndex) >= 3) & (repIndex > windowLow)) /* intentional overflow */
+ if ( ((U32)((dictLimit-1) - repIndex) >= 3) /* intentional overflow : do not test positions overlapping 2 memory segments */
+ & (offset_1 <= curr - windowLow) ) /* equivalent to `curr > repIndex >= windowLow` */
if (MEM_read32(ip) == MEM_read32(repMatch)) {
/* repcode detected */
const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend;
size_t const repLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd, prefixStart) + 4;
int const gain2 = (int)(repLength * 4);
- int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 1);
+ int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offcode)) + 1);
if ((repLength >= 4) && (gain2 > gain1))
- matchLength = repLength, offset = 0, start = ip;
+ matchLength = repLength, offcode = STORE_REPCODE_1, start = ip;
} }
/* search match, depth 2 */
{ size_t offset2=999999999;
- size_t const ml2 = searchMax(ms, ip, iend, &offset2);
- int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)offset2+1)); /* raw approx */
- int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)offset+1) + 7);
+ size_t const ml2 = ZSTD_searchMax(ms, ip, iend, &offset2, mls, rowLog, searchMethod, ZSTD_extDict);
+ int const gain2 = (int)(ml2*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offset2))); /* raw approx */
+ int const gain1 = (int)(matchLength*4 - ZSTD_highbit32((U32)STORED_TO_OFFBASE(offcode)) + 7);
if ((ml2 >= 4) && (gain2 > gain1)) {
- matchLength = ml2, offset = offset2, start = ip;
+ matchLength = ml2, offcode = offset2, start = ip;
continue;
} } }
break; /* nothing found : store previous solution */
}
/* catch up */
- if (offset) {
- U32 const matchIndex = (U32)((start-base) - (offset - ZSTD_REP_MOVE));
+ if (STORED_IS_OFFSET(offcode)) {
+ U32 const matchIndex = (U32)((size_t)(start-base) - STORED_OFFSET(offcode));
const BYTE* match = (matchIndex < dictLimit) ? dictBase + matchIndex : base + matchIndex;
const BYTE* const mStart = (matchIndex < dictLimit) ? dictStart : prefixStart;
while ((start>anchor) && (match>mStart) && (start[-1] == match[-1])) { start--; match--; matchLength++; } /* catch up */
- offset_2 = offset_1; offset_1 = (U32)(offset - ZSTD_REP_MOVE);
+ offset_2 = offset_1; offset_1 = (U32)STORED_OFFSET(offcode);
}
/* store sequence */
_storeSequence:
- { size_t const litLength = start - anchor;
- ZSTD_storeSeq(seqStore, litLength, anchor, iend, (U32)offset, matchLength-MINMATCH);
+ { size_t const litLength = (size_t)(start - anchor);
+ ZSTD_storeSeq(seqStore, litLength, anchor, iend, (U32)offcode, matchLength);
anchor = ip = start + matchLength;
}
const U32 repIndex = repCurrent - offset_2;
const BYTE* const repBase = repIndex < dictLimit ? dictBase : base;
const BYTE* const repMatch = repBase + repIndex;
- if (((U32)((dictLimit-1) - repIndex) >= 3) & (repIndex > windowLow)) /* intentional overflow */
+ if ( ((U32)((dictLimit-1) - repIndex) >= 3) /* intentional overflow : do not test positions overlapping 2 memory segments */
+ & (offset_2 <= repCurrent - windowLow) ) /* equivalent to `curr > repIndex >= windowLow` */
if (MEM_read32(ip) == MEM_read32(repMatch)) {
/* repcode detected we should take it */
const BYTE* const repEnd = repIndex < dictLimit ? dictEnd : iend;
matchLength = ZSTD_count_2segments(ip+4, repMatch+4, iend, repEnd, prefixStart) + 4;
- offset = offset_2; offset_2 = offset_1; offset_1 = (U32)offset; /* swap offset history */
- ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, matchLength-MINMATCH);
+ offcode = offset_2; offset_2 = offset_1; offset_1 = (U32)offcode; /* swap offset history */
+ ZSTD_storeSeq(seqStore, 0, anchor, iend, STORE_REPCODE_1, matchLength);
ip += matchLength;
anchor = ip;
continue; /* faster when present ... (?) */
{
return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_binaryTree, 2);
}
+
+size_t ZSTD_compressBlock_greedy_extDict_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+{
+ return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 0);
+}
+
+size_t ZSTD_compressBlock_lazy_extDict_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+
+{
+ return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 1);
+}
+
+size_t ZSTD_compressBlock_lazy2_extDict_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize)
+
+{
+ return ZSTD_compressBlock_lazy_extDict_generic(ms, seqStore, rep, src, srcSize, search_rowHash, 2);
+}
#define ZSTD_LAZY_DDSS_BUCKET_LOG 2
U32 ZSTD_insertAndFindFirstIndex(ZSTD_matchState_t* ms, const BYTE* ip);
+void ZSTD_row_update(ZSTD_matchState_t* const ms, const BYTE* ip);
void ZSTD_dedicatedDictSearch_lazy_loadDictionary(ZSTD_matchState_t* ms, const BYTE* const ip);
size_t ZSTD_compressBlock_greedy(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_lazy2_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_lazy_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_greedy_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
size_t ZSTD_compressBlock_btlazy2_dictMatchState(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
size_t ZSTD_compressBlock_greedy_dictMatchState(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_lazy2_dictMatchState_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_lazy_dictMatchState_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_greedy_dictMatchState_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
size_t ZSTD_compressBlock_lazy2_dedicatedDictSearch(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
size_t ZSTD_compressBlock_greedy_dedicatedDictSearch(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_lazy2_dedicatedDictSearch_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_lazy_dedicatedDictSearch_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_greedy_dedicatedDictSearch_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
size_t ZSTD_compressBlock_greedy_extDict(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
size_t ZSTD_compressBlock_lazy2_extDict(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_greedy_extDict_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_lazy_extDict_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
+size_t ZSTD_compressBlock_lazy2_extDict_row(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ void const* src, size_t srcSize);
size_t ZSTD_compressBlock_btlazy2_extDict(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
void const* src, size_t srcSize);
+
#endif /* ZSTD_LAZY_H */
}
}
+/* ZSTD_ldm_gear_reset()
+ * Feeds [data, data + minMatchLength) into the hash without registering any
+ * splits. This effectively resets the hash state. This is used when skipping
+ * over data, either at the beginning of a block, or skipping sections.
+ */
+static void ZSTD_ldm_gear_reset(ldmRollingHashState_t* state,
+ BYTE const* data, size_t minMatchLength)
+{
+ U64 hash = state->rolling;
+ size_t n = 0;
+
+#define GEAR_ITER_ONCE() do { \
+ hash = (hash << 1) + ZSTD_ldm_gearTab[data[n] & 0xff]; \
+ n += 1; \
+ } while (0)
+ while (n + 3 < minMatchLength) {
+ GEAR_ITER_ONCE();
+ GEAR_ITER_ONCE();
+ GEAR_ITER_ONCE();
+ GEAR_ITER_ONCE();
+ }
+ while (n < minMatchLength) {
+ GEAR_ITER_ONCE();
+ }
+#undef GEAR_ITER_ONCE
+}
+
/* ZSTD_ldm_gear_feed():
*
* Registers in the splits array all the split points found in the first
size_t const ldmBucketSize = ((size_t)1) << (params.hashLog - ldmBucketSizeLog);
size_t const totalSize = ZSTD_cwksp_alloc_size(ldmBucketSize)
+ ZSTD_cwksp_alloc_size(ldmHSize * sizeof(ldmEntry_t));
- return params.enableLdm ? totalSize : 0;
+ return params.enableLdm == ZSTD_ps_enable ? totalSize : 0;
}
size_t ZSTD_ldm_getMaxNbSeq(ldmParams_t params, size_t maxChunkSize)
{
- return params.enableLdm ? (maxChunkSize / params.minMatchLength) : 0;
+ return params.enableLdm == ZSTD_ps_enable ? (maxChunkSize / params.minMatchLength) : 0;
}
/* ZSTD_ldm_getBucket() :
while (ip < iend) {
size_t hashed;
unsigned n;
-
+
numSplits = 0;
hashed = ZSTD_ldm_gear_feed(&hashState, ip, iend - ip, splits, &numSplits);
/* Initialize the rolling hash state with the first minMatchLength bytes */
ZSTD_ldm_gear_init(&hashState, params);
- {
- size_t n = 0;
-
- while (n < minMatchLength) {
- numSplits = 0;
- n += ZSTD_ldm_gear_feed(&hashState, ip + n, minMatchLength - n,
- splits, &numSplits);
- }
- ip += minMatchLength;
- }
+ ZSTD_ldm_gear_reset(&hashState, ip, minMatchLength);
+ ip += minMatchLength;
while (ip < ilimit) {
size_t hashed;
for (n = 0; n < numSplits; n++) {
size_t forwardMatchLength = 0, backwardMatchLength = 0,
bestMatchLength = 0, mLength;
+ U32 offset;
BYTE const* const split = candidates[n].split;
U32 const checksum = candidates[n].checksum;
U32 const hash = candidates[n].hash;
}
/* Match found */
+ offset = (U32)(split - base) - bestEntry->offset;
mLength = forwardMatchLength + backwardMatchLength;
{
- U32 const offset = (U32)(split - base) - bestEntry->offset;
rawSeq* const seq = rawSeqStore->seq + rawSeqStore->size;
/* Out of sequence storage */
ZSTD_ldm_insertEntry(ldmState, hash, newEntry, *params);
anchor = split + forwardMatchLength;
+
+ /* If we find a match that ends after the data that we've hashed
+ * then we have a repeating, overlapping, pattern. E.g. all zeros.
+ * If one repetition of the pattern matches our `stopMask` then all
+ * repetitions will. We don't need to insert them all into out table,
+ * only the first one. So skip over overlapping matches.
+ * This is a major speed boost (20x) for compressing a single byte
+ * repeated, when that byte ends up in the table.
+ */
+ if (anchor > ip + hashed) {
+ ZSTD_ldm_gear_reset(&hashState, anchor - minMatchLength, minMatchLength);
+ /* Continue the outer loop at anchor (ip + hashed == anchor). */
+ ip = anchor - hashed;
+ break;
+ }
}
ip += hashed;
assert(chunkStart < iend);
/* 1. Perform overflow correction if necessary. */
- if (ZSTD_window_needOverflowCorrection(ldmState->window, chunkEnd)) {
+ if (ZSTD_window_needOverflowCorrection(ldmState->window, 0, maxDist, ldmState->loadedDictEnd, chunkStart, chunkEnd)) {
U32 const ldmHSize = 1U << params->hashLog;
U32 const correction = ZSTD_window_correctOverflow(
&ldmState->window, /* cycleLog */ 0, maxDist, chunkStart);
return 0;
}
-void ZSTD_ldm_skipSequences(rawSeqStore_t* rawSeqStore, size_t srcSize, U32 const minMatch) {
+void
+ZSTD_ldm_skipSequences(rawSeqStore_t* rawSeqStore, size_t srcSize, U32 const minMatch)
+{
while (srcSize > 0 && rawSeqStore->pos < rawSeqStore->size) {
rawSeq* seq = rawSeqStore->seq + rawSeqStore->pos;
if (srcSize <= seq->litLength) {
size_t ZSTD_ldm_blockCompress(rawSeqStore_t* rawSeqStore,
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ ZSTD_paramSwitch_e useRowMatchFinder,
void const* src, size_t srcSize)
{
const ZSTD_compressionParameters* const cParams = &ms->cParams;
unsigned const minMatch = cParams->minMatch;
ZSTD_blockCompressor const blockCompressor =
- ZSTD_selectBlockCompressor(cParams->strategy, ZSTD_matchState_dictMode(ms));
+ ZSTD_selectBlockCompressor(cParams->strategy, useRowMatchFinder, ZSTD_matchState_dictMode(ms));
/* Input bounds */
BYTE const* const istart = (BYTE const*)src;
BYTE const* const iend = istart + srcSize;
rep[0] = sequence.offset;
/* Store the sequence */
ZSTD_storeSeq(seqStore, newLitLength, ip - newLitLength, iend,
- sequence.offset + ZSTD_REP_MOVE,
- sequence.matchLength - MINMATCH);
+ STORE_OFFSET(sequence.offset),
+ sequence.matchLength);
ip += sequence.matchLength;
}
}
*/
size_t ZSTD_ldm_blockCompress(rawSeqStore_t* rawSeqStore,
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ ZSTD_paramSwitch_e useRowMatchFinder,
void const* src, size_t srcSize);
/*
#ifndef ZSTD_LDM_GEARTAB_H
#define ZSTD_LDM_GEARTAB_H
-static U64 ZSTD_ldm_gearTab[256] = {
+#include "../common/compiler.h" /* UNUSED_ATTR */
+#include "../common/mem.h" /* U64 */
+
+static UNUSED_ATTR const U64 ZSTD_ldm_gearTab[256] = {
0xf5b8f72c5f77775c, 0x84935f266b7ac412, 0xb647ada9ca730ccc,
0xb065bb4b114fb1de, 0x34584e7e8c3a9fd0, 0x4e97e17c6ae26b05,
0x3a03d743bc99a604, 0xcecd042422c4044f, 0x76de76c58524259e,
* You may select, at your option, one of the above-listed licenses.
*/
-/*
- * Disable inlining for the optimal parser for the kernel build.
- * It is unlikely to be used in the kernel, and where it is used
- * latency shouldn't matter because it is very slow to begin with.
- * We prefer a ~180KB binary size win over faster optimal parsing.
- *
- * TODO(https://github.com/facebook/zstd/issues/2862):
- * Improve the code size of the optimal parser in general, so we
- * don't need this hack for the kernel build.
- */
-#define ZSTD_NO_INLINE 1
-
#include "zstd_compress_internal.h"
#include "hist.h"
#include "zstd_opt.h"
#define ZSTD_LITFREQ_ADD 2 /* scaling factor for litFreq, so that frequencies adapt faster to new stats */
-#define ZSTD_FREQ_DIV 4 /* log factor when using previous stats to init next stats */
#define ZSTD_MAX_PRICE (1<<30)
#define ZSTD_PREDEF_THRESHOLD 1024 /* if srcSize < ZSTD_PREDEF_THRESHOLD, symbols' cost is assumed static, directly determined by pre-defined distributions */
* Price functions for optimal parser
***************************************/
-#if 0 /* approximation at bit level */
+#if 0 /* approximation at bit level (for tests) */
# define BITCOST_ACCURACY 0
# define BITCOST_MULTIPLIER (1 << BITCOST_ACCURACY)
-# define WEIGHT(stat) ((void)opt, ZSTD_bitWeight(stat))
-#elif 0 /* fractional bit accuracy */
+# define WEIGHT(stat, opt) ((void)opt, ZSTD_bitWeight(stat))
+#elif 0 /* fractional bit accuracy (for tests) */
# define BITCOST_ACCURACY 8
# define BITCOST_MULTIPLIER (1 << BITCOST_ACCURACY)
# define WEIGHT(stat,opt) ((void)opt, ZSTD_fracWeight(stat))
static int ZSTD_compressedLiterals(optState_t const* const optPtr)
{
- return optPtr->literalCompressionMode != ZSTD_lcm_uncompressed;
+ return optPtr->literalCompressionMode != ZSTD_ps_disable;
}
static void ZSTD_setBasePrices(optState_t* optPtr, int optLevel)
}
-/* ZSTD_downscaleStat() :
- * reduce all elements in table by a factor 2^(ZSTD_FREQ_DIV+malus)
- * return the resulting sum of elements */
-static U32 ZSTD_downscaleStat(unsigned* table, U32 lastEltIndex, int malus)
+static U32 sum_u32(const unsigned table[], size_t nbElts)
+{
+ size_t n;
+ U32 total = 0;
+ for (n=0; n<nbElts; n++) {
+ total += table[n];
+ }
+ return total;
+}
+
+static U32 ZSTD_downscaleStats(unsigned* table, U32 lastEltIndex, U32 shift)
{
U32 s, sum=0;
- DEBUGLOG(5, "ZSTD_downscaleStat (nbElts=%u)", (unsigned)lastEltIndex+1);
- assert(ZSTD_FREQ_DIV+malus > 0 && ZSTD_FREQ_DIV+malus < 31);
+ DEBUGLOG(5, "ZSTD_downscaleStats (nbElts=%u, shift=%u)", (unsigned)lastEltIndex+1, (unsigned)shift);
+ assert(shift < 30);
for (s=0; s<lastEltIndex+1; s++) {
- table[s] = 1 + (table[s] >> (ZSTD_FREQ_DIV+malus));
+ table[s] = 1 + (table[s] >> shift);
sum += table[s];
}
return sum;
}
+/* ZSTD_scaleStats() :
+ * reduce all elements in table is sum too large
+ * return the resulting sum of elements */
+static U32 ZSTD_scaleStats(unsigned* table, U32 lastEltIndex, U32 logTarget)
+{
+ U32 const prevsum = sum_u32(table, lastEltIndex+1);
+ U32 const factor = prevsum >> logTarget;
+ DEBUGLOG(5, "ZSTD_scaleStats (nbElts=%u, target=%u)", (unsigned)lastEltIndex+1, (unsigned)logTarget);
+ assert(logTarget < 30);
+ if (factor <= 1) return prevsum;
+ return ZSTD_downscaleStats(table, lastEltIndex, ZSTD_highbit32(factor));
+}
+
/* ZSTD_rescaleFreqs() :
* if first block (detected by optPtr->litLengthSum == 0) : init statistics
* take hints from dictionary if there is one
- * or init from zero, using src for literals stats, or flat 1 for match symbols
+ * and init from zero if there is none,
+ * using src for literals stats, and baseline stats for sequence symbols
* otherwise downscale existing stats, to be used as seed for next block.
*/
static void
optPtr->litSum = 0;
for (lit=0; lit<=MaxLit; lit++) {
U32 const scaleLog = 11; /* scale to 2K */
- U32 const bitCost = HUF_getNbBits(optPtr->symbolCosts->huf.CTable, lit);
+ U32 const bitCost = HUF_getNbBitsFromCTable(optPtr->symbolCosts->huf.CTable, lit);
assert(bitCost <= scaleLog);
optPtr->litFreq[lit] = bitCost ? 1 << (scaleLog-bitCost) : 1 /*minimum to calculate cost*/;
optPtr->litSum += optPtr->litFreq[lit];
if (compressedLiterals) {
unsigned lit = MaxLit;
HIST_count_simple(optPtr->litFreq, &lit, src, srcSize); /* use raw first block to init statistics */
- optPtr->litSum = ZSTD_downscaleStat(optPtr->litFreq, MaxLit, 1);
+ optPtr->litSum = ZSTD_downscaleStats(optPtr->litFreq, MaxLit, 8);
}
- { unsigned ll;
- for (ll=0; ll<=MaxLL; ll++)
- optPtr->litLengthFreq[ll] = 1;
+ { unsigned const baseLLfreqs[MaxLL+1] = {
+ 4, 2, 1, 1, 1, 1, 1, 1,
+ 1, 1, 1, 1, 1, 1, 1, 1,
+ 1, 1, 1, 1, 1, 1, 1, 1,
+ 1, 1, 1, 1, 1, 1, 1, 1,
+ 1, 1, 1, 1
+ };
+ ZSTD_memcpy(optPtr->litLengthFreq, baseLLfreqs, sizeof(baseLLfreqs));
+ optPtr->litLengthSum = sum_u32(baseLLfreqs, MaxLL+1);
}
- optPtr->litLengthSum = MaxLL+1;
{ unsigned ml;
for (ml=0; ml<=MaxML; ml++)
}
optPtr->matchLengthSum = MaxML+1;
- { unsigned of;
- for (of=0; of<=MaxOff; of++)
- optPtr->offCodeFreq[of] = 1;
+ { unsigned const baseOFCfreqs[MaxOff+1] = {
+ 6, 2, 1, 1, 2, 3, 4, 4,
+ 4, 3, 2, 1, 1, 1, 1, 1,
+ 1, 1, 1, 1, 1, 1, 1, 1,
+ 1, 1, 1, 1, 1, 1, 1, 1
+ };
+ ZSTD_memcpy(optPtr->offCodeFreq, baseOFCfreqs, sizeof(baseOFCfreqs));
+ optPtr->offCodeSum = sum_u32(baseOFCfreqs, MaxOff+1);
}
- optPtr->offCodeSum = MaxOff+1;
+
}
} else { /* new block : re-use previous statistics, scaled down */
if (compressedLiterals)
- optPtr->litSum = ZSTD_downscaleStat(optPtr->litFreq, MaxLit, 1);
- optPtr->litLengthSum = ZSTD_downscaleStat(optPtr->litLengthFreq, MaxLL, 0);
- optPtr->matchLengthSum = ZSTD_downscaleStat(optPtr->matchLengthFreq, MaxML, 0);
- optPtr->offCodeSum = ZSTD_downscaleStat(optPtr->offCodeFreq, MaxOff, 0);
+ optPtr->litSum = ZSTD_scaleStats(optPtr->litFreq, MaxLit, 12);
+ optPtr->litLengthSum = ZSTD_scaleStats(optPtr->litLengthFreq, MaxLL, 11);
+ optPtr->matchLengthSum = ZSTD_scaleStats(optPtr->matchLengthFreq, MaxML, 11);
+ optPtr->offCodeSum = ZSTD_scaleStats(optPtr->offCodeFreq, MaxOff, 11);
}
ZSTD_setBasePrices(optPtr, optLevel);
* cost of literalLength symbol */
static U32 ZSTD_litLengthPrice(U32 const litLength, const optState_t* const optPtr, int optLevel)
{
- if (optPtr->priceType == zop_predef) return WEIGHT(litLength, optLevel);
+ assert(litLength <= ZSTD_BLOCKSIZE_MAX);
+ if (optPtr->priceType == zop_predef)
+ return WEIGHT(litLength, optLevel);
+ /* We can't compute the litLength price for sizes >= ZSTD_BLOCKSIZE_MAX
+ * because it isn't representable in the zstd format. So instead just
+ * call it 1 bit more than ZSTD_BLOCKSIZE_MAX - 1. In this case the block
+ * would be all literals.
+ */
+ if (litLength == ZSTD_BLOCKSIZE_MAX)
+ return BITCOST_MULTIPLIER + ZSTD_litLengthPrice(ZSTD_BLOCKSIZE_MAX - 1, optPtr, optLevel);
/* dynamic statistics */
{ U32 const llCode = ZSTD_LLcode(litLength);
/* ZSTD_getMatchPrice() :
* Provides the cost of the match part (offset + matchLength) of a sequence
* Must be combined with ZSTD_fullLiteralsCost() to get the full cost of a sequence.
- * optLevel: when <2, favors small offset for decompression speed (improved cache efficiency) */
+ * @offcode : expects a scale where 0,1,2 are repcodes 1-3, and 3+ are real_offsets+2
+ * @optLevel: when <2, favors small offset for decompression speed (improved cache efficiency)
+ */
FORCE_INLINE_TEMPLATE U32
-ZSTD_getMatchPrice(U32 const offset,
+ZSTD_getMatchPrice(U32 const offcode,
U32 const matchLength,
const optState_t* const optPtr,
int const optLevel)
{
U32 price;
- U32 const offCode = ZSTD_highbit32(offset+1);
+ U32 const offCode = ZSTD_highbit32(STORED_TO_OFFBASE(offcode));
U32 const mlBase = matchLength - MINMATCH;
assert(matchLength >= MINMATCH);
optPtr->litLengthSum++;
}
- /* match offset code (0-2=>repCode; 3+=>offset+2) */
- { U32 const offCode = ZSTD_highbit32(offsetCode+1);
+ /* offset code : expected to follow storeSeq() numeric representation */
+ { U32 const offCode = ZSTD_highbit32(STORED_TO_OFFBASE(offsetCode));
assert(offCode <= MaxOff);
optPtr->offCodeFreq[offCode]++;
optPtr->offCodeSum++;
/* Update hashTable3 up to ip (excluded)
Assumption : always within prefix (i.e. not within extDict) */
-static U32 ZSTD_insertAndFindFirstIndexHash3 (ZSTD_matchState_t* ms,
+static U32 ZSTD_insertAndFindFirstIndexHash3 (const ZSTD_matchState_t* ms,
U32* nextToUpdate3,
const BYTE* const ip)
{
* Binary Tree search
***************************************/
/* ZSTD_insertBt1() : add one or multiple positions to tree.
- * ip : assumed <= iend-8 .
+ * @param ip assumed <= iend-8 .
+ * @param target The target of ZSTD_updateTree_internal() - we are filling to this position
* @return : nb of positions added */
static U32 ZSTD_insertBt1(
- ZSTD_matchState_t* ms,
+ const ZSTD_matchState_t* ms,
const BYTE* const ip, const BYTE* const iend,
+ U32 const target,
U32 const mls, const int extDict)
{
const ZSTD_compressionParameters* const cParams = &ms->cParams;
U32* smallerPtr = bt + 2*(curr&btMask);
U32* largerPtr = smallerPtr + 1;
U32 dummy32; /* to be nullified at the end */
- U32 const windowLow = ms->window.lowLimit;
+ /* windowLow is based on target because
+ * we only need positions that will be in the window at the end of the tree update.
+ */
+ U32 const windowLow = ZSTD_getLowestMatchIndex(ms, target, cParams->windowLog);
U32 matchEndIdx = curr+8+1;
size_t bestLength = 8;
U32 nbCompares = 1U << cParams->searchLog;
DEBUGLOG(8, "ZSTD_insertBt1 (%u)", curr);
+ assert(curr <= target);
assert(ip <= iend-8); /* required for h calculation */
hashTable[h] = curr; /* Update Hash Table */
idx, target, dictMode);
while(idx < target) {
- U32 const forward = ZSTD_insertBt1(ms, base+idx, iend, mls, dictMode == ZSTD_extDict);
+ U32 const forward = ZSTD_insertBt1(ms, base+idx, iend, target, mls, dictMode == ZSTD_extDict);
assert(idx < (U32)(idx + forward));
idx += forward;
}
DEBUGLOG(8, "found repCode %u (ll0:%u, offset:%u) of length %u",
repCode, ll0, repOffset, repLen);
bestLength = repLen;
- matches[mnum].off = repCode - ll0;
+ matches[mnum].off = STORE_REPCODE(repCode - ll0 + 1); /* expect value between 1 and 3 */
matches[mnum].len = (U32)repLen;
mnum++;
if ( (repLen > sufficient_len)
bestLength = mlen;
assert(curr > matchIndex3);
assert(mnum==0); /* no prior solution */
- matches[0].off = (curr - matchIndex3) + ZSTD_REP_MOVE;
+ matches[0].off = STORE_OFFSET(curr - matchIndex3);
matches[0].len = (U32)mlen;
mnum = 1;
if ( (mlen > sufficient_len) |
return 1;
} } }
/* no dictMatchState lookup: dicts don't have a populated HC3 table */
- }
+ } /* if (mls == 3) */
hashTable[h] = curr; /* Update Hash Table */
if (matchLength > bestLength) {
DEBUGLOG(8, "found match of length %u at distance %u (offCode=%u)",
- (U32)matchLength, curr - matchIndex, curr - matchIndex + ZSTD_REP_MOVE);
+ (U32)matchLength, curr - matchIndex, STORE_OFFSET(curr - matchIndex));
assert(matchEndIdx > matchIndex);
if (matchLength > matchEndIdx - matchIndex)
matchEndIdx = matchIndex + (U32)matchLength;
bestLength = matchLength;
- matches[mnum].off = (curr - matchIndex) + ZSTD_REP_MOVE;
+ matches[mnum].off = STORE_OFFSET(curr - matchIndex);
matches[mnum].len = (U32)matchLength;
mnum++;
if ( (matchLength > ZSTD_OPT_NUM)
| (ip+matchLength == iLimit) /* equal : no way to know if inf or sup */) {
if (dictMode == ZSTD_dictMatchState) nbCompares = 0; /* break should also skip searching dms */
break; /* drop, to preserve bt consistency (miss a little bit of compression) */
- }
- }
+ } }
if (match[matchLength] < ip[matchLength]) {
/* match smaller than current */
if (matchLength > bestLength) {
matchIndex = dictMatchIndex + dmsIndexDelta;
DEBUGLOG(8, "found dms match of length %u at distance %u (offCode=%u)",
- (U32)matchLength, curr - matchIndex, curr - matchIndex + ZSTD_REP_MOVE);
+ (U32)matchLength, curr - matchIndex, STORE_OFFSET(curr - matchIndex));
if (matchLength > matchEndIdx - matchIndex)
matchEndIdx = matchIndex + (U32)matchLength;
bestLength = matchLength;
- matches[mnum].off = (curr - matchIndex) + ZSTD_REP_MOVE;
+ matches[mnum].off = STORE_OFFSET(curr - matchIndex);
matches[mnum].len = (U32)matchLength;
mnum++;
if ( (matchLength > ZSTD_OPT_NUM)
| (ip+matchLength == iLimit) /* equal : no way to know if inf or sup */) {
break; /* drop, to guarantee consistency (miss a little bit of compression) */
- }
- }
+ } }
if (dictMatchIndex <= dmsBtLow) { break; } /* beyond tree size, stop the search */
if (match[matchLength] < ip[matchLength]) {
/* match is larger than current */
commonLengthLarger = matchLength;
dictMatchIndex = nextPtr[0];
- }
- }
- }
+ } } } /* if (dictMode == ZSTD_dictMatchState) */
assert(matchEndIdx > curr+8);
ms->nextToUpdate = matchEndIdx - 8; /* skip repetitive patterns */
return mnum;
}
-
-FORCE_INLINE_TEMPLATE U32 ZSTD_BtGetAllMatches (
- ZSTD_match_t* matches, /* store result (match found, increasing size) in this table */
- ZSTD_matchState_t* ms,
- U32* nextToUpdate3,
- const BYTE* ip, const BYTE* const iHighLimit, const ZSTD_dictMode_e dictMode,
- const U32 rep[ZSTD_REP_NUM],
- U32 const ll0,
- U32 const lengthToBeat)
+typedef U32 (*ZSTD_getAllMatchesFn)(
+ ZSTD_match_t*,
+ ZSTD_matchState_t*,
+ U32*,
+ const BYTE*,
+ const BYTE*,
+ const U32 rep[ZSTD_REP_NUM],
+ U32 const ll0,
+ U32 const lengthToBeat);
+
+FORCE_INLINE_TEMPLATE U32 ZSTD_btGetAllMatches_internal(
+ ZSTD_match_t* matches,
+ ZSTD_matchState_t* ms,
+ U32* nextToUpdate3,
+ const BYTE* ip,
+ const BYTE* const iHighLimit,
+ const U32 rep[ZSTD_REP_NUM],
+ U32 const ll0,
+ U32 const lengthToBeat,
+ const ZSTD_dictMode_e dictMode,
+ const U32 mls)
{
- const ZSTD_compressionParameters* const cParams = &ms->cParams;
- U32 const matchLengthSearch = cParams->minMatch;
- DEBUGLOG(8, "ZSTD_BtGetAllMatches");
- if (ip < ms->window.base + ms->nextToUpdate) return 0; /* skipped area */
- ZSTD_updateTree_internal(ms, ip, iHighLimit, matchLengthSearch, dictMode);
- switch(matchLengthSearch)
- {
- case 3 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 3);
- default :
- case 4 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 4);
- case 5 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 5);
- case 7 :
- case 6 : return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, 6);
+ assert(BOUNDED(3, ms->cParams.minMatch, 6) == mls);
+ DEBUGLOG(8, "ZSTD_BtGetAllMatches(dictMode=%d, mls=%u)", (int)dictMode, mls);
+ if (ip < ms->window.base + ms->nextToUpdate)
+ return 0; /* skipped area */
+ ZSTD_updateTree_internal(ms, ip, iHighLimit, mls, dictMode);
+ return ZSTD_insertBtAndGetAllMatches(matches, ms, nextToUpdate3, ip, iHighLimit, dictMode, rep, ll0, lengthToBeat, mls);
+}
+
+#define ZSTD_BT_GET_ALL_MATCHES_FN(dictMode, mls) ZSTD_btGetAllMatches_##dictMode##_##mls
+
+#define GEN_ZSTD_BT_GET_ALL_MATCHES_(dictMode, mls) \
+ static U32 ZSTD_BT_GET_ALL_MATCHES_FN(dictMode, mls)( \
+ ZSTD_match_t* matches, \
+ ZSTD_matchState_t* ms, \
+ U32* nextToUpdate3, \
+ const BYTE* ip, \
+ const BYTE* const iHighLimit, \
+ const U32 rep[ZSTD_REP_NUM], \
+ U32 const ll0, \
+ U32 const lengthToBeat) \
+ { \
+ return ZSTD_btGetAllMatches_internal( \
+ matches, ms, nextToUpdate3, ip, iHighLimit, \
+ rep, ll0, lengthToBeat, ZSTD_##dictMode, mls); \
+ }
+
+#define GEN_ZSTD_BT_GET_ALL_MATCHES(dictMode) \
+ GEN_ZSTD_BT_GET_ALL_MATCHES_(dictMode, 3) \
+ GEN_ZSTD_BT_GET_ALL_MATCHES_(dictMode, 4) \
+ GEN_ZSTD_BT_GET_ALL_MATCHES_(dictMode, 5) \
+ GEN_ZSTD_BT_GET_ALL_MATCHES_(dictMode, 6)
+
+GEN_ZSTD_BT_GET_ALL_MATCHES(noDict)
+GEN_ZSTD_BT_GET_ALL_MATCHES(extDict)
+GEN_ZSTD_BT_GET_ALL_MATCHES(dictMatchState)
+
+#define ZSTD_BT_GET_ALL_MATCHES_ARRAY(dictMode) \
+ { \
+ ZSTD_BT_GET_ALL_MATCHES_FN(dictMode, 3), \
+ ZSTD_BT_GET_ALL_MATCHES_FN(dictMode, 4), \
+ ZSTD_BT_GET_ALL_MATCHES_FN(dictMode, 5), \
+ ZSTD_BT_GET_ALL_MATCHES_FN(dictMode, 6) \
}
+
+static ZSTD_getAllMatchesFn
+ZSTD_selectBtGetAllMatches(ZSTD_matchState_t const* ms, ZSTD_dictMode_e const dictMode)
+{
+ ZSTD_getAllMatchesFn const getAllMatchesFns[3][4] = {
+ ZSTD_BT_GET_ALL_MATCHES_ARRAY(noDict),
+ ZSTD_BT_GET_ALL_MATCHES_ARRAY(extDict),
+ ZSTD_BT_GET_ALL_MATCHES_ARRAY(dictMatchState)
+ };
+ U32 const mls = BOUNDED(3, ms->cParams.minMatch, 6);
+ assert((U32)dictMode < 3);
+ assert(mls - 3 < 4);
+ return getAllMatchesFns[(int)dictMode][mls - 3];
}
/* ***********************
/* Struct containing info needed to make decision about ldm inclusion */
typedef struct {
- rawSeqStore_t seqStore; /* External match candidates store for this block */
- U32 startPosInBlock; /* Start position of the current match candidate */
- U32 endPosInBlock; /* End position of the current match candidate */
- U32 offset; /* Offset of the match candidate */
+ rawSeqStore_t seqStore; /* External match candidates store for this block */
+ U32 startPosInBlock; /* Start position of the current match candidate */
+ U32 endPosInBlock; /* End position of the current match candidate */
+ U32 offset; /* Offset of the match candidate */
} ZSTD_optLdm_t;
/* ZSTD_optLdm_skipRawSeqStoreBytes():
- * Moves forward in rawSeqStore by nbBytes, which will update the fields 'pos' and 'posInSequence'.
+ * Moves forward in @rawSeqStore by @nbBytes,
+ * which will update the fields 'pos' and 'posInSequence'.
*/
-static void ZSTD_optLdm_skipRawSeqStoreBytes(rawSeqStore_t* rawSeqStore, size_t nbBytes) {
+static void ZSTD_optLdm_skipRawSeqStoreBytes(rawSeqStore_t* rawSeqStore, size_t nbBytes)
+{
U32 currPos = (U32)(rawSeqStore->posInSequence + nbBytes);
while (currPos && rawSeqStore->pos < rawSeqStore->size) {
rawSeq currSeq = rawSeqStore->seq[rawSeqStore->pos];
* Calculates the beginning and end of the next match in the current block.
* Updates 'pos' and 'posInSequence' of the ldmSeqStore.
*/
-static void ZSTD_opt_getNextMatchAndUpdateSeqStore(ZSTD_optLdm_t* optLdm, U32 currPosInBlock,
- U32 blockBytesRemaining) {
+static void
+ZSTD_opt_getNextMatchAndUpdateSeqStore(ZSTD_optLdm_t* optLdm, U32 currPosInBlock,
+ U32 blockBytesRemaining)
+{
rawSeq currSeq;
U32 currBlockEndPos;
U32 literalsBytesRemaining;
optLdm->endPosInBlock = UINT_MAX;
return;
}
- /* Calculate appropriate bytes left in matchLength and litLength after adjusting
- based on ldmSeqStore->posInSequence */
+ /* Calculate appropriate bytes left in matchLength and litLength
+ * after adjusting based on ldmSeqStore->posInSequence */
currSeq = optLdm->seqStore.seq[optLdm->seqStore.pos];
assert(optLdm->seqStore.posInSequence <= currSeq.litLength + currSeq.matchLength);
currBlockEndPos = currPosInBlock + blockBytesRemaining;
}
/* ZSTD_optLdm_maybeAddMatch():
- * Adds a match if it's long enough, based on it's 'matchStartPosInBlock'
- * and 'matchEndPosInBlock', into 'matches'. Maintains the correct ordering of 'matches'
+ * Adds a match if it's long enough,
+ * based on it's 'matchStartPosInBlock' and 'matchEndPosInBlock',
+ * into 'matches'. Maintains the correct ordering of 'matches'.
*/
static void ZSTD_optLdm_maybeAddMatch(ZSTD_match_t* matches, U32* nbMatches,
- ZSTD_optLdm_t* optLdm, U32 currPosInBlock) {
- U32 posDiff = currPosInBlock - optLdm->startPosInBlock;
+ const ZSTD_optLdm_t* optLdm, U32 currPosInBlock)
+{
+ U32 const posDiff = currPosInBlock - optLdm->startPosInBlock;
/* Note: ZSTD_match_t actually contains offCode and matchLength (before subtracting MINMATCH) */
- U32 candidateMatchLength = optLdm->endPosInBlock - optLdm->startPosInBlock - posDiff;
- U32 candidateOffCode = optLdm->offset + ZSTD_REP_MOVE;
+ U32 const candidateMatchLength = optLdm->endPosInBlock - optLdm->startPosInBlock - posDiff;
/* Ensure that current block position is not outside of the match */
if (currPosInBlock < optLdm->startPosInBlock
}
if (*nbMatches == 0 || ((candidateMatchLength > matches[*nbMatches-1].len) && *nbMatches < ZSTD_OPT_NUM)) {
+ U32 const candidateOffCode = STORE_OFFSET(optLdm->offset);
DEBUGLOG(6, "ZSTD_optLdm_maybeAddMatch(): Adding ldm candidate match (offCode: %u matchLength %u) at block position=%u",
candidateOffCode, candidateMatchLength, currPosInBlock);
matches[*nbMatches].len = candidateMatchLength;
/* ZSTD_optLdm_processMatchCandidate():
* Wrapper function to update ldm seq store and call ldm functions as necessary.
*/
-static void ZSTD_optLdm_processMatchCandidate(ZSTD_optLdm_t* optLdm, ZSTD_match_t* matches, U32* nbMatches,
- U32 currPosInBlock, U32 remainingBytes) {
+static void
+ZSTD_optLdm_processMatchCandidate(ZSTD_optLdm_t* optLdm,
+ ZSTD_match_t* matches, U32* nbMatches,
+ U32 currPosInBlock, U32 remainingBytes)
+{
if (optLdm->seqStore.size == 0 || optLdm->seqStore.pos >= optLdm->seqStore.size) {
return;
}
* at the end of a match from the ldm seq store, and will often be some bytes
* over beyond matchEndPosInBlock. As such, we need to correct for these "overshoots"
*/
- U32 posOvershoot = currPosInBlock - optLdm->endPosInBlock;
+ U32 const posOvershoot = currPosInBlock - optLdm->endPosInBlock;
ZSTD_optLdm_skipRawSeqStoreBytes(&optLdm->seqStore, posOvershoot);
- }
+ }
ZSTD_opt_getNextMatchAndUpdateSeqStore(optLdm, currPosInBlock, remainingBytes);
}
ZSTD_optLdm_maybeAddMatch(matches, nbMatches, optLdm, currPosInBlock);
}
+
/*-*******************************
* Optimal parser
*********************************/
-
static U32 ZSTD_totalLen(ZSTD_optimal_t sol)
{
return sol.litlen + sol.mlen;
const BYTE* const prefixStart = base + ms->window.dictLimit;
const ZSTD_compressionParameters* const cParams = &ms->cParams;
+ ZSTD_getAllMatchesFn getAllMatches = ZSTD_selectBtGetAllMatches(ms, dictMode);
+
U32 const sufficient_len = MIN(cParams->targetLength, ZSTD_OPT_NUM -1);
U32 const minMatch = (cParams->minMatch == 3) ? 3 : 4;
U32 nextToUpdate3 = ms->nextToUpdate;
/* find first match */
{ U32 const litlen = (U32)(ip - anchor);
U32 const ll0 = !litlen;
- U32 nbMatches = ZSTD_BtGetAllMatches(matches, ms, &nextToUpdate3, ip, iend, dictMode, rep, ll0, minMatch);
+ U32 nbMatches = getAllMatches(matches, ms, &nextToUpdate3, ip, iend, rep, ll0, minMatch);
ZSTD_optLdm_processMatchCandidate(&optLdm, matches, &nbMatches,
(U32)(ip-istart), (U32)(iend - ip));
if (!nbMatches) { ip++; continue; }
* in every price. We include the literal length to avoid negative
* prices when we subtract the previous literal length.
*/
- opt[0].price = ZSTD_litLengthPrice(litlen, optStatePtr, optLevel);
+ opt[0].price = (int)ZSTD_litLengthPrice(litlen, optStatePtr, optLevel);
/* large match -> immediate encoding */
{ U32 const maxML = matches[nbMatches-1].len;
- U32 const maxOffset = matches[nbMatches-1].off;
+ U32 const maxOffcode = matches[nbMatches-1].off;
DEBUGLOG(6, "found %u matches of maxLength=%u and maxOffCode=%u at cPos=%u => start new series",
- nbMatches, maxML, maxOffset, (U32)(ip-prefixStart));
+ nbMatches, maxML, maxOffcode, (U32)(ip-prefixStart));
if (maxML > sufficient_len) {
lastSequence.litlen = litlen;
lastSequence.mlen = maxML;
- lastSequence.off = maxOffset;
+ lastSequence.off = maxOffcode;
DEBUGLOG(6, "large match (%u>%u), immediate encoding",
maxML, sufficient_len);
cur = 0;
} }
/* set prices for first matches starting position == 0 */
- { U32 const literalsPrice = opt[0].price + ZSTD_litLengthPrice(0, optStatePtr, optLevel);
+ assert(opt[0].price >= 0);
+ { U32 const literalsPrice = (U32)opt[0].price + ZSTD_litLengthPrice(0, optStatePtr, optLevel);
U32 pos;
U32 matchNb;
for (pos = 1; pos < minMatch; pos++) {
opt[pos].price = ZSTD_MAX_PRICE; /* mlen, litlen and price will be fixed during forward scanning */
}
for (matchNb = 0; matchNb < nbMatches; matchNb++) {
- U32 const offset = matches[matchNb].off;
+ U32 const offcode = matches[matchNb].off;
U32 const end = matches[matchNb].len;
for ( ; pos <= end ; pos++ ) {
- U32 const matchPrice = ZSTD_getMatchPrice(offset, pos, optStatePtr, optLevel);
+ U32 const matchPrice = ZSTD_getMatchPrice(offcode, pos, optStatePtr, optLevel);
U32 const sequencePrice = literalsPrice + matchPrice;
DEBUGLOG(7, "rPos:%u => set initial price : %.2f",
pos, ZSTD_fCost(sequencePrice));
opt[pos].mlen = pos;
- opt[pos].off = offset;
+ opt[pos].off = offcode;
opt[pos].litlen = litlen;
- opt[pos].price = sequencePrice;
+ opt[pos].price = (int)sequencePrice;
} }
last_pos = pos-1;
}
/* Fix current position with one literal if cheaper */
{ U32 const litlen = (opt[cur-1].mlen == 0) ? opt[cur-1].litlen + 1 : 1;
int const price = opt[cur-1].price
- + ZSTD_rawLiteralsCost(ip+cur-1, 1, optStatePtr, optLevel)
- + ZSTD_litLengthPrice(litlen, optStatePtr, optLevel)
- - ZSTD_litLengthPrice(litlen-1, optStatePtr, optLevel);
+ + (int)ZSTD_rawLiteralsCost(ip+cur-1, 1, optStatePtr, optLevel)
+ + (int)ZSTD_litLengthPrice(litlen, optStatePtr, optLevel)
+ - (int)ZSTD_litLengthPrice(litlen-1, optStatePtr, optLevel);
assert(price < 1000000000); /* overflow check */
if (price <= opt[cur].price) {
DEBUGLOG(7, "cPos:%zi==rPos:%u : better price (%.2f<=%.2f) using literal (ll==%u) (hist:%u,%u,%u)",
assert(cur >= opt[cur].mlen);
if (opt[cur].mlen != 0) {
U32 const prev = cur - opt[cur].mlen;
- repcodes_t newReps = ZSTD_updateRep(opt[prev].rep, opt[cur].off, opt[cur].litlen==0);
+ repcodes_t const newReps = ZSTD_newRep(opt[prev].rep, opt[cur].off, opt[cur].litlen==0);
ZSTD_memcpy(opt[cur].rep, &newReps, sizeof(repcodes_t));
} else {
ZSTD_memcpy(opt[cur].rep, opt[cur - 1].rep, sizeof(repcodes_t));
continue; /* skip unpromising positions; about ~+6% speed, -0.01 ratio */
}
+ assert(opt[cur].price >= 0);
{ U32 const ll0 = (opt[cur].mlen != 0);
U32 const litlen = (opt[cur].mlen == 0) ? opt[cur].litlen : 0;
- U32 const previousPrice = opt[cur].price;
+ U32 const previousPrice = (U32)opt[cur].price;
U32 const basePrice = previousPrice + ZSTD_litLengthPrice(0, optStatePtr, optLevel);
- U32 nbMatches = ZSTD_BtGetAllMatches(matches, ms, &nextToUpdate3, inr, iend, dictMode, opt[cur].rep, ll0, minMatch);
+ U32 nbMatches = getAllMatches(matches, ms, &nextToUpdate3, inr, iend, opt[cur].rep, ll0, minMatch);
U32 matchNb;
ZSTD_optLdm_processMatchCandidate(&optLdm, matches, &nbMatches,
for (mlen = lastML; mlen >= startML; mlen--) { /* scan downward */
U32 const pos = cur + mlen;
- int const price = basePrice + ZSTD_getMatchPrice(offset, mlen, optStatePtr, optLevel);
+ int const price = (int)basePrice + (int)ZSTD_getMatchPrice(offset, mlen, optStatePtr, optLevel);
if ((pos > last_pos) || (price < opt[pos].price)) {
DEBUGLOG(7, "rPos:%u (ml=%2u) => new better price (%.2f<%.2f)",
* update them while traversing the sequences.
*/
if (lastSequence.mlen != 0) {
- repcodes_t reps = ZSTD_updateRep(opt[cur].rep, lastSequence.off, lastSequence.litlen==0);
+ repcodes_t const reps = ZSTD_newRep(opt[cur].rep, lastSequence.off, lastSequence.litlen==0);
ZSTD_memcpy(rep, &reps, sizeof(reps));
} else {
ZSTD_memcpy(rep, opt[cur].rep, sizeof(repcodes_t));
assert(anchor + llen <= iend);
ZSTD_updateStats(optStatePtr, llen, anchor, offCode, mlen);
- ZSTD_storeSeq(seqStore, llen, anchor, iend, offCode, mlen-MINMATCH);
+ ZSTD_storeSeq(seqStore, llen, anchor, iend, offCode, mlen);
anchor += advance;
ip = anchor;
} }
return (size_t)(iend - anchor);
}
+static size_t ZSTD_compressBlock_opt0(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ const void* src, size_t srcSize, const ZSTD_dictMode_e dictMode)
+{
+ return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 0 /* optLevel */, dictMode);
+}
+
+static size_t ZSTD_compressBlock_opt2(
+ ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
+ const void* src, size_t srcSize, const ZSTD_dictMode_e dictMode)
+{
+ return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /* optLevel */, dictMode);
+}
size_t ZSTD_compressBlock_btopt(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
const void* src, size_t srcSize)
{
DEBUGLOG(5, "ZSTD_compressBlock_btopt");
- return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 0 /*optLevel*/, ZSTD_noDict);
+ return ZSTD_compressBlock_opt0(ms, seqStore, rep, src, srcSize, ZSTD_noDict);
}
-/* used in 2-pass strategy */
-static U32 ZSTD_upscaleStat(unsigned* table, U32 lastEltIndex, int bonus)
-{
- U32 s, sum=0;
- assert(ZSTD_FREQ_DIV+bonus >= 0);
- for (s=0; s<lastEltIndex+1; s++) {
- table[s] <<= ZSTD_FREQ_DIV+bonus;
- table[s]--;
- sum += table[s];
- }
- return sum;
-}
-/* used in 2-pass strategy */
-MEM_STATIC void ZSTD_upscaleStats(optState_t* optPtr)
-{
- if (ZSTD_compressedLiterals(optPtr))
- optPtr->litSum = ZSTD_upscaleStat(optPtr->litFreq, MaxLit, 0);
- optPtr->litLengthSum = ZSTD_upscaleStat(optPtr->litLengthFreq, MaxLL, 0);
- optPtr->matchLengthSum = ZSTD_upscaleStat(optPtr->matchLengthFreq, MaxML, 0);
- optPtr->offCodeSum = ZSTD_upscaleStat(optPtr->offCodeFreq, MaxOff, 0);
-}
/* ZSTD_initStats_ultra():
* make a first compression pass, just to seed stats with more accurate starting values.
assert(ms->window.dictLimit == ms->window.lowLimit); /* no dictionary */
assert(ms->window.dictLimit - ms->nextToUpdate <= 1); /* no prefix (note: intentional overflow, defined as 2-complement) */
- ZSTD_compressBlock_opt_generic(ms, seqStore, tmpRep, src, srcSize, 2 /*optLevel*/, ZSTD_noDict); /* generate stats into ms->opt*/
+ ZSTD_compressBlock_opt2(ms, seqStore, tmpRep, src, srcSize, ZSTD_noDict); /* generate stats into ms->opt*/
/* invalidate first scan from history */
ZSTD_resetSeqStore(seqStore);
ms->window.lowLimit = ms->window.dictLimit;
ms->nextToUpdate = ms->window.dictLimit;
- /* re-inforce weight of collected statistics */
- ZSTD_upscaleStats(&ms->opt);
}
size_t ZSTD_compressBlock_btultra(
const void* src, size_t srcSize)
{
DEBUGLOG(5, "ZSTD_compressBlock_btultra (srcSize=%zu)", srcSize);
- return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_noDict);
+ return ZSTD_compressBlock_opt2(ms, seqStore, rep, src, srcSize, ZSTD_noDict);
}
size_t ZSTD_compressBlock_btultra2(
ZSTD_initStats_ultra(ms, seqStore, rep, src, srcSize);
}
- return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_noDict);
+ return ZSTD_compressBlock_opt2(ms, seqStore, rep, src, srcSize, ZSTD_noDict);
}
size_t ZSTD_compressBlock_btopt_dictMatchState(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
const void* src, size_t srcSize)
{
- return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 0 /*optLevel*/, ZSTD_dictMatchState);
+ return ZSTD_compressBlock_opt0(ms, seqStore, rep, src, srcSize, ZSTD_dictMatchState);
}
size_t ZSTD_compressBlock_btultra_dictMatchState(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
const void* src, size_t srcSize)
{
- return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_dictMatchState);
+ return ZSTD_compressBlock_opt2(ms, seqStore, rep, src, srcSize, ZSTD_dictMatchState);
}
size_t ZSTD_compressBlock_btopt_extDict(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
const void* src, size_t srcSize)
{
- return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 0 /*optLevel*/, ZSTD_extDict);
+ return ZSTD_compressBlock_opt0(ms, seqStore, rep, src, srcSize, ZSTD_extDict);
}
size_t ZSTD_compressBlock_btultra_extDict(
ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
const void* src, size_t srcSize)
{
- return ZSTD_compressBlock_opt_generic(ms, seqStore, rep, src, srcSize, 2 /*optLevel*/, ZSTD_extDict);
+ return ZSTD_compressBlock_opt2(ms, seqStore, rep, src, srcSize, ZSTD_extDict);
}
/* note : no btultra2 variant for extDict nor dictMatchState,
#define HUF_STATIC_LINKING_ONLY
#include "../common/huf.h"
#include "../common/error_private.h"
+#include "../common/zstd_internal.h"
+
+/* **************************************************************
+* Constants
+****************************************************************/
+
+#define HUF_DECODER_FAST_TABLELOG 11
/* **************************************************************
* Macros
#error "Cannot force the use of the X1 and X2 decoders at the same time!"
#endif
+#if ZSTD_ENABLE_ASM_X86_64_BMI2 && DYNAMIC_BMI2
+# define HUF_ASM_X86_64_BMI2_ATTRS BMI2_TARGET_ATTRIBUTE
+#else
+# define HUF_ASM_X86_64_BMI2_ATTRS
+#endif
+
+#define HUF_EXTERN_C
+#define HUF_ASM_DECL HUF_EXTERN_C
+
+#if DYNAMIC_BMI2 || (ZSTD_ENABLE_ASM_X86_64_BMI2 && defined(__BMI2__))
+# define HUF_NEED_BMI2_FUNCTION 1
+#else
+# define HUF_NEED_BMI2_FUNCTION 0
+#endif
+
+#if !(ZSTD_ENABLE_ASM_X86_64_BMI2 && defined(__BMI2__))
+# define HUF_NEED_DEFAULT_FUNCTION 1
+#else
+# define HUF_NEED_DEFAULT_FUNCTION 0
+#endif
/* **************************************************************
* Error Management
return fn##_body(dst, dstSize, cSrc, cSrcSize, DTable); \
} \
\
- static TARGET_ATTRIBUTE("bmi2") size_t fn##_bmi2( \
+ static BMI2_TARGET_ATTRIBUTE size_t fn##_bmi2( \
void* dst, size_t dstSize, \
const void* cSrc, size_t cSrcSize, \
const HUF_DTable* DTable) \
return dtd;
}
+#if ZSTD_ENABLE_ASM_X86_64_BMI2
+
+static size_t HUF_initDStream(BYTE const* ip) {
+ BYTE const lastByte = ip[7];
+ size_t const bitsConsumed = lastByte ? 8 - BIT_highbit32(lastByte) : 0;
+ size_t const value = MEM_readLEST(ip) | 1;
+ assert(bitsConsumed <= 8);
+ return value << bitsConsumed;
+}
+typedef struct {
+ BYTE const* ip[4];
+ BYTE* op[4];
+ U64 bits[4];
+ void const* dt;
+ BYTE const* ilimit;
+ BYTE* oend;
+ BYTE const* iend[4];
+} HUF_DecompressAsmArgs;
+
+/*
+ * Initializes args for the asm decoding loop.
+ * @returns 0 on success
+ * 1 if the fallback implementation should be used.
+ * Or an error code on failure.
+ */
+static size_t HUF_DecompressAsmArgs_init(HUF_DecompressAsmArgs* args, void* dst, size_t dstSize, void const* src, size_t srcSize, const HUF_DTable* DTable)
+{
+ void const* dt = DTable + 1;
+ U32 const dtLog = HUF_getDTableDesc(DTable).tableLog;
+
+ const BYTE* const ilimit = (const BYTE*)src + 6 + 8;
+
+ BYTE* const oend = (BYTE*)dst + dstSize;
+
+ /* The following condition is false on x32 platform,
+ * but HUF_asm is not compatible with this ABI */
+ if (!(MEM_isLittleEndian() && !MEM_32bits())) return 1;
+
+ /* strict minimum : jump table + 1 byte per stream */
+ if (srcSize < 10)
+ return ERROR(corruption_detected);
+
+ /* Must have at least 8 bytes per stream because we don't handle initializing smaller bit containers.
+ * If table log is not correct at this point, fallback to the old decoder.
+ * On small inputs we don't have enough data to trigger the fast loop, so use the old decoder.
+ */
+ if (dtLog != HUF_DECODER_FAST_TABLELOG)
+ return 1;
+
+ /* Read the jump table. */
+ {
+ const BYTE* const istart = (const BYTE*)src;
+ size_t const length1 = MEM_readLE16(istart);
+ size_t const length2 = MEM_readLE16(istart+2);
+ size_t const length3 = MEM_readLE16(istart+4);
+ size_t const length4 = srcSize - (length1 + length2 + length3 + 6);
+ args->iend[0] = istart + 6; /* jumpTable */
+ args->iend[1] = args->iend[0] + length1;
+ args->iend[2] = args->iend[1] + length2;
+ args->iend[3] = args->iend[2] + length3;
+
+ /* HUF_initDStream() requires this, and this small of an input
+ * won't benefit from the ASM loop anyways.
+ * length1 must be >= 16 so that ip[0] >= ilimit before the loop
+ * starts.
+ */
+ if (length1 < 16 || length2 < 8 || length3 < 8 || length4 < 8)
+ return 1;
+ if (length4 > srcSize) return ERROR(corruption_detected); /* overflow */
+ }
+ /* ip[] contains the position that is currently loaded into bits[]. */
+ args->ip[0] = args->iend[1] - sizeof(U64);
+ args->ip[1] = args->iend[2] - sizeof(U64);
+ args->ip[2] = args->iend[3] - sizeof(U64);
+ args->ip[3] = (BYTE const*)src + srcSize - sizeof(U64);
+
+ /* op[] contains the output pointers. */
+ args->op[0] = (BYTE*)dst;
+ args->op[1] = args->op[0] + (dstSize+3)/4;
+ args->op[2] = args->op[1] + (dstSize+3)/4;
+ args->op[3] = args->op[2] + (dstSize+3)/4;
+
+ /* No point to call the ASM loop for tiny outputs. */
+ if (args->op[3] >= oend)
+ return 1;
+
+ /* bits[] is the bit container.
+ * It is read from the MSB down to the LSB.
+ * It is shifted left as it is read, and zeros are
+ * shifted in. After the lowest valid bit a 1 is
+ * set, so that CountTrailingZeros(bits[]) can be used
+ * to count how many bits we've consumed.
+ */
+ args->bits[0] = HUF_initDStream(args->ip[0]);
+ args->bits[1] = HUF_initDStream(args->ip[1]);
+ args->bits[2] = HUF_initDStream(args->ip[2]);
+ args->bits[3] = HUF_initDStream(args->ip[3]);
+
+ /* If ip[] >= ilimit, it is guaranteed to be safe to
+ * reload bits[]. It may be beyond its section, but is
+ * guaranteed to be valid (>= istart).
+ */
+ args->ilimit = ilimit;
+
+ args->oend = oend;
+ args->dt = dt;
+
+ return 0;
+}
+
+static size_t HUF_initRemainingDStream(BIT_DStream_t* bit, HUF_DecompressAsmArgs const* args, int stream, BYTE* segmentEnd)
+{
+ /* Validate that we haven't overwritten. */
+ if (args->op[stream] > segmentEnd)
+ return ERROR(corruption_detected);
+ /* Validate that we haven't read beyond iend[].
+ * Note that ip[] may be < iend[] because the MSB is
+ * the next bit to read, and we may have consumed 100%
+ * of the stream, so down to iend[i] - 8 is valid.
+ */
+ if (args->ip[stream] < args->iend[stream] - 8)
+ return ERROR(corruption_detected);
+
+ /* Construct the BIT_DStream_t. */
+ bit->bitContainer = MEM_readLE64(args->ip[stream]);
+ bit->bitsConsumed = ZSTD_countTrailingZeros((size_t)args->bits[stream]);
+ bit->start = (const char*)args->iend[0];
+ bit->limitPtr = bit->start + sizeof(size_t);
+ bit->ptr = (const char*)args->ip[stream];
+
+ return 0;
+}
+#endif
+
#ifndef HUF_FORCE_DECOMPRESS_X2
/*-***************************/
/* single-symbol decoding */
/*-***************************/
-typedef struct { BYTE byte; BYTE nbBits; } HUF_DEltX1; /* single-symbol decoding */
+typedef struct { BYTE nbBits; BYTE byte; } HUF_DEltX1; /* single-symbol decoding */
/*
* Packs 4 HUF_DEltX1 structs into a U64. This is used to lay down 4 entries at
static U64 HUF_DEltX1_set4(BYTE symbol, BYTE nbBits) {
U64 D4;
if (MEM_isLittleEndian()) {
- D4 = symbol + (nbBits << 8);
- } else {
D4 = (symbol << 8) + nbBits;
+ } else {
+ D4 = symbol + (nbBits << 8);
}
D4 *= 0x0001000100010001ULL;
return D4;
}
+/*
+ * Increase the tableLog to targetTableLog and rescales the stats.
+ * If tableLog > targetTableLog this is a no-op.
+ * @returns New tableLog
+ */
+static U32 HUF_rescaleStats(BYTE* huffWeight, U32* rankVal, U32 nbSymbols, U32 tableLog, U32 targetTableLog)
+{
+ if (tableLog > targetTableLog)
+ return tableLog;
+ if (tableLog < targetTableLog) {
+ U32 const scale = targetTableLog - tableLog;
+ U32 s;
+ /* Increase the weight for all non-zero probability symbols by scale. */
+ for (s = 0; s < nbSymbols; ++s) {
+ huffWeight[s] += (BYTE)((huffWeight[s] == 0) ? 0 : scale);
+ }
+ /* Update rankVal to reflect the new weights.
+ * All weights except 0 get moved to weight + scale.
+ * Weights [1, scale] are empty.
+ */
+ for (s = targetTableLog; s > scale; --s) {
+ rankVal[s] = rankVal[s - scale];
+ }
+ for (s = scale; s > 0; --s) {
+ rankVal[s] = 0;
+ }
+ }
+ return targetTableLog;
+}
+
typedef struct {
U32 rankVal[HUF_TABLELOG_ABSOLUTEMAX + 1];
U32 rankStart[HUF_TABLELOG_ABSOLUTEMAX + 1];
iSize = HUF_readStats_wksp(wksp->huffWeight, HUF_SYMBOLVALUE_MAX + 1, wksp->rankVal, &nbSymbols, &tableLog, src, srcSize, wksp->statsWksp, sizeof(wksp->statsWksp), bmi2);
if (HUF_isError(iSize)) return iSize;
+
/* Table header */
{ DTableDesc dtd = HUF_getDTableDesc(DTable);
+ U32 const maxTableLog = dtd.maxTableLog + 1;
+ U32 const targetTableLog = MIN(maxTableLog, HUF_DECODER_FAST_TABLELOG);
+ tableLog = HUF_rescaleStats(wksp->huffWeight, wksp->rankVal, nbSymbols, tableLog, targetTableLog);
if (tableLog > (U32)(dtd.maxTableLog+1)) return ERROR(tableLog_tooLarge); /* DTable too small, Huffman tree cannot fit in */
dtd.tableType = 0;
dtd.tableLog = (BYTE)tableLog;
/* fill DTable
* We fill all entries of each weight in order.
- * That way length is a constant for each iteration of the outter loop.
+ * That way length is a constant for each iteration of the outer loop.
* We can switch based on the length to a different inner loop which is
* optimized for that particular case.
*/
BYTE* const pStart = p;
/* up to 4 symbols at a time */
- while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd-3)) {
- HUF_DECODE_SYMBOLX1_2(p, bitDPtr);
- HUF_DECODE_SYMBOLX1_1(p, bitDPtr);
- HUF_DECODE_SYMBOLX1_2(p, bitDPtr);
- HUF_DECODE_SYMBOLX1_0(p, bitDPtr);
+ if ((pEnd - p) > 3) {
+ while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd-3)) {
+ HUF_DECODE_SYMBOLX1_2(p, bitDPtr);
+ HUF_DECODE_SYMBOLX1_1(p, bitDPtr);
+ HUF_DECODE_SYMBOLX1_2(p, bitDPtr);
+ HUF_DECODE_SYMBOLX1_0(p, bitDPtr);
+ }
+ } else {
+ BIT_reloadDStream(bitDPtr);
}
/* [0-3] symbols remaining */
U32 endSignal = 1;
if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */
+ if (opStart4 > oend) return ERROR(corruption_detected); /* overflow */
CHECK_F( BIT_initDStream(&bitD1, istart1, length1) );
CHECK_F( BIT_initDStream(&bitD2, istart2, length2) );
CHECK_F( BIT_initDStream(&bitD3, istart3, length3) );
CHECK_F( BIT_initDStream(&bitD4, istart4, length4) );
/* up to 16 symbols per loop (4 symbols per stream) in 64-bit mode */
- for ( ; (endSignal) & (op4 < olimit) ; ) {
- HUF_DECODE_SYMBOLX1_2(op1, &bitD1);
- HUF_DECODE_SYMBOLX1_2(op2, &bitD2);
- HUF_DECODE_SYMBOLX1_2(op3, &bitD3);
- HUF_DECODE_SYMBOLX1_2(op4, &bitD4);
- HUF_DECODE_SYMBOLX1_1(op1, &bitD1);
- HUF_DECODE_SYMBOLX1_1(op2, &bitD2);
- HUF_DECODE_SYMBOLX1_1(op3, &bitD3);
- HUF_DECODE_SYMBOLX1_1(op4, &bitD4);
- HUF_DECODE_SYMBOLX1_2(op1, &bitD1);
- HUF_DECODE_SYMBOLX1_2(op2, &bitD2);
- HUF_DECODE_SYMBOLX1_2(op3, &bitD3);
- HUF_DECODE_SYMBOLX1_2(op4, &bitD4);
- HUF_DECODE_SYMBOLX1_0(op1, &bitD1);
- HUF_DECODE_SYMBOLX1_0(op2, &bitD2);
- HUF_DECODE_SYMBOLX1_0(op3, &bitD3);
- HUF_DECODE_SYMBOLX1_0(op4, &bitD4);
- endSignal &= BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished;
- endSignal &= BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished;
- endSignal &= BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished;
- endSignal &= BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished;
+ if ((size_t)(oend - op4) >= sizeof(size_t)) {
+ for ( ; (endSignal) & (op4 < olimit) ; ) {
+ HUF_DECODE_SYMBOLX1_2(op1, &bitD1);
+ HUF_DECODE_SYMBOLX1_2(op2, &bitD2);
+ HUF_DECODE_SYMBOLX1_2(op3, &bitD3);
+ HUF_DECODE_SYMBOLX1_2(op4, &bitD4);
+ HUF_DECODE_SYMBOLX1_1(op1, &bitD1);
+ HUF_DECODE_SYMBOLX1_1(op2, &bitD2);
+ HUF_DECODE_SYMBOLX1_1(op3, &bitD3);
+ HUF_DECODE_SYMBOLX1_1(op4, &bitD4);
+ HUF_DECODE_SYMBOLX1_2(op1, &bitD1);
+ HUF_DECODE_SYMBOLX1_2(op2, &bitD2);
+ HUF_DECODE_SYMBOLX1_2(op3, &bitD3);
+ HUF_DECODE_SYMBOLX1_2(op4, &bitD4);
+ HUF_DECODE_SYMBOLX1_0(op1, &bitD1);
+ HUF_DECODE_SYMBOLX1_0(op2, &bitD2);
+ HUF_DECODE_SYMBOLX1_0(op3, &bitD3);
+ HUF_DECODE_SYMBOLX1_0(op4, &bitD4);
+ endSignal &= BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished;
+ endSignal &= BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished;
+ endSignal &= BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished;
+ endSignal &= BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished;
+ }
}
/* check corruption */
}
}
+#if HUF_NEED_BMI2_FUNCTION
+static BMI2_TARGET_ATTRIBUTE
+size_t HUF_decompress4X1_usingDTable_internal_bmi2(void* dst, size_t dstSize, void const* cSrc,
+ size_t cSrcSize, HUF_DTable const* DTable) {
+ return HUF_decompress4X1_usingDTable_internal_body(dst, dstSize, cSrc, cSrcSize, DTable);
+}
+#endif
+
+#if HUF_NEED_DEFAULT_FUNCTION
+static
+size_t HUF_decompress4X1_usingDTable_internal_default(void* dst, size_t dstSize, void const* cSrc,
+ size_t cSrcSize, HUF_DTable const* DTable) {
+ return HUF_decompress4X1_usingDTable_internal_body(dst, dstSize, cSrc, cSrcSize, DTable);
+}
+#endif
+
+#if ZSTD_ENABLE_ASM_X86_64_BMI2
+
+HUF_ASM_DECL void HUF_decompress4X1_usingDTable_internal_bmi2_asm_loop(HUF_DecompressAsmArgs* args) ZSTDLIB_HIDDEN;
+
+static HUF_ASM_X86_64_BMI2_ATTRS
+size_t
+HUF_decompress4X1_usingDTable_internal_bmi2_asm(
+ void* dst, size_t dstSize,
+ const void* cSrc, size_t cSrcSize,
+ const HUF_DTable* DTable)
+{
+ void const* dt = DTable + 1;
+ const BYTE* const iend = (const BYTE*)cSrc + 6;
+ BYTE* const oend = (BYTE*)dst + dstSize;
+ HUF_DecompressAsmArgs args;
+ {
+ size_t const ret = HUF_DecompressAsmArgs_init(&args, dst, dstSize, cSrc, cSrcSize, DTable);
+ FORWARD_IF_ERROR(ret, "Failed to init asm args");
+ if (ret != 0)
+ return HUF_decompress4X1_usingDTable_internal_bmi2(dst, dstSize, cSrc, cSrcSize, DTable);
+ }
+
+ assert(args.ip[0] >= args.ilimit);
+ HUF_decompress4X1_usingDTable_internal_bmi2_asm_loop(&args);
+
+ /* Our loop guarantees that ip[] >= ilimit and that we haven't
+ * overwritten any op[].
+ */
+ assert(args.ip[0] >= iend);
+ assert(args.ip[1] >= iend);
+ assert(args.ip[2] >= iend);
+ assert(args.ip[3] >= iend);
+ assert(args.op[3] <= oend);
+ (void)iend;
+
+ /* finish bit streams one by one. */
+ {
+ size_t const segmentSize = (dstSize+3) / 4;
+ BYTE* segmentEnd = (BYTE*)dst;
+ int i;
+ for (i = 0; i < 4; ++i) {
+ BIT_DStream_t bit;
+ if (segmentSize <= (size_t)(oend - segmentEnd))
+ segmentEnd += segmentSize;
+ else
+ segmentEnd = oend;
+ FORWARD_IF_ERROR(HUF_initRemainingDStream(&bit, &args, i, segmentEnd), "corruption");
+ /* Decompress and validate that we've produced exactly the expected length. */
+ args.op[i] += HUF_decodeStreamX1(args.op[i], &bit, segmentEnd, (HUF_DEltX1 const*)dt, HUF_DECODER_FAST_TABLELOG);
+ if (args.op[i] != segmentEnd) return ERROR(corruption_detected);
+ }
+ }
+
+ /* decoded size */
+ return dstSize;
+}
+#endif /* ZSTD_ENABLE_ASM_X86_64_BMI2 */
typedef size_t (*HUF_decompress_usingDTable_t)(void *dst, size_t dstSize,
const void *cSrc,
const HUF_DTable *DTable);
HUF_DGEN(HUF_decompress1X1_usingDTable_internal)
-HUF_DGEN(HUF_decompress4X1_usingDTable_internal)
+static size_t HUF_decompress4X1_usingDTable_internal(void* dst, size_t dstSize, void const* cSrc,
+ size_t cSrcSize, HUF_DTable const* DTable, int bmi2)
+{
+#if DYNAMIC_BMI2
+ if (bmi2) {
+# if ZSTD_ENABLE_ASM_X86_64_BMI2
+ return HUF_decompress4X1_usingDTable_internal_bmi2_asm(dst, dstSize, cSrc, cSrcSize, DTable);
+# else
+ return HUF_decompress4X1_usingDTable_internal_bmi2(dst, dstSize, cSrc, cSrcSize, DTable);
+# endif
+ }
+#else
+ (void)bmi2;
+#endif
+
+#if ZSTD_ENABLE_ASM_X86_64_BMI2 && defined(__BMI2__)
+ return HUF_decompress4X1_usingDTable_internal_bmi2_asm(dst, dstSize, cSrc, cSrcSize, DTable);
+#else
+ return HUF_decompress4X1_usingDTable_internal_default(dst, dstSize, cSrc, cSrcSize, DTable);
+#endif
+}
size_t HUF_decompress1X1_usingDTable(
/* *************************/
typedef struct { U16 sequence; BYTE nbBits; BYTE length; } HUF_DEltX2; /* double-symbols decoding */
-typedef struct { BYTE symbol; BYTE weight; } sortedSymbol_t;
+typedef struct { BYTE symbol; } sortedSymbol_t;
typedef U32 rankValCol_t[HUF_TABLELOG_MAX + 1];
typedef rankValCol_t rankVal_t[HUF_TABLELOG_MAX];
+/*
+ * Constructs a HUF_DEltX2 in a U32.
+ */
+static U32 HUF_buildDEltX2U32(U32 symbol, U32 nbBits, U32 baseSeq, int level)
+{
+ U32 seq;
+ DEBUG_STATIC_ASSERT(offsetof(HUF_DEltX2, sequence) == 0);
+ DEBUG_STATIC_ASSERT(offsetof(HUF_DEltX2, nbBits) == 2);
+ DEBUG_STATIC_ASSERT(offsetof(HUF_DEltX2, length) == 3);
+ DEBUG_STATIC_ASSERT(sizeof(HUF_DEltX2) == sizeof(U32));
+ if (MEM_isLittleEndian()) {
+ seq = level == 1 ? symbol : (baseSeq + (symbol << 8));
+ return seq + (nbBits << 16) + ((U32)level << 24);
+ } else {
+ seq = level == 1 ? (symbol << 8) : ((baseSeq << 8) + symbol);
+ return (seq << 16) + (nbBits << 8) + (U32)level;
+ }
+}
-/* HUF_fillDTableX2Level2() :
- * `rankValOrigin` must be a table of at least (HUF_TABLELOG_MAX + 1) U32 */
-static void HUF_fillDTableX2Level2(HUF_DEltX2* DTable, U32 sizeLog, const U32 consumed,
- const U32* rankValOrigin, const int minWeight,
- const sortedSymbol_t* sortedSymbols, const U32 sortedListSize,
- U32 nbBitsBaseline, U16 baseSeq, U32* wksp, size_t wkspSize)
+/*
+ * Constructs a HUF_DEltX2.
+ */
+static HUF_DEltX2 HUF_buildDEltX2(U32 symbol, U32 nbBits, U32 baseSeq, int level)
{
HUF_DEltX2 DElt;
- U32* rankVal = wksp;
+ U32 const val = HUF_buildDEltX2U32(symbol, nbBits, baseSeq, level);
+ DEBUG_STATIC_ASSERT(sizeof(DElt) == sizeof(val));
+ ZSTD_memcpy(&DElt, &val, sizeof(val));
+ return DElt;
+}
+
+/*
+ * Constructs 2 HUF_DEltX2s and packs them into a U64.
+ */
+static U64 HUF_buildDEltX2U64(U32 symbol, U32 nbBits, U16 baseSeq, int level)
+{
+ U32 DElt = HUF_buildDEltX2U32(symbol, nbBits, baseSeq, level);
+ return (U64)DElt + ((U64)DElt << 32);
+}
- assert(wkspSize >= HUF_TABLELOG_MAX + 1);
- (void)wkspSize;
- /* get pre-calculated rankVal */
- ZSTD_memcpy(rankVal, rankValOrigin, sizeof(U32) * (HUF_TABLELOG_MAX + 1));
+/*
+ * Fills the DTable rank with all the symbols from [begin, end) that are each
+ * nbBits long.
+ *
+ * @param DTableRank The start of the rank in the DTable.
+ * @param begin The first symbol to fill (inclusive).
+ * @param end The last symbol to fill (exclusive).
+ * @param nbBits Each symbol is nbBits long.
+ * @param tableLog The table log.
+ * @param baseSeq If level == 1 { 0 } else { the first level symbol }
+ * @param level The level in the table. Must be 1 or 2.
+ */
+static void HUF_fillDTableX2ForWeight(
+ HUF_DEltX2* DTableRank,
+ sortedSymbol_t const* begin, sortedSymbol_t const* end,
+ U32 nbBits, U32 tableLog,
+ U16 baseSeq, int const level)
+{
+ U32 const length = 1U << ((tableLog - nbBits) & 0x1F /* quiet static-analyzer */);
+ const sortedSymbol_t* ptr;
+ assert(level >= 1 && level <= 2);
+ switch (length) {
+ case 1:
+ for (ptr = begin; ptr != end; ++ptr) {
+ HUF_DEltX2 const DElt = HUF_buildDEltX2(ptr->symbol, nbBits, baseSeq, level);
+ *DTableRank++ = DElt;
+ }
+ break;
+ case 2:
+ for (ptr = begin; ptr != end; ++ptr) {
+ HUF_DEltX2 const DElt = HUF_buildDEltX2(ptr->symbol, nbBits, baseSeq, level);
+ DTableRank[0] = DElt;
+ DTableRank[1] = DElt;
+ DTableRank += 2;
+ }
+ break;
+ case 4:
+ for (ptr = begin; ptr != end; ++ptr) {
+ U64 const DEltX2 = HUF_buildDEltX2U64(ptr->symbol, nbBits, baseSeq, level);
+ ZSTD_memcpy(DTableRank + 0, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTableRank + 2, &DEltX2, sizeof(DEltX2));
+ DTableRank += 4;
+ }
+ break;
+ case 8:
+ for (ptr = begin; ptr != end; ++ptr) {
+ U64 const DEltX2 = HUF_buildDEltX2U64(ptr->symbol, nbBits, baseSeq, level);
+ ZSTD_memcpy(DTableRank + 0, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTableRank + 2, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTableRank + 4, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTableRank + 6, &DEltX2, sizeof(DEltX2));
+ DTableRank += 8;
+ }
+ break;
+ default:
+ for (ptr = begin; ptr != end; ++ptr) {
+ U64 const DEltX2 = HUF_buildDEltX2U64(ptr->symbol, nbBits, baseSeq, level);
+ HUF_DEltX2* const DTableRankEnd = DTableRank + length;
+ for (; DTableRank != DTableRankEnd; DTableRank += 8) {
+ ZSTD_memcpy(DTableRank + 0, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTableRank + 2, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTableRank + 4, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTableRank + 6, &DEltX2, sizeof(DEltX2));
+ }
+ }
+ break;
+ }
+}
- /* fill skipped values */
+/* HUF_fillDTableX2Level2() :
+ * `rankValOrigin` must be a table of at least (HUF_TABLELOG_MAX + 1) U32 */
+static void HUF_fillDTableX2Level2(HUF_DEltX2* DTable, U32 targetLog, const U32 consumedBits,
+ const U32* rankVal, const int minWeight, const int maxWeight1,
+ const sortedSymbol_t* sortedSymbols, U32 const* rankStart,
+ U32 nbBitsBaseline, U16 baseSeq)
+{
+ /* Fill skipped values (all positions up to rankVal[minWeight]).
+ * These are positions only get a single symbol because the combined weight
+ * is too large.
+ */
if (minWeight>1) {
- U32 i, skipSize = rankVal[minWeight];
- MEM_writeLE16(&(DElt.sequence), baseSeq);
- DElt.nbBits = (BYTE)(consumed);
- DElt.length = 1;
- for (i = 0; i < skipSize; i++)
- DTable[i] = DElt;
+ U32 const length = 1U << ((targetLog - consumedBits) & 0x1F /* quiet static-analyzer */);
+ U64 const DEltX2 = HUF_buildDEltX2U64(baseSeq, consumedBits, /* baseSeq */ 0, /* level */ 1);
+ int const skipSize = rankVal[minWeight];
+ assert(length > 1);
+ assert((U32)skipSize < length);
+ switch (length) {
+ case 2:
+ assert(skipSize == 1);
+ ZSTD_memcpy(DTable, &DEltX2, sizeof(DEltX2));
+ break;
+ case 4:
+ assert(skipSize <= 4);
+ ZSTD_memcpy(DTable + 0, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTable + 2, &DEltX2, sizeof(DEltX2));
+ break;
+ default:
+ {
+ int i;
+ for (i = 0; i < skipSize; i += 8) {
+ ZSTD_memcpy(DTable + i + 0, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTable + i + 2, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTable + i + 4, &DEltX2, sizeof(DEltX2));
+ ZSTD_memcpy(DTable + i + 6, &DEltX2, sizeof(DEltX2));
+ }
+ }
+ }
}
- /* fill DTable */
- { U32 s; for (s=0; s<sortedListSize; s++) { /* note : sortedSymbols already skipped */
- const U32 symbol = sortedSymbols[s].symbol;
- const U32 weight = sortedSymbols[s].weight;
- const U32 nbBits = nbBitsBaseline - weight;
- const U32 length = 1 << (sizeLog-nbBits);
- const U32 start = rankVal[weight];
- U32 i = start;
- const U32 end = start + length;
-
- MEM_writeLE16(&(DElt.sequence), (U16)(baseSeq + (symbol << 8)));
- DElt.nbBits = (BYTE)(nbBits + consumed);
- DElt.length = 2;
- do { DTable[i++] = DElt; } while (i<end); /* since length >= 1 */
-
- rankVal[weight] += length;
- } }
+ /* Fill each of the second level symbols by weight. */
+ {
+ int w;
+ for (w = minWeight; w < maxWeight1; ++w) {
+ int const begin = rankStart[w];
+ int const end = rankStart[w+1];
+ U32 const nbBits = nbBitsBaseline - w;
+ U32 const totalBits = nbBits + consumedBits;
+ HUF_fillDTableX2ForWeight(
+ DTable + rankVal[w],
+ sortedSymbols + begin, sortedSymbols + end,
+ totalBits, targetLog,
+ baseSeq, /* level */ 2);
+ }
+ }
}
-
static void HUF_fillDTableX2(HUF_DEltX2* DTable, const U32 targetLog,
- const sortedSymbol_t* sortedList, const U32 sortedListSize,
+ const sortedSymbol_t* sortedList,
const U32* rankStart, rankVal_t rankValOrigin, const U32 maxWeight,
- const U32 nbBitsBaseline, U32* wksp, size_t wkspSize)
+ const U32 nbBitsBaseline)
{
- U32* rankVal = wksp;
+ U32* const rankVal = rankValOrigin[0];
const int scaleLog = nbBitsBaseline - targetLog; /* note : targetLog >= srcLog, hence scaleLog <= 1 */
const U32 minBits = nbBitsBaseline - maxWeight;
- U32 s;
-
- assert(wkspSize >= HUF_TABLELOG_MAX + 1);
- wksp += HUF_TABLELOG_MAX + 1;
- wkspSize -= HUF_TABLELOG_MAX + 1;
-
- ZSTD_memcpy(rankVal, rankValOrigin, sizeof(U32) * (HUF_TABLELOG_MAX + 1));
-
- /* fill DTable */
- for (s=0; s<sortedListSize; s++) {
- const U16 symbol = sortedList[s].symbol;
- const U32 weight = sortedList[s].weight;
- const U32 nbBits = nbBitsBaseline - weight;
- const U32 start = rankVal[weight];
- const U32 length = 1 << (targetLog-nbBits);
-
- if (targetLog-nbBits >= minBits) { /* enough room for a second symbol */
- U32 sortedRank;
+ int w;
+ int const wEnd = (int)maxWeight + 1;
+
+ /* Fill DTable in order of weight. */
+ for (w = 1; w < wEnd; ++w) {
+ int const begin = (int)rankStart[w];
+ int const end = (int)rankStart[w+1];
+ U32 const nbBits = nbBitsBaseline - w;
+
+ if (targetLog-nbBits >= minBits) {
+ /* Enough room for a second symbol. */
+ int start = rankVal[w];
+ U32 const length = 1U << ((targetLog - nbBits) & 0x1F /* quiet static-analyzer */);
int minWeight = nbBits + scaleLog;
+ int s;
if (minWeight < 1) minWeight = 1;
- sortedRank = rankStart[minWeight];
- HUF_fillDTableX2Level2(DTable+start, targetLog-nbBits, nbBits,
- rankValOrigin[nbBits], minWeight,
- sortedList+sortedRank, sortedListSize-sortedRank,
- nbBitsBaseline, symbol, wksp, wkspSize);
+ /* Fill the DTable for every symbol of weight w.
+ * These symbols get at least 1 second symbol.
+ */
+ for (s = begin; s != end; ++s) {
+ HUF_fillDTableX2Level2(
+ DTable + start, targetLog, nbBits,
+ rankValOrigin[nbBits], minWeight, wEnd,
+ sortedList, rankStart,
+ nbBitsBaseline, sortedList[s].symbol);
+ start += length;
+ }
} else {
- HUF_DEltX2 DElt;
- MEM_writeLE16(&(DElt.sequence), symbol);
- DElt.nbBits = (BYTE)(nbBits);
- DElt.length = 1;
- { U32 const end = start + length;
- U32 u;
- for (u = start; u < end; u++) DTable[u] = DElt;
- } }
- rankVal[weight] += length;
+ /* Only a single symbol. */
+ HUF_fillDTableX2ForWeight(
+ DTable + rankVal[w],
+ sortedList + begin, sortedList + end,
+ nbBits, targetLog,
+ /* baseSeq */ 0, /* level */ 1);
+ }
}
}
typedef struct {
rankValCol_t rankVal[HUF_TABLELOG_MAX];
U32 rankStats[HUF_TABLELOG_MAX + 1];
- U32 rankStart0[HUF_TABLELOG_MAX + 2];
+ U32 rankStart0[HUF_TABLELOG_MAX + 3];
sortedSymbol_t sortedSymbol[HUF_SYMBOLVALUE_MAX + 1];
BYTE weightList[HUF_SYMBOLVALUE_MAX + 1];
U32 calleeWksp[HUF_READ_STATS_WORKSPACE_SIZE_U32];
const void* src, size_t srcSize,
void* workSpace, size_t wkspSize)
{
- U32 tableLog, maxW, sizeOfSort, nbSymbols;
+ return HUF_readDTableX2_wksp_bmi2(DTable, src, srcSize, workSpace, wkspSize, /* bmi2 */ 0);
+}
+
+size_t HUF_readDTableX2_wksp_bmi2(HUF_DTable* DTable,
+ const void* src, size_t srcSize,
+ void* workSpace, size_t wkspSize, int bmi2)
+{
+ U32 tableLog, maxW, nbSymbols;
DTableDesc dtd = HUF_getDTableDesc(DTable);
- U32 const maxTableLog = dtd.maxTableLog;
+ U32 maxTableLog = dtd.maxTableLog;
size_t iSize;
void* dtPtr = DTable+1; /* force compiler to avoid strict-aliasing */
HUF_DEltX2* const dt = (HUF_DEltX2*)dtPtr;
if (maxTableLog > HUF_TABLELOG_MAX) return ERROR(tableLog_tooLarge);
/* ZSTD_memset(weightList, 0, sizeof(weightList)); */ /* is not necessary, even though some analyzer complain ... */
- iSize = HUF_readStats_wksp(wksp->weightList, HUF_SYMBOLVALUE_MAX + 1, wksp->rankStats, &nbSymbols, &tableLog, src, srcSize, wksp->calleeWksp, sizeof(wksp->calleeWksp), /* bmi2 */ 0);
+ iSize = HUF_readStats_wksp(wksp->weightList, HUF_SYMBOLVALUE_MAX + 1, wksp->rankStats, &nbSymbols, &tableLog, src, srcSize, wksp->calleeWksp, sizeof(wksp->calleeWksp), bmi2);
if (HUF_isError(iSize)) return iSize;
/* check result */
if (tableLog > maxTableLog) return ERROR(tableLog_tooLarge); /* DTable can't fit code depth */
+ if (tableLog <= HUF_DECODER_FAST_TABLELOG && maxTableLog > HUF_DECODER_FAST_TABLELOG) maxTableLog = HUF_DECODER_FAST_TABLELOG;
/* find maxWeight */
for (maxW = tableLog; wksp->rankStats[maxW]==0; maxW--) {} /* necessarily finds a solution before 0 */
rankStart[w] = curr;
}
rankStart[0] = nextRankStart; /* put all 0w symbols at the end of sorted list*/
- sizeOfSort = nextRankStart;
+ rankStart[maxW+1] = nextRankStart;
}
/* sort symbols by weight */
U32 const w = wksp->weightList[s];
U32 const r = rankStart[w]++;
wksp->sortedSymbol[r].symbol = (BYTE)s;
- wksp->sortedSymbol[r].weight = (BYTE)w;
}
rankStart[0] = 0; /* forget 0w symbols; this is beginning of weight(1) */
}
} } } }
HUF_fillDTableX2(dt, maxTableLog,
- wksp->sortedSymbol, sizeOfSort,
+ wksp->sortedSymbol,
wksp->rankStart0, wksp->rankVal, maxW,
- tableLog+1,
- wksp->calleeWksp, sizeof(wksp->calleeWksp) / sizeof(U32));
+ tableLog+1);
dtd.tableLog = (BYTE)maxTableLog;
dtd.tableType = 1;
HUF_decodeSymbolX2(void* op, BIT_DStream_t* DStream, const HUF_DEltX2* dt, const U32 dtLog)
{
size_t const val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
- ZSTD_memcpy(op, dt+val, 2);
+ ZSTD_memcpy(op, &dt[val].sequence, 2);
BIT_skipBits(DStream, dt[val].nbBits);
return dt[val].length;
}
HUF_decodeLastSymbolX2(void* op, BIT_DStream_t* DStream, const HUF_DEltX2* dt, const U32 dtLog)
{
size_t const val = BIT_lookBitsFast(DStream, dtLog); /* note : dtLog >= 1 */
- ZSTD_memcpy(op, dt+val, 1);
- if (dt[val].length==1) BIT_skipBits(DStream, dt[val].nbBits);
- else {
+ ZSTD_memcpy(op, &dt[val].sequence, 1);
+ if (dt[val].length==1) {
+ BIT_skipBits(DStream, dt[val].nbBits);
+ } else {
if (DStream->bitsConsumed < (sizeof(DStream->bitContainer)*8)) {
BIT_skipBits(DStream, dt[val].nbBits);
if (DStream->bitsConsumed > (sizeof(DStream->bitContainer)*8))
/* ugly hack; works only because it's the last symbol. Note : can't easily extract nbBits from just this symbol */
DStream->bitsConsumed = (sizeof(DStream->bitContainer)*8);
- } }
+ }
+ }
return 1;
}
BYTE* const pStart = p;
/* up to 8 symbols at a time */
- while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd-(sizeof(bitDPtr->bitContainer)-1))) {
- HUF_DECODE_SYMBOLX2_2(p, bitDPtr);
- HUF_DECODE_SYMBOLX2_1(p, bitDPtr);
- HUF_DECODE_SYMBOLX2_2(p, bitDPtr);
- HUF_DECODE_SYMBOLX2_0(p, bitDPtr);
+ if ((size_t)(pEnd - p) >= sizeof(bitDPtr->bitContainer)) {
+ if (dtLog <= 11 && MEM_64bits()) {
+ /* up to 10 symbols at a time */
+ while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd-9)) {
+ HUF_DECODE_SYMBOLX2_0(p, bitDPtr);
+ HUF_DECODE_SYMBOLX2_0(p, bitDPtr);
+ HUF_DECODE_SYMBOLX2_0(p, bitDPtr);
+ HUF_DECODE_SYMBOLX2_0(p, bitDPtr);
+ HUF_DECODE_SYMBOLX2_0(p, bitDPtr);
+ }
+ } else {
+ /* up to 8 symbols at a time */
+ while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p < pEnd-(sizeof(bitDPtr->bitContainer)-1))) {
+ HUF_DECODE_SYMBOLX2_2(p, bitDPtr);
+ HUF_DECODE_SYMBOLX2_1(p, bitDPtr);
+ HUF_DECODE_SYMBOLX2_2(p, bitDPtr);
+ HUF_DECODE_SYMBOLX2_0(p, bitDPtr);
+ }
+ }
+ } else {
+ BIT_reloadDStream(bitDPtr);
}
/* closer to end : up to 2 symbols at a time */
- while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p <= pEnd-2))
- HUF_DECODE_SYMBOLX2_0(p, bitDPtr);
+ if ((size_t)(pEnd - p) >= 2) {
+ while ((BIT_reloadDStream(bitDPtr) == BIT_DStream_unfinished) & (p <= pEnd-2))
+ HUF_DECODE_SYMBOLX2_0(p, bitDPtr);
- while (p <= pEnd-2)
- HUF_DECODE_SYMBOLX2_0(p, bitDPtr); /* no need to reload : reached the end of DStream */
+ while (p <= pEnd-2)
+ HUF_DECODE_SYMBOLX2_0(p, bitDPtr); /* no need to reload : reached the end of DStream */
+ }
if (p < pEnd)
p += HUF_decodeLastSymbolX2(p, bitDPtr, dt, dtLog);
/* decoded size */
return dstSize;
}
-
FORCE_INLINE_TEMPLATE size_t
HUF_decompress4X2_usingDTable_internal_body(
void* dst, size_t dstSize,
U32 const dtLog = dtd.tableLog;
if (length4 > cSrcSize) return ERROR(corruption_detected); /* overflow */
+ if (opStart4 > oend) return ERROR(corruption_detected); /* overflow */
CHECK_F( BIT_initDStream(&bitD1, istart1, length1) );
CHECK_F( BIT_initDStream(&bitD2, istart2, length2) );
CHECK_F( BIT_initDStream(&bitD3, istart3, length3) );
CHECK_F( BIT_initDStream(&bitD4, istart4, length4) );
/* 16-32 symbols per loop (4-8 symbols per stream) */
- for ( ; (endSignal) & (op4 < olimit); ) {
+ if ((size_t)(oend - op4) >= sizeof(size_t)) {
+ for ( ; (endSignal) & (op4 < olimit); ) {
#if defined(__clang__) && (defined(__x86_64__) || defined(__i386__))
- HUF_DECODE_SYMBOLX2_2(op1, &bitD1);
- HUF_DECODE_SYMBOLX2_1(op1, &bitD1);
- HUF_DECODE_SYMBOLX2_2(op1, &bitD1);
- HUF_DECODE_SYMBOLX2_0(op1, &bitD1);
- HUF_DECODE_SYMBOLX2_2(op2, &bitD2);
- HUF_DECODE_SYMBOLX2_1(op2, &bitD2);
- HUF_DECODE_SYMBOLX2_2(op2, &bitD2);
- HUF_DECODE_SYMBOLX2_0(op2, &bitD2);
- endSignal &= BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished;
- endSignal &= BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished;
- HUF_DECODE_SYMBOLX2_2(op3, &bitD3);
- HUF_DECODE_SYMBOLX2_1(op3, &bitD3);
- HUF_DECODE_SYMBOLX2_2(op3, &bitD3);
- HUF_DECODE_SYMBOLX2_0(op3, &bitD3);
- HUF_DECODE_SYMBOLX2_2(op4, &bitD4);
- HUF_DECODE_SYMBOLX2_1(op4, &bitD4);
- HUF_DECODE_SYMBOLX2_2(op4, &bitD4);
- HUF_DECODE_SYMBOLX2_0(op4, &bitD4);
- endSignal &= BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished;
- endSignal &= BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished;
+ HUF_DECODE_SYMBOLX2_2(op1, &bitD1);
+ HUF_DECODE_SYMBOLX2_1(op1, &bitD1);
+ HUF_DECODE_SYMBOLX2_2(op1, &bitD1);
+ HUF_DECODE_SYMBOLX2_0(op1, &bitD1);
+ HUF_DECODE_SYMBOLX2_2(op2, &bitD2);
+ HUF_DECODE_SYMBOLX2_1(op2, &bitD2);
+ HUF_DECODE_SYMBOLX2_2(op2, &bitD2);
+ HUF_DECODE_SYMBOLX2_0(op2, &bitD2);
+ endSignal &= BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished;
+ endSignal &= BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished;
+ HUF_DECODE_SYMBOLX2_2(op3, &bitD3);
+ HUF_DECODE_SYMBOLX2_1(op3, &bitD3);
+ HUF_DECODE_SYMBOLX2_2(op3, &bitD3);
+ HUF_DECODE_SYMBOLX2_0(op3, &bitD3);
+ HUF_DECODE_SYMBOLX2_2(op4, &bitD4);
+ HUF_DECODE_SYMBOLX2_1(op4, &bitD4);
+ HUF_DECODE_SYMBOLX2_2(op4, &bitD4);
+ HUF_DECODE_SYMBOLX2_0(op4, &bitD4);
+ endSignal &= BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished;
+ endSignal &= BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished;
#else
- HUF_DECODE_SYMBOLX2_2(op1, &bitD1);
- HUF_DECODE_SYMBOLX2_2(op2, &bitD2);
- HUF_DECODE_SYMBOLX2_2(op3, &bitD3);
- HUF_DECODE_SYMBOLX2_2(op4, &bitD4);
- HUF_DECODE_SYMBOLX2_1(op1, &bitD1);
- HUF_DECODE_SYMBOLX2_1(op2, &bitD2);
- HUF_DECODE_SYMBOLX2_1(op3, &bitD3);
- HUF_DECODE_SYMBOLX2_1(op4, &bitD4);
- HUF_DECODE_SYMBOLX2_2(op1, &bitD1);
- HUF_DECODE_SYMBOLX2_2(op2, &bitD2);
- HUF_DECODE_SYMBOLX2_2(op3, &bitD3);
- HUF_DECODE_SYMBOLX2_2(op4, &bitD4);
- HUF_DECODE_SYMBOLX2_0(op1, &bitD1);
- HUF_DECODE_SYMBOLX2_0(op2, &bitD2);
- HUF_DECODE_SYMBOLX2_0(op3, &bitD3);
- HUF_DECODE_SYMBOLX2_0(op4, &bitD4);
- endSignal = (U32)LIKELY((U32)
- (BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished)
- & (BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished)
- & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished)
- & (BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished));
+ HUF_DECODE_SYMBOLX2_2(op1, &bitD1);
+ HUF_DECODE_SYMBOLX2_2(op2, &bitD2);
+ HUF_DECODE_SYMBOLX2_2(op3, &bitD3);
+ HUF_DECODE_SYMBOLX2_2(op4, &bitD4);
+ HUF_DECODE_SYMBOLX2_1(op1, &bitD1);
+ HUF_DECODE_SYMBOLX2_1(op2, &bitD2);
+ HUF_DECODE_SYMBOLX2_1(op3, &bitD3);
+ HUF_DECODE_SYMBOLX2_1(op4, &bitD4);
+ HUF_DECODE_SYMBOLX2_2(op1, &bitD1);
+ HUF_DECODE_SYMBOLX2_2(op2, &bitD2);
+ HUF_DECODE_SYMBOLX2_2(op3, &bitD3);
+ HUF_DECODE_SYMBOLX2_2(op4, &bitD4);
+ HUF_DECODE_SYMBOLX2_0(op1, &bitD1);
+ HUF_DECODE_SYMBOLX2_0(op2, &bitD2);
+ HUF_DECODE_SYMBOLX2_0(op3, &bitD3);
+ HUF_DECODE_SYMBOLX2_0(op4, &bitD4);
+ endSignal = (U32)LIKELY((U32)
+ (BIT_reloadDStreamFast(&bitD1) == BIT_DStream_unfinished)
+ & (BIT_reloadDStreamFast(&bitD2) == BIT_DStream_unfinished)
+ & (BIT_reloadDStreamFast(&bitD3) == BIT_DStream_unfinished)
+ & (BIT_reloadDStreamFast(&bitD4) == BIT_DStream_unfinished));
#endif
+ }
}
/* check corruption */
}
}
+#if HUF_NEED_BMI2_FUNCTION
+static BMI2_TARGET_ATTRIBUTE
+size_t HUF_decompress4X2_usingDTable_internal_bmi2(void* dst, size_t dstSize, void const* cSrc,
+ size_t cSrcSize, HUF_DTable const* DTable) {
+ return HUF_decompress4X2_usingDTable_internal_body(dst, dstSize, cSrc, cSrcSize, DTable);
+}
+#endif
+
+#if HUF_NEED_DEFAULT_FUNCTION
+static
+size_t HUF_decompress4X2_usingDTable_internal_default(void* dst, size_t dstSize, void const* cSrc,
+ size_t cSrcSize, HUF_DTable const* DTable) {
+ return HUF_decompress4X2_usingDTable_internal_body(dst, dstSize, cSrc, cSrcSize, DTable);
+}
+#endif
+
+#if ZSTD_ENABLE_ASM_X86_64_BMI2
+
+HUF_ASM_DECL void HUF_decompress4X2_usingDTable_internal_bmi2_asm_loop(HUF_DecompressAsmArgs* args) ZSTDLIB_HIDDEN;
+
+static HUF_ASM_X86_64_BMI2_ATTRS size_t
+HUF_decompress4X2_usingDTable_internal_bmi2_asm(
+ void* dst, size_t dstSize,
+ const void* cSrc, size_t cSrcSize,
+ const HUF_DTable* DTable) {
+ void const* dt = DTable + 1;
+ const BYTE* const iend = (const BYTE*)cSrc + 6;
+ BYTE* const oend = (BYTE*)dst + dstSize;
+ HUF_DecompressAsmArgs args;
+ {
+ size_t const ret = HUF_DecompressAsmArgs_init(&args, dst, dstSize, cSrc, cSrcSize, DTable);
+ FORWARD_IF_ERROR(ret, "Failed to init asm args");
+ if (ret != 0)
+ return HUF_decompress4X2_usingDTable_internal_bmi2(dst, dstSize, cSrc, cSrcSize, DTable);
+ }
+
+ assert(args.ip[0] >= args.ilimit);
+ HUF_decompress4X2_usingDTable_internal_bmi2_asm_loop(&args);
+
+ /* note : op4 already verified within main loop */
+ assert(args.ip[0] >= iend);
+ assert(args.ip[1] >= iend);
+ assert(args.ip[2] >= iend);
+ assert(args.ip[3] >= iend);
+ assert(args.op[3] <= oend);
+ (void)iend;
+
+ /* finish bitStreams one by one */
+ {
+ size_t const segmentSize = (dstSize+3) / 4;
+ BYTE* segmentEnd = (BYTE*)dst;
+ int i;
+ for (i = 0; i < 4; ++i) {
+ BIT_DStream_t bit;
+ if (segmentSize <= (size_t)(oend - segmentEnd))
+ segmentEnd += segmentSize;
+ else
+ segmentEnd = oend;
+ FORWARD_IF_ERROR(HUF_initRemainingDStream(&bit, &args, i, segmentEnd), "corruption");
+ args.op[i] += HUF_decodeStreamX2(args.op[i], &bit, segmentEnd, (HUF_DEltX2 const*)dt, HUF_DECODER_FAST_TABLELOG);
+ if (args.op[i] != segmentEnd)
+ return ERROR(corruption_detected);
+ }
+ }
+
+ /* decoded size */
+ return dstSize;
+}
+#endif /* ZSTD_ENABLE_ASM_X86_64_BMI2 */
+
+static size_t HUF_decompress4X2_usingDTable_internal(void* dst, size_t dstSize, void const* cSrc,
+ size_t cSrcSize, HUF_DTable const* DTable, int bmi2)
+{
+#if DYNAMIC_BMI2
+ if (bmi2) {
+# if ZSTD_ENABLE_ASM_X86_64_BMI2
+ return HUF_decompress4X2_usingDTable_internal_bmi2_asm(dst, dstSize, cSrc, cSrcSize, DTable);
+# else
+ return HUF_decompress4X2_usingDTable_internal_bmi2(dst, dstSize, cSrc, cSrcSize, DTable);
+# endif
+ }
+#else
+ (void)bmi2;
+#endif
+
+#if ZSTD_ENABLE_ASM_X86_64_BMI2 && defined(__BMI2__)
+ return HUF_decompress4X2_usingDTable_internal_bmi2_asm(dst, dstSize, cSrc, cSrcSize, DTable);
+#else
+ return HUF_decompress4X2_usingDTable_internal_default(dst, dstSize, cSrc, cSrcSize, DTable);
+#endif
+}
+
HUF_DGEN(HUF_decompress1X2_usingDTable_internal)
-HUF_DGEN(HUF_decompress4X2_usingDTable_internal)
size_t HUF_decompress1X2_usingDTable(
void* dst, size_t dstSize,
#if !defined(HUF_FORCE_DECOMPRESS_X1) && !defined(HUF_FORCE_DECOMPRESS_X2)
typedef struct { U32 tableTime; U32 decode256Time; } algo_time_t;
-static const algo_time_t algoTime[16 /* Quantization */][3 /* single, double, quad */] =
+static const algo_time_t algoTime[16 /* Quantization */][2 /* single, double */] =
{
/* single, double, quad */
- {{0,0}, {1,1}, {2,2}}, /* Q==0 : impossible */
- {{0,0}, {1,1}, {2,2}}, /* Q==1 : impossible */
- {{ 38,130}, {1313, 74}, {2151, 38}}, /* Q == 2 : 12-18% */
- {{ 448,128}, {1353, 74}, {2238, 41}}, /* Q == 3 : 18-25% */
- {{ 556,128}, {1353, 74}, {2238, 47}}, /* Q == 4 : 25-32% */
- {{ 714,128}, {1418, 74}, {2436, 53}}, /* Q == 5 : 32-38% */
- {{ 883,128}, {1437, 74}, {2464, 61}}, /* Q == 6 : 38-44% */
- {{ 897,128}, {1515, 75}, {2622, 68}}, /* Q == 7 : 44-50% */
- {{ 926,128}, {1613, 75}, {2730, 75}}, /* Q == 8 : 50-56% */
- {{ 947,128}, {1729, 77}, {3359, 77}}, /* Q == 9 : 56-62% */
- {{1107,128}, {2083, 81}, {4006, 84}}, /* Q ==10 : 62-69% */
- {{1177,128}, {2379, 87}, {4785, 88}}, /* Q ==11 : 69-75% */
- {{1242,128}, {2415, 93}, {5155, 84}}, /* Q ==12 : 75-81% */
- {{1349,128}, {2644,106}, {5260,106}}, /* Q ==13 : 81-87% */
- {{1455,128}, {2422,124}, {4174,124}}, /* Q ==14 : 87-93% */
- {{ 722,128}, {1891,145}, {1936,146}}, /* Q ==15 : 93-99% */
+ {{0,0}, {1,1}}, /* Q==0 : impossible */
+ {{0,0}, {1,1}}, /* Q==1 : impossible */
+ {{ 150,216}, { 381,119}}, /* Q == 2 : 12-18% */
+ {{ 170,205}, { 514,112}}, /* Q == 3 : 18-25% */
+ {{ 177,199}, { 539,110}}, /* Q == 4 : 25-32% */
+ {{ 197,194}, { 644,107}}, /* Q == 5 : 32-38% */
+ {{ 221,192}, { 735,107}}, /* Q == 6 : 38-44% */
+ {{ 256,189}, { 881,106}}, /* Q == 7 : 44-50% */
+ {{ 359,188}, {1167,109}}, /* Q == 8 : 50-56% */
+ {{ 582,187}, {1570,114}}, /* Q == 9 : 56-62% */
+ {{ 688,187}, {1712,122}}, /* Q ==10 : 62-69% */
+ {{ 825,186}, {1965,136}}, /* Q ==11 : 69-75% */
+ {{ 976,185}, {2131,150}}, /* Q ==12 : 75-81% */
+ {{1180,186}, {2070,175}}, /* Q ==13 : 81-87% */
+ {{1377,185}, {1731,202}}, /* Q ==14 : 87-93% */
+ {{1412,185}, {1695,202}}, /* Q ==15 : 93-99% */
};
#endif
U32 const D256 = (U32)(dstSize >> 8);
U32 const DTime0 = algoTime[Q][0].tableTime + (algoTime[Q][0].decode256Time * D256);
U32 DTime1 = algoTime[Q][1].tableTime + (algoTime[Q][1].decode256Time * D256);
- DTime1 += DTime1 >> 3; /* advantage to algorithm using less memory, to reduce cache eviction */
+ DTime1 += DTime1 >> 5; /* small advantage to algorithm using less memory, to reduce cache eviction */
return DTime1 < DTime0;
}
#endif
* Dependencies
*********************************************************/
#include "../common/zstd_deps.h" /* ZSTD_memcpy, ZSTD_memmove, ZSTD_memset */
-#include "../common/cpu.h" /* bmi2 */
#include "../common/mem.h" /* low level memory routines */
#define FSE_STATIC_LINKING_ONLY
#include "../common/fse.h"
dctx->inBuffSize = 0;
dctx->outBuffSize = 0;
dctx->streamStage = zdss_init;
- dctx->legacyContext = NULL;
- dctx->previousLegacyVersion = 0;
dctx->noForwardProgress = 0;
dctx->oversizedDuration = 0;
- dctx->bmi2 = ZSTD_cpuid_bmi2(ZSTD_cpuid());
+#if DYNAMIC_BMI2
+ dctx->bmi2 = ZSTD_cpuSupportsBmi2();
+#endif
dctx->ddictSet = NULL;
ZSTD_DCtx_resetParameters(dctx);
#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
return dctx;
}
-ZSTD_DCtx* ZSTD_createDCtx_advanced(ZSTD_customMem customMem)
-{
+static ZSTD_DCtx* ZSTD_createDCtx_internal(ZSTD_customMem customMem) {
if ((!customMem.customAlloc) ^ (!customMem.customFree)) return NULL;
{ ZSTD_DCtx* const dctx = (ZSTD_DCtx*)ZSTD_customMalloc(sizeof(*dctx), customMem);
}
}
+ZSTD_DCtx* ZSTD_createDCtx_advanced(ZSTD_customMem customMem)
+{
+ return ZSTD_createDCtx_internal(customMem);
+}
+
ZSTD_DCtx* ZSTD_createDCtx(void)
{
DEBUGLOG(3, "ZSTD_createDCtx");
- return ZSTD_createDCtx_advanced(ZSTD_defaultCMem);
+ return ZSTD_createDCtx_internal(ZSTD_defaultCMem);
}
static void ZSTD_clearDict(ZSTD_DCtx* dctx)
return 0;
}
+/*! ZSTD_isSkippableFrame() :
+ * Tells if the content of `buffer` starts with a valid Frame Identifier for a skippable frame.
+ * Note : Frame Identifier is 4 bytes. If `size < 4`, @return will always be 0.
+ */
+unsigned ZSTD_isSkippableFrame(const void* buffer, size_t size)
+{
+ if (size < ZSTD_FRAMEIDSIZE) return 0;
+ { U32 const magic = MEM_readLE32(buffer);
+ if ((magic & ZSTD_MAGIC_SKIPPABLE_MASK) == ZSTD_MAGIC_SKIPPABLE_START) return 1;
+ }
+ return 0;
+}
+
/* ZSTD_frameHeaderSize_internal() :
* srcSize must be large enough to reach header size fields.
* note : only works for formats ZSTD_f_zstd1 and ZSTD_f_zstd1_magicless.
return ZSTD_getFrameHeader_advanced(zfhPtr, src, srcSize, ZSTD_f_zstd1);
}
-
/* ZSTD_getFrameContentSize() :
* compatible with legacy mode
* @return : decompressed size of the single frame pointed to be `src` if known, otherwise
}
}
+/*! ZSTD_readSkippableFrame() :
+ * Retrieves a zstd skippable frame containing data given by src, and writes it to dst buffer.
+ *
+ * The parameter magicVariant will receive the magicVariant that was supplied when the frame was written,
+ * i.e. magicNumber - ZSTD_MAGIC_SKIPPABLE_START. This can be NULL if the caller is not interested
+ * in the magicVariant.
+ *
+ * Returns an error if destination buffer is not large enough, or if the frame is not skippable.
+ *
+ * @return : number of bytes written or a ZSTD error.
+ */
+ZSTDLIB_API size_t ZSTD_readSkippableFrame(void* dst, size_t dstCapacity, unsigned* magicVariant,
+ const void* src, size_t srcSize)
+{
+ U32 const magicNumber = MEM_readLE32(src);
+ size_t skippableFrameSize = readSkippableFrameSize(src, srcSize);
+ size_t skippableContentSize = skippableFrameSize - ZSTD_SKIPPABLEHEADERSIZE;
+
+ /* check input validity */
+ RETURN_ERROR_IF(!ZSTD_isSkippableFrame(src, srcSize), frameParameter_unsupported, "");
+ RETURN_ERROR_IF(skippableFrameSize < ZSTD_SKIPPABLEHEADERSIZE || skippableFrameSize > srcSize, srcSize_wrong, "");
+ RETURN_ERROR_IF(skippableContentSize > dstCapacity, dstSize_tooSmall, "");
+
+ /* deliver payload */
+ if (skippableContentSize > 0 && dst != NULL)
+ ZSTD_memcpy(dst, (const BYTE *)src + ZSTD_SKIPPABLEHEADERSIZE, skippableContentSize);
+ if (magicVariant != NULL)
+ *magicVariant = magicNumber - ZSTD_MAGIC_SKIPPABLE_START;
+ return skippableContentSize;
+}
+
/* ZSTD_findDecompressedSize() :
* compatible with legacy mode
* `srcSize` must be the exact length of some number of ZSTD compressed and/or
switch(blockProperties.blockType)
{
case bt_compressed:
- decodedSize = ZSTD_decompressBlock_internal(dctx, op, (size_t)(oend-op), ip, cBlockSize, /* frame */ 1);
+ decodedSize = ZSTD_decompressBlock_internal(dctx, op, (size_t)(oend-op), ip, cBlockSize, /* frame */ 1, not_streaming);
break;
case bt_raw :
decodedSize = ZSTD_copyRawBlock(op, (size_t)(oend-op), ip, cBlockSize);
{
#if defined(ZSTD_HEAPMODE) && (ZSTD_HEAPMODE>=1)
size_t regenSize;
- ZSTD_DCtx* const dctx = ZSTD_createDCtx();
+ ZSTD_DCtx* const dctx = ZSTD_createDCtx_internal(ZSTD_defaultCMem);
RETURN_ERROR_IF(dctx==NULL, memory_allocation, "NULL pointer!");
regenSize = ZSTD_decompressDCtx(dctx, dst, dstCapacity, src, srcSize);
ZSTD_freeDCtx(dctx);
size_t ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx* dctx) { return dctx->expected; }
/*
- * Similar to ZSTD_nextSrcSizeToDecompress(), but when when a block input can be streamed,
+ * Similar to ZSTD_nextSrcSizeToDecompress(), but when a block input can be streamed,
* we allow taking a partial block as the input. Currently only raw uncompressed blocks can
* be streamed.
*
return dctx->expected;
if (dctx->bType != bt_raw)
return dctx->expected;
- return MIN(MAX(inputSize, 1), dctx->expected);
+ return BOUNDED(1, inputSize, dctx->expected);
}
ZSTD_nextInputType_e ZSTD_nextInputType(ZSTD_DCtx* dctx) {
{
case bt_compressed:
DEBUGLOG(5, "ZSTD_decompressContinue: case bt_compressed");
- rSize = ZSTD_decompressBlock_internal(dctx, dst, dstCapacity, src, srcSize, /* frame */ 1);
+ rSize = ZSTD_decompressBlock_internal(dctx, dst, dstCapacity, src, srcSize, /* frame */ 1, is_streaming);
dctx->expected = 0; /* Streaming not supported */
break;
case bt_raw :
ZSTD_DStream* ZSTD_createDStream(void)
{
DEBUGLOG(3, "ZSTD_createDStream");
- return ZSTD_createDStream_advanced(ZSTD_defaultCMem);
+ return ZSTD_createDCtx_internal(ZSTD_defaultCMem);
}
ZSTD_DStream* ZSTD_initStaticDStream(void *workspace, size_t workspaceSize)
ZSTD_DStream* ZSTD_createDStream_advanced(ZSTD_customMem customMem)
{
- return ZSTD_createDCtx_advanced(customMem);
+ return ZSTD_createDCtx_internal(customMem);
}
size_t ZSTD_freeDStream(ZSTD_DStream* zds)
size_t ZSTD_decodingBufferSize_min(unsigned long long windowSize, unsigned long long frameContentSize)
{
size_t const blockSize = (size_t) MIN(windowSize, ZSTD_BLOCKSIZE_MAX);
- unsigned long long const neededRBSize = windowSize + blockSize + (WILDCOPY_OVERLENGTH * 2);
+ /* space is needed to store the litbuffer after the output of a given block without stomping the extDict of a previous run, as well as to cover both windows against wildcopy*/
+ unsigned long long const neededRBSize = windowSize + blockSize + ZSTD_BLOCKSIZE_MAX + (WILDCOPY_OVERLENGTH * 2);
unsigned long long const neededSize = MIN(frameContentSize, neededRBSize);
size_t const minRBSize = (size_t) neededSize;
RETURN_ERROR_IF((unsigned long long)minRBSize != neededSize,
DEBUGLOG(5, "stage zdss_init => transparent reset ");
zds->streamStage = zdss_loadHeader;
zds->lhSize = zds->inPos = zds->outStart = zds->outEnd = 0;
- zds->legacyVersion = 0;
zds->hostageByte = 0;
zds->expectedOutBuffer = *output;
ZSTD_FALLTHROUGH;
}
}
+/* Allocate buffer for literals, either overlapping current dst, or split between dst and litExtraBuffer, or stored entirely within litExtraBuffer */
+static void ZSTD_allocateLiteralsBuffer(ZSTD_DCtx* dctx, void* const dst, const size_t dstCapacity, const size_t litSize,
+ const streaming_operation streaming, const size_t expectedWriteSize, const unsigned splitImmediately)
+{
+ if (streaming == not_streaming && dstCapacity > ZSTD_BLOCKSIZE_MAX + WILDCOPY_OVERLENGTH + litSize + WILDCOPY_OVERLENGTH)
+ {
+ /* room for litbuffer to fit without read faulting */
+ dctx->litBuffer = (BYTE*)dst + ZSTD_BLOCKSIZE_MAX + WILDCOPY_OVERLENGTH;
+ dctx->litBufferEnd = dctx->litBuffer + litSize;
+ dctx->litBufferLocation = ZSTD_in_dst;
+ }
+ else if (litSize > ZSTD_LITBUFFEREXTRASIZE)
+ {
+ /* won't fit in litExtraBuffer, so it will be split between end of dst and extra buffer */
+ if (splitImmediately) {
+ /* won't fit in litExtraBuffer, so it will be split between end of dst and extra buffer */
+ dctx->litBuffer = (BYTE*)dst + expectedWriteSize - litSize + ZSTD_LITBUFFEREXTRASIZE - WILDCOPY_OVERLENGTH;
+ dctx->litBufferEnd = dctx->litBuffer + litSize - ZSTD_LITBUFFEREXTRASIZE;
+ }
+ else {
+ /* initially this will be stored entirely in dst during huffman decoding, it will partially shifted to litExtraBuffer after */
+ dctx->litBuffer = (BYTE*)dst + expectedWriteSize - litSize;
+ dctx->litBufferEnd = (BYTE*)dst + expectedWriteSize;
+ }
+ dctx->litBufferLocation = ZSTD_split;
+ }
+ else
+ {
+ /* fits entirely within litExtraBuffer, so no split is necessary */
+ dctx->litBuffer = dctx->litExtraBuffer;
+ dctx->litBufferEnd = dctx->litBuffer + litSize;
+ dctx->litBufferLocation = ZSTD_not_in_dst;
+ }
+}
/* Hidden declaration for fullbench */
size_t ZSTD_decodeLiteralsBlock(ZSTD_DCtx* dctx,
- const void* src, size_t srcSize);
+ const void* src, size_t srcSize,
+ void* dst, size_t dstCapacity, const streaming_operation streaming);
/*! ZSTD_decodeLiteralsBlock() :
+ * Where it is possible to do so without being stomped by the output during decompression, the literals block will be stored
+ * in the dstBuffer. If there is room to do so, it will be stored in full in the excess dst space after where the current
+ * block will be output. Otherwise it will be stored at the end of the current dst blockspace, with a small portion being
+ * stored in dctx->litExtraBuffer to help keep it "ahead" of the current output write.
+ *
* @return : nb of bytes read from src (< srcSize )
* note : symbol not declared but exposed for fullbench */
size_t ZSTD_decodeLiteralsBlock(ZSTD_DCtx* dctx,
- const void* src, size_t srcSize) /* note : srcSize < BLOCKSIZE */
+ const void* src, size_t srcSize, /* note : srcSize < BLOCKSIZE */
+ void* dst, size_t dstCapacity, const streaming_operation streaming)
{
DEBUGLOG(5, "ZSTD_decodeLiteralsBlock");
RETURN_ERROR_IF(srcSize < MIN_CBLOCK_SIZE, corruption_detected, "");
U32 const lhlCode = (istart[0] >> 2) & 3;
U32 const lhc = MEM_readLE32(istart);
size_t hufSuccess;
+ size_t expectedWriteSize = MIN(ZSTD_BLOCKSIZE_MAX, dstCapacity);
switch(lhlCode)
{
case 0: case 1: default: /* note : default is impossible, since lhlCode into [0..3] */
litCSize = (lhc >> 22) + ((size_t)istart[4] << 10);
break;
}
+ RETURN_ERROR_IF(litSize > 0 && dst == NULL, dstSize_tooSmall, "NULL not handled");
RETURN_ERROR_IF(litSize > ZSTD_BLOCKSIZE_MAX, corruption_detected, "");
RETURN_ERROR_IF(litCSize + lhSize > srcSize, corruption_detected, "");
+ RETURN_ERROR_IF(expectedWriteSize < litSize , dstSize_tooSmall, "");
+ ZSTD_allocateLiteralsBuffer(dctx, dst, dstCapacity, litSize, streaming, expectedWriteSize, 0);
/* prefetch huffman table if cold */
if (dctx->ddictIsCold && (litSize > 768 /* heuristic */)) {
if (singleStream) {
hufSuccess = HUF_decompress1X_usingDTable_bmi2(
dctx->litBuffer, litSize, istart+lhSize, litCSize,
- dctx->HUFptr, dctx->bmi2);
+ dctx->HUFptr, ZSTD_DCtx_get_bmi2(dctx));
} else {
hufSuccess = HUF_decompress4X_usingDTable_bmi2(
dctx->litBuffer, litSize, istart+lhSize, litCSize,
- dctx->HUFptr, dctx->bmi2);
+ dctx->HUFptr, ZSTD_DCtx_get_bmi2(dctx));
}
} else {
if (singleStream) {
hufSuccess = HUF_decompress1X1_DCtx_wksp_bmi2(
dctx->entropy.hufTable, dctx->litBuffer, litSize,
istart+lhSize, litCSize, dctx->workspace,
- sizeof(dctx->workspace), dctx->bmi2);
+ sizeof(dctx->workspace), ZSTD_DCtx_get_bmi2(dctx));
#endif
} else {
hufSuccess = HUF_decompress4X_hufOnly_wksp_bmi2(
dctx->entropy.hufTable, dctx->litBuffer, litSize,
istart+lhSize, litCSize, dctx->workspace,
- sizeof(dctx->workspace), dctx->bmi2);
+ sizeof(dctx->workspace), ZSTD_DCtx_get_bmi2(dctx));
}
}
+ if (dctx->litBufferLocation == ZSTD_split)
+ {
+ ZSTD_memcpy(dctx->litExtraBuffer, dctx->litBufferEnd - ZSTD_LITBUFFEREXTRASIZE, ZSTD_LITBUFFEREXTRASIZE);
+ ZSTD_memmove(dctx->litBuffer + ZSTD_LITBUFFEREXTRASIZE - WILDCOPY_OVERLENGTH, dctx->litBuffer, litSize - ZSTD_LITBUFFEREXTRASIZE);
+ dctx->litBuffer += ZSTD_LITBUFFEREXTRASIZE - WILDCOPY_OVERLENGTH;
+ dctx->litBufferEnd -= WILDCOPY_OVERLENGTH;
+ }
RETURN_ERROR_IF(HUF_isError(hufSuccess), corruption_detected, "");
dctx->litSize = litSize;
dctx->litEntropy = 1;
if (litEncType==set_compressed) dctx->HUFptr = dctx->entropy.hufTable;
- ZSTD_memset(dctx->litBuffer + dctx->litSize, 0, WILDCOPY_OVERLENGTH);
return litCSize + lhSize;
}
case set_basic:
{ size_t litSize, lhSize;
U32 const lhlCode = ((istart[0]) >> 2) & 3;
+ size_t expectedWriteSize = MIN(ZSTD_BLOCKSIZE_MAX, dstCapacity);
switch(lhlCode)
{
case 0: case 2: default: /* note : default is impossible, since lhlCode into [0..3] */
break;
}
+ RETURN_ERROR_IF(litSize > 0 && dst == NULL, dstSize_tooSmall, "NULL not handled");
+ RETURN_ERROR_IF(expectedWriteSize < litSize, dstSize_tooSmall, "");
+ ZSTD_allocateLiteralsBuffer(dctx, dst, dstCapacity, litSize, streaming, expectedWriteSize, 1);
if (lhSize+litSize+WILDCOPY_OVERLENGTH > srcSize) { /* risk reading beyond src buffer with wildcopy */
RETURN_ERROR_IF(litSize+lhSize > srcSize, corruption_detected, "");
- ZSTD_memcpy(dctx->litBuffer, istart+lhSize, litSize);
+ if (dctx->litBufferLocation == ZSTD_split)
+ {
+ ZSTD_memcpy(dctx->litBuffer, istart + lhSize, litSize - ZSTD_LITBUFFEREXTRASIZE);
+ ZSTD_memcpy(dctx->litExtraBuffer, istart + lhSize + litSize - ZSTD_LITBUFFEREXTRASIZE, ZSTD_LITBUFFEREXTRASIZE);
+ }
+ else
+ {
+ ZSTD_memcpy(dctx->litBuffer, istart + lhSize, litSize);
+ }
dctx->litPtr = dctx->litBuffer;
dctx->litSize = litSize;
- ZSTD_memset(dctx->litBuffer + dctx->litSize, 0, WILDCOPY_OVERLENGTH);
return lhSize+litSize;
}
/* direct reference into compressed stream */
dctx->litPtr = istart+lhSize;
dctx->litSize = litSize;
+ dctx->litBufferEnd = dctx->litPtr + litSize;
+ dctx->litBufferLocation = ZSTD_not_in_dst;
return lhSize+litSize;
}
case set_rle:
{ U32 const lhlCode = ((istart[0]) >> 2) & 3;
size_t litSize, lhSize;
+ size_t expectedWriteSize = MIN(ZSTD_BLOCKSIZE_MAX, dstCapacity);
switch(lhlCode)
{
case 0: case 2: default: /* note : default is impossible, since lhlCode into [0..3] */
RETURN_ERROR_IF(srcSize<4, corruption_detected, "srcSize >= MIN_CBLOCK_SIZE == 3; here we need lhSize+1 = 4");
break;
}
+ RETURN_ERROR_IF(litSize > 0 && dst == NULL, dstSize_tooSmall, "NULL not handled");
RETURN_ERROR_IF(litSize > ZSTD_BLOCKSIZE_MAX, corruption_detected, "");
- ZSTD_memset(dctx->litBuffer, istart[lhSize], litSize + WILDCOPY_OVERLENGTH);
+ RETURN_ERROR_IF(expectedWriteSize < litSize, dstSize_tooSmall, "");
+ ZSTD_allocateLiteralsBuffer(dctx, dst, dstCapacity, litSize, streaming, expectedWriteSize, 1);
+ if (dctx->litBufferLocation == ZSTD_split)
+ {
+ ZSTD_memset(dctx->litBuffer, istart[lhSize], litSize - ZSTD_LITBUFFEREXTRASIZE);
+ ZSTD_memset(dctx->litExtraBuffer, istart[lhSize], ZSTD_LITBUFFEREXTRASIZE);
+ }
+ else
+ {
+ ZSTD_memset(dctx->litBuffer, istart[lhSize], litSize);
+ }
dctx->litPtr = dctx->litBuffer;
dctx->litSize = litSize;
return lhSize+1;
}; /* ML_defaultDTable */
-static void ZSTD_buildSeqTable_rle(ZSTD_seqSymbol* dt, U32 baseValue, U32 nbAddBits)
+static void ZSTD_buildSeqTable_rle(ZSTD_seqSymbol* dt, U32 baseValue, U8 nbAddBits)
{
void* ptr = dt;
ZSTD_seqSymbol_header* const DTableH = (ZSTD_seqSymbol_header*)ptr;
cell->nbBits = 0;
cell->nextState = 0;
assert(nbAddBits < 255);
- cell->nbAdditionalBits = (BYTE)nbAddBits;
+ cell->nbAdditionalBits = nbAddBits;
cell->baseValue = baseValue;
}
FORCE_INLINE_TEMPLATE
void ZSTD_buildFSETable_body(ZSTD_seqSymbol* dt,
const short* normalizedCounter, unsigned maxSymbolValue,
- const U32* baseValue, const U32* nbAdditionalBits,
+ const U32* baseValue, const U8* nbAdditionalBits,
unsigned tableLog, void* wksp, size_t wkspSize)
{
ZSTD_seqSymbol* const tableDecode = dt+1;
tableDecode[u].nbBits = (BYTE) (tableLog - BIT_highbit32(nextState) );
tableDecode[u].nextState = (U16) ( (nextState << tableDecode[u].nbBits) - tableSize);
assert(nbAdditionalBits[symbol] < 255);
- tableDecode[u].nbAdditionalBits = (BYTE)nbAdditionalBits[symbol];
+ tableDecode[u].nbAdditionalBits = nbAdditionalBits[symbol];
tableDecode[u].baseValue = baseValue[symbol];
}
}
/* Avoids the FORCE_INLINE of the _body() function. */
static void ZSTD_buildFSETable_body_default(ZSTD_seqSymbol* dt,
const short* normalizedCounter, unsigned maxSymbolValue,
- const U32* baseValue, const U32* nbAdditionalBits,
+ const U32* baseValue, const U8* nbAdditionalBits,
unsigned tableLog, void* wksp, size_t wkspSize)
{
ZSTD_buildFSETable_body(dt, normalizedCounter, maxSymbolValue,
}
#if DYNAMIC_BMI2
-TARGET_ATTRIBUTE("bmi2") static void ZSTD_buildFSETable_body_bmi2(ZSTD_seqSymbol* dt,
+BMI2_TARGET_ATTRIBUTE static void ZSTD_buildFSETable_body_bmi2(ZSTD_seqSymbol* dt,
const short* normalizedCounter, unsigned maxSymbolValue,
- const U32* baseValue, const U32* nbAdditionalBits,
+ const U32* baseValue, const U8* nbAdditionalBits,
unsigned tableLog, void* wksp, size_t wkspSize)
{
ZSTD_buildFSETable_body(dt, normalizedCounter, maxSymbolValue,
void ZSTD_buildFSETable(ZSTD_seqSymbol* dt,
const short* normalizedCounter, unsigned maxSymbolValue,
- const U32* baseValue, const U32* nbAdditionalBits,
+ const U32* baseValue, const U8* nbAdditionalBits,
unsigned tableLog, void* wksp, size_t wkspSize, int bmi2)
{
#if DYNAMIC_BMI2
static size_t ZSTD_buildSeqTable(ZSTD_seqSymbol* DTableSpace, const ZSTD_seqSymbol** DTablePtr,
symbolEncodingType_e type, unsigned max, U32 maxLog,
const void* src, size_t srcSize,
- const U32* baseValue, const U32* nbAdditionalBits,
+ const U32* baseValue, const U8* nbAdditionalBits,
const ZSTD_seqSymbol* defaultTable, U32 flagRepeatTable,
int ddictIsCold, int nbSeq, U32* wksp, size_t wkspSize,
int bmi2)
RETURN_ERROR_IF((*(const BYTE*)src) > max, corruption_detected, "");
{ U32 const symbol = *(const BYTE*)src;
U32 const baseline = baseValue[symbol];
- U32 const nbBits = nbAdditionalBits[symbol];
+ U8 const nbBits = nbAdditionalBits[symbol];
ZSTD_buildSeqTable_rle(DTableSpace, baseline, nbBits);
}
*DTablePtr = DTableSpace;
LL_defaultDTable, dctx->fseEntropy,
dctx->ddictIsCold, nbSeq,
dctx->workspace, sizeof(dctx->workspace),
- dctx->bmi2);
+ ZSTD_DCtx_get_bmi2(dctx));
RETURN_ERROR_IF(ZSTD_isError(llhSize), corruption_detected, "ZSTD_buildSeqTable failed");
ip += llhSize;
}
OF_defaultDTable, dctx->fseEntropy,
dctx->ddictIsCold, nbSeq,
dctx->workspace, sizeof(dctx->workspace),
- dctx->bmi2);
+ ZSTD_DCtx_get_bmi2(dctx));
RETURN_ERROR_IF(ZSTD_isError(ofhSize), corruption_detected, "ZSTD_buildSeqTable failed");
ip += ofhSize;
}
ML_defaultDTable, dctx->fseEntropy,
dctx->ddictIsCold, nbSeq,
dctx->workspace, sizeof(dctx->workspace),
- dctx->bmi2);
+ ZSTD_DCtx_get_bmi2(dctx));
RETURN_ERROR_IF(ZSTD_isError(mlhSize), corruption_detected, "ZSTD_buildSeqTable failed");
ip += mlhSize;
}
size_t litLength;
size_t matchLength;
size_t offset;
- const BYTE* match;
} seq_t;
typedef struct {
ZSTD_fseState stateOffb;
ZSTD_fseState stateML;
size_t prevOffset[ZSTD_REP_NUM];
- const BYTE* prefixStart;
- const BYTE* dictEnd;
- size_t pos;
} seqState_t;
/*! ZSTD_overlapCopy8() :
* - ZSTD_overlap_src_before_dst: The src and dst may overlap and may be any distance apart.
* The src buffer must be before the dst buffer.
*/
-static void ZSTD_safecopy(BYTE* op, BYTE* const oend_w, BYTE const* ip, ptrdiff_t length, ZSTD_overlap_e ovtype) {
+static void ZSTD_safecopy(BYTE* op, const BYTE* const oend_w, BYTE const* ip, ptrdiff_t length, ZSTD_overlap_e ovtype) {
ptrdiff_t const diff = op - ip;
BYTE* const oend = op + length;
/* Copy 8 bytes and ensure the offset >= 8 when there can be overlap. */
assert(length >= 8);
ZSTD_overlapCopy8(&op, &ip, diff);
+ length -= 8;
assert(op - ip >= 8);
assert(op <= oend);
}
assert(oend > oend_w);
ZSTD_wildcopy(op, ip, oend_w - op, ovtype);
ip += oend_w - op;
- op = oend_w;
+ op += oend_w - op;
+ }
+ /* Handle the leftovers. */
+ while (op < oend) *op++ = *ip++;
+}
+
+/* ZSTD_safecopyDstBeforeSrc():
+ * This version allows overlap with dst before src, or handles the non-overlap case with dst after src
+ * Kept separate from more common ZSTD_safecopy case to avoid performance impact to the safecopy common case */
+static void ZSTD_safecopyDstBeforeSrc(BYTE* op, BYTE const* ip, ptrdiff_t length) {
+ ptrdiff_t const diff = op - ip;
+ BYTE* const oend = op + length;
+
+ if (length < 8 || diff > -8) {
+ /* Handle short lengths, close overlaps, and dst not before src. */
+ while (op < oend) *op++ = *ip++;
+ return;
+ }
+
+ if (op <= oend - WILDCOPY_OVERLENGTH && diff < -WILDCOPY_VECLEN) {
+ ZSTD_wildcopy(op, ip, oend - WILDCOPY_OVERLENGTH - op, ZSTD_no_overlap);
+ ip += oend - WILDCOPY_OVERLENGTH - op;
+ op += oend - WILDCOPY_OVERLENGTH - op;
}
+
/* Handle the leftovers. */
while (op < oend) *op++ = *ip++;
}
*/
FORCE_NOINLINE
size_t ZSTD_execSequenceEnd(BYTE* op,
- BYTE* const oend, seq_t sequence,
- const BYTE** litPtr, const BYTE* const litLimit,
- const BYTE* const prefixStart, const BYTE* const virtualStart, const BYTE* const dictEnd)
+ BYTE* const oend, seq_t sequence,
+ const BYTE** litPtr, const BYTE* const litLimit,
+ const BYTE* const prefixStart, const BYTE* const virtualStart, const BYTE* const dictEnd)
{
BYTE* const oLitEnd = op + sequence.litLength;
size_t const sequenceLength = sequence.litLength + sequence.matchLength;
if (sequence.offset > (size_t)(oLitEnd - prefixStart)) {
/* offset beyond prefix */
RETURN_ERROR_IF(sequence.offset > (size_t)(oLitEnd - virtualStart), corruption_detected, "");
- match = dictEnd - (prefixStart-match);
+ match = dictEnd - (prefixStart - match);
if (match + sequence.matchLength <= dictEnd) {
ZSTD_memmove(oLitEnd, match, sequence.matchLength);
return sequenceLength;
}
/* span extDict & currentPrefixSegment */
{ size_t const length1 = dictEnd - match;
- ZSTD_memmove(oLitEnd, match, length1);
- op = oLitEnd + length1;
- sequence.matchLength -= length1;
- match = prefixStart;
- } }
+ ZSTD_memmove(oLitEnd, match, length1);
+ op = oLitEnd + length1;
+ sequence.matchLength -= length1;
+ match = prefixStart;
+ }
+ }
+ ZSTD_safecopy(op, oend_w, match, sequence.matchLength, ZSTD_overlap_src_before_dst);
+ return sequenceLength;
+}
+
+/* ZSTD_execSequenceEndSplitLitBuffer():
+ * This version is intended to be used during instances where the litBuffer is still split. It is kept separate to avoid performance impact for the good case.
+ */
+FORCE_NOINLINE
+size_t ZSTD_execSequenceEndSplitLitBuffer(BYTE* op,
+ BYTE* const oend, const BYTE* const oend_w, seq_t sequence,
+ const BYTE** litPtr, const BYTE* const litLimit,
+ const BYTE* const prefixStart, const BYTE* const virtualStart, const BYTE* const dictEnd)
+{
+ BYTE* const oLitEnd = op + sequence.litLength;
+ size_t const sequenceLength = sequence.litLength + sequence.matchLength;
+ const BYTE* const iLitEnd = *litPtr + sequence.litLength;
+ const BYTE* match = oLitEnd - sequence.offset;
+
+
+ /* bounds checks : careful of address space overflow in 32-bit mode */
+ RETURN_ERROR_IF(sequenceLength > (size_t)(oend - op), dstSize_tooSmall, "last match must fit within dstBuffer");
+ RETURN_ERROR_IF(sequence.litLength > (size_t)(litLimit - *litPtr), corruption_detected, "try to read beyond literal buffer");
+ assert(op < op + sequenceLength);
+ assert(oLitEnd < op + sequenceLength);
+
+ /* copy literals */
+ RETURN_ERROR_IF(op > *litPtr && op < *litPtr + sequence.litLength, dstSize_tooSmall, "output should not catch up to and overwrite literal buffer");
+ ZSTD_safecopyDstBeforeSrc(op, *litPtr, sequence.litLength);
+ op = oLitEnd;
+ *litPtr = iLitEnd;
+
+ /* copy Match */
+ if (sequence.offset > (size_t)(oLitEnd - prefixStart)) {
+ /* offset beyond prefix */
+ RETURN_ERROR_IF(sequence.offset > (size_t)(oLitEnd - virtualStart), corruption_detected, "");
+ match = dictEnd - (prefixStart - match);
+ if (match + sequence.matchLength <= dictEnd) {
+ ZSTD_memmove(oLitEnd, match, sequence.matchLength);
+ return sequenceLength;
+ }
+ /* span extDict & currentPrefixSegment */
+ { size_t const length1 = dictEnd - match;
+ ZSTD_memmove(oLitEnd, match, length1);
+ op = oLitEnd + length1;
+ sequence.matchLength -= length1;
+ match = prefixStart;
+ }
+ }
ZSTD_safecopy(op, oend_w, match, sequence.matchLength, ZSTD_overlap_src_before_dst);
return sequenceLength;
}
HINT_INLINE
size_t ZSTD_execSequence(BYTE* op,
- BYTE* const oend, seq_t sequence,
- const BYTE** litPtr, const BYTE* const litLimit,
- const BYTE* const prefixStart, const BYTE* const virtualStart, const BYTE* const dictEnd)
+ BYTE* const oend, seq_t sequence,
+ const BYTE** litPtr, const BYTE* const litLimit,
+ const BYTE* const prefixStart, const BYTE* const virtualStart, const BYTE* const dictEnd)
{
BYTE* const oLitEnd = op + sequence.litLength;
size_t const sequenceLength = sequence.litLength + sequence.matchLength;
* - 32-bit mode and the match length overflows
*/
if (UNLIKELY(
+ iLitEnd > litLimit ||
+ oMatchEnd > oend_w ||
+ (MEM_32bits() && (size_t)(oend - op) < sequenceLength + WILDCOPY_OVERLENGTH)))
+ return ZSTD_execSequenceEnd(op, oend, sequence, litPtr, litLimit, prefixStart, virtualStart, dictEnd);
+
+ /* Assumptions (everything else goes into ZSTD_execSequenceEnd()) */
+ assert(op <= oLitEnd /* No overflow */);
+ assert(oLitEnd < oMatchEnd /* Non-zero match & no overflow */);
+ assert(oMatchEnd <= oend /* No underflow */);
+ assert(iLitEnd <= litLimit /* Literal length is in bounds */);
+ assert(oLitEnd <= oend_w /* Can wildcopy literals */);
+ assert(oMatchEnd <= oend_w /* Can wildcopy matches */);
+
+ /* Copy Literals:
+ * Split out litLength <= 16 since it is nearly always true. +1.6% on gcc-9.
+ * We likely don't need the full 32-byte wildcopy.
+ */
+ assert(WILDCOPY_OVERLENGTH >= 16);
+ ZSTD_copy16(op, (*litPtr));
+ if (UNLIKELY(sequence.litLength > 16)) {
+ ZSTD_wildcopy(op + 16, (*litPtr) + 16, sequence.litLength - 16, ZSTD_no_overlap);
+ }
+ op = oLitEnd;
+ *litPtr = iLitEnd; /* update for next sequence */
+
+ /* Copy Match */
+ if (sequence.offset > (size_t)(oLitEnd - prefixStart)) {
+ /* offset beyond prefix -> go into extDict */
+ RETURN_ERROR_IF(UNLIKELY(sequence.offset > (size_t)(oLitEnd - virtualStart)), corruption_detected, "");
+ match = dictEnd + (match - prefixStart);
+ if (match + sequence.matchLength <= dictEnd) {
+ ZSTD_memmove(oLitEnd, match, sequence.matchLength);
+ return sequenceLength;
+ }
+ /* span extDict & currentPrefixSegment */
+ { size_t const length1 = dictEnd - match;
+ ZSTD_memmove(oLitEnd, match, length1);
+ op = oLitEnd + length1;
+ sequence.matchLength -= length1;
+ match = prefixStart;
+ }
+ }
+ /* Match within prefix of 1 or more bytes */
+ assert(op <= oMatchEnd);
+ assert(oMatchEnd <= oend_w);
+ assert(match >= prefixStart);
+ assert(sequence.matchLength >= 1);
+
+ /* Nearly all offsets are >= WILDCOPY_VECLEN bytes, which means we can use wildcopy
+ * without overlap checking.
+ */
+ if (LIKELY(sequence.offset >= WILDCOPY_VECLEN)) {
+ /* We bet on a full wildcopy for matches, since we expect matches to be
+ * longer than literals (in general). In silesia, ~10% of matches are longer
+ * than 16 bytes.
+ */
+ ZSTD_wildcopy(op, match, (ptrdiff_t)sequence.matchLength, ZSTD_no_overlap);
+ return sequenceLength;
+ }
+ assert(sequence.offset < WILDCOPY_VECLEN);
+
+ /* Copy 8 bytes and spread the offset to be >= 8. */
+ ZSTD_overlapCopy8(&op, &match, sequence.offset);
+
+ /* If the match length is > 8 bytes, then continue with the wildcopy. */
+ if (sequence.matchLength > 8) {
+ assert(op < oMatchEnd);
+ ZSTD_wildcopy(op, match, (ptrdiff_t)sequence.matchLength - 8, ZSTD_overlap_src_before_dst);
+ }
+ return sequenceLength;
+}
+
+HINT_INLINE
+size_t ZSTD_execSequenceSplitLitBuffer(BYTE* op,
+ BYTE* const oend, const BYTE* const oend_w, seq_t sequence,
+ const BYTE** litPtr, const BYTE* const litLimit,
+ const BYTE* const prefixStart, const BYTE* const virtualStart, const BYTE* const dictEnd)
+{
+ BYTE* const oLitEnd = op + sequence.litLength;
+ size_t const sequenceLength = sequence.litLength + sequence.matchLength;
+ BYTE* const oMatchEnd = op + sequenceLength; /* risk : address space overflow (32-bits) */
+ const BYTE* const iLitEnd = *litPtr + sequence.litLength;
+ const BYTE* match = oLitEnd - sequence.offset;
+
+ assert(op != NULL /* Precondition */);
+ assert(oend_w < oend /* No underflow */);
+ /* Handle edge cases in a slow path:
+ * - Read beyond end of literals
+ * - Match end is within WILDCOPY_OVERLIMIT of oend
+ * - 32-bit mode and the match length overflows
+ */
+ if (UNLIKELY(
iLitEnd > litLimit ||
oMatchEnd > oend_w ||
(MEM_32bits() && (size_t)(oend - op) < sequenceLength + WILDCOPY_OVERLENGTH)))
- return ZSTD_execSequenceEnd(op, oend, sequence, litPtr, litLimit, prefixStart, virtualStart, dictEnd);
+ return ZSTD_execSequenceEndSplitLitBuffer(op, oend, oend_w, sequence, litPtr, litLimit, prefixStart, virtualStart, dictEnd);
/* Assumptions (everything else goes into ZSTD_execSequenceEnd()) */
assert(op <= oLitEnd /* No overflow */);
return sequenceLength;
}
+
static void
ZSTD_initFseState(ZSTD_fseState* DStatePtr, BIT_DStream_t* bitD, const ZSTD_seqSymbol* dt)
{
}
FORCE_INLINE_TEMPLATE void
-ZSTD_updateFseState(ZSTD_fseState* DStatePtr, BIT_DStream_t* bitD)
-{
- ZSTD_seqSymbol const DInfo = DStatePtr->table[DStatePtr->state];
- U32 const nbBits = DInfo.nbBits;
- size_t const lowBits = BIT_readBits(bitD, nbBits);
- DStatePtr->state = DInfo.nextState + lowBits;
-}
-
-FORCE_INLINE_TEMPLATE void
-ZSTD_updateFseStateWithDInfo(ZSTD_fseState* DStatePtr, BIT_DStream_t* bitD, ZSTD_seqSymbol const DInfo)
+ZSTD_updateFseStateWithDInfo(ZSTD_fseState* DStatePtr, BIT_DStream_t* bitD, U16 nextState, U32 nbBits)
{
- U32 const nbBits = DInfo.nbBits;
size_t const lowBits = BIT_readBits(bitD, nbBits);
- DStatePtr->state = DInfo.nextState + lowBits;
+ DStatePtr->state = nextState + lowBits;
}
/* We need to add at most (ZSTD_WINDOWLOG_MAX_32 - 1) bits to read the maximum
: 0)
typedef enum { ZSTD_lo_isRegularOffset, ZSTD_lo_isLongOffset=1 } ZSTD_longOffset_e;
-typedef enum { ZSTD_p_noPrefetch=0, ZSTD_p_prefetch=1 } ZSTD_prefetch_e;
FORCE_INLINE_TEMPLATE seq_t
-ZSTD_decodeSequence(seqState_t* seqState, const ZSTD_longOffset_e longOffsets, const ZSTD_prefetch_e prefetch)
+ZSTD_decodeSequence(seqState_t* seqState, const ZSTD_longOffset_e longOffsets)
{
seq_t seq;
- ZSTD_seqSymbol const llDInfo = seqState->stateLL.table[seqState->stateLL.state];
- ZSTD_seqSymbol const mlDInfo = seqState->stateML.table[seqState->stateML.state];
- ZSTD_seqSymbol const ofDInfo = seqState->stateOffb.table[seqState->stateOffb.state];
- U32 const llBase = llDInfo.baseValue;
- U32 const mlBase = mlDInfo.baseValue;
- U32 const ofBase = ofDInfo.baseValue;
- BYTE const llBits = llDInfo.nbAdditionalBits;
- BYTE const mlBits = mlDInfo.nbAdditionalBits;
- BYTE const ofBits = ofDInfo.nbAdditionalBits;
- BYTE const totalBits = llBits+mlBits+ofBits;
-
- /* sequence */
- { size_t offset;
- if (ofBits > 1) {
- ZSTD_STATIC_ASSERT(ZSTD_lo_isLongOffset == 1);
- ZSTD_STATIC_ASSERT(LONG_OFFSETS_MAX_EXTRA_BITS_32 == 5);
- assert(ofBits <= MaxOff);
- if (MEM_32bits() && longOffsets && (ofBits >= STREAM_ACCUMULATOR_MIN_32)) {
- U32 const extraBits = ofBits - MIN(ofBits, 32 - seqState->DStream.bitsConsumed);
- offset = ofBase + (BIT_readBitsFast(&seqState->DStream, ofBits - extraBits) << extraBits);
- BIT_reloadDStream(&seqState->DStream);
- if (extraBits) offset += BIT_readBitsFast(&seqState->DStream, extraBits);
- assert(extraBits <= LONG_OFFSETS_MAX_EXTRA_BITS_32); /* to avoid another reload */
- } else {
- offset = ofBase + BIT_readBitsFast(&seqState->DStream, ofBits/*>0*/); /* <= (ZSTD_WINDOWLOG_MAX-1) bits */
- if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream);
- }
- seqState->prevOffset[2] = seqState->prevOffset[1];
- seqState->prevOffset[1] = seqState->prevOffset[0];
- seqState->prevOffset[0] = offset;
- } else {
- U32 const ll0 = (llBase == 0);
- if (LIKELY((ofBits == 0))) {
- if (LIKELY(!ll0))
- offset = seqState->prevOffset[0];
- else {
- offset = seqState->prevOffset[1];
- seqState->prevOffset[1] = seqState->prevOffset[0];
- seqState->prevOffset[0] = offset;
+ const ZSTD_seqSymbol* const llDInfo = seqState->stateLL.table + seqState->stateLL.state;
+ const ZSTD_seqSymbol* const mlDInfo = seqState->stateML.table + seqState->stateML.state;
+ const ZSTD_seqSymbol* const ofDInfo = seqState->stateOffb.table + seqState->stateOffb.state;
+ seq.matchLength = mlDInfo->baseValue;
+ seq.litLength = llDInfo->baseValue;
+ { U32 const ofBase = ofDInfo->baseValue;
+ BYTE const llBits = llDInfo->nbAdditionalBits;
+ BYTE const mlBits = mlDInfo->nbAdditionalBits;
+ BYTE const ofBits = ofDInfo->nbAdditionalBits;
+ BYTE const totalBits = llBits+mlBits+ofBits;
+
+ U16 const llNext = llDInfo->nextState;
+ U16 const mlNext = mlDInfo->nextState;
+ U16 const ofNext = ofDInfo->nextState;
+ U32 const llnbBits = llDInfo->nbBits;
+ U32 const mlnbBits = mlDInfo->nbBits;
+ U32 const ofnbBits = ofDInfo->nbBits;
+ /*
+ * As gcc has better branch and block analyzers, sometimes it is only
+ * valuable to mark likelyness for clang, it gives around 3-4% of
+ * performance.
+ */
+
+ /* sequence */
+ { size_t offset;
+ #if defined(__clang__)
+ if (LIKELY(ofBits > 1)) {
+ #else
+ if (ofBits > 1) {
+ #endif
+ ZSTD_STATIC_ASSERT(ZSTD_lo_isLongOffset == 1);
+ ZSTD_STATIC_ASSERT(LONG_OFFSETS_MAX_EXTRA_BITS_32 == 5);
+ assert(ofBits <= MaxOff);
+ if (MEM_32bits() && longOffsets && (ofBits >= STREAM_ACCUMULATOR_MIN_32)) {
+ U32 const extraBits = ofBits - MIN(ofBits, 32 - seqState->DStream.bitsConsumed);
+ offset = ofBase + (BIT_readBitsFast(&seqState->DStream, ofBits - extraBits) << extraBits);
+ BIT_reloadDStream(&seqState->DStream);
+ if (extraBits) offset += BIT_readBitsFast(&seqState->DStream, extraBits);
+ assert(extraBits <= LONG_OFFSETS_MAX_EXTRA_BITS_32); /* to avoid another reload */
+ } else {
+ offset = ofBase + BIT_readBitsFast(&seqState->DStream, ofBits/*>0*/); /* <= (ZSTD_WINDOWLOG_MAX-1) bits */
+ if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream);
}
+ seqState->prevOffset[2] = seqState->prevOffset[1];
+ seqState->prevOffset[1] = seqState->prevOffset[0];
+ seqState->prevOffset[0] = offset;
} else {
- offset = ofBase + ll0 + BIT_readBitsFast(&seqState->DStream, 1);
- { size_t temp = (offset==3) ? seqState->prevOffset[0] - 1 : seqState->prevOffset[offset];
- temp += !temp; /* 0 is not valid; input is corrupted; force offset to 1 */
- if (offset != 1) seqState->prevOffset[2] = seqState->prevOffset[1];
- seqState->prevOffset[1] = seqState->prevOffset[0];
- seqState->prevOffset[0] = offset = temp;
- } } }
- seq.offset = offset;
- }
-
- seq.matchLength = mlBase;
- if (mlBits > 0)
- seq.matchLength += BIT_readBitsFast(&seqState->DStream, mlBits/*>0*/);
-
- if (MEM_32bits() && (mlBits+llBits >= STREAM_ACCUMULATOR_MIN_32-LONG_OFFSETS_MAX_EXTRA_BITS_32))
- BIT_reloadDStream(&seqState->DStream);
- if (MEM_64bits() && UNLIKELY(totalBits >= STREAM_ACCUMULATOR_MIN_64-(LLFSELog+MLFSELog+OffFSELog)))
- BIT_reloadDStream(&seqState->DStream);
- /* Ensure there are enough bits to read the rest of data in 64-bit mode. */
- ZSTD_STATIC_ASSERT(16+LLFSELog+MLFSELog+OffFSELog < STREAM_ACCUMULATOR_MIN_64);
-
- seq.litLength = llBase;
- if (llBits > 0)
- seq.litLength += BIT_readBitsFast(&seqState->DStream, llBits/*>0*/);
-
- if (MEM_32bits())
- BIT_reloadDStream(&seqState->DStream);
-
- DEBUGLOG(6, "seq: litL=%u, matchL=%u, offset=%u",
- (U32)seq.litLength, (U32)seq.matchLength, (U32)seq.offset);
-
- if (prefetch == ZSTD_p_prefetch) {
- size_t const pos = seqState->pos + seq.litLength;
- const BYTE* const matchBase = (seq.offset > pos) ? seqState->dictEnd : seqState->prefixStart;
- seq.match = matchBase + pos - seq.offset; /* note : this operation can overflow when seq.offset is really too large, which can only happen when input is corrupted.
- * No consequence though : no memory access will occur, offset is only used for prefetching */
- seqState->pos = pos + seq.matchLength;
- }
-
- /* ANS state update
- * gcc-9.0.0 does 2.5% worse with ZSTD_updateFseStateWithDInfo().
- * clang-9.2.0 does 7% worse with ZSTD_updateFseState().
- * Naturally it seems like ZSTD_updateFseStateWithDInfo() should be the
- * better option, so it is the default for other compilers. But, if you
- * measure that it is worse, please put up a pull request.
- */
- {
-#if !defined(__clang__)
- const int kUseUpdateFseState = 1;
-#else
- const int kUseUpdateFseState = 0;
-#endif
- if (kUseUpdateFseState) {
- ZSTD_updateFseState(&seqState->stateLL, &seqState->DStream); /* <= 9 bits */
- ZSTD_updateFseState(&seqState->stateML, &seqState->DStream); /* <= 9 bits */
- if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream); /* <= 18 bits */
- ZSTD_updateFseState(&seqState->stateOffb, &seqState->DStream); /* <= 8 bits */
- } else {
- ZSTD_updateFseStateWithDInfo(&seqState->stateLL, &seqState->DStream, llDInfo); /* <= 9 bits */
- ZSTD_updateFseStateWithDInfo(&seqState->stateML, &seqState->DStream, mlDInfo); /* <= 9 bits */
- if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream); /* <= 18 bits */
- ZSTD_updateFseStateWithDInfo(&seqState->stateOffb, &seqState->DStream, ofDInfo); /* <= 8 bits */
+ U32 const ll0 = (llDInfo->baseValue == 0);
+ if (LIKELY((ofBits == 0))) {
+ offset = seqState->prevOffset[ll0];
+ seqState->prevOffset[1] = seqState->prevOffset[!ll0];
+ seqState->prevOffset[0] = offset;
+ } else {
+ offset = ofBase + ll0 + BIT_readBitsFast(&seqState->DStream, 1);
+ { size_t temp = (offset==3) ? seqState->prevOffset[0] - 1 : seqState->prevOffset[offset];
+ temp += !temp; /* 0 is not valid; input is corrupted; force offset to 1 */
+ if (offset != 1) seqState->prevOffset[2] = seqState->prevOffset[1];
+ seqState->prevOffset[1] = seqState->prevOffset[0];
+ seqState->prevOffset[0] = offset = temp;
+ } } }
+ seq.offset = offset;
}
+
+ #if defined(__clang__)
+ if (UNLIKELY(mlBits > 0))
+ #else
+ if (mlBits > 0)
+ #endif
+ seq.matchLength += BIT_readBitsFast(&seqState->DStream, mlBits/*>0*/);
+
+ if (MEM_32bits() && (mlBits+llBits >= STREAM_ACCUMULATOR_MIN_32-LONG_OFFSETS_MAX_EXTRA_BITS_32))
+ BIT_reloadDStream(&seqState->DStream);
+ if (MEM_64bits() && UNLIKELY(totalBits >= STREAM_ACCUMULATOR_MIN_64-(LLFSELog+MLFSELog+OffFSELog)))
+ BIT_reloadDStream(&seqState->DStream);
+ /* Ensure there are enough bits to read the rest of data in 64-bit mode. */
+ ZSTD_STATIC_ASSERT(16+LLFSELog+MLFSELog+OffFSELog < STREAM_ACCUMULATOR_MIN_64);
+
+ #if defined(__clang__)
+ if (UNLIKELY(llBits > 0))
+ #else
+ if (llBits > 0)
+ #endif
+ seq.litLength += BIT_readBitsFast(&seqState->DStream, llBits/*>0*/);
+
+ if (MEM_32bits())
+ BIT_reloadDStream(&seqState->DStream);
+
+ DEBUGLOG(6, "seq: litL=%u, matchL=%u, offset=%u",
+ (U32)seq.litLength, (U32)seq.matchLength, (U32)seq.offset);
+
+ ZSTD_updateFseStateWithDInfo(&seqState->stateLL, &seqState->DStream, llNext, llnbBits); /* <= 9 bits */
+ ZSTD_updateFseStateWithDInfo(&seqState->stateML, &seqState->DStream, mlNext, mlnbBits); /* <= 9 bits */
+ if (MEM_32bits()) BIT_reloadDStream(&seqState->DStream); /* <= 18 bits */
+ ZSTD_updateFseStateWithDInfo(&seqState->stateOffb, &seqState->DStream, ofNext, ofnbBits); /* <= 8 bits */
}
return seq;
#endif
#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG
+
+
FORCE_INLINE_TEMPLATE size_t
DONT_VECTORIZE
-ZSTD_decompressSequences_body( ZSTD_DCtx* dctx,
+ZSTD_decompressSequences_bodySplitLitBuffer( ZSTD_DCtx* dctx,
void* dst, size_t maxDstSize,
const void* seqStart, size_t seqSize, int nbSeq,
const ZSTD_longOffset_e isLongOffset,
BYTE* const oend = ostart + maxDstSize;
BYTE* op = ostart;
const BYTE* litPtr = dctx->litPtr;
- const BYTE* const litEnd = litPtr + dctx->litSize;
+ const BYTE* litBufferEnd = dctx->litBufferEnd;
const BYTE* const prefixStart = (const BYTE*) (dctx->prefixStart);
const BYTE* const vBase = (const BYTE*) (dctx->virtualStart);
const BYTE* const dictEnd = (const BYTE*) (dctx->dictEnd);
- DEBUGLOG(5, "ZSTD_decompressSequences_body");
+ DEBUGLOG(5, "ZSTD_decompressSequences_bodySplitLitBuffer");
(void)frame;
/* Regen sequences */
if (nbSeq) {
seqState_t seqState;
- size_t error = 0;
dctx->fseEntropy = 1;
{ U32 i; for (i=0; i<ZSTD_REP_NUM; i++) seqState.prevOffset[i] = dctx->entropy.rep[i]; }
RETURN_ERROR_IF(
BIT_DStream_endOfBuffer < BIT_DStream_completed &&
BIT_DStream_completed < BIT_DStream_overflow);
+ /* decompress without overrunning litPtr begins */
+ {
+ seq_t sequence = ZSTD_decodeSequence(&seqState, isLongOffset);
+ /* Align the decompression loop to 32 + 16 bytes.
+ *
+ * zstd compiled with gcc-9 on an Intel i9-9900k shows 10% decompression
+ * speed swings based on the alignment of the decompression loop. This
+ * performance swing is caused by parts of the decompression loop falling
+ * out of the DSB. The entire decompression loop should fit in the DSB,
+ * when it can't we get much worse performance. You can measure if you've
+ * hit the good case or the bad case with this perf command for some
+ * compressed file test.zst:
+ *
+ * perf stat -e cycles -e instructions -e idq.all_dsb_cycles_any_uops \
+ * -e idq.all_mite_cycles_any_uops -- ./zstd -tq test.zst
+ *
+ * If you see most cycles served out of the MITE you've hit the bad case.
+ * If you see most cycles served out of the DSB you've hit the good case.
+ * If it is pretty even then you may be in an okay case.
+ *
+ * This issue has been reproduced on the following CPUs:
+ * - Kabylake: Macbook Pro (15-inch, 2019) 2.4 GHz Intel Core i9
+ * Use Instruments->Counters to get DSB/MITE cycles.
+ * I never got performance swings, but I was able to
+ * go from the good case of mostly DSB to half of the
+ * cycles served from MITE.
+ * - Coffeelake: Intel i9-9900k
+ * - Coffeelake: Intel i7-9700k
+ *
+ * I haven't been able to reproduce the instability or DSB misses on any
+ * of the following CPUS:
+ * - Haswell
+ * - Broadwell: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GH
+ * - Skylake
+ *
+ * Alignment is done for each of the three major decompression loops:
+ * - ZSTD_decompressSequences_bodySplitLitBuffer - presplit section of the literal buffer
+ * - ZSTD_decompressSequences_bodySplitLitBuffer - postsplit section of the literal buffer
+ * - ZSTD_decompressSequences_body
+ * Alignment choices are made to minimize large swings on bad cases and influence on performance
+ * from changes external to this code, rather than to overoptimize on the current commit.
+ *
+ * If you are seeing performance stability this script can help test.
+ * It tests on 4 commits in zstd where I saw performance change.
+ *
+ * https://gist.github.com/terrelln/9889fc06a423fd5ca6e99351564473f4
+ */
#if defined(__x86_64__)
- /* Align the decompression loop to 32 + 16 bytes.
- *
- * zstd compiled with gcc-9 on an Intel i9-9900k shows 10% decompression
- * speed swings based on the alignment of the decompression loop. This
- * performance swing is caused by parts of the decompression loop falling
- * out of the DSB. The entire decompression loop should fit in the DSB,
- * when it can't we get much worse performance. You can measure if you've
- * hit the good case or the bad case with this perf command for some
- * compressed file test.zst:
- *
- * perf stat -e cycles -e instructions -e idq.all_dsb_cycles_any_uops \
- * -e idq.all_mite_cycles_any_uops -- ./zstd -tq test.zst
- *
- * If you see most cycles served out of the MITE you've hit the bad case.
- * If you see most cycles served out of the DSB you've hit the good case.
- * If it is pretty even then you may be in an okay case.
- *
- * I've been able to reproduce this issue on the following CPUs:
- * - Kabylake: Macbook Pro (15-inch, 2019) 2.4 GHz Intel Core i9
- * Use Instruments->Counters to get DSB/MITE cycles.
- * I never got performance swings, but I was able to
- * go from the good case of mostly DSB to half of the
- * cycles served from MITE.
- * - Coffeelake: Intel i9-9900k
- *
- * I haven't been able to reproduce the instability or DSB misses on any
- * of the following CPUS:
- * - Haswell
- * - Broadwell: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GH
- * - Skylake
- *
- * If you are seeing performance stability this script can help test.
- * It tests on 4 commits in zstd where I saw performance change.
- *
- * https://gist.github.com/terrelln/9889fc06a423fd5ca6e99351564473f4
- */
- __asm__(".p2align 5");
- __asm__("nop");
- __asm__(".p2align 4");
+ __asm__(".p2align 6");
+# if __GNUC__ >= 7
+ /* good for gcc-7, gcc-9, and gcc-11 */
+ __asm__("nop");
+ __asm__(".p2align 5");
+ __asm__("nop");
+ __asm__(".p2align 4");
+# if __GNUC__ == 8 || __GNUC__ == 10
+ /* good for gcc-8 and gcc-10 */
+ __asm__("nop");
+ __asm__(".p2align 3");
+# endif
+# endif
+#endif
+
+ /* Handle the initial state where litBuffer is currently split between dst and litExtraBuffer */
+ for (; litPtr + sequence.litLength <= dctx->litBufferEnd; ) {
+ size_t const oneSeqSize = ZSTD_execSequenceSplitLitBuffer(op, oend, litPtr + sequence.litLength - WILDCOPY_OVERLENGTH, sequence, &litPtr, litBufferEnd, prefixStart, vBase, dictEnd);
+#if defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) && defined(FUZZING_ASSERT_VALID_SEQUENCE)
+ assert(!ZSTD_isError(oneSeqSize));
+ if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequence, prefixStart, vBase);
+#endif
+ if (UNLIKELY(ZSTD_isError(oneSeqSize)))
+ return oneSeqSize;
+ DEBUGLOG(6, "regenerated sequence size : %u", (U32)oneSeqSize);
+ op += oneSeqSize;
+ if (UNLIKELY(!--nbSeq))
+ break;
+ BIT_reloadDStream(&(seqState.DStream));
+ sequence = ZSTD_decodeSequence(&seqState, isLongOffset);
+ }
+
+ /* If there are more sequences, they will need to read literals from litExtraBuffer; copy over the remainder from dst and update litPtr and litEnd */
+ if (nbSeq > 0) {
+ const size_t leftoverLit = dctx->litBufferEnd - litPtr;
+ if (leftoverLit)
+ {
+ RETURN_ERROR_IF(leftoverLit > (size_t)(oend - op), dstSize_tooSmall, "remaining lit must fit within dstBuffer");
+ ZSTD_safecopyDstBeforeSrc(op, litPtr, leftoverLit);
+ sequence.litLength -= leftoverLit;
+ op += leftoverLit;
+ }
+ litPtr = dctx->litExtraBuffer;
+ litBufferEnd = dctx->litExtraBuffer + ZSTD_LITBUFFEREXTRASIZE;
+ dctx->litBufferLocation = ZSTD_not_in_dst;
+ {
+ size_t const oneSeqSize = ZSTD_execSequence(op, oend, sequence, &litPtr, litBufferEnd, prefixStart, vBase, dictEnd);
+#if defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) && defined(FUZZING_ASSERT_VALID_SEQUENCE)
+ assert(!ZSTD_isError(oneSeqSize));
+ if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequence, prefixStart, vBase);
+#endif
+ if (UNLIKELY(ZSTD_isError(oneSeqSize)))
+ return oneSeqSize;
+ DEBUGLOG(6, "regenerated sequence size : %u", (U32)oneSeqSize);
+ op += oneSeqSize;
+ if (--nbSeq)
+ BIT_reloadDStream(&(seqState.DStream));
+ }
+ }
+ }
+
+ if (nbSeq > 0) /* there is remaining lit from extra buffer */
+ {
+
+#if defined(__x86_64__)
+ __asm__(".p2align 6");
+ __asm__("nop");
+# if __GNUC__ != 7
+ /* worse for gcc-7 better for gcc-8, gcc-9, and gcc-10 and clang */
+ __asm__(".p2align 4");
+ __asm__("nop");
+ __asm__(".p2align 3");
+# elif __GNUC__ >= 11
+ __asm__(".p2align 3");
+# else
+ __asm__(".p2align 5");
+ __asm__("nop");
+ __asm__(".p2align 3");
+# endif
+#endif
+
+ for (; ; ) {
+ seq_t const sequence = ZSTD_decodeSequence(&seqState, isLongOffset);
+ size_t const oneSeqSize = ZSTD_execSequence(op, oend, sequence, &litPtr, litBufferEnd, prefixStart, vBase, dictEnd);
+#if defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) && defined(FUZZING_ASSERT_VALID_SEQUENCE)
+ assert(!ZSTD_isError(oneSeqSize));
+ if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequence, prefixStart, vBase);
+#endif
+ if (UNLIKELY(ZSTD_isError(oneSeqSize)))
+ return oneSeqSize;
+ DEBUGLOG(6, "regenerated sequence size : %u", (U32)oneSeqSize);
+ op += oneSeqSize;
+ if (UNLIKELY(!--nbSeq))
+ break;
+ BIT_reloadDStream(&(seqState.DStream));
+ }
+ }
+
+ /* check if reached exact end */
+ DEBUGLOG(5, "ZSTD_decompressSequences_bodySplitLitBuffer: after decode loop, remaining nbSeq : %i", nbSeq);
+ RETURN_ERROR_IF(nbSeq, corruption_detected, "");
+ RETURN_ERROR_IF(BIT_reloadDStream(&seqState.DStream) < BIT_DStream_completed, corruption_detected, "");
+ /* save reps for next block */
+ { U32 i; for (i=0; i<ZSTD_REP_NUM; i++) dctx->entropy.rep[i] = (U32)(seqState.prevOffset[i]); }
+ }
+
+ /* last literal segment */
+ if (dctx->litBufferLocation == ZSTD_split) /* split hasn't been reached yet, first get dst then copy litExtraBuffer */
+ {
+ size_t const lastLLSize = litBufferEnd - litPtr;
+ RETURN_ERROR_IF(lastLLSize > (size_t)(oend - op), dstSize_tooSmall, "");
+ if (op != NULL) {
+ ZSTD_memmove(op, litPtr, lastLLSize);
+ op += lastLLSize;
+ }
+ litPtr = dctx->litExtraBuffer;
+ litBufferEnd = dctx->litExtraBuffer + ZSTD_LITBUFFEREXTRASIZE;
+ dctx->litBufferLocation = ZSTD_not_in_dst;
+ }
+ { size_t const lastLLSize = litBufferEnd - litPtr;
+ RETURN_ERROR_IF(lastLLSize > (size_t)(oend-op), dstSize_tooSmall, "");
+ if (op != NULL) {
+ ZSTD_memcpy(op, litPtr, lastLLSize);
+ op += lastLLSize;
+ }
+ }
+
+ return op-ostart;
+}
+
+FORCE_INLINE_TEMPLATE size_t
+DONT_VECTORIZE
+ZSTD_decompressSequences_body(ZSTD_DCtx* dctx,
+ void* dst, size_t maxDstSize,
+ const void* seqStart, size_t seqSize, int nbSeq,
+ const ZSTD_longOffset_e isLongOffset,
+ const int frame)
+{
+ const BYTE* ip = (const BYTE*)seqStart;
+ const BYTE* const iend = ip + seqSize;
+ BYTE* const ostart = (BYTE*)dst;
+ BYTE* const oend = dctx->litBufferLocation == ZSTD_not_in_dst ? ostart + maxDstSize : dctx->litBuffer;
+ BYTE* op = ostart;
+ const BYTE* litPtr = dctx->litPtr;
+ const BYTE* const litEnd = litPtr + dctx->litSize;
+ const BYTE* const prefixStart = (const BYTE*)(dctx->prefixStart);
+ const BYTE* const vBase = (const BYTE*)(dctx->virtualStart);
+ const BYTE* const dictEnd = (const BYTE*)(dctx->dictEnd);
+ DEBUGLOG(5, "ZSTD_decompressSequences_body");
+ (void)frame;
+
+ /* Regen sequences */
+ if (nbSeq) {
+ seqState_t seqState;
+ dctx->fseEntropy = 1;
+ { U32 i; for (i = 0; i < ZSTD_REP_NUM; i++) seqState.prevOffset[i] = dctx->entropy.rep[i]; }
+ RETURN_ERROR_IF(
+ ERR_isError(BIT_initDStream(&seqState.DStream, ip, iend - ip)),
+ corruption_detected, "");
+ ZSTD_initFseState(&seqState.stateLL, &seqState.DStream, dctx->LLTptr);
+ ZSTD_initFseState(&seqState.stateOffb, &seqState.DStream, dctx->OFTptr);
+ ZSTD_initFseState(&seqState.stateML, &seqState.DStream, dctx->MLTptr);
+ assert(dst != NULL);
+
+ ZSTD_STATIC_ASSERT(
+ BIT_DStream_unfinished < BIT_DStream_completed &&
+ BIT_DStream_endOfBuffer < BIT_DStream_completed &&
+ BIT_DStream_completed < BIT_DStream_overflow);
+
+#if defined(__x86_64__)
+ __asm__(".p2align 6");
+ __asm__("nop");
+# if __GNUC__ >= 7
+ __asm__(".p2align 5");
+ __asm__("nop");
+ __asm__(".p2align 3");
+# else
+ __asm__(".p2align 4");
+ __asm__("nop");
+ __asm__(".p2align 3");
+# endif
#endif
+
for ( ; ; ) {
- seq_t const sequence = ZSTD_decodeSequence(&seqState, isLongOffset, ZSTD_p_noPrefetch);
+ seq_t const sequence = ZSTD_decodeSequence(&seqState, isLongOffset);
size_t const oneSeqSize = ZSTD_execSequence(op, oend, sequence, &litPtr, litEnd, prefixStart, vBase, dictEnd);
#if defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) && defined(FUZZING_ASSERT_VALID_SEQUENCE)
assert(!ZSTD_isError(oneSeqSize));
if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequence, prefixStart, vBase);
#endif
+ if (UNLIKELY(ZSTD_isError(oneSeqSize)))
+ return oneSeqSize;
DEBUGLOG(6, "regenerated sequence size : %u", (U32)oneSeqSize);
- BIT_reloadDStream(&(seqState.DStream));
op += oneSeqSize;
- /* gcc and clang both don't like early returns in this loop.
- * Instead break and check for an error at the end of the loop.
- */
- if (UNLIKELY(ZSTD_isError(oneSeqSize))) {
- error = oneSeqSize;
+ if (UNLIKELY(!--nbSeq))
break;
- }
- if (UNLIKELY(!--nbSeq)) break;
+ BIT_reloadDStream(&(seqState.DStream));
}
/* check if reached exact end */
DEBUGLOG(5, "ZSTD_decompressSequences_body: after decode loop, remaining nbSeq : %i", nbSeq);
- if (ZSTD_isError(error)) return error;
RETURN_ERROR_IF(nbSeq, corruption_detected, "");
RETURN_ERROR_IF(BIT_reloadDStream(&seqState.DStream) < BIT_DStream_completed, corruption_detected, "");
/* save reps for next block */
{
return ZSTD_decompressSequences_body(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame);
}
+
+static size_t
+ZSTD_decompressSequencesSplitLitBuffer_default(ZSTD_DCtx* dctx,
+ void* dst, size_t maxDstSize,
+ const void* seqStart, size_t seqSize, int nbSeq,
+ const ZSTD_longOffset_e isLongOffset,
+ const int frame)
+{
+ return ZSTD_decompressSequences_bodySplitLitBuffer(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame);
+}
#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG */
#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT
+
+FORCE_INLINE_TEMPLATE size_t
+ZSTD_prefetchMatch(size_t prefetchPos, seq_t const sequence,
+ const BYTE* const prefixStart, const BYTE* const dictEnd)
+{
+ prefetchPos += sequence.litLength;
+ { const BYTE* const matchBase = (sequence.offset > prefetchPos) ? dictEnd : prefixStart;
+ const BYTE* const match = matchBase + prefetchPos - sequence.offset; /* note : this operation can overflow when seq.offset is really too large, which can only happen when input is corrupted.
+ * No consequence though : memory address is only used for prefetching, not for dereferencing */
+ PREFETCH_L1(match); PREFETCH_L1(match+CACHELINE_SIZE); /* note : it's safe to invoke PREFETCH() on any memory address, including invalid ones */
+ }
+ return prefetchPos + sequence.matchLength;
+}
+
+/* This decoding function employs prefetching
+ * to reduce latency impact of cache misses.
+ * It's generally employed when block contains a significant portion of long-distance matches
+ * or when coupled with a "cold" dictionary */
FORCE_INLINE_TEMPLATE size_t
ZSTD_decompressSequencesLong_body(
ZSTD_DCtx* dctx,
const BYTE* ip = (const BYTE*)seqStart;
const BYTE* const iend = ip + seqSize;
BYTE* const ostart = (BYTE*)dst;
- BYTE* const oend = ostart + maxDstSize;
+ BYTE* const oend = dctx->litBufferLocation == ZSTD_in_dst ? dctx->litBuffer : ostart + maxDstSize;
BYTE* op = ostart;
const BYTE* litPtr = dctx->litPtr;
- const BYTE* const litEnd = litPtr + dctx->litSize;
+ const BYTE* litBufferEnd = dctx->litBufferEnd;
const BYTE* const prefixStart = (const BYTE*) (dctx->prefixStart);
const BYTE* const dictStart = (const BYTE*) (dctx->virtualStart);
const BYTE* const dictEnd = (const BYTE*) (dctx->dictEnd);
/* Regen sequences */
if (nbSeq) {
-#define STORED_SEQS 4
+#define STORED_SEQS 8
#define STORED_SEQS_MASK (STORED_SEQS-1)
-#define ADVANCED_SEQS 4
+#define ADVANCED_SEQS STORED_SEQS
seq_t sequences[STORED_SEQS];
int const seqAdvance = MIN(nbSeq, ADVANCED_SEQS);
seqState_t seqState;
int seqNb;
+ size_t prefetchPos = (size_t)(op-prefixStart); /* track position relative to prefixStart */
+
dctx->fseEntropy = 1;
{ int i; for (i=0; i<ZSTD_REP_NUM; i++) seqState.prevOffset[i] = dctx->entropy.rep[i]; }
- seqState.prefixStart = prefixStart;
- seqState.pos = (size_t)(op-prefixStart);
- seqState.dictEnd = dictEnd;
assert(dst != NULL);
assert(iend >= ip);
RETURN_ERROR_IF(
/* prepare in advance */
for (seqNb=0; (BIT_reloadDStream(&seqState.DStream) <= BIT_DStream_completed) && (seqNb<seqAdvance); seqNb++) {
- sequences[seqNb] = ZSTD_decodeSequence(&seqState, isLongOffset, ZSTD_p_prefetch);
- PREFETCH_L1(sequences[seqNb].match); PREFETCH_L1(sequences[seqNb].match + sequences[seqNb].matchLength - 1); /* note : it's safe to invoke PREFETCH() on any memory address, including invalid ones */
+ seq_t const sequence = ZSTD_decodeSequence(&seqState, isLongOffset);
+ prefetchPos = ZSTD_prefetchMatch(prefetchPos, sequence, prefixStart, dictEnd);
+ sequences[seqNb] = sequence;
}
RETURN_ERROR_IF(seqNb<seqAdvance, corruption_detected, "");
- /* decode and decompress */
- for ( ; (BIT_reloadDStream(&(seqState.DStream)) <= BIT_DStream_completed) && (seqNb<nbSeq) ; seqNb++) {
- seq_t const sequence = ZSTD_decodeSequence(&seqState, isLongOffset, ZSTD_p_prefetch);
- size_t const oneSeqSize = ZSTD_execSequence(op, oend, sequences[(seqNb-ADVANCED_SEQS) & STORED_SEQS_MASK], &litPtr, litEnd, prefixStart, dictStart, dictEnd);
+ /* decompress without stomping litBuffer */
+ for (; (BIT_reloadDStream(&(seqState.DStream)) <= BIT_DStream_completed) && (seqNb < nbSeq); seqNb++) {
+ seq_t sequence = ZSTD_decodeSequence(&seqState, isLongOffset);
+ size_t oneSeqSize;
+
+ if (dctx->litBufferLocation == ZSTD_split && litPtr + sequences[(seqNb - ADVANCED_SEQS) & STORED_SEQS_MASK].litLength > dctx->litBufferEnd)
+ {
+ /* lit buffer is reaching split point, empty out the first buffer and transition to litExtraBuffer */
+ const size_t leftoverLit = dctx->litBufferEnd - litPtr;
+ if (leftoverLit)
+ {
+ RETURN_ERROR_IF(leftoverLit > (size_t)(oend - op), dstSize_tooSmall, "remaining lit must fit within dstBuffer");
+ ZSTD_safecopyDstBeforeSrc(op, litPtr, leftoverLit);
+ sequences[(seqNb - ADVANCED_SEQS) & STORED_SEQS_MASK].litLength -= leftoverLit;
+ op += leftoverLit;
+ }
+ litPtr = dctx->litExtraBuffer;
+ litBufferEnd = dctx->litExtraBuffer + ZSTD_LITBUFFEREXTRASIZE;
+ dctx->litBufferLocation = ZSTD_not_in_dst;
+ oneSeqSize = ZSTD_execSequence(op, oend, sequences[(seqNb - ADVANCED_SEQS) & STORED_SEQS_MASK], &litPtr, litBufferEnd, prefixStart, dictStart, dictEnd);
#if defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) && defined(FUZZING_ASSERT_VALID_SEQUENCE)
- assert(!ZSTD_isError(oneSeqSize));
- if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequences[(seqNb-ADVANCED_SEQS) & STORED_SEQS_MASK], prefixStart, dictStart);
+ assert(!ZSTD_isError(oneSeqSize));
+ if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequences[(seqNb - ADVANCED_SEQS) & STORED_SEQS_MASK], prefixStart, dictStart);
#endif
- if (ZSTD_isError(oneSeqSize)) return oneSeqSize;
- PREFETCH_L1(sequence.match); PREFETCH_L1(sequence.match + sequence.matchLength - 1); /* note : it's safe to invoke PREFETCH() on any memory address, including invalid ones */
- sequences[seqNb & STORED_SEQS_MASK] = sequence;
- op += oneSeqSize;
+ if (ZSTD_isError(oneSeqSize)) return oneSeqSize;
+
+ prefetchPos = ZSTD_prefetchMatch(prefetchPos, sequence, prefixStart, dictEnd);
+ sequences[seqNb & STORED_SEQS_MASK] = sequence;
+ op += oneSeqSize;
+ }
+ else
+ {
+ /* lit buffer is either wholly contained in first or second split, or not split at all*/
+ oneSeqSize = dctx->litBufferLocation == ZSTD_split ?
+ ZSTD_execSequenceSplitLitBuffer(op, oend, litPtr + sequences[(seqNb - ADVANCED_SEQS) & STORED_SEQS_MASK].litLength - WILDCOPY_OVERLENGTH, sequences[(seqNb - ADVANCED_SEQS) & STORED_SEQS_MASK], &litPtr, litBufferEnd, prefixStart, dictStart, dictEnd) :
+ ZSTD_execSequence(op, oend, sequences[(seqNb - ADVANCED_SEQS) & STORED_SEQS_MASK], &litPtr, litBufferEnd, prefixStart, dictStart, dictEnd);
+#if defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) && defined(FUZZING_ASSERT_VALID_SEQUENCE)
+ assert(!ZSTD_isError(oneSeqSize));
+ if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequences[(seqNb - ADVANCED_SEQS) & STORED_SEQS_MASK], prefixStart, dictStart);
+#endif
+ if (ZSTD_isError(oneSeqSize)) return oneSeqSize;
+
+ prefetchPos = ZSTD_prefetchMatch(prefetchPos, sequence, prefixStart, dictEnd);
+ sequences[seqNb & STORED_SEQS_MASK] = sequence;
+ op += oneSeqSize;
+ }
}
RETURN_ERROR_IF(seqNb<nbSeq, corruption_detected, "");
/* finish queue */
seqNb -= seqAdvance;
for ( ; seqNb<nbSeq ; seqNb++) {
- size_t const oneSeqSize = ZSTD_execSequence(op, oend, sequences[seqNb&STORED_SEQS_MASK], &litPtr, litEnd, prefixStart, dictStart, dictEnd);
+ seq_t *sequence = &(sequences[seqNb&STORED_SEQS_MASK]);
+ if (dctx->litBufferLocation == ZSTD_split && litPtr + sequence->litLength > dctx->litBufferEnd)
+ {
+ const size_t leftoverLit = dctx->litBufferEnd - litPtr;
+ if (leftoverLit)
+ {
+ RETURN_ERROR_IF(leftoverLit > (size_t)(oend - op), dstSize_tooSmall, "remaining lit must fit within dstBuffer");
+ ZSTD_safecopyDstBeforeSrc(op, litPtr, leftoverLit);
+ sequence->litLength -= leftoverLit;
+ op += leftoverLit;
+ }
+ litPtr = dctx->litExtraBuffer;
+ litBufferEnd = dctx->litExtraBuffer + ZSTD_LITBUFFEREXTRASIZE;
+ dctx->litBufferLocation = ZSTD_not_in_dst;
+ {
+ size_t const oneSeqSize = ZSTD_execSequence(op, oend, *sequence, &litPtr, litBufferEnd, prefixStart, dictStart, dictEnd);
#if defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) && defined(FUZZING_ASSERT_VALID_SEQUENCE)
- assert(!ZSTD_isError(oneSeqSize));
- if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequences[seqNb&STORED_SEQS_MASK], prefixStart, dictStart);
+ assert(!ZSTD_isError(oneSeqSize));
+ if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequences[seqNb&STORED_SEQS_MASK], prefixStart, dictStart);
#endif
- if (ZSTD_isError(oneSeqSize)) return oneSeqSize;
- op += oneSeqSize;
+ if (ZSTD_isError(oneSeqSize)) return oneSeqSize;
+ op += oneSeqSize;
+ }
+ }
+ else
+ {
+ size_t const oneSeqSize = dctx->litBufferLocation == ZSTD_split ?
+ ZSTD_execSequenceSplitLitBuffer(op, oend, litPtr + sequence->litLength - WILDCOPY_OVERLENGTH, *sequence, &litPtr, litBufferEnd, prefixStart, dictStart, dictEnd) :
+ ZSTD_execSequence(op, oend, *sequence, &litPtr, litBufferEnd, prefixStart, dictStart, dictEnd);
+#if defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) && defined(FUZZING_ASSERT_VALID_SEQUENCE)
+ assert(!ZSTD_isError(oneSeqSize));
+ if (frame) ZSTD_assertValidSequence(dctx, op, oend, sequences[seqNb&STORED_SEQS_MASK], prefixStart, dictStart);
+#endif
+ if (ZSTD_isError(oneSeqSize)) return oneSeqSize;
+ op += oneSeqSize;
+ }
}
/* save reps for next block */
}
/* last literal segment */
- { size_t const lastLLSize = litEnd - litPtr;
+ if (dctx->litBufferLocation == ZSTD_split) /* first deplete literal buffer in dst, then copy litExtraBuffer */
+ {
+ size_t const lastLLSize = litBufferEnd - litPtr;
+ RETURN_ERROR_IF(lastLLSize > (size_t)(oend - op), dstSize_tooSmall, "");
+ if (op != NULL) {
+ ZSTD_memmove(op, litPtr, lastLLSize);
+ op += lastLLSize;
+ }
+ litPtr = dctx->litExtraBuffer;
+ litBufferEnd = dctx->litExtraBuffer + ZSTD_LITBUFFEREXTRASIZE;
+ }
+ { size_t const lastLLSize = litBufferEnd - litPtr;
RETURN_ERROR_IF(lastLLSize > (size_t)(oend-op), dstSize_tooSmall, "");
if (op != NULL) {
- ZSTD_memcpy(op, litPtr, lastLLSize);
+ ZSTD_memmove(op, litPtr, lastLLSize);
op += lastLLSize;
}
}
#if DYNAMIC_BMI2
#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG
-static TARGET_ATTRIBUTE("bmi2") size_t
+static BMI2_TARGET_ATTRIBUTE size_t
DONT_VECTORIZE
ZSTD_decompressSequences_bmi2(ZSTD_DCtx* dctx,
void* dst, size_t maxDstSize,
{
return ZSTD_decompressSequences_body(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame);
}
+static BMI2_TARGET_ATTRIBUTE size_t
+DONT_VECTORIZE
+ZSTD_decompressSequencesSplitLitBuffer_bmi2(ZSTD_DCtx* dctx,
+ void* dst, size_t maxDstSize,
+ const void* seqStart, size_t seqSize, int nbSeq,
+ const ZSTD_longOffset_e isLongOffset,
+ const int frame)
+{
+ return ZSTD_decompressSequences_bodySplitLitBuffer(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame);
+}
#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG */
#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT
-static TARGET_ATTRIBUTE("bmi2") size_t
+static BMI2_TARGET_ATTRIBUTE size_t
ZSTD_decompressSequencesLong_bmi2(ZSTD_DCtx* dctx,
void* dst, size_t maxDstSize,
const void* seqStart, size_t seqSize, int nbSeq,
{
DEBUGLOG(5, "ZSTD_decompressSequences");
#if DYNAMIC_BMI2
- if (dctx->bmi2) {
+ if (ZSTD_DCtx_get_bmi2(dctx)) {
return ZSTD_decompressSequences_bmi2(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame);
}
#endif
- return ZSTD_decompressSequences_default(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame);
+ return ZSTD_decompressSequences_default(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame);
+}
+static size_t
+ZSTD_decompressSequencesSplitLitBuffer(ZSTD_DCtx* dctx, void* dst, size_t maxDstSize,
+ const void* seqStart, size_t seqSize, int nbSeq,
+ const ZSTD_longOffset_e isLongOffset,
+ const int frame)
+{
+ DEBUGLOG(5, "ZSTD_decompressSequencesSplitLitBuffer");
+#if DYNAMIC_BMI2
+ if (ZSTD_DCtx_get_bmi2(dctx)) {
+ return ZSTD_decompressSequencesSplitLitBuffer_bmi2(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame);
+ }
+#endif
+ return ZSTD_decompressSequencesSplitLitBuffer_default(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame);
}
#endif /* ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG */
{
DEBUGLOG(5, "ZSTD_decompressSequencesLong");
#if DYNAMIC_BMI2
- if (dctx->bmi2) {
+ if (ZSTD_DCtx_get_bmi2(dctx)) {
return ZSTD_decompressSequencesLong_bmi2(dctx, dst, maxDstSize, seqStart, seqSize, nbSeq, isLongOffset, frame);
}
#endif
size_t
ZSTD_decompressBlock_internal(ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity,
- const void* src, size_t srcSize, const int frame)
+ const void* src, size_t srcSize, const int frame, const streaming_operation streaming)
{ /* blockType == blockCompressed */
const BYTE* ip = (const BYTE*)src;
/* isLongOffset must be true if there are long offsets.
RETURN_ERROR_IF(srcSize >= ZSTD_BLOCKSIZE_MAX, srcSize_wrong, "");
/* Decode literals section */
- { size_t const litCSize = ZSTD_decodeLiteralsBlock(dctx, src, srcSize);
+ { size_t const litCSize = ZSTD_decodeLiteralsBlock(dctx, src, srcSize, dst, dstCapacity, streaming);
DEBUGLOG(5, "ZSTD_decodeLiteralsBlock : %u", (U32)litCSize);
if (ZSTD_isError(litCSize)) return litCSize;
ip += litCSize;
#ifndef ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG
/* else */
- return ZSTD_decompressSequences(dctx, dst, dstCapacity, ip, srcSize, nbSeq, isLongOffset, frame);
+ if (dctx->litBufferLocation == ZSTD_split)
+ return ZSTD_decompressSequencesSplitLitBuffer(dctx, dst, dstCapacity, ip, srcSize, nbSeq, isLongOffset, frame);
+ else
+ return ZSTD_decompressSequences(dctx, dst, dstCapacity, ip, srcSize, nbSeq, isLongOffset, frame);
#endif
}
}
{
size_t dSize;
ZSTD_checkContinuity(dctx, dst, dstCapacity);
- dSize = ZSTD_decompressBlock_internal(dctx, dst, dstCapacity, src, srcSize, /* frame */ 0);
+ dSize = ZSTD_decompressBlock_internal(dctx, dst, dstCapacity, src, srcSize, /* frame */ 0, not_streaming);
dctx->previousDstEnd = (char*)dst + dSize;
return dSize;
}
*/
+ /* Streaming state is used to inform allocation of the literal buffer */
+typedef enum {
+ not_streaming = 0,
+ is_streaming = 1
+} streaming_operation;
+
/* ZSTD_decompressBlock_internal() :
* decompress block, starting at `src`,
* into destination buffer `dst`.
*/
size_t ZSTD_decompressBlock_internal(ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity,
- const void* src, size_t srcSize, const int frame);
+ const void* src, size_t srcSize, const int frame, const streaming_operation streaming);
/* ZSTD_buildFSETable() :
* generate FSE decoding table for one symbol (ll, ml or off)
*/
void ZSTD_buildFSETable(ZSTD_seqSymbol* dt,
const short* normalizedCounter, unsigned maxSymbolValue,
- const U32* baseValue, const U32* nbAdditionalBits,
+ const U32* baseValue, const U8* nbAdditionalBits,
unsigned tableLog, void* wksp, size_t wkspSize,
int bmi2);
* Dependencies
*********************************************************/
#include "../common/mem.h" /* BYTE, U16, U32 */
-#include "../common/zstd_internal.h" /* ZSTD_seqSymbol */
+#include "../common/zstd_internal.h" /* constants : MaxLL, MaxML, MaxOff, LLFSELog, etc. */
0xFFFD, 0x1FFFD, 0x3FFFD, 0x7FFFD, 0xFFFFD, 0x1FFFFD, 0x3FFFFD, 0x7FFFFD,
0xFFFFFD, 0x1FFFFFD, 0x3FFFFFD, 0x7FFFFFD, 0xFFFFFFD, 0x1FFFFFFD, 0x3FFFFFFD, 0x7FFFFFFD };
-static UNUSED_ATTR const U32 OF_bits[MaxOff+1] = {
+static UNUSED_ATTR const U8 OF_bits[MaxOff+1] = {
0, 1, 2, 3, 4, 5, 6, 7,
8, 9, 10, 11, 12, 13, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23,
size_t ddictPtrCount;
} ZSTD_DDictHashSet;
+#ifndef ZSTD_DECODER_INTERNAL_BUFFER
+# define ZSTD_DECODER_INTERNAL_BUFFER (1 << 16)
+#endif
+
+#define ZSTD_LBMIN 64
+#define ZSTD_LBMAX (128 << 10)
+
+/* extra buffer, compensates when dst is not large enough to store litBuffer */
+#define ZSTD_LITBUFFEREXTRASIZE BOUNDED(ZSTD_LBMIN, ZSTD_DECODER_INTERNAL_BUFFER, ZSTD_LBMAX)
+
+typedef enum {
+ ZSTD_not_in_dst = 0, /* Stored entirely within litExtraBuffer */
+ ZSTD_in_dst = 1, /* Stored entirely within dst (in memory after current output write) */
+ ZSTD_split = 2 /* Split between litExtraBuffer and dst */
+} ZSTD_litLocation_e;
+
struct ZSTD_DCtx_s
{
const ZSTD_seqSymbol* LLTptr;
size_t litSize;
size_t rleSize;
size_t staticSize;
+#if DYNAMIC_BMI2 != 0
int bmi2; /* == 1 if the CPU supports BMI2 and 0 otherwise. CPU support is determined dynamically once per context lifetime. */
+#endif
/* dictionary */
ZSTD_DDict* ddictLocal;
size_t outStart;
size_t outEnd;
size_t lhSize;
- void* legacyContext;
- U32 previousLegacyVersion;
- U32 legacyVersion;
U32 hostageByte;
int noForwardProgress;
ZSTD_bufferMode_e outBufferMode;
ZSTD_outBuffer expectedOutBuffer;
/* workspace */
- BYTE litBuffer[ZSTD_BLOCKSIZE_MAX + WILDCOPY_OVERLENGTH];
+ BYTE* litBuffer;
+ const BYTE* litBufferEnd;
+ ZSTD_litLocation_e litBufferLocation;
+ BYTE litExtraBuffer[ZSTD_LITBUFFEREXTRASIZE + WILDCOPY_OVERLENGTH]; /* literal buffer can be split between storage within dst and within this scratch buffer */
BYTE headerBuffer[ZSTD_FRAMEHEADERSIZE_MAX];
size_t oversizedDuration;
/* Tracing */
}; /* typedef'd to ZSTD_DCtx within "zstd.h" */
+MEM_STATIC int ZSTD_DCtx_get_bmi2(const struct ZSTD_DCtx_s *dctx) {
+#if DYNAMIC_BMI2 != 0
+ return dctx->bmi2;
+#else
+ (void)dctx;
+ return 0;
+#endif
+}
/*-*******************************************************
* Shared internal functions
* decompression.
*/
+/*
+ * Disable the ASM Huffman implementation because we need to
+ * include all the sources.
+ */
+#define ZSTD_DISABLE_ASM 1
+
#include "common/debug.c"
#include "common/entropy_common.c"
#include "common/error_private.c"
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+ OR BSD-3-Clause
+/*
+ * Copyright (c) Facebook, Inc.
+ * All rights reserved.
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+ */
+
+#include <linux/module.h>
+
+#include "common/huf.h"
+#include "common/fse.h"
+#include "common/zstd_internal.h"
+
+// Export symbols shared by compress and decompress into a common module
+
+#undef ZSTD_isError /* defined within zstd_internal.h */
+EXPORT_SYMBOL_GPL(FSE_readNCount);
+EXPORT_SYMBOL_GPL(HUF_readStats);
+EXPORT_SYMBOL_GPL(HUF_readStats_wksp);
+EXPORT_SYMBOL_GPL(ZSTD_isError);
+EXPORT_SYMBOL_GPL(ZSTD_getErrorName);
+EXPORT_SYMBOL_GPL(ZSTD_getErrorCode);
+EXPORT_SYMBOL_GPL(ZSTD_customMalloc);
+EXPORT_SYMBOL_GPL(ZSTD_customCalloc);
+EXPORT_SYMBOL_GPL(ZSTD_customFree);
+
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_DESCRIPTION("Zstd Common");
size_t zstd_reset_cstream(zstd_cstream *cstream,
unsigned long long pledged_src_size)
{
- return ZSTD_resetCStream(cstream, pledged_src_size);
+ if (pledged_src_size == 0)
+ pledged_src_size = ZSTD_CONTENTSIZE_UNKNOWN;
+ ZSTD_FORWARD_IF_ERR( ZSTD_CCtx_reset(cstream, ZSTD_reset_session_only) );
+ ZSTD_FORWARD_IF_ERR( ZSTD_CCtx_setPledgedSrcSize(cstream, pledged_src_size) );
+ return 0;
}
EXPORT_SYMBOL(zstd_reset_cstream);
comment "GUP_TEST needs to have DEBUG_FS enabled"
depends on !GUP_TEST && !DEBUG_FS
-config GUP_GET_PTE_LOW_HIGH
+config GUP_GET_PXX_LOW_HIGH
bool
config ARCH_HAS_PTE_SPECIAL
pmdp = pmd_offset_lockless(pudp, pud, addr);
do {
- pmd_t pmd = READ_ONCE(*pmdp);
+ pmd_t pmd = pmdp_get_lockless(pmdp);
next = pmd_addr_end(addr, end);
if (!pmd_present(pmd))
if (pin_longterm_test_nr_pages)
unpin_user_pages(pin_longterm_test_pages,
pin_longterm_test_nr_pages);
- kfree(pin_longterm_test_pages);
+ kvfree(pin_longterm_test_pages);
pin_longterm_test_pages = NULL;
pin_longterm_test_nr_pages = 0;
}
fast = !!(args.flags & PIN_LONGTERM_TEST_FLAG_USE_FAST);
if (!fast && mmap_read_lock_killable(current->mm)) {
- kfree(pages);
+ kvfree(pages);
return -EINTR;
}
* huge or device mapping one and compute corresponding pfn
* values.
*/
- pmd = pmd_read_atomic(pmdp);
- barrier();
+ pmd = pmdp_get_lockless(pmdp);
if (!pmd_devmap(pmd) && !pmd_trans_huge(pmd))
goto again;
if (!*pmd)
return SCAN_PMD_NULL;
- pmde = pmd_read_atomic(*pmd);
+ pmde = pmdp_get_lockless(*pmd);
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
/* See comments in pmd_none_or_trans_huge_or_clear_bad() */
#include <linux/mutex.h>
#include <linux/rcupdate.h>
#include <linux/stacktrace.h>
+#include <linux/stackdepot.h>
#include <linux/cache.h>
#include <linux/percpu.h>
#include <linux/memblock.h>
u32 checksum;
/* memory ranges to be scanned inside an object (empty for all) */
struct hlist_head area_list;
- unsigned long trace[MAX_TRACE];
- unsigned int trace_len;
+ depot_stack_handle_t trace_handle;
unsigned long jiffies; /* creation timestamp */
pid_t pid; /* pid of the current task */
char comm[TASK_COMM_LEN]; /* executable name */
struct kmemleak_object *object)
{
int i;
+ unsigned long *entries;
+ unsigned int nr_entries;
unsigned int msecs_age = jiffies_to_msecs(jiffies - object->jiffies);
+ nr_entries = stack_depot_fetch(object->trace_handle, &entries);
warn_or_seq_printf(seq, "unreferenced object 0x%08lx (size %zu):\n",
- object->pointer, object->size);
+ object->pointer, object->size);
warn_or_seq_printf(seq, " comm \"%s\", pid %d, jiffies %lu (age %d.%03ds)\n",
- object->comm, object->pid, object->jiffies,
- msecs_age / 1000, msecs_age % 1000);
+ object->comm, object->pid, object->jiffies,
+ msecs_age / 1000, msecs_age % 1000);
hex_dump_object(seq, object);
warn_or_seq_printf(seq, " backtrace:\n");
- for (i = 0; i < object->trace_len; i++) {
- void *ptr = (void *)object->trace[i];
- warn_or_seq_printf(seq, " [<%p>] %pS\n", ptr, ptr);
+ for (i = 0; i < nr_entries; i++) {
+ void *ptr = (void *)entries[i];
+ warn_or_seq_printf(seq, " [<%pK>] %pS\n", ptr, ptr);
}
}
static void dump_object_info(struct kmemleak_object *object)
{
pr_notice("Object 0x%08lx (size %zu):\n",
- object->pointer, object->size);
+ object->pointer, object->size);
pr_notice(" comm \"%s\", pid %d, jiffies %lu\n",
- object->comm, object->pid, object->jiffies);
+ object->comm, object->pid, object->jiffies);
pr_notice(" min_count = %d\n", object->min_count);
pr_notice(" count = %d\n", object->count);
pr_notice(" flags = 0x%x\n", object->flags);
pr_notice(" checksum = %u\n", object->checksum);
pr_notice(" backtrace:\n");
- stack_trace_print(object->trace, object->trace_len, 4);
+ if (object->trace_handle)
+ stack_depot_print(object->trace_handle);
}
/*
return object;
}
-/*
- * Save stack trace to the given array of MAX_TRACE size.
- */
-static int __save_stack_trace(unsigned long *trace)
+static noinline depot_stack_handle_t set_track_prepare(void)
{
- return stack_trace_save(trace, MAX_TRACE, 2);
+ depot_stack_handle_t trace_handle;
+ unsigned long entries[MAX_TRACE];
+ unsigned int nr_entries;
+
+ if (!kmemleak_initialized)
+ return 0;
+ nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 3);
+ trace_handle = stack_depot_save(entries, nr_entries, GFP_NOWAIT);
+
+ return trace_handle;
}
/*
}
/* kernel backtrace */
- object->trace_len = __save_stack_trace(object->trace);
+ object->trace_handle = set_track_prepare();
raw_spin_lock_irqsave(&kmemleak_lock, flags);
rb_link_node(&object->rb_node, rb_parent, link);
rb_insert_color(&object->rb_node, is_phys ? &object_phys_tree_root :
&object_tree_root);
-
list_add_tail_rcu(&object->object_list, &object_list);
out:
raw_spin_unlock_irqrestore(&kmemleak_lock, flags);
}
raw_spin_lock_irqsave(&object->lock, flags);
- object->trace_len = __save_stack_trace(object->trace);
+ object->trace_handle = set_track_prepare();
raw_spin_unlock_irqrestore(&object->lock, flags);
put_object(object);
if (kmemleak_error)
return;
+ stack_depot_init();
jiffies_min_age = msecs_to_jiffies(MSECS_MIN_AGE);
jiffies_scan_wait = msecs_to_jiffies(SECS_SCAN_WAIT * 1000);
static int wp_clean_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
- pmd_t pmdval = pmd_read_atomic(pmd);
+ pmd_t pmdval = pmdp_get_lockless(pmd);
if (!pmd_trans_unstable(&pmdval))
return 0;
*/
static inline int pmd_none_or_clear_bad_unless_trans_huge(pmd_t *pmd)
{
- pmd_t pmdval = pmd_read_atomic(pmd);
+ pmd_t pmdval = pmdp_get_lockless(pmd);
/* See pmd_none_or_trans_huge_or_clear_bad for info on barrier */
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
break;
}
- dst_pmdval = pmd_read_atomic(dst_pmd);
+ dst_pmdval = pmdp_get_lockless(dst_pmd);
/*
* If the dst_pmd is mapped as THP don't
* override it and just be strict.
/* walk_pte_range() may call get_next_vma() */
vma = args->vma;
for (i = pmd_index(start), addr = start; addr != end; i++, addr = next) {
- pmd_t val = pmd_read_atomic(pmd + i);
-
- /* for pmd_read_atomic() */
- barrier();
+ pmd_t val = pmdp_get_lockless(pmd + i);
next = pmd_addr_end(addr, end);
}
csocket->sk->sk_allocation = GFP_NOIO;
+ csocket->sk->sk_use_task_frag = false;
file = sock_alloc_file(csocket, 0, NULL);
if (IS_ERR(file)) {
pr_err("%s (%d): failed to map fd\n",
};
-static int atm_uevent(struct device *cdev, struct kobj_uevent_env *env)
+static int atm_uevent(const struct device *cdev, struct kobj_uevent_env *env)
{
- struct atm_dev *adev;
+ const struct atm_dev *adev;
if (!cdev)
return -ENODEV;
if (err < 0)
goto out;
- set_memory_ro((long)image, 1);
- set_memory_x((long)image, 1);
+ set_memory_rox((long)image, 1);
prog_ret = dummy_ops_call_op(image, args);
err = dummy_ops_copy_args(args);
kfree(p);
}
-static void brport_get_ownership(struct kobject *kobj, kuid_t *uid, kgid_t *gid)
+static void brport_get_ownership(const struct kobject *kobj, kuid_t *uid, kgid_t *gid)
{
struct net_bridge_port *p = kobj_to_brport(kobj);
if (ret)
return ret;
sock->sk->sk_allocation = GFP_NOFS;
+ sock->sk->sk_use_task_frag = false;
#ifdef CONFIG_LOCKDEP
lockdep_set_class(&sock->sk->sk_lock, &socket_class);
continue;
}
+ devl_lock(devlink);
err = devlink_nl_fill(msg, devlink, DEVLINK_CMD_NEW,
NETLINK_CB(cb->skb).portid,
cb->nlh->nlmsg_seq, NLM_F_MULTI);
+ devl_unlock(devlink);
devlink_put(devlink);
+
if (err)
goto out;
idx++;
devl_assert_locked(devlink);
/* Free all snapshots of region */
+ mutex_lock(®ion->snapshot_lock);
list_for_each_entry_safe(snapshot, ts, ®ion->snapshot_list, list)
devlink_region_snapshot_del(region, snapshot);
+ mutex_unlock(®ion->snapshot_lock);
list_del(®ion->list);
mutex_destroy(®ion->snapshot_lock);
netdev_put(queue->dev, &queue->dev_tracker);
}
-static const void *rx_queue_namespace(struct kobject *kobj)
+static const void *rx_queue_namespace(const struct kobject *kobj)
{
struct netdev_rx_queue *queue = to_rx_queue(kobj);
struct device *dev = &queue->dev->dev;
return ns;
}
-static void rx_queue_get_ownership(struct kobject *kobj,
+static void rx_queue_get_ownership(const struct kobject *kobj,
kuid_t *uid, kgid_t *gid)
{
const struct net *net = rx_queue_namespace(kobj);
netdev_put(queue->dev, &queue->dev_tracker);
}
-static const void *netdev_queue_namespace(struct kobject *kobj)
+static const void *netdev_queue_namespace(const struct kobject *kobj)
{
struct netdev_queue *queue = to_netdev_queue(kobj);
struct device *dev = &queue->dev->dev;
return ns;
}
-static void netdev_queue_get_ownership(struct kobject *kobj,
+static void netdev_queue_get_ownership(const struct kobject *kobj,
kuid_t *uid, kgid_t *gid)
{
const struct net *net = netdev_queue_namespace(kobj);
};
EXPORT_SYMBOL_GPL(net_ns_type_operations);
-static int netdev_uevent(struct device *d, struct kobj_uevent_env *env)
+static int netdev_uevent(const struct device *d, struct kobj_uevent_env *env)
{
- struct net_device *dev = to_net_dev(d);
+ const struct net_device *dev = to_net_dev(d);
int retval;
/* pass interface to uevent. */
netdev_freemem(dev);
}
-static const void *net_namespace(struct device *d)
+static const void *net_namespace(const struct device *d)
{
- struct net_device *dev = to_net_dev(d);
+ const struct net_device *dev = to_net_dev(d);
return dev_net(dev);
}
-static void net_get_ownership(struct device *d, kuid_t *uid, kgid_t *gid)
+static void net_get_ownership(const struct device *d, kuid_t *uid, kgid_t *gid)
{
- struct net_device *dev = to_net_dev(d);
+ const struct net_device *dev = to_net_dev(d);
const struct net *net = dev_net(dev);
net_ns_get_ownership(net, uid, gid);
insp = list;
} else {
/* Eaten partially. */
+ if (skb_is_gso(skb) && !list->head_frag &&
+ skb_headlen(list))
+ skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY;
if (skb_shared(list)) {
/* Sucks! We need to fork list. :-( */
sk->sk_rcvbuf = READ_ONCE(sysctl_rmem_default);
sk->sk_sndbuf = READ_ONCE(sysctl_wmem_default);
sk->sk_state = TCP_CLOSE;
+ sk->sk_use_task_frag = true;
sk_set_socket(sk, sock);
sock_set_flag(sk, SOCK_ZAPPED);
/* First the read buffer. */
__skb_queue_purge(&sk->sk_receive_queue);
+ /* Next, the error queue.
+ * We need to use queue lock, because other threads might
+ * add packets to the queue without socket lock being held.
+ */
+ skb_queue_purge(&sk->sk_error_queue);
+
/* Next, the write queue. */
WARN_ON_ONCE(!skb_queue_empty(&sk->sk_write_queue));
return;
pause = READ_ONCE(net->ipv4.sysctl_tcp_plb_suspend_rto_sec) * HZ;
- pause += prandom_u32_max(pause);
+ pause += get_random_u32_below(pause);
plb->pause_until = tcp_jiffies32 + pause;
/* Reset PLB state upon RTO, since an RTO causes a sk_rethink_txhash() call
struct mctp_dev *mdev;
mdev = mctp_dev_get_rtnl(dev);
- if (mdev && !mctp_known(dev)) {
- // Sanity check, should match what was set in mctp_register
- netdev_warn(dev, "%s: BUG mctp_ptr set for unknown type %d",
- __func__, dev->type);
- return;
- }
if (!mdev)
return;
struct mctp_dev *mdev;
/* Already registered? */
- mdev = rtnl_dereference(dev->mctp_ptr);
-
- if (mdev) {
- if (!mctp_known(dev))
- netdev_warn(dev, "%s: BUG mctp_ptr set for unknown type %d",
- __func__, dev->type);
+ if (rtnl_dereference(dev->mctp_ptr))
return 0;
- }
/* only register specific types */
if (!mctp_known(dev))
break;
case IP_VS_SO_SET_DELDEST:
ret = ip_vs_del_dest(svc, &udest);
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ ret = -EINVAL;
+ break;
}
out_unlock:
const __be32 *addr, const __be32 *mask)
{
struct flow_action_entry *entry;
- int i, j;
+ int i;
- for (i = 0, j = 0; i < sizeof(struct in6_addr) / sizeof(u32); i += sizeof(u32), j++) {
+ for (i = 0; i < sizeof(struct in6_addr) / sizeof(u32); i++) {
entry = flow_action_entry_next(flow_rule);
flow_offload_mangle(entry, FLOW_ACT_MANGLE_HDR_TYPE_IP6,
- offset + i, &addr[j], mask);
+ offset + i * sizeof(u32), &addr[i], mask);
}
}
struct sw_flow_mask mask;
struct sk_buff *reply;
struct datapath *dp;
+ struct sw_flow_key *key;
struct sw_flow_actions *acts;
struct sw_flow_match match;
u32 ufid_flags = ovs_nla_get_ufid_flags(a[OVS_FLOW_ATTR_UFID_FLAGS]);
}
/* Extract key. */
- ovs_match_init(&match, &new_flow->key, false, &mask);
+ key = kzalloc(sizeof(*key), GFP_KERNEL);
+ if (!key) {
+ error = -ENOMEM;
+ goto err_kfree_key;
+ }
+
+ ovs_match_init(&match, key, false, &mask);
error = ovs_nla_get_match(net, &match, a[OVS_FLOW_ATTR_KEY],
a[OVS_FLOW_ATTR_MASK], log);
if (error)
goto err_kfree_flow;
+ ovs_flow_mask_key(&new_flow->key, key, true, &mask);
+
/* Extract flow identifier. */
error = ovs_nla_get_identifier(&new_flow->id, a[OVS_FLOW_ATTR_UFID],
- &new_flow->key, log);
+ key, log);
if (error)
goto err_kfree_flow;
- /* unmasked key is needed to match when ufid is not used. */
- if (ovs_identifier_is_key(&new_flow->id))
- match.key = new_flow->id.unmasked_key;
-
- ovs_flow_mask_key(&new_flow->key, &new_flow->key, true, &mask);
-
/* Validate actions. */
error = ovs_nla_copy_actions(net, a[OVS_FLOW_ATTR_ACTIONS],
&new_flow->key, &acts, log);
if (ovs_identifier_is_ufid(&new_flow->id))
flow = ovs_flow_tbl_lookup_ufid(&dp->table, &new_flow->id);
if (!flow)
- flow = ovs_flow_tbl_lookup(&dp->table, &new_flow->key);
+ flow = ovs_flow_tbl_lookup(&dp->table, key);
if (likely(!flow)) {
rcu_assign_pointer(new_flow->sf_acts, acts);
if (reply)
ovs_notify(&dp_flow_genl_family, reply, info);
+
+ kfree(key);
return 0;
err_unlock_ovs:
ovs_nla_free_flow_actions(acts);
err_kfree_flow:
ovs_flow_free(new_flow, false);
+err_kfree_key:
+ kfree(key);
error:
return error;
}
kfree(rfkill);
}
-static int rfkill_dev_uevent(struct device *dev, struct kobj_uevent_env *env)
+static int rfkill_dev_uevent(const struct device *dev, struct kobj_uevent_env *env)
{
struct rfkill *rfkill = to_rfkill(dev);
unsigned long flags;
struct hlist_node link;
struct socket *socket; /* my UDP socket */
struct task_struct *io_thread;
+ struct completion io_thread_ready; /* Indication that the I/O thread started */
struct rxrpc_sock __rcu *service; /* Service(s) listening on this endpoint */
struct rw_semaphore defrag_sem; /* control re-enablement of IP DF bit */
struct sk_buff_head rx_queue; /* Received packets */
*/
int rxrpc_service_prealloc(struct rxrpc_sock *, gfp_t);
void rxrpc_discard_prealloc(struct rxrpc_sock *);
-bool rxrpc_new_incoming_call(struct rxrpc_local *, struct rxrpc_peer *,
- struct rxrpc_connection *, struct sockaddr_rxrpc *,
- struct sk_buff *);
+int rxrpc_new_incoming_call(struct rxrpc_local *, struct rxrpc_peer *,
+ struct rxrpc_connection *, struct sockaddr_rxrpc *,
+ struct sk_buff *);
void rxrpc_accept_incoming_calls(struct rxrpc_local *);
int rxrpc_user_charge_accept(struct rxrpc_sock *, unsigned long);
struct rxrpc_peer *rxrpc_get_peer(struct rxrpc_peer *, enum rxrpc_peer_trace);
struct rxrpc_peer *rxrpc_get_peer_maybe(struct rxrpc_peer *, enum rxrpc_peer_trace);
void rxrpc_put_peer(struct rxrpc_peer *, enum rxrpc_peer_trace);
-void rxrpc_put_peer_locked(struct rxrpc_peer *, enum rxrpc_peer_trace);
/*
* proc.c
* If we want to report an error, we mark the skb with the packet type and
* abort code and return false.
*/
-bool rxrpc_new_incoming_call(struct rxrpc_local *local,
- struct rxrpc_peer *peer,
- struct rxrpc_connection *conn,
- struct sockaddr_rxrpc *peer_srx,
- struct sk_buff *skb)
+int rxrpc_new_incoming_call(struct rxrpc_local *local,
+ struct rxrpc_peer *peer,
+ struct rxrpc_connection *conn,
+ struct sockaddr_rxrpc *peer_srx,
+ struct sk_buff *skb)
{
const struct rxrpc_security *sec = NULL;
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
/* Don't set up a call for anything other than the first DATA packet. */
if (sp->hdr.seq != 1 ||
sp->hdr.type != RXRPC_PACKET_TYPE_DATA)
- return true; /* Just discard */
+ return 0; /* Just discard */
rcu_read_lock();
_leave(" = %p{%d}", call, call->debug_id);
rxrpc_input_call_event(call, skb);
rxrpc_put_call(call, rxrpc_call_put_input);
- return true;
+ return 0;
unsupported_service:
trace_rxrpc_abort(0, "INV", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
reject:
rcu_read_unlock();
_leave(" = f [%u]", skb->mark);
- return false;
+ return -EPROTO;
discard:
rcu_read_unlock();
- return true;
+ return 0;
}
/*
call->tx_total_len = p->tx_total_len;
call->key = key_get(cp->key);
call->local = rxrpc_get_local(cp->local, rxrpc_local_get_call);
+ call->security_level = cp->security_level;
if (p->kernel)
__set_bit(RXRPC_CALL_KERNEL, &call->flags);
if (cp->upgrade)
call->conn = rxrpc_get_connection(conn, rxrpc_conn_get_activate_call);
call->cid = conn->proto.cid | channel;
call->call_id = call_id;
- call->security = conn->security;
- call->security_ix = conn->security_ix;
call->dest_srx.srx_service = conn->service_id;
trace_rxrpc_connect_call(call);
skb->mark = RXRPC_SKB_MARK_REJECT_ABORT;
reject_packet:
rxrpc_reject_packet(local, skb);
- return ret;
+ return 0;
}
/*
if (rxrpc_to_client(sp))
goto bad_message;
if (rxrpc_new_incoming_call(conn->local, conn->peer, conn,
- peer_srx, skb))
+ peer_srx, skb) == 0)
return 0;
goto reject_packet;
}
struct rxrpc_local *local = data;
struct rxrpc_call *call;
struct sk_buff *skb;
+ bool should_stop;
+
+ complete(&local->io_thread_ready);
skb_queue_head_init(&rx_queue);
}
set_current_state(TASK_INTERRUPTIBLE);
+ should_stop = kthread_should_stop();
if (!skb_queue_empty(&local->rx_queue) ||
!list_empty(&local->call_attend_q)) {
__set_current_state(TASK_RUNNING);
continue;
}
- if (kthread_should_stop())
+ if (should_stop)
break;
schedule();
}
local->rxnet = rxnet;
INIT_HLIST_NODE(&local->link);
init_rwsem(&local->defrag_sem);
+ init_completion(&local->io_thread_ready);
skb_queue_head_init(&local->rx_queue);
INIT_LIST_HEAD(&local->call_attend_q);
local->client_bundles = RB_ROOT;
goto error_sock;
}
+ wait_for_completion(&local->io_thread_ready);
local->io_thread = io_thread;
_leave(" = 0");
return 0;
*/
void rxrpc_unuse_local(struct rxrpc_local *local, enum rxrpc_local_trace why)
{
- unsigned int debug_id = local->debug_id;
+ unsigned int debug_id;
int r, u;
if (local) {
+ debug_id = local->debug_id;
r = refcount_read(&local->ref);
u = atomic_dec_return(&local->active_users);
trace_rxrpc_local(debug_id, why, r, u);
struct rxrpc_peer *peer;
const u8 mask = ARRAY_SIZE(rxnet->peer_keepalive) - 1;
time64_t keepalive_at;
+ bool use;
int slot;
spin_lock(&rxnet->peer_hash_lock);
if (!rxrpc_get_peer_maybe(peer, rxrpc_peer_get_keepalive))
continue;
- if (__rxrpc_use_local(peer->local, rxrpc_local_use_peer_keepalive)) {
- spin_unlock(&rxnet->peer_hash_lock);
+ use = __rxrpc_use_local(peer->local, rxrpc_local_use_peer_keepalive);
+ spin_unlock(&rxnet->peer_hash_lock);
+ if (use) {
keepalive_at = peer->last_tx_at + RXRPC_KEEPALIVE_TIME;
slot = keepalive_at - base;
_debug("%02x peer %u t=%d {%pISp}",
spin_lock(&rxnet->peer_hash_lock);
list_add_tail(&peer->keepalive_link,
&rxnet->peer_keepalive[slot & mask]);
+ spin_unlock(&rxnet->peer_hash_lock);
rxrpc_unuse_local(peer->local, rxrpc_local_unuse_peer_keepalive);
}
- rxrpc_put_peer_locked(peer, rxrpc_peer_put_keepalive);
+ rxrpc_put_peer(peer, rxrpc_peer_put_keepalive);
+ spin_lock(&rxnet->peer_hash_lock);
}
spin_unlock(&rxnet->peer_hash_lock);
rxrpc_peer_init_rtt(peer);
peer->cong_ssthresh = RXRPC_TX_MAX_WINDOW;
- trace_rxrpc_peer(peer->debug_id, why, 1);
+ trace_rxrpc_peer(peer->debug_id, 1, why);
}
_leave(" = %p", peer);
int r;
__refcount_inc(&peer->ref, &r);
- trace_rxrpc_peer(peer->debug_id, why, r + 1);
+ trace_rxrpc_peer(peer->debug_id, r + 1, why);
return peer;
}
}
/*
- * Drop a ref on a peer record where the caller already holds the
- * peer_hash_lock.
- */
-void rxrpc_put_peer_locked(struct rxrpc_peer *peer, enum rxrpc_peer_trace why)
-{
- unsigned int debug_id = peer->debug_id;
- bool dead;
- int r;
-
- dead = __refcount_dec_and_test(&peer->ref, &r);
- trace_rxrpc_peer(debug_id, r - 1, why);
- if (dead) {
- hash_del_rcu(&peer->hash_link);
- list_del_init(&peer->keepalive_link);
- rxrpc_free_peer(peer);
- }
-}
-
-/*
* Make sure all peer records have been discarded.
*/
void rxrpc_destroy_all_peers(struct rxrpc_net *rxnet)
struct rxperf_call *call = container_of(work, struct rxperf_call, work);
enum rxperf_call_state state;
u32 abort_code, remote_abort = 0;
- int ret;
+ int ret = 0;
if (call->state == RXPERF_CALL_COMPLETE)
return;
*/
int rxrpc_init_client_call_security(struct rxrpc_call *call)
{
- const struct rxrpc_security *sec;
+ const struct rxrpc_security *sec = &rxrpc_no_security;
struct rxrpc_key_token *token;
struct key *key = call->key;
int ret;
if (!key)
- return 0;
+ goto found;
ret = key_validate(key);
if (ret < 0)
found:
call->security = sec;
- _leave(" = 0");
+ call->security_ix = sec->security_index;
return 0;
}
if (call->tx_total_len != -1 ||
call->tx_pending ||
call->tx_top != 0)
- goto error_put;
+ goto out_put_unlock;
call->tx_total_len = p.call.tx_total_len;
}
}
* the value carried.
*/
if (em_hdr->flags & TCF_EM_SIMPLE) {
+ if (em->ops->datalen > 0)
+ goto errout;
if (data_len < sizeof(u32))
goto errout;
em->data = *(u32 *) data;
}
static const struct kobj_ns_type_operations *
-rpc_sysfs_object_child_ns_type(struct kobject *kobj)
+rpc_sysfs_object_child_ns_type(const struct kobject *kobj)
{
return &net_ns_type_operations;
}
kfree(xprt);
}
-static const void *rpc_sysfs_client_namespace(struct kobject *kobj)
+static const void *rpc_sysfs_client_namespace(const struct kobject *kobj)
{
return container_of(kobj, struct rpc_sysfs_client, kobject)->net;
}
-static const void *rpc_sysfs_xprt_switch_namespace(struct kobject *kobj)
+static const void *rpc_sysfs_xprt_switch_namespace(const struct kobject *kobj)
{
return container_of(kobj, struct rpc_sysfs_xprt_switch, kobject)->net;
}
-static const void *rpc_sysfs_xprt_namespace(struct kobject *kobj)
+static const void *rpc_sysfs_xprt_namespace(const struct kobject *kobj)
{
return container_of(kobj, struct rpc_sysfs_xprt,
kobject)->xprt->xprt_net;
sk->sk_write_space = xs_udp_write_space;
sk->sk_state_change = xs_local_state_change;
sk->sk_error_report = xs_error_report;
+ sk->sk_use_task_frag = false;
xprt_clear_connected(xprt);
sk->sk_user_data = xprt;
sk->sk_data_ready = xs_data_ready;
sk->sk_write_space = xs_udp_write_space;
+ sk->sk_use_task_frag = false;
xprt_set_connected(xprt);
sk->sk_state_change = xs_tcp_state_change;
sk->sk_write_space = xs_tcp_write_space;
sk->sk_error_report = xs_error_report;
+ sk->sk_use_task_frag = false;
/* socket options */
sock_reset_flag(sk, SOCK_LINGER);
unix_state_lock(sk);
err = 0;
- if (unix_peer(sk) == other) {
+ if (sk->sk_type == SOCK_SEQPACKET) {
+ /* We are here only when racing with unix_release_sock()
+ * is clearing @other. Never change state to TCP_CLOSE
+ * unlike SOCK_DGRAM wants.
+ */
+ unix_state_unlock(sk);
+ err = -EPIPE;
+ } else if (unix_peer(sk) == other) {
unix_peer(sk) = NULL;
unix_dgram_peer_wake_disconnect_wakeup(sk, other);
+ sk->sk_state = TCP_CLOSE;
unix_state_unlock(sk);
- sk->sk_state = TCP_CLOSE;
unix_dgram_disconnected(sk, other);
sock_put(other);
err = -ECONNREFUSED;
#define WIPHY_PM_OPS NULL
#endif
-static const void *wiphy_namespace(struct device *d)
+static const void *wiphy_namespace(const struct device *d)
{
struct wiphy *wiphy = container_of(d, struct wiphy, dev);
/* avoid using task_frag */
sk->sk_allocation = GFP_ATOMIC;
+ sk->sk_use_task_frag = false;
return 0;
__u16 vcpu_num;
__u16 vmid;
-/* POST_STANDARD_VM_UUID1, refer to https://github.com/projectacrn/acrn-hypervisor/blob/master/hypervisor/include/common/vm_uuids.h */
-guid_t vm_uuid = GUID_INIT(0x385479d2, 0xd625, 0xe811, 0x86, 0x4e, 0xcb, 0x7a, 0x18, 0xb3, 0x46, 0x43);
int hsm_fd;
int is_running = 1;
}
hsm_fd = open("/dev/acrn_hsm", O_RDWR|O_CLOEXEC);
- memcpy(&create_vm.uuid, &vm_uuid, 16);
create_vm.ioreq_buf = (__u64)io_req_buf;
ret = ioctl(hsm_fd, ACRN_IOCTL_CREATE_VM, &create_vm);
printf("Created VM! [%d]\n", ret);
trace_foo_with_template_print("I have to be different", cnt);
- trace_foo_rel_loc("Hello __rel_loc", cnt, bitmask);
+ trace_foo_rel_loc("Hello __rel_loc", cnt, bitmask, current->cpus_ptr);
}
static void simple_thread_func(int cnt)
*
* __assign_bitmask(target_cpus, cpumask_bits(bar), nr_cpumask_bits);
*
+ * __cpumask: This is pretty much the same as __bitmask but is specific for
+ * CPU masks. The type displayed to the user via the format files will
+ * be "cpumaks_t" such that user space may deal with them differently
+ * if they choose to do so, and the bits is always set to nr_cpumask_bits.
+ *
+ * __cpumask(target_cpu)
+ *
+ * To assign a cpumask, use the __assign_cpumask() helper macro.
+ *
+ * __assign_cpumask(target_cpus, cpumask_bits(bar));
*
* fast_assign: This is a C like function that is used to store the items
* into the ring buffer. A special variable called "__entry" will be the
* This is also used to print out the data from the trace files.
* Again, the __entry macro is used to access the data from the ring buffer.
*
- * Note, __dynamic_array, __string, and __bitmask require special helpers
- * to access the data.
+ * Note, __dynamic_array, __string, __bitmask and __cpumask require special
+ * helpers to access the data.
*
* For __dynamic_array(int, foo, bar) use __get_dynamic_array(foo)
* Use __get_dynamic_array_len(foo) to get the length of the array
*
* For __bitmask(target_cpus, nr_cpumask_bits) use __get_bitmask(target_cpus)
*
+ * For __cpumask(target_cpus) use __get_cpumask(target_cpus)
+ *
*
* Note, that for both the assign and the printk, __entry is the handler
* to the data structure in the ring buffer, and is defined by the
__dynamic_array(int, list, __length_of(lst))
__string( str, string )
__bitmask( cpus, num_possible_cpus() )
+ __cpumask( cpum )
__vstring( vstr, fmt, va )
),
__assign_str(str, string);
__assign_vstr(vstr, fmt, va);
__assign_bitmask(cpus, cpumask_bits(mask), num_possible_cpus());
+ __assign_cpumask(cpum, cpumask_bits(mask));
),
- TP_printk("foo %s %d %s %s %s %s (%s) %s", __entry->foo, __entry->bar,
+ TP_printk("foo %s %d %s %s %s %s (%s) (%s) %s", __entry->foo, __entry->bar,
/*
* Notice here the use of some helper functions. This includes:
__print_array(__get_dynamic_array(list),
__get_dynamic_array_len(list) / sizeof(int),
sizeof(int)),
- __get_str(str), __get_bitmask(cpus), __get_str(vstr))
+ __get_str(str), __get_bitmask(cpus), __get_cpumask(cpum),
+ __get_str(vstr))
);
/*
TRACE_EVENT(foo_rel_loc,
- TP_PROTO(const char *foo, int bar, unsigned long *mask),
+ TP_PROTO(const char *foo, int bar, unsigned long *mask, const cpumask_t *cpus),
- TP_ARGS(foo, bar, mask),
+ TP_ARGS(foo, bar, mask, cpus),
TP_STRUCT__entry(
__rel_string( foo, foo )
__field( int, bar )
__rel_bitmask( bitmask,
BITS_PER_BYTE * sizeof(unsigned long) )
+ __rel_cpumask( cpumask )
),
TP_fast_assign(
__entry->bar = bar;
__assign_rel_bitmask(bitmask, mask,
BITS_PER_BYTE * sizeof(unsigned long));
+ __assign_rel_cpumask(cpumask, cpus);
),
- TP_printk("foo_rel_loc %s, %d, %s", __get_rel_str(foo), __entry->bar,
- __get_rel_bitmask(bitmask))
+ TP_printk("foo_rel_loc %s, %d, %s, %s", __get_rel_str(foo), __entry->bar,
+ __get_rel_bitmask(bitmask),
+ __get_rel_cpumask(cpumask))
);
#endif
space := $(empty) $(empty)
space_escape := _-_SPACE_-_
pound := \#
+define newline
+
+
+endef
+
+###
+# Comparison macros.
+# Usage: $(call test-lt, $(CONFIG_LLD_VERSION), 150000)
+#
+# Use $(intcmp ...) if supported. (Make >= 4.4)
+# Otherwise, fall back to the 'test' shell command.
+ifeq ($(intcmp 1,0,,,y),y)
+test-ge = $(intcmp $(strip $1)0, $(strip $2)0,,y,y)
+test-gt = $(intcmp $(strip $1)0, $(strip $2)0,,,y)
+else
+test-ge = $(shell test $(strip $1)0 -ge $(strip $2)0 && echo y)
+test-gt = $(shell test $(strip $1)0 -gt $(strip $2)0 && echo y)
+endif
+test-le = $(call test-ge, $2, $1)
+test-lt = $(call test-gt, $2, $1)
###
# Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o
stringify = $(squote)$(quote)$1$(quote)$(squote)
###
+# The path to Kbuild or Makefile. Kbuild has precedence over Makefile.
+kbuild-dir = $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
+kbuild-file = $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
+
+###
+# Read a file, replacing newlines with spaces
+#
+# Make 4.2 or later can read a file by using its builtin function.
+ifneq ($(filter-out 3.% 4.0 4.1, $(MAKE_VERSION)),)
+read-file = $(subst $(newline),$(space),$(file < $1))
+else
+read-file = $(shell cat $1 2>/dev/null)
+endif
+
+###
# Easy method for doing a status message
kecho := :
quiet_kecho := echo
make-cmd = $(call escsq,$(subst $(pound),$$(pound),$(subst $$,$$$$,$(cmd_$(1)))))
# Find any prerequisites that are newer than target or that do not exist.
-# (This is not true for now; $? should contain any non-existent prerequisites,
-# but it does not work as expected when .SECONDARY is present. This seems a bug
-# of GNU Make.)
# PHONY targets skipped in both cases.
newer-prereqs = $(filter-out $(PHONY),$?)
.DELETE_ON_ERROR:
# do not delete intermediate files automatically
+#
+# .NOTINTERMEDIATE is more correct, but only available on newer Make versions.
+# Make 4.4 introduced .NOTINTERMEDIATE, and it appears in .FEATURES, but the
+# global .NOTINTERMEDIATE does not work. We can use it on Make > 4.4.
+# Use .SECONDARY for older Make versions, but "newer-prereq" cannot detect
+# deleted files.
+ifneq ($(and $(filter notintermediate, $(.FEATURES)),$(filter-out 4.4,$(MAKE_VERSION))),)
+.NOTINTERMEDIATE:
+else
.SECONDARY:
+endif
all:
src := $(subst /generated,,$(obj))
--include $(src)/Kbuild
+
+include $(srctree)/scripts/Kbuild.include
+-include $(kbuild-file)
# $(generic)/Kbuild lists mandatory-y. Exclude um since it is a special case.
ifneq ($(SRCARCH),um)
include $(srctree)/$(generic)/Kbuild
endif
-include $(srctree)/scripts/Kbuild.include
-
redundant := $(filter $(mandatory-y) $(generated-y), $(generic-y))
redundant += $(foreach f, $(generic-y), $(if $(wildcard $(srctree)/$(src)/$(f)),$(f)))
redundant := $(sort $(redundant))
include $(srctree)/scripts/Kbuild.include
include $(srctree)/scripts/Makefile.compiler
-
-# The filename Kbuild has precedence over Makefile
-kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
-include $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
-
+include $(kbuild-file)
include $(srctree)/scripts/Makefile.lib
# Do not include hostprogs rules unless needed.
cmd_check_local_export = $(srctree)/scripts/check-local-export $@
+ifneq ($(findstring 1, $(KBUILD_EXTRA_WARN)),)
+cmd_warn_shared_object = $(if $(word 2, $(modname-multi)),$(warning $(kbuild-file): $*.o is added to multiple modules: $(modname-multi)))
+endif
+
define rule_cc_o_c
$(call cmd_and_fixdep,cc_o_c)
$(call cmd,gen_ksymdeps)
$(call cmd,gen_objtooldep)
$(call cmd,gen_symversions_c)
$(call cmd,record_mcount)
+ $(call cmd,warn_shared_object)
endef
define rule_as_o_S
$(call cmd,check_local_export)
$(call cmd,gen_objtooldep)
$(call cmd,gen_symversions_S)
+ $(call cmd,warn_shared_object)
endef
# Built-in and composite module parts
# modules.order unless contained modules are updated.
cmd_modules_order = { $(foreach m, $(real-prereqs), \
- $(if $(filter %/modules.order, $m), cat $m, echo $(patsubst %.o,%.ko,$m));) :; } \
+ $(if $(filter %/modules.order, $m), cat $m, echo $m);) :; } \
> $@
$(obj)/modules.order: $(obj-m) FORCE
__clean:
include $(srctree)/scripts/Kbuild.include
-
-# The filename Kbuild has precedence over Makefile
-kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
-include $(or $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Makefile)
+include $(kbuild-file)
# Figure out what we need to build from the various variables
# ==========================================================================
# gcc-min-version
# Usage: cflags-$(call gcc-min-version, 70100) += -foo
-gcc-min-version = $(shell [ $(CONFIG_GCC_VERSION)0 -ge $(1)0 ] && echo y)
+gcc-min-version = $(call test-ge, $(CONFIG_GCC_VERSION), $1)
# clang-min-version
# Usage: cflags-$(call clang-min-version, 110000) += -foo
-clang-min-version = $(shell [ $(CONFIG_CLANG_VERSION)0 -ge $(1)0 ] && echo y)
+clang-min-version = $(call test-ge, $(CONFIG_CLANG_VERSION), $1)
# ld-option
# Usage: KBUILD_LDFLAGS += $(call ld-option, -X, -Y)
DEBUG_RUSTFLAGS += -Cdebuginfo=2
endif
-ifdef CONFIG_DEBUG_INFO_COMPRESSED
+ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZLIB
DEBUG_CFLAGS += -gz=zlib
KBUILD_AFLAGS += -gz=zlib
KBUILD_LDFLAGS += --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS += -gz=zstd
+KBUILD_AFLAGS += -gz=zstd
+KBUILD_LDFLAGS += --compress-debug-sections=zstd
endif
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
include include/config/auto.conf
include $(srctree)/scripts/Kbuild.include
-include $(src)/Makefile
+include $(kbuild-file)
dtbs := $(addprefix $(dst)/, $(dtb-y) $(if $(CONFIG_OF_ALL_DTBS),$(dtb-)))
subdirs := $(addprefix $(obj)/, $(subdir-y) $(subdir-m))
KBUILD_CFLAGS += -Wno-type-limits
KBUILD_CFLAGS += -Wno-shift-negative-value
+KBUILD_CPPFLAGS += -Wundef
KBUILD_CPPFLAGS += -DKBUILD_EXTRA_WARN1
else
objtool-args-$(CONFIG_X86_KERNEL_IBT) += --ibt
objtool-args-$(CONFIG_FINEIBT) += --cfi
objtool-args-$(CONFIG_FTRACE_MCOUNT_USE_OBJTOOL) += --mcount
+ifdef CONFIG_FTRACE_MCOUNT_USE_OBJTOOL
+objtool-args-$(CONFIG_HAVE_OBJTOOL_NOP_MCOUNT) += --mnop
+endif
objtool-args-$(CONFIG_UNWINDER_ORC) += --orc
objtool-args-$(CONFIG_RETPOLINE) += --retpoline
objtool-args-$(CONFIG_RETHUNK) += --rethunk
include $(srctree)/scripts/Makefile.lib
# find all modules listed in modules.order
-modules := $(sort $(shell cat $(MODORDER)))
+modules := $(call read-file, $(MODORDER))
-__modfinal: $(modules)
+__modfinal: $(modules:%.o=%.ko)
@:
# modname and part-of-module are set to make c_flags define proper module flags
printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
# Re-generate module BTFs if either module's .ko or vmlinux changed
-$(modules): %.ko: %.o %.mod.o scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),vmlinux) FORCE
+%.ko: %.o %.mod.o scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),vmlinux) FORCE
+$(call if_changed_except,ld_ko_o,vmlinux)
ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+$(if $(newer-prereqs),$(call cmd,btf_ko))
endif
-targets += $(modules) $(modules:.ko=.mod.o)
+targets += $(modules:%.o=%.ko) $(modules:%.o=%.mod.o)
# Add FORCE to the prequisites of a target to force it to be always rebuilt.
# ---------------------------------------------------------------------------
include include/config/auto.conf
include $(srctree)/scripts/Kbuild.include
-modules := $(sort $(shell cat $(MODORDER)))
+modules := $(call read-file, $(MODORDER))
ifeq ($(KBUILD_EXTMOD),)
dst := $(MODLIB)/kernel
suffix-$(CONFIG_MODULE_COMPRESS_XZ) := .xz
suffix-$(CONFIG_MODULE_COMPRESS_ZSTD) := .zst
-modules := $(patsubst $(extmod_prefix)%, $(dst)/%$(suffix-y), $(modules))
+modules := $(patsubst $(extmod_prefix)%.o, $(dst)/%.ko$(suffix-y), $(modules))
__modinst: $(modules)
@:
include include/config/auto.conf
include $(srctree)/scripts/Kbuild.include
+MODPOST = scripts/mod/modpost
+
modpost-args = \
$(if $(CONFIG_MODVERSIONS),-m) \
$(if $(CONFIG_MODULE_SRCVERSION_ALL),-a) \
$(if $(CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS)$(KBUILD_NSDEPS),-N) \
-o $@
+modpost-deps := $(MODPOST)
+
# 'make -i -k' ignores compile errors, and builds as many modules as possible.
ifneq ($(findstring i,$(filter-out --%,$(MAKEFLAGS))),)
modpost-args += -n
.vmlinux.objs: vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
$(call if_changed,vmlinux_objs)
-vmlinux.o-if-present := $(wildcard vmlinux.o)
-output-symdump := vmlinux.symvers
-
-ifdef KBUILD_MODULES
-output-symdump := $(if $(vmlinux.o-if-present), Module.symvers, modules-only.symvers)
-missing-input := $(filter-out $(vmlinux.o-if-present),vmlinux.o)
+ifeq ($(wildcard vmlinux.o),)
+missing-input := vmlinux.o
+output-symdump := modules-only.symvers
+else
+modpost-args += vmlinux.o
+modpost-deps += vmlinux.o
+output-symdump := $(if $(KBUILD_MODULES), Module.symvers, vmlinux.symvers)
endif
else
src := $(obj)
# Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS
-include $(or $(wildcard $(src)/Kbuild), $(src)/Makefile)
+include $(kbuild-file)
-module.symvers-if-present := $(wildcard Module.symvers)
output-symdump := $(KBUILD_EXTMOD)/Module.symvers
-missing-input := $(filter-out $(module.symvers-if-present), Module.symvers)
-modpost-args += -e $(addprefix -i ,$(module.symvers-if-present) $(KBUILD_EXTRA_SYMBOLS))
+ifeq ($(wildcard Module.symvers),)
+missing-input := Module.symvers
+else
+modpost-args += -i Module.symvers
+modpost-deps += Module.symvers
+endif
+
+modpost-args += -e $(addprefix -i , $(KBUILD_EXTRA_SYMBOLS))
endif # ($(KBUILD_EXTMOD),)
modpost-args += -w
endif
-modorder-if-needed := $(if $(KBUILD_MODULES), $(MODORDER))
-
-MODPOST = scripts/mod/modpost
+ifdef KBUILD_MODULES
+modpost-args += -T $(MODORDER)
+modpost-deps += $(MODORDER)
+endif
# Read out modules.order to pass in modpost.
# Otherwise, allmodconfig would fail with "Argument list too long".
echo >&2 "WARNING: $(missing-input) is missing."; \
echo >&2 " Modules may not have dependencies or modversions."; \
echo >&2 " You may get many unresolved symbol warnings.";) \
- sed 's/ko$$/o/' $(or $(modorder-if-needed), /dev/null) | $(MODPOST) $(modpost-args) -T - $(vmlinux.o-if-present)
+ $(MODPOST) $(modpost-args)
targets += $(output-symdump)
-$(output-symdump): $(modorder-if-needed) $(vmlinux.o-if-present) $(module.symvers-if-present) $(MODPOST) FORCE
+$(output-symdump): $(modpost-deps) FORCE
$(call if_changed,modpost)
__modpost: $(output-symdump)
+rpmbuild $(RPMOPTS) --target $(UTS_MACHINE)-linux -ta $(KERNELPATH).tar.gz \
--define='_smp_mflags %{nil}'
+# srcrpm-pkg
+# ---------------------------------------------------------------------------
+PHONY += srcrpm-pkg
+srcrpm-pkg:
+ $(MAKE) clean
+ $(CONFIG_SHELL) $(MKSPEC) >$(objtree)/kernel.spec
+ $(call cmd,src_tar,$(KERNELPATH),kernel.spec)
+ +rpmbuild $(RPMOPTS) --target $(UTS_MACHINE)-linux -ts $(KERNELPATH).tar.gz \
+ --define='_smp_mflags %{nil}' --define='_srcrpmdir $(srctree)'
+
# binrpm-pkg
# ---------------------------------------------------------------------------
PHONY += binrpm-pkg
"""
with open(modorder) as f:
for line in f:
- ko = line.rstrip()
- base, ext = os.path.splitext(ko)
- if ext != '.ko':
- sys.exit('{}: module path must end with .ko'.format(ko))
+ obj = line.rstrip()
+ base, ext = os.path.splitext(obj)
+ if ext != '.o':
+ sys.exit('{}: module path must end with .o'.format(obj))
mod = base + '.mod'
# Read from *.mod, to get a list of objects that compose the module.
with open(mod) as m:
EOT
{
- [ -n "${read_modorder}" ] && sed 's/ko$/usyms/' modules.order | xargs cat
+ [ -n "${read_modorder}" ] && sed 's/o$/usyms/' modules.order | xargs cat
echo "$needed_symbols"
[ -n "$ksym_wl" ] && cat "$ksym_wl"
} | sed -e 's/ /\n/g' | sed -n -e '/^$/!p' |
opts = [x for x in flags.split(" ") if x.startswith("--jobserver")]
# Parse out R,W file descriptor numbers and set them nonblocking.
- fds = opts[0].split("=", 1)[1]
+ # If the MAKEFLAGS variable contains multiple instances of the
+ # --jobserver-auth= option, the last one is relevant.
+ fds = opts[-1].split("=", 1)[1]
reader, writer = [int(x) for x in fds.split(",", 1)]
# Open a private copy of reader to avoid setting nonblocking
# on an unexpecting process with the same reader fd.
* This software may be used and distributed according to the terms
* of the GNU General Public License, incorporated herein by reference.
*
- * Usage: nm -n vmlinux | scripts/kallsyms [--all-symbols] > symbols.S
+ * Usage: kallsyms [--all-symbols] [--absolute-percpu]
+ * [--base-relative] in.map > out.S
*
* Table compression uses all the unused char codes on the symbols and
* maps these to the most used substrings (tokens). For instance, it might
# SPDX-License-Identifier: GPL-2.0-only
/conf
/[gmnq]conf
-/[gmnq]conf-cfg
+/[gmnq]conf-cflags
+/[gmnq]conf-libs
+/qconf-bin
/qconf-moc.cc
hostprogs += nconf
nconf-objs := nconf.o nconf.gui.o $(common-objs)
-HOSTLDLIBS_nconf = $(shell . $(obj)/nconf-cfg && echo $$libs)
-HOSTCFLAGS_nconf.o = $(shell . $(obj)/nconf-cfg && echo $$cflags)
-HOSTCFLAGS_nconf.gui.o = $(shell . $(obj)/nconf-cfg && echo $$cflags)
+HOSTLDLIBS_nconf = $(call read-file, $(obj)/nconf-libs)
+HOSTCFLAGS_nconf.o = $(call read-file, $(obj)/nconf-cflags)
+HOSTCFLAGS_nconf.gui.o = $(call read-file, $(obj)/nconf-cflags)
-$(obj)/nconf.o $(obj)/nconf.gui.o: $(obj)/nconf-cfg
+$(obj)/nconf: | $(obj)/nconf-libs
+$(obj)/nconf.o $(obj)/nconf.gui.o: | $(obj)/nconf-cflags
# mconf: Used for the menuconfig target based on lxdialog
hostprogs += mconf
checklist.o inputbox.o menubox.o textbox.o util.o yesno.o)
mconf-objs := mconf.o $(lxdialog) $(common-objs)
-HOSTLDLIBS_mconf = $(shell . $(obj)/mconf-cfg && echo $$libs)
+HOSTLDLIBS_mconf = $(call read-file, $(obj)/mconf-libs)
$(foreach f, mconf.o $(lxdialog), \
- $(eval HOSTCFLAGS_$f = $$(shell . $(obj)/mconf-cfg && echo $$$$cflags)))
+ $(eval HOSTCFLAGS_$f = $$(call read-file, $(obj)/mconf-cflags)))
-$(addprefix $(obj)/, mconf.o $(lxdialog)): $(obj)/mconf-cfg
+$(obj)/mconf: | $(obj)/mconf-libs
+$(addprefix $(obj)/, mconf.o $(lxdialog)): | $(obj)/mconf-cflags
# qconf: Used for the xconfig target based on Qt
hostprogs += qconf
qconf-cxxobjs := qconf.o qconf-moc.o
qconf-objs := images.o $(common-objs)
-HOSTLDLIBS_qconf = $(shell . $(obj)/qconf-cfg && echo $$libs)
-HOSTCXXFLAGS_qconf.o = $(shell . $(obj)/qconf-cfg && echo $$cflags)
-HOSTCXXFLAGS_qconf-moc.o = $(shell . $(obj)/qconf-cfg && echo $$cflags)
-
-$(obj)/qconf.o: $(obj)/qconf-cfg
+HOSTLDLIBS_qconf = $(call read-file, $(obj)/qconf-libs)
+HOSTCXXFLAGS_qconf.o = -std=c++11 -fPIC $(call read-file, $(obj)/qconf-cflags)
+HOSTCXXFLAGS_qconf-moc.o = -std=c++11 -fPIC $(call read-file, $(obj)/qconf-cflags)
+$(obj)/qconf: | $(obj)/qconf-libs
+$(obj)/qconf.o $(obj)/qconf-moc.o: | $(obj)/qconf-cflags
quiet_cmd_moc = MOC $@
- cmd_moc = $(shell . $(obj)/qconf-cfg && echo $$moc) $< -o $@
+ cmd_moc = $(call read-file, $(obj)/qconf-bin)/moc $< -o $@
-$(obj)/qconf-moc.cc: $(src)/qconf.h $(obj)/qconf-cfg FORCE
+$(obj)/qconf-moc.cc: $(src)/qconf.h FORCE | $(obj)/qconf-bin
$(call if_changed,moc)
targets += qconf-moc.cc
hostprogs += gconf
gconf-objs := gconf.o images.o $(common-objs)
-HOSTLDLIBS_gconf = $(shell . $(obj)/gconf-cfg && echo $$libs)
-HOSTCFLAGS_gconf.o = $(shell . $(obj)/gconf-cfg && echo $$cflags)
+HOSTLDLIBS_gconf = $(call read-file, $(obj)/gconf-libs)
+HOSTCFLAGS_gconf.o = $(call read-file, $(obj)/gconf-cflags)
-$(obj)/gconf.o: $(obj)/gconf-cfg
+$(obj)/gconf: | $(obj)/gconf-libs
+$(obj)/gconf.o: | $(obj)/gconf-cflags
# check if necessary packages are available, and configure build flags
-filechk_conf_cfg = $(CONFIG_SHELL) $<
+cmd_conf_cfg = $< $(addprefix $(obj)/$*conf-, cflags libs bin)
-$(obj)/%conf-cfg: $(src)/%conf-cfg.sh FORCE
- $(call filechk,conf_cfg)
+$(obj)/%conf-cflags $(obj)/%conf-libs $(obj)/%conf-bin: $(src)/%conf-cfg.sh
+ $(call cmd,conf_cfg)
-clean-files += *conf-cfg
+clean-files += *conf-cflags *conf-libs *conf-bin
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
+cflags=$1
+libs=$2
+
PKG="gtk+-2.0 gmodule-2.0 libglade-2.0"
if [ -z "$(command -v ${HOSTPKG_CONFIG})" ]; then
exit 1
fi
-echo cflags=\"$(${HOSTPKG_CONFIG} --cflags $PKG)\"
-echo libs=\"$(${HOSTPKG_CONFIG} --libs $PKG)\"
+${HOSTPKG_CONFIG} --cflags ${PKG} > ${cflags}
+${HOSTPKG_CONFIG} --libs ${PKG} > ${libs}
void str_free(struct gstr *gs);
void str_append(struct gstr *gs, const char *s);
void str_printf(struct gstr *gs, const char *fmt, ...);
-const char *str_get(struct gstr *gs);
+char *str_get(struct gstr *gs);
/* menu.c */
void _menu_init(void);
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
+cflags=$1
+libs=$2
+
PKG="ncursesw"
PKG2="ncurses"
if [ -n "$(command -v ${HOSTPKG_CONFIG})" ]; then
if ${HOSTPKG_CONFIG} --exists $PKG; then
- echo cflags=\"$(${HOSTPKG_CONFIG} --cflags $PKG)\"
- echo libs=\"$(${HOSTPKG_CONFIG} --libs $PKG)\"
+ ${HOSTPKG_CONFIG} --cflags ${PKG} > ${cflags}
+ ${HOSTPKG_CONFIG} --libs ${PKG} > ${libs}
exit 0
fi
- if ${HOSTPKG_CONFIG} --exists $PKG2; then
- echo cflags=\"$(${HOSTPKG_CONFIG} --cflags $PKG2)\"
- echo libs=\"$(${HOSTPKG_CONFIG} --libs $PKG2)\"
+ if ${HOSTPKG_CONFIG} --exists ${PKG2}; then
+ ${HOSTPKG_CONFIG} --cflags ${PKG2} > ${cflags}
+ ${HOSTPKG_CONFIG} --libs ${PKG2} > ${libs}
exit 0
fi
fi
# (Even if it is installed, some distributions such as openSUSE cannot
# find ncurses by pkg-config.)
if [ -f /usr/include/ncursesw/ncurses.h ]; then
- echo cflags=\"-D_GNU_SOURCE -I/usr/include/ncursesw\"
- echo libs=\"-lncursesw\"
+ echo -D_GNU_SOURCE -I/usr/include/ncursesw > ${cflags}
+ echo -lncursesw > ${libs}
exit 0
fi
if [ -f /usr/include/ncurses/ncurses.h ]; then
- echo cflags=\"-D_GNU_SOURCE -I/usr/include/ncurses\"
- echo libs=\"-lncurses\"
+ echo -D_GNU_SOURCE -I/usr/include/ncurses > ${cflags}
+ echo -lncurses > ${libs}
exit 0
fi
# As a final fallback before giving up, check if $HOSTCC knows of a default
# ncurses installation (e.g. from a vendor-specific sysroot).
if echo '#include <ncurses.h>' | ${HOSTCC} -E - >/dev/null 2>&1; then
- echo cflags=\"-D_GNU_SOURCE\"
- echo libs=\"-lncurses\"
+ echo -D_GNU_SOURCE > ${cflags}
+ echo -lncurses > ${libs}
exit 0
fi
res = get_relations_str(sym_arr, &head);
set_subtitle();
- dres = show_textbox_ext("Search Results", (char *)
- str_get(&res), 0, 0, keys, &vscroll,
- &hscroll, &update_text, (void *)
+ dres = show_textbox_ext("Search Results", str_get(&res), 0, 0,
+ keys, &vscroll, &hscroll, &update_text,
&data);
again = false;
for (i = 0; i < JUMP_NB && keys[i]; i++)
menu = prop->menu;
for (i = 0; menu != &rootmenu && i < 8; menu = menu->parent) {
- bool accessible = menu_is_visible(menu);
-
submenu[i++] = menu;
- if (location == NULL && accessible)
+ if (location == NULL && menu_is_visible(menu))
location = menu;
}
if (head && location) {
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
+cflags=$1
+libs=$2
+
PKG="ncursesw menuw panelw"
PKG2="ncurses menu panel"
if [ -n "$(command -v ${HOSTPKG_CONFIG})" ]; then
if ${HOSTPKG_CONFIG} --exists $PKG; then
- echo cflags=\"$(${HOSTPKG_CONFIG} --cflags $PKG)\"
- echo libs=\"$(${HOSTPKG_CONFIG} --libs $PKG)\"
+ ${HOSTPKG_CONFIG} --cflags ${PKG} > ${cflags}
+ ${HOSTPKG_CONFIG} --libs ${PKG} > ${libs}
exit 0
fi
if ${HOSTPKG_CONFIG} --exists $PKG2; then
- echo cflags=\"$(${HOSTPKG_CONFIG} --cflags $PKG2)\"
- echo libs=\"$(${HOSTPKG_CONFIG} --libs $PKG2)\"
+ ${HOSTPKG_CONFIG} --cflags ${PKG2} > ${cflags}
+ ${HOSTPKG_CONFIG} --libs ${PKG2} > ${libs}
exit 0
fi
fi
# (Even if it is installed, some distributions such as openSUSE cannot
# find ncurses by pkg-config.)
if [ -f /usr/include/ncursesw/ncurses.h ]; then
- echo cflags=\"-D_GNU_SOURCE -I/usr/include/ncursesw\"
- echo libs=\"-lncursesw -lmenuw -lpanelw\"
+ echo -D_GNU_SOURCE -I/usr/include/ncursesw > ${cflags}
+ echo -lncursesw -lmenuw -lpanelw > ${libs}
exit 0
fi
if [ -f /usr/include/ncurses/ncurses.h ]; then
- echo cflags=\"-D_GNU_SOURCE -I/usr/include/ncurses\"
- echo libs=\"-lncurses -lmenu -lpanel\"
+ echo -D_GNU_SOURCE -I/usr/include/ncurses > ${cflags}
+ echo -lncurses -lmenu -lpanel > ${libs}
exit 0
fi
if [ -f /usr/include/ncurses.h ]; then
- echo cflags=\"-D_GNU_SOURCE\"
- echo libs=\"-lncurses -lmenu -lpanel\"
+ echo -D_GNU_SOURCE > ${cflags}
+ echo -lncurses -lmenu -lpanel > ${libs}
exit 0
fi
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
+cflags=$1
+libs=$2
+bin=$3
+
PKG="Qt5Core Qt5Gui Qt5Widgets"
if [ -z "$(command -v ${HOSTPKG_CONFIG})" ]; then
fi
if ${HOSTPKG_CONFIG} --exists $PKG; then
- echo cflags=\"-std=c++11 -fPIC $(${HOSTPKG_CONFIG} --cflags $PKG)\"
- echo libs=\"$(${HOSTPKG_CONFIG} --libs $PKG)\"
- echo moc=\"$(${HOSTPKG_CONFIG} --variable=host_bins Qt5Core)/moc\"
+ ${HOSTPKG_CONFIG} --cflags ${PKG} > ${cflags}
+ ${HOSTPKG_CONFIG} --libs ${PKG} > ${libs}
+ ${HOSTPKG_CONFIG} --variable=host_bins Qt5Core > ${bin}
exit 0
fi
}
/* Retrieve value of growable string */
-const char *str_get(struct gstr *gs)
+char *str_get(struct gstr *gs)
{
return gs->s;
}
case "$1" in
binutils)
- echo 2.23.0
+ echo 2.25.0
;;
gcc)
echo 5.1.0
typedef uint32_t __u32;
typedef uint16_t __u16;
typedef unsigned char __u8;
+
+/* UUID types for backward compatibility, don't use in new code */
typedef struct {
__u8 b[16];
} guid_t;
-/* backwards compatibility, don't use in new code */
-typedef struct {
- __u8 b[16];
-} uuid_le;
typedef struct {
__u8 b[16];
} uuid_t;
+
#define UUID_STRING_LEN 36
+/* MEI UUID type, don't use anywhere else */
+typedef struct {
+ __u8 b[16];
+} uuid_le;
+
/* Big exception to the "don't include kernel headers into userspace, which
* even potentially has different endianness and word sizes, since
* we handle those differences explicitly below */
int i;
if (size % id_size || size < id_size) {
- fatal("%s: sizeof(struct %s_device_id)=%lu is not a modulo "
- "of the size of "
- "section __mod_%s__<identifier>_device_table=%lu.\n"
- "Fix definition of struct %s_device_id "
- "in mod_devicetable.h\n",
+ fatal("%s: sizeof(struct %s_device_id)=%lu is not a modulo of the size of section __mod_%s__<identifier>_device_table=%lu.\n"
+ "Fix definition of struct %s_device_id in mod_devicetable.h\n",
modname, device_id, id_size, device_id, size, device_id);
}
/* Verify last one is a terminator */
for (i = 0; i < id_size; i++ ) {
if (*(uint8_t*)(symval+size-id_size+i)) {
- fprintf(stderr,"%s: struct %s_device_id is %lu bytes. "
- "The last of %lu is:\n",
+ fprintf(stderr,
+ "%s: struct %s_device_id is %lu bytes. The last of %lu is:\n",
modname, device_id, id_size, size / id_size);
for (i = 0; i < id_size; i++ )
fprintf(stderr,"0x%02x ",
*(uint8_t*)(symval+size-id_size+i) );
fprintf(stderr,"\n");
- fatal("%s: struct %s_device_id is not terminated "
- "with a NULL entry!\n", modname, device_id);
+ fatal("%s: struct %s_device_id is not terminated with a NULL entry!\n",
+ modname, device_id);
}
}
}
DEF_FIELD(symval, amba_id, mask);
if ((id & mask) != id)
- fatal("%s: Masked-off bit(s) of AMBA device ID are non-zero: "
- "id=0x%08X, mask=0x%08X. Please fix this driver.\n",
+ fatal("%s: Masked-off bit(s) of AMBA device ID are non-zero: id=0x%08X, mask=0x%08X. Please fix this driver.\n",
filename, id, mask);
p += sprintf(alias, "amba:d");
int nobits = sechdrs[i].sh_type == SHT_NOBITS;
if (!nobits && sechdrs[i].sh_offset > info->size) {
- fatal("%s is truncated. sechdrs[i].sh_offset=%lu > "
- "sizeof(*hrd)=%zu\n", filename,
- (unsigned long)sechdrs[i].sh_offset,
+ fatal("%s is truncated. sechdrs[i].sh_offset=%lu > sizeof(*hrd)=%zu\n",
+ filename, (unsigned long)sechdrs[i].sh_offset,
sizeof(*hdr));
return 0;
}
#define ALL_EXIT_SECTIONS EXIT_SECTIONS, ALL_XXXEXIT_SECTIONS
#define DATA_SECTIONS ".data", ".data.rel"
-#define TEXT_SECTIONS ".text", ".text.unlikely", ".sched.text", \
+#define TEXT_SECTIONS ".text", ".text.*", ".sched.text", \
".kprobes.text", ".cpuidle.text", ".noinstr.text"
#define OTHER_TEXT_SECTIONS ".ref.text", ".head.text", ".spinlock.text", \
- ".fixup", ".entry.text", ".exception.text", ".text.*", \
+ ".fixup", ".entry.text", ".exception.text", \
".coldtext", ".softirqentry.text"
#define INIT_SECTIONS ".init.*"
get_pretty_name(is_function(tosym),
&to_pretty_name, &to_pretty_name_p);
- warn("%s(%s+0x%lx): Section mismatch in reference"
- " from the %s %s%s to the %s %s:%s%s\n",
+ warn("%s(%s+0x%lx): Section mismatch in reference from the %s %s%s to the %s %s:%s%s\n",
modname, fromsec, (long)r->r_offset, from_pretty_name,
fromsym_name, from_pretty_name_p,
to_pretty_name, tosec, tosym_name, to_pretty_name_p);
#define R_RISCV_SUB32 39
#endif
+#ifndef EM_LOONGARCH
+#define EM_LOONGARCH 258
+#endif
+
+#ifndef R_LARCH_SUB32
+#define R_LARCH_SUB32 55
+#endif
+
static void section_rela(const char *modname, struct elf_info *elf,
Elf_Shdr *sechdr)
{
ELF_R_TYPE(r.r_info) == R_RISCV_SUB32)
continue;
break;
+ case EM_LOONGARCH:
+ if (!strcmp("__ex_table", fromsec) &&
+ ELF_R_TYPE(r.r_info) == R_LARCH_SUB32)
+ continue;
+ break;
}
sym = elf->symtab_start + r_sym;
/* Skip special sections */
FILE *in = stdin;
char fname[PATH_MAX];
- if (strcmp(filename, "-") != 0) {
- in = fopen(filename, "r");
- if (!in)
- fatal("Can't open filenames file %s: %m", filename);
- }
+ in = fopen(filename, "r");
+ if (!in)
+ fatal("Can't open filenames file %s: %m", filename);
while (fgets(fname, PATH_MAX, in) != NULL) {
if (strends(fname, "\n"))
read_symbols(fname);
}
- if (in != stdin)
- fclose(in);
+ fclose(in);
}
#define SZ 500
static inline void md4_transform_helper(struct md4_ctx *ctx)
{
- le32_to_cpu_array(ctx->block, sizeof(ctx->block) / sizeof(uint32_t));
+ le32_to_cpu_array(ctx->block, ARRAY_SIZE(ctx->block));
md4_transform(ctx->hash, ctx->block);
}
le32_to_cpu_array(mctx->block, (sizeof(mctx->block) -
sizeof(uint64_t)) / sizeof(uint32_t));
md4_transform(mctx->hash, mctx->block);
- cpu_to_le32_array(mctx->hash, sizeof(mctx->hash) / sizeof(uint32_t));
+ cpu_to_le32_array(mctx->hash, ARRAY_SIZE(mctx->hash));
snprintf(out, len, "%08X%08X%08X%08X",
mctx->hash[0], mctx->hash[1], mctx->hash[2], mctx->hash[3]);
for m in $(sed 's:.*/::' "$1" | sort | uniq -d)
do
echo "error: the following would cause module name conflict:" >&2
- sed -n "/\/$m/s:^: :p" "$1" >&2
+ sed -n "/\/$m/s:^\(.*\)\.o\$: \1.ko:p" "$1" >&2
exit_code=1
done
}
fi
;;
arm64)
- for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo ; do
+ for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do
if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then
cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
break
Priority: optional
Maintainer: $maintainer
Rules-Requires-Root: no
-Build-Depends: bc, rsync, kmod, cpio, bison, flex | flex:native $extra_build_depends
+Build-Depends: bc, rsync, kmod, cpio, bison, flex $extra_build_depends
Homepage: https://www.kernel.org/
Package: $packagename-$version
--exclude=*.o --exclude=*.ko --exclude=*.cmd --exclude=Documentation \
--exclude=.config.old --exclude=.missing-syscalls.d --exclude=*.s"
+test -n "$LOCALVERSION" && MAKE="$MAKE LOCALVERSION=$LOCALVERSION"
+
# We can label the here-doc lines for conditional output to the spec file
#
# Labels:
URL: https://www.kernel.org
$S Source: kernel-$__KERNELRELEASE.tar.gz
Provides: $PROVIDES
+$S BuildRequires: bc binutils bison dwarves elfutils-libelf-devel flex
+$S BuildRequires: gcc make openssl openssl-devel perl python3 rsync
+
# $UTS_MACHINE as a fallback of _arch in case
# /usr/lib/rpm/platform/*/macros was not included.
%define _arch %{?_arch:$UTS_MACHINE}
$S$M
$S %prep
$S %setup -q
+$S rm -f scripts/basic/fixdep scripts/kconfig/conf
+$S rm -f tools/objtool/{fixdep,objtool}
$S
$S %build
$S $MAKE %{?_smp_mflags} KBUILD_BUILD_VERSION=%{release}
#define R_AARCH64_ABS64 257
#endif
+#ifndef EM_LOONGARCH
+#define EM_LOONGARCH 258
+#define R_LARCH_32 1
+#define R_LARCH_64 2
+#define R_LARCH_MARK_LA 20
+#define R_LARCH_SOP_PUSH_PLT_PCREL 29
+#endif
+
#define R_ARM_PC24 1
#define R_ARM_THM_CALL 10
#define R_ARM_CALL 28
return ELF64_R_TYPE(w8(rp->r_info)) != R_AARCH64_CALL26;
}
+static int LARCH32_is_fake_mcount(Elf32_Rel const *rp)
+{
+ switch (ELF64_R_TYPE(w(rp->r_info))) {
+ case R_LARCH_MARK_LA:
+ case R_LARCH_SOP_PUSH_PLT_PCREL:
+ return 0;
+ }
+
+ return 1;
+}
+
+static int LARCH64_is_fake_mcount(Elf64_Rel const *rp)
+{
+ switch (ELF64_R_TYPE(w(rp->r_info))) {
+ case R_LARCH_MARK_LA:
+ case R_LARCH_SOP_PUSH_PLT_PCREL:
+ return 0;
+ }
+
+ return 1;
+}
+
/* 64-bit EM_MIPS has weird ELF64_Rela.r_info.
* http://techpubs.sgi.com/library/manuals/4000/007-4658-001/pdf/007-4658-001.pdf
* We interpret Table 29 Relocation Operation (Elf64_Rel, Elf64_Rela) [p.40]
break;
case EM_IA_64: reltype = R_IA64_IMM64; break;
case EM_MIPS: /* reltype: e_class */ break;
+ case EM_LOONGARCH: /* reltype: e_class */ break;
case EM_PPC: reltype = R_PPC_ADDR32; break;
case EM_PPC64: reltype = R_PPC64_ADDR64; break;
case EM_S390: /* reltype: e_class */ break;
reltype = R_MIPS_32;
is_fake_mcount32 = MIPS32_is_fake_mcount;
}
+ if (w2(ehdr->e_machine) == EM_LOONGARCH) {
+ reltype = R_LARCH_32;
+ is_fake_mcount32 = LARCH32_is_fake_mcount;
+ }
if (do32(ehdr, fname, reltype) < 0)
goto out;
break;
Elf64_r_info = MIPS64_r_info;
is_fake_mcount64 = MIPS64_is_fake_mcount;
}
+ if (w2(ghdr->e_machine) == EM_LOONGARCH) {
+ reltype = R_LARCH_64;
+ is_fake_mcount64 = LARCH64_is_fake_mcount;
+ }
if (do64(ghdr, fname, reltype) < 0)
goto out;
break;
rm -f scripts/extract-cert
rm -f arch/x86/purgatory/kexec-purgatory.c
+
+rm -f scripts/kconfig/[gmnq]conf-cfg
switch (r2(&ehdr->e_machine)) {
case EM_386:
case EM_AARCH64:
+ case EM_LOONGARCH:
case EM_RISCV:
case EM_S390:
case EM_X86_64:
case EM_ARCOMPACT:
case EM_ARCV2:
case EM_ARM:
- case EM_LOONGARCH:
case EM_MICROBLAZE:
case EM_MIPS:
case EM_XTENSA:
char *items[];
};
+/*
+ * These comparators are needed nowhere outside of ima so just define them here.
+ * This pattern should hopefully never be needed outside of ima.
+ */
+static inline bool vfsuid_gt_kuid(vfsuid_t vfsuid, kuid_t kuid)
+{
+ return __vfsuid_val(vfsuid) > __kuid_val(kuid);
+}
+
+static inline bool vfsgid_gt_kgid(vfsgid_t vfsgid, kgid_t kgid)
+{
+ return __vfsgid_val(vfsgid) > __kgid_val(kgid);
+}
+
+static inline bool vfsuid_lt_kuid(vfsuid_t vfsuid, kuid_t kuid)
+{
+ return __vfsuid_val(vfsuid) < __kuid_val(kuid);
+}
+
+static inline bool vfsgid_lt_kgid(vfsgid_t vfsgid, kgid_t kgid)
+{
+ return __vfsgid_val(vfsgid) < __kgid_val(kgid);
+}
+
struct ima_rule_entry {
struct list_head list;
int action;
memset(&res, 0, sizeof(res));
+ res.hw_ops = &sdw_intel_cnl_hw_ops;
res.mmio_base = sdev->bar[HDA_DSP_BAR];
res.shim_base = hdev->desc->sdw_shim_base;
res.alh_base = hdev->desc->sdw_alh_base;
MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA);
MODULE_IMPORT_NS(SND_INTEL_SOUNDWIRE_ACPI);
MODULE_IMPORT_NS(SOUNDWIRE_INTEL_INIT);
+MODULE_IMPORT_NS(SOUNDWIRE_INTEL);
MODULE_AUTHOR("Alan Cox");
MODULE_LICENSE("GPL");
-static char *sound_devnode(struct device *dev, umode_t *mode)
+static char *sound_devnode(const struct device *dev, umode_t *mode)
{
if (MAJOR(dev->devt) == SOUND_MAJOR)
return NULL;
/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
#ifndef TOOLS_ARCH_PARISC_UAPI_ASM_MMAN_FIX_H
#define TOOLS_ARCH_PARISC_UAPI_ASM_MMAN_FIX_H
-#define MADV_DODUMP 70
+#define MADV_DODUMP 17
#define MADV_DOFORK 11
-#define MADV_DONTDUMP 69
+#define MADV_DONTDUMP 16
#define MADV_DONTFORK 10
#define MADV_DONTNEED 4
#define MADV_FREE 8
-#define MADV_HUGEPAGE 67
-#define MADV_MERGEABLE 65
-#define MADV_NOHUGEPAGE 68
+#define MADV_HUGEPAGE 14
+#define MADV_MERGEABLE 12
+#define MADV_NOHUGEPAGE 15
#define MADV_NORMAL 0
#define MADV_RANDOM 1
#define MADV_REMOVE 9
#define MADV_SEQUENTIAL 2
-#define MADV_UNMERGEABLE 66
+#define MADV_UNMERGEABLE 13
#define MADV_WILLNEED 3
#define MAP_ANONYMOUS 0x10
#define MAP_DENYWRITE 0x0800
MAKEFLAGS += --no-print-directory
+INSTALL = install
+
+
+# Use DESTDIR for installing into a different root directory.
+# This is useful for building a package. The program will be
+# installed in this directory as if it was the root directory.
+# Then the build tool can move it later.
+DESTDIR ?=
+DESTDIR_SQ = '$(subst ','\'',$(DESTDIR))'
+
LIBFILE = $(OUTPUT)libapi.a
CFLAGS := $(EXTRA_WARNINGS) $(EXTRA_CFLAGS)
API_IN := $(OUTPUT)libapi-in.o
+ifeq ($(LP64), 1)
+ libdir_relative = lib64
+else
+ libdir_relative = lib
+endif
+
+prefix ?=
+libdir = $(prefix)/$(libdir_relative)
+
+# Shell quotes
+libdir_SQ = $(subst ','\'',$(libdir))
+
all:
export srctree OUTPUT CC LD CFLAGS V
include $(srctree)/tools/build/Makefile.include
+include $(srctree)/tools/scripts/Makefile.include
all: fixdep $(LIBFILE)
$(LIBFILE): $(API_IN)
$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(API_IN)
+define do_install_mkdir
+ if [ ! -d '$(DESTDIR_SQ)$1' ]; then \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$1'; \
+ fi
+endef
+
+define do_install
+ if [ ! -d '$2' ]; then \
+ $(INSTALL) -d -m 755 '$2'; \
+ fi; \
+ $(INSTALL) $1 $(if $3,-m $3,) '$2'
+endef
+
+install_lib: $(LIBFILE)
+ $(call QUIET_INSTALL, $(LIBFILE)) \
+ $(call do_install_mkdir,$(libdir_SQ)); \
+ cp -fpR $(LIBFILE) $(DESTDIR)$(libdir_SQ)
+
+HDRS := cpu.h debug.h io.h
+FD_HDRS := fd/array.h
+FS_HDRS := fs/fs.h fs/tracing_path.h
+INSTALL_HDRS_PFX := $(DESTDIR)$(prefix)/include/api
+INSTALL_HDRS := $(addprefix $(INSTALL_HDRS_PFX)/, $(HDRS))
+INSTALL_FD_HDRS := $(addprefix $(INSTALL_HDRS_PFX)/, $(FD_HDRS))
+INSTALL_FS_HDRS := $(addprefix $(INSTALL_HDRS_PFX)/, $(FS_HDRS))
+
+$(INSTALL_HDRS): $(INSTALL_HDRS_PFX)/%.h: %.h
+ $(call QUIET_INSTALL, $@) \
+ $(call do_install,$<,$(INSTALL_HDRS_PFX)/,644)
+
+$(INSTALL_FD_HDRS): $(INSTALL_HDRS_PFX)/fd/%.h: fd/%.h
+ $(call QUIET_INSTALL, $@) \
+ $(call do_install,$<,$(INSTALL_HDRS_PFX)/fd/,644)
+
+$(INSTALL_FS_HDRS): $(INSTALL_HDRS_PFX)/fs/%.h: fs/%.h
+ $(call QUIET_INSTALL, $@) \
+ $(call do_install,$<,$(INSTALL_HDRS_PFX)/fs/,644)
+
+install_headers: $(INSTALL_HDRS) $(INSTALL_FD_HDRS) $(INSTALL_FS_HDRS)
+ $(call QUIET_INSTALL, libapi_headers)
+
+install: install_lib install_headers
+
clean:
$(call QUIET_CLEAN, libapi) $(RM) $(LIBFILE); \
find $(or $(OUTPUT),.) -name \*.o -or -name \*.o.cmd -or -name \*.o.d | xargs $(RM)
return dir;
}
+int tracing_events__scandir_alphasort(struct dirent ***namelist)
+{
+ char *path = get_tracing_file("events");
+ int ret;
+
+ if (!path) {
+ *namelist = NULL;
+ return 0;
+ }
+
+ ret = scandir(path, namelist, NULL, alphasort);
+ put_events_file(path);
+
+ return ret;
+}
+
int tracing_path__strerror_open_tp(int err, char *buf, size_t size,
const char *sys, const char *name)
{
#include <dirent.h>
DIR *tracing_events__opendir(void);
+int tracing_events__scandir_alphasort(struct dirent ***namelist);
void tracing_path_set(const char *mountpoint);
const char *tracing_path_mount(void);
$(call do_install,$<,$(prefix)/include/bpf,644)
install_headers: $(BPF_GENERATED) $(INSTALL_SRC_HDRS) $(INSTALL_GEN_HDRS)
+ $(call QUIET_INSTALL, libbpf_headers)
install_pkgconfig: $(PC_FILE)
$(call QUIET_INSTALL, $(PC_FILE)) \
endef
define do_install
- if [ ! -d '$(DESTDIR_SQ)$2' ]; then \
- $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$2'; \
- fi; \
- $(INSTALL) $1 $(if $3,-m $3,) '$(DESTDIR_SQ)$2'
+ if [ ! -d '$2' ]; then \
+ $(INSTALL) -d -m 755 '$2'; \
+ fi; \
+ $(INSTALL) $1 $(if $3,-m $3,) '$2'
endef
install_lib: libs
$(call do_install_mkdir,$(libdir_SQ)); \
cp -fpR $(LIBPERF_ALL) $(DESTDIR)$(libdir_SQ)
-install_headers:
- $(call QUIET_INSTALL, headers) \
- $(call do_install,include/perf/core.h,$(prefix)/include/perf,644); \
- $(call do_install,include/perf/cpumap.h,$(prefix)/include/perf,644); \
- $(call do_install,include/perf/threadmap.h,$(prefix)/include/perf,644); \
- $(call do_install,include/perf/evlist.h,$(prefix)/include/perf,644); \
- $(call do_install,include/perf/evsel.h,$(prefix)/include/perf,644); \
- $(call do_install,include/perf/event.h,$(prefix)/include/perf,644); \
- $(call do_install,include/perf/mmap.h,$(prefix)/include/perf,644);
+HDRS := bpf_perf.h core.h cpumap.h threadmap.h evlist.h evsel.h event.h mmap.h
+INTERNAL_HDRS := cpumap.h evlist.h evsel.h lib.h mmap.h threadmap.h xyarray.h
+
+INSTALL_HDRS_PFX := $(DESTDIR)$(prefix)/include/perf
+INSTALL_HDRS := $(addprefix $(INSTALL_HDRS_PFX)/, $(HDRS))
+INSTALL_INTERNAL_HDRS_PFX := $(DESTDIR)$(prefix)/include/internal
+INSTALL_INTERNAL_HDRS := $(addprefix $(INSTALL_INTERNAL_HDRS_PFX)/, $(INTERNAL_HDRS))
+
+$(INSTALL_HDRS): $(INSTALL_HDRS_PFX)/%.h: include/perf/%.h
+ $(call QUIET_INSTALL, $@) \
+ $(call do_install,$<,$(INSTALL_HDRS_PFX)/,644)
+
+$(INSTALL_INTERNAL_HDRS): $(INSTALL_INTERNAL_HDRS_PFX)/%.h: include/internal/%.h
+ $(call QUIET_INSTALL, $@) \
+ $(call do_install,$<,$(INSTALL_INTERNAL_HDRS_PFX)/,644)
+
+install_headers: $(INSTALL_HDRS) $(INSTALL_INTERNAL_HDRS)
+ $(call QUIET_INSTALL, libperf_headers)
install_pkgconfig: $(LIBPERF_PC)
$(call QUIET_INSTALL, $(LIBPERF_PC)) \
- $(call do_install,$(LIBPERF_PC),$(libdir_SQ)/pkgconfig,644)
+ $(call do_install,$(LIBPERF_PC),$(DESTDIR_SQ)$(libdir_SQ)/pkgconfig,644)
install_doc:
$(Q)$(MAKE) -C Documentation install-man install-html install-examples
#define __LIBPERF_CPUMAP_H
#include <perf/core.h>
-#include <perf/cpumap.h>
#include <stdio.h>
#include <stdbool.h>
int cpu;
};
+struct perf_cpu_map;
+
LIBPERF_API struct perf_cpu_map *perf_cpu_map__dummy_new(void);
LIBPERF_API struct perf_cpu_map *perf_cpu_map__default_new(void);
LIBPERF_API struct perf_cpu_map *perf_cpu_map__new(const char *cpu_list);
MAKEFLAGS += --no-print-directory
+INSTALL = install
+
+# Use DESTDIR for installing into a different root directory.
+# This is useful for building a package. The program will be
+# installed in this directory as if it was the root directory.
+# Then the build tool can move it later.
+DESTDIR ?=
+DESTDIR_SQ = '$(subst ','\'',$(DESTDIR))'
+
LIBFILE = $(OUTPUT)libsubcmd.a
CFLAGS := -ggdb3 -Wall -Wextra -std=gnu99 -fPIC
SUBCMD_IN := $(OUTPUT)libsubcmd-in.o
+ifeq ($(LP64), 1)
+ libdir_relative = lib64
+else
+ libdir_relative = lib
+endif
+
+prefix ?=
+libdir = $(prefix)/$(libdir_relative)
+
+# Shell quotes
+libdir_SQ = $(subst ','\'',$(libdir))
+
all:
export srctree OUTPUT CC LD CFLAGS V
$(LIBFILE): $(SUBCMD_IN)
$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(SUBCMD_IN)
+define do_install_mkdir
+ if [ ! -d '$(DESTDIR_SQ)$1' ]; then \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$1'; \
+ fi
+endef
+
+define do_install
+ if [ ! -d '$2' ]; then \
+ $(INSTALL) -d -m 755 '$2'; \
+ fi; \
+ $(INSTALL) $1 $(if $3,-m $3,) '$2'
+endef
+
+install_lib: $(LIBFILE)
+ $(call QUIET_INSTALL, $(LIBFILE)) \
+ $(call do_install_mkdir,$(libdir_SQ)); \
+ cp -fpR $(LIBFILE) $(DESTDIR)$(libdir_SQ)
+
+HDRS := exec-cmd.h help.h pager.h parse-options.h run-command.h
+INSTALL_HDRS_PFX := $(DESTDIR)$(prefix)/include/subcmd
+INSTALL_HDRS := $(addprefix $(INSTALL_HDRS_PFX)/, $(HDRS))
+
+$(INSTALL_HDRS): $(INSTALL_HDRS_PFX)/%.h: %.h
+ $(call QUIET_INSTALL, $@) \
+ $(call do_install,$<,$(INSTALL_HDRS_PFX)/,644)
+
+install_headers: $(INSTALL_HDRS)
+ $(call QUIET_INSTALL, libsubcmd_headers)
+
+install: install_lib install_headers
+
clean:
$(call QUIET_CLEAN, libsubcmd) $(RM) $(LIBFILE); \
find $(or $(OUTPUT),.) -name \*.o -or -name \*.o.cmd -or -name \*.o.d | xargs $(RM)
--- /dev/null
+libsymbol-y += kallsyms.o
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+include ../../scripts/Makefile.include
+include ../../scripts/utilities.mak # QUIET_CLEAN
+
+ifeq ($(srctree),)
+srctree := $(patsubst %/,%,$(dir $(CURDIR)))
+srctree := $(patsubst %/,%,$(dir $(srctree)))
+srctree := $(patsubst %/,%,$(dir $(srctree)))
+#$(info Determined 'srctree' to be $(srctree))
+endif
+
+CC ?= $(CROSS_COMPILE)gcc
+AR ?= $(CROSS_COMPILE)ar
+LD ?= $(CROSS_COMPILE)ld
+
+MAKEFLAGS += --no-print-directory
+
+INSTALL = install
+
+
+# Use DESTDIR for installing into a different root directory.
+# This is useful for building a package. The program will be
+# installed in this directory as if it was the root directory.
+# Then the build tool can move it later.
+DESTDIR ?=
+DESTDIR_SQ = '$(subst ','\'',$(DESTDIR))'
+
+LIBFILE = $(OUTPUT)libsymbol.a
+
+CFLAGS := $(EXTRA_WARNINGS) $(EXTRA_CFLAGS)
+CFLAGS += -ggdb3 -Wall -Wextra -std=gnu11 -U_FORTIFY_SOURCE -fPIC
+
+ifeq ($(DEBUG),0)
+ifeq ($(CC_NO_CLANG), 0)
+ CFLAGS += -O3
+else
+ CFLAGS += -O6
+endif
+endif
+
+ifeq ($(DEBUG),0)
+ CFLAGS += -D_FORTIFY_SOURCE
+endif
+
+# Treat warnings as errors unless directed not to
+ifneq ($(WERROR),0)
+ CFLAGS += -Werror
+endif
+
+CFLAGS += -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
+
+CFLAGS += -I$(srctree)/tools/lib
+CFLAGS += -I$(srctree)/tools/include
+
+RM = rm -f
+
+SYMBOL_IN := $(OUTPUT)libsymbol-in.o
+
+ifeq ($(LP64), 1)
+ libdir_relative = lib64
+else
+ libdir_relative = lib
+endif
+
+prefix ?=
+libdir = $(prefix)/$(libdir_relative)
+
+# Shell quotes
+libdir_SQ = $(subst ','\'',$(libdir))
+
+all:
+
+export srctree OUTPUT CC LD CFLAGS V
+include $(srctree)/tools/build/Makefile.include
+include $(srctree)/tools/scripts/Makefile.include
+
+all: fixdep $(LIBFILE)
+
+$(SYMBOL_IN): FORCE
+ @$(MAKE) $(build)=libsymbol
+
+$(LIBFILE): $(SYMBOL_IN)
+ $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(SYMBOL_IN)
+
+define do_install_mkdir
+ if [ ! -d '$(DESTDIR_SQ)$1' ]; then \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$1'; \
+ fi
+endef
+
+define do_install
+ if [ ! -d '$2' ]; then \
+ $(INSTALL) -d -m 755 '$2'; \
+ fi; \
+ $(INSTALL) $1 $(if $3,-m $3,) '$2'
+endef
+
+install_lib: $(LIBFILE)
+ $(call QUIET_INSTALL, $(LIBFILE)) \
+ $(call do_install_mkdir,$(libdir_SQ)); \
+ cp -fpR $(LIBFILE) $(DESTDIR)$(libdir_SQ)
+
+HDRS := kallsyms.h
+INSTALL_HDRS_PFX := $(DESTDIR)$(prefix)/include/symbol
+INSTALL_HDRS := $(addprefix $(INSTALL_HDRS_PFX)/, $(HDRS))
+
+$(INSTALL_HDRS): $(INSTALL_HDRS_PFX)/%.h: %.h
+ $(call QUIET_INSTALL, $@) \
+ $(call do_install,$<,$(INSTALL_HDRS_PFX)/,644)
+
+install_headers: $(INSTALL_HDRS)
+ $(call QUIET_INSTALL, libsymbol_headers)
+
+install: install_lib install_headers
+
+clean:
+ $(call QUIET_CLEAN, libsymbol) $(RM) $(LIBFILE); \
+ find $(or $(OUTPUT),.) -name \*.o -or -name \*.o.cmd -or -name \*.o.d | xargs $(RM)
+
+FORCE:
+
+.PHONY: clean FORCE
+++ /dev/null
-# SPDX-License-Identifier: GPL-2.0-only
-TRACEEVENT-CFLAGS
-libtraceevent-dynamic-list
-libtraceevent.so.*
+++ /dev/null
-libtraceevent-y += event-parse.o
-libtraceevent-y += event-plugin.o
-libtraceevent-y += trace-seq.o
-libtraceevent-y += parse-filter.o
-libtraceevent-y += parse-utils.o
-libtraceevent-y += kbuffer-parse.o
-libtraceevent-y += tep_strerror.o
-libtraceevent-y += event-parse-api.o
+++ /dev/null
-include ../../../scripts/Makefile.include
-include ../../../scripts/utilities.mak
-
-# This Makefile and manpage XSL files were taken from tools/perf/Documentation
-# and modified for libtraceevent.
-
-MAN3_TXT= \
- $(wildcard libtraceevent-*.txt) \
- libtraceevent.txt
-
-MAN_TXT = $(MAN3_TXT)
-_MAN_XML=$(patsubst %.txt,%.xml,$(MAN_TXT))
-_MAN_HTML=$(patsubst %.txt,%.html,$(MAN_TXT))
-_DOC_MAN3=$(patsubst %.txt,%.3,$(MAN3_TXT))
-
-MAN_XML=$(addprefix $(OUTPUT),$(_MAN_XML))
-MAN_HTML=$(addprefix $(OUTPUT),$(_MAN_HTML))
-DOC_MAN3=$(addprefix $(OUTPUT),$(_DOC_MAN3))
-
-# Make the path relative to DESTDIR, not prefix
-ifndef DESTDIR
-prefix?=$(HOME)
-endif
-bindir?=$(prefix)/bin
-htmldir?=$(prefix)/share/doc/libtraceevent-doc
-pdfdir?=$(prefix)/share/doc/libtraceevent-doc
-mandir?=$(prefix)/share/man
-man3dir=$(mandir)/man3
-
-ASCIIDOC=asciidoc
-ASCIIDOC_EXTRA = --unsafe -f asciidoc.conf
-ASCIIDOC_HTML = xhtml11
-MANPAGE_XSL = manpage-normal.xsl
-XMLTO_EXTRA =
-INSTALL?=install
-RM ?= rm -f
-
-ifdef USE_ASCIIDOCTOR
-ASCIIDOC = asciidoctor
-ASCIIDOC_EXTRA = -a compat-mode
-ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions
-ASCIIDOC_EXTRA += -a mansource="libtraceevent" -a manmanual="libtraceevent Manual"
-ASCIIDOC_HTML = xhtml5
-endif
-
-XMLTO=xmlto
-
-_tmp_tool_path := $(call get-executable,$(ASCIIDOC))
-ifeq ($(_tmp_tool_path),)
- missing_tools = $(ASCIIDOC)
-endif
-
-ifndef USE_ASCIIDOCTOR
-_tmp_tool_path := $(call get-executable,$(XMLTO))
-ifeq ($(_tmp_tool_path),)
- missing_tools += $(XMLTO)
-endif
-endif
-
-#
-# For asciidoc ...
-# -7.1.2, no extra settings are needed.
-# 8.0-, set ASCIIDOC8.
-#
-
-#
-# For docbook-xsl ...
-# -1.68.1, set ASCIIDOC_NO_ROFF? (based on changelog from 1.73.0)
-# 1.69.0, no extra settings are needed?
-# 1.69.1-1.71.0, set DOCBOOK_SUPPRESS_SP?
-# 1.71.1, no extra settings are needed?
-# 1.72.0, set DOCBOOK_XSL_172.
-# 1.73.0-, set ASCIIDOC_NO_ROFF
-#
-
-#
-# If you had been using DOCBOOK_XSL_172 in an attempt to get rid
-# of 'the ".ft C" problem' in your generated manpages, and you
-# instead ended up with weird characters around callouts, try
-# using ASCIIDOC_NO_ROFF instead (it works fine with ASCIIDOC8).
-#
-
-ifdef ASCIIDOC8
-ASCIIDOC_EXTRA += -a asciidoc7compatible
-endif
-ifdef DOCBOOK_XSL_172
-ASCIIDOC_EXTRA += -a libtraceevent-asciidoc-no-roff
-MANPAGE_XSL = manpage-1.72.xsl
-else
- ifdef ASCIIDOC_NO_ROFF
- # docbook-xsl after 1.72 needs the regular XSL, but will not
- # pass-thru raw roff codes from asciidoc.conf, so turn them off.
- ASCIIDOC_EXTRA += -a libtraceevent-asciidoc-no-roff
- endif
-endif
-ifdef MAN_BOLD_LITERAL
-XMLTO_EXTRA += -m manpage-bold-literal.xsl
-endif
-ifdef DOCBOOK_SUPPRESS_SP
-XMLTO_EXTRA += -m manpage-suppress-sp.xsl
-endif
-
-SHELL_PATH ?= $(SHELL)
-# Shell quote;
-SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
-
-DESTDIR ?=
-DESTDIR_SQ = '$(subst ','\'',$(DESTDIR))'
-
-export DESTDIR DESTDIR_SQ
-
-#
-# Please note that there is a minor bug in asciidoc.
-# The version after 6.0.3 _will_ include the patch found here:
-# http://marc.theaimsgroup.com/?l=libtraceevent&m=111558757202243&w=2
-#
-# Until that version is released you may have to apply the patch
-# yourself - yes, all 6 characters of it!
-#
-QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir
-QUIET_SUBDIR1 =
-
-ifneq ($(findstring $(MAKEFLAGS),w),w)
-PRINT_DIR = --no-print-directory
-else # "make -w"
-NO_SUBDIR = :
-endif
-
-ifneq ($(findstring $(MAKEFLAGS),s),s)
-ifneq ($(V),1)
- QUIET_ASCIIDOC = @echo ' ASCIIDOC '$@;
- QUIET_XMLTO = @echo ' XMLTO '$@;
- QUIET_SUBDIR0 = +@subdir=
- QUIET_SUBDIR1 = ;$(NO_SUBDIR) \
- echo ' SUBDIR ' $$subdir; \
- $(MAKE) $(PRINT_DIR) -C $$subdir
- export V
-endif
-endif
-
-all: html man
-
-man: man3
-man3: $(DOC_MAN3)
-
-html: $(MAN_HTML)
-
-$(MAN_HTML) $(DOC_MAN3): asciidoc.conf
-
-install: install-man
-
-check-man-tools:
-ifdef missing_tools
- $(error "You need to install $(missing_tools) for man pages")
-endif
-
-do-install-man: man
- $(call QUIET_INSTALL, Documentation-man) \
- $(INSTALL) -d -m 755 $(DESTDIR)$(man3dir); \
- $(INSTALL) -m 644 $(DOC_MAN3) $(DESTDIR)$(man3dir);
-
-install-man: check-man-tools man do-install-man
-
-uninstall: uninstall-man
-
-uninstall-man:
- $(call QUIET_UNINST, Documentation-man) \
- $(Q)$(RM) $(addprefix $(DESTDIR)$(man3dir)/,$(DOC_MAN3))
-
-
-ifdef missing_tools
- DO_INSTALL_MAN = $(warning Please install $(missing_tools) to have the man pages installed)
-else
- DO_INSTALL_MAN = do-install-man
-endif
-
-CLEAN_FILES = \
- $(MAN_XML) $(addsuffix +,$(MAN_XML)) \
- $(MAN_HTML) $(addsuffix +,$(MAN_HTML)) \
- $(DOC_MAN3) *.3
-
-clean:
- $(call QUIET_CLEAN, Documentation) $(RM) $(CLEAN_FILES)
-
-ifdef USE_ASCIIDOCTOR
-$(OUTPUT)%.3 : $(OUTPUT)%.txt
- $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
- $(ASCIIDOC) -b manpage -d manpage \
- $(ASCIIDOC_EXTRA) -alibtraceevent_version=$(EVENT_PARSE_VERSION) -o $@+ $< && \
- mv $@+ $@
-endif
-
-$(OUTPUT)%.3 : $(OUTPUT)%.xml
- $(QUIET_XMLTO)$(RM) $@ && \
- $(XMLTO) -o $(OUTPUT). -m $(MANPAGE_XSL) $(XMLTO_EXTRA) man $<
-
-$(OUTPUT)%.xml : %.txt
- $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
- $(ASCIIDOC) -b docbook -d manpage \
- $(ASCIIDOC_EXTRA) -alibtraceevent_version=$(EVENT_PARSE_VERSION) -o $@+ $< && \
- mv $@+ $@
-
-$(MAN_HTML): $(OUTPUT)%.html : %.txt
- $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
- $(ASCIIDOC) -b $(ASCIIDOC_HTML) -d manpage \
- $(ASCIIDOC_EXTRA) -aperf_version=$(EVENT_PARSE_VERSION) -o $@+ $< && \
- mv $@+ $@
+++ /dev/null
-## linktep: macro
-#
-# Usage: linktep:command[manpage-section]
-#
-# Note, {0} is the manpage section, while {target} is the command.
-#
-# Show TEP link as: <command>(<section>); if section is defined, else just show
-# the command.
-
-[macros]
-(?su)[\\]?(?P<name>linktep):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
-
-[attributes]
-asterisk=*
-plus=+
-caret=^
-startsb=[
-endsb=]
-tilde=~
-
-ifdef::backend-docbook[]
-[linktep-inlinemacro]
-{0%{target}}
-{0#<citerefentry>}
-{0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
-{0#</citerefentry>}
-endif::backend-docbook[]
-
-ifdef::backend-docbook[]
-ifndef::tep-asciidoc-no-roff[]
-# "unbreak" docbook-xsl v1.68 for manpages. v1.69 works with or without this.
-# v1.72 breaks with this because it replaces dots not in roff requests.
-[listingblock]
-<example><title>{title}</title>
-<literallayout>
-ifdef::doctype-manpage[]
- .ft C
-endif::doctype-manpage[]
-|
-ifdef::doctype-manpage[]
- .ft
-endif::doctype-manpage[]
-</literallayout>
-{title#}</example>
-endif::tep-asciidoc-no-roff[]
-
-ifdef::tep-asciidoc-no-roff[]
-ifdef::doctype-manpage[]
-# The following two small workarounds insert a simple paragraph after screen
-[listingblock]
-<example><title>{title}</title>
-<literallayout>
-|
-</literallayout><simpara></simpara>
-{title#}</example>
-
-[verseblock]
-<formalpara{id? id="{id}"}><title>{title}</title><para>
-{title%}<literallayout{id? id="{id}"}>
-{title#}<literallayout>
-|
-</literallayout>
-{title#}</para></formalpara>
-{title%}<simpara></simpara>
-endif::doctype-manpage[]
-endif::tep-asciidoc-no-roff[]
-endif::backend-docbook[]
-
-ifdef::doctype-manpage[]
-ifdef::backend-docbook[]
-[header]
-template::[header-declarations]
-<refentry>
-<refmeta>
-<refentrytitle>{mantitle}</refentrytitle>
-<manvolnum>{manvolnum}</manvolnum>
-<refmiscinfo class="source">libtraceevent</refmiscinfo>
-<refmiscinfo class="version">{libtraceevent_version}</refmiscinfo>
-<refmiscinfo class="manual">libtraceevent Manual</refmiscinfo>
-</refmeta>
-<refnamediv>
- <refname>{manname1}</refname>
- <refname>{manname2}</refname>
- <refname>{manname3}</refname>
- <refname>{manname4}</refname>
- <refname>{manname5}</refname>
- <refname>{manname6}</refname>
- <refname>{manname7}</refname>
- <refname>{manname8}</refname>
- <refname>{manname9}</refname>
- <refname>{manname10}</refname>
- <refname>{manname11}</refname>
- <refname>{manname12}</refname>
- <refname>{manname13}</refname>
- <refname>{manname14}</refname>
- <refname>{manname15}</refname>
- <refname>{manname16}</refname>
- <refname>{manname17}</refname>
- <refname>{manname18}</refname>
- <refname>{manname19}</refname>
- <refname>{manname20}</refname>
- <refname>{manname21}</refname>
- <refname>{manname22}</refname>
- <refname>{manname23}</refname>
- <refname>{manname24}</refname>
- <refname>{manname25}</refname>
- <refname>{manname26}</refname>
- <refname>{manname27}</refname>
- <refname>{manname28}</refname>
- <refname>{manname29}</refname>
- <refname>{manname30}</refname>
- <refpurpose>{manpurpose}</refpurpose>
-</refnamediv>
-endif::backend-docbook[]
-endif::doctype-manpage[]
-
-ifdef::backend-xhtml11[]
-[linktep-inlinemacro]
-<a href="{target}.html">{target}{0?({0})}</a>
-endif::backend-xhtml11[]
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_register_comm, tep_override_comm, tep_pid_is_registered,
-tep_data_comm_from_pid, tep_data_pid_from_comm, tep_cmdline_pid -
-Manage pid to process name mappings.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-int *tep_register_comm*(struct tep_handle pass:[*]_tep_, const char pass:[*]_comm_, int _pid_);
-int *tep_override_comm*(struct tep_handle pass:[*]_tep_, const char pass:[*]_comm_, int _pid_);
-bool *tep_is_pid_registered*(struct tep_handle pass:[*]_tep_, int _pid_);
-const char pass:[*]*tep_data_comm_from_pid*(struct tep_handle pass:[*]_pevent_, int _pid_);
-struct cmdline pass:[*]*tep_data_pid_from_comm*(struct tep_handle pass:[*]_pevent_, const char pass:[*]_comm_, struct cmdline pass:[*]_next_);
-int *tep_cmdline_pid*(struct tep_handle pass:[*]_pevent_, struct cmdline pass:[*]_cmdline_);
---
-
-DESCRIPTION
------------
-These functions can be used to handle the mapping between pid and process name.
-The library builds a cache of these mappings, which is used to display the name
-of the process, instead of its pid. This information can be retrieved from
-tracefs/saved_cmdlines file.
-
-The _tep_register_comm()_ function registers a _pid_ / process name mapping.
-If a command with the same _pid_ is already registered, an error is returned.
-The _pid_ argument is the process ID, the _comm_ argument is the process name,
-_tep_ is the event context. The _comm_ is duplicated internally.
-
-The _tep_override_comm()_ function registers a _pid_ / process name mapping.
-If a process with the same pid is already registered, the process name string is
-udapted with the new one. The _pid_ argument is the process ID, the _comm_
-argument is the process name, _tep_ is the event context. The _comm_ is
-duplicated internally.
-
-The _tep_is_pid_registered()_ function checks if a pid has a process name
-mapping registered. The _pid_ argument is the process ID, _tep_ is the event
-context.
-
-The _tep_data_comm_from_pid()_ function returns the process name for a given
-pid. The _pid_ argument is the process ID, _tep_ is the event context.
-The returned string should not be freed, but will be freed when the _tep_
-handler is closed.
-
-The _tep_data_pid_from_comm()_ function returns a pid for a given process name.
-The _comm_ argument is the process name, _tep_ is the event context.
-The argument _next_ is the cmdline structure to search for the next pid.
-As there may be more than one pid for a given process, the result of this call
-can be passed back into a recurring call in the _next_ parameter, to search for
-the next pid. If _next_ is NULL, it will return the first pid associated with
-the _comm_. The function performs a linear search, so it may be slow.
-
-The _tep_cmdline_pid()_ function returns the pid associated with a given
-_cmdline_. The _tep_ argument is the event context.
-
-RETURN VALUE
-------------
-_tep_register_comm()_ function returns 0 on success. In case of an error -1 is
-returned and errno is set to indicate the cause of the problem: ENOMEM, if there
-is not enough memory to duplicate the _comm_ or EEXIST if a mapping for this
-_pid_ is already registered.
-
-_tep_override_comm()_ function returns 0 on success. In case of an error -1 is
-returned and errno is set to indicate the cause of the problem: ENOMEM, if there
-is not enough memory to duplicate the _comm_.
-
-_tep_is_pid_registered()_ function returns true if the _pid_ has a process name
-mapped to it, false otherwise.
-
-_tep_data_comm_from_pid()_ function returns the process name as string, or the
-string "<...>" if there is no mapping for the given pid.
-
-_tep_data_pid_from_comm()_ function returns a pointer to a struct cmdline, that
-holds a pid for a given process, or NULL if none is found. This result can be
-passed back into a recurring call as the _next_ parameter of the function.
-
-_tep_cmdline_pid()_ functions returns the pid for the give cmdline. If _cmdline_
- is NULL, then -1 is returned.
-
-EXAMPLE
--------
-The following example registers pid for command "ls", in context of event _tep_
-and performs various searches for pid / process name mappings:
-[source,c]
---
-#include <event-parse.h>
-...
-int ret;
-int ls_pid = 1021;
-struct tep_handle *tep = tep_alloc();
-...
- ret = tep_register_comm(tep, "ls", ls_pid);
- if (ret != 0 && errno == EEXIST)
- ret = tep_override_comm(tep, "ls", ls_pid);
- if (ret != 0) {
- /* Failed to register pid / command mapping */
- }
-...
- if (tep_is_pid_registered(tep, ls_pid) == 0) {
- /* Command mapping for ls_pid is not registered */
- }
-...
- const char *comm = tep_data_comm_from_pid(tep, ls_pid);
- if (comm) {
- /* Found process name for ls_pid */
- }
-...
- int pid;
- struct cmdline *cmd = tep_data_pid_from_comm(tep, "ls", NULL);
- while (cmd) {
- pid = tep_cmdline_pid(tep, cmd);
- /* Found pid for process "ls" */
- cmd = tep_data_pid_from_comm(tep, "ls", cmd);
- }
---
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_get_cpus, tep_set_cpus - Get / set the number of CPUs, which have a tracing
-buffer representing it. Note, the buffer may be empty.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-int *tep_get_cpus*(struct tep_handle pass:[*]_tep_);
-void *tep_set_cpus*(struct tep_handle pass:[*]_tep_, int _cpus_);
---
-
-DESCRIPTION
------------
-The _tep_get_cpus()_ function gets the number of CPUs, which have a tracing
-buffer representing it. The _tep_ argument is trace event parser context.
-
-The _tep_set_cpus()_ function sets the number of CPUs, which have a tracing
-buffer representing it. The _tep_ argument is trace event parser context.
-The _cpu_ argument is the number of CPUs with tracing data.
-
-RETURN VALUE
-------------
-The _tep_get_cpus()_ functions returns the number of CPUs, which have tracing
-data recorded.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
- tep_set_cpus(tep, 5);
-...
- printf("We have tracing data for %d CPUs", tep_get_cpus(tep));
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_read_number - Reads a number from raw data.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-unsigned long long *tep_read_number*(struct tep_handle pass:[*]_tep_, const void pass:[*]_ptr_, int _size_);
---
-
-DESCRIPTION
------------
-The _tep_read_number()_ function reads an integer from raw data, taking into
-account the endianness of the raw data and the current host. The _tep_ argument
-is the trace event parser context. The _ptr_ is a pointer to the raw data, where
-the integer is, and the _size_ is the size of the integer.
-
-RETURN VALUE
-------------
-The _tep_read_number()_ function returns the integer in the byte order of
-the current host. In case of an error, 0 is returned.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-void process_record(struct tep_record *record)
-{
- int offset = 24;
- int data = tep_read_number(tep, record->data + offset, 4);
-
- /* Read the 4 bytes at the offset 24 of data as an integer */
-}
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_find_event,tep_find_event_by_name,tep_find_event_by_record -
-Find events by given key.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-struct tep_event pass:[*]*tep_find_event*(struct tep_handle pass:[*]_tep_, int _id_);
-struct tep_event pass:[*]*tep_find_event_by_name*(struct tep_handle pass:[*]_tep_, const char pass:[*]_sys_, const char pass:[*]_name_);
-struct tep_event pass:[*]*tep_find_event_by_record*(struct tep_handle pass:[*]_tep_, struct tep_record pass:[*]_record_);
---
-
-DESCRIPTION
------------
-This set of functions can be used to search for an event, based on a given
-criteria. All functions require a pointer to a _tep_, trace event parser
-context.
-
-The _tep_find_event()_ function searches for an event by given event _id_. The
-event ID is assigned dynamically and can be viewed in event's format file,
-"ID" field.
-
-The tep_find_event_by_name()_ function searches for an event by given
-event _name_, under the system _sys_. If the _sys_ is NULL (not specified),
-the first event with _name_ is returned.
-
-The tep_find_event_by_record()_ function searches for an event from a given
-_record_.
-
-RETURN VALUE
-------------
-All these functions return a pointer to the found event, or NULL if there is no
-such event.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-struct tep_event *event;
-
-event = tep_find_event(tep, 1857);
-if (event == NULL) {
- /* There is no event with ID 1857 */
-}
-
-event = tep_find_event_by_name(tep, "kvm", "kvm_exit");
-if (event == NULL) {
- /* There is no kvm_exit event, from kvm system */
-}
-
-void event_from_record(struct tep_record *record)
-{
- struct tep_event *event = tep_find_event_by_record(tep, record);
- if (event == NULL) {
- /* There is no event from given record */
- }
-}
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_get_event, tep_get_first_event, tep_get_events_count - Access events.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-struct tep_event pass:[*]*tep_get_event*(struct tep_handle pass:[*]_tep_, int _index_);
-struct tep_event pass:[*]*tep_get_first_event*(struct tep_handle pass:[*]_tep_);
-int *tep_get_events_count*(struct tep_handle pass:[*]_tep_);
---
-
-DESCRIPTION
------------
-The _tep_get_event()_ function returns a pointer to event at the given _index_.
-The _tep_ argument is trace event parser context, the _index_ is the index of
-the requested event.
-
-The _tep_get_first_event()_ function returns a pointer to the first event.
-As events are stored in an array, this function returns the pointer to the
-beginning of the array. The _tep_ argument is trace event parser context.
-
-The _tep_get_events_count()_ function returns the number of the events
-in the array. The _tep_ argument is trace event parser context.
-
-RETURN VALUE
-------------
-The _tep_get_event()_ returns a pointer to the event located at _index_.
-NULL is returned in case of error, in case there are no events or _index_ is
-out of range.
-
-The _tep_get_first_event()_ returns a pointer to the first event. NULL is
-returned in case of error, or in case there are no events.
-
-The _tep_get_events_count()_ returns the number of the events. 0 is
-returned in case of error, or in case there are no events.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-int i,count = tep_get_events_count(tep);
-struct tep_event *event, *events = tep_get_first_event(tep);
-
-if (events == NULL) {
- /* There are no events */
-} else {
- for (i = 0; i < count; i++) {
- event = (events+i);
- /* process events[i] */
- }
-
- /* Get the last event */
- event = tep_get_event(tep, count-1);
-}
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_list_events, tep_list_events_copy -
-Get list of events, sorted by given criteria.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-enum *tep_event_sort_type* {
- _TEP_EVENT_SORT_ID_,
- _TEP_EVENT_SORT_NAME_,
- _TEP_EVENT_SORT_SYSTEM_,
-};
-
-struct tep_event pass:[*]pass:[*]*tep_list_events*(struct tep_handle pass:[*]_tep_, enum tep_event_sort_type _sort_type_);
-struct tep_event pass:[*]pass:[*]*tep_list_events_copy*(struct tep_handle pass:[*]_tep_, enum tep_event_sort_type _sort_type_);
---
-
-DESCRIPTION
------------
-The _tep_list_events()_ function returns an array of pointers to the events,
-sorted by the _sort_type_ criteria. The last element of the array is NULL.
-The returned memory must not be freed, it is managed by the library.
-The function is not thread safe. The _tep_ argument is trace event parser
-context. The _sort_type_ argument is the required sort criteria:
-[verse]
---
- _TEP_EVENT_SORT_ID_ - sort by the event ID.
- _TEP_EVENT_SORT_NAME_ - sort by the event (name, system, id) triplet.
- _TEP_EVENT_SORT_SYSTEM_ - sort by the event (system, name, id) triplet.
---
-
-The _tep_list_events_copy()_ is a thread safe version of _tep_list_events()_.
-It has the same behavior, but the returned array is allocated internally and
-must be freed by the caller. Note that the content of the array must not be
-freed (see the EXAMPLE below).
-
-RETURN VALUE
-------------
-The _tep_list_events()_ function returns an array of pointers to events.
-In case of an error, NULL is returned. The returned array must not be freed,
-it is managed by the library.
-
-The _tep_list_events_copy()_ function returns an array of pointers to events.
-In case of an error, NULL is returned. The returned array must be freed by
-the caller.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-int i;
-struct tep_event_format **events;
-
-i=0;
-events = tep_list_events(tep, TEP_EVENT_SORT_ID);
-if (events == NULL) {
- /* Failed to get the events, sorted by ID */
-} else {
- while(events[i]) {
- /* walk through the list of the events, sorted by ID */
- i++;
- }
-}
-
-i=0;
-events = tep_list_events_copy(tep, TEP_EVENT_SORT_NAME);
-if (events == NULL) {
- /* Failed to get the events, sorted by name */
-} else {
- while(events[i]) {
- /* walk through the list of the events, sorted by name */
- i++;
- }
- free(events);
-}
-
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_print_event - Writes event information into a trace sequence.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-*#include <trace-seq.h>*
-
-void *tep_print_event*(struct tep_handle pass:[*]_tep_, struct trace_seqpass:[*]_s_, struct tep_record pass:[*]_record_, const char pass:[*]_fmt_, _..._)
---
-
-DESCRIPTION
------------
-
-The _tep_print_event()_ function parses the event information of the given
-_record_ and writes it into the trace sequence _s_, according to the format
-string _fmt_. The desired information is specified after the format string.
-The _fmt_ is printf-like format string, following arguments are supported:
-[verse]
---
- TEP_PRINT_PID, "%d" - PID of the event.
- TEP_PRINT_CPU, "%d" - Event CPU.
- TEP_PRINT_COMM, "%s" - Event command string.
- TEP_PRINT_NAME, "%s" - Event name.
- TEP_PRINT_LATENCY, "%s" - Latency of the event. It prints 4 or more
- fields - interrupt state, scheduling state,
- current context, and preemption count.
- Field 1 is the interrupt enabled state:
- d : Interrupts are disabled
- . : Interrupts are enabled
- X : The architecture does not support this
- information
- Field 2 is the "need resched" state.
- N : The task is set to call the scheduler when
- possible, as another higher priority task
- may need to be scheduled in.
- . : The task is not set to call the scheduler.
- Field 3 is the context state.
- . : Normal context
- s : Soft interrupt context
- h : Hard interrupt context
- H : Hard interrupt context which triggered
- during soft interrupt context.
- z : NMI context
- Z : NMI context which triggered during hard
- interrupt context
- Field 4 is the preemption count.
- . : The preempt count is zero.
- On preemptible kernels (where the task can be scheduled
- out in arbitrary locations while in kernel context), the
- preempt count, when non zero, will prevent the kernel
- from scheduling out the current task. The preempt count
- number is displayed when it is not zero.
- Depending on the kernel, it may show other fields
- (lock depth, or migration disabled, which are unique to
- specialized kernels).
- TEP_PRINT_TIME, %d - event time stamp. A divisor and precision can be
- specified as part of this format string:
- "%precision.divisord". Example:
- "%3.1000d" - divide the time by 1000 and print the first
- 3 digits before the dot. Thus, the time stamp
- "123456000" will be printed as "123.456"
- TEP_PRINT_INFO, "%s" - event information.
- TEP_PRINT_INFO_RAW, "%s" - event information, in raw format.
-
---
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-#include <trace-seq.h>
-...
-struct trace_seq seq;
-trace_seq_init(&seq);
-struct tep_handle *tep = tep_alloc();
-...
-void print_my_event(struct tep_record *record)
-{
- trace_seq_reset(&seq);
- tep_print_event(tep, s, record, "%16s-%-5d [%03d] %s %6.1000d %s %s",
- TEP_PRINT_COMM, TEP_PRINT_PID, TEP_PRINT_CPU,
- TEP_PRINT_LATENCY, TEP_PRINT_TIME, TEP_PRINT_NAME,
- TEP_PRINT_INFO);
-}
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*trace-seq.h*
- Header file to include in order to have access to trace sequences related APIs.
- Trace sequences are used to allow a function to call several other functions
- to create a string of data to use.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_find_common_field, tep_find_field, tep_find_any_field -
-Search for a field in an event.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-struct tep_format_field pass:[*]*tep_find_common_field*(struct tep_event pass:[*]_event_, const char pass:[*]_name_);
-struct tep_format_field pass:[*]*tep_find_field*(struct tep_event_ormat pass:[*]_event_, const char pass:[*]_name_);
-struct tep_format_field pass:[*]*tep_find_any_field*(struct tep_event pass:[*]_event_, const char pass:[*]_name_);
---
-
-DESCRIPTION
------------
-These functions search for a field with given name in an event. The field
-returned can be used to find the field content from within a data record.
-
-The _tep_find_common_field()_ function searches for a common field with _name_
-in the _event_.
-
-The _tep_find_field()_ function searches for an event specific field with
-_name_ in the _event_.
-
-The _tep_find_any_field()_ function searches for any field with _name_ in the
-_event_.
-
-RETURN VALUE
-------------
-The _tep_find_common_field(), _tep_find_field()_ and _tep_find_any_field()_
-functions return a pointer to the found field, or NULL in case there is no field
-with the requested name.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-void get_htimer_info(struct tep_handle *tep, struct tep_record *record)
-{
- struct tep_format_field *field;
- struct tep_event *event;
- long long softexpires;
- int mode;
- int pid;
-
- event = tep_find_event_by_name(tep, "timer", "hrtimer_start");
-
- field = tep_find_common_field(event, "common_pid");
- if (field == NULL) {
- /* Cannot find "common_pid" field in the event */
- } else {
- /* Get pid from the data record */
- pid = tep_read_number(tep, record->data + field->offset,
- field->size);
- }
-
- field = tep_find_field(event, "softexpires");
- if (field == NULL) {
- /* Cannot find "softexpires" event specific field in the event */
- } else {
- /* Get softexpires parameter from the data record */
- softexpires = tep_read_number(tep, record->data + field->offset,
- field->size);
- }
-
- field = tep_find_any_field(event, "mode");
- if (field == NULL) {
- /* Cannot find "mode" field in the event */
- } else
- {
- /* Get mode parameter from the data record */
- mode = tep_read_number(tep, record->data + field->offset,
- field->size);
- }
-}
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_get_any_field_val, tep_get_common_field_val, tep_get_field_val,
-tep_get_field_raw - Get value of a field.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-*#include <trace-seq.h>*
-
-int *tep_get_any_field_val*(struct trace_seq pass:[*]_s_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, unsigned long long pass:[*]_val_, int _err_);
-int *tep_get_common_field_val*(struct trace_seq pass:[*]_s_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, unsigned long long pass:[*]_val_, int _err_);
-int *tep_get_field_val*(struct trace_seq pass:[*]_s_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, unsigned long long pass:[*]_val_, int _err_);
-void pass:[*]*tep_get_field_raw*(struct trace_seq pass:[*]_s_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, int pass:[*]_len_, int _err_);
---
-
-DESCRIPTION
------------
-These functions can be used to find a field and retrieve its value.
-
-The _tep_get_any_field_val()_ function searches in the _record_ for a field
-with _name_, part of the _event_. If the field is found, its value is stored in
-_val_. If there is an error and _err_ is not zero, then an error string is
-written into _s_.
-
-The _tep_get_common_field_val()_ function does the same as
-_tep_get_any_field_val()_, but searches only in the common fields. This works
-for any event as all events include the common fields.
-
-The _tep_get_field_val()_ function does the same as _tep_get_any_field_val()_,
-but searches only in the event specific fields.
-
-The _tep_get_field_raw()_ function searches in the _record_ for a field with
-_name_, part of the _event_. If the field is found, a pointer to where the field
-exists in the record's raw data is returned. The size of the data is stored in
-_len_. If there is an error and _err_ is not zero, then an error string is
-written into _s_.
-
-RETURN VALUE
-------------
-The _tep_get_any_field_val()_, _tep_get_common_field_val()_ and
-_tep_get_field_val()_ functions return 0 on success, or -1 in case of an error.
-
-The _tep_get_field_raw()_ function returns a pointer to field's raw data, and
-places the length of this data in _len_. In case of an error NULL is returned.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-#include <trace-seq.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-struct tep_event *event = tep_find_event_by_name(tep, "kvm", "kvm_exit");
-...
-void process_record(struct tep_record *record)
-{
- int len;
- char *comm;
- struct tep_event_format *event;
- unsigned long long val;
-
- event = tep_find_event_by_record(pevent, record);
- if (event != NULL) {
- if (tep_get_common_field_val(NULL, event, "common_type",
- record, &val, 0) == 0) {
- /* Got the value of common type field */
- }
- if (tep_get_field_val(NULL, event, "pid", record, &val, 0) == 0) {
- /* Got the value of pid specific field */
- }
- comm = tep_get_field_raw(NULL, event, "comm", record, &len, 0);
- if (comm != NULL) {
- /* Got a pointer to the comm event specific field */
- }
- }
-}
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*trace-seq.h*
- Header file to include in order to have access to trace sequences
- related APIs. Trace sequences are used to allow a function to call
- several other functions to create a string of data to use.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_print_field, tep_print_fields, tep_print_num_field, tep_print_func_field -
-Print the field content.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-*#include <trace-seq.h>*
-
-void *tep_print_field*(struct trace_seq pass:[*]_s_, void pass:[*]_data_, struct tep_format_field pass:[*]_field_);
-void *tep_print_fields*(struct trace_seq pass:[*]_s_, void pass:[*]_data_, int _size_, struct tep_event pass:[*]_event_);
-int *tep_print_num_field*(struct trace_seq pass:[*]_s_, const char pass:[*]_fmt_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, int _err_);
-int *tep_print_func_field*(struct trace_seq pass:[*]_s_, const char pass:[*]_fmt_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, int _err_);
---
-
-DESCRIPTION
------------
-These functions print recorded field's data, according to the field's type.
-
-The _tep_print_field()_ function extracts from the recorded raw _data_ value of
-the _field_ and prints it into _s_, according to the field type.
-
-The _tep_print_fields()_ prints each field name followed by the record's field
-value according to the field's type:
-[verse]
---
-"field1_name=field1_value field2_name=field2_value ..."
---
-It iterates all fields of the _event_, and calls _tep_print_field()_ for each of
-them.
-
-The _tep_print_num_field()_ function prints a numeric field with given format
-string. A search is performed in the _event_ for a field with _name_. If such
-field is found, its value is extracted from the _record_ and is printed in the
-_s_, according to the given format string _fmt_. If the argument _err_ is
-non-zero, and an error occures - it is printed in the _s_.
-
-The _tep_print_func_field()_ function prints a function field with given format
-string. A search is performed in the _event_ for a field with _name_. If such
-field is found, its value is extracted from the _record_. The value is assumed
-to be a function address, and a search is perform to find the name of this
-function. The function name (if found) and its address are printed in the _s_,
-according to the given format string _fmt_. If the argument _err_ is non-zero,
-and an error occures - it is printed in _s_.
-
-RETURN VALUE
-------------
-The _tep_print_num_field()_ and _tep_print_func_field()_ functions return 1
-on success, -1 in case of an error or 0 if the print buffer _s_ is full.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-#include <trace-seq.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-struct trace_seq seq;
-trace_seq_init(&seq);
-struct tep_event *event = tep_find_event_by_name(tep, "timer", "hrtimer_start");
-...
-void process_record(struct tep_record *record)
-{
- struct tep_format_field *field_pid = tep_find_common_field(event, "common_pid");
-
- trace_seq_reset(&seq);
-
- /* Print the value of "common_pid" */
- tep_print_field(&seq, record->data, field_pid);
-
- /* Print all fields of the "hrtimer_start" event */
- tep_print_fields(&seq, record->data, record->size, event);
-
- /* Print the value of "expires" field with custom format string */
- tep_print_num_field(&seq, " timer expires in %llu ", event, "expires", record, 0);
-
- /* Print the address and the name of "function" field with custom format string */
- tep_print_func_field(&seq, " timer function is %s ", event, "function", record, 0);
- }
- ...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*trace-seq.h*
- Header file to include in order to have access to trace sequences related APIs.
- Trace sequences are used to allow a function to call several other functions
- to create a string of data to use.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_read_number_field - Reads a number from raw data.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-int *tep_read_number_field*(struct tep_format_field pass:[*]_field_, const void pass:[*]_data_, unsigned long long pass:[*]_value_);
---
-
-DESCRIPTION
------------
-The _tep_read_number_field()_ function reads the value of the _field_ from the
-raw _data_ and stores it in the _value_. The function sets the _value_ according
-to the endianness of the raw data and the current machine and stores it in
-_value_.
-
-RETURN VALUE
-------------
-The _tep_read_number_field()_ function retunrs 0 in case of success, or -1 in
-case of an error.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-struct tep_event *event = tep_find_event_by_name(tep, "timer", "hrtimer_start");
-...
-void process_record(struct tep_record *record)
-{
- unsigned long long pid;
- struct tep_format_field *field_pid = tep_find_common_field(event, "common_pid");
-
- if (tep_read_number_field(field_pid, record->data, &pid) != 0) {
- /* Failed to get "common_pid" value */
- }
-}
-...
---
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_event_common_fields, tep_event_fields - Get a list of fields for an event.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-struct tep_format_field pass:[*]pass:[*]*tep_event_common_fields*(struct tep_event pass:[*]_event_);
-struct tep_format_field pass:[*]pass:[*]*tep_event_fields*(struct tep_event pass:[*]_event_);
---
-
-DESCRIPTION
------------
-The _tep_event_common_fields()_ function returns an array of pointers to common
-fields for the _event_. The array is allocated in the function and must be freed
-by free(). The last element of the array is NULL.
-
-The _tep_event_fields()_ function returns an array of pointers to event specific
-fields for the _event_. The array is allocated in the function and must be freed
-by free(). The last element of the array is NULL.
-
-RETURN VALUE
-------------
-Both _tep_event_common_fields()_ and _tep_event_fields()_ functions return
-an array of pointers to tep_format_field structures in case of success, or
-NULL in case of an error.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-int i;
-struct tep_format_field **fields;
-struct tep_event *event = tep_find_event_by_name(tep, "kvm", "kvm_exit");
-if (event != NULL) {
- fields = tep_event_common_fields(event);
- if (fields != NULL) {
- i = 0;
- while (fields[i]) {
- /*
- walk through the list of the common fields
- of the kvm_exit event
- */
- i++;
- }
- free(fields);
- }
- fields = tep_event_fields(event);
- if (fields != NULL) {
- i = 0;
- while (fields[i]) {
- /*
- walk through the list of the event specific
- fields of the kvm_exit event
- */
- i++;
- }
- free(fields);
- }
-}
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_is_file_bigendian, tep_set_file_bigendian - Get / set the endianness of the
-raw data being accessed by the tep handler.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-enum *tep_endian* {
- TEP_LITTLE_ENDIAN = 0,
- TEP_BIG_ENDIAN
-};
-
-bool *tep_is_file_bigendian*(struct tep_handle pass:[*]_tep_);
-void *tep_set_file_bigendian*(struct tep_handle pass:[*]_tep_, enum tep_endian _endian_);
-
---
-DESCRIPTION
------------
-The _tep_is_file_bigendian()_ function gets the endianness of the raw data,
-being accessed by the tep handler. The _tep_ argument is trace event parser
-context.
-
-The _tep_set_file_bigendian()_ function sets the endianness of raw data being
-accessed by the tep handler. The _tep_ argument is trace event parser context.
-[verse]
---
-The _endian_ argument is the endianness:
- _TEP_LITTLE_ENDIAN_ - the raw data is in little endian format,
- _TEP_BIG_ENDIAN_ - the raw data is in big endian format.
---
-RETURN VALUE
-------------
-The _tep_is_file_bigendian()_ function returns true if the data is in bigendian
-format, false otherwise.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
- tep_set_file_bigendian(tep, TEP_LITTLE_ENDIAN);
-...
- if (tep_is_file_bigendian(tep)) {
- /* The raw data is in big endian */
- } else {
- /* The raw data is in little endian */
- }
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_filter_alloc, tep_filter_free, tep_filter_reset, tep_filter_make_string,
-tep_filter_copy, tep_filter_compare, tep_filter_match, tep_event_filtered,
-tep_filter_remove_event, tep_filter_strerror, tep_filter_add_filter_str -
-Event filter related APIs.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-struct tep_event_filter pass:[*]*tep_filter_alloc*(struct tep_handle pass:[*]_tep_);
-void *tep_filter_free*(struct tep_event_filter pass:[*]_filter_);
-void *tep_filter_reset*(struct tep_event_filter pass:[*]_filter_);
-enum tep_errno *tep_filter_add_filter_str*(struct tep_event_filter pass:[*]_filter_, const char pass:[*]_filter_str_);
-int *tep_event_filtered*(struct tep_event_filter pass:[*]_filter_, int _event_id_);
-int *tep_filter_remove_event*(struct tep_event_filter pass:[*]_filter_, int _event_id_);
-enum tep_errno *tep_filter_match*(struct tep_event_filter pass:[*]_filter_, struct tep_record pass:[*]_record_);
-int *tep_filter_copy*(struct tep_event_filter pass:[*]_dest_, struct tep_event_filter pass:[*]_source_);
-int *tep_filter_compare*(struct tep_event_filter pass:[*]_filter1_, struct tep_event_filter pass:[*]_filter2_);
-char pass:[*]*tep_filter_make_string*(struct tep_event_filter pass:[*]_filter_, int _event_id_);
-int *tep_filter_strerror*(struct tep_event_filter pass:[*]_filter_, enum tep_errno _err_, char pass:[*]buf, size_t _buflen_);
---
-
-DESCRIPTION
------------
-Filters can be attached to traced events. They can be used to filter out various
-events when outputting them. Each event can be filtered based on its parameters,
-described in the event's format file. This set of functions can be used to
-create, delete, modify and attach event filters.
-
-The _tep_filter_alloc()_ function creates a new event filter. The _tep_ argument
-is the trace event parser context.
-
-The _tep_filter_free()_ function frees an event filter and all resources that it
-had used.
-
-The _tep_filter_reset()_ function removes all rules from an event filter and
-resets it.
-
-The _tep_filter_add_filter_str()_ function adds a new rule to the _filter_. The
-_filter_str_ argument is the filter string, that contains the rule.
-
-The _tep_event_filtered()_ function checks if the event with _event_id_ has
-_filter_.
-
-The _tep_filter_remove_event()_ function removes a _filter_ for an event with
-_event_id_.
-
-The _tep_filter_match()_ function tests if a _record_ matches given _filter_.
-
-The _tep_filter_copy()_ function copies a _source_ filter into a _dest_ filter.
-
-The _tep_filter_compare()_ function compares two filers - _filter1_ and _filter2_.
-
-The _tep_filter_make_string()_ function constructs a string, displaying
-the _filter_ contents for given _event_id_.
-
-The _tep_filter_strerror()_ function copies the _filter_ error buffer into the
-given _buf_ with the size _buflen_. If the error buffer is empty, in the _buf_
-is copied a string, describing the error _err_.
-
-RETURN VALUE
-------------
-The _tep_filter_alloc()_ function returns a pointer to the newly created event
-filter, or NULL in case of an error.
-
-The _tep_filter_add_filter_str()_ function returns 0 if the rule was
-successfully added or a negative error code. Use _tep_filter_strerror()_ to see
-actual error message in case of an error.
-
-The _tep_event_filtered()_ function returns 1 if the filter is found for given
-event, or 0 otherwise.
-
-The _tep_filter_remove_event()_ function returns 1 if the vent was removed, or
-0 if the event was not found.
-
-The _tep_filter_match()_ function returns _tep_errno_, according to the result:
-[verse]
---
-_pass:[TEP_ERRNO__FILTER_MATCH]_ - filter found for event, the record matches.
-_pass:[TEP_ERRNO__FILTER_MISS]_ - filter found for event, the record does not match.
-_pass:[TEP_ERRNO__FILTER_NOT_FOUND]_ - no filter found for record's event.
-_pass:[TEP_ERRNO__NO_FILTER]_ - no rules in the filter.
---
-or any other _tep_errno_, if an error occurred during the test.
-
-The _tep_filter_copy()_ function returns 0 on success or -1 if not all rules
- were copied.
-
-The _tep_filter_compare()_ function returns 1 if the two filters hold the same
-content, or 0 if they do not.
-
-The _tep_filter_make_string()_ function returns a string, which must be freed
-with free(), or NULL in case of an error.
-
-The _tep_filter_strerror()_ function returns 0 if message was filled
-successfully, or -1 in case of an error.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-char errstr[200];
-int ret;
-
-struct tep_event_filter *filter = tep_filter_alloc(tep);
-struct tep_event_filter *filter1 = tep_filter_alloc(tep);
-ret = tep_filter_add_filter_str(filter, "sched/sched_wakeup:target_cpu==1");
-if(ret < 0) {
- tep_filter_strerror(filter, ret, errstr, sizeof(errstr));
- /* Failed to add a new rule to the filter, the error string is in errstr */
-}
-if (tep_filter_copy(filter1, filter) != 0) {
- /* Failed to copy filter in filter1 */
-}
-...
-if (tep_filter_compare(filter, filter1) != 1) {
- /* Both filters are different */
-}
-...
-void process_record(struct tep_handle *tep, struct tep_record *record)
-{
- struct tep_event *event;
- char *fstring;
-
- event = tep_find_event_by_record(tep, record);
-
- if (tep_event_filtered(filter, event->id) == 1) {
- /* The event has filter */
- fstring = tep_filter_make_string(filter, event->id);
- if (fstring != NULL) {
- /* The filter for the event is in fstring */
- free(fstring);
- }
- }
-
- switch (tep_filter_match(filter, record)) {
- case TEP_ERRNO__FILTER_MATCH:
- /* The filter matches the record */
- break;
- case TEP_ERRNO__FILTER_MISS:
- /* The filter does not match the record */
- break;
- case TEP_ERRNO__FILTER_NOT_FOUND:
- /* No filter found for record's event */
- break;
- case TEP_ERRNO__NO_FILTER:
- /* There are no rules in the filter */
- break
- default:
- /* An error occurred during the test */
- break;
- }
-
- if (tep_filter_remove_event(filter, event->id) == 1) {
- /* The event was removed from the filter */
- }
-}
-
-...
-tep_filter_reset(filter);
-...
-tep_filter_free(filter);
-tep_filter_free(filter1);
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_find_function, tep_find_function_address, tep_set_function_resolver,
-tep_reset_function_resolver, tep_register_function, tep_register_print_string -
-function related tep APIs
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-typedef char pass:[*](*tep_func_resolver_t*)(void pass:[*]_priv_, unsigned long long pass:[*]_addrp_, char pass:[**]_modp_);
-int *tep_set_function_resolver*(struct tep_handle pass:[*]_tep_, tep_func_resolver_t pass:[*]_func_, void pass:[*]_priv_);
-void *tep_reset_function_resolver*(struct tep_handle pass:[*]_tep_);
-const char pass:[*]*tep_find_function*(struct tep_handle pass:[*]_tep_, unsigned long long _addr_);
-unsigned long long *tep_find_function_address*(struct tep_handle pass:[*]_tep_, unsigned long long _addr_);
-int *tep_register_function*(struct tep_handle pass:[*]_tep_, char pass:[*]_name_, unsigned long long _addr_, char pass:[*]_mod_);
-int *tep_register_print_string*(struct tep_handle pass:[*]_tep_, const char pass:[*]_fmt_, unsigned long long _addr_);
---
-
-DESCRIPTION
------------
-Some tools may have already a way to resolve the kernel functions. These APIs
-allow them to keep using it instead of duplicating all the entries inside.
-
-The _tep_func_resolver_t_ type is the prototype of the alternative kernel
-functions resolver. This function receives a pointer to its custom context
-(set with the _tep_set_function_resolver()_ call ) and the address of a kernel
-function, which has to be resolved. In case of success, it should return
-the name of the function and its module (if any) in _modp_.
-
-The _tep_set_function_resolver()_ function registers _func_ as an alternative
-kernel functions resolver. The _tep_ argument is trace event parser context.
-The _priv_ argument is a custom context of the _func_ function. The function
-resolver is used by the APIs _tep_find_function()_,
-_tep_find_function_address()_, and _tep_print_func_field()_ to resolve
-a function address to a function name.
-
-The _tep_reset_function_resolver()_ function resets the kernel functions
-resolver to the default function. The _tep_ argument is trace event parser
-context.
-
-
-These APIs can be used to find function name and start address, by given
-address. The given address does not have to be exact, it will select
-the function that would contain it.
-
-The _tep_find_function()_ function returns the function name, which contains the
-given address _addr_. The _tep_ argument is the trace event parser context.
-
-The _tep_find_function_address()_ function returns the function start address,
-by given address _addr_. The _addr_ does not have to be exact, it will select
-the function that would contain it. The _tep_ argument is the trace event
-parser context.
-
-The _tep_register_function()_ function registers a function name mapped to an
-address and (optional) module. This mapping is used in case the function tracer
-or events have "%pS" parameter in its format string. It is common to pass in
-the kallsyms function names with their corresponding addresses with this
-function. The _tep_ argument is the trace event parser context. The _name_ is
-the name of the function, the string is copied internally. The _addr_ is the
-start address of the function. The _mod_ is the kernel module the function may
-be in (NULL for none).
-
-The _tep_register_print_string()_ function registers a string by the address
-it was stored in the kernel. Some strings internal to the kernel with static
-address are passed to certain events. The "%s" in the event's format field
-which has an address needs to know what string would be at that address. The
-tep_register_print_string() supplies the parsing with the mapping between kernel
-addresses and those strings. The _tep_ argument is the trace event parser
-context. The _fmt_ is the string to register, it is copied internally.
-The _addr_ is the address the string was located at.
-
-
-RETURN VALUE
-------------
-The _tep_set_function_resolver()_ function returns 0 in case of success, or -1
-in case of an error.
-
-The _tep_find_function()_ function returns the function name, or NULL in case
-it cannot be found.
-
-The _tep_find_function_address()_ function returns the function start address,
-or 0 in case it cannot be found.
-
-The _tep_register_function()_ function returns 0 in case of success. In case of
-an error -1 is returned, and errno is set to the appropriate error number.
-
-The _tep_register_print_string()_ function returns 0 in case of success. In case
-of an error -1 is returned, and errno is set to the appropriate error number.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-char *my_resolve_kernel_addr(void *context,
- unsigned long long *addrp, char **modp)
-{
- struct db *function_database = context;
- struct symbol *sym = sql_lookup(function_database, *addrp);
-
- if (!sym)
- return NULL;
-
- *modp = sym->module_name;
- return sym->name;
-}
-
-void show_function( unsigned long long addr)
-{
- unsigned long long fstart;
- const char *fname;
-
- if (tep_set_function_resolver(tep, my_resolve_kernel_addr,
- function_database) != 0) {
- /* failed to register my_resolve_kernel_addr */
- }
-
- /* These APIs use my_resolve_kernel_addr() to resolve the addr */
- fname = tep_find_function(tep, addr);
- fstart = tep_find_function_address(tep, addr);
-
- /*
- addr is in function named fname, starting at fstart address,
- at offset (addr - fstart)
- */
-
- tep_reset_function_resolver(tep);
-
-}
-...
- if (tep_register_function(tep, "kvm_exit",
- (unsigned long long) 0x12345678, "kvm") != 0) {
- /* Failed to register kvm_exit address mapping */
- }
-...
- if (tep_register_print_string(tep, "print string",
- (unsigned long long) 0x87654321, NULL) != 0) {
- /* Failed to register "print string" address mapping */
- }
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_find_function,tep_find_function_address - Find function name / start address.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-const char pass:[*]*tep_find_function*(struct tep_handle pass:[*]_tep_, unsigned long long _addr_);
-unsigned long long *tep_find_function_address*(struct tep_handle pass:[*]_tep_, unsigned long long _addr_);
---
-
-DESCRIPTION
------------
-These functions can be used to find function name and start address, by given
-address. The given address does not have to be exact, it will select the function
-that would contain it.
-
-The _tep_find_function()_ function returns the function name, which contains the
-given address _addr_. The _tep_ argument is the trace event parser context.
-
-The _tep_find_function_address()_ function returns the function start address,
-by given address _addr_. The _addr_ does not have to be exact, it will select the
-function that would contain it. The _tep_ argument is the trace event parser context.
-
-RETURN VALUE
-------------
-The _tep_find_function()_ function returns the function name, or NULL in case
-it cannot be found.
-
-The _tep_find_function_address()_ function returns the function start address,
-or 0 in case it cannot be found.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-void show_function( unsigned long long addr)
-{
- const char *fname = tep_find_function(tep, addr);
- unsigned long long fstart = tep_find_function_address(tep, addr);
-
- /* addr is in function named fname, starting at fstart address, at offset (addr - fstart) */
-}
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_alloc, tep_free,tep_ref, tep_unref,tep_get_ref - Create, destroy, manage
-references of trace event parser context.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-struct tep_handle pass:[*]*tep_alloc*(void);
-void *tep_free*(struct tep_handle pass:[*]_tep_);
-void *tep_ref*(struct tep_handle pass:[*]_tep_);
-void *tep_unref*(struct tep_handle pass:[*]_tep_);
-int *tep_get_ref*(struct tep_handle pass:[*]_tep_);
---
-
-DESCRIPTION
------------
-These are the main functions to create and destroy tep_handle - the main
-structure, representing the trace event parser context. This context is used as
-the input parameter of most library APIs.
-
-The _tep_alloc()_ function allocates and initializes the tep context.
-
-The _tep_free()_ function will decrement the reference of the _tep_ handler.
-When there is no more references, then it will free the handler, as well
-as clean up all its resources that it had used. The argument _tep_ is
-the pointer to the trace event parser context.
-
-The _tep_ref()_ function adds a reference to the _tep_ handler.
-
-The _tep_unref()_ function removes a reference from the _tep_ handler. When
-the last reference is removed, the _tep_ is destroyed, and all resources that
-it had used are cleaned up.
-
-The _tep_ref_get()_ functions gets the current references of the _tep_ handler.
-
-RETURN VALUE
-------------
-_tep_alloc()_ returns a pointer to a newly created tep_handle structure.
-NULL is returned in case there is not enough free memory to allocate it.
-
-_tep_ref_get()_ returns the current references of _tep_.
-If _tep_ is NULL, 0 is returned.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-
-...
-struct tep_handle *tep = tep_alloc();
-...
-int ref = tep_get_ref(tep);
-tep_ref(tep);
-if ( (ref+1) != tep_get_ref(tep)) {
- /* Something wrong happened, the counter is not incremented by 1 */
-}
-tep_unref(tep);
-...
-tep_free(tep);
-...
---
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_get_header_page_size, tep_get_header_timestamp_size, tep_is_old_format -
-Get the data stored in the header page, in kernel context.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-int *tep_get_header_page_size*(struct tep_handle pass:[*]_tep_);
-int *tep_get_header_timestamp_size*(struct tep_handle pass:[*]_tep_);
-bool *tep_is_old_format*(struct tep_handle pass:[*]_tep_);
---
-DESCRIPTION
------------
-These functions retrieve information from kernel context, stored in tracefs
-events/header_page. Old kernels do not have header page info, so default values
-from user space context are used.
-
-The _tep_get_header_page_size()_ function returns the size of a long integer,
-in kernel context. The _tep_ argument is trace event parser context.
-This information is retrieved from tracefs events/header_page, "commit" field.
-
-The _tep_get_header_timestamp_size()_ function returns the size of timestamps,
-in kernel context. The _tep_ argument is trace event parser context. This
-information is retrieved from tracefs events/header_page, "timestamp" field.
-
-The _tep_is_old_format()_ function returns true if the kernel predates
-the addition of events/header_page, otherwise it returns false.
-
-RETURN VALUE
-------------
-The _tep_get_header_page_size()_ function returns the size of a long integer,
-in bytes.
-
-The _tep_get_header_timestamp_size()_ function returns the size of timestamps,
-in bytes.
-
-The _tep_is_old_format()_ function returns true, if an old kernel is used to
-generate the tracing data, which has no event/header_page. If the kernel is new,
-or _tep_ is NULL, false is returned.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
- int longsize;
- int timesize;
- bool old;
-
- longsize = tep_get_header_page_size(tep);
- timesize = tep_get_header_timestamp_size(tep);
- old = tep_is_old_format(tep);
-
- printf ("%s kernel is used to generate the tracing data.\n",
- old?"Old":"New");
- printf("The size of a long integer is %d bytes.\n", longsize);
- printf("The timestamps size is %d bytes.\n", timesize);
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_is_bigendian, tep_is_local_bigendian, tep_set_local_bigendian - Get / set
-the endianness of the local machine.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-enum *tep_endian* {
- TEP_LITTLE_ENDIAN = 0,
- TEP_BIG_ENDIAN
-};
-
-int *tep_is_bigendian*(void);
-bool *tep_is_local_bigendian*(struct tep_handle pass:[*]_tep_);
-void *tep_set_local_bigendian*(struct tep_handle pass:[*]_tep_, enum tep_endian _endian_);
---
-
-DESCRIPTION
------------
-
-The _tep_is_bigendian()_ gets the endianness of the machine, executing
-the function.
-
-The _tep_is_local_bigendian()_ function gets the endianness of the local
-machine, saved in the _tep_ handler. The _tep_ argument is the trace event
-parser context. This API is a bit faster than _tep_is_bigendian()_, as it
-returns cached endianness of the local machine instead of checking it each time.
-
-The _tep_set_local_bigendian()_ function sets the endianness of the local
-machine in the _tep_ handler. The _tep_ argument is trace event parser context.
-The _endian_ argument is the endianness:
-[verse]
---
- _TEP_LITTLE_ENDIAN_ - the machine is little endian,
- _TEP_BIG_ENDIAN_ - the machine is big endian.
---
-
-RETURN VALUE
-------------
-The _tep_is_bigendian()_ function returns non zero if the endianness of the
-machine, executing the code, is big endian and zero otherwise.
-
-The _tep_is_local_bigendian()_ function returns true, if the endianness of the
-local machine, saved in the _tep_ handler, is big endian, or false otherwise.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
- if (tep_is_bigendian())
- tep_set_local_bigendian(tep, TEP_BIG_ENDIAN);
- else
- tep_set_local_bigendian(tep, TEP_LITTLE_ENDIAN);
-...
- if (tep_is_local_bigendian(tep))
- printf("This machine you are running on is bigendian\n");
- else
- printf("This machine you are running on is little endian\n");
-
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_get_long_size, tep_set_long_size - Get / set the size of a long integer on
-the machine, where the trace is generated, in bytes
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-int *tep_get_long_size*(strucqt tep_handle pass:[*]_tep_);
-void *tep_set_long_size*(struct tep_handle pass:[*]_tep_, int _long_size_);
---
-
-DESCRIPTION
------------
-The _tep_get_long_size()_ function returns the size of a long integer on the machine,
-where the trace is generated. The _tep_ argument is trace event parser context.
-
-The _tep_set_long_size()_ function sets the size of a long integer on the machine,
-where the trace is generated. The _tep_ argument is trace event parser context.
-The _long_size_ is the size of a long integer, in bytes.
-
-RETURN VALUE
-------------
-The _tep_get_long_size()_ function returns the size of a long integer on the machine,
-where the trace is generated, in bytes.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-tep_set_long_size(tep, 4);
-...
-int long_size = tep_get_long_size(tep);
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_get_page_size, tep_set_page_size - Get / set the size of a memory page on
-the machine, where the trace is generated
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-int *tep_get_page_size*(struct tep_handle pass:[*]_tep_);
-void *tep_set_page_size*(struct tep_handle pass:[*]_tep_, int _page_size_);
---
-
-DESCRIPTION
------------
-The _tep_get_page_size()_ function returns the size of a memory page on
-the machine, where the trace is generated. The _tep_ argument is trace
-event parser context.
-
-The _tep_set_page_size()_ function stores in the _tep_ context the size of a
-memory page on the machine, where the trace is generated.
-The _tep_ argument is trace event parser context.
-The _page_size_ argument is the size of a memory page, in bytes.
-
-RETURN VALUE
-------------
-The _tep_get_page_size()_ function returns size of the memory page, in bytes.
-
-EXAMPLE
--------
-[source,c]
---
-#include <unistd.h>
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
- int page_size = getpagesize();
-
- tep_set_page_size(tep, page_size);
-
- printf("The page size for this machine is %d\n", tep_get_page_size(tep));
-
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_parse_event, tep_parse_format - Parse the event format information
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-enum tep_errno *tep_parse_event*(struct tep_handle pass:[*]_tep_, const char pass:[*]_buf_, unsigned long _size_, const char pass:[*]_sys_);
-enum tep_errno *tep_parse_format*(struct tep_handle pass:[*]_tep_, struct tep_event pass:[*]pass:[*]_eventp_, const char pass:[*]_buf_, unsigned long _size_, const char pass:[*]_sys_);
---
-
-DESCRIPTION
------------
-The _tep_parse_event()_ function parses the event format and creates an event
-structure to quickly parse raw data for a given event. The _tep_ argument is
-the trace event parser context. The created event structure is stored in the
-_tep_ context. The _buf_ argument is a buffer with _size_, where the event
-format data is. The event format data can be taken from
-tracefs/events/.../.../format files. The _sys_ argument is the system of
-the event.
-
-The _tep_parse_format()_ function does the same as _tep_parse_event()_. The only
-difference is in the extra _eventp_ argument, where the newly created event
-structure is returned.
-
-RETURN VALUE
-------------
-Both _tep_parse_event()_ and _tep_parse_format()_ functions return 0 on success,
-or TEP_ERRNO__... in case of an error.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-char *buf;
-int size;
-struct tep_event *event = NULL;
-buf = read_file("/sys/kernel/tracing/events/ftrace/print/format", &size);
-if (tep_parse_event(tep, buf, size, "ftrace") != 0) {
- /* Failed to parse the ftrace print format */
-}
-
-if (tep_parse_format(tep, &event, buf, size, "ftrace") != 0) {
- /* Failed to parse the ftrace print format */
-}
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_parse_header_page - Parses the data stored in the header page.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-int *tep_parse_header_page*(struct tep_handle pass:[*]_tep_, char pass:[*]_buf_, unsigned long _size_, int _long_size_);
---
-
-DESCRIPTION
------------
-The _tep_parse_header_page()_ function parses the header page data from _buf_,
-and initializes the _tep_, trace event parser context, with it. The buffer
-_buf_ is with _size_, and is supposed to be copied from
-tracefs/events/header_page.
-
-Some old kernels do not have header page info, in this case the
-_tep_parse_header_page()_ function can be called with _size_ equal to 0. The
-_tep_ context is initialized with default values. The _long_size_ can be used in
-this use case, to set the size of a long integer to be used.
-
-RETURN VALUE
-------------
-The _tep_parse_header_page()_ function returns 0 in case of success, or -1
-in case of an error.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-char *buf;
-int size;
-buf = read_file("/sys/kernel/tracing/events/header_page", &size);
-if (tep_parse_header_page(tep, buf, size, sizeof(unsigned long)) != 0) {
- /* Failed to parse the header page */
-}
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_load_plugins, tep_unload_plugins, tep_load_plugins_hook - Load / unload traceevent plugins.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-struct tep_plugin_list pass:[*]*tep_load_plugins*(struct tep_handle pass:[*]_tep_);
-void *tep_unload_plugins*(struct tep_plugin_list pass:[*]_plugin_list_, struct tep_handle pass:[*]_tep_);
-void *tep_load_plugins_hook*(struct tep_handle pass:[*]_tep_, const char pass:[*]_suffix_,
- void (pass:[*]_load_plugin_)(struct tep_handle pass:[*]tep,
- const char pass:[*]path,
- const char pass:[*]name,
- void pass:[*]data),
- void pass:[*]_data_);
---
-
-DESCRIPTION
------------
-The _tep_load_plugins()_ function loads all plugins, located in the plugin
-directories. The _tep_ argument is trace event parser context.
-The plugin directories are :
-[verse]
---
- - Directories, specified in _tep_->plugins_dir with priority TEP_PLUGIN_FIRST
- - System's plugin directory, defined at the library compile time. It
- depends on the library installation prefix and usually is
- _(install_preffix)/lib/traceevent/plugins_
- - Directory, defined by the environment variable _TRACEEVENT_PLUGIN_DIR_
- - User's plugin directory, located at _~/.local/lib/traceevent/plugins_
- - Directories, specified in _tep_->plugins_dir with priority TEP_PLUGIN_LAST
---
-Loading of plugins can be controlled by the _tep_flags_, using the
-_tep_set_flag()_ API:
-[verse]
---
- _TEP_DISABLE_SYS_PLUGINS_ - do not load plugins, located in
- the system's plugin directory.
- _TEP_DISABLE_PLUGINS_ - do not load any plugins.
---
-The _tep_set_flag()_ API needs to be called before _tep_load_plugins()_, if
-loading of all plugins is not the desired case.
-
-The _tep_unload_plugins()_ function unloads the plugins, previously loaded by
-_tep_load_plugins()_. The _tep_ argument is trace event parser context. The
-_plugin_list_ is the list of loaded plugins, returned by
-the _tep_load_plugins()_ function.
-
-The _tep_load_plugins_hook_ function walks through all directories with plugins
-and calls user specified _load_plugin()_ hook for each plugin file. Only files
-with given _suffix_ are considered to be plugins. The _data_ is a user specified
-context, passed to _load_plugin()_. Directories and the walk order are the same
-as in _tep_load_plugins()_ API.
-
-RETURN VALUE
-------------
-The _tep_load_plugins()_ function returns a list of successfully loaded plugins,
-or NULL in case no plugins are loaded.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-struct tep_plugin_list *plugins = tep_load_plugins(tep);
-if (plugins == NULL) {
- /* no plugins are loaded */
-}
-...
-tep_unload_plugins(plugins, tep);
-...
-void print_plugin(struct tep_handle *tep, const char *path,
- const char *name, void *data)
-{
- pritnf("Found libtraceevent plugin %s/%s\n", path, name);
-}
-...
-tep_load_plugins_hook(tep, ".so", print_plugin, NULL);
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_, _tep_set_flag(3)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_data_type, tep_data_pid,tep_data_preempt_count, tep_data_flags -
-Extract common fields from a record.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-enum *trace_flag_type* {
- _TRACE_FLAG_IRQS_OFF_,
- _TRACE_FLAG_IRQS_NOSUPPORT_,
- _TRACE_FLAG_NEED_RESCHED_,
- _TRACE_FLAG_HARDIRQ_,
- _TRACE_FLAG_SOFTIRQ_,
-};
-
-int *tep_data_type*(struct tep_handle pass:[*]_tep_, struct tep_record pass:[*]_rec_);
-int *tep_data_pid*(struct tep_handle pass:[*]_tep_, struct tep_record pass:[*]_rec_);
-int *tep_data_preempt_count*(struct tep_handle pass:[*]_tep_, struct tep_record pass:[*]_rec_);
-int *tep_data_flags*(struct tep_handle pass:[*]_tep_, struct tep_record pass:[*]_rec_);
---
-
-DESCRIPTION
------------
-This set of functions can be used to extract common fields from a record.
-
-The _tep_data_type()_ function gets the event id from the record _rec_.
-It reads the "common_type" field. The _tep_ argument is the trace event parser
-context.
-
-The _tep_data_pid()_ function gets the process id from the record _rec_.
-It reads the "common_pid" field. The _tep_ argument is the trace event parser
-context.
-
-The _tep_data_preempt_count()_ function gets the preemption count from the
-record _rec_. It reads the "common_preempt_count" field. The _tep_ argument is
-the trace event parser context.
-
-The _tep_data_flags()_ function gets the latency flags from the record _rec_.
-It reads the "common_flags" field. The _tep_ argument is the trace event parser
-context. Supported latency flags are:
-[verse]
---
- _TRACE_FLAG_IRQS_OFF_, Interrupts are disabled.
- _TRACE_FLAG_IRQS_NOSUPPORT_, Reading IRQ flag is not supported by the architecture.
- _TRACE_FLAG_NEED_RESCHED_, Task needs rescheduling.
- _TRACE_FLAG_HARDIRQ_, Hard IRQ is running.
- _TRACE_FLAG_SOFTIRQ_, Soft IRQ is running.
---
-
-RETURN VALUE
-------------
-The _tep_data_type()_ function returns an integer, representing the event id.
-
-The _tep_data_pid()_ function returns an integer, representing the process id
-
-The _tep_data_preempt_count()_ function returns an integer, representing the
-preemption count.
-
-The _tep_data_flags()_ function returns an integer, representing the latency
-flags. Look at the _trace_flag_type_ enum for supported flags.
-
-All these functions in case of an error return a negative integer.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-void process_record(struct tep_record *record)
-{
- int data;
-
- data = tep_data_type(tep, record);
- if (data >= 0) {
- /* Got the ID of the event */
- }
-
- data = tep_data_pid(tep, record);
- if (data >= 0) {
- /* Got the process ID */
- }
-
- data = tep_data_preempt_count(tep, record);
- if (data >= 0) {
- /* Got the preemption count */
- }
-
- data = tep_data_flags(tep, record);
- if (data >= 0) {
- /* Got the latency flags */
- }
-}
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_register_event_handler, tep_unregister_event_handler - Register /
-unregisters a callback function to parse an event information.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-enum *tep_reg_handler* {
- _TEP_REGISTER_SUCCESS_,
- _TEP_REGISTER_SUCCESS_OVERWRITE_,
-};
-
-int *tep_register_event_handler*(struct tep_handle pass:[*]_tep_, int _id_, const char pass:[*]_sys_name_, const char pass:[*]_event_name_, tep_event_handler_func _func_, void pass:[*]_context_);
-int *tep_unregister_event_handler*(struct tep_handle pass:[*]tep, int id, const char pass:[*]sys_name, const char pass:[*]event_name, tep_event_handler_func func, void pass:[*]_context_);
-
-typedef int (*pass:[*]tep_event_handler_func*)(struct trace_seq pass:[*]s, struct tep_record pass:[*]record, struct tep_event pass:[*]event, void pass:[*]context);
---
-
-DESCRIPTION
------------
-The _tep_register_event_handler()_ function registers a handler function,
-which is going to be called to parse the information for a given event.
-The _tep_ argument is the trace event parser context. The _id_ argument is
-the id of the event. The _sys_name_ argument is the name of the system,
-the event belongs to. The _event_name_ argument is the name of the event.
-If _id_ is >= 0, it is used to find the event, otherwise _sys_name_ and
-_event_name_ are used. The _func_ is a pointer to the function, which is going
-to be called to parse the event information. The _context_ argument is a pointer
-to the context data, which will be passed to the _func_. If a handler function
-for the same event is already registered, it will be overridden with the new
-one. This mechanism allows a developer to override the parsing of a given event.
-If for some reason the default print format is not sufficient, the developer
-can register a function for an event to be used to parse the data instead.
-
-The _tep_unregister_event_handler()_ function unregisters the handler function,
-previously registered with _tep_register_event_handler()_. The _tep_ argument
-is the trace event parser context. The _id_, _sys_name_, _event_name_, _func_,
-and _context_ are the same arguments, as when the callback function _func_ was
-registered.
-
-The _tep_event_handler_func_ is the type of the custom event handler
-function. The _s_ argument is the trace sequence, it can be used to create a
-custom string, describing the event. A _record_ to get the event from is passed
-as input parameter and also the _event_ - the handle to the record's event. The
-_context_ is custom context, set when the custom event handler is registered.
-
-RETURN VALUE
-------------
-The _tep_register_event_handler()_ function returns _TEP_REGISTER_SUCCESS_
-if the new handler is registered successfully or
-_TEP_REGISTER_SUCCESS_OVERWRITE_ if an existing handler is overwritten.
-If there is not enough memory to complete the registration,
-TEP_ERRNO__MEM_ALLOC_FAILED is returned.
-
-The _tep_unregister_event_handler()_ function returns 0 if _func_ was removed
-successful or, -1 if the event was not found.
-
-The _tep_event_handler_func_ should return -1 in case of an error,
-or 0 otherwise.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-#include <trace-seq.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-int timer_expire_handler(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- trace_seq_printf(s, "hrtimer=");
-
- if (tep_print_num_field(s, "0x%llx", event, "timer", record, 0) == -1)
- tep_print_num_field(s, "0x%llx", event, "hrtimer", record, 1);
-
- trace_seq_printf(s, " now=");
-
- tep_print_num_field(s, "%llu", event, "now", record, 1);
-
- tep_print_func_field(s, " function=%s", event, "function", record, 0);
-
- return 0;
-}
-...
- int ret;
-
- ret = tep_register_event_handler(tep, -1, "timer", "hrtimer_expire_entry",
- timer_expire_handler, NULL);
- if (ret < 0) {
- char buf[32];
-
- tep_strerror(tep, ret, buf, 32)
- printf("Failed to register handler for hrtimer_expire_entry: %s\n", buf);
- } else {
- switch (ret) {
- case TEP_REGISTER_SUCCESS:
- printf ("Registered handler for hrtimer_expire_entry\n");
- break;
- case TEP_REGISTER_SUCCESS_OVERWRITE:
- printf ("Overwrote handler for hrtimer_expire_entry\n");
- break;
- }
- }
-...
- ret = tep_unregister_event_handler(tep, -1, "timer", "hrtimer_expire_entry",
- timer_expire_handler, NULL);
- if ( ret )
- printf ("Failed to unregister handler for hrtimer_expire_entry\n");
-
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*trace-seq.h*
- Header file to include in order to have access to trace sequences
- related APIs. Trace sequences are used to allow a function to call
- several other functions to create a string of data to use.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_register_print_function,tep_unregister_print_function -
-Registers / Unregisters a helper function.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-enum *tep_func_arg_type* {
- TEP_FUNC_ARG_VOID,
- TEP_FUNC_ARG_INT,
- TEP_FUNC_ARG_LONG,
- TEP_FUNC_ARG_STRING,
- TEP_FUNC_ARG_PTR,
- TEP_FUNC_ARG_MAX_TYPES
-};
-
-typedef unsigned long long (*pass:[*]tep_func_handler*)(struct trace_seq pass:[*]s, unsigned long long pass:[*]args);
-
-int *tep_register_print_function*(struct tep_handle pass:[*]_tep_, tep_func_handler _func_, enum tep_func_arg_type _ret_type_, char pass:[*]_name_, _..._);
-int *tep_unregister_print_function*(struct tep_handle pass:[*]_tep_, tep_func_handler _func_, char pass:[*]_name_);
---
-
-DESCRIPTION
------------
-Some events may have helper functions in the print format arguments.
-This allows a plugin to dynamically create a way to process one of
-these functions.
-
-The _tep_register_print_function()_ registers such helper function. The _tep_
-argument is the trace event parser context. The _func_ argument is a pointer
-to the helper function. The _ret_type_ argument is the return type of the
-helper function, value from the _tep_func_arg_type_ enum. The _name_ is the name
-of the helper function, as seen in the print format arguments. The _..._ is a
-variable list of _tep_func_arg_type_ enums, the _func_ function arguments.
-This list must end with _TEP_FUNC_ARG_VOID_. See 'EXAMPLE' section.
-
-The _tep_unregister_print_function()_ unregisters a helper function, previously
-registered with _tep_register_print_function()_. The _tep_ argument is the
-trace event parser context. The _func_ and _name_ arguments are the same, used
-when the helper function was registered.
-
-The _tep_func_handler_ is the type of the helper function. The _s_ argument is
-the trace sequence, it can be used to create a custom string.
-The _args_ is a list of arguments, defined when the helper function was
-registered.
-
-RETURN VALUE
-------------
-The _tep_register_print_function()_ function returns 0 in case of success.
-In case of an error, TEP_ERRNO_... code is returned.
-
-The _tep_unregister_print_function()_ returns 0 in case of success, or -1 in
-case of an error.
-
-EXAMPLE
--------
-Some events have internal functions calls, that appear in the print format
-output. For example "tracefs/events/i915/g4x_wm/format" has:
-[source,c]
---
-print fmt: "pipe %c, frame=%u, scanline=%u, wm %d/%d/%d, sr %s/%d/%d/%d, hpll %s/%d/%d/%d, fbc %s",
- ((REC->pipe) + 'A'), REC->frame, REC->scanline, REC->primary,
- REC->sprite, REC->cursor, yesno(REC->cxsr), REC->sr_plane,
- REC->sr_cursor, REC->sr_fbc, yesno(REC->hpll), REC->hpll_plane,
- REC->hpll_cursor, REC->hpll_fbc, yesno(REC->fbc)
---
-Notice the call to function _yesno()_ in the print arguments. In the kernel
-context, this function has the following implementation:
-[source,c]
---
-static const char *yesno(int x)
-{
- static const char *yes = "yes";
- static const char *no = "no";
-
- return x ? yes : no;
-}
---
-The user space event parser has no idea how to handle this _yesno()_ function.
-The _tep_register_print_function()_ API can be used to register a user space
-helper function, mapped to the kernel's _yesno()_:
-[source,c]
---
-#include <event-parse.h>
-#include <trace-seq.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-static const char *yes_no_helper(int x)
-{
- return x ? "yes" : "no";
-}
-...
- if ( tep_register_print_function(tep,
- yes_no_helper,
- TEP_FUNC_ARG_STRING,
- "yesno",
- TEP_FUNC_ARG_INT,
- TEP_FUNC_ARG_VOID) != 0) {
- /* Failed to register yes_no_helper function */
- }
-
-/*
- Now, when the event parser encounters this yesno() function, it will know
- how to handle it.
-*/
-...
- if (tep_unregister_print_function(tep, yes_no_helper, "yesno") != 0) {
- /* Failed to unregister yes_no_helper function */
- }
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*trace-seq.h*
- Header file to include in order to have access to trace sequences
- related APIs. Trace sequences are used to allow a function to call
- several other functions to create a string of data to use.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_set_flag, tep_clear_flag, tep_test_flag -
-Manage flags of trace event parser context.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-enum *tep_flag* {
- _TEP_NSEC_OUTPUT_,
- _TEP_DISABLE_SYS_PLUGINS_,
- _TEP_DISABLE_PLUGINS_
-};
-void *tep_set_flag*(struct tep_handle pass:[*]_tep_, enum tep_flag _flag_);
-void *tep_clear_flag*(struct tep_handle pass:[*]_tep_, enum tep_flag _flag_);
-bool *tep_test_flag*(struct tep_handle pass:[*]_tep_, enum tep_flag _flag_);
---
-
-DESCRIPTION
------------
-Trace event parser context flags are defined in *enum tep_flag*:
-[verse]
---
-_TEP_NSEC_OUTPUT_ - print event's timestamp in nano seconds, instead of micro seconds.
-_TEP_DISABLE_SYS_PLUGINS_ - disable plugins, located in system's plugin
- directory. This directory is defined at library compile
- time, and usually depends on library installation
- prefix: (install_preffix)/lib/traceevent/plugins
-_TEP_DISABLE_PLUGINS_ - disable all library plugins:
- - in system's plugin directory
- - in directory, defined by the environment variable _TRACEEVENT_PLUGIN_DIR_
- - in user's home directory, _~/.traceevent/plugins_
---
-Note: plugin related flags must me set before calling _tep_load_plugins()_ API.
-
-The _tep_set_flag()_ function sets _flag_ to _tep_ context.
-
-The _tep_clear_flag()_ function clears _flag_ from _tep_ context.
-
-The _tep_test_flag()_ function tests if _flag_ is set to _tep_ context.
-
-RETURN VALUE
-------------
-_tep_test_flag()_ function returns true if _flag_ is set, false otherwise.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-/* Print timestamps in nanoseconds */
-tep_set_flag(tep, TEP_NSEC_OUTPUT);
-...
-if (tep_test_flag(tep, TEP_NSEC_OUTPUT)) {
- /* print timestamps in nanoseconds */
-} else {
- /* print timestamps in microseconds */
-}
-...
-/* Print timestamps in microseconds */
-tep_clear_flag(tep, TEP_NSEC_OUTPUT);
-...
---
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-tep_strerror - Returns a string describing regular errno and tep error number.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-int *tep_strerror*(struct tep_handle pass:[*]_tep_, enum tep_errno _errnum_, char pass:[*]_buf_, size_t _buflen_);
-
---
-DESCRIPTION
------------
-The _tep_strerror()_ function converts tep error number into a human
-readable string.
-The _tep_ argument is trace event parser context. The _errnum_ is a regular
-errno, defined in errno.h, or a tep error number. The string, describing this
-error number is copied in the _buf_ argument. The _buflen_ argument is
-the size of the _buf_.
-
-It as a thread safe wrapper around strerror_r(). The library function has two
-different behaviors - POSIX and GNU specific. The _tep_strerror()_ API always
-behaves as the POSIX version - the error string is copied in the user supplied
-buffer.
-
-RETURN VALUE
-------------
-The _tep_strerror()_ function returns 0, if a valid _errnum_ is passed and the
-string is copied into _buf_. If _errnum_ is not a valid error number,
--1 is returned and _buf_ is not modified.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-...
-struct tep_handle *tep = tep_alloc();
-...
-char buf[32];
-char *pool = calloc(1, 128);
-if (tep == NULL) {
- tep_strerror(tep, TEP_ERRNO__MEM_ALLOC_FAILED, buf, 32);
- printf ("The pool is not initialized, %s", buf);
-}
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-trace_seq_init, trace_seq_destroy, trace_seq_reset, trace_seq_terminate,
-trace_seq_putc, trace_seq_puts, trace_seq_printf, trace_seq_vprintf,
-trace_seq_do_fprintf, trace_seq_do_printf -
-Initialize / destroy a trace sequence.
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-*#include <trace-seq.h>*
-
-void *trace_seq_init*(struct trace_seq pass:[*]_s_);
-void *trace_seq_destroy*(struct trace_seq pass:[*]_s_);
-void *trace_seq_reset*(struct trace_seq pass:[*]_s_);
-void *trace_seq_terminate*(struct trace_seq pass:[*]_s_);
-int *trace_seq_putc*(struct trace_seq pass:[*]_s_, unsigned char _c_);
-int *trace_seq_puts*(struct trace_seq pass:[*]_s_, const char pass:[*]_str_);
-int *trace_seq_printf*(struct trace_seq pass:[*]_s_, const char pass:[*]_fmt_, _..._);
-int *trace_seq_vprintf*(struct trace_seq pass:[*]_s_, const char pass:[*]_fmt_, va_list _args_);
-int *trace_seq_do_printf*(struct trace_seq pass:[*]_s_);
-int *trace_seq_do_fprintf*(struct trace_seq pass:[*]_s_, FILE pass:[*]_fp_);
---
-
-DESCRIPTION
------------
-Trace sequences are used to allow a function to call several other functions
-to create a string of data to use.
-
-The _trace_seq_init()_ function initializes the trace sequence _s_.
-
-The _trace_seq_destroy()_ function destroys the trace sequence _s_ and frees
-all its resources that it had used.
-
-The _trace_seq_reset()_ function re-initializes the trace sequence _s_. All
-characters already written in _s_ will be deleted.
-
-The _trace_seq_terminate()_ function terminates the trace sequence _s_. It puts
-the null character pass:['\0'] at the end of the buffer.
-
-The _trace_seq_putc()_ function puts a single character _c_ in the trace
-sequence _s_.
-
-The _trace_seq_puts()_ function puts a NULL terminated string _str_ in the
-trace sequence _s_.
-
-The _trace_seq_printf()_ function puts a formated string _fmt _with
-variable arguments _..._ in the trace sequence _s_.
-
-The _trace_seq_vprintf()_ function puts a formated string _fmt _with
-list of arguments _args_ in the trace sequence _s_.
-
-The _trace_seq_do_printf()_ function prints the buffer of trace sequence _s_ to
-the standard output stdout.
-
-The _trace_seq_do_fprintf()_ function prints the buffer of trace sequence _s_
-to the given file _fp_.
-
-RETURN VALUE
-------------
-Both _trace_seq_putc()_ and _trace_seq_puts()_ functions return the number of
-characters put in the trace sequence, or 0 in case of an error
-
-Both _trace_seq_printf()_ and _trace_seq_vprintf()_ functions return 0 if the
-trace oversizes the buffer's free space, the number of characters printed, or
-a negative value in case of an error.
-
-Both _trace_seq_do_printf()_ and _trace_seq_do_fprintf()_ functions return the
-number of printed characters, or -1 in case of an error.
-
-EXAMPLE
--------
-[source,c]
---
-#include <event-parse.h>
-#include <trace-seq.h>
-...
-struct trace_seq seq;
-trace_seq_init(&seq);
-...
-void foo_seq_print(struct trace_seq *tseq, char *format, ...)
-{
- va_list ap;
- va_start(ap, format);
- if (trace_seq_vprintf(tseq, format, ap) <= 0) {
- /* Failed to print in the trace sequence */
- }
- va_end(ap);
-}
-
-trace_seq_reset(&seq);
-
-char *str = " MAN page example";
-if (trace_seq_puts(&seq, str) != strlen(str)) {
- /* Failed to put str in the trace sequence */
-}
-if (trace_seq_putc(&seq, ':') != 1) {
- /* Failed to put ':' in the trace sequence */
-}
-if (trace_seq_printf(&seq, " trace sequence: %d", 1) <= 0) {
- /* Failed to print in the trace sequence */
-}
-foo_seq_print( &seq, " %d\n", 2);
-
-trace_seq_terminate(&seq);
-...
-
-if (trace_seq_do_printf(&seq) < 0 ) {
- /* Failed to print the sequence buffer to the standard output */
-}
-FILE *fp = fopen("trace.txt", "w");
-if (trace_seq_do_fprintf(&seq, fp) < 0 ) [
- /* Failed to print the sequence buffer to the trace.txt file */
-}
-
-trace_seq_destroy(&seq);
-...
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*trace-seq.h*
- Header file to include in order to have access to trace sequences related APIs.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_libtraceevent(3)_, _trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-libtraceevent(3)
-================
-
-NAME
-----
-libtraceevent - Linux kernel trace event library
-
-SYNOPSIS
---------
-[verse]
---
-*#include <event-parse.h>*
-
-Management of tep handler data structure and access of its members:
- struct tep_handle pass:[*]*tep_alloc*(void);
- void *tep_free*(struct tep_handle pass:[*]_tep_);
- void *tep_ref*(struct tep_handle pass:[*]_tep_);
- void *tep_unref*(struct tep_handle pass:[*]_tep_);
- int *tep_get_ref*(struct tep_handle pass:[*]_tep_);
- void *tep_set_flag*(struct tep_handle pass:[*]_tep_, enum tep_flag _flag_);
- void *tep_clear_flag*(struct tep_handle pass:[*]_tep_, enum tep_flag _flag_);
- bool *tep_test_flag*(struct tep_handle pass:[*]_tep_, enum tep_flag _flags_);
- int *tep_get_cpus*(struct tep_handle pass:[*]_tep_);
- void *tep_set_cpus*(struct tep_handle pass:[*]_tep_, int _cpus_);
- int *tep_get_long_size*(strucqt tep_handle pass:[*]_tep_);
- void *tep_set_long_size*(struct tep_handle pass:[*]_tep_, int _long_size_);
- int *tep_get_page_size*(struct tep_handle pass:[*]_tep_);
- void *tep_set_page_size*(struct tep_handle pass:[*]_tep_, int _page_size_);
- int *tep_get_header_page_size*(struct tep_handle pass:[*]_tep_);
- int *tep_get_header_timestamp_size*(struct tep_handle pass:[*]_tep_);
- bool *tep_is_old_format*(struct tep_handle pass:[*]_tep_);
- int *tep_strerror*(struct tep_handle pass:[*]_tep_, enum tep_errno _errnum_, char pass:[*]_buf_, size_t _buflen_);
-
-Register / unregister APIs:
- int *tep_register_function*(struct tep_handle pass:[*]_tep_, char pass:[*]_name_, unsigned long long _addr_, char pass:[*]_mod_);
- int *tep_register_event_handler*(struct tep_handle pass:[*]_tep_, int _id_, const char pass:[*]_sys_name_, const char pass:[*]_event_name_, tep_event_handler_func _func_, void pass:[*]_context_);
- int *tep_unregister_event_handler*(struct tep_handle pass:[*]tep, int id, const char pass:[*]sys_name, const char pass:[*]event_name, tep_event_handler_func func, void pass:[*]_context_);
- int *tep_register_print_string*(struct tep_handle pass:[*]_tep_, const char pass:[*]_fmt_, unsigned long long _addr_);
- int *tep_register_print_function*(struct tep_handle pass:[*]_tep_, tep_func_handler _func_, enum tep_func_arg_type _ret_type_, char pass:[*]_name_, _..._);
- int *tep_unregister_print_function*(struct tep_handle pass:[*]_tep_, tep_func_handler _func_, char pass:[*]_name_);
-
-Plugins management:
- struct tep_plugin_list pass:[*]*tep_load_plugins*(struct tep_handle pass:[*]_tep_);
- void *tep_unload_plugins*(struct tep_plugin_list pass:[*]_plugin_list_, struct tep_handle pass:[*]_tep_);
- char pass:[*]pass:[*]*tep_plugin_list_options*(void);
- void *tep_plugin_free_options_list*(char pass:[*]pass:[*]_list_);
- int *tep_plugin_add_options*(const char pass:[*]_name_, struct tep_plugin_option pass:[*]_options_);
- void *tep_plugin_remove_options*(struct tep_plugin_option pass:[*]_options_);
- void *tep_print_plugins*(struct trace_seq pass:[*]_s_, const char pass:[*]_prefix_, const char pass:[*]_suffix_, const struct tep_plugin_list pass:[*]_list_);
-
-Event related APIs:
- struct tep_event pass:[*]*tep_get_event*(struct tep_handle pass:[*]_tep_, int _index_);
- struct tep_event pass:[*]*tep_get_first_event*(struct tep_handle pass:[*]_tep_);
- int *tep_get_events_count*(struct tep_handle pass:[*]_tep_);
- struct tep_event pass:[*]pass:[*]*tep_list_events*(struct tep_handle pass:[*]_tep_, enum tep_event_sort_type _sort_type_);
- struct tep_event pass:[*]pass:[*]*tep_list_events_copy*(struct tep_handle pass:[*]_tep_, enum tep_event_sort_type _sort_type_);
- void *tep_print_event*(struct tep_handle pass:[*]_tep_, struct trace_seq pass:[*]_s_, struct tep_record pass:[*]_record_, const char pass:[*]_fmt_, _..._);
-
-Event finding:
- struct tep_event pass:[*]*tep_find_event*(struct tep_handle pass:[*]_tep_, int _id_);
- struct tep_event pass:[*]*tep_find_event_by_name*(struct tep_handle pass:[*]_tep_, const char pass:[*]_sys_, const char pass:[*]_name_);
- struct tep_event pass:[*]*tep_find_event_by_record*(struct tep_handle pass:[*]_tep_, struct tep_record pass:[*]_record_);
-
-Parsing of event files:
- int *tep_parse_header_page*(struct tep_handle pass:[*]_tep_, char pass:[*]_buf_, unsigned long _size_, int _long_size_);
- enum tep_errno *tep_parse_event*(struct tep_handle pass:[*]_tep_, const char pass:[*]_buf_, unsigned long _size_, const char pass:[*]_sys_);
- enum tep_errno *tep_parse_format*(struct tep_handle pass:[*]_tep_, struct tep_event pass:[*]pass:[*]_eventp_, const char pass:[*]_buf_, unsigned long _size_, const char pass:[*]_sys_);
-
-APIs related to fields from event's format files:
- struct tep_format_field pass:[*]pass:[*]*tep_event_common_fields*(struct tep_event pass:[*]_event_);
- struct tep_format_field pass:[*]pass:[*]*tep_event_fields*(struct tep_event pass:[*]_event_);
- void pass:[*]*tep_get_field_raw*(struct trace_seq pass:[*]_s_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, int pass:[*]_len_, int _err_);
- int *tep_get_field_val*(struct trace_seq pass:[*]_s_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, unsigned long long pass:[*]_val_, int _err_);
- int *tep_get_common_field_val*(struct trace_seq pass:[*]_s_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, unsigned long long pass:[*]_val_, int _err_);
- int *tep_get_any_field_val*(struct trace_seq pass:[*]_s_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, unsigned long long pass:[*]_val_, int _err_);
- int *tep_read_number_field*(struct tep_format_field pass:[*]_field_, const void pass:[*]_data_, unsigned long long pass:[*]_value_);
-
-Event fields printing:
- void *tep_print_field*(struct trace_seq pass:[*]_s_, void pass:[*]_data_, struct tep_format_field pass:[*]_field_);
- void *tep_print_fields*(struct trace_seq pass:[*]_s_, void pass:[*]_data_, int _size_, struct tep_event pass:[*]_event_);
- int *tep_print_num_field*(struct trace_seq pass:[*]_s_, const char pass:[*]_fmt_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, int _err_);
- int *tep_print_func_field*(struct trace_seq pass:[*]_s_, const char pass:[*]_fmt_, struct tep_event pass:[*]_event_, const char pass:[*]_name_, struct tep_record pass:[*]_record_, int _err_);
-
-Event fields finding:
- struct tep_format_field pass:[*]*tep_find_common_field*(struct tep_event pass:[*]_event_, const char pass:[*]_name_);
- struct tep_format_field pass:[*]*tep_find_field*(struct tep_event_ormat pass:[*]_event_, const char pass:[*]_name_);
- struct tep_format_field pass:[*]*tep_find_any_field*(struct tep_event pass:[*]_event_, const char pass:[*]_name_);
-
-Functions resolver:
- int *tep_set_function_resolver*(struct tep_handle pass:[*]_tep_, tep_func_resolver_t pass:[*]_func_, void pass:[*]_priv_);
- void *tep_reset_function_resolver*(struct tep_handle pass:[*]_tep_);
- const char pass:[*]*tep_find_function*(struct tep_handle pass:[*]_tep_, unsigned long long _addr_);
- unsigned long long *tep_find_function_address*(struct tep_handle pass:[*]_tep_, unsigned long long _addr_);
-
-Filter management:
- struct tep_event_filter pass:[*]*tep_filter_alloc*(struct tep_handle pass:[*]_tep_);
- enum tep_errno *tep_filter_add_filter_str*(struct tep_event_filter pass:[*]_filter_, const char pass:[*]_filter_str_);
- enum tep_errno *tep_filter_match*(struct tep_event_filter pass:[*]_filter_, struct tep_record pass:[*]_record_);
- int *tep_filter_strerror*(struct tep_event_filter pass:[*]_filter_, enum tep_errno _err_, char pass:[*]buf, size_t _buflen_);
- int *tep_event_filtered*(struct tep_event_filter pass:[*]_filter_, int _event_id_);
- void *tep_filter_reset*(struct tep_event_filter pass:[*]_filter_);
- void *tep_filter_free*(struct tep_event_filter pass:[*]_filter_);
- char pass:[*]*tep_filter_make_string*(struct tep_event_filter pass:[*]_filter_, int _event_id_);
- int *tep_filter_remove_event*(struct tep_event_filter pass:[*]_filter_, int _event_id_);
- int *tep_filter_copy*(struct tep_event_filter pass:[*]_dest_, struct tep_event_filter pass:[*]_source_);
- int *tep_filter_compare*(struct tep_event_filter pass:[*]_filter1_, struct tep_event_filter pass:[*]_filter2_);
-
-Parsing various data from the records:
- int *tep_data_type*(struct tep_handle pass:[*]_tep_, struct tep_record pass:[*]_rec_);
- int *tep_data_pid*(struct tep_handle pass:[*]_tep_, struct tep_record pass:[*]_rec_);
- int *tep_data_preempt_count*(struct tep_handle pass:[*]_tep_, struct tep_record pass:[*]_rec_);
- int *tep_data_flags*(struct tep_handle pass:[*]_tep_, struct tep_record pass:[*]_rec_);
-
-Command and task related APIs:
- const char pass:[*]*tep_data_comm_from_pid*(struct tep_handle pass:[*]_tep_, int _pid_);
- struct cmdline pass:[*]*tep_data_pid_from_comm*(struct tep_handle pass:[*]_tep_, const char pass:[*]_comm_, struct cmdline pass:[*]_next_);
- int *tep_register_comm*(struct tep_handle pass:[*]_tep_, const char pass:[*]_comm_, int _pid_);
- int *tep_override_comm*(struct tep_handle pass:[*]_tep_, const char pass:[*]_comm_, int _pid_);
- bool *tep_is_pid_registered*(struct tep_handle pass:[*]_tep_, int _pid_);
- int *tep_cmdline_pid*(struct tep_handle pass:[*]_tep_, struct cmdline pass:[*]_cmdline_);
-
-Endian related APIs:
- int *tep_is_bigendian*(void);
- unsigned long long *tep_read_number*(struct tep_handle pass:[*]_tep_, const void pass:[*]_ptr_, int _size_);
- bool *tep_is_file_bigendian*(struct tep_handle pass:[*]_tep_);
- void *tep_set_file_bigendian*(struct tep_handle pass:[*]_tep_, enum tep_endian _endian_);
- bool *tep_is_local_bigendian*(struct tep_handle pass:[*]_tep_);
- void *tep_set_local_bigendian*(struct tep_handle pass:[*]_tep_, enum tep_endian _endian_);
-
-Trace sequences:
-*#include <trace-seq.h>*
- void *trace_seq_init*(struct trace_seq pass:[*]_s_);
- void *trace_seq_reset*(struct trace_seq pass:[*]_s_);
- void *trace_seq_destroy*(struct trace_seq pass:[*]_s_);
- int *trace_seq_printf*(struct trace_seq pass:[*]_s_, const char pass:[*]_fmt_, ...);
- int *trace_seq_vprintf*(struct trace_seq pass:[*]_s_, const char pass:[*]_fmt_, va_list _args_);
- int *trace_seq_puts*(struct trace_seq pass:[*]_s_, const char pass:[*]_str_);
- int *trace_seq_putc*(struct trace_seq pass:[*]_s_, unsigned char _c_);
- void *trace_seq_terminate*(struct trace_seq pass:[*]_s_);
- int *trace_seq_do_fprintf*(struct trace_seq pass:[*]_s_, FILE pass:[*]_fp_);
- int *trace_seq_do_printf*(struct trace_seq pass:[*]_s_);
---
-
-DESCRIPTION
------------
-The libtraceevent(3) library provides APIs to access kernel tracepoint events,
-located in the tracefs file system under the events directory.
-
-ENVIRONMENT
------------
-[verse]
---
-TRACEEVENT_PLUGIN_DIR
- Additional plugin directory. All shared object files, located in this directory will be loaded as traceevent plugins.
---
-
-FILES
------
-[verse]
---
-*event-parse.h*
- Header file to include in order to have access to the library APIs.
-*trace-seq.h*
- Header file to include in order to have access to trace sequences related APIs.
- Trace sequences are used to allow a function to call several other functions
- to create a string of data to use.
-*-ltraceevent*
- Linker switch to add when building a program that uses the library.
---
-
-SEE ALSO
---------
-_trace-cmd(1)_
-
-AUTHOR
-------
-[verse]
---
-*Steven Rostedt* <rostedt@goodmis.org>, author of *libtraceevent*.
-*Tzvetomir Stoyanov* <tz.stoyanov@gmail.com>, author of this man page.
---
-REPORTING BUGS
---------------
-Report bugs to <linux-trace-devel@vger.kernel.org>
-
-LICENSE
--------
-libtraceevent is Free Software licensed under the GNU LGPL 2.1
-
-RESOURCES
----------
-https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+++ /dev/null
-<!-- manpage-1.72.xsl:
- special settings for manpages rendered from asciidoc+docbook
- handles peculiarities in docbook-xsl 1.72.0 -->
-<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
- version="1.0">
-
-<xsl:import href="manpage-base.xsl"/>
-
-<!-- these are the special values for the roff control characters
- needed for docbook-xsl 1.72.0 -->
-<xsl:param name="git.docbook.backslash">▓</xsl:param>
-<xsl:param name="git.docbook.dot" >⌂</xsl:param>
-
-</xsl:stylesheet>
+++ /dev/null
-<!-- manpage-base.xsl:
- special formatting for manpages rendered from asciidoc+docbook -->
-<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
- version="1.0">
-
-<!-- these params silence some output from xmlto -->
-<xsl:param name="man.output.quietly" select="1"/>
-<xsl:param name="refentry.meta.get.quietly" select="1"/>
-
-<!-- convert asciidoc callouts to man page format;
- git.docbook.backslash and git.docbook.dot params
- must be supplied by another XSL file or other means -->
-<xsl:template match="co">
- <xsl:value-of select="concat(
- $git.docbook.backslash,'fB(',
- substring-after(@id,'-'),')',
- $git.docbook.backslash,'fR')"/>
-</xsl:template>
-<xsl:template match="calloutlist">
- <xsl:value-of select="$git.docbook.dot"/>
- <xsl:text>sp </xsl:text>
- <xsl:apply-templates/>
- <xsl:text> </xsl:text>
-</xsl:template>
-<xsl:template match="callout">
- <xsl:value-of select="concat(
- $git.docbook.backslash,'fB',
- substring-after(@arearefs,'-'),
- '. ',$git.docbook.backslash,'fR')"/>
- <xsl:apply-templates/>
- <xsl:value-of select="$git.docbook.dot"/>
- <xsl:text>br </xsl:text>
-</xsl:template>
-
-</xsl:stylesheet>
+++ /dev/null
-<!-- manpage-bold-literal.xsl:
- special formatting for manpages rendered from asciidoc+docbook -->
-<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
- version="1.0">
-
-<!-- render literal text as bold (instead of plain or monospace);
- this makes literal text easier to distinguish in manpages
- viewed on a tty -->
-<xsl:template match="literal">
- <xsl:value-of select="$git.docbook.backslash"/>
- <xsl:text>fB</xsl:text>
- <xsl:apply-templates/>
- <xsl:value-of select="$git.docbook.backslash"/>
- <xsl:text>fR</xsl:text>
-</xsl:template>
-
-</xsl:stylesheet>
+++ /dev/null
-<!-- manpage-normal.xsl:
- special settings for manpages rendered from asciidoc+docbook
- handles anything we want to keep away from docbook-xsl 1.72.0 -->
-<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
- version="1.0">
-
-<xsl:import href="manpage-base.xsl"/>
-
-<!-- these are the normal values for the roff control characters -->
-<xsl:param name="git.docbook.backslash">\</xsl:param>
-<xsl:param name="git.docbook.dot" >.</xsl:param>
-
-</xsl:stylesheet>
+++ /dev/null
-<!-- manpage-suppress-sp.xsl:
- special settings for manpages rendered from asciidoc+docbook
- handles erroneous, inline .sp in manpage output of some
- versions of docbook-xsl -->
-<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
- version="1.0">
-
-<!-- attempt to work around spurious .sp at the tail of the line
- that some versions of docbook stylesheets seem to add -->
-<xsl:template match="simpara">
- <xsl:variable name="content">
- <xsl:apply-templates/>
- </xsl:variable>
- <xsl:value-of select="normalize-space($content)"/>
- <xsl:if test="not(ancestor::authorblurb) and
- not(ancestor::personblurb)">
- <xsl:text> </xsl:text>
- </xsl:if>
-</xsl:template>
-
-</xsl:stylesheet>
+++ /dev/null
-# SPDX-License-Identifier: GPL-2.0
-# trace-cmd version
-EP_VERSION = 1
-EP_PATCHLEVEL = 1
-EP_EXTRAVERSION = 0
-
-# file format version
-FILE_VERSION = 6
-
-MAKEFLAGS += --no-print-directory
-
-
-# Makefiles suck: This macro sets a default value of $(2) for the
-# variable named by $(1), unless the variable has been set by
-# environment or command line. This is necessary for CC and AR
-# because make sets default values, so the simpler ?= approach
-# won't work as expected.
-define allow-override
- $(if $(or $(findstring environment,$(origin $(1))),\
- $(findstring command line,$(origin $(1)))),,\
- $(eval $(1) = $(2)))
-endef
-
-# Allow setting CC and AR, or setting CROSS_COMPILE as a prefix.
-$(call allow-override,CC,$(CROSS_COMPILE)gcc)
-$(call allow-override,AR,$(CROSS_COMPILE)ar)
-$(call allow-override,NM,$(CROSS_COMPILE)nm)
-$(call allow-override,PKG_CONFIG,pkg-config)
-
-EXT = -std=gnu99
-INSTALL = install
-
-# Use DESTDIR for installing into a different root directory.
-# This is useful for building a package. The program will be
-# installed in this directory as if it was the root directory.
-# Then the build tool can move it later.
-DESTDIR ?=
-DESTDIR_SQ = '$(subst ','\'',$(DESTDIR))'
-
-LP64 := $(shell echo __LP64__ | ${CC} ${CFLAGS} -E -x c - | tail -n 1)
-ifeq ($(LP64), 1)
- libdir_relative_temp = lib64
-else
- libdir_relative_temp = lib
-endif
-
-libdir_relative ?= $(libdir_relative_temp)
-prefix ?= /usr/local
-libdir = $(prefix)/$(libdir_relative)
-man_dir = $(prefix)/share/man
-man_dir_SQ = '$(subst ','\'',$(man_dir))'
-pkgconfig_dir ?= $(word 1,$(shell $(PKG_CONFIG) \
- --variable pc_path pkg-config | tr ":" " "))
-includedir_relative = traceevent
-includedir = $(prefix)/include/$(includedir_relative)
-includedir_SQ = '$(subst ','\'',$(includedir))'
-
-export man_dir man_dir_SQ INSTALL
-export DESTDIR DESTDIR_SQ
-export EVENT_PARSE_VERSION
-
-include ../../scripts/Makefile.include
-
-# copy a bit from Linux kbuild
-
-ifeq ("$(origin V)", "command line")
- VERBOSE = $(V)
-endif
-ifndef VERBOSE
- VERBOSE = 0
-endif
-
-ifeq ($(srctree),)
-srctree := $(patsubst %/,%,$(dir $(CURDIR)))
-srctree := $(patsubst %/,%,$(dir $(srctree)))
-srctree := $(patsubst %/,%,$(dir $(srctree)))
-#$(info Determined 'srctree' to be $(srctree))
-endif
-
-export prefix libdir src obj
-
-# Shell quotes
-libdir_SQ = $(subst ','\'',$(libdir))
-libdir_relative_SQ = $(subst ','\'',$(libdir_relative))
-
-CONFIG_INCLUDES =
-CONFIG_LIBS =
-CONFIG_FLAGS =
-
-VERSION = $(EP_VERSION)
-PATCHLEVEL = $(EP_PATCHLEVEL)
-EXTRAVERSION = $(EP_EXTRAVERSION)
-
-OBJ = $@
-N =
-
-EVENT_PARSE_VERSION = $(EP_VERSION).$(EP_PATCHLEVEL).$(EP_EXTRAVERSION)
-
-LIB_TARGET = libtraceevent.a libtraceevent.so.$(EVENT_PARSE_VERSION)
-LIB_INSTALL = libtraceevent.a libtraceevent.so*
-LIB_INSTALL := $(addprefix $(OUTPUT),$(LIB_INSTALL))
-
-INCLUDES = -I. -I $(srctree)/tools/include $(CONFIG_INCLUDES)
-
-# Set compile option CFLAGS
-ifdef EXTRA_CFLAGS
- CFLAGS := $(EXTRA_CFLAGS)
-else
- CFLAGS := -g -Wall
-endif
-
-# Append required CFLAGS
-override CFLAGS += -fPIC
-override CFLAGS += $(CONFIG_FLAGS) $(INCLUDES) $(PLUGIN_DIR_SQ)
-override CFLAGS += $(udis86-flags) -D_GNU_SOURCE
-
-ifeq ($(VERBOSE),1)
- Q =
-else
- Q = @
-endif
-
-# Disable command line variables (CFLAGS) override from top
-# level Makefile (perf), otherwise build Makefile will get
-# the same command line setup.
-MAKEOVERRIDES=
-
-export srctree OUTPUT CC LD CFLAGS V
-build := -f $(srctree)/tools/build/Makefile.build dir=. obj
-
-TE_IN := $(OUTPUT)libtraceevent-in.o
-LIB_TARGET := $(addprefix $(OUTPUT),$(LIB_TARGET))
-
-CMD_TARGETS = $(LIB_TARGET)
-
-TARGETS = $(CMD_TARGETS)
-
-all: all_cmd plugins
-
-all_cmd: $(CMD_TARGETS)
-
-$(TE_IN): force
- $(Q)$(MAKE) $(build)=libtraceevent
-
-$(OUTPUT)libtraceevent.so.$(EVENT_PARSE_VERSION): $(TE_IN)
- $(QUIET_LINK)$(CC) --shared $(LDFLAGS) $^ -Wl,-soname,libtraceevent.so.$(EP_VERSION) -o $@
- @ln -sf $(@F) $(OUTPUT)libtraceevent.so
- @ln -sf $(@F) $(OUTPUT)libtraceevent.so.$(EP_VERSION)
-
-$(OUTPUT)libtraceevent.a: $(TE_IN)
- $(QUIET_LINK)$(RM) $@; $(AR) rcs $@ $^
-
-$(OUTPUT)%.so: $(OUTPUT)%-in.o
- $(QUIET_LINK)$(CC) $(CFLAGS) -shared $(LDFLAGS) -nostartfiles -o $@ $^
-
-define make_version.h
- (echo '/* This file is automatically generated. Do not modify. */'; \
- echo \#define VERSION_CODE $(shell \
- expr $(VERSION) \* 256 + $(PATCHLEVEL)); \
- echo '#define EXTRAVERSION ' $(EXTRAVERSION); \
- echo '#define VERSION_STRING "'$(VERSION).$(PATCHLEVEL).$(EXTRAVERSION)'"'; \
- echo '#define FILE_VERSION '$(FILE_VERSION); \
- ) > $1
-endef
-
-define update_version.h
- ($(call make_version.h, $@.tmp); \
- if [ -r $@ ] && cmp -s $@ $@.tmp; then \
- rm -f $@.tmp; \
- else \
- echo ' UPDATE $@'; \
- mv -f $@.tmp $@; \
- fi);
-endef
-
-ep_version.h: force
- $(Q)$(N)$(call update_version.h)
-
-VERSION_FILES = ep_version.h
-
-define update_dir
- (echo $1 > $@.tmp; \
- if [ -r $@ ] && cmp -s $@ $@.tmp; then \
- rm -f $@.tmp; \
- else \
- echo ' UPDATE $@'; \
- mv -f $@.tmp $@; \
- fi);
-endef
-
-tags: force
- $(RM) tags
- find . -name '*.[ch]' | xargs ctags --extra=+f --c-kinds=+px \
- --regex-c++='/_PE\(([^,)]*).*/TEP_ERRNO__\1/'
-
-TAGS: force
- $(RM) TAGS
- find . -name '*.[ch]' | xargs etags \
- --regex='/_PE(\([^,)]*\).*/TEP_ERRNO__\1/'
-
-define do_install_mkdir
- if [ ! -d '$(DESTDIR_SQ)$1' ]; then \
- $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$1'; \
- fi
-endef
-
-define do_install
- $(call do_install_mkdir,$2); \
- $(INSTALL) $(if $3,-m $3,) $1 '$(DESTDIR_SQ)$2'
-endef
-
-PKG_CONFIG_SOURCE_FILE = libtraceevent.pc
-PKG_CONFIG_FILE := $(addprefix $(OUTPUT),$(PKG_CONFIG_SOURCE_FILE))
-define do_install_pkgconfig_file
- if [ -n "${pkgconfig_dir}" ]; then \
- cp -f ${PKG_CONFIG_SOURCE_FILE}.template ${PKG_CONFIG_FILE}; \
- sed -i "s|INSTALL_PREFIX|${1}|g" ${PKG_CONFIG_FILE}; \
- sed -i "s|LIB_VERSION|${EVENT_PARSE_VERSION}|g" ${PKG_CONFIG_FILE}; \
- sed -i "s|LIB_DIR|${libdir}|g" ${PKG_CONFIG_FILE}; \
- sed -i "s|HEADER_DIR|$(includedir)|g" ${PKG_CONFIG_FILE}; \
- $(call do_install,$(PKG_CONFIG_FILE),$(pkgconfig_dir),644); \
- else \
- (echo Failed to locate pkg-config directory) 1>&2; \
- fi
-endef
-
-install_lib: all_cmd install_plugins install_headers install_pkgconfig
- $(call QUIET_INSTALL, $(LIB_TARGET)) \
- $(call do_install_mkdir,$(libdir_SQ)); \
- cp -fpR $(LIB_INSTALL) $(DESTDIR)$(libdir_SQ)
-
-install_pkgconfig:
- $(call QUIET_INSTALL, $(PKG_CONFIG_FILE)) \
- $(call do_install_pkgconfig_file,$(prefix))
-
-install_headers:
- $(call QUIET_INSTALL, headers) \
- $(call do_install,event-parse.h,$(includedir_SQ),644); \
- $(call do_install,event-utils.h,$(includedir_SQ),644); \
- $(call do_install,trace-seq.h,$(includedir_SQ),644); \
- $(call do_install,kbuffer.h,$(includedir_SQ),644)
-
-install: install_lib
-
-clean: clean_plugins
- $(call QUIET_CLEAN, libtraceevent) \
- $(RM) *.o *~ $(TARGETS) *.a *.so $(VERSION_FILES) .*.d .*.cmd; \
- $(RM) TRACEEVENT-CFLAGS tags TAGS; \
- $(RM) $(PKG_CONFIG_FILE)
-
-PHONY += doc
-doc:
- $(call descend,Documentation)
-
-PHONY += doc-clean
-doc-clean:
- $(call descend,Documentation,clean)
-
-PHONY += doc-install
-doc-install:
- $(call descend,Documentation,install)
-
-PHONY += doc-uninstall
-doc-uninstall:
- $(call descend,Documentation,uninstall)
-
-PHONY += help
-help:
- @echo 'Possible targets:'
- @echo''
- @echo ' all - default, compile the library and the'\
- 'plugins'
- @echo ' plugins - compile the plugins'
- @echo ' install - install the library, the plugins,'\
- 'the header and pkgconfig files'
- @echo ' clean - clean the library and the plugins object files'
- @echo ' doc - compile the documentation files - man'\
- 'and html pages, in the Documentation directory'
- @echo ' doc-clean - clean the documentation files'
- @echo ' doc-install - install the man pages'
- @echo ' doc-uninstall - uninstall the man pages'
- @echo''
-
-PHONY += plugins
-plugins:
- $(call descend,plugins)
-
-PHONY += install_plugins
-install_plugins:
- $(call descend,plugins,install)
-
-PHONY += clean_plugins
-clean_plugins:
- $(call descend,plugins,clean)
-
-force:
-
-# Declare the contents of the .PHONY variable as phony. We keep that
-# information in a variable so we can use it in if_changed and friends.
-.PHONY: $(PHONY)
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-
-#include "event-parse.h"
-#include "event-parse-local.h"
-#include "event-utils.h"
-
-/**
- * tep_get_event - returns the event with the given index
- * @tep: a handle to the tep_handle
- * @index: index of the requested event, in the range 0 .. nr_events
- *
- * This returns pointer to the element of the events array with the given index
- * If @tep is NULL, or @index is not in the range 0 .. nr_events, NULL is returned.
- */
-struct tep_event *tep_get_event(struct tep_handle *tep, int index)
-{
- if (tep && tep->events && index < tep->nr_events)
- return tep->events[index];
-
- return NULL;
-}
-
-/**
- * tep_get_first_event - returns the first event in the events array
- * @tep: a handle to the tep_handle
- *
- * This returns pointer to the first element of the events array
- * If @tep is NULL, NULL is returned.
- */
-struct tep_event *tep_get_first_event(struct tep_handle *tep)
-{
- return tep_get_event(tep, 0);
-}
-
-/**
- * tep_get_events_count - get the number of defined events
- * @tep: a handle to the tep_handle
- *
- * This returns number of elements in event array
- * If @tep is NULL, 0 is returned.
- */
-int tep_get_events_count(struct tep_handle *tep)
-{
- if (tep)
- return tep->nr_events;
- return 0;
-}
-
-/**
- * tep_set_flag - set event parser flag
- * @tep: a handle to the tep_handle
- * @flag: flag, or combination of flags to be set
- * can be any combination from enum tep_flag
- *
- * This sets a flag or combination of flags from enum tep_flag
- */
-void tep_set_flag(struct tep_handle *tep, int flag)
-{
- if (tep)
- tep->flags |= flag;
-}
-
-/**
- * tep_clear_flag - clear event parser flag
- * @tep: a handle to the tep_handle
- * @flag: flag to be cleared
- *
- * This clears a tep flag
- */
-void tep_clear_flag(struct tep_handle *tep, enum tep_flag flag)
-{
- if (tep)
- tep->flags &= ~flag;
-}
-
-/**
- * tep_test_flag - check the state of event parser flag
- * @tep: a handle to the tep_handle
- * @flag: flag to be checked
- *
- * This returns the state of the requested tep flag.
- * Returns: true if the flag is set, false otherwise.
- */
-bool tep_test_flag(struct tep_handle *tep, enum tep_flag flag)
-{
- if (tep)
- return tep->flags & flag;
- return false;
-}
-
-__hidden unsigned short data2host2(struct tep_handle *tep, unsigned short data)
-{
- unsigned short swap;
-
- if (!tep || tep->host_bigendian == tep->file_bigendian)
- return data;
-
- swap = ((data & 0xffULL) << 8) |
- ((data & (0xffULL << 8)) >> 8);
-
- return swap;
-}
-
-__hidden unsigned int data2host4(struct tep_handle *tep, unsigned int data)
-{
- unsigned int swap;
-
- if (!tep || tep->host_bigendian == tep->file_bigendian)
- return data;
-
- swap = ((data & 0xffULL) << 24) |
- ((data & (0xffULL << 8)) << 8) |
- ((data & (0xffULL << 16)) >> 8) |
- ((data & (0xffULL << 24)) >> 24);
-
- return swap;
-}
-
-__hidden unsigned long long
-data2host8(struct tep_handle *tep, unsigned long long data)
-{
- unsigned long long swap;
-
- if (!tep || tep->host_bigendian == tep->file_bigendian)
- return data;
-
- swap = ((data & 0xffULL) << 56) |
- ((data & (0xffULL << 8)) << 40) |
- ((data & (0xffULL << 16)) << 24) |
- ((data & (0xffULL << 24)) << 8) |
- ((data & (0xffULL << 32)) >> 8) |
- ((data & (0xffULL << 40)) >> 24) |
- ((data & (0xffULL << 48)) >> 40) |
- ((data & (0xffULL << 56)) >> 56);
-
- return swap;
-}
-
-/**
- * tep_get_header_page_size - get size of the header page
- * @tep: a handle to the tep_handle
- *
- * This returns size of the header page
- * If @tep is NULL, 0 is returned.
- */
-int tep_get_header_page_size(struct tep_handle *tep)
-{
- if (tep)
- return tep->header_page_size_size;
- return 0;
-}
-
-/**
- * tep_get_header_timestamp_size - get size of the timestamp in the header page
- * @tep: a handle to the tep_handle
- *
- * This returns size of the timestamp in the header page
- * If @tep is NULL, 0 is returned.
- */
-int tep_get_header_timestamp_size(struct tep_handle *tep)
-{
- if (tep)
- return tep->header_page_ts_size;
- return 0;
-}
-
-/**
- * tep_get_cpus - get the number of CPUs
- * @tep: a handle to the tep_handle
- *
- * This returns the number of CPUs
- * If @tep is NULL, 0 is returned.
- */
-int tep_get_cpus(struct tep_handle *tep)
-{
- if (tep)
- return tep->cpus;
- return 0;
-}
-
-/**
- * tep_set_cpus - set the number of CPUs
- * @tep: a handle to the tep_handle
- *
- * This sets the number of CPUs
- */
-void tep_set_cpus(struct tep_handle *tep, int cpus)
-{
- if (tep)
- tep->cpus = cpus;
-}
-
-/**
- * tep_get_long_size - get the size of a long integer on the traced machine
- * @tep: a handle to the tep_handle
- *
- * This returns the size of a long integer on the traced machine
- * If @tep is NULL, 0 is returned.
- */
-int tep_get_long_size(struct tep_handle *tep)
-{
- if (tep)
- return tep->long_size;
- return 0;
-}
-
-/**
- * tep_set_long_size - set the size of a long integer on the traced machine
- * @tep: a handle to the tep_handle
- * @size: size, in bytes, of a long integer
- *
- * This sets the size of a long integer on the traced machine
- */
-void tep_set_long_size(struct tep_handle *tep, int long_size)
-{
- if (tep)
- tep->long_size = long_size;
-}
-
-/**
- * tep_get_page_size - get the size of a memory page on the traced machine
- * @tep: a handle to the tep_handle
- *
- * This returns the size of a memory page on the traced machine
- * If @tep is NULL, 0 is returned.
- */
-int tep_get_page_size(struct tep_handle *tep)
-{
- if (tep)
- return tep->page_size;
- return 0;
-}
-
-/**
- * tep_set_page_size - set the size of a memory page on the traced machine
- * @tep: a handle to the tep_handle
- * @_page_size: size of a memory page, in bytes
- *
- * This sets the size of a memory page on the traced machine
- */
-void tep_set_page_size(struct tep_handle *tep, int _page_size)
-{
- if (tep)
- tep->page_size = _page_size;
-}
-
-/**
- * tep_is_file_bigendian - return the endian of the file
- * @tep: a handle to the tep_handle
- *
- * This returns true if the file is in big endian order
- * If @tep is NULL, false is returned.
- */
-bool tep_is_file_bigendian(struct tep_handle *tep)
-{
- if (tep)
- return (tep->file_bigendian == TEP_BIG_ENDIAN);
- return false;
-}
-
-/**
- * tep_set_file_bigendian - set if the file is in big endian order
- * @tep: a handle to the tep_handle
- * @endian: non zero, if the file is in big endian order
- *
- * This sets if the file is in big endian order
- */
-void tep_set_file_bigendian(struct tep_handle *tep, enum tep_endian endian)
-{
- if (tep)
- tep->file_bigendian = endian;
-}
-
-/**
- * tep_is_local_bigendian - return the endian of the saved local machine
- * @tep: a handle to the tep_handle
- *
- * This returns true if the saved local machine in @tep is big endian.
- * If @tep is NULL, false is returned.
- */
-bool tep_is_local_bigendian(struct tep_handle *tep)
-{
- if (tep)
- return (tep->host_bigendian == TEP_BIG_ENDIAN);
- return 0;
-}
-
-/**
- * tep_set_local_bigendian - set the stored local machine endian order
- * @tep: a handle to the tep_handle
- * @endian: non zero, if the local host has big endian order
- *
- * This sets the endian order for the local machine.
- */
-void tep_set_local_bigendian(struct tep_handle *tep, enum tep_endian endian)
-{
- if (tep)
- tep->host_bigendian = endian;
-}
-
-/**
- * tep_is_old_format - get if an old kernel is used
- * @tep: a handle to the tep_handle
- *
- * This returns true, if an old kernel is used to generate the tracing events or
- * false if a new kernel is used. Old kernels did not have header page info.
- * If @tep is NULL, false is returned.
- */
-bool tep_is_old_format(struct tep_handle *tep)
-{
- if (tep)
- return tep->old_format;
- return false;
-}
-
-/**
- * tep_set_test_filters - set a flag to test a filter string
- * @tep: a handle to the tep_handle
- * @test_filters: the new value of the test_filters flag
- *
- * This sets a flag to test a filter string. If this flag is set, when
- * tep_filter_add_filter_str() API as called,it will print the filter string
- * instead of adding it.
- */
-void tep_set_test_filters(struct tep_handle *tep, int test_filters)
-{
- if (tep)
- tep->test_filters = test_filters;
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-
-#ifndef _PARSE_EVENTS_INT_H
-#define _PARSE_EVENTS_INT_H
-
-struct tep_cmdline;
-struct cmdline_list;
-struct func_map;
-struct func_list;
-struct event_handler;
-struct func_resolver;
-struct tep_plugins_dir;
-
-#define __hidden __attribute__((visibility ("hidden")))
-
-struct tep_handle {
- int ref_count;
-
- int header_page_ts_offset;
- int header_page_ts_size;
- int header_page_size_offset;
- int header_page_size_size;
- int header_page_data_offset;
- int header_page_data_size;
- int header_page_overwrite;
-
- enum tep_endian file_bigendian;
- enum tep_endian host_bigendian;
-
- int old_format;
-
- int cpus;
- int long_size;
- int page_size;
-
- struct tep_cmdline *cmdlines;
- struct cmdline_list *cmdlist;
- int cmdline_count;
-
- struct func_map *func_map;
- struct func_resolver *func_resolver;
- struct func_list *funclist;
- unsigned int func_count;
-
- struct printk_map *printk_map;
- struct printk_list *printklist;
- unsigned int printk_count;
-
- struct tep_event **events;
- int nr_events;
- struct tep_event **sort_events;
- enum tep_event_sort_type last_type;
-
- int type_offset;
- int type_size;
-
- int pid_offset;
- int pid_size;
-
- int pc_offset;
- int pc_size;
-
- int flags_offset;
- int flags_size;
-
- int ld_offset;
- int ld_size;
-
- int test_filters;
-
- int flags;
-
- struct tep_format_field *bprint_ip_field;
- struct tep_format_field *bprint_fmt_field;
- struct tep_format_field *bprint_buf_field;
-
- struct event_handler *handlers;
- struct tep_function_handler *func_handlers;
-
- /* cache */
- struct tep_event *last_event;
-
- struct tep_plugins_dir *plugins_dir;
-};
-
-enum tep_print_parse_type {
- PRINT_FMT_STRING,
- PRINT_FMT_ARG_DIGIT,
- PRINT_FMT_ARG_POINTER,
- PRINT_FMT_ARG_STRING,
-};
-
-struct tep_print_parse {
- struct tep_print_parse *next;
-
- char *format;
- int ls;
- enum tep_print_parse_type type;
- struct tep_print_arg *arg;
- struct tep_print_arg *len_as_arg;
-};
-
-void free_tep_event(struct tep_event *event);
-void free_tep_format_field(struct tep_format_field *field);
-void free_tep_plugin_paths(struct tep_handle *tep);
-
-unsigned short data2host2(struct tep_handle *tep, unsigned short data);
-unsigned int data2host4(struct tep_handle *tep, unsigned int data);
-unsigned long long data2host8(struct tep_handle *tep, unsigned long long data);
-
-/* access to the internal parser */
-int peek_char(void);
-void init_input_buf(const char *buf, unsigned long long size);
-unsigned long long get_input_buf_ptr(void);
-const char *get_input_buf(void);
-enum tep_event_type read_token(char **tok);
-void free_token(char *tok);
-
-#endif /* _PARSE_EVENTS_INT_H */
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- *
- * The parts for function graph printing was taken and modified from the
- * Linux Kernel that were written by
- * - Copyright (C) 2009 Frederic Weisbecker,
- * Frederic Weisbecker gave his permission to relicense the code to
- * the Lesser General Public License.
- */
-#include <inttypes.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <stdarg.h>
-#include <ctype.h>
-#include <errno.h>
-#include <stdint.h>
-#include <limits.h>
-#include <linux/time64.h>
-
-#include <netinet/in.h>
-#include "event-parse.h"
-
-#include "event-parse-local.h"
-#include "event-utils.h"
-#include "trace-seq.h"
-
-static const char *input_buf;
-static unsigned long long input_buf_ptr;
-static unsigned long long input_buf_siz;
-
-static int is_flag_field;
-static int is_symbolic_field;
-
-static int show_warning = 1;
-
-#define do_warning(fmt, ...) \
- do { \
- if (show_warning) \
- warning(fmt, ##__VA_ARGS__); \
- } while (0)
-
-#define do_warning_event(event, fmt, ...) \
- do { \
- if (!show_warning) \
- continue; \
- \
- if (event) \
- warning("[%s:%s] " fmt, event->system, \
- event->name, ##__VA_ARGS__); \
- else \
- warning(fmt, ##__VA_ARGS__); \
- } while (0)
-
-/**
- * init_input_buf - init buffer for parsing
- * @buf: buffer to parse
- * @size: the size of the buffer
- *
- * Initializes the internal buffer that tep_read_token() will parse.
- */
-__hidden void init_input_buf(const char *buf, unsigned long long size)
-{
- input_buf = buf;
- input_buf_siz = size;
- input_buf_ptr = 0;
-}
-
-__hidden const char *get_input_buf(void)
-{
- return input_buf;
-}
-
-__hidden unsigned long long get_input_buf_ptr(void)
-{
- return input_buf_ptr;
-}
-
-struct event_handler {
- struct event_handler *next;
- int id;
- const char *sys_name;
- const char *event_name;
- tep_event_handler_func func;
- void *context;
-};
-
-struct func_params {
- struct func_params *next;
- enum tep_func_arg_type type;
-};
-
-struct tep_function_handler {
- struct tep_function_handler *next;
- enum tep_func_arg_type ret_type;
- char *name;
- tep_func_handler func;
- struct func_params *params;
- int nr_args;
-};
-
-static unsigned long long
-process_defined_func(struct trace_seq *s, void *data, int size,
- struct tep_event *event, struct tep_print_arg *arg);
-
-static void free_func_handle(struct tep_function_handler *func);
-
-void breakpoint(void)
-{
- static int x;
- x++;
-}
-
-static struct tep_print_arg *alloc_arg(void)
-{
- return calloc(1, sizeof(struct tep_print_arg));
-}
-
-struct tep_cmdline {
- char *comm;
- int pid;
-};
-
-static int cmdline_cmp(const void *a, const void *b)
-{
- const struct tep_cmdline *ca = a;
- const struct tep_cmdline *cb = b;
-
- if (ca->pid < cb->pid)
- return -1;
- if (ca->pid > cb->pid)
- return 1;
-
- return 0;
-}
-
-/* Looking for where to place the key */
-static int cmdline_slot_cmp(const void *a, const void *b)
-{
- const struct tep_cmdline *ca = a;
- const struct tep_cmdline *cb = b;
- const struct tep_cmdline *cb1 = cb + 1;
-
- if (ca->pid < cb->pid)
- return -1;
-
- if (ca->pid > cb->pid) {
- if (ca->pid <= cb1->pid)
- return 0;
- return 1;
- }
-
- return 0;
-}
-
-struct cmdline_list {
- struct cmdline_list *next;
- char *comm;
- int pid;
-};
-
-static int cmdline_init(struct tep_handle *tep)
-{
- struct cmdline_list *cmdlist = tep->cmdlist;
- struct cmdline_list *item;
- struct tep_cmdline *cmdlines;
- int i;
-
- cmdlines = malloc(sizeof(*cmdlines) * tep->cmdline_count);
- if (!cmdlines)
- return -1;
-
- i = 0;
- while (cmdlist) {
- cmdlines[i].pid = cmdlist->pid;
- cmdlines[i].comm = cmdlist->comm;
- i++;
- item = cmdlist;
- cmdlist = cmdlist->next;
- free(item);
- }
-
- qsort(cmdlines, tep->cmdline_count, sizeof(*cmdlines), cmdline_cmp);
-
- tep->cmdlines = cmdlines;
- tep->cmdlist = NULL;
-
- return 0;
-}
-
-static const char *find_cmdline(struct tep_handle *tep, int pid)
-{
- const struct tep_cmdline *comm;
- struct tep_cmdline key;
-
- if (!pid)
- return "<idle>";
-
- if (!tep->cmdlines && cmdline_init(tep))
- return "<not enough memory for cmdlines!>";
-
- key.pid = pid;
-
- comm = bsearch(&key, tep->cmdlines, tep->cmdline_count,
- sizeof(*tep->cmdlines), cmdline_cmp);
-
- if (comm)
- return comm->comm;
- return "<...>";
-}
-
-/**
- * tep_is_pid_registered - return if a pid has a cmdline registered
- * @tep: a handle to the trace event parser context
- * @pid: The pid to check if it has a cmdline registered with.
- *
- * Returns true if the pid has a cmdline mapped to it
- * false otherwise.
- */
-bool tep_is_pid_registered(struct tep_handle *tep, int pid)
-{
- const struct tep_cmdline *comm;
- struct tep_cmdline key;
-
- if (!pid)
- return true;
-
- if (!tep->cmdlines && cmdline_init(tep))
- return false;
-
- key.pid = pid;
-
- comm = bsearch(&key, tep->cmdlines, tep->cmdline_count,
- sizeof(*tep->cmdlines), cmdline_cmp);
-
- if (comm)
- return true;
- return false;
-}
-
-/*
- * If the command lines have been converted to an array, then
- * we must add this pid. This is much slower than when cmdlines
- * are added before the array is initialized.
- */
-static int add_new_comm(struct tep_handle *tep,
- const char *comm, int pid, bool override)
-{
- struct tep_cmdline *cmdlines = tep->cmdlines;
- struct tep_cmdline *cmdline;
- struct tep_cmdline key;
- char *new_comm;
- int cnt;
-
- if (!pid)
- return 0;
-
- /* avoid duplicates */
- key.pid = pid;
-
- cmdline = bsearch(&key, tep->cmdlines, tep->cmdline_count,
- sizeof(*tep->cmdlines), cmdline_cmp);
- if (cmdline) {
- if (!override) {
- errno = EEXIST;
- return -1;
- }
- new_comm = strdup(comm);
- if (!new_comm) {
- errno = ENOMEM;
- return -1;
- }
- free(cmdline->comm);
- cmdline->comm = new_comm;
-
- return 0;
- }
-
- cmdlines = realloc(cmdlines, sizeof(*cmdlines) * (tep->cmdline_count + 1));
- if (!cmdlines) {
- errno = ENOMEM;
- return -1;
- }
- tep->cmdlines = cmdlines;
-
- key.comm = strdup(comm);
- if (!key.comm) {
- errno = ENOMEM;
- return -1;
- }
-
- if (!tep->cmdline_count) {
- /* no entries yet */
- tep->cmdlines[0] = key;
- tep->cmdline_count++;
- return 0;
- }
-
- /* Now find where we want to store the new cmdline */
- cmdline = bsearch(&key, tep->cmdlines, tep->cmdline_count - 1,
- sizeof(*tep->cmdlines), cmdline_slot_cmp);
-
- cnt = tep->cmdline_count;
- if (cmdline) {
- /* cmdline points to the one before the spot we want */
- cmdline++;
- cnt -= cmdline - tep->cmdlines;
-
- } else {
- /* The new entry is either before or after the list */
- if (key.pid > tep->cmdlines[tep->cmdline_count - 1].pid) {
- tep->cmdlines[tep->cmdline_count++] = key;
- return 0;
- }
- cmdline = &tep->cmdlines[0];
- }
- memmove(cmdline + 1, cmdline, (cnt * sizeof(*cmdline)));
- *cmdline = key;
-
- tep->cmdline_count++;
-
- return 0;
-}
-
-static int _tep_register_comm(struct tep_handle *tep,
- const char *comm, int pid, bool override)
-{
- struct cmdline_list *item;
-
- if (tep->cmdlines)
- return add_new_comm(tep, comm, pid, override);
-
- item = malloc(sizeof(*item));
- if (!item)
- return -1;
-
- if (comm)
- item->comm = strdup(comm);
- else
- item->comm = strdup("<...>");
- if (!item->comm) {
- free(item);
- return -1;
- }
- item->pid = pid;
- item->next = tep->cmdlist;
-
- tep->cmdlist = item;
- tep->cmdline_count++;
-
- return 0;
-}
-
-/**
- * tep_register_comm - register a pid / comm mapping
- * @tep: a handle to the trace event parser context
- * @comm: the command line to register
- * @pid: the pid to map the command line to
- *
- * This adds a mapping to search for command line names with
- * a given pid. The comm is duplicated. If a command with the same pid
- * already exist, -1 is returned and errno is set to EEXIST
- */
-int tep_register_comm(struct tep_handle *tep, const char *comm, int pid)
-{
- return _tep_register_comm(tep, comm, pid, false);
-}
-
-/**
- * tep_override_comm - register a pid / comm mapping
- * @tep: a handle to the trace event parser context
- * @comm: the command line to register
- * @pid: the pid to map the command line to
- *
- * This adds a mapping to search for command line names with
- * a given pid. The comm is duplicated. If a command with the same pid
- * already exist, the command string is udapted with the new one
- */
-int tep_override_comm(struct tep_handle *tep, const char *comm, int pid)
-{
- if (!tep->cmdlines && cmdline_init(tep)) {
- errno = ENOMEM;
- return -1;
- }
- return _tep_register_comm(tep, comm, pid, true);
-}
-
-struct func_map {
- unsigned long long addr;
- char *func;
- char *mod;
-};
-
-struct func_list {
- struct func_list *next;
- unsigned long long addr;
- char *func;
- char *mod;
-};
-
-static int func_cmp(const void *a, const void *b)
-{
- const struct func_map *fa = a;
- const struct func_map *fb = b;
-
- if (fa->addr < fb->addr)
- return -1;
- if (fa->addr > fb->addr)
- return 1;
-
- return 0;
-}
-
-/*
- * We are searching for a record in between, not an exact
- * match.
- */
-static int func_bcmp(const void *a, const void *b)
-{
- const struct func_map *fa = a;
- const struct func_map *fb = b;
-
- if ((fa->addr == fb->addr) ||
-
- (fa->addr > fb->addr &&
- fa->addr < (fb+1)->addr))
- return 0;
-
- if (fa->addr < fb->addr)
- return -1;
-
- return 1;
-}
-
-static int func_map_init(struct tep_handle *tep)
-{
- struct func_list *funclist;
- struct func_list *item;
- struct func_map *func_map;
- int i;
-
- func_map = malloc(sizeof(*func_map) * (tep->func_count + 1));
- if (!func_map)
- return -1;
-
- funclist = tep->funclist;
-
- i = 0;
- while (funclist) {
- func_map[i].func = funclist->func;
- func_map[i].addr = funclist->addr;
- func_map[i].mod = funclist->mod;
- i++;
- item = funclist;
- funclist = funclist->next;
- free(item);
- }
-
- qsort(func_map, tep->func_count, sizeof(*func_map), func_cmp);
-
- /*
- * Add a special record at the end.
- */
- func_map[tep->func_count].func = NULL;
- func_map[tep->func_count].addr = 0;
- func_map[tep->func_count].mod = NULL;
-
- tep->func_map = func_map;
- tep->funclist = NULL;
-
- return 0;
-}
-
-static struct func_map *
-__find_func(struct tep_handle *tep, unsigned long long addr)
-{
- struct func_map *func;
- struct func_map key;
-
- if (!tep->func_map)
- func_map_init(tep);
-
- key.addr = addr;
-
- func = bsearch(&key, tep->func_map, tep->func_count,
- sizeof(*tep->func_map), func_bcmp);
-
- return func;
-}
-
-struct func_resolver {
- tep_func_resolver_t *func;
- void *priv;
- struct func_map map;
-};
-
-/**
- * tep_set_function_resolver - set an alternative function resolver
- * @tep: a handle to the trace event parser context
- * @resolver: function to be used
- * @priv: resolver function private state.
- *
- * Some tools may have already a way to resolve kernel functions, allow them to
- * keep using it instead of duplicating all the entries inside tep->funclist.
- */
-int tep_set_function_resolver(struct tep_handle *tep,
- tep_func_resolver_t *func, void *priv)
-{
- struct func_resolver *resolver = malloc(sizeof(*resolver));
-
- if (resolver == NULL)
- return -1;
-
- resolver->func = func;
- resolver->priv = priv;
-
- free(tep->func_resolver);
- tep->func_resolver = resolver;
-
- return 0;
-}
-
-/**
- * tep_reset_function_resolver - reset alternative function resolver
- * @tep: a handle to the trace event parser context
- *
- * Stop using whatever alternative resolver was set, use the default
- * one instead.
- */
-void tep_reset_function_resolver(struct tep_handle *tep)
-{
- free(tep->func_resolver);
- tep->func_resolver = NULL;
-}
-
-static struct func_map *
-find_func(struct tep_handle *tep, unsigned long long addr)
-{
- struct func_map *map;
-
- if (!tep->func_resolver)
- return __find_func(tep, addr);
-
- map = &tep->func_resolver->map;
- map->mod = NULL;
- map->addr = addr;
- map->func = tep->func_resolver->func(tep->func_resolver->priv,
- &map->addr, &map->mod);
- if (map->func == NULL)
- return NULL;
-
- return map;
-}
-
-/**
- * tep_find_function - find a function by a given address
- * @tep: a handle to the trace event parser context
- * @addr: the address to find the function with
- *
- * Returns a pointer to the function stored that has the given
- * address. Note, the address does not have to be exact, it
- * will select the function that would contain the address.
- */
-const char *tep_find_function(struct tep_handle *tep, unsigned long long addr)
-{
- struct func_map *map;
-
- map = find_func(tep, addr);
- if (!map)
- return NULL;
-
- return map->func;
-}
-
-/**
- * tep_find_function_address - find a function address by a given address
- * @tep: a handle to the trace event parser context
- * @addr: the address to find the function with
- *
- * Returns the address the function starts at. This can be used in
- * conjunction with tep_find_function to print both the function
- * name and the function offset.
- */
-unsigned long long
-tep_find_function_address(struct tep_handle *tep, unsigned long long addr)
-{
- struct func_map *map;
-
- map = find_func(tep, addr);
- if (!map)
- return 0;
-
- return map->addr;
-}
-
-/**
- * tep_register_function - register a function with a given address
- * @tep: a handle to the trace event parser context
- * @function: the function name to register
- * @addr: the address the function starts at
- * @mod: the kernel module the function may be in (NULL for none)
- *
- * This registers a function name with an address and module.
- * The @func passed in is duplicated.
- */
-int tep_register_function(struct tep_handle *tep, char *func,
- unsigned long long addr, char *mod)
-{
- struct func_list *item = malloc(sizeof(*item));
-
- if (!item)
- return -1;
-
- item->next = tep->funclist;
- item->func = strdup(func);
- if (!item->func)
- goto out_free;
-
- if (mod) {
- item->mod = strdup(mod);
- if (!item->mod)
- goto out_free_func;
- } else
- item->mod = NULL;
- item->addr = addr;
-
- tep->funclist = item;
- tep->func_count++;
-
- return 0;
-
-out_free_func:
- free(item->func);
- item->func = NULL;
-out_free:
- free(item);
- errno = ENOMEM;
- return -1;
-}
-
-/**
- * tep_print_funcs - print out the stored functions
- * @tep: a handle to the trace event parser context
- *
- * This prints out the stored functions.
- */
-void tep_print_funcs(struct tep_handle *tep)
-{
- int i;
-
- if (!tep->func_map)
- func_map_init(tep);
-
- for (i = 0; i < (int)tep->func_count; i++) {
- printf("%016llx %s",
- tep->func_map[i].addr,
- tep->func_map[i].func);
- if (tep->func_map[i].mod)
- printf(" [%s]\n", tep->func_map[i].mod);
- else
- printf("\n");
- }
-}
-
-struct printk_map {
- unsigned long long addr;
- char *printk;
-};
-
-struct printk_list {
- struct printk_list *next;
- unsigned long long addr;
- char *printk;
-};
-
-static int printk_cmp(const void *a, const void *b)
-{
- const struct printk_map *pa = a;
- const struct printk_map *pb = b;
-
- if (pa->addr < pb->addr)
- return -1;
- if (pa->addr > pb->addr)
- return 1;
-
- return 0;
-}
-
-static int printk_map_init(struct tep_handle *tep)
-{
- struct printk_list *printklist;
- struct printk_list *item;
- struct printk_map *printk_map;
- int i;
-
- printk_map = malloc(sizeof(*printk_map) * (tep->printk_count + 1));
- if (!printk_map)
- return -1;
-
- printklist = tep->printklist;
-
- i = 0;
- while (printklist) {
- printk_map[i].printk = printklist->printk;
- printk_map[i].addr = printklist->addr;
- i++;
- item = printklist;
- printklist = printklist->next;
- free(item);
- }
-
- qsort(printk_map, tep->printk_count, sizeof(*printk_map), printk_cmp);
-
- tep->printk_map = printk_map;
- tep->printklist = NULL;
-
- return 0;
-}
-
-static struct printk_map *
-find_printk(struct tep_handle *tep, unsigned long long addr)
-{
- struct printk_map *printk;
- struct printk_map key;
-
- if (!tep->printk_map && printk_map_init(tep))
- return NULL;
-
- key.addr = addr;
-
- printk = bsearch(&key, tep->printk_map, tep->printk_count,
- sizeof(*tep->printk_map), printk_cmp);
-
- return printk;
-}
-
-/**
- * tep_register_print_string - register a string by its address
- * @tep: a handle to the trace event parser context
- * @fmt: the string format to register
- * @addr: the address the string was located at
- *
- * This registers a string by the address it was stored in the kernel.
- * The @fmt passed in is duplicated.
- */
-int tep_register_print_string(struct tep_handle *tep, const char *fmt,
- unsigned long long addr)
-{
- struct printk_list *item = malloc(sizeof(*item));
- char *p;
-
- if (!item)
- return -1;
-
- item->next = tep->printklist;
- item->addr = addr;
-
- /* Strip off quotes and '\n' from the end */
- if (fmt[0] == '"')
- fmt++;
- item->printk = strdup(fmt);
- if (!item->printk)
- goto out_free;
-
- p = item->printk + strlen(item->printk) - 1;
- if (*p == '"')
- *p = 0;
-
- p -= 2;
- if (strcmp(p, "\\n") == 0)
- *p = 0;
-
- tep->printklist = item;
- tep->printk_count++;
-
- return 0;
-
-out_free:
- free(item);
- errno = ENOMEM;
- return -1;
-}
-
-/**
- * tep_print_printk - print out the stored strings
- * @tep: a handle to the trace event parser context
- *
- * This prints the string formats that were stored.
- */
-void tep_print_printk(struct tep_handle *tep)
-{
- int i;
-
- if (!tep->printk_map)
- printk_map_init(tep);
-
- for (i = 0; i < (int)tep->printk_count; i++) {
- printf("%016llx %s\n",
- tep->printk_map[i].addr,
- tep->printk_map[i].printk);
- }
-}
-
-static struct tep_event *alloc_event(void)
-{
- return calloc(1, sizeof(struct tep_event));
-}
-
-static int add_event(struct tep_handle *tep, struct tep_event *event)
-{
- int i;
- struct tep_event **events = realloc(tep->events, sizeof(event) *
- (tep->nr_events + 1));
- if (!events)
- return -1;
-
- tep->events = events;
-
- for (i = 0; i < tep->nr_events; i++) {
- if (tep->events[i]->id > event->id)
- break;
- }
- if (i < tep->nr_events)
- memmove(&tep->events[i + 1],
- &tep->events[i],
- sizeof(event) * (tep->nr_events - i));
-
- tep->events[i] = event;
- tep->nr_events++;
-
- event->tep = tep;
-
- return 0;
-}
-
-static int event_item_type(enum tep_event_type type)
-{
- switch (type) {
- case TEP_EVENT_ITEM ... TEP_EVENT_SQUOTE:
- return 1;
- case TEP_EVENT_ERROR ... TEP_EVENT_DELIM:
- default:
- return 0;
- }
-}
-
-static void free_flag_sym(struct tep_print_flag_sym *fsym)
-{
- struct tep_print_flag_sym *next;
-
- while (fsym) {
- next = fsym->next;
- free(fsym->value);
- free(fsym->str);
- free(fsym);
- fsym = next;
- }
-}
-
-static void free_arg(struct tep_print_arg *arg)
-{
- struct tep_print_arg *farg;
-
- if (!arg)
- return;
-
- switch (arg->type) {
- case TEP_PRINT_ATOM:
- free(arg->atom.atom);
- break;
- case TEP_PRINT_FIELD:
- free(arg->field.name);
- break;
- case TEP_PRINT_FLAGS:
- free_arg(arg->flags.field);
- free(arg->flags.delim);
- free_flag_sym(arg->flags.flags);
- break;
- case TEP_PRINT_SYMBOL:
- free_arg(arg->symbol.field);
- free_flag_sym(arg->symbol.symbols);
- break;
- case TEP_PRINT_HEX:
- case TEP_PRINT_HEX_STR:
- free_arg(arg->hex.field);
- free_arg(arg->hex.size);
- break;
- case TEP_PRINT_INT_ARRAY:
- free_arg(arg->int_array.field);
- free_arg(arg->int_array.count);
- free_arg(arg->int_array.el_size);
- break;
- case TEP_PRINT_TYPE:
- free(arg->typecast.type);
- free_arg(arg->typecast.item);
- break;
- case TEP_PRINT_STRING:
- case TEP_PRINT_BSTRING:
- free(arg->string.string);
- break;
- case TEP_PRINT_BITMASK:
- free(arg->bitmask.bitmask);
- break;
- case TEP_PRINT_DYNAMIC_ARRAY:
- case TEP_PRINT_DYNAMIC_ARRAY_LEN:
- free(arg->dynarray.index);
- break;
- case TEP_PRINT_OP:
- free(arg->op.op);
- free_arg(arg->op.left);
- free_arg(arg->op.right);
- break;
- case TEP_PRINT_FUNC:
- while (arg->func.args) {
- farg = arg->func.args;
- arg->func.args = farg->next;
- free_arg(farg);
- }
- break;
-
- case TEP_PRINT_NULL:
- default:
- break;
- }
-
- free(arg);
-}
-
-static enum tep_event_type get_type(int ch)
-{
- if (ch == '\n')
- return TEP_EVENT_NEWLINE;
- if (isspace(ch))
- return TEP_EVENT_SPACE;
- if (isalnum(ch) || ch == '_')
- return TEP_EVENT_ITEM;
- if (ch == '\'')
- return TEP_EVENT_SQUOTE;
- if (ch == '"')
- return TEP_EVENT_DQUOTE;
- if (!isprint(ch))
- return TEP_EVENT_NONE;
- if (ch == '(' || ch == ')' || ch == ',')
- return TEP_EVENT_DELIM;
-
- return TEP_EVENT_OP;
-}
-
-static int __read_char(void)
-{
- if (input_buf_ptr >= input_buf_siz)
- return -1;
-
- return input_buf[input_buf_ptr++];
-}
-
-/**
- * peek_char - peek at the next character that will be read
- *
- * Returns the next character read, or -1 if end of buffer.
- */
-__hidden int peek_char(void)
-{
- if (input_buf_ptr >= input_buf_siz)
- return -1;
-
- return input_buf[input_buf_ptr];
-}
-
-static int extend_token(char **tok, char *buf, int size)
-{
- char *newtok = realloc(*tok, size);
-
- if (!newtok) {
- free(*tok);
- *tok = NULL;
- return -1;
- }
-
- if (!*tok)
- strcpy(newtok, buf);
- else
- strcat(newtok, buf);
- *tok = newtok;
-
- return 0;
-}
-
-static enum tep_event_type force_token(const char *str, char **tok);
-
-static enum tep_event_type __read_token(char **tok)
-{
- char buf[BUFSIZ];
- int ch, last_ch, quote_ch, next_ch;
- int i = 0;
- int tok_size = 0;
- enum tep_event_type type;
-
- *tok = NULL;
-
-
- ch = __read_char();
- if (ch < 0)
- return TEP_EVENT_NONE;
-
- type = get_type(ch);
- if (type == TEP_EVENT_NONE)
- return type;
-
- buf[i++] = ch;
-
- switch (type) {
- case TEP_EVENT_NEWLINE:
- case TEP_EVENT_DELIM:
- if (asprintf(tok, "%c", ch) < 0)
- return TEP_EVENT_ERROR;
-
- return type;
-
- case TEP_EVENT_OP:
- switch (ch) {
- case '-':
- next_ch = peek_char();
- if (next_ch == '>') {
- buf[i++] = __read_char();
- break;
- }
- /* fall through */
- case '+':
- case '|':
- case '&':
- case '>':
- case '<':
- last_ch = ch;
- ch = peek_char();
- if (ch != last_ch)
- goto test_equal;
- buf[i++] = __read_char();
- switch (last_ch) {
- case '>':
- case '<':
- goto test_equal;
- default:
- break;
- }
- break;
- case '!':
- case '=':
- goto test_equal;
- default: /* what should we do instead? */
- break;
- }
- buf[i] = 0;
- *tok = strdup(buf);
- return type;
-
- test_equal:
- ch = peek_char();
- if (ch == '=')
- buf[i++] = __read_char();
- goto out;
-
- case TEP_EVENT_DQUOTE:
- case TEP_EVENT_SQUOTE:
- /* don't keep quotes */
- i--;
- quote_ch = ch;
- last_ch = 0;
- concat:
- do {
- if (i == (BUFSIZ - 1)) {
- buf[i] = 0;
- tok_size += BUFSIZ;
-
- if (extend_token(tok, buf, tok_size) < 0)
- return TEP_EVENT_NONE;
- i = 0;
- }
- last_ch = ch;
- ch = __read_char();
- buf[i++] = ch;
- /* the '\' '\' will cancel itself */
- if (ch == '\\' && last_ch == '\\')
- last_ch = 0;
- } while (ch != quote_ch || last_ch == '\\');
- /* remove the last quote */
- i--;
-
- /*
- * For strings (double quotes) check the next token.
- * If it is another string, concatinate the two.
- */
- if (type == TEP_EVENT_DQUOTE) {
- unsigned long long save_input_buf_ptr = input_buf_ptr;
-
- do {
- ch = __read_char();
- } while (isspace(ch));
- if (ch == '"')
- goto concat;
- input_buf_ptr = save_input_buf_ptr;
- }
-
- goto out;
-
- case TEP_EVENT_ERROR ... TEP_EVENT_SPACE:
- case TEP_EVENT_ITEM:
- default:
- break;
- }
-
- while (get_type(peek_char()) == type) {
- if (i == (BUFSIZ - 1)) {
- buf[i] = 0;
- tok_size += BUFSIZ;
-
- if (extend_token(tok, buf, tok_size) < 0)
- return TEP_EVENT_NONE;
- i = 0;
- }
- ch = __read_char();
- buf[i++] = ch;
- }
-
- out:
- buf[i] = 0;
- if (extend_token(tok, buf, tok_size + i + 1) < 0)
- return TEP_EVENT_NONE;
-
- if (type == TEP_EVENT_ITEM) {
- /*
- * Older versions of the kernel has a bug that
- * creates invalid symbols and will break the mac80211
- * parsing. This is a work around to that bug.
- *
- * See Linux kernel commit:
- * 811cb50baf63461ce0bdb234927046131fc7fa8b
- */
- if (strcmp(*tok, "LOCAL_PR_FMT") == 0) {
- free(*tok);
- *tok = NULL;
- return force_token("\"%s\" ", tok);
- } else if (strcmp(*tok, "STA_PR_FMT") == 0) {
- free(*tok);
- *tok = NULL;
- return force_token("\" sta:%pM\" ", tok);
- } else if (strcmp(*tok, "VIF_PR_FMT") == 0) {
- free(*tok);
- *tok = NULL;
- return force_token("\" vif:%p(%d)\" ", tok);
- }
- }
-
- return type;
-}
-
-static enum tep_event_type force_token(const char *str, char **tok)
-{
- const char *save_input_buf;
- unsigned long long save_input_buf_ptr;
- unsigned long long save_input_buf_siz;
- enum tep_event_type type;
-
- /* save off the current input pointers */
- save_input_buf = input_buf;
- save_input_buf_ptr = input_buf_ptr;
- save_input_buf_siz = input_buf_siz;
-
- init_input_buf(str, strlen(str));
-
- type = __read_token(tok);
-
- /* reset back to original token */
- input_buf = save_input_buf;
- input_buf_ptr = save_input_buf_ptr;
- input_buf_siz = save_input_buf_siz;
-
- return type;
-}
-
-/**
- * free_token - free a token returned by tep_read_token
- * @token: the token to free
- */
-__hidden void free_token(char *tok)
-{
- if (tok)
- free(tok);
-}
-
-/**
- * read_token - access to utilities to use the tep parser
- * @tok: The token to return
- *
- * This will parse tokens from the string given by
- * tep_init_data().
- *
- * Returns the token type.
- */
-__hidden enum tep_event_type read_token(char **tok)
-{
- enum tep_event_type type;
-
- for (;;) {
- type = __read_token(tok);
- if (type != TEP_EVENT_SPACE)
- return type;
-
- free_token(*tok);
- }
-
- /* not reached */
- *tok = NULL;
- return TEP_EVENT_NONE;
-}
-
-/* no newline */
-static enum tep_event_type read_token_item(char **tok)
-{
- enum tep_event_type type;
-
- for (;;) {
- type = __read_token(tok);
- if (type != TEP_EVENT_SPACE && type != TEP_EVENT_NEWLINE)
- return type;
- free_token(*tok);
- *tok = NULL;
- }
-
- /* not reached */
- *tok = NULL;
- return TEP_EVENT_NONE;
-}
-
-static int test_type(enum tep_event_type type, enum tep_event_type expect)
-{
- if (type != expect) {
- do_warning("Error: expected type %d but read %d",
- expect, type);
- return -1;
- }
- return 0;
-}
-
-static int test_type_token(enum tep_event_type type, const char *token,
- enum tep_event_type expect, const char *expect_tok)
-{
- if (type != expect) {
- do_warning("Error: expected type %d but read %d",
- expect, type);
- return -1;
- }
-
- if (strcmp(token, expect_tok) != 0) {
- do_warning("Error: expected '%s' but read '%s'",
- expect_tok, token);
- return -1;
- }
- return 0;
-}
-
-static int __read_expect_type(enum tep_event_type expect, char **tok, int newline_ok)
-{
- enum tep_event_type type;
-
- if (newline_ok)
- type = read_token(tok);
- else
- type = read_token_item(tok);
- return test_type(type, expect);
-}
-
-static int read_expect_type(enum tep_event_type expect, char **tok)
-{
- return __read_expect_type(expect, tok, 1);
-}
-
-static int __read_expected(enum tep_event_type expect, const char *str,
- int newline_ok)
-{
- enum tep_event_type type;
- char *token;
- int ret;
-
- if (newline_ok)
- type = read_token(&token);
- else
- type = read_token_item(&token);
-
- ret = test_type_token(type, token, expect, str);
-
- free_token(token);
-
- return ret;
-}
-
-static int read_expected(enum tep_event_type expect, const char *str)
-{
- return __read_expected(expect, str, 1);
-}
-
-static int read_expected_item(enum tep_event_type expect, const char *str)
-{
- return __read_expected(expect, str, 0);
-}
-
-static char *event_read_name(void)
-{
- char *token;
-
- if (read_expected(TEP_EVENT_ITEM, "name") < 0)
- return NULL;
-
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- return NULL;
-
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto fail;
-
- return token;
-
- fail:
- free_token(token);
- return NULL;
-}
-
-static int event_read_id(void)
-{
- char *token;
- int id;
-
- if (read_expected_item(TEP_EVENT_ITEM, "ID") < 0)
- return -1;
-
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- return -1;
-
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto fail;
-
- id = strtoul(token, NULL, 0);
- free_token(token);
- return id;
-
- fail:
- free_token(token);
- return -1;
-}
-
-static int field_is_string(struct tep_format_field *field)
-{
- if ((field->flags & TEP_FIELD_IS_ARRAY) &&
- (strstr(field->type, "char") || strstr(field->type, "u8") ||
- strstr(field->type, "s8")))
- return 1;
-
- return 0;
-}
-
-static int field_is_dynamic(struct tep_format_field *field)
-{
- if (strncmp(field->type, "__data_loc", 10) == 0)
- return 1;
-
- return 0;
-}
-
-static int field_is_relative_dynamic(struct tep_format_field *field)
-{
- if (strncmp(field->type, "__rel_loc", 9) == 0)
- return 1;
-
- return 0;
-}
-
-static int field_is_long(struct tep_format_field *field)
-{
- /* includes long long */
- if (strstr(field->type, "long"))
- return 1;
-
- return 0;
-}
-
-static unsigned int type_size(const char *name)
-{
- /* This covers all TEP_FIELD_IS_STRING types. */
- static struct {
- const char *type;
- unsigned int size;
- } table[] = {
- { "u8", 1 },
- { "u16", 2 },
- { "u32", 4 },
- { "u64", 8 },
- { "s8", 1 },
- { "s16", 2 },
- { "s32", 4 },
- { "s64", 8 },
- { "char", 1 },
- { },
- };
- int i;
-
- for (i = 0; table[i].type; i++) {
- if (!strcmp(table[i].type, name))
- return table[i].size;
- }
-
- return 0;
-}
-
-static int append(char **buf, const char *delim, const char *str)
-{
- char *new_buf;
-
- new_buf = realloc(*buf, strlen(*buf) + strlen(delim) + strlen(str) + 1);
- if (!new_buf)
- return -1;
- strcat(new_buf, delim);
- strcat(new_buf, str);
- *buf = new_buf;
- return 0;
-}
-
-static int event_read_fields(struct tep_event *event, struct tep_format_field **fields)
-{
- struct tep_format_field *field = NULL;
- enum tep_event_type type;
- char *token;
- char *last_token;
- char *delim = " ";
- int count = 0;
- int ret;
-
- do {
- unsigned int size_dynamic = 0;
-
- type = read_token(&token);
- if (type == TEP_EVENT_NEWLINE) {
- free_token(token);
- return count;
- }
-
- count++;
-
- if (test_type_token(type, token, TEP_EVENT_ITEM, "field"))
- goto fail;
- free_token(token);
-
- type = read_token(&token);
- /*
- * The ftrace fields may still use the "special" name.
- * Just ignore it.
- */
- if (event->flags & TEP_EVENT_FL_ISFTRACE &&
- type == TEP_EVENT_ITEM && strcmp(token, "special") == 0) {
- free_token(token);
- type = read_token(&token);
- }
-
- if (test_type_token(type, token, TEP_EVENT_OP, ":") < 0)
- goto fail;
-
- free_token(token);
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto fail;
-
- last_token = token;
-
- field = calloc(1, sizeof(*field));
- if (!field)
- goto fail;
-
- field->event = event;
-
- /* read the rest of the type */
- for (;;) {
- type = read_token(&token);
- if (type == TEP_EVENT_ITEM ||
- (type == TEP_EVENT_OP && strcmp(token, "*") == 0) ||
- /*
- * Some of the ftrace fields are broken and have
- * an illegal "." in them.
- */
- (event->flags & TEP_EVENT_FL_ISFTRACE &&
- type == TEP_EVENT_OP && strcmp(token, ".") == 0)) {
-
- if (strcmp(token, "*") == 0)
- field->flags |= TEP_FIELD_IS_POINTER;
-
- if (field->type) {
- ret = append(&field->type, delim, last_token);
- free(last_token);
- if (ret < 0)
- goto fail;
- } else
- field->type = last_token;
- last_token = token;
- delim = " ";
- continue;
- }
-
- /* Handle __attribute__((user)) */
- if ((type == TEP_EVENT_DELIM) &&
- strcmp("__attribute__", last_token) == 0 &&
- token[0] == '(') {
- int depth = 1;
- int ret;
-
- ret = append(&field->type, " ", last_token);
- ret |= append(&field->type, "", "(");
- if (ret < 0)
- goto fail;
-
- delim = " ";
- while ((type = read_token(&token)) != TEP_EVENT_NONE) {
- if (type == TEP_EVENT_DELIM) {
- if (token[0] == '(')
- depth++;
- else if (token[0] == ')')
- depth--;
- if (!depth)
- break;
- ret = append(&field->type, "", token);
- delim = "";
- } else {
- ret = append(&field->type, delim, token);
- delim = " ";
- }
- if (ret < 0)
- goto fail;
- free(last_token);
- last_token = token;
- }
- continue;
- }
- break;
- }
-
- if (!field->type) {
- do_warning_event(event, "%s: no type found", __func__);
- goto fail;
- }
- field->name = field->alias = last_token;
-
- if (test_type(type, TEP_EVENT_OP))
- goto fail;
-
- if (strcmp(token, "[") == 0) {
- enum tep_event_type last_type = type;
- char *brackets = token;
-
- field->flags |= TEP_FIELD_IS_ARRAY;
-
- type = read_token(&token);
-
- if (type == TEP_EVENT_ITEM)
- field->arraylen = strtoul(token, NULL, 0);
- else
- field->arraylen = 0;
-
- while (strcmp(token, "]") != 0) {
- const char *delim;
-
- if (last_type == TEP_EVENT_ITEM &&
- type == TEP_EVENT_ITEM)
- delim = " ";
- else
- delim = "";
-
- last_type = type;
-
- ret = append(&brackets, delim, token);
- if (ret < 0) {
- free(brackets);
- goto fail;
- }
- /* We only care about the last token */
- field->arraylen = strtoul(token, NULL, 0);
- free_token(token);
- type = read_token(&token);
- if (type == TEP_EVENT_NONE) {
- free(brackets);
- do_warning_event(event, "failed to find token");
- goto fail;
- }
- }
-
- free_token(token);
-
- ret = append(&brackets, "", "]");
- if (ret < 0) {
- free(brackets);
- goto fail;
- }
-
- /* add brackets to type */
-
- type = read_token(&token);
- /*
- * If the next token is not an OP, then it is of
- * the format: type [] item;
- */
- if (type == TEP_EVENT_ITEM) {
- ret = append(&field->type, " ", field->name);
- if (ret < 0) {
- free(brackets);
- goto fail;
- }
- ret = append(&field->type, "", brackets);
-
- size_dynamic = type_size(field->name);
- free_token(field->name);
- field->name = field->alias = token;
- type = read_token(&token);
- } else {
- ret = append(&field->type, "", brackets);
- if (ret < 0) {
- free(brackets);
- goto fail;
- }
- }
- free(brackets);
- }
-
- if (field_is_string(field))
- field->flags |= TEP_FIELD_IS_STRING;
- if (field_is_dynamic(field))
- field->flags |= TEP_FIELD_IS_DYNAMIC;
- if (field_is_relative_dynamic(field))
- field->flags |= TEP_FIELD_IS_DYNAMIC | TEP_FIELD_IS_RELATIVE;
- if (field_is_long(field))
- field->flags |= TEP_FIELD_IS_LONG;
-
- if (test_type_token(type, token, TEP_EVENT_OP, ";"))
- goto fail;
- free_token(token);
-
- if (read_expected(TEP_EVENT_ITEM, "offset") < 0)
- goto fail_expect;
-
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- goto fail_expect;
-
- if (read_expect_type(TEP_EVENT_ITEM, &token))
- goto fail;
- field->offset = strtoul(token, NULL, 0);
- free_token(token);
-
- if (read_expected(TEP_EVENT_OP, ";") < 0)
- goto fail_expect;
-
- if (read_expected(TEP_EVENT_ITEM, "size") < 0)
- goto fail_expect;
-
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- goto fail_expect;
-
- if (read_expect_type(TEP_EVENT_ITEM, &token))
- goto fail;
- field->size = strtoul(token, NULL, 0);
- free_token(token);
-
- if (read_expected(TEP_EVENT_OP, ";") < 0)
- goto fail_expect;
-
- type = read_token(&token);
- if (type != TEP_EVENT_NEWLINE) {
- /* newer versions of the kernel have a "signed" type */
- if (test_type_token(type, token, TEP_EVENT_ITEM, "signed"))
- goto fail;
-
- free_token(token);
-
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- goto fail_expect;
-
- if (read_expect_type(TEP_EVENT_ITEM, &token))
- goto fail;
-
- if (strtoul(token, NULL, 0))
- field->flags |= TEP_FIELD_IS_SIGNED;
-
- free_token(token);
- if (read_expected(TEP_EVENT_OP, ";") < 0)
- goto fail_expect;
-
- if (read_expect_type(TEP_EVENT_NEWLINE, &token))
- goto fail;
- }
-
- free_token(token);
-
- if (field->flags & TEP_FIELD_IS_ARRAY) {
- if (field->arraylen)
- field->elementsize = field->size / field->arraylen;
- else if (field->flags & TEP_FIELD_IS_DYNAMIC)
- field->elementsize = size_dynamic;
- else if (field->flags & TEP_FIELD_IS_STRING)
- field->elementsize = 1;
- else if (field->flags & TEP_FIELD_IS_LONG)
- field->elementsize = event->tep ?
- event->tep->long_size :
- sizeof(long);
- } else
- field->elementsize = field->size;
-
- *fields = field;
- fields = &field->next;
-
- } while (1);
-
- return 0;
-
-fail:
- free_token(token);
-fail_expect:
- if (field) {
- free(field->type);
- free(field->name);
- free(field);
- }
- return -1;
-}
-
-static int event_read_format(struct tep_event *event)
-{
- char *token;
- int ret;
-
- if (read_expected_item(TEP_EVENT_ITEM, "format") < 0)
- return -1;
-
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- return -1;
-
- if (read_expect_type(TEP_EVENT_NEWLINE, &token))
- goto fail;
- free_token(token);
-
- ret = event_read_fields(event, &event->format.common_fields);
- if (ret < 0)
- return ret;
- event->format.nr_common = ret;
-
- ret = event_read_fields(event, &event->format.fields);
- if (ret < 0)
- return ret;
- event->format.nr_fields = ret;
-
- return 0;
-
- fail:
- free_token(token);
- return -1;
-}
-
-static enum tep_event_type
-process_arg_token(struct tep_event *event, struct tep_print_arg *arg,
- char **tok, enum tep_event_type type);
-
-static enum tep_event_type
-process_arg(struct tep_event *event, struct tep_print_arg *arg, char **tok)
-{
- enum tep_event_type type;
- char *token;
-
- type = read_token(&token);
- *tok = token;
-
- return process_arg_token(event, arg, tok, type);
-}
-
-static enum tep_event_type
-process_op(struct tep_event *event, struct tep_print_arg *arg, char **tok);
-
-/*
- * For __print_symbolic() and __print_flags, we need to completely
- * evaluate the first argument, which defines what to print next.
- */
-static enum tep_event_type
-process_field_arg(struct tep_event *event, struct tep_print_arg *arg, char **tok)
-{
- enum tep_event_type type;
-
- type = process_arg(event, arg, tok);
-
- while (type == TEP_EVENT_OP) {
- type = process_op(event, arg, tok);
- }
-
- return type;
-}
-
-static enum tep_event_type
-process_cond(struct tep_event *event, struct tep_print_arg *top, char **tok)
-{
- struct tep_print_arg *arg, *left, *right;
- enum tep_event_type type;
- char *token = NULL;
-
- arg = alloc_arg();
- left = alloc_arg();
- right = alloc_arg();
-
- if (!arg || !left || !right) {
- do_warning_event(event, "%s: not enough memory!", __func__);
- /* arg will be freed at out_free */
- free_arg(left);
- free_arg(right);
- goto out_free;
- }
-
- arg->type = TEP_PRINT_OP;
- arg->op.left = left;
- arg->op.right = right;
-
- *tok = NULL;
- type = process_arg(event, left, &token);
-
- again:
- if (type == TEP_EVENT_ERROR)
- goto out_free;
-
- /* Handle other operations in the arguments */
- if (type == TEP_EVENT_OP && strcmp(token, ":") != 0) {
- type = process_op(event, left, &token);
- goto again;
- }
-
- if (test_type_token(type, token, TEP_EVENT_OP, ":"))
- goto out_free;
-
- arg->op.op = token;
-
- type = process_arg(event, right, &token);
-
- top->op.right = arg;
-
- *tok = token;
- return type;
-
-out_free:
- /* Top may point to itself */
- top->op.right = NULL;
- free_token(token);
- free_arg(arg);
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_array(struct tep_event *event, struct tep_print_arg *top, char **tok)
-{
- struct tep_print_arg *arg;
- enum tep_event_type type;
- char *token = NULL;
-
- arg = alloc_arg();
- if (!arg) {
- do_warning_event(event, "%s: not enough memory!", __func__);
- /* '*tok' is set to top->op.op. No need to free. */
- *tok = NULL;
- return TEP_EVENT_ERROR;
- }
-
- *tok = NULL;
- type = process_arg(event, arg, &token);
- if (test_type_token(type, token, TEP_EVENT_OP, "]"))
- goto out_free;
-
- top->op.right = arg;
-
- free_token(token);
- type = read_token_item(&token);
- *tok = token;
-
- return type;
-
-out_free:
- free_token(token);
- free_arg(arg);
- return TEP_EVENT_ERROR;
-}
-
-static int get_op_prio(char *op)
-{
- if (!op[1]) {
- switch (op[0]) {
- case '~':
- case '!':
- return 4;
- case '*':
- case '/':
- case '%':
- return 6;
- case '+':
- case '-':
- return 7;
- /* '>>' and '<<' are 8 */
- case '<':
- case '>':
- return 9;
- /* '==' and '!=' are 10 */
- case '&':
- return 11;
- case '^':
- return 12;
- case '|':
- return 13;
- case '?':
- return 16;
- default:
- do_warning("unknown op '%c'", op[0]);
- return -1;
- }
- } else {
- if (strcmp(op, "++") == 0 ||
- strcmp(op, "--") == 0) {
- return 3;
- } else if (strcmp(op, ">>") == 0 ||
- strcmp(op, "<<") == 0) {
- return 8;
- } else if (strcmp(op, ">=") == 0 ||
- strcmp(op, "<=") == 0) {
- return 9;
- } else if (strcmp(op, "==") == 0 ||
- strcmp(op, "!=") == 0) {
- return 10;
- } else if (strcmp(op, "&&") == 0) {
- return 14;
- } else if (strcmp(op, "||") == 0) {
- return 15;
- } else {
- do_warning("unknown op '%s'", op);
- return -1;
- }
- }
-}
-
-static int set_op_prio(struct tep_print_arg *arg)
-{
-
- /* single ops are the greatest */
- if (!arg->op.left || arg->op.left->type == TEP_PRINT_NULL)
- arg->op.prio = 0;
- else
- arg->op.prio = get_op_prio(arg->op.op);
-
- return arg->op.prio;
-}
-
-/* Note, *tok does not get freed, but will most likely be saved */
-static enum tep_event_type
-process_op(struct tep_event *event, struct tep_print_arg *arg, char **tok)
-{
- struct tep_print_arg *left, *right = NULL;
- enum tep_event_type type;
- char *token;
-
- /* the op is passed in via tok */
- token = *tok;
-
- if (arg->type == TEP_PRINT_OP && !arg->op.left) {
- /* handle single op */
- if (token[1]) {
- do_warning_event(event, "bad op token %s", token);
- goto out_free;
- }
- switch (token[0]) {
- case '~':
- case '!':
- case '+':
- case '-':
- break;
- default:
- do_warning_event(event, "bad op token %s", token);
- goto out_free;
-
- }
-
- /* make an empty left */
- left = alloc_arg();
- if (!left)
- goto out_warn_free;
-
- left->type = TEP_PRINT_NULL;
- arg->op.left = left;
-
- right = alloc_arg();
- if (!right)
- goto out_warn_free;
-
- arg->op.right = right;
-
- /* do not free the token, it belongs to an op */
- *tok = NULL;
- type = process_arg(event, right, tok);
-
- } else if (strcmp(token, "?") == 0) {
-
- left = alloc_arg();
- if (!left)
- goto out_warn_free;
-
- /* copy the top arg to the left */
- *left = *arg;
-
- arg->type = TEP_PRINT_OP;
- arg->op.op = token;
- arg->op.left = left;
- arg->op.prio = 0;
-
- /* it will set arg->op.right */
- type = process_cond(event, arg, tok);
-
- } else if (strcmp(token, ">>") == 0 ||
- strcmp(token, "<<") == 0 ||
- strcmp(token, "&") == 0 ||
- strcmp(token, "|") == 0 ||
- strcmp(token, "&&") == 0 ||
- strcmp(token, "||") == 0 ||
- strcmp(token, "-") == 0 ||
- strcmp(token, "+") == 0 ||
- strcmp(token, "*") == 0 ||
- strcmp(token, "^") == 0 ||
- strcmp(token, "/") == 0 ||
- strcmp(token, "%") == 0 ||
- strcmp(token, "<") == 0 ||
- strcmp(token, ">") == 0 ||
- strcmp(token, "<=") == 0 ||
- strcmp(token, ">=") == 0 ||
- strcmp(token, "==") == 0 ||
- strcmp(token, "!=") == 0) {
-
- left = alloc_arg();
- if (!left)
- goto out_warn_free;
-
- /* copy the top arg to the left */
- *left = *arg;
-
- arg->type = TEP_PRINT_OP;
- arg->op.op = token;
- arg->op.left = left;
- arg->op.right = NULL;
-
- if (set_op_prio(arg) == -1) {
- event->flags |= TEP_EVENT_FL_FAILED;
- /* arg->op.op (= token) will be freed at out_free */
- arg->op.op = NULL;
- goto out_free;
- }
-
- type = read_token_item(&token);
- *tok = token;
-
- /* could just be a type pointer */
- if ((strcmp(arg->op.op, "*") == 0) &&
- type == TEP_EVENT_DELIM && (strcmp(token, ")") == 0)) {
- int ret;
-
- if (left->type != TEP_PRINT_ATOM) {
- do_warning_event(event, "bad pointer type");
- goto out_free;
- }
- ret = append(&left->atom.atom, " ", "*");
- if (ret < 0)
- goto out_warn_free;
-
- free(arg->op.op);
- *arg = *left;
- free(left);
-
- return type;
- }
-
- right = alloc_arg();
- if (!right)
- goto out_warn_free;
-
- type = process_arg_token(event, right, tok, type);
- if (type == TEP_EVENT_ERROR) {
- free_arg(right);
- /* token was freed in process_arg_token() via *tok */
- token = NULL;
- goto out_free;
- }
-
- if (right->type == TEP_PRINT_OP &&
- get_op_prio(arg->op.op) < get_op_prio(right->op.op)) {
- struct tep_print_arg tmp;
-
- /* rotate ops according to the priority */
- arg->op.right = right->op.left;
-
- tmp = *arg;
- *arg = *right;
- *right = tmp;
-
- arg->op.left = right;
- } else {
- arg->op.right = right;
- }
-
- } else if (strcmp(token, "[") == 0) {
-
- left = alloc_arg();
- if (!left)
- goto out_warn_free;
-
- *left = *arg;
-
- arg->type = TEP_PRINT_OP;
- arg->op.op = token;
- arg->op.left = left;
-
- arg->op.prio = 0;
-
- /* it will set arg->op.right */
- type = process_array(event, arg, tok);
-
- } else {
- do_warning_event(event, "unknown op '%s'", token);
- event->flags |= TEP_EVENT_FL_FAILED;
- /* the arg is now the left side */
- goto out_free;
- }
-
- if (type == TEP_EVENT_OP && strcmp(*tok, ":") != 0) {
- int prio;
-
- /* higher prios need to be closer to the root */
- prio = get_op_prio(*tok);
-
- if (prio > arg->op.prio)
- return process_op(event, arg, tok);
-
- return process_op(event, right, tok);
- }
-
- return type;
-
-out_warn_free:
- do_warning_event(event, "%s: not enough memory!", __func__);
-out_free:
- free_token(token);
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_entry(struct tep_event *event __maybe_unused, struct tep_print_arg *arg,
- char **tok)
-{
- enum tep_event_type type;
- char *field;
- char *token;
-
- if (read_expected(TEP_EVENT_OP, "->") < 0)
- goto out_err;
-
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto out_free;
- field = token;
-
- arg->type = TEP_PRINT_FIELD;
- arg->field.name = field;
-
- if (is_flag_field) {
- arg->field.field = tep_find_any_field(event, arg->field.name);
- arg->field.field->flags |= TEP_FIELD_IS_FLAG;
- is_flag_field = 0;
- } else if (is_symbolic_field) {
- arg->field.field = tep_find_any_field(event, arg->field.name);
- arg->field.field->flags |= TEP_FIELD_IS_SYMBOLIC;
- is_symbolic_field = 0;
- }
-
- type = read_token(&token);
- *tok = token;
-
- return type;
-
- out_free:
- free_token(token);
- out_err:
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static int alloc_and_process_delim(struct tep_event *event, char *next_token,
- struct tep_print_arg **print_arg)
-{
- struct tep_print_arg *field;
- enum tep_event_type type;
- char *token;
- int ret = 0;
-
- field = alloc_arg();
- if (!field) {
- do_warning_event(event, "%s: not enough memory!", __func__);
- errno = ENOMEM;
- return -1;
- }
-
- type = process_arg(event, field, &token);
-
- if (test_type_token(type, token, TEP_EVENT_DELIM, next_token)) {
- errno = EINVAL;
- ret = -1;
- free_arg(field);
- goto out_free_token;
- }
-
- *print_arg = field;
-
-out_free_token:
- free_token(token);
-
- return ret;
-}
-
-static char *arg_eval (struct tep_print_arg *arg);
-
-static unsigned long long
-eval_type_str(unsigned long long val, const char *type, int pointer)
-{
- int sign = 0;
- char *ref;
- int len;
-
- len = strlen(type);
-
- if (pointer) {
-
- if (type[len-1] != '*') {
- do_warning("pointer expected with non pointer type");
- return val;
- }
-
- ref = malloc(len);
- if (!ref) {
- do_warning("%s: not enough memory!", __func__);
- return val;
- }
- memcpy(ref, type, len);
-
- /* chop off the " *" */
- ref[len - 2] = 0;
-
- val = eval_type_str(val, ref, 0);
- free(ref);
- return val;
- }
-
- /* check if this is a pointer */
- if (type[len - 1] == '*')
- return val;
-
- /* Try to figure out the arg size*/
- if (strncmp(type, "struct", 6) == 0)
- /* all bets off */
- return val;
-
- if (strcmp(type, "u8") == 0)
- return val & 0xff;
-
- if (strcmp(type, "u16") == 0)
- return val & 0xffff;
-
- if (strcmp(type, "u32") == 0)
- return val & 0xffffffff;
-
- if (strcmp(type, "u64") == 0 ||
- strcmp(type, "s64") == 0)
- return val;
-
- if (strcmp(type, "s8") == 0)
- return (unsigned long long)(char)val & 0xff;
-
- if (strcmp(type, "s16") == 0)
- return (unsigned long long)(short)val & 0xffff;
-
- if (strcmp(type, "s32") == 0)
- return (unsigned long long)(int)val & 0xffffffff;
-
- if (strncmp(type, "unsigned ", 9) == 0) {
- sign = 0;
- type += 9;
- }
-
- if (strcmp(type, "char") == 0) {
- if (sign)
- return (unsigned long long)(char)val & 0xff;
- else
- return val & 0xff;
- }
-
- if (strcmp(type, "short") == 0) {
- if (sign)
- return (unsigned long long)(short)val & 0xffff;
- else
- return val & 0xffff;
- }
-
- if (strcmp(type, "int") == 0) {
- if (sign)
- return (unsigned long long)(int)val & 0xffffffff;
- else
- return val & 0xffffffff;
- }
-
- return val;
-}
-
-/*
- * Try to figure out the type.
- */
-static unsigned long long
-eval_type(unsigned long long val, struct tep_print_arg *arg, int pointer)
-{
- if (arg->type != TEP_PRINT_TYPE) {
- do_warning("expected type argument");
- return 0;
- }
-
- return eval_type_str(val, arg->typecast.type, pointer);
-}
-
-static int arg_num_eval(struct tep_print_arg *arg, long long *val)
-{
- long long left, right;
- int ret = 1;
-
- switch (arg->type) {
- case TEP_PRINT_ATOM:
- *val = strtoll(arg->atom.atom, NULL, 0);
- break;
- case TEP_PRINT_TYPE:
- ret = arg_num_eval(arg->typecast.item, val);
- if (!ret)
- break;
- *val = eval_type(*val, arg, 0);
- break;
- case TEP_PRINT_OP:
- switch (arg->op.op[0]) {
- case '|':
- ret = arg_num_eval(arg->op.left, &left);
- if (!ret)
- break;
- ret = arg_num_eval(arg->op.right, &right);
- if (!ret)
- break;
- if (arg->op.op[1])
- *val = left || right;
- else
- *val = left | right;
- break;
- case '&':
- ret = arg_num_eval(arg->op.left, &left);
- if (!ret)
- break;
- ret = arg_num_eval(arg->op.right, &right);
- if (!ret)
- break;
- if (arg->op.op[1])
- *val = left && right;
- else
- *val = left & right;
- break;
- case '<':
- ret = arg_num_eval(arg->op.left, &left);
- if (!ret)
- break;
- ret = arg_num_eval(arg->op.right, &right);
- if (!ret)
- break;
- switch (arg->op.op[1]) {
- case 0:
- *val = left < right;
- break;
- case '<':
- *val = left << right;
- break;
- case '=':
- *val = left <= right;
- break;
- default:
- do_warning("unknown op '%s'", arg->op.op);
- ret = 0;
- }
- break;
- case '>':
- ret = arg_num_eval(arg->op.left, &left);
- if (!ret)
- break;
- ret = arg_num_eval(arg->op.right, &right);
- if (!ret)
- break;
- switch (arg->op.op[1]) {
- case 0:
- *val = left > right;
- break;
- case '>':
- *val = left >> right;
- break;
- case '=':
- *val = left >= right;
- break;
- default:
- do_warning("unknown op '%s'", arg->op.op);
- ret = 0;
- }
- break;
- case '=':
- ret = arg_num_eval(arg->op.left, &left);
- if (!ret)
- break;
- ret = arg_num_eval(arg->op.right, &right);
- if (!ret)
- break;
-
- if (arg->op.op[1] != '=') {
- do_warning("unknown op '%s'", arg->op.op);
- ret = 0;
- } else
- *val = left == right;
- break;
- case '!':
- ret = arg_num_eval(arg->op.left, &left);
- if (!ret)
- break;
- ret = arg_num_eval(arg->op.right, &right);
- if (!ret)
- break;
-
- switch (arg->op.op[1]) {
- case '=':
- *val = left != right;
- break;
- default:
- do_warning("unknown op '%s'", arg->op.op);
- ret = 0;
- }
- break;
- case '-':
- /* check for negative */
- if (arg->op.left->type == TEP_PRINT_NULL)
- left = 0;
- else
- ret = arg_num_eval(arg->op.left, &left);
- if (!ret)
- break;
- ret = arg_num_eval(arg->op.right, &right);
- if (!ret)
- break;
- *val = left - right;
- break;
- case '+':
- if (arg->op.left->type == TEP_PRINT_NULL)
- left = 0;
- else
- ret = arg_num_eval(arg->op.left, &left);
- if (!ret)
- break;
- ret = arg_num_eval(arg->op.right, &right);
- if (!ret)
- break;
- *val = left + right;
- break;
- case '~':
- ret = arg_num_eval(arg->op.right, &right);
- if (!ret)
- break;
- *val = ~right;
- break;
- default:
- do_warning("unknown op '%s'", arg->op.op);
- ret = 0;
- }
- break;
-
- case TEP_PRINT_NULL:
- case TEP_PRINT_FIELD ... TEP_PRINT_SYMBOL:
- case TEP_PRINT_STRING:
- case TEP_PRINT_BSTRING:
- case TEP_PRINT_BITMASK:
- default:
- do_warning("invalid eval type %d", arg->type);
- ret = 0;
-
- }
- return ret;
-}
-
-static char *arg_eval (struct tep_print_arg *arg)
-{
- long long val;
- static char buf[24];
-
- switch (arg->type) {
- case TEP_PRINT_ATOM:
- return arg->atom.atom;
- case TEP_PRINT_TYPE:
- return arg_eval(arg->typecast.item);
- case TEP_PRINT_OP:
- if (!arg_num_eval(arg, &val))
- break;
- sprintf(buf, "%lld", val);
- return buf;
-
- case TEP_PRINT_NULL:
- case TEP_PRINT_FIELD ... TEP_PRINT_SYMBOL:
- case TEP_PRINT_STRING:
- case TEP_PRINT_BSTRING:
- case TEP_PRINT_BITMASK:
- default:
- do_warning("invalid eval type %d", arg->type);
- break;
- }
-
- return NULL;
-}
-
-static enum tep_event_type
-process_fields(struct tep_event *event, struct tep_print_flag_sym **list, char **tok)
-{
- enum tep_event_type type;
- struct tep_print_arg *arg = NULL;
- struct tep_print_flag_sym *field;
- char *token = *tok;
- char *value;
-
- do {
- free_token(token);
- type = read_token_item(&token);
- if (test_type_token(type, token, TEP_EVENT_OP, "{"))
- break;
-
- arg = alloc_arg();
- if (!arg)
- goto out_free;
-
- free_token(token);
- type = process_arg(event, arg, &token);
-
- if (type == TEP_EVENT_OP)
- type = process_op(event, arg, &token);
-
- if (type == TEP_EVENT_ERROR)
- goto out_free;
-
- if (test_type_token(type, token, TEP_EVENT_DELIM, ","))
- goto out_free;
-
- field = calloc(1, sizeof(*field));
- if (!field)
- goto out_free;
-
- value = arg_eval(arg);
- if (value == NULL)
- goto out_free_field;
- field->value = strdup(value);
- if (field->value == NULL)
- goto out_free_field;
-
- free_arg(arg);
- arg = alloc_arg();
- if (!arg)
- goto out_free;
-
- free_token(token);
- type = process_arg(event, arg, &token);
- if (test_type_token(type, token, TEP_EVENT_OP, "}"))
- goto out_free_field;
-
- value = arg_eval(arg);
- if (value == NULL)
- goto out_free_field;
- field->str = strdup(value);
- if (field->str == NULL)
- goto out_free_field;
- free_arg(arg);
- arg = NULL;
-
- *list = field;
- list = &field->next;
-
- free_token(token);
- type = read_token_item(&token);
- } while (type == TEP_EVENT_DELIM && strcmp(token, ",") == 0);
-
- *tok = token;
- return type;
-
-out_free_field:
- free_flag_sym(field);
-out_free:
- free_arg(arg);
- free_token(token);
- *tok = NULL;
-
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_flags(struct tep_event *event, struct tep_print_arg *arg, char **tok)
-{
- struct tep_print_arg *field;
- enum tep_event_type type;
- char *token = NULL;
-
- memset(arg, 0, sizeof(*arg));
- arg->type = TEP_PRINT_FLAGS;
-
- field = alloc_arg();
- if (!field) {
- do_warning_event(event, "%s: not enough memory!", __func__);
- goto out_free;
- }
-
- type = process_field_arg(event, field, &token);
-
- /* Handle operations in the first argument */
- while (type == TEP_EVENT_OP)
- type = process_op(event, field, &token);
-
- if (test_type_token(type, token, TEP_EVENT_DELIM, ","))
- goto out_free_field;
- free_token(token);
-
- arg->flags.field = field;
-
- type = read_token_item(&token);
- if (event_item_type(type)) {
- arg->flags.delim = token;
- type = read_token_item(&token);
- }
-
- if (test_type_token(type, token, TEP_EVENT_DELIM, ","))
- goto out_free;
-
- type = process_fields(event, &arg->flags.flags, &token);
- if (test_type_token(type, token, TEP_EVENT_DELIM, ")"))
- goto out_free;
-
- free_token(token);
- type = read_token_item(tok);
- return type;
-
-out_free_field:
- free_arg(field);
-out_free:
- free_token(token);
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_symbols(struct tep_event *event, struct tep_print_arg *arg, char **tok)
-{
- struct tep_print_arg *field;
- enum tep_event_type type;
- char *token = NULL;
-
- memset(arg, 0, sizeof(*arg));
- arg->type = TEP_PRINT_SYMBOL;
-
- field = alloc_arg();
- if (!field) {
- do_warning_event(event, "%s: not enough memory!", __func__);
- goto out_free;
- }
-
- type = process_field_arg(event, field, &token);
-
- if (test_type_token(type, token, TEP_EVENT_DELIM, ","))
- goto out_free_field;
-
- arg->symbol.field = field;
-
- type = process_fields(event, &arg->symbol.symbols, &token);
- if (test_type_token(type, token, TEP_EVENT_DELIM, ")"))
- goto out_free;
-
- free_token(token);
- type = read_token_item(tok);
- return type;
-
-out_free_field:
- free_arg(field);
-out_free:
- free_token(token);
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_hex_common(struct tep_event *event, struct tep_print_arg *arg,
- char **tok, enum tep_print_arg_type type)
-{
- memset(arg, 0, sizeof(*arg));
- arg->type = type;
-
- if (alloc_and_process_delim(event, ",", &arg->hex.field))
- goto out;
-
- if (alloc_and_process_delim(event, ")", &arg->hex.size))
- goto free_field;
-
- return read_token_item(tok);
-
-free_field:
- free_arg(arg->hex.field);
- arg->hex.field = NULL;
-out:
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_hex(struct tep_event *event, struct tep_print_arg *arg, char **tok)
-{
- return process_hex_common(event, arg, tok, TEP_PRINT_HEX);
-}
-
-static enum tep_event_type
-process_hex_str(struct tep_event *event, struct tep_print_arg *arg,
- char **tok)
-{
- return process_hex_common(event, arg, tok, TEP_PRINT_HEX_STR);
-}
-
-static enum tep_event_type
-process_int_array(struct tep_event *event, struct tep_print_arg *arg, char **tok)
-{
- memset(arg, 0, sizeof(*arg));
- arg->type = TEP_PRINT_INT_ARRAY;
-
- if (alloc_and_process_delim(event, ",", &arg->int_array.field))
- goto out;
-
- if (alloc_and_process_delim(event, ",", &arg->int_array.count))
- goto free_field;
-
- if (alloc_and_process_delim(event, ")", &arg->int_array.el_size))
- goto free_size;
-
- return read_token_item(tok);
-
-free_size:
- free_arg(arg->int_array.count);
- arg->int_array.count = NULL;
-free_field:
- free_arg(arg->int_array.field);
- arg->int_array.field = NULL;
-out:
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_dynamic_array(struct tep_event *event, struct tep_print_arg *arg, char **tok)
-{
- struct tep_format_field *field;
- enum tep_event_type type;
- char *token;
-
- memset(arg, 0, sizeof(*arg));
- arg->type = TEP_PRINT_DYNAMIC_ARRAY;
-
- /*
- * The item within the parenthesis is another field that holds
- * the index into where the array starts.
- */
- type = read_token(&token);
- *tok = token;
- if (type != TEP_EVENT_ITEM)
- goto out_free;
-
- /* Find the field */
-
- field = tep_find_field(event, token);
- if (!field)
- goto out_free;
-
- arg->dynarray.field = field;
- arg->dynarray.index = 0;
-
- if (read_expected(TEP_EVENT_DELIM, ")") < 0)
- goto out_free;
-
- free_token(token);
- type = read_token_item(&token);
- *tok = token;
- if (type != TEP_EVENT_OP || strcmp(token, "[") != 0)
- return type;
-
- free_token(token);
- arg = alloc_arg();
- if (!arg) {
- do_warning_event(event, "%s: not enough memory!", __func__);
- *tok = NULL;
- return TEP_EVENT_ERROR;
- }
-
- type = process_arg(event, arg, &token);
- if (type == TEP_EVENT_ERROR)
- goto out_free_arg;
-
- if (!test_type_token(type, token, TEP_EVENT_OP, "]"))
- goto out_free_arg;
-
- free_token(token);
- type = read_token_item(tok);
- return type;
-
- out_free_arg:
- free_arg(arg);
- out_free:
- free_token(token);
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_dynamic_array_len(struct tep_event *event, struct tep_print_arg *arg,
- char **tok)
-{
- struct tep_format_field *field;
- enum tep_event_type type;
- char *token;
-
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto out_free;
-
- arg->type = TEP_PRINT_DYNAMIC_ARRAY_LEN;
-
- /* Find the field */
- field = tep_find_field(event, token);
- if (!field)
- goto out_free;
-
- arg->dynarray.field = field;
- arg->dynarray.index = 0;
-
- if (read_expected(TEP_EVENT_DELIM, ")") < 0)
- goto out_err;
-
- free_token(token);
- type = read_token(&token);
- *tok = token;
-
- return type;
-
- out_free:
- free_token(token);
- out_err:
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_paren(struct tep_event *event, struct tep_print_arg *arg, char **tok)
-{
- struct tep_print_arg *item_arg;
- enum tep_event_type type;
- char *token;
-
- type = process_arg(event, arg, &token);
-
- if (type == TEP_EVENT_ERROR)
- goto out_free;
-
- if (type == TEP_EVENT_OP)
- type = process_op(event, arg, &token);
-
- if (type == TEP_EVENT_ERROR)
- goto out_free;
-
- if (test_type_token(type, token, TEP_EVENT_DELIM, ")"))
- goto out_free;
-
- free_token(token);
- type = read_token_item(&token);
-
- /*
- * If the next token is an item or another open paren, then
- * this was a typecast.
- */
- if (event_item_type(type) ||
- (type == TEP_EVENT_DELIM && strcmp(token, "(") == 0)) {
-
- /* make this a typecast and contine */
-
- /* prevous must be an atom */
- if (arg->type != TEP_PRINT_ATOM) {
- do_warning_event(event, "previous needed to be TEP_PRINT_ATOM");
- goto out_free;
- }
-
- item_arg = alloc_arg();
- if (!item_arg) {
- do_warning_event(event, "%s: not enough memory!",
- __func__);
- goto out_free;
- }
-
- arg->type = TEP_PRINT_TYPE;
- arg->typecast.type = arg->atom.atom;
- arg->typecast.item = item_arg;
- type = process_arg_token(event, item_arg, &token, type);
-
- }
-
- *tok = token;
- return type;
-
- out_free:
- free_token(token);
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-
-static enum tep_event_type
-process_str(struct tep_event *event __maybe_unused, struct tep_print_arg *arg,
- char **tok)
-{
- enum tep_event_type type;
- char *token;
-
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto out_free;
-
- arg->type = TEP_PRINT_STRING;
- arg->string.string = token;
- arg->string.field = NULL;
-
- if (read_expected(TEP_EVENT_DELIM, ")") < 0)
- goto out_err;
-
- type = read_token(&token);
- *tok = token;
-
- return type;
-
- out_free:
- free_token(token);
- out_err:
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_bitmask(struct tep_event *event __maybe_unused, struct tep_print_arg *arg,
- char **tok)
-{
- enum tep_event_type type;
- char *token;
-
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto out_free;
-
- arg->type = TEP_PRINT_BITMASK;
- arg->bitmask.bitmask = token;
- arg->bitmask.field = NULL;
-
- if (read_expected(TEP_EVENT_DELIM, ")") < 0)
- goto out_err;
-
- type = read_token(&token);
- *tok = token;
-
- return type;
-
- out_free:
- free_token(token);
- out_err:
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static struct tep_function_handler *
-find_func_handler(struct tep_handle *tep, char *func_name)
-{
- struct tep_function_handler *func;
-
- if (!tep)
- return NULL;
-
- for (func = tep->func_handlers; func; func = func->next) {
- if (strcmp(func->name, func_name) == 0)
- break;
- }
-
- return func;
-}
-
-static void remove_func_handler(struct tep_handle *tep, char *func_name)
-{
- struct tep_function_handler *func;
- struct tep_function_handler **next;
-
- next = &tep->func_handlers;
- while ((func = *next)) {
- if (strcmp(func->name, func_name) == 0) {
- *next = func->next;
- free_func_handle(func);
- break;
- }
- next = &func->next;
- }
-}
-
-static enum tep_event_type
-process_func_handler(struct tep_event *event, struct tep_function_handler *func,
- struct tep_print_arg *arg, char **tok)
-{
- struct tep_print_arg **next_arg;
- struct tep_print_arg *farg;
- enum tep_event_type type;
- char *token;
- int i;
-
- arg->type = TEP_PRINT_FUNC;
- arg->func.func = func;
-
- *tok = NULL;
-
- next_arg = &(arg->func.args);
- for (i = 0; i < func->nr_args; i++) {
- farg = alloc_arg();
- if (!farg) {
- do_warning_event(event, "%s: not enough memory!",
- __func__);
- return TEP_EVENT_ERROR;
- }
-
- type = process_arg(event, farg, &token);
- if (i < (func->nr_args - 1)) {
- if (type != TEP_EVENT_DELIM || strcmp(token, ",") != 0) {
- do_warning_event(event,
- "Error: function '%s()' expects %d arguments but event %s only uses %d",
- func->name, func->nr_args,
- event->name, i + 1);
- goto err;
- }
- } else {
- if (type != TEP_EVENT_DELIM || strcmp(token, ")") != 0) {
- do_warning_event(event,
- "Error: function '%s()' only expects %d arguments but event %s has more",
- func->name, func->nr_args, event->name);
- goto err;
- }
- }
-
- *next_arg = farg;
- next_arg = &(farg->next);
- free_token(token);
- }
-
- type = read_token(&token);
- *tok = token;
-
- return type;
-
-err:
- free_arg(farg);
- free_token(token);
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_builtin_expect(struct tep_event *event, struct tep_print_arg *arg, char **tok)
-{
- enum tep_event_type type;
- char *token = NULL;
-
- /* Handle __builtin_expect( cond, #) */
- type = process_arg(event, arg, &token);
-
- if (type != TEP_EVENT_DELIM || token[0] != ',')
- goto out_free;
-
- free_token(token);
-
- /* We don't care what the second parameter is of the __builtin_expect() */
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto out_free;
-
- if (read_expected(TEP_EVENT_DELIM, ")") < 0)
- goto out_free;
-
- free_token(token);
- type = read_token_item(tok);
- return type;
-
-out_free:
- free_token(token);
- *tok = NULL;
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_function(struct tep_event *event, struct tep_print_arg *arg,
- char *token, char **tok)
-{
- struct tep_function_handler *func;
-
- if (strcmp(token, "__print_flags") == 0) {
- free_token(token);
- is_flag_field = 1;
- return process_flags(event, arg, tok);
- }
- if (strcmp(token, "__print_symbolic") == 0) {
- free_token(token);
- is_symbolic_field = 1;
- return process_symbols(event, arg, tok);
- }
- if (strcmp(token, "__print_hex") == 0) {
- free_token(token);
- return process_hex(event, arg, tok);
- }
- if (strcmp(token, "__print_hex_str") == 0) {
- free_token(token);
- return process_hex_str(event, arg, tok);
- }
- if (strcmp(token, "__print_array") == 0) {
- free_token(token);
- return process_int_array(event, arg, tok);
- }
- if (strcmp(token, "__get_str") == 0 ||
- strcmp(token, "__get_rel_str") == 0) {
- free_token(token);
- return process_str(event, arg, tok);
- }
- if (strcmp(token, "__get_bitmask") == 0 ||
- strcmp(token, "__get_rel_bitmask") == 0) {
- free_token(token);
- return process_bitmask(event, arg, tok);
- }
- if (strcmp(token, "__get_dynamic_array") == 0 ||
- strcmp(token, "__get_rel_dynamic_array") == 0) {
- free_token(token);
- return process_dynamic_array(event, arg, tok);
- }
- if (strcmp(token, "__get_dynamic_array_len") == 0 ||
- strcmp(token, "__get_rel_dynamic_array_len") == 0) {
- free_token(token);
- return process_dynamic_array_len(event, arg, tok);
- }
- if (strcmp(token, "__builtin_expect") == 0) {
- free_token(token);
- return process_builtin_expect(event, arg, tok);
- }
-
- func = find_func_handler(event->tep, token);
- if (func) {
- free_token(token);
- return process_func_handler(event, func, arg, tok);
- }
-
- do_warning_event(event, "function %s not defined", token);
- free_token(token);
- return TEP_EVENT_ERROR;
-}
-
-static enum tep_event_type
-process_arg_token(struct tep_event *event, struct tep_print_arg *arg,
- char **tok, enum tep_event_type type)
-{
- char *token;
- char *atom;
-
- token = *tok;
-
- switch (type) {
- case TEP_EVENT_ITEM:
- if (strcmp(token, "REC") == 0) {
- free_token(token);
- type = process_entry(event, arg, &token);
- break;
- }
- atom = token;
- /* test the next token */
- type = read_token_item(&token);
-
- /*
- * If the next token is a parenthesis, then this
- * is a function.
- */
- if (type == TEP_EVENT_DELIM && strcmp(token, "(") == 0) {
- free_token(token);
- token = NULL;
- /* this will free atom. */
- type = process_function(event, arg, atom, &token);
- break;
- }
- /* atoms can be more than one token long */
- while (type == TEP_EVENT_ITEM) {
- int ret;
-
- ret = append(&atom, " ", token);
- if (ret < 0) {
- free(atom);
- *tok = NULL;
- free_token(token);
- return TEP_EVENT_ERROR;
- }
- free_token(token);
- type = read_token_item(&token);
- }
-
- arg->type = TEP_PRINT_ATOM;
- arg->atom.atom = atom;
- break;
-
- case TEP_EVENT_DQUOTE:
- case TEP_EVENT_SQUOTE:
- arg->type = TEP_PRINT_ATOM;
- arg->atom.atom = token;
- type = read_token_item(&token);
- break;
- case TEP_EVENT_DELIM:
- if (strcmp(token, "(") == 0) {
- free_token(token);
- type = process_paren(event, arg, &token);
- break;
- }
- case TEP_EVENT_OP:
- /* handle single ops */
- arg->type = TEP_PRINT_OP;
- arg->op.op = token;
- arg->op.left = NULL;
- type = process_op(event, arg, &token);
-
- /* On error, the op is freed */
- if (type == TEP_EVENT_ERROR)
- arg->op.op = NULL;
-
- /* return error type if errored */
- break;
-
- case TEP_EVENT_ERROR ... TEP_EVENT_NEWLINE:
- default:
- do_warning_event(event, "unexpected type %d", type);
- return TEP_EVENT_ERROR;
- }
- *tok = token;
-
- return type;
-}
-
-static int event_read_print_args(struct tep_event *event, struct tep_print_arg **list)
-{
- enum tep_event_type type = TEP_EVENT_ERROR;
- struct tep_print_arg *arg;
- char *token;
- int args = 0;
-
- do {
- if (type == TEP_EVENT_NEWLINE) {
- type = read_token_item(&token);
- continue;
- }
-
- arg = alloc_arg();
- if (!arg) {
- do_warning_event(event, "%s: not enough memory!",
- __func__);
- return -1;
- }
-
- type = process_arg(event, arg, &token);
-
- if (type == TEP_EVENT_ERROR) {
- free_token(token);
- free_arg(arg);
- return -1;
- }
-
- *list = arg;
- args++;
-
- if (type == TEP_EVENT_OP) {
- type = process_op(event, arg, &token);
- free_token(token);
- if (type == TEP_EVENT_ERROR) {
- *list = NULL;
- free_arg(arg);
- return -1;
- }
- list = &arg->next;
- continue;
- }
-
- if (type == TEP_EVENT_DELIM && strcmp(token, ",") == 0) {
- free_token(token);
- *list = arg;
- list = &arg->next;
- continue;
- }
- break;
- } while (type != TEP_EVENT_NONE);
-
- if (type != TEP_EVENT_NONE && type != TEP_EVENT_ERROR)
- free_token(token);
-
- return args;
-}
-
-static int event_read_print(struct tep_event *event)
-{
- enum tep_event_type type;
- char *token;
- int ret;
-
- if (read_expected_item(TEP_EVENT_ITEM, "print") < 0)
- return -1;
-
- if (read_expected(TEP_EVENT_ITEM, "fmt") < 0)
- return -1;
-
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- return -1;
-
- if (read_expect_type(TEP_EVENT_DQUOTE, &token) < 0)
- goto fail;
-
- concat:
- event->print_fmt.format = token;
- event->print_fmt.args = NULL;
-
- /* ok to have no arg */
- type = read_token_item(&token);
-
- if (type == TEP_EVENT_NONE)
- return 0;
-
- /* Handle concatenation of print lines */
- if (type == TEP_EVENT_DQUOTE) {
- char *cat;
-
- if (asprintf(&cat, "%s%s", event->print_fmt.format, token) < 0)
- goto fail;
- free_token(token);
- free_token(event->print_fmt.format);
- event->print_fmt.format = NULL;
- token = cat;
- goto concat;
- }
-
- if (test_type_token(type, token, TEP_EVENT_DELIM, ","))
- goto fail;
-
- free_token(token);
-
- ret = event_read_print_args(event, &event->print_fmt.args);
- if (ret < 0)
- return -1;
-
- return ret;
-
- fail:
- free_token(token);
- return -1;
-}
-
-/**
- * tep_find_common_field - return a common field by event
- * @event: handle for the event
- * @name: the name of the common field to return
- *
- * Returns a common field from the event by the given @name.
- * This only searches the common fields and not all field.
- */
-struct tep_format_field *
-tep_find_common_field(struct tep_event *event, const char *name)
-{
- struct tep_format_field *format;
-
- for (format = event->format.common_fields;
- format; format = format->next) {
- if (strcmp(format->name, name) == 0)
- break;
- }
-
- return format;
-}
-
-/**
- * tep_find_field - find a non-common field
- * @event: handle for the event
- * @name: the name of the non-common field
- *
- * Returns a non-common field by the given @name.
- * This does not search common fields.
- */
-struct tep_format_field *
-tep_find_field(struct tep_event *event, const char *name)
-{
- struct tep_format_field *format;
-
- for (format = event->format.fields;
- format; format = format->next) {
- if (strcmp(format->name, name) == 0)
- break;
- }
-
- return format;
-}
-
-/**
- * tep_find_any_field - find any field by name
- * @event: handle for the event
- * @name: the name of the field
- *
- * Returns a field by the given @name.
- * This searches the common field names first, then
- * the non-common ones if a common one was not found.
- */
-struct tep_format_field *
-tep_find_any_field(struct tep_event *event, const char *name)
-{
- struct tep_format_field *format;
-
- format = tep_find_common_field(event, name);
- if (format)
- return format;
- return tep_find_field(event, name);
-}
-
-/**
- * tep_read_number - read a number from data
- * @tep: a handle to the trace event parser context
- * @ptr: the raw data
- * @size: the size of the data that holds the number
- *
- * Returns the number (converted to host) from the
- * raw data.
- */
-unsigned long long tep_read_number(struct tep_handle *tep,
- const void *ptr, int size)
-{
- unsigned long long val;
-
- switch (size) {
- case 1:
- return *(unsigned char *)ptr;
- case 2:
- return data2host2(tep, *(unsigned short *)ptr);
- case 4:
- return data2host4(tep, *(unsigned int *)ptr);
- case 8:
- memcpy(&val, (ptr), sizeof(unsigned long long));
- return data2host8(tep, val);
- default:
- /* BUG! */
- return 0;
- }
-}
-
-/**
- * tep_read_number_field - read a number from data
- * @field: a handle to the field
- * @data: the raw data to read
- * @value: the value to place the number in
- *
- * Reads raw data according to a field offset and size,
- * and translates it into @value.
- *
- * Returns 0 on success, -1 otherwise.
- */
-int tep_read_number_field(struct tep_format_field *field, const void *data,
- unsigned long long *value)
-{
- if (!field)
- return -1;
- switch (field->size) {
- case 1:
- case 2:
- case 4:
- case 8:
- *value = tep_read_number(field->event->tep,
- data + field->offset, field->size);
- return 0;
- default:
- return -1;
- }
-}
-
-static int get_common_info(struct tep_handle *tep,
- const char *type, int *offset, int *size)
-{
- struct tep_event *event;
- struct tep_format_field *field;
-
- /*
- * All events should have the same common elements.
- * Pick any event to find where the type is;
- */
- if (!tep->events) {
- do_warning("no event_list!");
- return -1;
- }
-
- event = tep->events[0];
- field = tep_find_common_field(event, type);
- if (!field)
- return -1;
-
- *offset = field->offset;
- *size = field->size;
-
- return 0;
-}
-
-static int __parse_common(struct tep_handle *tep, void *data,
- int *size, int *offset, const char *name)
-{
- int ret;
-
- if (!*size) {
- ret = get_common_info(tep, name, offset, size);
- if (ret < 0)
- return ret;
- }
- return tep_read_number(tep, data + *offset, *size);
-}
-
-static int trace_parse_common_type(struct tep_handle *tep, void *data)
-{
- return __parse_common(tep, data,
- &tep->type_size, &tep->type_offset,
- "common_type");
-}
-
-static int parse_common_pid(struct tep_handle *tep, void *data)
-{
- return __parse_common(tep, data,
- &tep->pid_size, &tep->pid_offset,
- "common_pid");
-}
-
-static int parse_common_pc(struct tep_handle *tep, void *data)
-{
- return __parse_common(tep, data,
- &tep->pc_size, &tep->pc_offset,
- "common_preempt_count");
-}
-
-static int parse_common_flags(struct tep_handle *tep, void *data)
-{
- return __parse_common(tep, data,
- &tep->flags_size, &tep->flags_offset,
- "common_flags");
-}
-
-static int parse_common_lock_depth(struct tep_handle *tep, void *data)
-{
- return __parse_common(tep, data,
- &tep->ld_size, &tep->ld_offset,
- "common_lock_depth");
-}
-
-static int parse_common_migrate_disable(struct tep_handle *tep, void *data)
-{
- return __parse_common(tep, data,
- &tep->ld_size, &tep->ld_offset,
- "common_migrate_disable");
-}
-
-static int events_id_cmp(const void *a, const void *b);
-
-/**
- * tep_find_event - find an event by given id
- * @tep: a handle to the trace event parser context
- * @id: the id of the event
- *
- * Returns an event that has a given @id.
- */
-struct tep_event *tep_find_event(struct tep_handle *tep, int id)
-{
- struct tep_event **eventptr;
- struct tep_event key;
- struct tep_event *pkey = &key;
-
- /* Check cache first */
- if (tep->last_event && tep->last_event->id == id)
- return tep->last_event;
-
- key.id = id;
-
- eventptr = bsearch(&pkey, tep->events, tep->nr_events,
- sizeof(*tep->events), events_id_cmp);
-
- if (eventptr) {
- tep->last_event = *eventptr;
- return *eventptr;
- }
-
- return NULL;
-}
-
-/**
- * tep_find_event_by_name - find an event by given name
- * @tep: a handle to the trace event parser context
- * @sys: the system name to search for
- * @name: the name of the event to search for
- *
- * This returns an event with a given @name and under the system
- * @sys. If @sys is NULL the first event with @name is returned.
- */
-struct tep_event *
-tep_find_event_by_name(struct tep_handle *tep,
- const char *sys, const char *name)
-{
- struct tep_event *event = NULL;
- int i;
-
- if (tep->last_event &&
- strcmp(tep->last_event->name, name) == 0 &&
- (!sys || strcmp(tep->last_event->system, sys) == 0))
- return tep->last_event;
-
- for (i = 0; i < tep->nr_events; i++) {
- event = tep->events[i];
- if (strcmp(event->name, name) == 0) {
- if (!sys)
- break;
- if (strcmp(event->system, sys) == 0)
- break;
- }
- }
- if (i == tep->nr_events)
- event = NULL;
-
- tep->last_event = event;
- return event;
-}
-
-static unsigned long long
-eval_num_arg(void *data, int size, struct tep_event *event, struct tep_print_arg *arg)
-{
- struct tep_handle *tep = event->tep;
- unsigned long long val = 0;
- unsigned long long left, right;
- struct tep_print_arg *typearg = NULL;
- struct tep_print_arg *larg;
- unsigned long offset;
- unsigned int field_size;
-
- switch (arg->type) {
- case TEP_PRINT_NULL:
- /* ?? */
- return 0;
- case TEP_PRINT_ATOM:
- return strtoull(arg->atom.atom, NULL, 0);
- case TEP_PRINT_FIELD:
- if (!arg->field.field) {
- arg->field.field = tep_find_any_field(event, arg->field.name);
- if (!arg->field.field)
- goto out_warning_field;
-
- }
- /* must be a number */
- val = tep_read_number(tep, data + arg->field.field->offset,
- arg->field.field->size);
- break;
- case TEP_PRINT_FLAGS:
- case TEP_PRINT_SYMBOL:
- case TEP_PRINT_INT_ARRAY:
- case TEP_PRINT_HEX:
- case TEP_PRINT_HEX_STR:
- break;
- case TEP_PRINT_TYPE:
- val = eval_num_arg(data, size, event, arg->typecast.item);
- return eval_type(val, arg, 0);
- case TEP_PRINT_STRING:
- case TEP_PRINT_BSTRING:
- case TEP_PRINT_BITMASK:
- return 0;
- case TEP_PRINT_FUNC: {
- struct trace_seq s;
- trace_seq_init(&s);
- val = process_defined_func(&s, data, size, event, arg);
- trace_seq_destroy(&s);
- return val;
- }
- case TEP_PRINT_OP:
- if (strcmp(arg->op.op, "[") == 0) {
- /*
- * Arrays are special, since we don't want
- * to read the arg as is.
- */
- right = eval_num_arg(data, size, event, arg->op.right);
-
- /* handle typecasts */
- larg = arg->op.left;
- while (larg->type == TEP_PRINT_TYPE) {
- if (!typearg)
- typearg = larg;
- larg = larg->typecast.item;
- }
-
- /* Default to long size */
- field_size = tep->long_size;
-
- switch (larg->type) {
- case TEP_PRINT_DYNAMIC_ARRAY:
- offset = tep_read_number(tep,
- data + larg->dynarray.field->offset,
- larg->dynarray.field->size);
- if (larg->dynarray.field->elementsize)
- field_size = larg->dynarray.field->elementsize;
- /*
- * The actual length of the dynamic array is stored
- * in the top half of the field, and the offset
- * is in the bottom half of the 32 bit field.
- */
- offset &= 0xffff;
- offset += right;
- break;
- case TEP_PRINT_FIELD:
- if (!larg->field.field) {
- larg->field.field =
- tep_find_any_field(event, larg->field.name);
- if (!larg->field.field) {
- arg = larg;
- goto out_warning_field;
- }
- }
- field_size = larg->field.field->elementsize;
- offset = larg->field.field->offset +
- right * larg->field.field->elementsize;
- break;
- default:
- goto default_op; /* oops, all bets off */
- }
- val = tep_read_number(tep,
- data + offset, field_size);
- if (typearg)
- val = eval_type(val, typearg, 1);
- break;
- } else if (strcmp(arg->op.op, "?") == 0) {
- left = eval_num_arg(data, size, event, arg->op.left);
- arg = arg->op.right;
- if (left)
- val = eval_num_arg(data, size, event, arg->op.left);
- else
- val = eval_num_arg(data, size, event, arg->op.right);
- break;
- }
- default_op:
- left = eval_num_arg(data, size, event, arg->op.left);
- right = eval_num_arg(data, size, event, arg->op.right);
- switch (arg->op.op[0]) {
- case '!':
- switch (arg->op.op[1]) {
- case 0:
- val = !right;
- break;
- case '=':
- val = left != right;
- break;
- default:
- goto out_warning_op;
- }
- break;
- case '~':
- val = ~right;
- break;
- case '|':
- if (arg->op.op[1])
- val = left || right;
- else
- val = left | right;
- break;
- case '&':
- if (arg->op.op[1])
- val = left && right;
- else
- val = left & right;
- break;
- case '<':
- switch (arg->op.op[1]) {
- case 0:
- val = left < right;
- break;
- case '<':
- val = left << right;
- break;
- case '=':
- val = left <= right;
- break;
- default:
- goto out_warning_op;
- }
- break;
- case '>':
- switch (arg->op.op[1]) {
- case 0:
- val = left > right;
- break;
- case '>':
- val = left >> right;
- break;
- case '=':
- val = left >= right;
- break;
- default:
- goto out_warning_op;
- }
- break;
- case '=':
- if (arg->op.op[1] != '=')
- goto out_warning_op;
-
- val = left == right;
- break;
- case '-':
- val = left - right;
- break;
- case '+':
- val = left + right;
- break;
- case '/':
- val = left / right;
- break;
- case '%':
- val = left % right;
- break;
- case '*':
- val = left * right;
- break;
- default:
- goto out_warning_op;
- }
- break;
- case TEP_PRINT_DYNAMIC_ARRAY_LEN:
- offset = tep_read_number(tep,
- data + arg->dynarray.field->offset,
- arg->dynarray.field->size);
- /*
- * The total allocated length of the dynamic array is
- * stored in the top half of the field, and the offset
- * is in the bottom half of the 32 bit field.
- */
- val = (unsigned long long)(offset >> 16);
- break;
- case TEP_PRINT_DYNAMIC_ARRAY:
- /* Without [], we pass the address to the dynamic data */
- offset = tep_read_number(tep,
- data + arg->dynarray.field->offset,
- arg->dynarray.field->size);
- /*
- * The total allocated length of the dynamic array is
- * stored in the top half of the field, and the offset
- * is in the bottom half of the 32 bit field.
- */
- offset &= 0xffff;
- val = (unsigned long long)((unsigned long)data + offset);
- break;
- default: /* not sure what to do there */
- return 0;
- }
- return val;
-
-out_warning_op:
- do_warning_event(event, "%s: unknown op '%s'", __func__, arg->op.op);
- return 0;
-
-out_warning_field:
- do_warning_event(event, "%s: field %s not found",
- __func__, arg->field.name);
- return 0;
-}
-
-struct flag {
- const char *name;
- unsigned long long value;
-};
-
-static const struct flag flags[] = {
- { "HI_SOFTIRQ", 0 },
- { "TIMER_SOFTIRQ", 1 },
- { "NET_TX_SOFTIRQ", 2 },
- { "NET_RX_SOFTIRQ", 3 },
- { "BLOCK_SOFTIRQ", 4 },
- { "IRQ_POLL_SOFTIRQ", 5 },
- { "TASKLET_SOFTIRQ", 6 },
- { "SCHED_SOFTIRQ", 7 },
- { "HRTIMER_SOFTIRQ", 8 },
- { "RCU_SOFTIRQ", 9 },
-
- { "HRTIMER_NORESTART", 0 },
- { "HRTIMER_RESTART", 1 },
-};
-
-static long long eval_flag(const char *flag)
-{
- int i;
-
- /*
- * Some flags in the format files do not get converted.
- * If the flag is not numeric, see if it is something that
- * we already know about.
- */
- if (isdigit(flag[0]))
- return strtoull(flag, NULL, 0);
-
- for (i = 0; i < (int)(sizeof(flags)/sizeof(flags[0])); i++)
- if (strcmp(flags[i].name, flag) == 0)
- return flags[i].value;
-
- return -1LL;
-}
-
-static void print_str_to_seq(struct trace_seq *s, const char *format,
- int len_arg, const char *str)
-{
- if (len_arg >= 0)
- trace_seq_printf(s, format, len_arg, str);
- else
- trace_seq_printf(s, format, str);
-}
-
-static void print_bitmask_to_seq(struct tep_handle *tep,
- struct trace_seq *s, const char *format,
- int len_arg, const void *data, int size)
-{
- int nr_bits = size * 8;
- int str_size = (nr_bits + 3) / 4;
- int len = 0;
- char buf[3];
- char *str;
- int index;
- int i;
-
- /*
- * The kernel likes to put in commas every 32 bits, we
- * can do the same.
- */
- str_size += (nr_bits - 1) / 32;
-
- str = malloc(str_size + 1);
- if (!str) {
- do_warning("%s: not enough memory!", __func__);
- return;
- }
- str[str_size] = 0;
-
- /* Start out with -2 for the two chars per byte */
- for (i = str_size - 2; i >= 0; i -= 2) {
- /*
- * data points to a bit mask of size bytes.
- * In the kernel, this is an array of long words, thus
- * endianness is very important.
- */
- if (tep->file_bigendian)
- index = size - (len + 1);
- else
- index = len;
-
- snprintf(buf, 3, "%02x", *((unsigned char *)data + index));
- memcpy(str + i, buf, 2);
- len++;
- if (!(len & 3) && i > 0) {
- i--;
- str[i] = ',';
- }
- }
-
- if (len_arg >= 0)
- trace_seq_printf(s, format, len_arg, str);
- else
- trace_seq_printf(s, format, str);
-
- free(str);
-}
-
-static void print_str_arg(struct trace_seq *s, void *data, int size,
- struct tep_event *event, const char *format,
- int len_arg, struct tep_print_arg *arg)
-{
- struct tep_handle *tep = event->tep;
- struct tep_print_flag_sym *flag;
- struct tep_format_field *field;
- struct printk_map *printk;
- long long val, fval;
- unsigned long long addr;
- char *str;
- unsigned char *hex;
- int print;
- int i, len;
-
- switch (arg->type) {
- case TEP_PRINT_NULL:
- /* ?? */
- return;
- case TEP_PRINT_ATOM:
- print_str_to_seq(s, format, len_arg, arg->atom.atom);
- return;
- case TEP_PRINT_FIELD:
- field = arg->field.field;
- if (!field) {
- field = tep_find_any_field(event, arg->field.name);
- if (!field) {
- str = arg->field.name;
- goto out_warning_field;
- }
- arg->field.field = field;
- }
- /* Zero sized fields, mean the rest of the data */
- len = field->size ? : size - field->offset;
-
- /*
- * Some events pass in pointers. If this is not an array
- * and the size is the same as long_size, assume that it
- * is a pointer.
- */
- if (!(field->flags & TEP_FIELD_IS_ARRAY) &&
- field->size == tep->long_size) {
-
- /* Handle heterogeneous recording and processing
- * architectures
- *
- * CASE I:
- * Traces recorded on 32-bit devices (32-bit
- * addressing) and processed on 64-bit devices:
- * In this case, only 32 bits should be read.
- *
- * CASE II:
- * Traces recorded on 64 bit devices and processed
- * on 32-bit devices:
- * In this case, 64 bits must be read.
- */
- addr = (tep->long_size == 8) ?
- *(unsigned long long *)(data + field->offset) :
- (unsigned long long)*(unsigned int *)(data + field->offset);
-
- /* Check if it matches a print format */
- printk = find_printk(tep, addr);
- if (printk)
- trace_seq_puts(s, printk->printk);
- else
- trace_seq_printf(s, "%llx", addr);
- break;
- }
- str = malloc(len + 1);
- if (!str) {
- do_warning_event(event, "%s: not enough memory!",
- __func__);
- return;
- }
- memcpy(str, data + field->offset, len);
- str[len] = 0;
- print_str_to_seq(s, format, len_arg, str);
- free(str);
- break;
- case TEP_PRINT_FLAGS:
- val = eval_num_arg(data, size, event, arg->flags.field);
- print = 0;
- for (flag = arg->flags.flags; flag; flag = flag->next) {
- fval = eval_flag(flag->value);
- if (!val && fval < 0) {
- print_str_to_seq(s, format, len_arg, flag->str);
- break;
- }
- if (fval > 0 && (val & fval) == fval) {
- if (print && arg->flags.delim)
- trace_seq_puts(s, arg->flags.delim);
- print_str_to_seq(s, format, len_arg, flag->str);
- print = 1;
- val &= ~fval;
- }
- }
- if (val) {
- if (print && arg->flags.delim)
- trace_seq_puts(s, arg->flags.delim);
- trace_seq_printf(s, "0x%llx", val);
- }
- break;
- case TEP_PRINT_SYMBOL:
- val = eval_num_arg(data, size, event, arg->symbol.field);
- for (flag = arg->symbol.symbols; flag; flag = flag->next) {
- fval = eval_flag(flag->value);
- if (val == fval) {
- print_str_to_seq(s, format, len_arg, flag->str);
- break;
- }
- }
- if (!flag)
- trace_seq_printf(s, "0x%llx", val);
- break;
- case TEP_PRINT_HEX:
- case TEP_PRINT_HEX_STR:
- if (arg->hex.field->type == TEP_PRINT_DYNAMIC_ARRAY) {
- unsigned long offset;
- offset = tep_read_number(tep,
- data + arg->hex.field->dynarray.field->offset,
- arg->hex.field->dynarray.field->size);
- hex = data + (offset & 0xffff);
- } else {
- field = arg->hex.field->field.field;
- if (!field) {
- str = arg->hex.field->field.name;
- field = tep_find_any_field(event, str);
- if (!field)
- goto out_warning_field;
- arg->hex.field->field.field = field;
- }
- hex = data + field->offset;
- }
- len = eval_num_arg(data, size, event, arg->hex.size);
- for (i = 0; i < len; i++) {
- if (i && arg->type == TEP_PRINT_HEX)
- trace_seq_putc(s, ' ');
- trace_seq_printf(s, "%02x", hex[i]);
- }
- break;
-
- case TEP_PRINT_INT_ARRAY: {
- void *num;
- int el_size;
-
- if (arg->int_array.field->type == TEP_PRINT_DYNAMIC_ARRAY) {
- unsigned long offset;
- struct tep_format_field *field =
- arg->int_array.field->dynarray.field;
- offset = tep_read_number(tep,
- data + field->offset,
- field->size);
- num = data + (offset & 0xffff);
- } else {
- field = arg->int_array.field->field.field;
- if (!field) {
- str = arg->int_array.field->field.name;
- field = tep_find_any_field(event, str);
- if (!field)
- goto out_warning_field;
- arg->int_array.field->field.field = field;
- }
- num = data + field->offset;
- }
- len = eval_num_arg(data, size, event, arg->int_array.count);
- el_size = eval_num_arg(data, size, event,
- arg->int_array.el_size);
- for (i = 0; i < len; i++) {
- if (i)
- trace_seq_putc(s, ' ');
-
- if (el_size == 1) {
- trace_seq_printf(s, "%u", *(uint8_t *)num);
- } else if (el_size == 2) {
- trace_seq_printf(s, "%u", *(uint16_t *)num);
- } else if (el_size == 4) {
- trace_seq_printf(s, "%u", *(uint32_t *)num);
- } else if (el_size == 8) {
- trace_seq_printf(s, "%"PRIu64, *(uint64_t *)num);
- } else {
- trace_seq_printf(s, "BAD SIZE:%d 0x%x",
- el_size, *(uint8_t *)num);
- el_size = 1;
- }
-
- num += el_size;
- }
- break;
- }
- case TEP_PRINT_TYPE:
- break;
- case TEP_PRINT_STRING: {
- int str_offset;
-
- if (!arg->string.field)
- arg->string.field = tep_find_any_field(event, arg->string.string);
- if (!arg->string.field)
- break;
-
- str_offset = data2host4(tep,
- *(unsigned int *)(data + arg->string.field->offset));
- str_offset &= 0xffff;
- if (arg->string.field->flags & TEP_FIELD_IS_RELATIVE)
- str_offset += arg->string.field->offset + arg->string.field->size;
- print_str_to_seq(s, format, len_arg, ((char *)data) + str_offset);
- break;
- }
- case TEP_PRINT_BSTRING:
- print_str_to_seq(s, format, len_arg, arg->string.string);
- break;
- case TEP_PRINT_BITMASK: {
- int bitmask_offset;
- int bitmask_size;
-
- if (!arg->bitmask.field)
- arg->bitmask.field = tep_find_any_field(event, arg->bitmask.bitmask);
- if (!arg->bitmask.field)
- break;
- bitmask_offset = data2host4(tep,
- *(unsigned int *)(data + arg->bitmask.field->offset));
- bitmask_size = bitmask_offset >> 16;
- bitmask_offset &= 0xffff;
- if (arg->bitmask.field->flags & TEP_FIELD_IS_RELATIVE)
- bitmask_offset += arg->bitmask.field->offset + arg->bitmask.field->size;
- print_bitmask_to_seq(tep, s, format, len_arg,
- data + bitmask_offset, bitmask_size);
- break;
- }
- case TEP_PRINT_OP:
- /*
- * The only op for string should be ? :
- */
- if (arg->op.op[0] != '?')
- return;
- val = eval_num_arg(data, size, event, arg->op.left);
- if (val)
- print_str_arg(s, data, size, event,
- format, len_arg, arg->op.right->op.left);
- else
- print_str_arg(s, data, size, event,
- format, len_arg, arg->op.right->op.right);
- break;
- case TEP_PRINT_FUNC:
- process_defined_func(s, data, size, event, arg);
- break;
- default:
- /* well... */
- break;
- }
-
- return;
-
-out_warning_field:
- do_warning_event(event, "%s: field %s not found",
- __func__, arg->field.name);
-}
-
-static unsigned long long
-process_defined_func(struct trace_seq *s, void *data, int size,
- struct tep_event *event, struct tep_print_arg *arg)
-{
- struct tep_function_handler *func_handle = arg->func.func;
- struct func_params *param;
- unsigned long long *args;
- unsigned long long ret;
- struct tep_print_arg *farg;
- struct trace_seq str;
- struct save_str {
- struct save_str *next;
- char *str;
- } *strings = NULL, *string;
- int i;
-
- if (!func_handle->nr_args) {
- ret = (*func_handle->func)(s, NULL);
- goto out;
- }
-
- farg = arg->func.args;
- param = func_handle->params;
-
- ret = ULLONG_MAX;
- args = malloc(sizeof(*args) * func_handle->nr_args);
- if (!args)
- goto out;
-
- for (i = 0; i < func_handle->nr_args; i++) {
- switch (param->type) {
- case TEP_FUNC_ARG_INT:
- case TEP_FUNC_ARG_LONG:
- case TEP_FUNC_ARG_PTR:
- args[i] = eval_num_arg(data, size, event, farg);
- break;
- case TEP_FUNC_ARG_STRING:
- trace_seq_init(&str);
- print_str_arg(&str, data, size, event, "%s", -1, farg);
- trace_seq_terminate(&str);
- string = malloc(sizeof(*string));
- if (!string) {
- do_warning_event(event, "%s(%d): malloc str",
- __func__, __LINE__);
- goto out_free;
- }
- string->next = strings;
- string->str = strdup(str.buffer);
- if (!string->str) {
- free(string);
- do_warning_event(event, "%s(%d): malloc str",
- __func__, __LINE__);
- goto out_free;
- }
- args[i] = (uintptr_t)string->str;
- strings = string;
- trace_seq_destroy(&str);
- break;
- default:
- /*
- * Something went totally wrong, this is not
- * an input error, something in this code broke.
- */
- do_warning_event(event, "Unexpected end of arguments\n");
- goto out_free;
- }
- farg = farg->next;
- param = param->next;
- }
-
- ret = (*func_handle->func)(s, args);
-out_free:
- free(args);
- while (strings) {
- string = strings;
- strings = string->next;
- free(string->str);
- free(string);
- }
-
- out:
- /* TBD : handle return type here */
- return ret;
-}
-
-static void free_args(struct tep_print_arg *args)
-{
- struct tep_print_arg *next;
-
- while (args) {
- next = args->next;
-
- free_arg(args);
- args = next;
- }
-}
-
-static struct tep_print_arg *make_bprint_args(char *fmt, void *data, int size, struct tep_event *event)
-{
- struct tep_handle *tep = event->tep;
- struct tep_format_field *field, *ip_field;
- struct tep_print_arg *args, *arg, **next;
- unsigned long long ip, val;
- char *ptr;
- void *bptr;
- int vsize = 0;
-
- field = tep->bprint_buf_field;
- ip_field = tep->bprint_ip_field;
-
- if (!field) {
- field = tep_find_field(event, "buf");
- if (!field) {
- do_warning_event(event, "can't find buffer field for binary printk");
- return NULL;
- }
- ip_field = tep_find_field(event, "ip");
- if (!ip_field) {
- do_warning_event(event, "can't find ip field for binary printk");
- return NULL;
- }
- tep->bprint_buf_field = field;
- tep->bprint_ip_field = ip_field;
- }
-
- ip = tep_read_number(tep, data + ip_field->offset, ip_field->size);
-
- /*
- * The first arg is the IP pointer.
- */
- args = alloc_arg();
- if (!args) {
- do_warning_event(event, "%s(%d): not enough memory!",
- __func__, __LINE__);
- return NULL;
- }
- arg = args;
- arg->next = NULL;
- next = &arg->next;
-
- arg->type = TEP_PRINT_ATOM;
-
- if (asprintf(&arg->atom.atom, "%lld", ip) < 0)
- goto out_free;
-
- /* skip the first "%ps: " */
- for (ptr = fmt + 5, bptr = data + field->offset;
- bptr < data + size && *ptr; ptr++) {
- int ls = 0;
-
- if (*ptr == '%') {
- process_again:
- ptr++;
- switch (*ptr) {
- case '%':
- break;
- case 'l':
- ls++;
- goto process_again;
- case 'L':
- ls = 2;
- goto process_again;
- case '0' ... '9':
- goto process_again;
- case '.':
- goto process_again;
- case 'z':
- case 'Z':
- ls = 1;
- goto process_again;
- case 'p':
- ls = 1;
- if (isalnum(ptr[1])) {
- ptr++;
- /* Check for special pointers */
- switch (*ptr) {
- case 's':
- case 'S':
- case 'x':
- break;
- case 'f':
- case 'F':
- /*
- * Pre-5.5 kernels use %pf and
- * %pF for printing symbols
- * while kernels since 5.5 use
- * %pfw for fwnodes. So check
- * %p[fF] isn't followed by 'w'.
- */
- if (ptr[1] != 'w')
- break;
- /* fall through */
- default:
- /*
- * Older kernels do not process
- * dereferenced pointers.
- * Only process if the pointer
- * value is a printable.
- */
- if (isprint(*(char *)bptr))
- goto process_string;
- }
- }
- /* fall through */
- case 'd':
- case 'u':
- case 'i':
- case 'x':
- case 'X':
- case 'o':
- switch (ls) {
- case 0:
- vsize = 4;
- break;
- case 1:
- vsize = tep->long_size;
- break;
- case 2:
- vsize = 8;
- break;
- default:
- vsize = ls; /* ? */
- break;
- }
- /* fall through */
- case '*':
- if (*ptr == '*')
- vsize = 4;
-
- /* the pointers are always 4 bytes aligned */
- bptr = (void *)(((unsigned long)bptr + 3) &
- ~3);
- val = tep_read_number(tep, bptr, vsize);
- bptr += vsize;
- arg = alloc_arg();
- if (!arg) {
- do_warning_event(event, "%s(%d): not enough memory!",
- __func__, __LINE__);
- goto out_free;
- }
- arg->next = NULL;
- arg->type = TEP_PRINT_ATOM;
- if (asprintf(&arg->atom.atom, "%lld", val) < 0) {
- free(arg);
- goto out_free;
- }
- *next = arg;
- next = &arg->next;
- /*
- * The '*' case means that an arg is used as the length.
- * We need to continue to figure out for what.
- */
- if (*ptr == '*')
- goto process_again;
-
- break;
- case 's':
- process_string:
- arg = alloc_arg();
- if (!arg) {
- do_warning_event(event, "%s(%d): not enough memory!",
- __func__, __LINE__);
- goto out_free;
- }
- arg->next = NULL;
- arg->type = TEP_PRINT_BSTRING;
- arg->string.string = strdup(bptr);
- if (!arg->string.string)
- goto out_free;
- bptr += strlen(bptr) + 1;
- *next = arg;
- next = &arg->next;
- default:
- break;
- }
- }
- }
-
- return args;
-
-out_free:
- free_args(args);
- return NULL;
-}
-
-static char *
-get_bprint_format(void *data, int size __maybe_unused,
- struct tep_event *event)
-{
- struct tep_handle *tep = event->tep;
- unsigned long long addr;
- struct tep_format_field *field;
- struct printk_map *printk;
- char *format;
-
- field = tep->bprint_fmt_field;
-
- if (!field) {
- field = tep_find_field(event, "fmt");
- if (!field) {
- do_warning_event(event, "can't find format field for binary printk");
- return NULL;
- }
- tep->bprint_fmt_field = field;
- }
-
- addr = tep_read_number(tep, data + field->offset, field->size);
-
- printk = find_printk(tep, addr);
- if (!printk) {
- if (asprintf(&format, "%%ps: (NO FORMAT FOUND at %llx)\n", addr) < 0)
- return NULL;
- return format;
- }
-
- if (asprintf(&format, "%s: %s", "%ps", printk->printk) < 0)
- return NULL;
-
- return format;
-}
-
-static int print_mac_arg(struct trace_seq *s, const char *format,
- void *data, int size, struct tep_event *event,
- struct tep_print_arg *arg)
-{
- const char *fmt = "%.2x:%.2x:%.2x:%.2x:%.2x:%.2x";
- bool reverse = false;
- unsigned char *buf;
- int ret = 0;
-
- if (arg->type == TEP_PRINT_FUNC) {
- process_defined_func(s, data, size, event, arg);
- return 0;
- }
-
- if (arg->type != TEP_PRINT_FIELD) {
- trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d",
- arg->type);
- return 0;
- }
-
- if (format[0] == 'm') {
- fmt = "%.2x%.2x%.2x%.2x%.2x%.2x";
- } else if (format[0] == 'M' && format[1] == 'F') {
- fmt = "%.2x-%.2x-%.2x-%.2x-%.2x-%.2x";
- ret++;
- }
- if (format[1] == 'R') {
- reverse = true;
- ret++;
- }
-
- if (!arg->field.field) {
- arg->field.field =
- tep_find_any_field(event, arg->field.name);
- if (!arg->field.field) {
- do_warning_event(event, "%s: field %s not found",
- __func__, arg->field.name);
- return ret;
- }
- }
- if (arg->field.field->size != 6) {
- trace_seq_printf(s, "INVALIDMAC");
- return ret;
- }
-
- buf = data + arg->field.field->offset;
- if (reverse)
- trace_seq_printf(s, fmt, buf[5], buf[4], buf[3], buf[2], buf[1], buf[0]);
- else
- trace_seq_printf(s, fmt, buf[0], buf[1], buf[2], buf[3], buf[4], buf[5]);
-
- return ret;
-}
-
-static int parse_ip4_print_args(struct tep_handle *tep,
- const char *ptr, bool *reverse)
-{
- int ret = 0;
-
- *reverse = false;
-
- /* hnbl */
- switch (*ptr) {
- case 'h':
- if (tep->file_bigendian)
- *reverse = false;
- else
- *reverse = true;
- ret++;
- break;
- case 'l':
- *reverse = true;
- ret++;
- break;
- case 'n':
- case 'b':
- ret++;
- /* fall through */
- default:
- *reverse = false;
- break;
- }
-
- return ret;
-}
-
-static void print_ip4_addr(struct trace_seq *s, char i, bool reverse, unsigned char *buf)
-{
- const char *fmt;
-
- if (i == 'i')
- fmt = "%03d.%03d.%03d.%03d";
- else
- fmt = "%d.%d.%d.%d";
-
- if (reverse)
- trace_seq_printf(s, fmt, buf[3], buf[2], buf[1], buf[0]);
- else
- trace_seq_printf(s, fmt, buf[0], buf[1], buf[2], buf[3]);
-
-}
-
-static inline bool ipv6_addr_v4mapped(const struct in6_addr *a)
-{
- return ((unsigned long)(a->s6_addr32[0] | a->s6_addr32[1]) |
- (unsigned long)(a->s6_addr32[2] ^ htonl(0x0000ffff))) == 0UL;
-}
-
-static inline bool ipv6_addr_is_isatap(const struct in6_addr *addr)
-{
- return (addr->s6_addr32[2] | htonl(0x02000000)) == htonl(0x02005EFE);
-}
-
-static void print_ip6c_addr(struct trace_seq *s, unsigned char *addr)
-{
- int i, j, range;
- unsigned char zerolength[8];
- int longest = 1;
- int colonpos = -1;
- uint16_t word;
- uint8_t hi, lo;
- bool needcolon = false;
- bool useIPv4;
- struct in6_addr in6;
-
- memcpy(&in6, addr, sizeof(struct in6_addr));
-
- useIPv4 = ipv6_addr_v4mapped(&in6) || ipv6_addr_is_isatap(&in6);
-
- memset(zerolength, 0, sizeof(zerolength));
-
- if (useIPv4)
- range = 6;
- else
- range = 8;
-
- /* find position of longest 0 run */
- for (i = 0; i < range; i++) {
- for (j = i; j < range; j++) {
- if (in6.s6_addr16[j] != 0)
- break;
- zerolength[i]++;
- }
- }
- for (i = 0; i < range; i++) {
- if (zerolength[i] > longest) {
- longest = zerolength[i];
- colonpos = i;
- }
- }
- if (longest == 1) /* don't compress a single 0 */
- colonpos = -1;
-
- /* emit address */
- for (i = 0; i < range; i++) {
- if (i == colonpos) {
- if (needcolon || i == 0)
- trace_seq_printf(s, ":");
- trace_seq_printf(s, ":");
- needcolon = false;
- i += longest - 1;
- continue;
- }
- if (needcolon) {
- trace_seq_printf(s, ":");
- needcolon = false;
- }
- /* hex u16 without leading 0s */
- word = ntohs(in6.s6_addr16[i]);
- hi = word >> 8;
- lo = word & 0xff;
- if (hi)
- trace_seq_printf(s, "%x%02x", hi, lo);
- else
- trace_seq_printf(s, "%x", lo);
-
- needcolon = true;
- }
-
- if (useIPv4) {
- if (needcolon)
- trace_seq_printf(s, ":");
- print_ip4_addr(s, 'I', false, &in6.s6_addr[12]);
- }
-
- return;
-}
-
-static void print_ip6_addr(struct trace_seq *s, char i, unsigned char *buf)
-{
- int j;
-
- for (j = 0; j < 16; j += 2) {
- trace_seq_printf(s, "%02x%02x", buf[j], buf[j+1]);
- if (i == 'I' && j < 14)
- trace_seq_printf(s, ":");
- }
-}
-
-/*
- * %pi4 print an IPv4 address with leading zeros
- * %pI4 print an IPv4 address without leading zeros
- * %pi6 print an IPv6 address without colons
- * %pI6 print an IPv6 address with colons
- * %pI6c print an IPv6 address in compressed form with colons
- * %pISpc print an IP address based on sockaddr; p adds port.
- */
-static int print_ipv4_arg(struct trace_seq *s, const char *ptr, char i,
- void *data, int size, struct tep_event *event,
- struct tep_print_arg *arg)
-{
- bool reverse = false;
- unsigned char *buf;
- int ret;
-
- ret = parse_ip4_print_args(event->tep, ptr, &reverse);
-
- if (arg->type == TEP_PRINT_FUNC) {
- process_defined_func(s, data, size, event, arg);
- return ret;
- }
-
- if (arg->type != TEP_PRINT_FIELD) {
- trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
- return ret;
- }
-
- if (!arg->field.field) {
- arg->field.field =
- tep_find_any_field(event, arg->field.name);
- if (!arg->field.field) {
- do_warning("%s: field %s not found",
- __func__, arg->field.name);
- return ret;
- }
- }
-
- buf = data + arg->field.field->offset;
-
- if (arg->field.field->size != 4) {
- trace_seq_printf(s, "INVALIDIPv4");
- return ret;
- }
-
- print_ip4_addr(s, i, reverse, buf);
- return ret;
-
-}
-
-static int print_ipv6_arg(struct trace_seq *s, const char *ptr, char i,
- void *data, int size, struct tep_event *event,
- struct tep_print_arg *arg)
-{
- char have_c = 0;
- unsigned char *buf;
- int rc = 0;
-
- /* pI6c */
- if (i == 'I' && *ptr == 'c') {
- have_c = 1;
- ptr++;
- rc++;
- }
-
- if (arg->type == TEP_PRINT_FUNC) {
- process_defined_func(s, data, size, event, arg);
- return rc;
- }
-
- if (arg->type != TEP_PRINT_FIELD) {
- trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
- return rc;
- }
-
- if (!arg->field.field) {
- arg->field.field =
- tep_find_any_field(event, arg->field.name);
- if (!arg->field.field) {
- do_warning("%s: field %s not found",
- __func__, arg->field.name);
- return rc;
- }
- }
-
- buf = data + arg->field.field->offset;
-
- if (arg->field.field->size != 16) {
- trace_seq_printf(s, "INVALIDIPv6");
- return rc;
- }
-
- if (have_c)
- print_ip6c_addr(s, buf);
- else
- print_ip6_addr(s, i, buf);
-
- return rc;
-}
-
-static int print_ipsa_arg(struct trace_seq *s, const char *ptr, char i,
- void *data, int size, struct tep_event *event,
- struct tep_print_arg *arg)
-{
- char have_c = 0, have_p = 0;
- unsigned char *buf;
- struct sockaddr_storage *sa;
- bool reverse = false;
- int rc = 0;
- int ret;
-
- /* pISpc */
- if (i == 'I') {
- if (*ptr == 'p') {
- have_p = 1;
- ptr++;
- rc++;
- }
- if (*ptr == 'c') {
- have_c = 1;
- ptr++;
- rc++;
- }
- }
- ret = parse_ip4_print_args(event->tep, ptr, &reverse);
- ptr += ret;
- rc += ret;
-
- if (arg->type == TEP_PRINT_FUNC) {
- process_defined_func(s, data, size, event, arg);
- return rc;
- }
-
- if (arg->type != TEP_PRINT_FIELD) {
- trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
- return rc;
- }
-
- if (!arg->field.field) {
- arg->field.field =
- tep_find_any_field(event, arg->field.name);
- if (!arg->field.field) {
- do_warning("%s: field %s not found",
- __func__, arg->field.name);
- return rc;
- }
- }
-
- sa = (struct sockaddr_storage *) (data + arg->field.field->offset);
-
- if (sa->ss_family == AF_INET) {
- struct sockaddr_in *sa4 = (struct sockaddr_in *) sa;
-
- if (arg->field.field->size < sizeof(struct sockaddr_in)) {
- trace_seq_printf(s, "INVALIDIPv4");
- return rc;
- }
-
- print_ip4_addr(s, i, reverse, (unsigned char *) &sa4->sin_addr);
- if (have_p)
- trace_seq_printf(s, ":%d", ntohs(sa4->sin_port));
-
-
- } else if (sa->ss_family == AF_INET6) {
- struct sockaddr_in6 *sa6 = (struct sockaddr_in6 *) sa;
-
- if (arg->field.field->size < sizeof(struct sockaddr_in6)) {
- trace_seq_printf(s, "INVALIDIPv6");
- return rc;
- }
-
- if (have_p)
- trace_seq_printf(s, "[");
-
- buf = (unsigned char *) &sa6->sin6_addr;
- if (have_c)
- print_ip6c_addr(s, buf);
- else
- print_ip6_addr(s, i, buf);
-
- if (have_p)
- trace_seq_printf(s, "]:%d", ntohs(sa6->sin6_port));
- }
-
- return rc;
-}
-
-static int print_ip_arg(struct trace_seq *s, const char *ptr,
- void *data, int size, struct tep_event *event,
- struct tep_print_arg *arg)
-{
- char i = *ptr; /* 'i' or 'I' */
- int rc = 1;
-
- /* IP version */
- ptr++;
-
- switch (*ptr) {
- case '4':
- rc += print_ipv4_arg(s, ptr + 1, i, data, size, event, arg);
- break;
- case '6':
- rc += print_ipv6_arg(s, ptr + 1, i, data, size, event, arg);
- break;
- case 'S':
- rc += print_ipsa_arg(s, ptr + 1, i, data, size, event, arg);
- break;
- default:
- return 0;
- }
-
- return rc;
-}
-
-static const int guid_index[16] = {3, 2, 1, 0, 5, 4, 7, 6, 8, 9, 10, 11, 12, 13, 14, 15};
-static const int uuid_index[16] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15};
-
-static int print_uuid_arg(struct trace_seq *s, const char *ptr,
- void *data, int size, struct tep_event *event,
- struct tep_print_arg *arg)
-{
- const int *index = uuid_index;
- char *format = "%02x";
- int ret = 0;
- char *buf;
- int i;
-
- switch (*(ptr + 1)) {
- case 'L':
- format = "%02X";
- /* fall through */
- case 'l':
- index = guid_index;
- ret++;
- break;
- case 'B':
- format = "%02X";
- /* fall through */
- case 'b':
- ret++;
- break;
- }
-
- if (arg->type == TEP_PRINT_FUNC) {
- process_defined_func(s, data, size, event, arg);
- return ret;
- }
-
- if (arg->type != TEP_PRINT_FIELD) {
- trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
- return ret;
- }
-
- if (!arg->field.field) {
- arg->field.field =
- tep_find_any_field(event, arg->field.name);
- if (!arg->field.field) {
- do_warning("%s: field %s not found",
- __func__, arg->field.name);
- return ret;
- }
- }
-
- if (arg->field.field->size != 16) {
- trace_seq_printf(s, "INVALIDUUID");
- return ret;
- }
-
- buf = data + arg->field.field->offset;
-
- for (i = 0; i < 16; i++) {
- trace_seq_printf(s, format, buf[index[i]] & 0xff);
- switch (i) {
- case 3:
- case 5:
- case 7:
- case 9:
- trace_seq_printf(s, "-");
- break;
- }
- }
-
- return ret;
-}
-
-static int print_raw_buff_arg(struct trace_seq *s, const char *ptr,
- void *data, int size, struct tep_event *event,
- struct tep_print_arg *arg, int print_len)
-{
- int plen = print_len;
- char *delim = " ";
- int ret = 0;
- char *buf;
- int i;
- unsigned long offset;
- int arr_len;
-
- switch (*(ptr + 1)) {
- case 'C':
- delim = ":";
- ret++;
- break;
- case 'D':
- delim = "-";
- ret++;
- break;
- case 'N':
- delim = "";
- ret++;
- break;
- }
-
- if (arg->type == TEP_PRINT_FUNC) {
- process_defined_func(s, data, size, event, arg);
- return ret;
- }
-
- if (arg->type != TEP_PRINT_DYNAMIC_ARRAY) {
- trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
- return ret;
- }
-
- offset = tep_read_number(event->tep,
- data + arg->dynarray.field->offset,
- arg->dynarray.field->size);
- arr_len = (unsigned long long)(offset >> 16);
- buf = data + (offset & 0xffff);
-
- if (arr_len < plen)
- plen = arr_len;
-
- if (plen < 1)
- return ret;
-
- trace_seq_printf(s, "%02x", buf[0] & 0xff);
- for (i = 1; i < plen; i++)
- trace_seq_printf(s, "%s%02x", delim, buf[i] & 0xff);
-
- return ret;
-}
-
-static int is_printable_array(char *p, unsigned int len)
-{
- unsigned int i;
-
- for (i = 0; i < len && p[i]; i++)
- if (!isprint(p[i]) && !isspace(p[i]))
- return 0;
- return 1;
-}
-
-void tep_print_field(struct trace_seq *s, void *data,
- struct tep_format_field *field)
-{
- unsigned long long val;
- unsigned int offset, len, i;
- struct tep_handle *tep = field->event->tep;
-
- if (field->flags & TEP_FIELD_IS_ARRAY) {
- offset = field->offset;
- len = field->size;
- if (field->flags & TEP_FIELD_IS_DYNAMIC) {
- val = tep_read_number(tep, data + offset, len);
- offset = val;
- len = offset >> 16;
- offset &= 0xffff;
- if (field->flags & TEP_FIELD_IS_RELATIVE)
- offset += field->offset + field->size;
- }
- if (field->flags & TEP_FIELD_IS_STRING &&
- is_printable_array(data + offset, len)) {
- trace_seq_printf(s, "%s", (char *)data + offset);
- } else {
- trace_seq_puts(s, "ARRAY[");
- for (i = 0; i < len; i++) {
- if (i)
- trace_seq_puts(s, ", ");
- trace_seq_printf(s, "%02x",
- *((unsigned char *)data + offset + i));
- }
- trace_seq_putc(s, ']');
- field->flags &= ~TEP_FIELD_IS_STRING;
- }
- } else {
- val = tep_read_number(tep, data + field->offset,
- field->size);
- if (field->flags & TEP_FIELD_IS_POINTER) {
- trace_seq_printf(s, "0x%llx", val);
- } else if (field->flags & TEP_FIELD_IS_SIGNED) {
- switch (field->size) {
- case 4:
- /*
- * If field is long then print it in hex.
- * A long usually stores pointers.
- */
- if (field->flags & TEP_FIELD_IS_LONG)
- trace_seq_printf(s, "0x%x", (int)val);
- else
- trace_seq_printf(s, "%d", (int)val);
- break;
- case 2:
- trace_seq_printf(s, "%2d", (short)val);
- break;
- case 1:
- trace_seq_printf(s, "%1d", (char)val);
- break;
- default:
- trace_seq_printf(s, "%lld", val);
- }
- } else {
- if (field->flags & TEP_FIELD_IS_LONG)
- trace_seq_printf(s, "0x%llx", val);
- else
- trace_seq_printf(s, "%llu", val);
- }
- }
-}
-
-void tep_print_fields(struct trace_seq *s, void *data,
- int size __maybe_unused, struct tep_event *event)
-{
- struct tep_format_field *field;
-
- field = event->format.fields;
- while (field) {
- trace_seq_printf(s, " %s=", field->name);
- tep_print_field(s, data, field);
- field = field->next;
- }
-}
-
-static int print_function(struct trace_seq *s, const char *format,
- void *data, int size, struct tep_event *event,
- struct tep_print_arg *arg)
-{
- struct func_map *func;
- unsigned long long val;
-
- val = eval_num_arg(data, size, event, arg);
- func = find_func(event->tep, val);
- if (func) {
- trace_seq_puts(s, func->func);
- if (*format == 'F' || *format == 'S')
- trace_seq_printf(s, "+0x%llx", val - func->addr);
- } else {
- if (event->tep->long_size == 4)
- trace_seq_printf(s, "0x%lx", (long)val);
- else
- trace_seq_printf(s, "0x%llx", (long long)val);
- }
-
- return 0;
-}
-
-static int print_arg_pointer(struct trace_seq *s, const char *format, int plen,
- void *data, int size,
- struct tep_event *event, struct tep_print_arg *arg)
-{
- unsigned long long val;
- int ret = 1;
-
- if (arg->type == TEP_PRINT_BSTRING) {
- trace_seq_puts(s, arg->string.string);
- return 0;
- }
- while (*format) {
- if (*format == 'p') {
- format++;
- break;
- }
- format++;
- }
-
- switch (*format) {
- case 'F':
- case 'f':
- case 'S':
- case 's':
- ret += print_function(s, format, data, size, event, arg);
- break;
- case 'M':
- case 'm':
- ret += print_mac_arg(s, format, data, size, event, arg);
- break;
- case 'I':
- case 'i':
- ret += print_ip_arg(s, format, data, size, event, arg);
- break;
- case 'U':
- ret += print_uuid_arg(s, format, data, size, event, arg);
- break;
- case 'h':
- ret += print_raw_buff_arg(s, format, data, size, event, arg, plen);
- break;
- default:
- ret = 0;
- val = eval_num_arg(data, size, event, arg);
- trace_seq_printf(s, "%p", (void *)(intptr_t)val);
- break;
- }
-
- return ret;
-
-}
-
-static int print_arg_number(struct trace_seq *s, const char *format, int plen,
- void *data, int size, int ls,
- struct tep_event *event, struct tep_print_arg *arg)
-{
- unsigned long long val;
-
- val = eval_num_arg(data, size, event, arg);
-
- switch (ls) {
- case -2:
- if (plen >= 0)
- trace_seq_printf(s, format, plen, (char)val);
- else
- trace_seq_printf(s, format, (char)val);
- break;
- case -1:
- if (plen >= 0)
- trace_seq_printf(s, format, plen, (short)val);
- else
- trace_seq_printf(s, format, (short)val);
- break;
- case 0:
- if (plen >= 0)
- trace_seq_printf(s, format, plen, (int)val);
- else
- trace_seq_printf(s, format, (int)val);
- break;
- case 1:
- if (plen >= 0)
- trace_seq_printf(s, format, plen, (long)val);
- else
- trace_seq_printf(s, format, (long)val);
- break;
- case 2:
- if (plen >= 0)
- trace_seq_printf(s, format, plen, (long long)val);
- else
- trace_seq_printf(s, format, (long long)val);
- break;
- default:
- do_warning_event(event, "bad count (%d)", ls);
- event->flags |= TEP_EVENT_FL_FAILED;
- }
- return 0;
-}
-
-
-static void print_arg_string(struct trace_seq *s, const char *format, int plen,
- void *data, int size,
- struct tep_event *event, struct tep_print_arg *arg)
-{
- struct trace_seq p;
-
- /* Use helper trace_seq */
- trace_seq_init(&p);
- print_str_arg(&p, data, size, event,
- format, plen, arg);
- trace_seq_terminate(&p);
- trace_seq_puts(s, p.buffer);
- trace_seq_destroy(&p);
-}
-
-static int parse_arg_format_pointer(const char *format)
-{
- int ret = 0;
- int index;
- int loop;
-
- switch (*format) {
- case 'F':
- case 'S':
- case 'f':
- case 's':
- ret++;
- break;
- case 'M':
- case 'm':
- /* [mM]R , [mM]F */
- switch (format[1]) {
- case 'R':
- case 'F':
- ret++;
- break;
- }
- ret++;
- break;
- case 'I':
- case 'i':
- index = 2;
- loop = 1;
- switch (format[1]) {
- case 'S':
- /*[S][pfs]*/
- while (loop) {
- switch (format[index]) {
- case 'p':
- case 'f':
- case 's':
- ret++;
- index++;
- break;
- default:
- loop = 0;
- break;
- }
- }
- /* fall through */
- case '4':
- /* [4S][hnbl] */
- switch (format[index]) {
- case 'h':
- case 'n':
- case 'l':
- case 'b':
- ret++;
- index++;
- break;
- }
- if (format[1] == '4') {
- ret++;
- break;
- }
- /* fall through */
- case '6':
- /* [6S]c */
- if (format[index] == 'c')
- ret++;
- ret++;
- break;
- }
- ret++;
- break;
- case 'U':
- switch (format[1]) {
- case 'L':
- case 'l':
- case 'B':
- case 'b':
- ret++;
- break;
- }
- ret++;
- break;
- case 'h':
- switch (format[1]) {
- case 'C':
- case 'D':
- case 'N':
- ret++;
- break;
- }
- ret++;
- break;
- default:
- break;
- }
-
- return ret;
-}
-
-static void free_parse_args(struct tep_print_parse *arg)
-{
- struct tep_print_parse *del;
-
- while (arg) {
- del = arg;
- arg = del->next;
- free(del->format);
- free(del);
- }
-}
-
-static int parse_arg_add(struct tep_print_parse **parse, char *format,
- enum tep_print_parse_type type,
- struct tep_print_arg *arg,
- struct tep_print_arg *len_as_arg,
- int ls)
-{
- struct tep_print_parse *parg = NULL;
-
- parg = calloc(1, sizeof(*parg));
- if (!parg)
- goto error;
- parg->format = strdup(format);
- if (!parg->format)
- goto error;
- parg->type = type;
- parg->arg = arg;
- parg->len_as_arg = len_as_arg;
- parg->ls = ls;
- *parse = parg;
- return 0;
-error:
- if (parg) {
- free(parg->format);
- free(parg);
- }
- return -1;
-}
-
-static int parse_arg_format(struct tep_print_parse **parse,
- struct tep_event *event,
- const char *format, struct tep_print_arg **arg)
-{
- struct tep_print_arg *len_arg = NULL;
- char print_format[32];
- const char *start = format;
- int ret = 0;
- int ls = 0;
- int res;
- int len;
-
- format++;
- ret++;
- for (; *format; format++) {
- switch (*format) {
- case '#':
- /* FIXME: need to handle properly */
- break;
- case 'h':
- ls--;
- break;
- case 'l':
- ls++;
- break;
- case 'L':
- ls = 2;
- break;
- case '.':
- case 'z':
- case 'Z':
- case '0' ... '9':
- case '-':
- break;
- case '*':
- /* The argument is the length. */
- if (!*arg) {
- do_warning_event(event, "no argument match");
- event->flags |= TEP_EVENT_FL_FAILED;
- goto out_failed;
- }
- if (len_arg) {
- do_warning_event(event, "argument already matched");
- event->flags |= TEP_EVENT_FL_FAILED;
- goto out_failed;
- }
- len_arg = *arg;
- *arg = (*arg)->next;
- break;
- case 'p':
- if (!*arg) {
- do_warning_event(event, "no argument match");
- event->flags |= TEP_EVENT_FL_FAILED;
- goto out_failed;
- }
- res = parse_arg_format_pointer(format + 1);
- if (res > 0) {
- format += res;
- ret += res;
- }
- len = ((unsigned long)format + 1) -
- (unsigned long)start;
- /* should never happen */
- if (len > 31) {
- do_warning_event(event, "bad format!");
- event->flags |= TEP_EVENT_FL_FAILED;
- len = 31;
- }
- memcpy(print_format, start, len);
- print_format[len] = 0;
-
- parse_arg_add(parse, print_format,
- PRINT_FMT_ARG_POINTER, *arg, len_arg, ls);
- *arg = (*arg)->next;
- ret++;
- return ret;
- case 'd':
- case 'u':
- case 'i':
- case 'x':
- case 'X':
- case 'o':
- if (!*arg) {
- do_warning_event(event, "no argument match");
- event->flags |= TEP_EVENT_FL_FAILED;
- goto out_failed;
- }
-
- len = ((unsigned long)format + 1) -
- (unsigned long)start;
-
- /* should never happen */
- if (len > 30) {
- do_warning_event(event, "bad format!");
- event->flags |= TEP_EVENT_FL_FAILED;
- len = 31;
- }
- memcpy(print_format, start, len);
- print_format[len] = 0;
-
- if (event->tep->long_size == 8 && ls == 1 &&
- sizeof(long) != 8) {
- char *p;
-
- /* make %l into %ll */
- if (ls == 1 && (p = strchr(print_format, 'l')))
- memmove(p+1, p, strlen(p)+1);
- ls = 2;
- }
- if (ls < -2 || ls > 2) {
- do_warning_event(event, "bad count (%d)", ls);
- event->flags |= TEP_EVENT_FL_FAILED;
- }
- parse_arg_add(parse, print_format,
- PRINT_FMT_ARG_DIGIT, *arg, len_arg, ls);
- *arg = (*arg)->next;
- ret++;
- return ret;
- case 's':
- if (!*arg) {
- do_warning_event(event, "no matching argument");
- event->flags |= TEP_EVENT_FL_FAILED;
- goto out_failed;
- }
-
- len = ((unsigned long)format + 1) -
- (unsigned long)start;
-
- /* should never happen */
- if (len > 31) {
- do_warning_event(event, "bad format!");
- event->flags |= TEP_EVENT_FL_FAILED;
- len = 31;
- }
-
- memcpy(print_format, start, len);
- print_format[len] = 0;
-
- parse_arg_add(parse, print_format,
- PRINT_FMT_ARG_STRING, *arg, len_arg, 0);
- *arg = (*arg)->next;
- ret++;
- return ret;
- default:
- snprintf(print_format, 32, ">%c<", *format);
- parse_arg_add(parse, print_format,
- PRINT_FMT_STRING, NULL, NULL, 0);
- ret++;
- return ret;
- }
- ret++;
- }
-
-out_failed:
- return ret;
-
-}
-
-static int parse_arg_string(struct tep_print_parse **parse, const char *format)
-{
- struct trace_seq s;
- int ret = 0;
-
- trace_seq_init(&s);
- for (; *format; format++) {
- if (*format == '\\') {
- format++;
- ret++;
- switch (*format) {
- case 'n':
- trace_seq_putc(&s, '\n');
- break;
- case 't':
- trace_seq_putc(&s, '\t');
- break;
- case 'r':
- trace_seq_putc(&s, '\r');
- break;
- case '\\':
- trace_seq_putc(&s, '\\');
- break;
- default:
- trace_seq_putc(&s, *format);
- break;
- }
- } else if (*format == '%') {
- if (*(format + 1) == '%') {
- trace_seq_putc(&s, '%');
- format++;
- ret++;
- } else
- break;
- } else
- trace_seq_putc(&s, *format);
-
- ret++;
- }
- trace_seq_terminate(&s);
- parse_arg_add(parse, s.buffer, PRINT_FMT_STRING, NULL, NULL, 0);
- trace_seq_destroy(&s);
-
- return ret;
-}
-
-static struct tep_print_parse *
-parse_args(struct tep_event *event, const char *format, struct tep_print_arg *arg)
-{
- struct tep_print_parse *parse_ret = NULL;
- struct tep_print_parse **parse = NULL;
- int ret;
- int len;
-
- len = strlen(format);
- while (*format) {
- if (!parse_ret)
- parse = &parse_ret;
- if (*format == '%' && *(format + 1) != '%')
- ret = parse_arg_format(parse, event, format, &arg);
- else
- ret = parse_arg_string(parse, format);
- if (*parse)
- parse = &((*parse)->next);
-
- len -= ret;
- if (len > 0)
- format += ret;
- else
- break;
- }
- return parse_ret;
-}
-
-static void print_event_cache(struct tep_print_parse *parse, struct trace_seq *s,
- void *data, int size, struct tep_event *event)
-{
- int len_arg;
-
- while (parse) {
- if (parse->len_as_arg)
- len_arg = eval_num_arg(data, size, event, parse->len_as_arg);
- switch (parse->type) {
- case PRINT_FMT_ARG_DIGIT:
- print_arg_number(s, parse->format,
- parse->len_as_arg ? len_arg : -1, data,
- size, parse->ls, event, parse->arg);
- break;
- case PRINT_FMT_ARG_POINTER:
- print_arg_pointer(s, parse->format,
- parse->len_as_arg ? len_arg : 1,
- data, size, event, parse->arg);
- break;
- case PRINT_FMT_ARG_STRING:
- print_arg_string(s, parse->format,
- parse->len_as_arg ? len_arg : -1,
- data, size, event, parse->arg);
- break;
- case PRINT_FMT_STRING:
- default:
- trace_seq_printf(s, "%s", parse->format);
- break;
- }
- parse = parse->next;
- }
-}
-
-static void pretty_print(struct trace_seq *s, void *data, int size, struct tep_event *event)
-{
- struct tep_print_parse *parse = event->print_fmt.print_cache;
- struct tep_print_arg *args = NULL;
- char *bprint_fmt = NULL;
-
- if (event->flags & TEP_EVENT_FL_FAILED) {
- trace_seq_printf(s, "[FAILED TO PARSE]");
- tep_print_fields(s, data, size, event);
- return;
- }
-
- if (event->flags & TEP_EVENT_FL_ISBPRINT) {
- bprint_fmt = get_bprint_format(data, size, event);
- args = make_bprint_args(bprint_fmt, data, size, event);
- parse = parse_args(event, bprint_fmt, args);
- }
-
- print_event_cache(parse, s, data, size, event);
-
- if (event->flags & TEP_EVENT_FL_ISBPRINT) {
- free_parse_args(parse);
- free_args(args);
- free(bprint_fmt);
- }
-}
-
-/*
- * This parses out the Latency format (interrupts disabled,
- * need rescheduling, in hard/soft interrupt, preempt count
- * and lock depth) and places it into the trace_seq.
- */
-static void data_latency_format(struct tep_handle *tep, struct trace_seq *s,
- char *format, struct tep_record *record)
-{
- static int check_lock_depth = 1;
- static int check_migrate_disable = 1;
- static int lock_depth_exists;
- static int migrate_disable_exists;
- unsigned int lat_flags;
- struct trace_seq sq;
- unsigned int pc;
- int lock_depth = 0;
- int migrate_disable = 0;
- int hardirq;
- int softirq;
- void *data = record->data;
-
- trace_seq_init(&sq);
- lat_flags = parse_common_flags(tep, data);
- pc = parse_common_pc(tep, data);
- /* lock_depth may not always exist */
- if (lock_depth_exists)
- lock_depth = parse_common_lock_depth(tep, data);
- else if (check_lock_depth) {
- lock_depth = parse_common_lock_depth(tep, data);
- if (lock_depth < 0)
- check_lock_depth = 0;
- else
- lock_depth_exists = 1;
- }
-
- /* migrate_disable may not always exist */
- if (migrate_disable_exists)
- migrate_disable = parse_common_migrate_disable(tep, data);
- else if (check_migrate_disable) {
- migrate_disable = parse_common_migrate_disable(tep, data);
- if (migrate_disable < 0)
- check_migrate_disable = 0;
- else
- migrate_disable_exists = 1;
- }
-
- hardirq = lat_flags & TRACE_FLAG_HARDIRQ;
- softirq = lat_flags & TRACE_FLAG_SOFTIRQ;
-
- trace_seq_printf(&sq, "%c%c%c",
- (lat_flags & TRACE_FLAG_IRQS_OFF) ? 'd' :
- (lat_flags & TRACE_FLAG_IRQS_NOSUPPORT) ?
- 'X' : '.',
- (lat_flags & TRACE_FLAG_NEED_RESCHED) ?
- 'N' : '.',
- (hardirq && softirq) ? 'H' :
- hardirq ? 'h' : softirq ? 's' : '.');
-
- if (pc)
- trace_seq_printf(&sq, "%x", pc);
- else
- trace_seq_printf(&sq, ".");
-
- if (migrate_disable_exists) {
- if (migrate_disable < 0)
- trace_seq_printf(&sq, ".");
- else
- trace_seq_printf(&sq, "%d", migrate_disable);
- }
-
- if (lock_depth_exists) {
- if (lock_depth < 0)
- trace_seq_printf(&sq, ".");
- else
- trace_seq_printf(&sq, "%d", lock_depth);
- }
-
- if (sq.state == TRACE_SEQ__MEM_ALLOC_FAILED) {
- s->state = TRACE_SEQ__MEM_ALLOC_FAILED;
- return;
- }
-
- trace_seq_terminate(&sq);
- trace_seq_puts(s, sq.buffer);
- trace_seq_destroy(&sq);
- trace_seq_terminate(s);
-}
-
-/**
- * tep_data_type - parse out the given event type
- * @tep: a handle to the trace event parser context
- * @rec: the record to read from
- *
- * This returns the event id from the @rec.
- */
-int tep_data_type(struct tep_handle *tep, struct tep_record *rec)
-{
- return trace_parse_common_type(tep, rec->data);
-}
-
-/**
- * tep_data_pid - parse the PID from record
- * @tep: a handle to the trace event parser context
- * @rec: the record to parse
- *
- * This returns the PID from a record.
- */
-int tep_data_pid(struct tep_handle *tep, struct tep_record *rec)
-{
- return parse_common_pid(tep, rec->data);
-}
-
-/**
- * tep_data_preempt_count - parse the preempt count from the record
- * @tep: a handle to the trace event parser context
- * @rec: the record to parse
- *
- * This returns the preempt count from a record.
- */
-int tep_data_preempt_count(struct tep_handle *tep, struct tep_record *rec)
-{
- return parse_common_pc(tep, rec->data);
-}
-
-/**
- * tep_data_flags - parse the latency flags from the record
- * @tep: a handle to the trace event parser context
- * @rec: the record to parse
- *
- * This returns the latency flags from a record.
- *
- * Use trace_flag_type enum for the flags (see event-parse.h).
- */
-int tep_data_flags(struct tep_handle *tep, struct tep_record *rec)
-{
- return parse_common_flags(tep, rec->data);
-}
-
-/**
- * tep_data_comm_from_pid - return the command line from PID
- * @tep: a handle to the trace event parser context
- * @pid: the PID of the task to search for
- *
- * This returns a pointer to the command line that has the given
- * @pid.
- */
-const char *tep_data_comm_from_pid(struct tep_handle *tep, int pid)
-{
- const char *comm;
-
- comm = find_cmdline(tep, pid);
- return comm;
-}
-
-static struct tep_cmdline *
-pid_from_cmdlist(struct tep_handle *tep, const char *comm, struct tep_cmdline *next)
-{
- struct cmdline_list *cmdlist = (struct cmdline_list *)next;
-
- if (cmdlist)
- cmdlist = cmdlist->next;
- else
- cmdlist = tep->cmdlist;
-
- while (cmdlist && strcmp(cmdlist->comm, comm) != 0)
- cmdlist = cmdlist->next;
-
- return (struct tep_cmdline *)cmdlist;
-}
-
-/**
- * tep_data_pid_from_comm - return the pid from a given comm
- * @tep: a handle to the trace event parser context
- * @comm: the cmdline to find the pid from
- * @next: the cmdline structure to find the next comm
- *
- * This returns the cmdline structure that holds a pid for a given
- * comm, or NULL if none found. As there may be more than one pid for
- * a given comm, the result of this call can be passed back into
- * a recurring call in the @next parameter, and then it will find the
- * next pid.
- * Also, it does a linear search, so it may be slow.
- */
-struct tep_cmdline *tep_data_pid_from_comm(struct tep_handle *tep, const char *comm,
- struct tep_cmdline *next)
-{
- struct tep_cmdline *cmdline;
-
- /*
- * If the cmdlines have not been converted yet, then use
- * the list.
- */
- if (!tep->cmdlines)
- return pid_from_cmdlist(tep, comm, next);
-
- if (next) {
- /*
- * The next pointer could have been still from
- * a previous call before cmdlines were created
- */
- if (next < tep->cmdlines ||
- next >= tep->cmdlines + tep->cmdline_count)
- next = NULL;
- else
- cmdline = next++;
- }
-
- if (!next)
- cmdline = tep->cmdlines;
-
- while (cmdline < tep->cmdlines + tep->cmdline_count) {
- if (strcmp(cmdline->comm, comm) == 0)
- return cmdline;
- cmdline++;
- }
- return NULL;
-}
-
-/**
- * tep_cmdline_pid - return the pid associated to a given cmdline
- * @tep: a handle to the trace event parser context
- * @cmdline: The cmdline structure to get the pid from
- *
- * Returns the pid for a give cmdline. If @cmdline is NULL, then
- * -1 is returned.
- */
-int tep_cmdline_pid(struct tep_handle *tep, struct tep_cmdline *cmdline)
-{
- struct cmdline_list *cmdlist = (struct cmdline_list *)cmdline;
-
- if (!cmdline)
- return -1;
-
- /*
- * If cmdlines have not been created yet, or cmdline is
- * not part of the array, then treat it as a cmdlist instead.
- */
- if (!tep->cmdlines ||
- cmdline < tep->cmdlines ||
- cmdline >= tep->cmdlines + tep->cmdline_count)
- return cmdlist->pid;
-
- return cmdline->pid;
-}
-
-/*
- * This parses the raw @data using the given @event information and
- * writes the print format into the trace_seq.
- */
-static void print_event_info(struct trace_seq *s, char *format, bool raw,
- struct tep_event *event, struct tep_record *record)
-{
- int print_pretty = 1;
-
- if (raw || (event->flags & TEP_EVENT_FL_PRINTRAW))
- tep_print_fields(s, record->data, record->size, event);
- else {
-
- if (event->handler && !(event->flags & TEP_EVENT_FL_NOHANDLE))
- print_pretty = event->handler(s, record, event,
- event->context);
-
- if (print_pretty)
- pretty_print(s, record->data, record->size, event);
- }
-
- trace_seq_terminate(s);
-}
-
-/**
- * tep_find_event_by_record - return the event from a given record
- * @tep: a handle to the trace event parser context
- * @record: The record to get the event from
- *
- * Returns the associated event for a given record, or NULL if non is
- * is found.
- */
-struct tep_event *
-tep_find_event_by_record(struct tep_handle *tep, struct tep_record *record)
-{
- int type;
-
- if (record->size < 0) {
- do_warning("ug! negative record size %d", record->size);
- return NULL;
- }
-
- type = trace_parse_common_type(tep, record->data);
-
- return tep_find_event(tep, type);
-}
-
-/*
- * Writes the timestamp of the record into @s. Time divisor and precision can be
- * specified as part of printf @format string. Example:
- * "%3.1000d" - divide the time by 1000 and print the first 3 digits
- * before the dot. Thus, the timestamp "123456000" will be printed as
- * "123.456"
- */
-static void print_event_time(struct tep_handle *tep, struct trace_seq *s,
- char *format, struct tep_event *event,
- struct tep_record *record)
-{
- unsigned long long time;
- char *divstr;
- int prec = 0, pr;
- int div = 0;
- int p10 = 1;
-
- if (isdigit(*(format + 1)))
- prec = atoi(format + 1);
- divstr = strchr(format, '.');
- if (divstr && isdigit(*(divstr + 1)))
- div = atoi(divstr + 1);
- time = record->ts;
- if (div) {
- time += div / 2;
- time /= div;
- }
- pr = prec;
- while (pr--)
- p10 *= 10;
-
- if (p10 > 1 && p10 < time)
- trace_seq_printf(s, "%5llu.%0*llu", time / p10, prec, time % p10);
- else
- trace_seq_printf(s, "%12llu", time);
-}
-
-struct print_event_type {
- enum {
- EVENT_TYPE_INT = 1,
- EVENT_TYPE_STRING,
- EVENT_TYPE_UNKNOWN,
- } type;
- char format[32];
-};
-
-static void print_string(struct tep_handle *tep, struct trace_seq *s,
- struct tep_record *record, struct tep_event *event,
- const char *arg, struct print_event_type *type)
-{
- const char *comm;
- int pid;
-
- if (strncmp(arg, TEP_PRINT_LATENCY, strlen(TEP_PRINT_LATENCY)) == 0) {
- data_latency_format(tep, s, type->format, record);
- } else if (strncmp(arg, TEP_PRINT_COMM, strlen(TEP_PRINT_COMM)) == 0) {
- pid = parse_common_pid(tep, record->data);
- comm = find_cmdline(tep, pid);
- trace_seq_printf(s, type->format, comm);
- } else if (strncmp(arg, TEP_PRINT_INFO_RAW, strlen(TEP_PRINT_INFO_RAW)) == 0) {
- print_event_info(s, type->format, true, event, record);
- } else if (strncmp(arg, TEP_PRINT_INFO, strlen(TEP_PRINT_INFO)) == 0) {
- print_event_info(s, type->format, false, event, record);
- } else if (strncmp(arg, TEP_PRINT_NAME, strlen(TEP_PRINT_NAME)) == 0) {
- trace_seq_printf(s, type->format, event->name);
- } else {
- trace_seq_printf(s, "[UNKNOWN TEP TYPE %s]", arg);
- }
-
-}
-
-static void print_int(struct tep_handle *tep, struct trace_seq *s,
- struct tep_record *record, struct tep_event *event,
- int arg, struct print_event_type *type)
-{
- int param;
-
- switch (arg) {
- case TEP_PRINT_CPU:
- param = record->cpu;
- break;
- case TEP_PRINT_PID:
- param = parse_common_pid(tep, record->data);
- break;
- case TEP_PRINT_TIME:
- return print_event_time(tep, s, type->format, event, record);
- default:
- return;
- }
- trace_seq_printf(s, type->format, param);
-}
-
-static int tep_print_event_param_type(char *format,
- struct print_event_type *type)
-{
- char *str = format + 1;
- int i = 1;
-
- type->type = EVENT_TYPE_UNKNOWN;
- while (*str) {
- switch (*str) {
- case 'd':
- case 'u':
- case 'i':
- case 'x':
- case 'X':
- case 'o':
- type->type = EVENT_TYPE_INT;
- break;
- case 's':
- type->type = EVENT_TYPE_STRING;
- break;
- }
- str++;
- i++;
- if (type->type != EVENT_TYPE_UNKNOWN)
- break;
- }
- memset(type->format, 0, 32);
- memcpy(type->format, format, i < 32 ? i : 31);
- return i;
-}
-
-/**
- * tep_print_event - Write various event information
- * @tep: a handle to the trace event parser context
- * @s: the trace_seq to write to
- * @record: The record to get the event from
- * @format: a printf format string. Supported event fileds:
- * TEP_PRINT_PID, "%d" - event PID
- * TEP_PRINT_CPU, "%d" - event CPU
- * TEP_PRINT_COMM, "%s" - event command string
- * TEP_PRINT_NAME, "%s" - event name
- * TEP_PRINT_LATENCY, "%s" - event latency
- * TEP_PRINT_TIME, %d - event time stamp. A divisor and precision
- * can be specified as part of this format string:
- * "%precision.divisord". Example:
- * "%3.1000d" - divide the time by 1000 and print the first
- * 3 digits before the dot. Thus, the time stamp
- * "123456000" will be printed as "123.456"
- * TEP_PRINT_INFO, "%s" - event information. If any width is specified in
- * the format string, the event information will be printed
- * in raw format.
- * Writes the specified event information into @s.
- */
-void tep_print_event(struct tep_handle *tep, struct trace_seq *s,
- struct tep_record *record, const char *fmt, ...)
-{
- struct print_event_type type;
- char *format = strdup(fmt);
- char *current = format;
- char *str = format;
- int offset;
- va_list args;
- struct tep_event *event;
-
- if (!format)
- return;
-
- event = tep_find_event_by_record(tep, record);
- va_start(args, fmt);
- while (*current) {
- current = strchr(str, '%');
- if (!current) {
- trace_seq_puts(s, str);
- break;
- }
- memset(&type, 0, sizeof(type));
- offset = tep_print_event_param_type(current, &type);
- *current = '\0';
- trace_seq_puts(s, str);
- current += offset;
- switch (type.type) {
- case EVENT_TYPE_STRING:
- print_string(tep, s, record, event,
- va_arg(args, char*), &type);
- break;
- case EVENT_TYPE_INT:
- print_int(tep, s, record, event,
- va_arg(args, int), &type);
- break;
- case EVENT_TYPE_UNKNOWN:
- default:
- trace_seq_printf(s, "[UNKNOWN TYPE]");
- break;
- }
- str = current;
-
- }
- va_end(args);
- free(format);
-}
-
-static int events_id_cmp(const void *a, const void *b)
-{
- struct tep_event * const * ea = a;
- struct tep_event * const * eb = b;
-
- if ((*ea)->id < (*eb)->id)
- return -1;
-
- if ((*ea)->id > (*eb)->id)
- return 1;
-
- return 0;
-}
-
-static int events_name_cmp(const void *a, const void *b)
-{
- struct tep_event * const * ea = a;
- struct tep_event * const * eb = b;
- int res;
-
- res = strcmp((*ea)->name, (*eb)->name);
- if (res)
- return res;
-
- res = strcmp((*ea)->system, (*eb)->system);
- if (res)
- return res;
-
- return events_id_cmp(a, b);
-}
-
-static int events_system_cmp(const void *a, const void *b)
-{
- struct tep_event * const * ea = a;
- struct tep_event * const * eb = b;
- int res;
-
- res = strcmp((*ea)->system, (*eb)->system);
- if (res)
- return res;
-
- res = strcmp((*ea)->name, (*eb)->name);
- if (res)
- return res;
-
- return events_id_cmp(a, b);
-}
-
-static struct tep_event **list_events_copy(struct tep_handle *tep)
-{
- struct tep_event **events;
-
- if (!tep)
- return NULL;
-
- events = malloc(sizeof(*events) * (tep->nr_events + 1));
- if (!events)
- return NULL;
-
- memcpy(events, tep->events, sizeof(*events) * tep->nr_events);
- events[tep->nr_events] = NULL;
- return events;
-}
-
-static void list_events_sort(struct tep_event **events, int nr_events,
- enum tep_event_sort_type sort_type)
-{
- int (*sort)(const void *a, const void *b);
-
- switch (sort_type) {
- case TEP_EVENT_SORT_ID:
- sort = events_id_cmp;
- break;
- case TEP_EVENT_SORT_NAME:
- sort = events_name_cmp;
- break;
- case TEP_EVENT_SORT_SYSTEM:
- sort = events_system_cmp;
- break;
- default:
- sort = NULL;
- }
-
- if (sort)
- qsort(events, nr_events, sizeof(*events), sort);
-}
-
-/**
- * tep_list_events - Get events, sorted by given criteria.
- * @tep: a handle to the tep context
- * @sort_type: desired sort order of the events in the array
- *
- * Returns an array of pointers to all events, sorted by the given
- * @sort_type criteria. The last element of the array is NULL. The returned
- * memory must not be freed, it is managed by the library.
- * The function is not thread safe.
- */
-struct tep_event **tep_list_events(struct tep_handle *tep,
- enum tep_event_sort_type sort_type)
-{
- struct tep_event **events;
-
- if (!tep)
- return NULL;
-
- events = tep->sort_events;
- if (events && tep->last_type == sort_type)
- return events;
-
- if (!events) {
- events = list_events_copy(tep);
- if (!events)
- return NULL;
-
- tep->sort_events = events;
-
- /* the internal events are sorted by id */
- if (sort_type == TEP_EVENT_SORT_ID) {
- tep->last_type = sort_type;
- return events;
- }
- }
-
- list_events_sort(events, tep->nr_events, sort_type);
- tep->last_type = sort_type;
-
- return events;
-}
-
-
-/**
- * tep_list_events_copy - Thread safe version of tep_list_events()
- * @tep: a handle to the tep context
- * @sort_type: desired sort order of the events in the array
- *
- * Returns an array of pointers to all events, sorted by the given
- * @sort_type criteria. The last element of the array is NULL. The returned
- * array is newly allocated inside the function and must be freed by the caller
- */
-struct tep_event **tep_list_events_copy(struct tep_handle *tep,
- enum tep_event_sort_type sort_type)
-{
- struct tep_event **events;
-
- if (!tep)
- return NULL;
-
- events = list_events_copy(tep);
- if (!events)
- return NULL;
-
- /* the internal events are sorted by id */
- if (sort_type == TEP_EVENT_SORT_ID)
- return events;
-
- list_events_sort(events, tep->nr_events, sort_type);
-
- return events;
-}
-
-static struct tep_format_field **
-get_event_fields(const char *type, const char *name,
- int count, struct tep_format_field *list)
-{
- struct tep_format_field **fields;
- struct tep_format_field *field;
- int i = 0;
-
- fields = malloc(sizeof(*fields) * (count + 1));
- if (!fields)
- return NULL;
-
- for (field = list; field; field = field->next) {
- fields[i++] = field;
- if (i == count + 1) {
- do_warning("event %s has more %s fields than specified",
- name, type);
- i--;
- break;
- }
- }
-
- if (i != count)
- do_warning("event %s has less %s fields than specified",
- name, type);
-
- fields[i] = NULL;
-
- return fields;
-}
-
-/**
- * tep_event_common_fields - return a list of common fields for an event
- * @event: the event to return the common fields of.
- *
- * Returns an allocated array of fields. The last item in the array is NULL.
- * The array must be freed with free().
- */
-struct tep_format_field **tep_event_common_fields(struct tep_event *event)
-{
- return get_event_fields("common", event->name,
- event->format.nr_common,
- event->format.common_fields);
-}
-
-/**
- * tep_event_fields - return a list of event specific fields for an event
- * @event: the event to return the fields of.
- *
- * Returns an allocated array of fields. The last item in the array is NULL.
- * The array must be freed with free().
- */
-struct tep_format_field **tep_event_fields(struct tep_event *event)
-{
- return get_event_fields("event", event->name,
- event->format.nr_fields,
- event->format.fields);
-}
-
-static void print_fields(struct trace_seq *s, struct tep_print_flag_sym *field)
-{
- trace_seq_printf(s, "{ %s, %s }", field->value, field->str);
- if (field->next) {
- trace_seq_puts(s, ", ");
- print_fields(s, field->next);
- }
-}
-
-/* for debugging */
-static void print_args(struct tep_print_arg *args)
-{
- int print_paren = 1;
- struct trace_seq s;
-
- switch (args->type) {
- case TEP_PRINT_NULL:
- printf("null");
- break;
- case TEP_PRINT_ATOM:
- printf("%s", args->atom.atom);
- break;
- case TEP_PRINT_FIELD:
- printf("REC->%s", args->field.name);
- break;
- case TEP_PRINT_FLAGS:
- printf("__print_flags(");
- print_args(args->flags.field);
- printf(", %s, ", args->flags.delim);
- trace_seq_init(&s);
- print_fields(&s, args->flags.flags);
- trace_seq_do_printf(&s);
- trace_seq_destroy(&s);
- printf(")");
- break;
- case TEP_PRINT_SYMBOL:
- printf("__print_symbolic(");
- print_args(args->symbol.field);
- printf(", ");
- trace_seq_init(&s);
- print_fields(&s, args->symbol.symbols);
- trace_seq_do_printf(&s);
- trace_seq_destroy(&s);
- printf(")");
- break;
- case TEP_PRINT_HEX:
- printf("__print_hex(");
- print_args(args->hex.field);
- printf(", ");
- print_args(args->hex.size);
- printf(")");
- break;
- case TEP_PRINT_HEX_STR:
- printf("__print_hex_str(");
- print_args(args->hex.field);
- printf(", ");
- print_args(args->hex.size);
- printf(")");
- break;
- case TEP_PRINT_INT_ARRAY:
- printf("__print_array(");
- print_args(args->int_array.field);
- printf(", ");
- print_args(args->int_array.count);
- printf(", ");
- print_args(args->int_array.el_size);
- printf(")");
- break;
- case TEP_PRINT_STRING:
- case TEP_PRINT_BSTRING:
- printf("__get_str(%s)", args->string.string);
- break;
- case TEP_PRINT_BITMASK:
- printf("__get_bitmask(%s)", args->bitmask.bitmask);
- break;
- case TEP_PRINT_TYPE:
- printf("(%s)", args->typecast.type);
- print_args(args->typecast.item);
- break;
- case TEP_PRINT_OP:
- if (strcmp(args->op.op, ":") == 0)
- print_paren = 0;
- if (print_paren)
- printf("(");
- print_args(args->op.left);
- printf(" %s ", args->op.op);
- print_args(args->op.right);
- if (print_paren)
- printf(")");
- break;
- default:
- /* we should warn... */
- return;
- }
- if (args->next) {
- printf("\n");
- print_args(args->next);
- }
-}
-
-static void parse_header_field(const char *field,
- int *offset, int *size, int mandatory)
-{
- unsigned long long save_input_buf_ptr;
- unsigned long long save_input_buf_siz;
- char *token;
- int type;
-
- save_input_buf_ptr = input_buf_ptr;
- save_input_buf_siz = input_buf_siz;
-
- if (read_expected(TEP_EVENT_ITEM, "field") < 0)
- return;
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- return;
-
- /* type */
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto fail;
- free_token(token);
-
- /*
- * If this is not a mandatory field, then test it first.
- */
- if (mandatory) {
- if (read_expected(TEP_EVENT_ITEM, field) < 0)
- return;
- } else {
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto fail;
- if (strcmp(token, field) != 0)
- goto discard;
- free_token(token);
- }
-
- if (read_expected(TEP_EVENT_OP, ";") < 0)
- return;
- if (read_expected(TEP_EVENT_ITEM, "offset") < 0)
- return;
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- return;
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto fail;
- *offset = atoi(token);
- free_token(token);
- if (read_expected(TEP_EVENT_OP, ";") < 0)
- return;
- if (read_expected(TEP_EVENT_ITEM, "size") < 0)
- return;
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- return;
- if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
- goto fail;
- *size = atoi(token);
- free_token(token);
- if (read_expected(TEP_EVENT_OP, ";") < 0)
- return;
- type = read_token(&token);
- if (type != TEP_EVENT_NEWLINE) {
- /* newer versions of the kernel have a "signed" type */
- if (type != TEP_EVENT_ITEM)
- goto fail;
-
- if (strcmp(token, "signed") != 0)
- goto fail;
-
- free_token(token);
-
- if (read_expected(TEP_EVENT_OP, ":") < 0)
- return;
-
- if (read_expect_type(TEP_EVENT_ITEM, &token))
- goto fail;
-
- free_token(token);
- if (read_expected(TEP_EVENT_OP, ";") < 0)
- return;
-
- if (read_expect_type(TEP_EVENT_NEWLINE, &token))
- goto fail;
- }
- fail:
- free_token(token);
- return;
-
- discard:
- input_buf_ptr = save_input_buf_ptr;
- input_buf_siz = save_input_buf_siz;
- *offset = 0;
- *size = 0;
- free_token(token);
-}
-
-/**
- * tep_parse_header_page - parse the data stored in the header page
- * @tep: a handle to the trace event parser context
- * @buf: the buffer storing the header page format string
- * @size: the size of @buf
- * @long_size: the long size to use if there is no header
- *
- * This parses the header page format for information on the
- * ring buffer used. The @buf should be copied from
- *
- * /sys/kernel/debug/tracing/events/header_page
- */
-int tep_parse_header_page(struct tep_handle *tep, char *buf, unsigned long size,
- int long_size)
-{
- int ignore;
-
- if (!size) {
- /*
- * Old kernels did not have header page info.
- * Sorry but we just use what we find here in user space.
- */
- tep->header_page_ts_size = sizeof(long long);
- tep->header_page_size_size = long_size;
- tep->header_page_data_offset = sizeof(long long) + long_size;
- tep->old_format = 1;
- return -1;
- }
- init_input_buf(buf, size);
-
- parse_header_field("timestamp", &tep->header_page_ts_offset,
- &tep->header_page_ts_size, 1);
- parse_header_field("commit", &tep->header_page_size_offset,
- &tep->header_page_size_size, 1);
- parse_header_field("overwrite", &tep->header_page_overwrite,
- &ignore, 0);
- parse_header_field("data", &tep->header_page_data_offset,
- &tep->header_page_data_size, 1);
-
- return 0;
-}
-
-static int event_matches(struct tep_event *event,
- int id, const char *sys_name,
- const char *event_name)
-{
- if (id >= 0 && id != event->id)
- return 0;
-
- if (event_name && (strcmp(event_name, event->name) != 0))
- return 0;
-
- if (sys_name && (strcmp(sys_name, event->system) != 0))
- return 0;
-
- return 1;
-}
-
-static void free_handler(struct event_handler *handle)
-{
- free((void *)handle->sys_name);
- free((void *)handle->event_name);
- free(handle);
-}
-
-static int find_event_handle(struct tep_handle *tep, struct tep_event *event)
-{
- struct event_handler *handle, **next;
-
- for (next = &tep->handlers; *next;
- next = &(*next)->next) {
- handle = *next;
- if (event_matches(event, handle->id,
- handle->sys_name,
- handle->event_name))
- break;
- }
-
- if (!(*next))
- return 0;
-
- pr_stat("overriding event (%d) %s:%s with new print handler",
- event->id, event->system, event->name);
-
- event->handler = handle->func;
- event->context = handle->context;
-
- *next = handle->next;
- free_handler(handle);
-
- return 1;
-}
-
-/**
- * parse_format - parse the event format
- * @buf: the buffer storing the event format string
- * @size: the size of @buf
- * @sys: the system the event belongs to
- *
- * This parses the event format and creates an event structure
- * to quickly parse raw data for a given event.
- *
- * These files currently come from:
- *
- * /sys/kernel/debug/tracing/events/.../.../format
- */
-static enum tep_errno parse_format(struct tep_event **eventp,
- struct tep_handle *tep, const char *buf,
- unsigned long size, const char *sys)
-{
- struct tep_event *event;
- int ret;
-
- init_input_buf(buf, size);
-
- *eventp = event = alloc_event();
- if (!event)
- return TEP_ERRNO__MEM_ALLOC_FAILED;
-
- event->name = event_read_name();
- if (!event->name) {
- /* Bad event? */
- ret = TEP_ERRNO__MEM_ALLOC_FAILED;
- goto event_alloc_failed;
- }
-
- if (strcmp(sys, "ftrace") == 0) {
- event->flags |= TEP_EVENT_FL_ISFTRACE;
-
- if (strcmp(event->name, "bprint") == 0)
- event->flags |= TEP_EVENT_FL_ISBPRINT;
- }
-
- event->id = event_read_id();
- if (event->id < 0) {
- ret = TEP_ERRNO__READ_ID_FAILED;
- /*
- * This isn't an allocation error actually.
- * But as the ID is critical, just bail out.
- */
- goto event_alloc_failed;
- }
-
- event->system = strdup(sys);
- if (!event->system) {
- ret = TEP_ERRNO__MEM_ALLOC_FAILED;
- goto event_alloc_failed;
- }
-
- /* Add tep to event so that it can be referenced */
- event->tep = tep;
-
- ret = event_read_format(event);
- if (ret < 0) {
- ret = TEP_ERRNO__READ_FORMAT_FAILED;
- goto event_parse_failed;
- }
-
- /*
- * If the event has an override, don't print warnings if the event
- * print format fails to parse.
- */
- if (tep && find_event_handle(tep, event))
- show_warning = 0;
-
- ret = event_read_print(event);
- show_warning = 1;
-
- if (ret < 0) {
- ret = TEP_ERRNO__READ_PRINT_FAILED;
- goto event_parse_failed;
- }
-
- if (!ret && (event->flags & TEP_EVENT_FL_ISFTRACE)) {
- struct tep_format_field *field;
- struct tep_print_arg *arg, **list;
-
- /* old ftrace had no args */
- list = &event->print_fmt.args;
- for (field = event->format.fields; field; field = field->next) {
- arg = alloc_arg();
- if (!arg) {
- event->flags |= TEP_EVENT_FL_FAILED;
- return TEP_ERRNO__OLD_FTRACE_ARG_FAILED;
- }
- arg->type = TEP_PRINT_FIELD;
- arg->field.name = strdup(field->name);
- if (!arg->field.name) {
- event->flags |= TEP_EVENT_FL_FAILED;
- free_arg(arg);
- return TEP_ERRNO__OLD_FTRACE_ARG_FAILED;
- }
- arg->field.field = field;
- *list = arg;
- list = &arg->next;
- }
- }
-
- if (!(event->flags & TEP_EVENT_FL_ISBPRINT))
- event->print_fmt.print_cache = parse_args(event,
- event->print_fmt.format,
- event->print_fmt.args);
-
- return 0;
-
- event_parse_failed:
- event->flags |= TEP_EVENT_FL_FAILED;
- return ret;
-
- event_alloc_failed:
- free(event->system);
- free(event->name);
- free(event);
- *eventp = NULL;
- return ret;
-}
-
-static enum tep_errno
-__parse_event(struct tep_handle *tep,
- struct tep_event **eventp,
- const char *buf, unsigned long size,
- const char *sys)
-{
- int ret = parse_format(eventp, tep, buf, size, sys);
- struct tep_event *event = *eventp;
-
- if (event == NULL)
- return ret;
-
- if (tep && add_event(tep, event)) {
- ret = TEP_ERRNO__MEM_ALLOC_FAILED;
- goto event_add_failed;
- }
-
-#define PRINT_ARGS 0
- if (PRINT_ARGS && event->print_fmt.args)
- print_args(event->print_fmt.args);
-
- return 0;
-
-event_add_failed:
- free_tep_event(event);
- return ret;
-}
-
-/**
- * tep_parse_format - parse the event format
- * @tep: a handle to the trace event parser context
- * @eventp: returned format
- * @buf: the buffer storing the event format string
- * @size: the size of @buf
- * @sys: the system the event belongs to
- *
- * This parses the event format and creates an event structure
- * to quickly parse raw data for a given event.
- *
- * These files currently come from:
- *
- * /sys/kernel/debug/tracing/events/.../.../format
- */
-enum tep_errno tep_parse_format(struct tep_handle *tep,
- struct tep_event **eventp,
- const char *buf,
- unsigned long size, const char *sys)
-{
- return __parse_event(tep, eventp, buf, size, sys);
-}
-
-/**
- * tep_parse_event - parse the event format
- * @tep: a handle to the trace event parser context
- * @buf: the buffer storing the event format string
- * @size: the size of @buf
- * @sys: the system the event belongs to
- *
- * This parses the event format and creates an event structure
- * to quickly parse raw data for a given event.
- *
- * These files currently come from:
- *
- * /sys/kernel/debug/tracing/events/.../.../format
- */
-enum tep_errno tep_parse_event(struct tep_handle *tep, const char *buf,
- unsigned long size, const char *sys)
-{
- struct tep_event *event = NULL;
- return __parse_event(tep, &event, buf, size, sys);
-}
-
-int get_field_val(struct trace_seq *s, struct tep_format_field *field,
- const char *name, struct tep_record *record,
- unsigned long long *val, int err)
-{
- if (!field) {
- if (err)
- trace_seq_printf(s, "<CANT FIND FIELD %s>", name);
- return -1;
- }
-
- if (tep_read_number_field(field, record->data, val)) {
- if (err)
- trace_seq_printf(s, " %s=INVALID", name);
- return -1;
- }
-
- return 0;
-}
-
-/**
- * tep_get_field_raw - return the raw pointer into the data field
- * @s: The seq to print to on error
- * @event: the event that the field is for
- * @name: The name of the field
- * @record: The record with the field name.
- * @len: place to store the field length.
- * @err: print default error if failed.
- *
- * Returns a pointer into record->data of the field and places
- * the length of the field in @len.
- *
- * On failure, it returns NULL.
- */
-void *tep_get_field_raw(struct trace_seq *s, struct tep_event *event,
- const char *name, struct tep_record *record,
- int *len, int err)
-{
- struct tep_format_field *field;
- void *data = record->data;
- unsigned offset;
- int dummy;
-
- if (!event)
- return NULL;
-
- field = tep_find_field(event, name);
-
- if (!field) {
- if (err)
- trace_seq_printf(s, "<CANT FIND FIELD %s>", name);
- return NULL;
- }
-
- /* Allow @len to be NULL */
- if (!len)
- len = &dummy;
-
- offset = field->offset;
- if (field->flags & TEP_FIELD_IS_DYNAMIC) {
- offset = tep_read_number(event->tep,
- data + offset, field->size);
- *len = offset >> 16;
- offset &= 0xffff;
- if (field->flags & TEP_FIELD_IS_RELATIVE)
- offset += field->offset + field->size;
- } else
- *len = field->size;
-
- return data + offset;
-}
-
-/**
- * tep_get_field_val - find a field and return its value
- * @s: The seq to print to on error
- * @event: the event that the field is for
- * @name: The name of the field
- * @record: The record with the field name.
- * @val: place to store the value of the field.
- * @err: print default error if failed.
- *
- * Returns 0 on success -1 on field not found.
- */
-int tep_get_field_val(struct trace_seq *s, struct tep_event *event,
- const char *name, struct tep_record *record,
- unsigned long long *val, int err)
-{
- struct tep_format_field *field;
-
- if (!event)
- return -1;
-
- field = tep_find_field(event, name);
-
- return get_field_val(s, field, name, record, val, err);
-}
-
-/**
- * tep_get_common_field_val - find a common field and return its value
- * @s: The seq to print to on error
- * @event: the event that the field is for
- * @name: The name of the field
- * @record: The record with the field name.
- * @val: place to store the value of the field.
- * @err: print default error if failed.
- *
- * Returns 0 on success -1 on field not found.
- */
-int tep_get_common_field_val(struct trace_seq *s, struct tep_event *event,
- const char *name, struct tep_record *record,
- unsigned long long *val, int err)
-{
- struct tep_format_field *field;
-
- if (!event)
- return -1;
-
- field = tep_find_common_field(event, name);
-
- return get_field_val(s, field, name, record, val, err);
-}
-
-/**
- * tep_get_any_field_val - find a any field and return its value
- * @s: The seq to print to on error
- * @event: the event that the field is for
- * @name: The name of the field
- * @record: The record with the field name.
- * @val: place to store the value of the field.
- * @err: print default error if failed.
- *
- * Returns 0 on success -1 on field not found.
- */
-int tep_get_any_field_val(struct trace_seq *s, struct tep_event *event,
- const char *name, struct tep_record *record,
- unsigned long long *val, int err)
-{
- struct tep_format_field *field;
-
- if (!event)
- return -1;
-
- field = tep_find_any_field(event, name);
-
- return get_field_val(s, field, name, record, val, err);
-}
-
-/**
- * tep_print_num_field - print a field and a format
- * @s: The seq to print to
- * @fmt: The printf format to print the field with.
- * @event: the event that the field is for
- * @name: The name of the field
- * @record: The record with the field name.
- * @err: print default error if failed.
- *
- * Returns positive value on success, negative in case of an error,
- * or 0 if buffer is full.
- */
-int tep_print_num_field(struct trace_seq *s, const char *fmt,
- struct tep_event *event, const char *name,
- struct tep_record *record, int err)
-{
- struct tep_format_field *field = tep_find_field(event, name);
- unsigned long long val;
-
- if (!field)
- goto failed;
-
- if (tep_read_number_field(field, record->data, &val))
- goto failed;
-
- return trace_seq_printf(s, fmt, val);
-
- failed:
- if (err)
- trace_seq_printf(s, "CAN'T FIND FIELD \"%s\"", name);
- return -1;
-}
-
-/**
- * tep_print_func_field - print a field and a format for function pointers
- * @s: The seq to print to
- * @fmt: The printf format to print the field with.
- * @event: the event that the field is for
- * @name: The name of the field
- * @record: The record with the field name.
- * @err: print default error if failed.
- *
- * Returns positive value on success, negative in case of an error,
- * or 0 if buffer is full.
- */
-int tep_print_func_field(struct trace_seq *s, const char *fmt,
- struct tep_event *event, const char *name,
- struct tep_record *record, int err)
-{
- struct tep_format_field *field = tep_find_field(event, name);
- struct tep_handle *tep = event->tep;
- unsigned long long val;
- struct func_map *func;
- char tmp[128];
-
- if (!field)
- goto failed;
-
- if (tep_read_number_field(field, record->data, &val))
- goto failed;
-
- func = find_func(tep, val);
-
- if (func)
- snprintf(tmp, 128, "%s/0x%llx", func->func, func->addr - val);
- else
- sprintf(tmp, "0x%08llx", val);
-
- return trace_seq_printf(s, fmt, tmp);
-
- failed:
- if (err)
- trace_seq_printf(s, "CAN'T FIND FIELD \"%s\"", name);
- return -1;
-}
-
-static void free_func_handle(struct tep_function_handler *func)
-{
- struct func_params *params;
-
- free(func->name);
-
- while (func->params) {
- params = func->params;
- func->params = params->next;
- free(params);
- }
-
- free(func);
-}
-
-/**
- * tep_register_print_function - register a helper function
- * @tep: a handle to the trace event parser context
- * @func: the function to process the helper function
- * @ret_type: the return type of the helper function
- * @name: the name of the helper function
- * @parameters: A list of enum tep_func_arg_type
- *
- * Some events may have helper functions in the print format arguments.
- * This allows a plugin to dynamically create a way to process one
- * of these functions.
- *
- * The @parameters is a variable list of tep_func_arg_type enums that
- * must end with TEP_FUNC_ARG_VOID.
- */
-int tep_register_print_function(struct tep_handle *tep,
- tep_func_handler func,
- enum tep_func_arg_type ret_type,
- char *name, ...)
-{
- struct tep_function_handler *func_handle;
- struct func_params **next_param;
- struct func_params *param;
- enum tep_func_arg_type type;
- va_list ap;
- int ret;
-
- func_handle = find_func_handler(tep, name);
- if (func_handle) {
- /*
- * This is most like caused by the users own
- * plugins updating the function. This overrides the
- * system defaults.
- */
- pr_stat("override of function helper '%s'", name);
- remove_func_handler(tep, name);
- }
-
- func_handle = calloc(1, sizeof(*func_handle));
- if (!func_handle) {
- do_warning("Failed to allocate function handler");
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
-
- func_handle->ret_type = ret_type;
- func_handle->name = strdup(name);
- func_handle->func = func;
- if (!func_handle->name) {
- do_warning("Failed to allocate function name");
- free(func_handle);
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
-
- next_param = &(func_handle->params);
- va_start(ap, name);
- for (;;) {
- type = va_arg(ap, enum tep_func_arg_type);
- if (type == TEP_FUNC_ARG_VOID)
- break;
-
- if (type >= TEP_FUNC_ARG_MAX_TYPES) {
- do_warning("Invalid argument type %d", type);
- ret = TEP_ERRNO__INVALID_ARG_TYPE;
- goto out_free;
- }
-
- param = malloc(sizeof(*param));
- if (!param) {
- do_warning("Failed to allocate function param");
- ret = TEP_ERRNO__MEM_ALLOC_FAILED;
- goto out_free;
- }
- param->type = type;
- param->next = NULL;
-
- *next_param = param;
- next_param = &(param->next);
-
- func_handle->nr_args++;
- }
- va_end(ap);
-
- func_handle->next = tep->func_handlers;
- tep->func_handlers = func_handle;
-
- return 0;
- out_free:
- va_end(ap);
- free_func_handle(func_handle);
- return ret;
-}
-
-/**
- * tep_unregister_print_function - unregister a helper function
- * @tep: a handle to the trace event parser context
- * @func: the function to process the helper function
- * @name: the name of the helper function
- *
- * This function removes existing print handler for function @name.
- *
- * Returns 0 if the handler was removed successully, -1 otherwise.
- */
-int tep_unregister_print_function(struct tep_handle *tep,
- tep_func_handler func, char *name)
-{
- struct tep_function_handler *func_handle;
-
- func_handle = find_func_handler(tep, name);
- if (func_handle && func_handle->func == func) {
- remove_func_handler(tep, name);
- return 0;
- }
- return -1;
-}
-
-static struct tep_event *search_event(struct tep_handle *tep, int id,
- const char *sys_name,
- const char *event_name)
-{
- struct tep_event *event;
-
- if (id >= 0) {
- /* search by id */
- event = tep_find_event(tep, id);
- if (!event)
- return NULL;
- if (event_name && (strcmp(event_name, event->name) != 0))
- return NULL;
- if (sys_name && (strcmp(sys_name, event->system) != 0))
- return NULL;
- } else {
- event = tep_find_event_by_name(tep, sys_name, event_name);
- if (!event)
- return NULL;
- }
- return event;
-}
-
-/**
- * tep_register_event_handler - register a way to parse an event
- * @tep: a handle to the trace event parser context
- * @id: the id of the event to register
- * @sys_name: the system name the event belongs to
- * @event_name: the name of the event
- * @func: the function to call to parse the event information
- * @context: the data to be passed to @func
- *
- * This function allows a developer to override the parsing of
- * a given event. If for some reason the default print format
- * is not sufficient, this function will register a function
- * for an event to be used to parse the data instead.
- *
- * If @id is >= 0, then it is used to find the event.
- * else @sys_name and @event_name are used.
- *
- * Returns:
- * TEP_REGISTER_SUCCESS_OVERWRITE if an existing handler is overwritten
- * TEP_REGISTER_SUCCESS if a new handler is registered successfully
- * negative TEP_ERRNO_... in case of an error
- *
- */
-int tep_register_event_handler(struct tep_handle *tep, int id,
- const char *sys_name, const char *event_name,
- tep_event_handler_func func, void *context)
-{
- struct tep_event *event;
- struct event_handler *handle;
-
- event = search_event(tep, id, sys_name, event_name);
- if (event == NULL)
- goto not_found;
-
- pr_stat("overriding event (%d) %s:%s with new print handler",
- event->id, event->system, event->name);
-
- event->handler = func;
- event->context = context;
- return TEP_REGISTER_SUCCESS_OVERWRITE;
-
- not_found:
- /* Save for later use. */
- handle = calloc(1, sizeof(*handle));
- if (!handle) {
- do_warning("Failed to allocate event handler");
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
-
- handle->id = id;
- if (event_name)
- handle->event_name = strdup(event_name);
- if (sys_name)
- handle->sys_name = strdup(sys_name);
-
- if ((event_name && !handle->event_name) ||
- (sys_name && !handle->sys_name)) {
- do_warning("Failed to allocate event/sys name");
- free((void *)handle->event_name);
- free((void *)handle->sys_name);
- free(handle);
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
-
- handle->func = func;
- handle->next = tep->handlers;
- tep->handlers = handle;
- handle->context = context;
-
- return TEP_REGISTER_SUCCESS;
-}
-
-static int handle_matches(struct event_handler *handler, int id,
- const char *sys_name, const char *event_name,
- tep_event_handler_func func, void *context)
-{
- if (id >= 0 && id != handler->id)
- return 0;
-
- if (event_name && (strcmp(event_name, handler->event_name) != 0))
- return 0;
-
- if (sys_name && (strcmp(sys_name, handler->sys_name) != 0))
- return 0;
-
- if (func != handler->func || context != handler->context)
- return 0;
-
- return 1;
-}
-
-/**
- * tep_unregister_event_handler - unregister an existing event handler
- * @tep: a handle to the trace event parser context
- * @id: the id of the event to unregister
- * @sys_name: the system name the handler belongs to
- * @event_name: the name of the event handler
- * @func: the function to call to parse the event information
- * @context: the data to be passed to @func
- *
- * This function removes existing event handler (parser).
- *
- * If @id is >= 0, then it is used to find the event.
- * else @sys_name and @event_name are used.
- *
- * Returns 0 if handler was removed successfully, -1 if event was not found.
- */
-int tep_unregister_event_handler(struct tep_handle *tep, int id,
- const char *sys_name, const char *event_name,
- tep_event_handler_func func, void *context)
-{
- struct tep_event *event;
- struct event_handler *handle;
- struct event_handler **next;
-
- event = search_event(tep, id, sys_name, event_name);
- if (event == NULL)
- goto not_found;
-
- if (event->handler == func && event->context == context) {
- pr_stat("removing override handler for event (%d) %s:%s. Going back to default handler.",
- event->id, event->system, event->name);
-
- event->handler = NULL;
- event->context = NULL;
- return 0;
- }
-
-not_found:
- for (next = &tep->handlers; *next; next = &(*next)->next) {
- handle = *next;
- if (handle_matches(handle, id, sys_name, event_name,
- func, context))
- break;
- }
-
- if (!(*next))
- return -1;
-
- *next = handle->next;
- free_handler(handle);
-
- return 0;
-}
-
-/**
- * tep_alloc - create a tep handle
- */
-struct tep_handle *tep_alloc(void)
-{
- struct tep_handle *tep = calloc(1, sizeof(*tep));
-
- if (tep) {
- tep->ref_count = 1;
- tep->host_bigendian = tep_is_bigendian();
- }
-
- return tep;
-}
-
-void tep_ref(struct tep_handle *tep)
-{
- tep->ref_count++;
-}
-
-int tep_get_ref(struct tep_handle *tep)
-{
- if (tep)
- return tep->ref_count;
- return 0;
-}
-
-__hidden void free_tep_format_field(struct tep_format_field *field)
-{
- free(field->type);
- if (field->alias != field->name)
- free(field->alias);
- free(field->name);
- free(field);
-}
-
-static void free_format_fields(struct tep_format_field *field)
-{
- struct tep_format_field *next;
-
- while (field) {
- next = field->next;
- free_tep_format_field(field);
- field = next;
- }
-}
-
-static void free_formats(struct tep_format *format)
-{
- free_format_fields(format->common_fields);
- free_format_fields(format->fields);
-}
-
-__hidden void free_tep_event(struct tep_event *event)
-{
- free(event->name);
- free(event->system);
-
- free_formats(&event->format);
-
- free(event->print_fmt.format);
- free_args(event->print_fmt.args);
- free_parse_args(event->print_fmt.print_cache);
- free(event);
-}
-
-/**
- * tep_free - free a tep handle
- * @tep: the tep handle to free
- */
-void tep_free(struct tep_handle *tep)
-{
- struct cmdline_list *cmdlist, *cmdnext;
- struct func_list *funclist, *funcnext;
- struct printk_list *printklist, *printknext;
- struct tep_function_handler *func_handler;
- struct event_handler *handle;
- int i;
-
- if (!tep)
- return;
-
- cmdlist = tep->cmdlist;
- funclist = tep->funclist;
- printklist = tep->printklist;
-
- tep->ref_count--;
- if (tep->ref_count)
- return;
-
- if (tep->cmdlines) {
- for (i = 0; i < tep->cmdline_count; i++)
- free(tep->cmdlines[i].comm);
- free(tep->cmdlines);
- }
-
- while (cmdlist) {
- cmdnext = cmdlist->next;
- free(cmdlist->comm);
- free(cmdlist);
- cmdlist = cmdnext;
- }
-
- if (tep->func_map) {
- for (i = 0; i < (int)tep->func_count; i++) {
- free(tep->func_map[i].func);
- free(tep->func_map[i].mod);
- }
- free(tep->func_map);
- }
-
- while (funclist) {
- funcnext = funclist->next;
- free(funclist->func);
- free(funclist->mod);
- free(funclist);
- funclist = funcnext;
- }
-
- while (tep->func_handlers) {
- func_handler = tep->func_handlers;
- tep->func_handlers = func_handler->next;
- free_func_handle(func_handler);
- }
-
- if (tep->printk_map) {
- for (i = 0; i < (int)tep->printk_count; i++)
- free(tep->printk_map[i].printk);
- free(tep->printk_map);
- }
-
- while (printklist) {
- printknext = printklist->next;
- free(printklist->printk);
- free(printklist);
- printklist = printknext;
- }
-
- for (i = 0; i < tep->nr_events; i++)
- free_tep_event(tep->events[i]);
-
- while (tep->handlers) {
- handle = tep->handlers;
- tep->handlers = handle->next;
- free_handler(handle);
- }
-
- free(tep->events);
- free(tep->sort_events);
- free(tep->func_resolver);
- free_tep_plugin_paths(tep);
-
- free(tep);
-}
-
-void tep_unref(struct tep_handle *tep)
-{
- tep_free(tep);
-}
+++ /dev/null
-/* SPDX-License-Identifier: LGPL-2.1 */
-/*
- * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-#ifndef _PARSE_EVENTS_H
-#define _PARSE_EVENTS_H
-
-#include <stdbool.h>
-#include <stdarg.h>
-#include <stdio.h>
-#include <regex.h>
-#include <string.h>
-
-#include "trace-seq.h"
-
-#ifndef __maybe_unused
-#define __maybe_unused __attribute__((unused))
-#endif
-
-#ifndef DEBUG_RECORD
-#define DEBUG_RECORD 0
-#endif
-
-struct tep_record {
- unsigned long long ts;
- unsigned long long offset;
- long long missed_events; /* buffer dropped events before */
- int record_size; /* size of binary record */
- int size; /* size of data */
- void *data;
- int cpu;
- int ref_count;
- int locked; /* Do not free, even if ref_count is zero */
- void *priv;
-#if DEBUG_RECORD
- struct tep_record *prev;
- struct tep_record *next;
- long alloc_addr;
-#endif
-};
-
-/* ----------------------- tep ----------------------- */
-
-struct tep_handle;
-struct tep_event;
-
-typedef int (*tep_event_handler_func)(struct trace_seq *s,
- struct tep_record *record,
- struct tep_event *event,
- void *context);
-
-typedef int (*tep_plugin_load_func)(struct tep_handle *tep);
-typedef int (*tep_plugin_unload_func)(struct tep_handle *tep);
-
-struct tep_plugin_option {
- struct tep_plugin_option *next;
- void *handle;
- char *file;
- char *name;
- char *plugin_alias;
- char *description;
- const char *value;
- void *priv;
- int set;
-};
-
-/*
- * Plugin hooks that can be called:
- *
- * TEP_PLUGIN_LOADER: (required)
- * The function name to initialized the plugin.
- *
- * int TEP_PLUGIN_LOADER(struct tep_handle *tep)
- *
- * TEP_PLUGIN_UNLOADER: (optional)
- * The function called just before unloading
- *
- * int TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
- *
- * TEP_PLUGIN_OPTIONS: (optional)
- * Plugin options that can be set before loading
- *
- * struct tep_plugin_option TEP_PLUGIN_OPTIONS[] = {
- * {
- * .name = "option-name",
- * .plugin_alias = "override-file-name", (optional)
- * .description = "description of option to show users",
- * },
- * {
- * .name = NULL,
- * },
- * };
- *
- * Array must end with .name = NULL;
- *
- *
- * .plugin_alias is used to give a shorter name to access
- * the vairable. Useful if a plugin handles more than one event.
- *
- * If .value is not set, then it is considered a boolean and only
- * .set will be processed. If .value is defined, then it is considered
- * a string option and .set will be ignored.
- *
- * TEP_PLUGIN_ALIAS: (optional)
- * The name to use for finding options (uses filename if not defined)
- */
-#define TEP_PLUGIN_LOADER tep_plugin_loader
-#define TEP_PLUGIN_UNLOADER tep_plugin_unloader
-#define TEP_PLUGIN_OPTIONS tep_plugin_options
-#define TEP_PLUGIN_ALIAS tep_plugin_alias
-#define _MAKE_STR(x) #x
-#define MAKE_STR(x) _MAKE_STR(x)
-#define TEP_PLUGIN_LOADER_NAME MAKE_STR(TEP_PLUGIN_LOADER)
-#define TEP_PLUGIN_UNLOADER_NAME MAKE_STR(TEP_PLUGIN_UNLOADER)
-#define TEP_PLUGIN_OPTIONS_NAME MAKE_STR(TEP_PLUGIN_OPTIONS)
-#define TEP_PLUGIN_ALIAS_NAME MAKE_STR(TEP_PLUGIN_ALIAS)
-
-enum tep_format_flags {
- TEP_FIELD_IS_ARRAY = 1,
- TEP_FIELD_IS_POINTER = 2,
- TEP_FIELD_IS_SIGNED = 4,
- TEP_FIELD_IS_STRING = 8,
- TEP_FIELD_IS_DYNAMIC = 16,
- TEP_FIELD_IS_LONG = 32,
- TEP_FIELD_IS_FLAG = 64,
- TEP_FIELD_IS_SYMBOLIC = 128,
- TEP_FIELD_IS_RELATIVE = 256,
-};
-
-struct tep_format_field {
- struct tep_format_field *next;
- struct tep_event *event;
- char *type;
- char *name;
- char *alias;
- int offset;
- int size;
- unsigned int arraylen;
- unsigned int elementsize;
- unsigned long flags;
-};
-
-struct tep_format {
- int nr_common;
- int nr_fields;
- struct tep_format_field *common_fields;
- struct tep_format_field *fields;
-};
-
-struct tep_print_arg_atom {
- char *atom;
-};
-
-struct tep_print_arg_string {
- char *string;
- struct tep_format_field *field;
-};
-
-struct tep_print_arg_bitmask {
- char *bitmask;
- struct tep_format_field *field;
-};
-
-struct tep_print_arg_field {
- char *name;
- struct tep_format_field *field;
-};
-
-struct tep_print_flag_sym {
- struct tep_print_flag_sym *next;
- char *value;
- char *str;
-};
-
-struct tep_print_arg_typecast {
- char *type;
- struct tep_print_arg *item;
-};
-
-struct tep_print_arg_flags {
- struct tep_print_arg *field;
- char *delim;
- struct tep_print_flag_sym *flags;
-};
-
-struct tep_print_arg_symbol {
- struct tep_print_arg *field;
- struct tep_print_flag_sym *symbols;
-};
-
-struct tep_print_arg_hex {
- struct tep_print_arg *field;
- struct tep_print_arg *size;
-};
-
-struct tep_print_arg_int_array {
- struct tep_print_arg *field;
- struct tep_print_arg *count;
- struct tep_print_arg *el_size;
-};
-
-struct tep_print_arg_dynarray {
- struct tep_format_field *field;
- struct tep_print_arg *index;
-};
-
-struct tep_print_arg;
-
-struct tep_print_arg_op {
- char *op;
- int prio;
- struct tep_print_arg *left;
- struct tep_print_arg *right;
-};
-
-struct tep_function_handler;
-
-struct tep_print_arg_func {
- struct tep_function_handler *func;
- struct tep_print_arg *args;
-};
-
-enum tep_print_arg_type {
- TEP_PRINT_NULL,
- TEP_PRINT_ATOM,
- TEP_PRINT_FIELD,
- TEP_PRINT_FLAGS,
- TEP_PRINT_SYMBOL,
- TEP_PRINT_HEX,
- TEP_PRINT_INT_ARRAY,
- TEP_PRINT_TYPE,
- TEP_PRINT_STRING,
- TEP_PRINT_BSTRING,
- TEP_PRINT_DYNAMIC_ARRAY,
- TEP_PRINT_OP,
- TEP_PRINT_FUNC,
- TEP_PRINT_BITMASK,
- TEP_PRINT_DYNAMIC_ARRAY_LEN,
- TEP_PRINT_HEX_STR,
-};
-
-struct tep_print_arg {
- struct tep_print_arg *next;
- enum tep_print_arg_type type;
- union {
- struct tep_print_arg_atom atom;
- struct tep_print_arg_field field;
- struct tep_print_arg_typecast typecast;
- struct tep_print_arg_flags flags;
- struct tep_print_arg_symbol symbol;
- struct tep_print_arg_hex hex;
- struct tep_print_arg_int_array int_array;
- struct tep_print_arg_func func;
- struct tep_print_arg_string string;
- struct tep_print_arg_bitmask bitmask;
- struct tep_print_arg_op op;
- struct tep_print_arg_dynarray dynarray;
- };
-};
-
-struct tep_print_parse;
-
-struct tep_print_fmt {
- char *format;
- struct tep_print_arg *args;
- struct tep_print_parse *print_cache;
-};
-
-struct tep_event {
- struct tep_handle *tep;
- char *name;
- int id;
- int flags;
- struct tep_format format;
- struct tep_print_fmt print_fmt;
- char *system;
- tep_event_handler_func handler;
- void *context;
-};
-
-enum {
- TEP_EVENT_FL_ISFTRACE = 0x01,
- TEP_EVENT_FL_ISPRINT = 0x02,
- TEP_EVENT_FL_ISBPRINT = 0x04,
- TEP_EVENT_FL_ISFUNCENT = 0x10,
- TEP_EVENT_FL_ISFUNCRET = 0x20,
- TEP_EVENT_FL_NOHANDLE = 0x40,
- TEP_EVENT_FL_PRINTRAW = 0x80,
-
- TEP_EVENT_FL_FAILED = 0x80000000
-};
-
-enum tep_event_sort_type {
- TEP_EVENT_SORT_ID,
- TEP_EVENT_SORT_NAME,
- TEP_EVENT_SORT_SYSTEM,
-};
-
-enum tep_event_type {
- TEP_EVENT_ERROR,
- TEP_EVENT_NONE,
- TEP_EVENT_SPACE,
- TEP_EVENT_NEWLINE,
- TEP_EVENT_OP,
- TEP_EVENT_DELIM,
- TEP_EVENT_ITEM,
- TEP_EVENT_DQUOTE,
- TEP_EVENT_SQUOTE,
-};
-
-typedef unsigned long long (*tep_func_handler)(struct trace_seq *s,
- unsigned long long *args);
-
-enum tep_func_arg_type {
- TEP_FUNC_ARG_VOID,
- TEP_FUNC_ARG_INT,
- TEP_FUNC_ARG_LONG,
- TEP_FUNC_ARG_STRING,
- TEP_FUNC_ARG_PTR,
- TEP_FUNC_ARG_MAX_TYPES
-};
-
-enum tep_flag {
- TEP_NSEC_OUTPUT = 1, /* output in NSECS */
- TEP_DISABLE_SYS_PLUGINS = 1 << 1,
- TEP_DISABLE_PLUGINS = 1 << 2,
-};
-
-#define TEP_ERRORS \
- _PE(MEM_ALLOC_FAILED, "failed to allocate memory"), \
- _PE(PARSE_EVENT_FAILED, "failed to parse event"), \
- _PE(READ_ID_FAILED, "failed to read event id"), \
- _PE(READ_FORMAT_FAILED, "failed to read event format"), \
- _PE(READ_PRINT_FAILED, "failed to read event print fmt"), \
- _PE(OLD_FTRACE_ARG_FAILED,"failed to allocate field name for ftrace"),\
- _PE(INVALID_ARG_TYPE, "invalid argument type"), \
- _PE(INVALID_EXP_TYPE, "invalid expression type"), \
- _PE(INVALID_OP_TYPE, "invalid operator type"), \
- _PE(INVALID_EVENT_NAME, "invalid event name"), \
- _PE(EVENT_NOT_FOUND, "no event found"), \
- _PE(SYNTAX_ERROR, "syntax error"), \
- _PE(ILLEGAL_RVALUE, "illegal rvalue"), \
- _PE(ILLEGAL_LVALUE, "illegal lvalue for string comparison"), \
- _PE(INVALID_REGEX, "regex did not compute"), \
- _PE(ILLEGAL_STRING_CMP, "illegal comparison for string"), \
- _PE(ILLEGAL_INTEGER_CMP,"illegal comparison for integer"), \
- _PE(REPARENT_NOT_OP, "cannot reparent other than OP"), \
- _PE(REPARENT_FAILED, "failed to reparent filter OP"), \
- _PE(BAD_FILTER_ARG, "bad arg in filter tree"), \
- _PE(UNEXPECTED_TYPE, "unexpected type (not a value)"), \
- _PE(ILLEGAL_TOKEN, "illegal token"), \
- _PE(INVALID_PAREN, "open parenthesis cannot come here"), \
- _PE(UNBALANCED_PAREN, "unbalanced number of parenthesis"), \
- _PE(UNKNOWN_TOKEN, "unknown token"), \
- _PE(FILTER_NOT_FOUND, "no filter found"), \
- _PE(NOT_A_NUMBER, "must have number field"), \
- _PE(NO_FILTER, "no filters exists"), \
- _PE(FILTER_MISS, "record does not match to filter")
-
-#undef _PE
-#define _PE(__code, __str) TEP_ERRNO__ ## __code
-enum tep_errno {
- TEP_ERRNO__SUCCESS = 0,
- TEP_ERRNO__FILTER_MATCH = TEP_ERRNO__SUCCESS,
-
- /*
- * Choose an arbitrary negative big number not to clash with standard
- * errno since SUS requires the errno has distinct positive values.
- * See 'Issue 6' in the link below.
- *
- * https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/errno.h.html
- */
- __TEP_ERRNO__START = -100000,
-
- TEP_ERRORS,
-
- __TEP_ERRNO__END,
-};
-#undef _PE
-
-struct tep_plugin_list;
-
-#define INVALID_PLUGIN_LIST_OPTION ((char **)((unsigned long)-1))
-
-enum tep_plugin_load_priority {
- TEP_PLUGIN_FIRST,
- TEP_PLUGIN_LAST,
-};
-
-int tep_add_plugin_path(struct tep_handle *tep, char *path,
- enum tep_plugin_load_priority prio);
-struct tep_plugin_list *tep_load_plugins(struct tep_handle *tep);
-void tep_unload_plugins(struct tep_plugin_list *plugin_list,
- struct tep_handle *tep);
-void tep_load_plugins_hook(struct tep_handle *tep, const char *suffix,
- void (*load_plugin)(struct tep_handle *tep,
- const char *path,
- const char *name,
- void *data),
- void *data);
-char **tep_plugin_list_options(void);
-void tep_plugin_free_options_list(char **list);
-int tep_plugin_add_options(const char *name,
- struct tep_plugin_option *options);
-int tep_plugin_add_option(const char *name, const char *val);
-void tep_plugin_remove_options(struct tep_plugin_option *options);
-void tep_plugin_print_options(struct trace_seq *s);
-void tep_print_plugins(struct trace_seq *s,
- const char *prefix, const char *suffix,
- const struct tep_plugin_list *list);
-
-/* tep_handle */
-typedef char *(tep_func_resolver_t)(void *priv,
- unsigned long long *addrp, char **modp);
-void tep_set_flag(struct tep_handle *tep, int flag);
-void tep_clear_flag(struct tep_handle *tep, enum tep_flag flag);
-bool tep_test_flag(struct tep_handle *tep, enum tep_flag flags);
-
-static inline int tep_is_bigendian(void)
-{
- unsigned char str[] = { 0x1, 0x2, 0x3, 0x4 };
- unsigned int val;
-
- memcpy(&val, str, 4);
- return val == 0x01020304;
-}
-
-/* taken from kernel/trace/trace.h */
-enum trace_flag_type {
- TRACE_FLAG_IRQS_OFF = 0x01,
- TRACE_FLAG_IRQS_NOSUPPORT = 0x02,
- TRACE_FLAG_NEED_RESCHED = 0x04,
- TRACE_FLAG_HARDIRQ = 0x08,
- TRACE_FLAG_SOFTIRQ = 0x10,
-};
-
-int tep_set_function_resolver(struct tep_handle *tep,
- tep_func_resolver_t *func, void *priv);
-void tep_reset_function_resolver(struct tep_handle *tep);
-int tep_register_comm(struct tep_handle *tep, const char *comm, int pid);
-int tep_override_comm(struct tep_handle *tep, const char *comm, int pid);
-int tep_register_function(struct tep_handle *tep, char *name,
- unsigned long long addr, char *mod);
-int tep_register_print_string(struct tep_handle *tep, const char *fmt,
- unsigned long long addr);
-bool tep_is_pid_registered(struct tep_handle *tep, int pid);
-
-struct tep_event *tep_get_event(struct tep_handle *tep, int index);
-
-#define TEP_PRINT_INFO "INFO"
-#define TEP_PRINT_INFO_RAW "INFO_RAW"
-#define TEP_PRINT_COMM "COMM"
-#define TEP_PRINT_LATENCY "LATENCY"
-#define TEP_PRINT_NAME "NAME"
-#define TEP_PRINT_PID 1U
-#define TEP_PRINT_TIME 2U
-#define TEP_PRINT_CPU 3U
-
-void tep_print_event(struct tep_handle *tep, struct trace_seq *s,
- struct tep_record *record, const char *fmt, ...)
- __attribute__ ((format (printf, 4, 5)));
-
-int tep_parse_header_page(struct tep_handle *tep, char *buf, unsigned long size,
- int long_size);
-
-enum tep_errno tep_parse_event(struct tep_handle *tep, const char *buf,
- unsigned long size, const char *sys);
-enum tep_errno tep_parse_format(struct tep_handle *tep,
- struct tep_event **eventp,
- const char *buf,
- unsigned long size, const char *sys);
-
-void *tep_get_field_raw(struct trace_seq *s, struct tep_event *event,
- const char *name, struct tep_record *record,
- int *len, int err);
-
-int tep_get_field_val(struct trace_seq *s, struct tep_event *event,
- const char *name, struct tep_record *record,
- unsigned long long *val, int err);
-int tep_get_common_field_val(struct trace_seq *s, struct tep_event *event,
- const char *name, struct tep_record *record,
- unsigned long long *val, int err);
-int tep_get_any_field_val(struct trace_seq *s, struct tep_event *event,
- const char *name, struct tep_record *record,
- unsigned long long *val, int err);
-
-int tep_print_num_field(struct trace_seq *s, const char *fmt,
- struct tep_event *event, const char *name,
- struct tep_record *record, int err);
-
-int tep_print_func_field(struct trace_seq *s, const char *fmt,
- struct tep_event *event, const char *name,
- struct tep_record *record, int err);
-
-enum tep_reg_handler {
- TEP_REGISTER_SUCCESS = 0,
- TEP_REGISTER_SUCCESS_OVERWRITE,
-};
-
-int tep_register_event_handler(struct tep_handle *tep, int id,
- const char *sys_name, const char *event_name,
- tep_event_handler_func func, void *context);
-int tep_unregister_event_handler(struct tep_handle *tep, int id,
- const char *sys_name, const char *event_name,
- tep_event_handler_func func, void *context);
-int tep_register_print_function(struct tep_handle *tep,
- tep_func_handler func,
- enum tep_func_arg_type ret_type,
- char *name, ...);
-int tep_unregister_print_function(struct tep_handle *tep,
- tep_func_handler func, char *name);
-
-struct tep_format_field *tep_find_common_field(struct tep_event *event, const char *name);
-struct tep_format_field *tep_find_field(struct tep_event *event, const char *name);
-struct tep_format_field *tep_find_any_field(struct tep_event *event, const char *name);
-
-const char *tep_find_function(struct tep_handle *tep, unsigned long long addr);
-unsigned long long
-tep_find_function_address(struct tep_handle *tep, unsigned long long addr);
-unsigned long long tep_read_number(struct tep_handle *tep, const void *ptr, int size);
-int tep_read_number_field(struct tep_format_field *field, const void *data,
- unsigned long long *value);
-
-struct tep_event *tep_get_first_event(struct tep_handle *tep);
-int tep_get_events_count(struct tep_handle *tep);
-struct tep_event *tep_find_event(struct tep_handle *tep, int id);
-
-struct tep_event *
-tep_find_event_by_name(struct tep_handle *tep, const char *sys, const char *name);
-struct tep_event *
-tep_find_event_by_record(struct tep_handle *tep, struct tep_record *record);
-
-int tep_data_type(struct tep_handle *tep, struct tep_record *rec);
-int tep_data_pid(struct tep_handle *tep, struct tep_record *rec);
-int tep_data_preempt_count(struct tep_handle *tep, struct tep_record *rec);
-int tep_data_flags(struct tep_handle *tep, struct tep_record *rec);
-const char *tep_data_comm_from_pid(struct tep_handle *tep, int pid);
-struct tep_cmdline;
-struct tep_cmdline *tep_data_pid_from_comm(struct tep_handle *tep, const char *comm,
- struct tep_cmdline *next);
-int tep_cmdline_pid(struct tep_handle *tep, struct tep_cmdline *cmdline);
-
-void tep_print_field(struct trace_seq *s, void *data,
- struct tep_format_field *field);
-void tep_print_fields(struct trace_seq *s, void *data,
- int size __maybe_unused, struct tep_event *event);
-int tep_strerror(struct tep_handle *tep, enum tep_errno errnum,
- char *buf, size_t buflen);
-
-struct tep_event **tep_list_events(struct tep_handle *tep, enum tep_event_sort_type);
-struct tep_event **tep_list_events_copy(struct tep_handle *tep,
- enum tep_event_sort_type);
-struct tep_format_field **tep_event_common_fields(struct tep_event *event);
-struct tep_format_field **tep_event_fields(struct tep_event *event);
-
-enum tep_endian {
- TEP_LITTLE_ENDIAN = 0,
- TEP_BIG_ENDIAN
-};
-int tep_get_cpus(struct tep_handle *tep);
-void tep_set_cpus(struct tep_handle *tep, int cpus);
-int tep_get_long_size(struct tep_handle *tep);
-void tep_set_long_size(struct tep_handle *tep, int long_size);
-int tep_get_page_size(struct tep_handle *tep);
-void tep_set_page_size(struct tep_handle *tep, int _page_size);
-bool tep_is_file_bigendian(struct tep_handle *tep);
-void tep_set_file_bigendian(struct tep_handle *tep, enum tep_endian endian);
-bool tep_is_local_bigendian(struct tep_handle *tep);
-void tep_set_local_bigendian(struct tep_handle *tep, enum tep_endian endian);
-int tep_get_header_page_size(struct tep_handle *tep);
-int tep_get_header_timestamp_size(struct tep_handle *tep);
-bool tep_is_old_format(struct tep_handle *tep);
-void tep_set_test_filters(struct tep_handle *tep, int test_filters);
-
-struct tep_handle *tep_alloc(void);
-void tep_free(struct tep_handle *tep);
-void tep_ref(struct tep_handle *tep);
-void tep_unref(struct tep_handle *tep);
-int tep_get_ref(struct tep_handle *tep);
-
-/* for debugging */
-void tep_print_funcs(struct tep_handle *tep);
-void tep_print_printk(struct tep_handle *tep);
-
-/* ----------------------- filtering ----------------------- */
-
-enum tep_filter_boolean_type {
- TEP_FILTER_FALSE,
- TEP_FILTER_TRUE,
-};
-
-enum tep_filter_op_type {
- TEP_FILTER_OP_AND = 1,
- TEP_FILTER_OP_OR,
- TEP_FILTER_OP_NOT,
-};
-
-enum tep_filter_cmp_type {
- TEP_FILTER_CMP_NONE,
- TEP_FILTER_CMP_EQ,
- TEP_FILTER_CMP_NE,
- TEP_FILTER_CMP_GT,
- TEP_FILTER_CMP_LT,
- TEP_FILTER_CMP_GE,
- TEP_FILTER_CMP_LE,
- TEP_FILTER_CMP_MATCH,
- TEP_FILTER_CMP_NOT_MATCH,
- TEP_FILTER_CMP_REGEX,
- TEP_FILTER_CMP_NOT_REGEX,
-};
-
-enum tep_filter_exp_type {
- TEP_FILTER_EXP_NONE,
- TEP_FILTER_EXP_ADD,
- TEP_FILTER_EXP_SUB,
- TEP_FILTER_EXP_MUL,
- TEP_FILTER_EXP_DIV,
- TEP_FILTER_EXP_MOD,
- TEP_FILTER_EXP_RSHIFT,
- TEP_FILTER_EXP_LSHIFT,
- TEP_FILTER_EXP_AND,
- TEP_FILTER_EXP_OR,
- TEP_FILTER_EXP_XOR,
- TEP_FILTER_EXP_NOT,
-};
-
-enum tep_filter_arg_type {
- TEP_FILTER_ARG_NONE,
- TEP_FILTER_ARG_BOOLEAN,
- TEP_FILTER_ARG_VALUE,
- TEP_FILTER_ARG_FIELD,
- TEP_FILTER_ARG_EXP,
- TEP_FILTER_ARG_OP,
- TEP_FILTER_ARG_NUM,
- TEP_FILTER_ARG_STR,
-};
-
-enum tep_filter_value_type {
- TEP_FILTER_NUMBER,
- TEP_FILTER_STRING,
- TEP_FILTER_CHAR
-};
-
-struct tep_filter_arg;
-
-struct tep_filter_arg_boolean {
- enum tep_filter_boolean_type value;
-};
-
-struct tep_filter_arg_field {
- struct tep_format_field *field;
-};
-
-struct tep_filter_arg_value {
- enum tep_filter_value_type type;
- union {
- char *str;
- unsigned long long val;
- };
-};
-
-struct tep_filter_arg_op {
- enum tep_filter_op_type type;
- struct tep_filter_arg *left;
- struct tep_filter_arg *right;
-};
-
-struct tep_filter_arg_exp {
- enum tep_filter_exp_type type;
- struct tep_filter_arg *left;
- struct tep_filter_arg *right;
-};
-
-struct tep_filter_arg_num {
- enum tep_filter_cmp_type type;
- struct tep_filter_arg *left;
- struct tep_filter_arg *right;
-};
-
-struct tep_filter_arg_str {
- enum tep_filter_cmp_type type;
- struct tep_format_field *field;
- char *val;
- char *buffer;
- regex_t reg;
-};
-
-struct tep_filter_arg {
- enum tep_filter_arg_type type;
- union {
- struct tep_filter_arg_boolean boolean;
- struct tep_filter_arg_field field;
- struct tep_filter_arg_value value;
- struct tep_filter_arg_op op;
- struct tep_filter_arg_exp exp;
- struct tep_filter_arg_num num;
- struct tep_filter_arg_str str;
- };
-};
-
-struct tep_filter_type {
- int event_id;
- struct tep_event *event;
- struct tep_filter_arg *filter;
-};
-
-#define TEP_FILTER_ERROR_BUFSZ 1024
-
-struct tep_event_filter {
- struct tep_handle *tep;
- int filters;
- struct tep_filter_type *event_filters;
- char error_buffer[TEP_FILTER_ERROR_BUFSZ];
-};
-
-struct tep_event_filter *tep_filter_alloc(struct tep_handle *tep);
-
-/* for backward compatibility */
-#define FILTER_NONE TEP_ERRNO__NO_FILTER
-#define FILTER_NOEXIST TEP_ERRNO__FILTER_NOT_FOUND
-#define FILTER_MISS TEP_ERRNO__FILTER_MISS
-#define FILTER_MATCH TEP_ERRNO__FILTER_MATCH
-
-enum tep_errno tep_filter_add_filter_str(struct tep_event_filter *filter,
- const char *filter_str);
-
-enum tep_errno tep_filter_match(struct tep_event_filter *filter,
- struct tep_record *record);
-
-int tep_filter_strerror(struct tep_event_filter *filter, enum tep_errno err,
- char *buf, size_t buflen);
-
-int tep_event_filtered(struct tep_event_filter *filter,
- int event_id);
-
-void tep_filter_reset(struct tep_event_filter *filter);
-
-void tep_filter_free(struct tep_event_filter *filter);
-
-char *tep_filter_make_string(struct tep_event_filter *filter, int event_id);
-
-int tep_filter_remove_event(struct tep_event_filter *filter,
- int event_id);
-
-int tep_filter_copy(struct tep_event_filter *dest, struct tep_event_filter *source);
-
-int tep_filter_compare(struct tep_event_filter *filter1, struct tep_event_filter *filter2);
-
-#endif /* _PARSE_EVENTS_H */
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-
-#include <ctype.h>
-#include <stdio.h>
-#include <string.h>
-#include <dlfcn.h>
-#include <stdlib.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <unistd.h>
-#include <dirent.h>
-#include <errno.h>
-#include "event-parse.h"
-#include "event-parse-local.h"
-#include "event-utils.h"
-#include "trace-seq.h"
-
-#define LOCAL_PLUGIN_DIR ".local/lib/traceevent/plugins/"
-
-static struct registered_plugin_options {
- struct registered_plugin_options *next;
- struct tep_plugin_option *options;
-} *registered_options;
-
-static struct trace_plugin_options {
- struct trace_plugin_options *next;
- char *plugin;
- char *option;
- char *value;
-} *trace_plugin_options;
-
-struct tep_plugin_list {
- struct tep_plugin_list *next;
- char *name;
- void *handle;
-};
-
-struct tep_plugins_dir {
- struct tep_plugins_dir *next;
- char *path;
- enum tep_plugin_load_priority prio;
-};
-
-static void lower_case(char *str)
-{
- if (!str)
- return;
- for (; *str; str++)
- *str = tolower(*str);
-}
-
-static int update_option_value(struct tep_plugin_option *op, const char *val)
-{
- char *op_val;
-
- if (!val) {
- /* toggle, only if option is boolean */
- if (op->value)
- /* Warn? */
- return 0;
- op->set ^= 1;
- return 0;
- }
-
- /*
- * If the option has a value then it takes a string
- * otherwise the option is a boolean.
- */
- if (op->value) {
- op->value = val;
- return 0;
- }
-
- /* Option is boolean, must be either "1", "0", "true" or "false" */
-
- op_val = strdup(val);
- if (!op_val)
- return -1;
- lower_case(op_val);
-
- if (strcmp(val, "1") == 0 || strcmp(val, "true") == 0)
- op->set = 1;
- else if (strcmp(val, "0") == 0 || strcmp(val, "false") == 0)
- op->set = 0;
- free(op_val);
-
- return 0;
-}
-
-/**
- * tep_plugin_list_options - get list of plugin options
- *
- * Returns an array of char strings that list the currently registered
- * plugin options in the format of <plugin>:<option>. This list can be
- * used by toggling the option.
- *
- * Returns NULL if there's no options registered. On error it returns
- * INVALID_PLUGIN_LIST_OPTION
- *
- * Must be freed with tep_plugin_free_options_list().
- */
-char **tep_plugin_list_options(void)
-{
- struct registered_plugin_options *reg;
- struct tep_plugin_option *op;
- char **list = NULL;
- char *name;
- int count = 0;
-
- for (reg = registered_options; reg; reg = reg->next) {
- for (op = reg->options; op->name; op++) {
- char *alias = op->plugin_alias ? op->plugin_alias : op->file;
- char **temp = list;
- int ret;
-
- ret = asprintf(&name, "%s:%s", alias, op->name);
- if (ret < 0)
- goto err;
-
- list = realloc(list, count + 2);
- if (!list) {
- list = temp;
- free(name);
- goto err;
- }
- list[count++] = name;
- list[count] = NULL;
- }
- }
- return list;
-
- err:
- while (--count >= 0)
- free(list[count]);
- free(list);
-
- return INVALID_PLUGIN_LIST_OPTION;
-}
-
-void tep_plugin_free_options_list(char **list)
-{
- int i;
-
- if (!list)
- return;
-
- if (list == INVALID_PLUGIN_LIST_OPTION)
- return;
-
- for (i = 0; list[i]; i++)
- free(list[i]);
-
- free(list);
-}
-
-static int
-update_option(const char *file, struct tep_plugin_option *option)
-{
- struct trace_plugin_options *op;
- char *plugin;
- int ret = 0;
-
- if (option->plugin_alias) {
- plugin = strdup(option->plugin_alias);
- if (!plugin)
- return -1;
- } else {
- char *p;
- plugin = strdup(file);
- if (!plugin)
- return -1;
- p = strstr(plugin, ".");
- if (p)
- *p = '\0';
- }
-
- /* first look for named options */
- for (op = trace_plugin_options; op; op = op->next) {
- if (!op->plugin)
- continue;
- if (strcmp(op->plugin, plugin) != 0)
- continue;
- if (strcmp(op->option, option->name) != 0)
- continue;
-
- ret = update_option_value(option, op->value);
- if (ret)
- goto out;
- break;
- }
-
- /* first look for unnamed options */
- for (op = trace_plugin_options; op; op = op->next) {
- if (op->plugin)
- continue;
- if (strcmp(op->option, option->name) != 0)
- continue;
-
- ret = update_option_value(option, op->value);
- break;
- }
-
- out:
- free(plugin);
- return ret;
-}
-
-/**
- * tep_plugin_add_options - Add a set of options by a plugin
- * @name: The name of the plugin adding the options
- * @options: The set of options being loaded
- *
- * Sets the options with the values that have been added by user.
- */
-int tep_plugin_add_options(const char *name,
- struct tep_plugin_option *options)
-{
- struct registered_plugin_options *reg;
-
- reg = malloc(sizeof(*reg));
- if (!reg)
- return -1;
- reg->next = registered_options;
- reg->options = options;
- registered_options = reg;
-
- while (options->name) {
- update_option(name, options);
- options++;
- }
- return 0;
-}
-
-/**
- * tep_plugin_remove_options - remove plugin options that were registered
- * @options: Options to removed that were registered with tep_plugin_add_options
- */
-void tep_plugin_remove_options(struct tep_plugin_option *options)
-{
- struct registered_plugin_options **last;
- struct registered_plugin_options *reg;
-
- for (last = ®istered_options; *last; last = &(*last)->next) {
- if ((*last)->options == options) {
- reg = *last;
- *last = reg->next;
- free(reg);
- return;
- }
- }
-}
-
-static int parse_option_name(char **option, char **plugin)
-{
- char *p;
-
- *plugin = NULL;
-
- if ((p = strstr(*option, ":"))) {
- *plugin = *option;
- *p = '\0';
- *option = strdup(p + 1);
- if (!*option)
- return -1;
- }
- return 0;
-}
-
-static struct tep_plugin_option *
-find_registered_option(const char *plugin, const char *option)
-{
- struct registered_plugin_options *reg;
- struct tep_plugin_option *op;
- const char *op_plugin;
-
- for (reg = registered_options; reg; reg = reg->next) {
- for (op = reg->options; op->name; op++) {
- if (op->plugin_alias)
- op_plugin = op->plugin_alias;
- else
- op_plugin = op->file;
-
- if (plugin && strcmp(plugin, op_plugin) != 0)
- continue;
- if (strcmp(option, op->name) != 0)
- continue;
-
- return op;
- }
- }
-
- return NULL;
-}
-
-static int process_option(const char *plugin, const char *option, const char *val)
-{
- struct tep_plugin_option *op;
-
- op = find_registered_option(plugin, option);
- if (!op)
- return 0;
-
- return update_option_value(op, val);
-}
-
-/**
- * tep_plugin_add_option - add an option/val pair to set plugin options
- * @name: The name of the option (format: <plugin>:<option> or just <option>)
- * @val: (optional) the value for the option
- *
- * Modify a plugin option. If @val is given than the value of the option
- * is set (note, some options just take a boolean, so @val must be either
- * "1" or "0" or "true" or "false").
- */
-int tep_plugin_add_option(const char *name, const char *val)
-{
- struct trace_plugin_options *op;
- char *option_str;
- char *plugin;
-
- option_str = strdup(name);
- if (!option_str)
- return -ENOMEM;
-
- if (parse_option_name(&option_str, &plugin) < 0)
- return -ENOMEM;
-
- /* If the option exists, update the val */
- for (op = trace_plugin_options; op; op = op->next) {
- /* Both must be NULL or not NULL */
- if ((!plugin || !op->plugin) && plugin != op->plugin)
- continue;
- if (plugin && strcmp(plugin, op->plugin) != 0)
- continue;
- if (strcmp(op->option, option_str) != 0)
- continue;
-
- /* update option */
- free(op->value);
- if (val) {
- op->value = strdup(val);
- if (!op->value)
- goto out_free;
- } else
- op->value = NULL;
-
- /* plugin and option_str don't get freed at the end */
- free(plugin);
- free(option_str);
-
- plugin = op->plugin;
- option_str = op->option;
- break;
- }
-
- /* If not found, create */
- if (!op) {
- op = malloc(sizeof(*op));
- if (!op)
- goto out_free;
- memset(op, 0, sizeof(*op));
- op->plugin = plugin;
- op->option = option_str;
- if (val) {
- op->value = strdup(val);
- if (!op->value) {
- free(op);
- goto out_free;
- }
- }
- op->next = trace_plugin_options;
- trace_plugin_options = op;
- }
-
- return process_option(plugin, option_str, val);
-
-out_free:
- free(plugin);
- free(option_str);
- return -ENOMEM;
-}
-
-static void print_op_data(struct trace_seq *s, const char *name,
- const char *op)
-{
- if (op)
- trace_seq_printf(s, "%8s:\t%s\n", name, op);
-}
-
-/**
- * tep_plugin_print_options - print out the registered plugin options
- * @s: The trace_seq descriptor to write the plugin options into
- *
- * Writes a list of options into trace_seq @s.
- */
-void tep_plugin_print_options(struct trace_seq *s)
-{
- struct registered_plugin_options *reg;
- struct tep_plugin_option *op;
-
- for (reg = registered_options; reg; reg = reg->next) {
- if (reg != registered_options)
- trace_seq_printf(s, "============\n");
- for (op = reg->options; op->name; op++) {
- if (op != reg->options)
- trace_seq_printf(s, "------------\n");
- print_op_data(s, "file", op->file);
- print_op_data(s, "plugin", op->plugin_alias);
- print_op_data(s, "option", op->name);
- print_op_data(s, "desc", op->description);
- print_op_data(s, "value", op->value);
- trace_seq_printf(s, "%8s:\t%d\n", "set", op->set);
- }
- }
-}
-
-/**
- * tep_print_plugins - print out the list of plugins loaded
- * @s: the trace_seq descripter to write to
- * @prefix: The prefix string to add before listing the option name
- * @suffix: The suffix string ot append after the option name
- * @list: The list of plugins (usually returned by tep_load_plugins()
- *
- * Writes to the trace_seq @s the list of plugins (files) that is
- * returned by tep_load_plugins(). Use @prefix and @suffix for formating:
- * @prefix = " ", @suffix = "\n".
- */
-void tep_print_plugins(struct trace_seq *s,
- const char *prefix, const char *suffix,
- const struct tep_plugin_list *list)
-{
- while (list) {
- trace_seq_printf(s, "%s%s%s", prefix, list->name, suffix);
- list = list->next;
- }
-}
-
-static void
-load_plugin(struct tep_handle *tep, const char *path,
- const char *file, void *data)
-{
- struct tep_plugin_list **plugin_list = data;
- struct tep_plugin_option *options;
- tep_plugin_load_func func;
- struct tep_plugin_list *list;
- const char *alias;
- char *plugin;
- void *handle;
- int ret;
-
- ret = asprintf(&plugin, "%s/%s", path, file);
- if (ret < 0) {
- warning("could not allocate plugin memory\n");
- return;
- }
-
- handle = dlopen(plugin, RTLD_NOW | RTLD_GLOBAL);
- if (!handle) {
- warning("could not load plugin '%s'\n%s\n",
- plugin, dlerror());
- goto out_free;
- }
-
- alias = dlsym(handle, TEP_PLUGIN_ALIAS_NAME);
- if (!alias)
- alias = file;
-
- options = dlsym(handle, TEP_PLUGIN_OPTIONS_NAME);
- if (options) {
- while (options->name) {
- ret = update_option(alias, options);
- if (ret < 0)
- goto out_free;
- options++;
- }
- }
-
- func = dlsym(handle, TEP_PLUGIN_LOADER_NAME);
- if (!func) {
- warning("could not find func '%s' in plugin '%s'\n%s\n",
- TEP_PLUGIN_LOADER_NAME, plugin, dlerror());
- goto out_free;
- }
-
- list = malloc(sizeof(*list));
- if (!list) {
- warning("could not allocate plugin memory\n");
- goto out_free;
- }
-
- list->next = *plugin_list;
- list->handle = handle;
- list->name = plugin;
- *plugin_list = list;
-
- pr_stat("registering plugin: %s", plugin);
- func(tep);
- return;
-
- out_free:
- free(plugin);
-}
-
-static void
-load_plugins_dir(struct tep_handle *tep, const char *suffix,
- const char *path,
- void (*load_plugin)(struct tep_handle *tep,
- const char *path,
- const char *name,
- void *data),
- void *data)
-{
- struct dirent *dent;
- struct stat st;
- DIR *dir;
- int ret;
-
- ret = stat(path, &st);
- if (ret < 0)
- return;
-
- if (!S_ISDIR(st.st_mode))
- return;
-
- dir = opendir(path);
- if (!dir)
- return;
-
- while ((dent = readdir(dir))) {
- const char *name = dent->d_name;
-
- if (strcmp(name, ".") == 0 ||
- strcmp(name, "..") == 0)
- continue;
-
- /* Only load plugins that end in suffix */
- if (strcmp(name + (strlen(name) - strlen(suffix)), suffix) != 0)
- continue;
-
- load_plugin(tep, path, name, data);
- }
-
- closedir(dir);
-}
-
-/**
- * tep_load_plugins_hook - call a user specified callback to load a plugin
- * @tep: handler to traceevent context
- * @suffix: filter only plugin files with given suffix
- * @load_plugin: user specified callback, called for each plugin file
- * @data: custom context, passed to @load_plugin
- *
- * Searches for traceevent plugin files and calls @load_plugin for each
- * The order of plugins search is:
- * - Directories, specified in @tep->plugins_dir and priority TEP_PLUGIN_FIRST
- * - Directory, specified at compile time with PLUGIN_TRACEEVENT_DIR
- * - Directory, specified by environment variable TRACEEVENT_PLUGIN_DIR
- * - In user's home: ~/.local/lib/traceevent/plugins/
- * - Directories, specified in @tep->plugins_dir and priority TEP_PLUGIN_LAST
- *
- */
-void tep_load_plugins_hook(struct tep_handle *tep, const char *suffix,
- void (*load_plugin)(struct tep_handle *tep,
- const char *path,
- const char *name,
- void *data),
- void *data)
-{
- struct tep_plugins_dir *dir = NULL;
- char *home;
- char *path;
- char *envdir;
- int ret;
-
- if (tep && tep->flags & TEP_DISABLE_PLUGINS)
- return;
-
- if (tep)
- dir = tep->plugins_dir;
- while (dir) {
- if (dir->prio == TEP_PLUGIN_FIRST)
- load_plugins_dir(tep, suffix, dir->path,
- load_plugin, data);
- dir = dir->next;
- }
-
- /*
- * If a system plugin directory was defined,
- * check that first.
- */
-#ifdef PLUGIN_DIR
- if (!tep || !(tep->flags & TEP_DISABLE_SYS_PLUGINS))
- load_plugins_dir(tep, suffix, PLUGIN_DIR,
- load_plugin, data);
-#endif
-
- /*
- * Next let the environment-set plugin directory
- * override the system defaults.
- */
- envdir = getenv("TRACEEVENT_PLUGIN_DIR");
- if (envdir)
- load_plugins_dir(tep, suffix, envdir, load_plugin, data);
-
- /*
- * Now let the home directory override the environment
- * or system defaults.
- */
- home = getenv("HOME");
- if (!home)
- return;
-
- ret = asprintf(&path, "%s/%s", home, LOCAL_PLUGIN_DIR);
- if (ret < 0) {
- warning("could not allocate plugin memory\n");
- return;
- }
-
- load_plugins_dir(tep, suffix, path, load_plugin, data);
-
- if (tep)
- dir = tep->plugins_dir;
- while (dir) {
- if (dir->prio == TEP_PLUGIN_LAST)
- load_plugins_dir(tep, suffix, dir->path,
- load_plugin, data);
- dir = dir->next;
- }
-
- free(path);
-}
-
-struct tep_plugin_list*
-tep_load_plugins(struct tep_handle *tep)
-{
- struct tep_plugin_list *list = NULL;
-
- tep_load_plugins_hook(tep, ".so", load_plugin, &list);
- return list;
-}
-
-/**
- * tep_add_plugin_path - Add a new plugin directory.
- * @tep: Trace event handler.
- * @path: Path to a directory. All plugin files in that
- * directory will be loaded.
- *@prio: Load priority of the plugins in that directory.
- *
- * Returns -1 in case of an error, 0 otherwise.
- */
-int tep_add_plugin_path(struct tep_handle *tep, char *path,
- enum tep_plugin_load_priority prio)
-{
- struct tep_plugins_dir *dir;
-
- if (!tep || !path)
- return -1;
-
- dir = calloc(1, sizeof(*dir));
- if (!dir)
- return -1;
-
- dir->path = strdup(path);
- if (!dir->path) {
- free(dir);
- return -1;
- }
- dir->prio = prio;
- dir->next = tep->plugins_dir;
- tep->plugins_dir = dir;
-
- return 0;
-}
-
-__hidden void free_tep_plugin_paths(struct tep_handle *tep)
-{
- struct tep_plugins_dir *dir;
-
- if (!tep)
- return;
-
- dir = tep->plugins_dir;
- while (dir) {
- tep->plugins_dir = tep->plugins_dir->next;
- free(dir->path);
- free(dir);
- dir = tep->plugins_dir;
- }
-}
-
-void
-tep_unload_plugins(struct tep_plugin_list *plugin_list, struct tep_handle *tep)
-{
- tep_plugin_unload_func func;
- struct tep_plugin_list *list;
-
- while (plugin_list) {
- list = plugin_list;
- plugin_list = list->next;
- func = dlsym(list->handle, TEP_PLUGIN_UNLOADER_NAME);
- if (func)
- func(tep);
- dlclose(list->handle);
- free(list->name);
- free(list);
- }
-}
+++ /dev/null
-/* SPDX-License-Identifier: LGPL-2.1 */
-/*
- * Copyright (C) 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-#ifndef __UTIL_H
-#define __UTIL_H
-
-#include <ctype.h>
-
-/* Can be overridden */
-void warning(const char *fmt, ...);
-void pr_stat(const char *fmt, ...);
-void vpr_stat(const char *fmt, va_list ap);
-
-/* Always available */
-void __warning(const char *fmt, ...);
-void __pr_stat(const char *fmt, ...);
-
-void __vwarning(const char *fmt, ...);
-void __vpr_stat(const char *fmt, ...);
-
-#define min(x, y) ({ \
- typeof(x) _min1 = (x); \
- typeof(y) _min2 = (y); \
- (void) (&_min1 == &_min2); \
- _min1 < _min2 ? _min1 : _min2; })
-
-static inline char *strim(char *string)
-{
- char *ret;
-
- if (!string)
- return NULL;
- while (*string) {
- if (!isspace(*string))
- break;
- string++;
- }
- ret = string;
-
- string = ret + strlen(ret) - 1;
- while (string > ret) {
- if (!isspace(*string))
- break;
- string--;
- }
- string[1] = 0;
-
- return ret;
-}
-
-static inline int has_text(const char *text)
-{
- if (!text)
- return 0;
-
- while (*text) {
- if (!isspace(*text))
- return 1;
- text++;
- }
-
- return 0;
-}
-
-#endif
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include "kbuffer.h"
-
-#define MISSING_EVENTS (1UL << 31)
-#define MISSING_STORED (1UL << 30)
-
-#define COMMIT_MASK ((1 << 27) - 1)
-
-enum {
- KBUFFER_FL_HOST_BIG_ENDIAN = (1<<0),
- KBUFFER_FL_BIG_ENDIAN = (1<<1),
- KBUFFER_FL_LONG_8 = (1<<2),
- KBUFFER_FL_OLD_FORMAT = (1<<3),
-};
-
-#define ENDIAN_MASK (KBUFFER_FL_HOST_BIG_ENDIAN | KBUFFER_FL_BIG_ENDIAN)
-
-/** kbuffer
- * @timestamp - timestamp of current event
- * @lost_events - # of lost events between this subbuffer and previous
- * @flags - special flags of the kbuffer
- * @subbuffer - pointer to the sub-buffer page
- * @data - pointer to the start of data on the sub-buffer page
- * @index - index from @data to the @curr event data
- * @curr - offset from @data to the start of current event
- * (includes metadata)
- * @next - offset from @data to the start of next event
- * @size - The size of data on @data
- * @start - The offset from @subbuffer where @data lives
- *
- * @read_4 - Function to read 4 raw bytes (may swap)
- * @read_8 - Function to read 8 raw bytes (may swap)
- * @read_long - Function to read a long word (4 or 8 bytes with needed swap)
- */
-struct kbuffer {
- unsigned long long timestamp;
- long long lost_events;
- unsigned long flags;
- void *subbuffer;
- void *data;
- unsigned int index;
- unsigned int curr;
- unsigned int next;
- unsigned int size;
- unsigned int start;
-
- unsigned int (*read_4)(void *ptr);
- unsigned long long (*read_8)(void *ptr);
- unsigned long long (*read_long)(struct kbuffer *kbuf, void *ptr);
- int (*next_event)(struct kbuffer *kbuf);
-};
-
-static void *zmalloc(size_t size)
-{
- return calloc(1, size);
-}
-
-static int host_is_bigendian(void)
-{
- unsigned char str[] = { 0x1, 0x2, 0x3, 0x4 };
- unsigned int *ptr;
-
- ptr = (unsigned int *)str;
- return *ptr == 0x01020304;
-}
-
-static int do_swap(struct kbuffer *kbuf)
-{
- return ((kbuf->flags & KBUFFER_FL_HOST_BIG_ENDIAN) + kbuf->flags) &
- ENDIAN_MASK;
-}
-
-static unsigned long long __read_8(void *ptr)
-{
- unsigned long long data = *(unsigned long long *)ptr;
-
- return data;
-}
-
-static unsigned long long __read_8_sw(void *ptr)
-{
- unsigned long long data = *(unsigned long long *)ptr;
- unsigned long long swap;
-
- swap = ((data & 0xffULL) << 56) |
- ((data & (0xffULL << 8)) << 40) |
- ((data & (0xffULL << 16)) << 24) |
- ((data & (0xffULL << 24)) << 8) |
- ((data & (0xffULL << 32)) >> 8) |
- ((data & (0xffULL << 40)) >> 24) |
- ((data & (0xffULL << 48)) >> 40) |
- ((data & (0xffULL << 56)) >> 56);
-
- return swap;
-}
-
-static unsigned int __read_4(void *ptr)
-{
- unsigned int data = *(unsigned int *)ptr;
-
- return data;
-}
-
-static unsigned int __read_4_sw(void *ptr)
-{
- unsigned int data = *(unsigned int *)ptr;
- unsigned int swap;
-
- swap = ((data & 0xffULL) << 24) |
- ((data & (0xffULL << 8)) << 8) |
- ((data & (0xffULL << 16)) >> 8) |
- ((data & (0xffULL << 24)) >> 24);
-
- return swap;
-}
-
-static unsigned long long read_8(struct kbuffer *kbuf, void *ptr)
-{
- return kbuf->read_8(ptr);
-}
-
-static unsigned int read_4(struct kbuffer *kbuf, void *ptr)
-{
- return kbuf->read_4(ptr);
-}
-
-static unsigned long long __read_long_8(struct kbuffer *kbuf, void *ptr)
-{
- return kbuf->read_8(ptr);
-}
-
-static unsigned long long __read_long_4(struct kbuffer *kbuf, void *ptr)
-{
- return kbuf->read_4(ptr);
-}
-
-static unsigned long long read_long(struct kbuffer *kbuf, void *ptr)
-{
- return kbuf->read_long(kbuf, ptr);
-}
-
-static int calc_index(struct kbuffer *kbuf, void *ptr)
-{
- return (unsigned long)ptr - (unsigned long)kbuf->data;
-}
-
-static int __next_event(struct kbuffer *kbuf);
-
-/**
- * kbuffer_alloc - allocat a new kbuffer
- * @size; enum to denote size of word
- * @endian: enum to denote endianness
- *
- * Allocates and returns a new kbuffer.
- */
-struct kbuffer *
-kbuffer_alloc(enum kbuffer_long_size size, enum kbuffer_endian endian)
-{
- struct kbuffer *kbuf;
- int flags = 0;
-
- switch (size) {
- case KBUFFER_LSIZE_4:
- break;
- case KBUFFER_LSIZE_8:
- flags |= KBUFFER_FL_LONG_8;
- break;
- default:
- return NULL;
- }
-
- switch (endian) {
- case KBUFFER_ENDIAN_LITTLE:
- break;
- case KBUFFER_ENDIAN_BIG:
- flags |= KBUFFER_FL_BIG_ENDIAN;
- break;
- default:
- return NULL;
- }
-
- kbuf = zmalloc(sizeof(*kbuf));
- if (!kbuf)
- return NULL;
-
- kbuf->flags = flags;
-
- if (host_is_bigendian())
- kbuf->flags |= KBUFFER_FL_HOST_BIG_ENDIAN;
-
- if (do_swap(kbuf)) {
- kbuf->read_8 = __read_8_sw;
- kbuf->read_4 = __read_4_sw;
- } else {
- kbuf->read_8 = __read_8;
- kbuf->read_4 = __read_4;
- }
-
- if (kbuf->flags & KBUFFER_FL_LONG_8)
- kbuf->read_long = __read_long_8;
- else
- kbuf->read_long = __read_long_4;
-
- /* May be changed by kbuffer_set_old_format() */
- kbuf->next_event = __next_event;
-
- return kbuf;
-}
-
-/** kbuffer_free - free an allocated kbuffer
- * @kbuf: The kbuffer to free
- *
- * Can take NULL as a parameter.
- */
-void kbuffer_free(struct kbuffer *kbuf)
-{
- free(kbuf);
-}
-
-static unsigned int type4host(struct kbuffer *kbuf,
- unsigned int type_len_ts)
-{
- if (kbuf->flags & KBUFFER_FL_BIG_ENDIAN)
- return (type_len_ts >> 29) & 3;
- else
- return type_len_ts & 3;
-}
-
-static unsigned int len4host(struct kbuffer *kbuf,
- unsigned int type_len_ts)
-{
- if (kbuf->flags & KBUFFER_FL_BIG_ENDIAN)
- return (type_len_ts >> 27) & 7;
- else
- return (type_len_ts >> 2) & 7;
-}
-
-static unsigned int type_len4host(struct kbuffer *kbuf,
- unsigned int type_len_ts)
-{
- if (kbuf->flags & KBUFFER_FL_BIG_ENDIAN)
- return (type_len_ts >> 27) & ((1 << 5) - 1);
- else
- return type_len_ts & ((1 << 5) - 1);
-}
-
-static unsigned int ts4host(struct kbuffer *kbuf,
- unsigned int type_len_ts)
-{
- if (kbuf->flags & KBUFFER_FL_BIG_ENDIAN)
- return type_len_ts & ((1 << 27) - 1);
- else
- return type_len_ts >> 5;
-}
-
-/*
- * Linux 2.6.30 and earlier (not much ealier) had a different
- * ring buffer format. It should be obsolete, but we handle it anyway.
- */
-enum old_ring_buffer_type {
- OLD_RINGBUF_TYPE_PADDING,
- OLD_RINGBUF_TYPE_TIME_EXTEND,
- OLD_RINGBUF_TYPE_TIME_STAMP,
- OLD_RINGBUF_TYPE_DATA,
-};
-
-static unsigned int old_update_pointers(struct kbuffer *kbuf)
-{
- unsigned long long extend;
- unsigned int type_len_ts;
- unsigned int type;
- unsigned int len;
- unsigned int delta;
- unsigned int length;
- void *ptr = kbuf->data + kbuf->curr;
-
- type_len_ts = read_4(kbuf, ptr);
- ptr += 4;
-
- type = type4host(kbuf, type_len_ts);
- len = len4host(kbuf, type_len_ts);
- delta = ts4host(kbuf, type_len_ts);
-
- switch (type) {
- case OLD_RINGBUF_TYPE_PADDING:
- kbuf->next = kbuf->size;
- return 0;
-
- case OLD_RINGBUF_TYPE_TIME_EXTEND:
- extend = read_4(kbuf, ptr);
- extend <<= TS_SHIFT;
- extend += delta;
- delta = extend;
- ptr += 4;
- length = 0;
- break;
-
- case OLD_RINGBUF_TYPE_TIME_STAMP:
- /* should never happen! */
- kbuf->curr = kbuf->size;
- kbuf->next = kbuf->size;
- kbuf->index = kbuf->size;
- return -1;
- default:
- if (len)
- length = len * 4;
- else {
- length = read_4(kbuf, ptr);
- length -= 4;
- ptr += 4;
- }
- break;
- }
-
- kbuf->timestamp += delta;
- kbuf->index = calc_index(kbuf, ptr);
- kbuf->next = kbuf->index + length;
-
- return type;
-}
-
-static int __old_next_event(struct kbuffer *kbuf)
-{
- int type;
-
- do {
- kbuf->curr = kbuf->next;
- if (kbuf->next >= kbuf->size)
- return -1;
- type = old_update_pointers(kbuf);
- } while (type == OLD_RINGBUF_TYPE_TIME_EXTEND || type == OLD_RINGBUF_TYPE_PADDING);
-
- return 0;
-}
-
-static unsigned int
-translate_data(struct kbuffer *kbuf, void *data, void **rptr,
- unsigned long long *delta, int *length)
-{
- unsigned long long extend;
- unsigned int type_len_ts;
- unsigned int type_len;
-
- type_len_ts = read_4(kbuf, data);
- data += 4;
-
- type_len = type_len4host(kbuf, type_len_ts);
- *delta = ts4host(kbuf, type_len_ts);
-
- switch (type_len) {
- case KBUFFER_TYPE_PADDING:
- *length = read_4(kbuf, data);
- break;
-
- case KBUFFER_TYPE_TIME_EXTEND:
- case KBUFFER_TYPE_TIME_STAMP:
- extend = read_4(kbuf, data);
- data += 4;
- extend <<= TS_SHIFT;
- extend += *delta;
- *delta = extend;
- *length = 0;
- break;
-
- case 0:
- *length = read_4(kbuf, data) - 4;
- *length = (*length + 3) & ~3;
- data += 4;
- break;
- default:
- *length = type_len * 4;
- break;
- }
-
- *rptr = data;
-
- return type_len;
-}
-
-static unsigned int update_pointers(struct kbuffer *kbuf)
-{
- unsigned long long delta;
- unsigned int type_len;
- int length;
- void *ptr = kbuf->data + kbuf->curr;
-
- type_len = translate_data(kbuf, ptr, &ptr, &delta, &length);
-
- if (type_len == KBUFFER_TYPE_TIME_STAMP)
- kbuf->timestamp = delta;
- else
- kbuf->timestamp += delta;
-
- kbuf->index = calc_index(kbuf, ptr);
- kbuf->next = kbuf->index + length;
-
- return type_len;
-}
-
-/**
- * kbuffer_translate_data - read raw data to get a record
- * @swap: Set to 1 if bytes in words need to be swapped when read
- * @data: The raw data to read
- * @size: Address to store the size of the event data.
- *
- * Returns a pointer to the event data. To determine the entire
- * record size (record metadata + data) just add the difference between
- * @data and the returned value to @size.
- */
-void *kbuffer_translate_data(int swap, void *data, unsigned int *size)
-{
- unsigned long long delta;
- struct kbuffer kbuf;
- int type_len;
- int length;
- void *ptr;
-
- if (swap) {
- kbuf.read_8 = __read_8_sw;
- kbuf.read_4 = __read_4_sw;
- kbuf.flags = host_is_bigendian() ? 0 : KBUFFER_FL_BIG_ENDIAN;
- } else {
- kbuf.read_8 = __read_8;
- kbuf.read_4 = __read_4;
- kbuf.flags = host_is_bigendian() ? KBUFFER_FL_BIG_ENDIAN: 0;
- }
-
- type_len = translate_data(&kbuf, data, &ptr, &delta, &length);
- switch (type_len) {
- case KBUFFER_TYPE_PADDING:
- case KBUFFER_TYPE_TIME_EXTEND:
- case KBUFFER_TYPE_TIME_STAMP:
- return NULL;
- }
-
- *size = length;
-
- return ptr;
-}
-
-static int __next_event(struct kbuffer *kbuf)
-{
- int type;
-
- do {
- kbuf->curr = kbuf->next;
- if (kbuf->next >= kbuf->size)
- return -1;
- type = update_pointers(kbuf);
- } while (type == KBUFFER_TYPE_TIME_EXTEND ||
- type == KBUFFER_TYPE_TIME_STAMP ||
- type == KBUFFER_TYPE_PADDING);
-
- return 0;
-}
-
-static int next_event(struct kbuffer *kbuf)
-{
- return kbuf->next_event(kbuf);
-}
-
-/**
- * kbuffer_next_event - increment the current pointer
- * @kbuf: The kbuffer to read
- * @ts: Address to store the next record's timestamp (may be NULL to ignore)
- *
- * Increments the pointers into the subbuffer of the kbuffer to point to the
- * next event so that the next kbuffer_read_event() will return a
- * new event.
- *
- * Returns the data of the next event if a new event exists on the subbuffer,
- * NULL otherwise.
- */
-void *kbuffer_next_event(struct kbuffer *kbuf, unsigned long long *ts)
-{
- int ret;
-
- if (!kbuf || !kbuf->subbuffer)
- return NULL;
-
- ret = next_event(kbuf);
- if (ret < 0)
- return NULL;
-
- if (ts)
- *ts = kbuf->timestamp;
-
- return kbuf->data + kbuf->index;
-}
-
-/**
- * kbuffer_load_subbuffer - load a new subbuffer into the kbuffer
- * @kbuf: The kbuffer to load
- * @subbuffer: The subbuffer to load into @kbuf.
- *
- * Load a new subbuffer (page) into @kbuf. This will reset all
- * the pointers and update the @kbuf timestamp. The next read will
- * return the first event on @subbuffer.
- *
- * Returns 0 on succes, -1 otherwise.
- */
-int kbuffer_load_subbuffer(struct kbuffer *kbuf, void *subbuffer)
-{
- unsigned long long flags;
- void *ptr = subbuffer;
-
- if (!kbuf || !subbuffer)
- return -1;
-
- kbuf->subbuffer = subbuffer;
-
- kbuf->timestamp = read_8(kbuf, ptr);
- ptr += 8;
-
- kbuf->curr = 0;
-
- if (kbuf->flags & KBUFFER_FL_LONG_8)
- kbuf->start = 16;
- else
- kbuf->start = 12;
-
- kbuf->data = subbuffer + kbuf->start;
-
- flags = read_long(kbuf, ptr);
- kbuf->size = (unsigned int)flags & COMMIT_MASK;
-
- if (flags & MISSING_EVENTS) {
- if (flags & MISSING_STORED) {
- ptr = kbuf->data + kbuf->size;
- kbuf->lost_events = read_long(kbuf, ptr);
- } else
- kbuf->lost_events = -1;
- } else
- kbuf->lost_events = 0;
-
- kbuf->index = 0;
- kbuf->next = 0;
-
- next_event(kbuf);
-
- return 0;
-}
-
-/**
- * kbuffer_subbuf_timestamp - read the timestamp from a sub buffer
- * @kbuf: The kbuffer to load
- * @subbuf: The subbuffer to read from.
- *
- * Return the timestamp from a subbuffer.
- */
-unsigned long long kbuffer_subbuf_timestamp(struct kbuffer *kbuf, void *subbuf)
-{
- return kbuf->read_8(subbuf);
-}
-
-/**
- * kbuffer_ptr_delta - read the delta field from a record
- * @kbuf: The kbuffer to load
- * @ptr: The record in the buffe.
- *
- * Return the timestamp delta from a record
- */
-unsigned int kbuffer_ptr_delta(struct kbuffer *kbuf, void *ptr)
-{
- unsigned int type_len_ts;
-
- type_len_ts = read_4(kbuf, ptr);
- return ts4host(kbuf, type_len_ts);
-}
-
-
-/**
- * kbuffer_read_event - read the next event in the kbuffer subbuffer
- * @kbuf: The kbuffer to read from
- * @ts: The address to store the timestamp of the event (may be NULL to ignore)
- *
- * Returns a pointer to the data part of the current event.
- * NULL if no event is left on the subbuffer.
- */
-void *kbuffer_read_event(struct kbuffer *kbuf, unsigned long long *ts)
-{
- if (!kbuf || !kbuf->subbuffer)
- return NULL;
-
- if (kbuf->curr >= kbuf->size)
- return NULL;
-
- if (ts)
- *ts = kbuf->timestamp;
- return kbuf->data + kbuf->index;
-}
-
-/**
- * kbuffer_timestamp - Return the timestamp of the current event
- * @kbuf: The kbuffer to read from
- *
- * Returns the timestamp of the current (next) event.
- */
-unsigned long long kbuffer_timestamp(struct kbuffer *kbuf)
-{
- return kbuf->timestamp;
-}
-
-/**
- * kbuffer_read_at_offset - read the event that is at offset
- * @kbuf: The kbuffer to read from
- * @offset: The offset into the subbuffer
- * @ts: The address to store the timestamp of the event (may be NULL to ignore)
- *
- * The @offset must be an index from the @kbuf subbuffer beginning.
- * If @offset is bigger than the stored subbuffer, NULL will be returned.
- *
- * Returns the data of the record that is at @offset. Note, @offset does
- * not need to be the start of the record, the offset just needs to be
- * in the record (or beginning of it).
- *
- * Note, the kbuf timestamp and pointers are updated to the
- * returned record. That is, kbuffer_read_event() will return the same
- * data and timestamp, and kbuffer_next_event() will increment from
- * this record.
- */
-void *kbuffer_read_at_offset(struct kbuffer *kbuf, int offset,
- unsigned long long *ts)
-{
- void *data;
-
- if (offset < kbuf->start)
- offset = 0;
- else
- offset -= kbuf->start;
-
- /* Reset the buffer */
- kbuffer_load_subbuffer(kbuf, kbuf->subbuffer);
- data = kbuffer_read_event(kbuf, ts);
-
- while (kbuf->curr < offset) {
- data = kbuffer_next_event(kbuf, ts);
- if (!data)
- break;
- }
-
- return data;
-}
-
-/**
- * kbuffer_subbuffer_size - the size of the loaded subbuffer
- * @kbuf: The kbuffer to read from
- *
- * Returns the size of the subbuffer. Note, this size is
- * where the last event resides. The stored subbuffer may actually be
- * bigger due to padding and such.
- */
-int kbuffer_subbuffer_size(struct kbuffer *kbuf)
-{
- return kbuf->size;
-}
-
-/**
- * kbuffer_curr_index - Return the index of the record
- * @kbuf: The kbuffer to read from
- *
- * Returns the index from the start of the data part of
- * the subbuffer to the current location. Note this is not
- * from the start of the subbuffer. An index of zero will
- * point to the first record. Use kbuffer_curr_offset() for
- * the actually offset (that can be used by kbuffer_read_at_offset())
- */
-int kbuffer_curr_index(struct kbuffer *kbuf)
-{
- return kbuf->curr;
-}
-
-/**
- * kbuffer_curr_offset - Return the offset of the record
- * @kbuf: The kbuffer to read from
- *
- * Returns the offset from the start of the subbuffer to the
- * current location.
- */
-int kbuffer_curr_offset(struct kbuffer *kbuf)
-{
- return kbuf->curr + kbuf->start;
-}
-
-/**
- * kbuffer_event_size - return the size of the event data
- * @kbuf: The kbuffer to read
- *
- * Returns the size of the event data (the payload not counting
- * the meta data of the record) of the current event.
- */
-int kbuffer_event_size(struct kbuffer *kbuf)
-{
- return kbuf->next - kbuf->index;
-}
-
-/**
- * kbuffer_curr_size - return the size of the entire record
- * @kbuf: The kbuffer to read
- *
- * Returns the size of the entire record (meta data and payload)
- * of the current event.
- */
-int kbuffer_curr_size(struct kbuffer *kbuf)
-{
- return kbuf->next - kbuf->curr;
-}
-
-/**
- * kbuffer_missed_events - return the # of missed events from last event.
- * @kbuf: The kbuffer to read from
- *
- * Returns the # of missed events (if recorded) before the current
- * event. Note, only events on the beginning of a subbuffer can
- * have missed events, all other events within the buffer will be
- * zero.
- */
-int kbuffer_missed_events(struct kbuffer *kbuf)
-{
- /* Only the first event can have missed events */
- if (kbuf->curr)
- return 0;
-
- return kbuf->lost_events;
-}
-
-/**
- * kbuffer_set_old_forma - set the kbuffer to use the old format parsing
- * @kbuf: The kbuffer to set
- *
- * This is obsolete (or should be). The first kernels to use the
- * new ring buffer had a slightly different ring buffer format
- * (2.6.30 and earlier). It is still somewhat supported by kbuffer,
- * but should not be counted on in the future.
- */
-void kbuffer_set_old_format(struct kbuffer *kbuf)
-{
- kbuf->flags |= KBUFFER_FL_OLD_FORMAT;
-
- kbuf->next_event = __old_next_event;
-}
-
-/**
- * kbuffer_start_of_data - return offset of where data starts on subbuffer
- * @kbuf: The kbuffer
- *
- * Returns the location on the subbuffer where the data starts.
- */
-int kbuffer_start_of_data(struct kbuffer *kbuf)
-{
- return kbuf->start;
-}
-
-/**
- * kbuffer_raw_get - get raw buffer info
- * @kbuf: The kbuffer
- * @subbuf: Start of mapped subbuffer
- * @info: Info descriptor to fill in
- *
- * For debugging. This can return internals of the ring buffer.
- * Expects to have info->next set to what it will read.
- * The type, length and timestamp delta will be filled in, and
- * @info->next will be updated to the next element.
- * The @subbuf is used to know if the info is passed the end of
- * data and NULL will be returned if it is.
- */
-struct kbuffer_raw_info *
-kbuffer_raw_get(struct kbuffer *kbuf, void *subbuf, struct kbuffer_raw_info *info)
-{
- unsigned long long flags;
- unsigned long long delta;
- unsigned int type_len;
- unsigned int size;
- int start;
- int length;
- void *ptr = info->next;
-
- if (!kbuf || !subbuf)
- return NULL;
-
- if (kbuf->flags & KBUFFER_FL_LONG_8)
- start = 16;
- else
- start = 12;
-
- flags = read_long(kbuf, subbuf + 8);
- size = (unsigned int)flags & COMMIT_MASK;
-
- if (ptr < subbuf || ptr >= subbuf + start + size)
- return NULL;
-
- type_len = translate_data(kbuf, ptr, &ptr, &delta, &length);
-
- info->next = ptr + length;
-
- info->type = type_len;
- info->delta = delta;
- info->length = length;
-
- return info;
-}
+++ /dev/null
-/* SPDX-License-Identifier: LGPL-2.1 */
-/*
- * Copyright (C) 2012 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-#ifndef _KBUFFER_H
-#define _KBUFFER_H
-
-#ifndef TS_SHIFT
-#define TS_SHIFT 27
-#endif
-
-enum kbuffer_endian {
- KBUFFER_ENDIAN_BIG,
- KBUFFER_ENDIAN_LITTLE,
-};
-
-enum kbuffer_long_size {
- KBUFFER_LSIZE_4,
- KBUFFER_LSIZE_8,
-};
-
-enum {
- KBUFFER_TYPE_PADDING = 29,
- KBUFFER_TYPE_TIME_EXTEND = 30,
- KBUFFER_TYPE_TIME_STAMP = 31,
-};
-
-struct kbuffer;
-
-struct kbuffer *kbuffer_alloc(enum kbuffer_long_size size, enum kbuffer_endian endian);
-void kbuffer_free(struct kbuffer *kbuf);
-int kbuffer_load_subbuffer(struct kbuffer *kbuf, void *subbuffer);
-void *kbuffer_read_event(struct kbuffer *kbuf, unsigned long long *ts);
-void *kbuffer_next_event(struct kbuffer *kbuf, unsigned long long *ts);
-unsigned long long kbuffer_timestamp(struct kbuffer *kbuf);
-unsigned long long kbuffer_subbuf_timestamp(struct kbuffer *kbuf, void *subbuf);
-unsigned int kbuffer_ptr_delta(struct kbuffer *kbuf, void *ptr);
-
-void *kbuffer_translate_data(int swap, void *data, unsigned int *size);
-
-void *kbuffer_read_at_offset(struct kbuffer *kbuf, int offset, unsigned long long *ts);
-
-int kbuffer_curr_index(struct kbuffer *kbuf);
-
-int kbuffer_curr_offset(struct kbuffer *kbuf);
-int kbuffer_curr_size(struct kbuffer *kbuf);
-int kbuffer_event_size(struct kbuffer *kbuf);
-int kbuffer_missed_events(struct kbuffer *kbuf);
-int kbuffer_subbuffer_size(struct kbuffer *kbuf);
-
-void kbuffer_set_old_format(struct kbuffer *kbuf);
-int kbuffer_start_of_data(struct kbuffer *kbuf);
-
-/* Debugging */
-
-struct kbuffer_raw_info {
- int type;
- int length;
- unsigned long long delta;
- void *next;
-};
-
-/* Read raw data */
-struct kbuffer_raw_info *kbuffer_raw_get(struct kbuffer *kbuf, void *subbuf,
- struct kbuffer_raw_info *info);
-
-#endif /* _K_BUFFER_H */
+++ /dev/null
-prefix=INSTALL_PREFIX
-libdir=LIB_DIR
-includedir=HEADER_DIR
-
-Name: libtraceevent
-URL: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
-Description: Linux kernel trace event library
-Version: LIB_VERSION
-Cflags: -I${includedir}
-Libs: -L${libdir} -ltraceevent
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <stdarg.h>
-#include <errno.h>
-#include <sys/types.h>
-
-#include "event-parse.h"
-#include "event-parse-local.h"
-#include "event-utils.h"
-
-#define COMM "COMM"
-#define CPU "CPU"
-
-static struct tep_format_field comm = {
- .name = "COMM",
-};
-
-static struct tep_format_field cpu = {
- .name = "CPU",
-};
-
-struct event_list {
- struct event_list *next;
- struct tep_event *event;
-};
-
-static void show_error(char *error_buf, const char *fmt, ...)
-{
- unsigned long long index;
- const char *input;
- va_list ap;
- int len;
- int i;
-
- input = get_input_buf();
- index = get_input_buf_ptr();
- len = input ? strlen(input) : 0;
-
- if (len) {
- strcpy(error_buf, input);
- error_buf[len] = '\n';
- for (i = 1; i < len && i < index; i++)
- error_buf[len+i] = ' ';
- error_buf[len + i] = '^';
- error_buf[len + i + 1] = '\n';
- len += i+2;
- }
-
- va_start(ap, fmt);
- vsnprintf(error_buf + len, TEP_FILTER_ERROR_BUFSZ - len, fmt, ap);
- va_end(ap);
-}
-
-static enum tep_event_type filter_read_token(char **tok)
-{
- enum tep_event_type type;
- char *token = NULL;
-
- do {
- free_token(token);
- type = read_token(&token);
- } while (type == TEP_EVENT_NEWLINE || type == TEP_EVENT_SPACE);
-
- /* If token is = or ! check to see if the next char is ~ */
- if (token &&
- (strcmp(token, "=") == 0 || strcmp(token, "!") == 0) &&
- peek_char() == '~') {
- /* append it */
- *tok = malloc(3);
- if (*tok == NULL) {
- free_token(token);
- return TEP_EVENT_ERROR;
- }
- sprintf(*tok, "%c%c", *token, '~');
- free_token(token);
- /* Now remove the '~' from the buffer */
- read_token(&token);
- free_token(token);
- } else
- *tok = token;
-
- return type;
-}
-
-static int filter_cmp(const void *a, const void *b)
-{
- const struct tep_filter_type *ea = a;
- const struct tep_filter_type *eb = b;
-
- if (ea->event_id < eb->event_id)
- return -1;
-
- if (ea->event_id > eb->event_id)
- return 1;
-
- return 0;
-}
-
-static struct tep_filter_type *
-find_filter_type(struct tep_event_filter *filter, int id)
-{
- struct tep_filter_type *filter_type;
- struct tep_filter_type key;
-
- key.event_id = id;
-
- filter_type = bsearch(&key, filter->event_filters,
- filter->filters,
- sizeof(*filter->event_filters),
- filter_cmp);
-
- return filter_type;
-}
-
-static struct tep_filter_type *
-add_filter_type(struct tep_event_filter *filter, int id)
-{
- struct tep_filter_type *filter_type;
- int i;
-
- filter_type = find_filter_type(filter, id);
- if (filter_type)
- return filter_type;
-
- filter_type = realloc(filter->event_filters,
- sizeof(*filter->event_filters) *
- (filter->filters + 1));
- if (!filter_type)
- return NULL;
-
- filter->event_filters = filter_type;
-
- for (i = 0; i < filter->filters; i++) {
- if (filter->event_filters[i].event_id > id)
- break;
- }
-
- if (i < filter->filters)
- memmove(&filter->event_filters[i+1],
- &filter->event_filters[i],
- sizeof(*filter->event_filters) *
- (filter->filters - i));
-
- filter_type = &filter->event_filters[i];
- filter_type->event_id = id;
- filter_type->event = tep_find_event(filter->tep, id);
- filter_type->filter = NULL;
-
- filter->filters++;
-
- return filter_type;
-}
-
-/**
- * tep_filter_alloc - create a new event filter
- * @tep: The tep that this filter is associated with
- */
-struct tep_event_filter *tep_filter_alloc(struct tep_handle *tep)
-{
- struct tep_event_filter *filter;
-
- filter = malloc(sizeof(*filter));
- if (filter == NULL)
- return NULL;
-
- memset(filter, 0, sizeof(*filter));
- filter->tep = tep;
- tep_ref(tep);
-
- return filter;
-}
-
-static struct tep_filter_arg *allocate_arg(void)
-{
- return calloc(1, sizeof(struct tep_filter_arg));
-}
-
-static void free_arg(struct tep_filter_arg *arg)
-{
- if (!arg)
- return;
-
- switch (arg->type) {
- case TEP_FILTER_ARG_NONE:
- case TEP_FILTER_ARG_BOOLEAN:
- break;
-
- case TEP_FILTER_ARG_NUM:
- free_arg(arg->num.left);
- free_arg(arg->num.right);
- break;
-
- case TEP_FILTER_ARG_EXP:
- free_arg(arg->exp.left);
- free_arg(arg->exp.right);
- break;
-
- case TEP_FILTER_ARG_STR:
- free(arg->str.val);
- regfree(&arg->str.reg);
- free(arg->str.buffer);
- break;
-
- case TEP_FILTER_ARG_VALUE:
- if (arg->value.type == TEP_FILTER_STRING ||
- arg->value.type == TEP_FILTER_CHAR)
- free(arg->value.str);
- break;
-
- case TEP_FILTER_ARG_OP:
- free_arg(arg->op.left);
- free_arg(arg->op.right);
- default:
- break;
- }
-
- free(arg);
-}
-
-static int add_event(struct event_list **events,
- struct tep_event *event)
-{
- struct event_list *list;
-
- list = malloc(sizeof(*list));
- if (list == NULL)
- return -1;
-
- list->next = *events;
- *events = list;
- list->event = event;
- return 0;
-}
-
-static int event_match(struct tep_event *event,
- regex_t *sreg, regex_t *ereg)
-{
- if (sreg) {
- return !regexec(sreg, event->system, 0, NULL, 0) &&
- !regexec(ereg, event->name, 0, NULL, 0);
- }
-
- return !regexec(ereg, event->system, 0, NULL, 0) ||
- !regexec(ereg, event->name, 0, NULL, 0);
-}
-
-static enum tep_errno
-find_event(struct tep_handle *tep, struct event_list **events,
- char *sys_name, char *event_name)
-{
- struct tep_event *event;
- regex_t ereg;
- regex_t sreg;
- int match = 0;
- int fail = 0;
- char *reg;
- int ret;
- int i;
-
- if (!event_name) {
- /* if no name is given, then swap sys and name */
- event_name = sys_name;
- sys_name = NULL;
- }
-
- ret = asprintf(®, "^%s$", event_name);
- if (ret < 0)
- return TEP_ERRNO__MEM_ALLOC_FAILED;
-
- ret = regcomp(&ereg, reg, REG_ICASE|REG_NOSUB);
- free(reg);
-
- if (ret)
- return TEP_ERRNO__INVALID_EVENT_NAME;
-
- if (sys_name) {
- ret = asprintf(®, "^%s$", sys_name);
- if (ret < 0) {
- regfree(&ereg);
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
-
- ret = regcomp(&sreg, reg, REG_ICASE|REG_NOSUB);
- free(reg);
- if (ret) {
- regfree(&ereg);
- return TEP_ERRNO__INVALID_EVENT_NAME;
- }
- }
-
- for (i = 0; i < tep->nr_events; i++) {
- event = tep->events[i];
- if (event_match(event, sys_name ? &sreg : NULL, &ereg)) {
- match = 1;
- if (add_event(events, event) < 0) {
- fail = 1;
- break;
- }
- }
- }
-
- regfree(&ereg);
- if (sys_name)
- regfree(&sreg);
-
- if (!match)
- return TEP_ERRNO__EVENT_NOT_FOUND;
- if (fail)
- return TEP_ERRNO__MEM_ALLOC_FAILED;
-
- return 0;
-}
-
-static void free_events(struct event_list *events)
-{
- struct event_list *event;
-
- while (events) {
- event = events;
- events = events->next;
- free(event);
- }
-}
-
-static enum tep_errno
-create_arg_item(struct tep_event *event, const char *token,
- enum tep_event_type type, struct tep_filter_arg **parg, char *error_str)
-{
- struct tep_format_field *field;
- struct tep_filter_arg *arg;
-
- arg = allocate_arg();
- if (arg == NULL) {
- show_error(error_str, "failed to allocate filter arg");
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
-
- switch (type) {
-
- case TEP_EVENT_SQUOTE:
- case TEP_EVENT_DQUOTE:
- arg->type = TEP_FILTER_ARG_VALUE;
- arg->value.type =
- type == TEP_EVENT_DQUOTE ? TEP_FILTER_STRING : TEP_FILTER_CHAR;
- arg->value.str = strdup(token);
- if (!arg->value.str) {
- free_arg(arg);
- show_error(error_str, "failed to allocate string filter arg");
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
- break;
- case TEP_EVENT_ITEM:
- /* if it is a number, then convert it */
- if (isdigit(token[0])) {
- arg->type = TEP_FILTER_ARG_VALUE;
- arg->value.type = TEP_FILTER_NUMBER;
- arg->value.val = strtoull(token, NULL, 0);
- break;
- }
- /* Consider this a field */
- field = tep_find_any_field(event, token);
- if (!field) {
- /* If token is 'COMM' or 'CPU' then it is special */
- if (strcmp(token, COMM) == 0) {
- field = &comm;
- } else if (strcmp(token, CPU) == 0) {
- field = &cpu;
- } else {
- /* not a field, Make it false */
- arg->type = TEP_FILTER_ARG_BOOLEAN;
- arg->boolean.value = TEP_FILTER_FALSE;
- break;
- }
- }
- arg->type = TEP_FILTER_ARG_FIELD;
- arg->field.field = field;
- break;
- default:
- free_arg(arg);
- show_error(error_str, "expected a value but found %s", token);
- return TEP_ERRNO__UNEXPECTED_TYPE;
- }
- *parg = arg;
- return 0;
-}
-
-static struct tep_filter_arg *
-create_arg_op(enum tep_filter_op_type btype)
-{
- struct tep_filter_arg *arg;
-
- arg = allocate_arg();
- if (!arg)
- return NULL;
-
- arg->type = TEP_FILTER_ARG_OP;
- arg->op.type = btype;
-
- return arg;
-}
-
-static struct tep_filter_arg *
-create_arg_exp(enum tep_filter_exp_type etype)
-{
- struct tep_filter_arg *arg;
-
- arg = allocate_arg();
- if (!arg)
- return NULL;
-
- arg->type = TEP_FILTER_ARG_EXP;
- arg->exp.type = etype;
-
- return arg;
-}
-
-static struct tep_filter_arg *
-create_arg_cmp(enum tep_filter_cmp_type ctype)
-{
- struct tep_filter_arg *arg;
-
- arg = allocate_arg();
- if (!arg)
- return NULL;
-
- /* Use NUM and change if necessary */
- arg->type = TEP_FILTER_ARG_NUM;
- arg->num.type = ctype;
-
- return arg;
-}
-
-static enum tep_errno
-add_right(struct tep_filter_arg *op, struct tep_filter_arg *arg, char *error_str)
-{
- struct tep_filter_arg *left;
- char *str;
- int op_type;
- int ret;
-
- switch (op->type) {
- case TEP_FILTER_ARG_EXP:
- if (op->exp.right)
- goto out_fail;
- op->exp.right = arg;
- break;
-
- case TEP_FILTER_ARG_OP:
- if (op->op.right)
- goto out_fail;
- op->op.right = arg;
- break;
-
- case TEP_FILTER_ARG_NUM:
- if (op->op.right)
- goto out_fail;
- /*
- * The arg must be num, str, or field
- */
- switch (arg->type) {
- case TEP_FILTER_ARG_VALUE:
- case TEP_FILTER_ARG_FIELD:
- break;
- default:
- show_error(error_str, "Illegal rvalue");
- return TEP_ERRNO__ILLEGAL_RVALUE;
- }
-
- /*
- * Depending on the type, we may need to
- * convert this to a string or regex.
- */
- switch (arg->value.type) {
- case TEP_FILTER_CHAR:
- /*
- * A char should be converted to number if
- * the string is 1 byte, and the compare
- * is not a REGEX.
- */
- if (strlen(arg->value.str) == 1 &&
- op->num.type != TEP_FILTER_CMP_REGEX &&
- op->num.type != TEP_FILTER_CMP_NOT_REGEX) {
- arg->value.type = TEP_FILTER_NUMBER;
- goto do_int;
- }
- /* fall through */
- case TEP_FILTER_STRING:
-
- /* convert op to a string arg */
- op_type = op->num.type;
- left = op->num.left;
- str = arg->value.str;
-
- /* reset the op for the new field */
- memset(op, 0, sizeof(*op));
-
- /*
- * If left arg was a field not found then
- * NULL the entire op.
- */
- if (left->type == TEP_FILTER_ARG_BOOLEAN) {
- free_arg(left);
- free_arg(arg);
- op->type = TEP_FILTER_ARG_BOOLEAN;
- op->boolean.value = TEP_FILTER_FALSE;
- break;
- }
-
- /* Left arg must be a field */
- if (left->type != TEP_FILTER_ARG_FIELD) {
- show_error(error_str,
- "Illegal lvalue for string comparison");
- return TEP_ERRNO__ILLEGAL_LVALUE;
- }
-
- /* Make sure this is a valid string compare */
- switch (op_type) {
- case TEP_FILTER_CMP_EQ:
- op_type = TEP_FILTER_CMP_MATCH;
- break;
- case TEP_FILTER_CMP_NE:
- op_type = TEP_FILTER_CMP_NOT_MATCH;
- break;
-
- case TEP_FILTER_CMP_REGEX:
- case TEP_FILTER_CMP_NOT_REGEX:
- ret = regcomp(&op->str.reg, str, REG_ICASE|REG_NOSUB);
- if (ret) {
- show_error(error_str,
- "RegEx '%s' did not compute",
- str);
- return TEP_ERRNO__INVALID_REGEX;
- }
- break;
- default:
- show_error(error_str,
- "Illegal comparison for string");
- return TEP_ERRNO__ILLEGAL_STRING_CMP;
- }
-
- op->type = TEP_FILTER_ARG_STR;
- op->str.type = op_type;
- op->str.field = left->field.field;
- op->str.val = strdup(str);
- if (!op->str.val) {
- show_error(error_str, "Failed to allocate string filter");
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
- /*
- * Need a buffer to copy data for tests
- */
- op->str.buffer = malloc(op->str.field->size + 1);
- if (!op->str.buffer) {
- show_error(error_str, "Failed to allocate string filter");
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
- /* Null terminate this buffer */
- op->str.buffer[op->str.field->size] = 0;
-
- /* We no longer have left or right args */
- free_arg(arg);
- free_arg(left);
-
- break;
-
- case TEP_FILTER_NUMBER:
-
- do_int:
- switch (op->num.type) {
- case TEP_FILTER_CMP_REGEX:
- case TEP_FILTER_CMP_NOT_REGEX:
- show_error(error_str,
- "Op not allowed with integers");
- return TEP_ERRNO__ILLEGAL_INTEGER_CMP;
-
- default:
- break;
- }
-
- /* numeric compare */
- op->num.right = arg;
- break;
- default:
- goto out_fail;
- }
- break;
- default:
- goto out_fail;
- }
-
- return 0;
-
- out_fail:
- show_error(error_str, "Syntax error");
- return TEP_ERRNO__SYNTAX_ERROR;
-}
-
-static struct tep_filter_arg *
-rotate_op_right(struct tep_filter_arg *a, struct tep_filter_arg *b)
-{
- struct tep_filter_arg *arg;
-
- arg = a->op.right;
- a->op.right = b;
- return arg;
-}
-
-static enum tep_errno add_left(struct tep_filter_arg *op, struct tep_filter_arg *arg)
-{
- switch (op->type) {
- case TEP_FILTER_ARG_EXP:
- if (arg->type == TEP_FILTER_ARG_OP)
- arg = rotate_op_right(arg, op);
- op->exp.left = arg;
- break;
-
- case TEP_FILTER_ARG_OP:
- op->op.left = arg;
- break;
- case TEP_FILTER_ARG_NUM:
- if (arg->type == TEP_FILTER_ARG_OP)
- arg = rotate_op_right(arg, op);
-
- /* left arg of compares must be a field */
- if (arg->type != TEP_FILTER_ARG_FIELD &&
- arg->type != TEP_FILTER_ARG_BOOLEAN)
- return TEP_ERRNO__INVALID_ARG_TYPE;
- op->num.left = arg;
- break;
- default:
- return TEP_ERRNO__INVALID_ARG_TYPE;
- }
- return 0;
-}
-
-enum op_type {
- OP_NONE,
- OP_BOOL,
- OP_NOT,
- OP_EXP,
- OP_CMP,
-};
-
-static enum op_type process_op(const char *token,
- enum tep_filter_op_type *btype,
- enum tep_filter_cmp_type *ctype,
- enum tep_filter_exp_type *etype)
-{
- *btype = TEP_FILTER_OP_NOT;
- *etype = TEP_FILTER_EXP_NONE;
- *ctype = TEP_FILTER_CMP_NONE;
-
- if (strcmp(token, "&&") == 0)
- *btype = TEP_FILTER_OP_AND;
- else if (strcmp(token, "||") == 0)
- *btype = TEP_FILTER_OP_OR;
- else if (strcmp(token, "!") == 0)
- return OP_NOT;
-
- if (*btype != TEP_FILTER_OP_NOT)
- return OP_BOOL;
-
- /* Check for value expressions */
- if (strcmp(token, "+") == 0) {
- *etype = TEP_FILTER_EXP_ADD;
- } else if (strcmp(token, "-") == 0) {
- *etype = TEP_FILTER_EXP_SUB;
- } else if (strcmp(token, "*") == 0) {
- *etype = TEP_FILTER_EXP_MUL;
- } else if (strcmp(token, "/") == 0) {
- *etype = TEP_FILTER_EXP_DIV;
- } else if (strcmp(token, "%") == 0) {
- *etype = TEP_FILTER_EXP_MOD;
- } else if (strcmp(token, ">>") == 0) {
- *etype = TEP_FILTER_EXP_RSHIFT;
- } else if (strcmp(token, "<<") == 0) {
- *etype = TEP_FILTER_EXP_LSHIFT;
- } else if (strcmp(token, "&") == 0) {
- *etype = TEP_FILTER_EXP_AND;
- } else if (strcmp(token, "|") == 0) {
- *etype = TEP_FILTER_EXP_OR;
- } else if (strcmp(token, "^") == 0) {
- *etype = TEP_FILTER_EXP_XOR;
- } else if (strcmp(token, "~") == 0)
- *etype = TEP_FILTER_EXP_NOT;
-
- if (*etype != TEP_FILTER_EXP_NONE)
- return OP_EXP;
-
- /* Check for compares */
- if (strcmp(token, "==") == 0)
- *ctype = TEP_FILTER_CMP_EQ;
- else if (strcmp(token, "!=") == 0)
- *ctype = TEP_FILTER_CMP_NE;
- else if (strcmp(token, "<") == 0)
- *ctype = TEP_FILTER_CMP_LT;
- else if (strcmp(token, ">") == 0)
- *ctype = TEP_FILTER_CMP_GT;
- else if (strcmp(token, "<=") == 0)
- *ctype = TEP_FILTER_CMP_LE;
- else if (strcmp(token, ">=") == 0)
- *ctype = TEP_FILTER_CMP_GE;
- else if (strcmp(token, "=~") == 0)
- *ctype = TEP_FILTER_CMP_REGEX;
- else if (strcmp(token, "!~") == 0)
- *ctype = TEP_FILTER_CMP_NOT_REGEX;
- else
- return OP_NONE;
-
- return OP_CMP;
-}
-
-static int check_op_done(struct tep_filter_arg *arg)
-{
- switch (arg->type) {
- case TEP_FILTER_ARG_EXP:
- return arg->exp.right != NULL;
-
- case TEP_FILTER_ARG_OP:
- return arg->op.right != NULL;
-
- case TEP_FILTER_ARG_NUM:
- return arg->num.right != NULL;
-
- case TEP_FILTER_ARG_STR:
- /* A string conversion is always done */
- return 1;
-
- case TEP_FILTER_ARG_BOOLEAN:
- /* field not found, is ok */
- return 1;
-
- default:
- return 0;
- }
-}
-
-enum filter_vals {
- FILTER_VAL_NORM,
- FILTER_VAL_FALSE,
- FILTER_VAL_TRUE,
-};
-
-static enum tep_errno
-reparent_op_arg(struct tep_filter_arg *parent, struct tep_filter_arg *old_child,
- struct tep_filter_arg *arg, char *error_str)
-{
- struct tep_filter_arg *other_child;
- struct tep_filter_arg **ptr;
-
- if (parent->type != TEP_FILTER_ARG_OP &&
- arg->type != TEP_FILTER_ARG_OP) {
- show_error(error_str, "can not reparent other than OP");
- return TEP_ERRNO__REPARENT_NOT_OP;
- }
-
- /* Get the sibling */
- if (old_child->op.right == arg) {
- ptr = &old_child->op.right;
- other_child = old_child->op.left;
- } else if (old_child->op.left == arg) {
- ptr = &old_child->op.left;
- other_child = old_child->op.right;
- } else {
- show_error(error_str, "Error in reparent op, find other child");
- return TEP_ERRNO__REPARENT_FAILED;
- }
-
- /* Detach arg from old_child */
- *ptr = NULL;
-
- /* Check for root */
- if (parent == old_child) {
- free_arg(other_child);
- *parent = *arg;
- /* Free arg without recussion */
- free(arg);
- return 0;
- }
-
- if (parent->op.right == old_child)
- ptr = &parent->op.right;
- else if (parent->op.left == old_child)
- ptr = &parent->op.left;
- else {
- show_error(error_str, "Error in reparent op");
- return TEP_ERRNO__REPARENT_FAILED;
- }
-
- *ptr = arg;
-
- free_arg(old_child);
- return 0;
-}
-
-/* Returns either filter_vals (success) or tep_errno (failfure) */
-static int test_arg(struct tep_filter_arg *parent, struct tep_filter_arg *arg,
- char *error_str)
-{
- int lval, rval;
-
- switch (arg->type) {
-
- /* bad case */
- case TEP_FILTER_ARG_BOOLEAN:
- return FILTER_VAL_FALSE + arg->boolean.value;
-
- /* good cases: */
- case TEP_FILTER_ARG_STR:
- case TEP_FILTER_ARG_VALUE:
- case TEP_FILTER_ARG_FIELD:
- return FILTER_VAL_NORM;
-
- case TEP_FILTER_ARG_EXP:
- lval = test_arg(arg, arg->exp.left, error_str);
- if (lval != FILTER_VAL_NORM)
- return lval;
- rval = test_arg(arg, arg->exp.right, error_str);
- if (rval != FILTER_VAL_NORM)
- return rval;
- return FILTER_VAL_NORM;
-
- case TEP_FILTER_ARG_NUM:
- lval = test_arg(arg, arg->num.left, error_str);
- if (lval != FILTER_VAL_NORM)
- return lval;
- rval = test_arg(arg, arg->num.right, error_str);
- if (rval != FILTER_VAL_NORM)
- return rval;
- return FILTER_VAL_NORM;
-
- case TEP_FILTER_ARG_OP:
- if (arg->op.type != TEP_FILTER_OP_NOT) {
- lval = test_arg(arg, arg->op.left, error_str);
- switch (lval) {
- case FILTER_VAL_NORM:
- break;
- case FILTER_VAL_TRUE:
- if (arg->op.type == TEP_FILTER_OP_OR)
- return FILTER_VAL_TRUE;
- rval = test_arg(arg, arg->op.right, error_str);
- if (rval != FILTER_VAL_NORM)
- return rval;
-
- return reparent_op_arg(parent, arg, arg->op.right,
- error_str);
-
- case FILTER_VAL_FALSE:
- if (arg->op.type == TEP_FILTER_OP_AND)
- return FILTER_VAL_FALSE;
- rval = test_arg(arg, arg->op.right, error_str);
- if (rval != FILTER_VAL_NORM)
- return rval;
-
- return reparent_op_arg(parent, arg, arg->op.right,
- error_str);
-
- default:
- return lval;
- }
- }
-
- rval = test_arg(arg, arg->op.right, error_str);
- switch (rval) {
- case FILTER_VAL_NORM:
- default:
- break;
-
- case FILTER_VAL_TRUE:
- if (arg->op.type == TEP_FILTER_OP_OR)
- return FILTER_VAL_TRUE;
- if (arg->op.type == TEP_FILTER_OP_NOT)
- return FILTER_VAL_FALSE;
-
- return reparent_op_arg(parent, arg, arg->op.left,
- error_str);
-
- case FILTER_VAL_FALSE:
- if (arg->op.type == TEP_FILTER_OP_AND)
- return FILTER_VAL_FALSE;
- if (arg->op.type == TEP_FILTER_OP_NOT)
- return FILTER_VAL_TRUE;
-
- return reparent_op_arg(parent, arg, arg->op.left,
- error_str);
- }
-
- return rval;
- default:
- show_error(error_str, "bad arg in filter tree");
- return TEP_ERRNO__BAD_FILTER_ARG;
- }
- return FILTER_VAL_NORM;
-}
-
-/* Remove any unknown event fields */
-static int collapse_tree(struct tep_filter_arg *arg,
- struct tep_filter_arg **arg_collapsed, char *error_str)
-{
- int ret;
-
- ret = test_arg(arg, arg, error_str);
- switch (ret) {
- case FILTER_VAL_NORM:
- break;
-
- case FILTER_VAL_TRUE:
- case FILTER_VAL_FALSE:
- free_arg(arg);
- arg = allocate_arg();
- if (arg) {
- arg->type = TEP_FILTER_ARG_BOOLEAN;
- arg->boolean.value = ret == FILTER_VAL_TRUE;
- } else {
- show_error(error_str, "Failed to allocate filter arg");
- ret = TEP_ERRNO__MEM_ALLOC_FAILED;
- }
- break;
-
- default:
- /* test_arg() already set the error_str */
- free_arg(arg);
- arg = NULL;
- break;
- }
-
- *arg_collapsed = arg;
- return ret;
-}
-
-static enum tep_errno
-process_filter(struct tep_event *event, struct tep_filter_arg **parg,
- char *error_str, int not)
-{
- enum tep_event_type type;
- char *token = NULL;
- struct tep_filter_arg *current_op = NULL;
- struct tep_filter_arg *current_exp = NULL;
- struct tep_filter_arg *left_item = NULL;
- struct tep_filter_arg *arg = NULL;
- enum op_type op_type;
- enum tep_filter_op_type btype;
- enum tep_filter_exp_type etype;
- enum tep_filter_cmp_type ctype;
- enum tep_errno ret;
-
- *parg = NULL;
-
- do {
- free(token);
- type = filter_read_token(&token);
- switch (type) {
- case TEP_EVENT_SQUOTE:
- case TEP_EVENT_DQUOTE:
- case TEP_EVENT_ITEM:
- ret = create_arg_item(event, token, type, &arg, error_str);
- if (ret < 0)
- goto fail;
- if (!left_item)
- left_item = arg;
- else if (current_exp) {
- ret = add_right(current_exp, arg, error_str);
- if (ret < 0)
- goto fail;
- left_item = NULL;
- /* Not's only one one expression */
- if (not) {
- arg = NULL;
- if (current_op)
- goto fail_syntax;
- free(token);
- *parg = current_exp;
- return 0;
- }
- } else
- goto fail_syntax;
- arg = NULL;
- break;
-
- case TEP_EVENT_DELIM:
- if (*token == ',') {
- show_error(error_str, "Illegal token ','");
- ret = TEP_ERRNO__ILLEGAL_TOKEN;
- goto fail;
- }
-
- if (*token == '(') {
- if (left_item) {
- show_error(error_str,
- "Open paren can not come after item");
- ret = TEP_ERRNO__INVALID_PAREN;
- goto fail;
- }
- if (current_exp) {
- show_error(error_str,
- "Open paren can not come after expression");
- ret = TEP_ERRNO__INVALID_PAREN;
- goto fail;
- }
-
- ret = process_filter(event, &arg, error_str, 0);
- if (ret != TEP_ERRNO__UNBALANCED_PAREN) {
- if (ret == 0) {
- show_error(error_str,
- "Unbalanced number of '('");
- ret = TEP_ERRNO__UNBALANCED_PAREN;
- }
- goto fail;
- }
- ret = 0;
-
- /* A not wants just one expression */
- if (not) {
- if (current_op)
- goto fail_syntax;
- *parg = arg;
- return 0;
- }
-
- if (current_op)
- ret = add_right(current_op, arg, error_str);
- else
- current_exp = arg;
-
- if (ret < 0)
- goto fail;
-
- } else { /* ')' */
- if (!current_op && !current_exp)
- goto fail_syntax;
-
- /* Make sure everything is finished at this level */
- if (current_exp && !check_op_done(current_exp))
- goto fail_syntax;
- if (current_op && !check_op_done(current_op))
- goto fail_syntax;
-
- if (current_op)
- *parg = current_op;
- else
- *parg = current_exp;
- free(token);
- return TEP_ERRNO__UNBALANCED_PAREN;
- }
- break;
-
- case TEP_EVENT_OP:
- op_type = process_op(token, &btype, &ctype, &etype);
-
- /* All expect a left arg except for NOT */
- switch (op_type) {
- case OP_BOOL:
- /* Logic ops need a left expression */
- if (!current_exp && !current_op)
- goto fail_syntax;
- /* fall through */
- case OP_NOT:
- /* logic only processes ops and exp */
- if (left_item)
- goto fail_syntax;
- break;
- case OP_EXP:
- case OP_CMP:
- if (!left_item)
- goto fail_syntax;
- break;
- case OP_NONE:
- show_error(error_str,
- "Unknown op token %s", token);
- ret = TEP_ERRNO__UNKNOWN_TOKEN;
- goto fail;
- }
-
- ret = 0;
- switch (op_type) {
- case OP_BOOL:
- arg = create_arg_op(btype);
- if (arg == NULL)
- goto fail_alloc;
- if (current_op)
- ret = add_left(arg, current_op);
- else
- ret = add_left(arg, current_exp);
- current_op = arg;
- current_exp = NULL;
- break;
-
- case OP_NOT:
- arg = create_arg_op(btype);
- if (arg == NULL)
- goto fail_alloc;
- if (current_op)
- ret = add_right(current_op, arg, error_str);
- if (ret < 0)
- goto fail;
- current_exp = arg;
- ret = process_filter(event, &arg, error_str, 1);
- if (ret < 0)
- goto fail;
- ret = add_right(current_exp, arg, error_str);
- if (ret < 0)
- goto fail;
- break;
-
- case OP_EXP:
- case OP_CMP:
- if (op_type == OP_EXP)
- arg = create_arg_exp(etype);
- else
- arg = create_arg_cmp(ctype);
- if (arg == NULL)
- goto fail_alloc;
-
- if (current_op)
- ret = add_right(current_op, arg, error_str);
- if (ret < 0)
- goto fail;
- ret = add_left(arg, left_item);
- if (ret < 0) {
- arg = NULL;
- goto fail_syntax;
- }
- current_exp = arg;
- break;
- default:
- break;
- }
- arg = NULL;
- if (ret < 0)
- goto fail_syntax;
- break;
- case TEP_EVENT_NONE:
- break;
- case TEP_EVENT_ERROR:
- goto fail_alloc;
- default:
- goto fail_syntax;
- }
- } while (type != TEP_EVENT_NONE);
-
- if (!current_op && !current_exp)
- goto fail_syntax;
-
- if (!current_op)
- current_op = current_exp;
-
- ret = collapse_tree(current_op, parg, error_str);
- /* collapse_tree() may free current_op, and updates parg accordingly */
- current_op = NULL;
- if (ret < 0)
- goto fail;
-
- free(token);
- return 0;
-
- fail_alloc:
- show_error(error_str, "failed to allocate filter arg");
- ret = TEP_ERRNO__MEM_ALLOC_FAILED;
- goto fail;
- fail_syntax:
- show_error(error_str, "Syntax error");
- ret = TEP_ERRNO__SYNTAX_ERROR;
- fail:
- free_arg(current_op);
- free_arg(current_exp);
- free_arg(arg);
- free(token);
- return ret;
-}
-
-static enum tep_errno
-process_event(struct tep_event *event, const char *filter_str,
- struct tep_filter_arg **parg, char *error_str)
-{
- int ret;
-
- init_input_buf(filter_str, strlen(filter_str));
-
- ret = process_filter(event, parg, error_str, 0);
- if (ret < 0)
- return ret;
-
- /* If parg is NULL, then make it into FALSE */
- if (!*parg) {
- *parg = allocate_arg();
- if (*parg == NULL)
- return TEP_ERRNO__MEM_ALLOC_FAILED;
-
- (*parg)->type = TEP_FILTER_ARG_BOOLEAN;
- (*parg)->boolean.value = TEP_FILTER_FALSE;
- }
-
- return 0;
-}
-
-static enum tep_errno
-filter_event(struct tep_event_filter *filter, struct tep_event *event,
- const char *filter_str, char *error_str)
-{
- struct tep_filter_type *filter_type;
- struct tep_filter_arg *arg;
- enum tep_errno ret;
-
- if (filter_str) {
- ret = process_event(event, filter_str, &arg, error_str);
- if (ret < 0)
- return ret;
-
- } else {
- /* just add a TRUE arg */
- arg = allocate_arg();
- if (arg == NULL)
- return TEP_ERRNO__MEM_ALLOC_FAILED;
-
- arg->type = TEP_FILTER_ARG_BOOLEAN;
- arg->boolean.value = TEP_FILTER_TRUE;
- }
-
- filter_type = add_filter_type(filter, event->id);
- if (filter_type == NULL) {
- free_arg(arg);
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
-
- if (filter_type->filter)
- free_arg(filter_type->filter);
- filter_type->filter = arg;
-
- return 0;
-}
-
-static void filter_init_error_buf(struct tep_event_filter *filter)
-{
- /* clear buffer to reset show error */
- init_input_buf("", 0);
- filter->error_buffer[0] = '\0';
-}
-
-/**
- * tep_filter_add_filter_str - add a new filter
- * @filter: the event filter to add to
- * @filter_str: the filter string that contains the filter
- *
- * Returns 0 if the filter was successfully added or a
- * negative error code. Use tep_filter_strerror() to see
- * actual error message in case of error.
- */
-enum tep_errno tep_filter_add_filter_str(struct tep_event_filter *filter,
- const char *filter_str)
-{
- struct tep_handle *tep = filter->tep;
- struct event_list *event;
- struct event_list *events = NULL;
- const char *filter_start;
- const char *next_event;
- char *this_event;
- char *event_name = NULL;
- char *sys_name = NULL;
- char *sp;
- enum tep_errno rtn = 0; /* TEP_ERRNO__SUCCESS */
- int len;
- int ret;
-
- filter_init_error_buf(filter);
-
- filter_start = strchr(filter_str, ':');
- if (filter_start)
- len = filter_start - filter_str;
- else
- len = strlen(filter_str);
-
- do {
- next_event = strchr(filter_str, ',');
- if (next_event &&
- (!filter_start || next_event < filter_start))
- len = next_event - filter_str;
- else if (filter_start)
- len = filter_start - filter_str;
- else
- len = strlen(filter_str);
-
- this_event = malloc(len + 1);
- if (this_event == NULL) {
- /* This can only happen when events is NULL, but still */
- free_events(events);
- return TEP_ERRNO__MEM_ALLOC_FAILED;
- }
- memcpy(this_event, filter_str, len);
- this_event[len] = 0;
-
- if (next_event)
- next_event++;
-
- filter_str = next_event;
-
- sys_name = strtok_r(this_event, "/", &sp);
- event_name = strtok_r(NULL, "/", &sp);
-
- if (!sys_name) {
- /* This can only happen when events is NULL, but still */
- free_events(events);
- free(this_event);
- return TEP_ERRNO__FILTER_NOT_FOUND;
- }
-
- /* Find this event */
- ret = find_event(tep, &events, strim(sys_name), strim(event_name));
- if (ret < 0) {
- free_events(events);
- free(this_event);
- return ret;
- }
- free(this_event);
- } while (filter_str);
-
- /* Skip the ':' */
- if (filter_start)
- filter_start++;
-
- /* filter starts here */
- for (event = events; event; event = event->next) {
- ret = filter_event(filter, event->event, filter_start,
- filter->error_buffer);
- /* Failures are returned if a parse error happened */
- if (ret < 0)
- rtn = ret;
-
- if (ret >= 0 && tep->test_filters) {
- char *test;
- test = tep_filter_make_string(filter, event->event->id);
- if (test) {
- printf(" '%s: %s'\n", event->event->name, test);
- free(test);
- }
- }
- }
-
- free_events(events);
-
- return rtn;
-}
-
-static void free_filter_type(struct tep_filter_type *filter_type)
-{
- free_arg(filter_type->filter);
-}
-
-/**
- * tep_filter_strerror - fill error message in a buffer
- * @filter: the event filter contains error
- * @err: the error code
- * @buf: the buffer to be filled in
- * @buflen: the size of the buffer
- *
- * Returns 0 if message was filled successfully, -1 if error
- */
-int tep_filter_strerror(struct tep_event_filter *filter, enum tep_errno err,
- char *buf, size_t buflen)
-{
- if (err <= __TEP_ERRNO__START || err >= __TEP_ERRNO__END)
- return -1;
-
- if (strlen(filter->error_buffer) > 0) {
- size_t len = snprintf(buf, buflen, "%s", filter->error_buffer);
-
- if (len > buflen)
- return -1;
- return 0;
- }
-
- return tep_strerror(filter->tep, err, buf, buflen);
-}
-
-/**
- * tep_filter_remove_event - remove a filter for an event
- * @filter: the event filter to remove from
- * @event_id: the event to remove a filter for
- *
- * Removes the filter saved for an event defined by @event_id
- * from the @filter.
- *
- * Returns 1: if an event was removed
- * 0: if the event was not found
- */
-int tep_filter_remove_event(struct tep_event_filter *filter,
- int event_id)
-{
- struct tep_filter_type *filter_type;
- unsigned long len;
-
- if (!filter->filters)
- return 0;
-
- filter_type = find_filter_type(filter, event_id);
-
- if (!filter_type)
- return 0;
-
- free_filter_type(filter_type);
-
- /* The filter_type points into the event_filters array */
- len = (unsigned long)(filter->event_filters + filter->filters) -
- (unsigned long)(filter_type + 1);
-
- memmove(filter_type, filter_type + 1, len);
- filter->filters--;
-
- memset(&filter->event_filters[filter->filters], 0,
- sizeof(*filter_type));
-
- return 1;
-}
-
-/**
- * tep_filter_reset - clear all filters in a filter
- * @filter: the event filter to reset
- *
- * Removes all filters from a filter and resets it.
- */
-void tep_filter_reset(struct tep_event_filter *filter)
-{
- int i;
-
- for (i = 0; i < filter->filters; i++)
- free_filter_type(&filter->event_filters[i]);
-
- free(filter->event_filters);
- filter->filters = 0;
- filter->event_filters = NULL;
-}
-
-void tep_filter_free(struct tep_event_filter *filter)
-{
- tep_unref(filter->tep);
-
- tep_filter_reset(filter);
-
- free(filter);
-}
-
-static char *arg_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg);
-
-static int copy_filter_type(struct tep_event_filter *filter,
- struct tep_event_filter *source,
- struct tep_filter_type *filter_type)
-{
- struct tep_filter_arg *arg;
- struct tep_event *event;
- const char *sys;
- const char *name;
- char *str;
-
- /* Can't assume that the tep's are the same */
- sys = filter_type->event->system;
- name = filter_type->event->name;
- event = tep_find_event_by_name(filter->tep, sys, name);
- if (!event)
- return -1;
-
- str = arg_to_str(source, filter_type->filter);
- if (!str)
- return -1;
-
- if (strcmp(str, "TRUE") == 0 || strcmp(str, "FALSE") == 0) {
- /* Add trivial event */
- arg = allocate_arg();
- if (arg == NULL) {
- free(str);
- return -1;
- }
-
- arg->type = TEP_FILTER_ARG_BOOLEAN;
- if (strcmp(str, "TRUE") == 0)
- arg->boolean.value = 1;
- else
- arg->boolean.value = 0;
-
- filter_type = add_filter_type(filter, event->id);
- if (filter_type == NULL) {
- free(str);
- free_arg(arg);
- return -1;
- }
-
- filter_type->filter = arg;
-
- free(str);
- return 0;
- }
-
- filter_event(filter, event, str, NULL);
- free(str);
-
- return 0;
-}
-
-/**
- * tep_filter_copy - copy a filter using another filter
- * @dest - the filter to copy to
- * @source - the filter to copy from
- *
- * Returns 0 on success and -1 if not all filters were copied
- */
-int tep_filter_copy(struct tep_event_filter *dest, struct tep_event_filter *source)
-{
- int ret = 0;
- int i;
-
- tep_filter_reset(dest);
-
- for (i = 0; i < source->filters; i++) {
- if (copy_filter_type(dest, source, &source->event_filters[i]))
- ret = -1;
- }
- return ret;
-}
-
-static int test_filter(struct tep_event *event, struct tep_filter_arg *arg,
- struct tep_record *record, enum tep_errno *err);
-
-static const char *
-get_comm(struct tep_event *event, struct tep_record *record)
-{
- const char *comm;
- int pid;
-
- pid = tep_data_pid(event->tep, record);
- comm = tep_data_comm_from_pid(event->tep, pid);
- return comm;
-}
-
-static unsigned long long
-get_value(struct tep_event *event,
- struct tep_format_field *field, struct tep_record *record)
-{
- unsigned long long val;
-
- /* Handle our dummy "comm" field */
- if (field == &comm) {
- const char *name;
-
- name = get_comm(event, record);
- return (unsigned long)name;
- }
-
- /* Handle our dummy "cpu" field */
- if (field == &cpu)
- return record->cpu;
-
- tep_read_number_field(field, record->data, &val);
-
- if (!(field->flags & TEP_FIELD_IS_SIGNED))
- return val;
-
- switch (field->size) {
- case 1:
- return (char)val;
- case 2:
- return (short)val;
- case 4:
- return (int)val;
- case 8:
- return (long long)val;
- }
- return val;
-}
-
-static unsigned long long
-get_arg_value(struct tep_event *event, struct tep_filter_arg *arg,
- struct tep_record *record, enum tep_errno *err);
-
-static unsigned long long
-get_exp_value(struct tep_event *event, struct tep_filter_arg *arg,
- struct tep_record *record, enum tep_errno *err)
-{
- unsigned long long lval, rval;
-
- lval = get_arg_value(event, arg->exp.left, record, err);
- rval = get_arg_value(event, arg->exp.right, record, err);
-
- if (*err) {
- /*
- * There was an error, no need to process anymore.
- */
- return 0;
- }
-
- switch (arg->exp.type) {
- case TEP_FILTER_EXP_ADD:
- return lval + rval;
-
- case TEP_FILTER_EXP_SUB:
- return lval - rval;
-
- case TEP_FILTER_EXP_MUL:
- return lval * rval;
-
- case TEP_FILTER_EXP_DIV:
- return lval / rval;
-
- case TEP_FILTER_EXP_MOD:
- return lval % rval;
-
- case TEP_FILTER_EXP_RSHIFT:
- return lval >> rval;
-
- case TEP_FILTER_EXP_LSHIFT:
- return lval << rval;
-
- case TEP_FILTER_EXP_AND:
- return lval & rval;
-
- case TEP_FILTER_EXP_OR:
- return lval | rval;
-
- case TEP_FILTER_EXP_XOR:
- return lval ^ rval;
-
- case TEP_FILTER_EXP_NOT:
- default:
- if (!*err)
- *err = TEP_ERRNO__INVALID_EXP_TYPE;
- }
- return 0;
-}
-
-static unsigned long long
-get_arg_value(struct tep_event *event, struct tep_filter_arg *arg,
- struct tep_record *record, enum tep_errno *err)
-{
- switch (arg->type) {
- case TEP_FILTER_ARG_FIELD:
- return get_value(event, arg->field.field, record);
-
- case TEP_FILTER_ARG_VALUE:
- if (arg->value.type != TEP_FILTER_NUMBER) {
- if (!*err)
- *err = TEP_ERRNO__NOT_A_NUMBER;
- }
- return arg->value.val;
-
- case TEP_FILTER_ARG_EXP:
- return get_exp_value(event, arg, record, err);
-
- default:
- if (!*err)
- *err = TEP_ERRNO__INVALID_ARG_TYPE;
- }
- return 0;
-}
-
-static int test_num(struct tep_event *event, struct tep_filter_arg *arg,
- struct tep_record *record, enum tep_errno *err)
-{
- unsigned long long lval, rval;
-
- lval = get_arg_value(event, arg->num.left, record, err);
- rval = get_arg_value(event, arg->num.right, record, err);
-
- if (*err) {
- /*
- * There was an error, no need to process anymore.
- */
- return 0;
- }
-
- switch (arg->num.type) {
- case TEP_FILTER_CMP_EQ:
- return lval == rval;
-
- case TEP_FILTER_CMP_NE:
- return lval != rval;
-
- case TEP_FILTER_CMP_GT:
- return lval > rval;
-
- case TEP_FILTER_CMP_LT:
- return lval < rval;
-
- case TEP_FILTER_CMP_GE:
- return lval >= rval;
-
- case TEP_FILTER_CMP_LE:
- return lval <= rval;
-
- default:
- if (!*err)
- *err = TEP_ERRNO__ILLEGAL_INTEGER_CMP;
- return 0;
- }
-}
-
-static const char *get_field_str(struct tep_filter_arg *arg, struct tep_record *record)
-{
- struct tep_event *event;
- struct tep_handle *tep;
- unsigned long long addr;
- const char *val = NULL;
- unsigned int size;
- char hex[64];
-
- /* If the field is not a string convert it */
- if (arg->str.field->flags & TEP_FIELD_IS_STRING) {
- val = record->data + arg->str.field->offset;
- size = arg->str.field->size;
-
- if (arg->str.field->flags & TEP_FIELD_IS_DYNAMIC) {
- addr = *(unsigned int *)val;
- size = addr >> 16;
- addr &= 0xffff;
- if (arg->str.field->flags & TEP_FIELD_IS_RELATIVE)
- addr += arg->str.field->offset + arg->str.field->size;
- val = record->data + addr;
- }
-
- /*
- * We need to copy the data since we can't be sure the field
- * is null terminated.
- */
- if (*(val + size - 1)) {
- /* copy it */
- memcpy(arg->str.buffer, val, arg->str.field->size);
- /* the buffer is already NULL terminated */
- val = arg->str.buffer;
- }
-
- } else {
- event = arg->str.field->event;
- tep = event->tep;
- addr = get_value(event, arg->str.field, record);
-
- if (arg->str.field->flags & (TEP_FIELD_IS_POINTER | TEP_FIELD_IS_LONG))
- /* convert to a kernel symbol */
- val = tep_find_function(tep, addr);
-
- if (val == NULL) {
- /* just use the hex of the string name */
- snprintf(hex, 64, "0x%llx", addr);
- val = hex;
- }
- }
-
- return val;
-}
-
-static int test_str(struct tep_event *event, struct tep_filter_arg *arg,
- struct tep_record *record, enum tep_errno *err)
-{
- const char *val;
-
- if (arg->str.field == &comm)
- val = get_comm(event, record);
- else
- val = get_field_str(arg, record);
-
- switch (arg->str.type) {
- case TEP_FILTER_CMP_MATCH:
- return strcmp(val, arg->str.val) == 0;
-
- case TEP_FILTER_CMP_NOT_MATCH:
- return strcmp(val, arg->str.val) != 0;
-
- case TEP_FILTER_CMP_REGEX:
- /* Returns zero on match */
- return !regexec(&arg->str.reg, val, 0, NULL, 0);
-
- case TEP_FILTER_CMP_NOT_REGEX:
- return regexec(&arg->str.reg, val, 0, NULL, 0);
-
- default:
- if (!*err)
- *err = TEP_ERRNO__ILLEGAL_STRING_CMP;
- return 0;
- }
-}
-
-static int test_op(struct tep_event *event, struct tep_filter_arg *arg,
- struct tep_record *record, enum tep_errno *err)
-{
- switch (arg->op.type) {
- case TEP_FILTER_OP_AND:
- return test_filter(event, arg->op.left, record, err) &&
- test_filter(event, arg->op.right, record, err);
-
- case TEP_FILTER_OP_OR:
- return test_filter(event, arg->op.left, record, err) ||
- test_filter(event, arg->op.right, record, err);
-
- case TEP_FILTER_OP_NOT:
- return !test_filter(event, arg->op.right, record, err);
-
- default:
- if (!*err)
- *err = TEP_ERRNO__INVALID_OP_TYPE;
- return 0;
- }
-}
-
-static int test_filter(struct tep_event *event, struct tep_filter_arg *arg,
- struct tep_record *record, enum tep_errno *err)
-{
- if (*err) {
- /*
- * There was an error, no need to process anymore.
- */
- return 0;
- }
-
- switch (arg->type) {
- case TEP_FILTER_ARG_BOOLEAN:
- /* easy case */
- return arg->boolean.value;
-
- case TEP_FILTER_ARG_OP:
- return test_op(event, arg, record, err);
-
- case TEP_FILTER_ARG_NUM:
- return test_num(event, arg, record, err);
-
- case TEP_FILTER_ARG_STR:
- return test_str(event, arg, record, err);
-
- case TEP_FILTER_ARG_EXP:
- case TEP_FILTER_ARG_VALUE:
- case TEP_FILTER_ARG_FIELD:
- /*
- * Expressions, fields and values evaluate
- * to true if they return non zero
- */
- return !!get_arg_value(event, arg, record, err);
-
- default:
- if (!*err)
- *err = TEP_ERRNO__INVALID_ARG_TYPE;
- return 0;
- }
-}
-
-/**
- * tep_event_filtered - return true if event has filter
- * @filter: filter struct with filter information
- * @event_id: event id to test if filter exists
- *
- * Returns 1 if filter found for @event_id
- * otherwise 0;
- */
-int tep_event_filtered(struct tep_event_filter *filter, int event_id)
-{
- struct tep_filter_type *filter_type;
-
- if (!filter->filters)
- return 0;
-
- filter_type = find_filter_type(filter, event_id);
-
- return filter_type ? 1 : 0;
-}
-
-/**
- * tep_filter_match - test if a record matches a filter
- * @filter: filter struct with filter information
- * @record: the record to test against the filter
- *
- * Returns: match result or error code (prefixed with TEP_ERRNO__)
- * FILTER_MATCH - filter found for event and @record matches
- * FILTER_MISS - filter found for event and @record does not match
- * FILTER_NOT_FOUND - no filter found for @record's event
- * NO_FILTER - if no filters exist
- * otherwise - error occurred during test
- */
-enum tep_errno tep_filter_match(struct tep_event_filter *filter,
- struct tep_record *record)
-{
- struct tep_handle *tep = filter->tep;
- struct tep_filter_type *filter_type;
- int event_id;
- int ret;
- enum tep_errno err = 0;
-
- filter_init_error_buf(filter);
-
- if (!filter->filters)
- return TEP_ERRNO__NO_FILTER;
-
- event_id = tep_data_type(tep, record);
-
- filter_type = find_filter_type(filter, event_id);
- if (!filter_type)
- return TEP_ERRNO__FILTER_NOT_FOUND;
-
- ret = test_filter(filter_type->event, filter_type->filter, record, &err);
- if (err)
- return err;
-
- return ret ? TEP_ERRNO__FILTER_MATCH : TEP_ERRNO__FILTER_MISS;
-}
-
-static char *op_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
-{
- char *str = NULL;
- char *left = NULL;
- char *right = NULL;
- char *op = NULL;
- int left_val = -1;
- int right_val = -1;
- int val;
-
- switch (arg->op.type) {
- case TEP_FILTER_OP_AND:
- op = "&&";
- /* fall through */
- case TEP_FILTER_OP_OR:
- if (!op)
- op = "||";
-
- left = arg_to_str(filter, arg->op.left);
- right = arg_to_str(filter, arg->op.right);
- if (!left || !right)
- break;
-
- /* Try to consolidate boolean values */
- if (strcmp(left, "TRUE") == 0)
- left_val = 1;
- else if (strcmp(left, "FALSE") == 0)
- left_val = 0;
-
- if (strcmp(right, "TRUE") == 0)
- right_val = 1;
- else if (strcmp(right, "FALSE") == 0)
- right_val = 0;
-
- if (left_val >= 0) {
- if ((arg->op.type == TEP_FILTER_OP_AND && !left_val) ||
- (arg->op.type == TEP_FILTER_OP_OR && left_val)) {
- /* Just return left value */
- str = left;
- left = NULL;
- break;
- }
- if (right_val >= 0) {
- /* just evaluate this. */
- val = 0;
- switch (arg->op.type) {
- case TEP_FILTER_OP_AND:
- val = left_val && right_val;
- break;
- case TEP_FILTER_OP_OR:
- val = left_val || right_val;
- break;
- default:
- break;
- }
- if (asprintf(&str, val ? "TRUE" : "FALSE") < 0)
- str = NULL;
- break;
- }
- }
- if (right_val >= 0) {
- if ((arg->op.type == TEP_FILTER_OP_AND && !right_val) ||
- (arg->op.type == TEP_FILTER_OP_OR && right_val)) {
- /* Just return right value */
- str = right;
- right = NULL;
- break;
- }
- /* The right value is meaningless */
- str = left;
- left = NULL;
- break;
- }
-
- if (asprintf(&str, "(%s) %s (%s)", left, op, right) < 0)
- str = NULL;
- break;
-
- case TEP_FILTER_OP_NOT:
- op = "!";
- right = arg_to_str(filter, arg->op.right);
- if (!right)
- break;
-
- /* See if we can consolidate */
- if (strcmp(right, "TRUE") == 0)
- right_val = 1;
- else if (strcmp(right, "FALSE") == 0)
- right_val = 0;
- if (right_val >= 0) {
- /* just return the opposite */
- if (asprintf(&str, right_val ? "FALSE" : "TRUE") < 0)
- str = NULL;
- break;
- }
- if (asprintf(&str, "%s(%s)", op, right) < 0)
- str = NULL;
- break;
-
- default:
- /* ?? */
- break;
- }
- free(left);
- free(right);
- return str;
-}
-
-static char *val_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
-{
- char *str = NULL;
-
- if (asprintf(&str, "%lld", arg->value.val) < 0)
- str = NULL;
-
- return str;
-}
-
-static char *field_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
-{
- return strdup(arg->field.field->name);
-}
-
-static char *exp_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
-{
- char *lstr;
- char *rstr;
- char *op;
- char *str = NULL;
-
- lstr = arg_to_str(filter, arg->exp.left);
- rstr = arg_to_str(filter, arg->exp.right);
- if (!lstr || !rstr)
- goto out;
-
- switch (arg->exp.type) {
- case TEP_FILTER_EXP_ADD:
- op = "+";
- break;
- case TEP_FILTER_EXP_SUB:
- op = "-";
- break;
- case TEP_FILTER_EXP_MUL:
- op = "*";
- break;
- case TEP_FILTER_EXP_DIV:
- op = "/";
- break;
- case TEP_FILTER_EXP_MOD:
- op = "%";
- break;
- case TEP_FILTER_EXP_RSHIFT:
- op = ">>";
- break;
- case TEP_FILTER_EXP_LSHIFT:
- op = "<<";
- break;
- case TEP_FILTER_EXP_AND:
- op = "&";
- break;
- case TEP_FILTER_EXP_OR:
- op = "|";
- break;
- case TEP_FILTER_EXP_XOR:
- op = "^";
- break;
- default:
- op = "[ERROR IN EXPRESSION TYPE]";
- break;
- }
-
- if (asprintf(&str, "%s %s %s", lstr, op, rstr) < 0)
- str = NULL;
-out:
- free(lstr);
- free(rstr);
-
- return str;
-}
-
-static char *num_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
-{
- char *lstr;
- char *rstr;
- char *str = NULL;
- char *op = NULL;
-
- lstr = arg_to_str(filter, arg->num.left);
- rstr = arg_to_str(filter, arg->num.right);
- if (!lstr || !rstr)
- goto out;
-
- switch (arg->num.type) {
- case TEP_FILTER_CMP_EQ:
- op = "==";
- /* fall through */
- case TEP_FILTER_CMP_NE:
- if (!op)
- op = "!=";
- /* fall through */
- case TEP_FILTER_CMP_GT:
- if (!op)
- op = ">";
- /* fall through */
- case TEP_FILTER_CMP_LT:
- if (!op)
- op = "<";
- /* fall through */
- case TEP_FILTER_CMP_GE:
- if (!op)
- op = ">=";
- /* fall through */
- case TEP_FILTER_CMP_LE:
- if (!op)
- op = "<=";
-
- if (asprintf(&str, "%s %s %s", lstr, op, rstr) < 0)
- str = NULL;
- break;
-
- default:
- /* ?? */
- break;
- }
-
-out:
- free(lstr);
- free(rstr);
- return str;
-}
-
-static char *str_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
-{
- char *str = NULL;
- char *op = NULL;
-
- switch (arg->str.type) {
- case TEP_FILTER_CMP_MATCH:
- op = "==";
- /* fall through */
- case TEP_FILTER_CMP_NOT_MATCH:
- if (!op)
- op = "!=";
- /* fall through */
- case TEP_FILTER_CMP_REGEX:
- if (!op)
- op = "=~";
- /* fall through */
- case TEP_FILTER_CMP_NOT_REGEX:
- if (!op)
- op = "!~";
-
- if (asprintf(&str, "%s %s \"%s\"",
- arg->str.field->name, op, arg->str.val) < 0)
- str = NULL;
- break;
-
- default:
- /* ?? */
- break;
- }
- return str;
-}
-
-static char *arg_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
-{
- char *str = NULL;
-
- switch (arg->type) {
- case TEP_FILTER_ARG_BOOLEAN:
- if (asprintf(&str, arg->boolean.value ? "TRUE" : "FALSE") < 0)
- str = NULL;
- return str;
-
- case TEP_FILTER_ARG_OP:
- return op_to_str(filter, arg);
-
- case TEP_FILTER_ARG_NUM:
- return num_to_str(filter, arg);
-
- case TEP_FILTER_ARG_STR:
- return str_to_str(filter, arg);
-
- case TEP_FILTER_ARG_VALUE:
- return val_to_str(filter, arg);
-
- case TEP_FILTER_ARG_FIELD:
- return field_to_str(filter, arg);
-
- case TEP_FILTER_ARG_EXP:
- return exp_to_str(filter, arg);
-
- default:
- /* ?? */
- return NULL;
- }
-
-}
-
-/**
- * tep_filter_make_string - return a string showing the filter
- * @filter: filter struct with filter information
- * @event_id: the event id to return the filter string with
- *
- * Returns a string that displays the filter contents.
- * This string must be freed with free(str).
- * NULL is returned if no filter is found or allocation failed.
- */
-char *
-tep_filter_make_string(struct tep_event_filter *filter, int event_id)
-{
- struct tep_filter_type *filter_type;
-
- if (!filter->filters)
- return NULL;
-
- filter_type = find_filter_type(filter, event_id);
-
- if (!filter_type)
- return NULL;
-
- return arg_to_str(filter, filter_type->filter);
-}
-
-/**
- * tep_filter_compare - compare two filters and return if they are the same
- * @filter1: Filter to compare with @filter2
- * @filter2: Filter to compare with @filter1
- *
- * Returns:
- * 1 if the two filters hold the same content.
- * 0 if they do not.
- */
-int tep_filter_compare(struct tep_event_filter *filter1, struct tep_event_filter *filter2)
-{
- struct tep_filter_type *filter_type1;
- struct tep_filter_type *filter_type2;
- char *str1, *str2;
- int result;
- int i;
-
- /* Do the easy checks first */
- if (filter1->filters != filter2->filters)
- return 0;
- if (!filter1->filters && !filter2->filters)
- return 1;
-
- /*
- * Now take a look at each of the events to see if they have the same
- * filters to them.
- */
- for (i = 0; i < filter1->filters; i++) {
- filter_type1 = &filter1->event_filters[i];
- filter_type2 = find_filter_type(filter2, filter_type1->event_id);
- if (!filter_type2)
- break;
- if (filter_type1->filter->type != filter_type2->filter->type)
- break;
- /* The best way to compare complex filters is with strings */
- str1 = arg_to_str(filter1, filter_type1->filter);
- str2 = arg_to_str(filter2, filter_type2->filter);
- if (str1 && str2)
- result = strcmp(str1, str2) != 0;
- else
- /* bail out if allocation fails */
- result = 1;
-
- free(str1);
- free(str2);
- if (result)
- break;
- }
-
- if (i < filter1->filters)
- return 0;
- return 1;
-}
-
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <stdarg.h>
-#include <errno.h>
-
-#define __weak __attribute__((weak))
-
-void __vwarning(const char *fmt, va_list ap)
-{
- if (errno)
- perror("libtraceevent");
- errno = 0;
-
- fprintf(stderr, " ");
- vfprintf(stderr, fmt, ap);
-
- fprintf(stderr, "\n");
-}
-
-void __warning(const char *fmt, ...)
-{
- va_list ap;
-
- va_start(ap, fmt);
- __vwarning(fmt, ap);
- va_end(ap);
-}
-
-void __weak warning(const char *fmt, ...)
-{
- va_list ap;
-
- va_start(ap, fmt);
- __vwarning(fmt, ap);
- va_end(ap);
-}
-
-void __vpr_stat(const char *fmt, va_list ap)
-{
- vprintf(fmt, ap);
- printf("\n");
-}
-
-void __pr_stat(const char *fmt, ...)
-{
- va_list ap;
-
- va_start(ap, fmt);
- __vpr_stat(fmt, ap);
- va_end(ap);
-}
-
-void __weak vpr_stat(const char *fmt, va_list ap)
-{
- __vpr_stat(fmt, ap);
-}
-
-void __weak pr_stat(const char *fmt, ...)
-{
- va_list ap;
-
- va_start(ap, fmt);
- __vpr_stat(fmt, ap);
- va_end(ap);
-}
+++ /dev/null
-plugin_jbd2-y += plugin_jbd2.o
-plugin_hrtimer-y += plugin_hrtimer.o
-plugin_kmem-y += plugin_kmem.o
-plugin_kvm-y += plugin_kvm.o
-plugin_mac80211-y += plugin_mac80211.o
-plugin_sched_switch-y += plugin_sched_switch.o
-plugin_function-y += plugin_function.o
-plugin_futex-y += plugin_futex.o
-plugin_xen-y += plugin_xen.o
-plugin_scsi-y += plugin_scsi.o
-plugin_cfg80211-y += plugin_cfg80211.o
-plugin_tlb-y += plugin_tlb.o
\ No newline at end of file
+++ /dev/null
-# SPDX-License-Identifier: GPL-2.0
-
-#MAKEFLAGS += --no-print-directory
-
-
-# Makefiles suck: This macro sets a default value of $(2) for the
-# variable named by $(1), unless the variable has been set by
-# environment or command line. This is necessary for CC and AR
-# because make sets default values, so the simpler ?= approach
-# won't work as expected.
-define allow-override
- $(if $(or $(findstring environment,$(origin $(1))),\
- $(findstring command line,$(origin $(1)))),,\
- $(eval $(1) = $(2)))
-endef
-
-# Allow setting CC and AR, or setting CROSS_COMPILE as a prefix.
-$(call allow-override,CC,$(CROSS_COMPILE)gcc)
-$(call allow-override,AR,$(CROSS_COMPILE)ar)
-$(call allow-override,NM,$(CROSS_COMPILE)nm)
-$(call allow-override,PKG_CONFIG,pkg-config)
-
-EXT = -std=gnu99
-INSTALL = install
-
-# Use DESTDIR for installing into a different root directory.
-# This is useful for building a package. The program will be
-# installed in this directory as if it was the root directory.
-# Then the build tool can move it later.
-DESTDIR ?=
-DESTDIR_SQ = '$(subst ','\'',$(DESTDIR))'
-
-LP64 := $(shell echo __LP64__ | ${CC} ${CFLAGS} -E -x c - | tail -n 1)
-ifeq ($(LP64), 1)
- libdir_relative_tmp = lib64
-else
- libdir_relative_tmp = lib
-endif
-
-libdir_relative ?= $(libdir_relative_tmp)
-prefix ?= /usr/local
-libdir = $(prefix)/$(libdir_relative)
-
-set_plugin_dir := 1
-
-# Set plugin_dir to preffered global plugin location
-# If we install under $HOME directory we go under
-# $(HOME)/.local/lib/traceevent/plugins
-#
-# We dont set PLUGIN_DIR in case we install under $HOME
-# directory, because by default the code looks under:
-# $(HOME)/.local/lib/traceevent/plugins by default.
-#
-ifeq ($(plugin_dir),)
-ifeq ($(prefix),$(HOME))
-override plugin_dir = $(HOME)/.local/lib/traceevent/plugins
-set_plugin_dir := 0
-else
-override plugin_dir = $(libdir)/traceevent/plugins
-endif
-endif
-
-ifeq ($(set_plugin_dir),1)
-PLUGIN_DIR = -DPLUGIN_DIR="$(plugin_dir)"
-PLUGIN_DIR_SQ = '$(subst ','\'',$(PLUGIN_DIR))'
-endif
-
-include ../../../scripts/Makefile.include
-
-# copy a bit from Linux kbuild
-
-ifeq ("$(origin V)", "command line")
- VERBOSE = $(V)
-endif
-ifndef VERBOSE
- VERBOSE = 0
-endif
-
-ifeq ($(srctree),)
-srctree := $(patsubst %/,%,$(dir $(CURDIR)))
-srctree := $(patsubst %/,%,$(dir $(srctree)))
-srctree := $(patsubst %/,%,$(dir $(srctree)))
-srctree := $(patsubst %/,%,$(dir $(srctree)))
-#$(info Determined 'srctree' to be $(srctree))
-endif
-
-export prefix libdir src obj
-
-# Shell quotes
-plugin_dir_SQ = $(subst ','\'',$(plugin_dir))
-
-CONFIG_INCLUDES =
-CONFIG_LIBS =
-CONFIG_FLAGS =
-
-OBJ = $@
-N =
-
-INCLUDES = -I. -I.. -I $(srctree)/tools/include $(CONFIG_INCLUDES)
-
-# Set compile option CFLAGS
-ifdef EXTRA_CFLAGS
- CFLAGS := $(EXTRA_CFLAGS)
-else
- CFLAGS := -g -Wall
-endif
-
-# Append required CFLAGS
-override CFLAGS += -fPIC
-override CFLAGS += $(CONFIG_FLAGS) $(INCLUDES) $(PLUGIN_DIR_SQ)
-override CFLAGS += $(udis86-flags) -D_GNU_SOURCE
-
-ifeq ($(VERBOSE),1)
- Q =
-else
- Q = @
-endif
-
-# Disable command line variables (CFLAGS) override from top
-# level Makefile (perf), otherwise build Makefile will get
-# the same command line setup.
-MAKEOVERRIDES=
-
-export srctree OUTPUT CC LD CFLAGS V
-
-build := -f $(srctree)/tools/build/Makefile.build dir=. obj
-
-DYNAMIC_LIST_FILE := $(OUTPUT)libtraceevent-dynamic-list
-
-PLUGINS = plugin_jbd2.so
-PLUGINS += plugin_hrtimer.so
-PLUGINS += plugin_kmem.so
-PLUGINS += plugin_kvm.so
-PLUGINS += plugin_mac80211.so
-PLUGINS += plugin_sched_switch.so
-PLUGINS += plugin_function.so
-PLUGINS += plugin_futex.so
-PLUGINS += plugin_xen.so
-PLUGINS += plugin_scsi.so
-PLUGINS += plugin_cfg80211.so
-PLUGINS += plugin_tlb.so
-
-PLUGINS := $(addprefix $(OUTPUT),$(PLUGINS))
-PLUGINS_IN := $(PLUGINS:.so=-in.o)
-
-plugins: $(PLUGINS) $(DYNAMIC_LIST_FILE)
-
-__plugin_obj = $(notdir $@)
- plugin_obj = $(__plugin_obj:-in.o=)
-
-$(PLUGINS_IN): force
- $(Q)$(MAKE) $(build)=$(plugin_obj)
-
-$(OUTPUT)libtraceevent-dynamic-list: $(PLUGINS)
- $(QUIET_GEN)$(call do_generate_dynamic_list_file, $(PLUGINS), $@)
-
-$(OUTPUT)%.so: $(OUTPUT)%-in.o
- $(QUIET_LINK)$(CC) $(CFLAGS) -shared $(LDFLAGS) -nostartfiles -o $@ $^
-
-define update_dir
- (echo $1 > $@.tmp; \
- if [ -r $@ ] && cmp -s $@ $@.tmp; then \
- rm -f $@.tmp; \
- else \
- echo ' UPDATE $@'; \
- mv -f $@.tmp $@; \
- fi);
-endef
-
-tags: force
- $(RM) tags
- find . -name '*.[ch]' | xargs ctags --extra=+f --c-kinds=+px \
- --regex-c++='/_PE\(([^,)]*).*/TEP_ERRNO__\1/'
-
-TAGS: force
- $(RM) TAGS
- find . -name '*.[ch]' | xargs etags \
- --regex='/_PE(\([^,)]*\).*/TEP_ERRNO__\1/'
-
-define do_install_mkdir
- if [ ! -d '$(DESTDIR_SQ)$1' ]; then \
- $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$1'; \
- fi
-endef
-
-define do_install
- $(call do_install_mkdir,$2); \
- $(INSTALL) $(if $3,-m $3,) $1 '$(DESTDIR_SQ)$2'
-endef
-
-define do_install_plugins
- for plugin in $1; do \
- $(call do_install,$$plugin,$(plugin_dir_SQ)); \
- done
-endef
-
-define do_generate_dynamic_list_file
- symbol_type=`$(NM) -u -D $1 | awk 'NF>1 {print $$1}' | \
- xargs echo "U w W" | tr 'w ' 'W\n' | sort -u | xargs echo`;\
- if [ "$$symbol_type" = "U W" ];then \
- (echo '{'; \
- $(NM) -u -D $1 | awk 'NF>1 {sub("@.*", "", $$2); print "\t"$$2";"}' | sort -u;\
- echo '};'; \
- ) > $2; \
- else \
- (echo Either missing one of [$1] or bad version of $(NM)) 1>&2;\
- fi
-endef
-
-install: $(PLUGINS)
- $(call QUIET_INSTALL, trace_plugins) \
- $(call do_install_plugins, $(PLUGINS))
-
-clean:
- $(call QUIET_CLEAN, trace_plugins) \
- $(RM) *.o *~ $(TARGETS) *.a *.so $(VERSION_FILES) .*.d .*.cmd; \
- $(RM) $(OUTPUT)libtraceevent-dynamic-list \
- $(RM) TRACEEVENT-CFLAGS tags TAGS;
-
-PHONY += force plugins
-force:
-
-# Declare the contents of the .PHONY variable as phony. We keep that
-# information in a variable so we can use it in if_changed and friends.
-.PHONY: $(PHONY)
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-#include <stdio.h>
-#include <string.h>
-#include <inttypes.h>
-#include <endian.h>
-#include "event-parse.h"
-
-/*
- * From glibc endian.h, for older systems where it is not present, e.g.: RHEL5,
- * Fedora6.
- */
-#ifndef le16toh
-# if __BYTE_ORDER == __LITTLE_ENDIAN
-# define le16toh(x) (x)
-# else
-# define le16toh(x) __bswap_16 (x)
-# endif
-#endif
-
-
-static unsigned long long
-process___le16_to_cpup(struct trace_seq *s, unsigned long long *args)
-{
- uint16_t *val = (uint16_t *) (unsigned long) args[0];
- return val ? (long long) le16toh(*val) : 0;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_print_function(tep,
- process___le16_to_cpup,
- TEP_FUNC_ARG_INT,
- "__le16_to_cpup",
- TEP_FUNC_ARG_PTR,
- TEP_FUNC_ARG_VOID);
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_print_function(tep, process___le16_to_cpup,
- "__le16_to_cpup");
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include "event-parse.h"
-#include "event-utils.h"
-#include "trace-seq.h"
-
-static struct func_stack {
- int size;
- char **stack;
-} *fstack;
-
-static int cpus = -1;
-
-#define STK_BLK 10
-
-struct tep_plugin_option plugin_options[] =
-{
- {
- .name = "parent",
- .plugin_alias = "ftrace",
- .description =
- "Print parent of functions for function events",
- },
- {
- .name = "indent",
- .plugin_alias = "ftrace",
- .description =
- "Try to show function call indents, based on parents",
- .set = 1,
- },
- {
- .name = "offset",
- .plugin_alias = "ftrace",
- .description =
- "Show function names as well as their offsets",
- .set = 0,
- },
- {
- .name = NULL,
- }
-};
-
-static struct tep_plugin_option *ftrace_parent = &plugin_options[0];
-static struct tep_plugin_option *ftrace_indent = &plugin_options[1];
-static struct tep_plugin_option *ftrace_offset = &plugin_options[2];
-
-static void add_child(struct func_stack *stack, const char *child, int pos)
-{
- int i;
-
- if (!child)
- return;
-
- if (pos < stack->size)
- free(stack->stack[pos]);
- else {
- char **ptr;
-
- ptr = realloc(stack->stack, sizeof(char *) *
- (stack->size + STK_BLK));
- if (!ptr) {
- warning("could not allocate plugin memory\n");
- return;
- }
-
- stack->stack = ptr;
-
- for (i = stack->size; i < stack->size + STK_BLK; i++)
- stack->stack[i] = NULL;
- stack->size += STK_BLK;
- }
-
- stack->stack[pos] = strdup(child);
-}
-
-static int add_and_get_index(const char *parent, const char *child, int cpu)
-{
- int i;
-
- if (cpu < 0)
- return 0;
-
- if (cpu > cpus) {
- struct func_stack *ptr;
-
- ptr = realloc(fstack, sizeof(*fstack) * (cpu + 1));
- if (!ptr) {
- warning("could not allocate plugin memory\n");
- return 0;
- }
-
- fstack = ptr;
-
- /* Account for holes in the cpu count */
- for (i = cpus + 1; i <= cpu; i++)
- memset(&fstack[i], 0, sizeof(fstack[i]));
- cpus = cpu;
- }
-
- for (i = 0; i < fstack[cpu].size && fstack[cpu].stack[i]; i++) {
- if (strcmp(parent, fstack[cpu].stack[i]) == 0) {
- add_child(&fstack[cpu], child, i+1);
- return i;
- }
- }
-
- /* Not found */
- add_child(&fstack[cpu], parent, 0);
- add_child(&fstack[cpu], child, 1);
- return 0;
-}
-
-static void show_function(struct trace_seq *s, struct tep_handle *tep,
- const char *func, unsigned long long function)
-{
- unsigned long long offset;
-
- trace_seq_printf(s, "%s", func);
- if (ftrace_offset->set) {
- offset = tep_find_function_address(tep, function);
- trace_seq_printf(s, "+0x%x ", (int)(function - offset));
- }
-}
-
-static int function_handler(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- struct tep_handle *tep = event->tep;
- unsigned long long function;
- unsigned long long pfunction;
- const char *func;
- const char *parent;
- int index = 0;
-
- if (tep_get_field_val(s, event, "ip", record, &function, 1))
- return trace_seq_putc(s, '!');
-
- func = tep_find_function(tep, function);
-
- if (tep_get_field_val(s, event, "parent_ip", record, &pfunction, 1))
- return trace_seq_putc(s, '!');
-
- parent = tep_find_function(tep, pfunction);
-
- if (parent && ftrace_indent->set)
- index = add_and_get_index(parent, func, record->cpu);
-
- trace_seq_printf(s, "%*s", index*3, "");
-
- if (func)
- show_function(s, tep, func, function);
- else
- trace_seq_printf(s, "0x%llx", function);
-
- if (ftrace_parent->set) {
- trace_seq_printf(s, " <-- ");
- if (parent)
- show_function(s, tep, parent, pfunction);
- else
- trace_seq_printf(s, "0x%llx", pfunction);
- }
-
- return 0;
-}
-
-static int
-trace_stack_handler(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- struct tep_format_field *field;
- unsigned long long addr;
- const char *func;
- int long_size;
- void *data = record->data;
-
- field = tep_find_any_field(event, "caller");
- if (!field) {
- trace_seq_printf(s, "<CANT FIND FIELD %s>", "caller");
- return 0;
- }
-
- trace_seq_puts(s, "<stack trace >\n");
-
- long_size = tep_get_long_size(event->tep);
-
- for (data += field->offset; data < record->data + record->size;
- data += long_size) {
- addr = tep_read_number(event->tep, data, long_size);
-
- if ((long_size == 8 && addr == (unsigned long long)-1) ||
- ((int)addr == -1))
- break;
-
- func = tep_find_function(event->tep, addr);
- if (func)
- trace_seq_printf(s, "=> %s (%llx)\n", func, addr);
- else
- trace_seq_printf(s, "=> %llx\n", addr);
- }
-
- return 0;
-}
-
-static int
-trace_raw_data_handler(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- struct tep_format_field *field;
- unsigned long long id;
- int long_size;
- void *data = record->data;
-
- if (tep_get_field_val(s, event, "id", record, &id, 1))
- return trace_seq_putc(s, '!');
-
- trace_seq_printf(s, "# %llx", id);
-
- field = tep_find_any_field(event, "buf");
- if (!field) {
- trace_seq_printf(s, "<CANT FIND FIELD %s>", "buf");
- return 0;
- }
-
- long_size = tep_get_long_size(event->tep);
-
- for (data += field->offset; data < record->data + record->size;
- data += long_size) {
- int size = sizeof(long);
- int left = (record->data + record->size) - data;
- int i;
-
- if (size > left)
- size = left;
-
- for (i = 0; i < size; i++)
- trace_seq_printf(s, " %02x", *(unsigned char *)(data + i));
- }
-
- return 0;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_event_handler(tep, -1, "ftrace", "function",
- function_handler, NULL);
-
- tep_register_event_handler(tep, -1, "ftrace", "kernel_stack",
- trace_stack_handler, NULL);
-
- tep_register_event_handler(tep, -1, "ftrace", "raw_data",
- trace_raw_data_handler, NULL);
-
- tep_plugin_add_options("ftrace", plugin_options);
-
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- int i, x;
-
- tep_unregister_event_handler(tep, -1, "ftrace", "function",
- function_handler, NULL);
-
- for (i = 0; i <= cpus; i++) {
- for (x = 0; x < fstack[i].size && fstack[i].stack[x]; x++)
- free(fstack[i].stack[x]);
- free(fstack[i].stack);
- }
-
- tep_plugin_remove_options(plugin_options);
-
- free(fstack);
- fstack = NULL;
- cpus = -1;
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2017 National Instruments Corp.
- *
- * Author: Julia Cartwright <julia@ni.com>
- *
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <linux/futex.h>
-
-#include "event-parse.h"
-
-#define ARRAY_SIZE(_a) (sizeof(_a) / sizeof((_a)[0]))
-
-struct futex_args {
- unsigned long long uaddr;
- unsigned long long op;
- unsigned long long val;
- unsigned long long utime; /* or val2 */
- unsigned long long uaddr2;
- unsigned long long val3;
-};
-
-struct futex_op {
- const char *name;
- const char *fmt_val;
- const char *fmt_utime;
- const char *fmt_uaddr2;
- const char *fmt_val3;
-};
-
-static const struct futex_op futex_op_tbl[] = {
- { "FUTEX_WAIT", " val=0x%08llx", " utime=0x%08llx", NULL, NULL },
- { "FUTEX_WAKE", " val=%llu", NULL, NULL, NULL },
- { "FUTEX_FD", " val=%llu", NULL, NULL, NULL },
- { "FUTEX_REQUEUE", " val=%llu", " val2=%llu", " uaddr2=0x%08llx", NULL },
- { "FUTEX_CMP_REQUEUE", " val=%llu", " val2=%llu", " uaddr2=0x%08llx", " val3=0x%08llx" },
- { "FUTEX_WAKE_OP", " val=%llu", " val2=%llu", " uaddr2=0x%08llx", " val3=0x%08llx" },
- { "FUTEX_LOCK_PI", NULL, " utime=0x%08llx", NULL, NULL },
- { "FUTEX_UNLOCK_PI", NULL, NULL, NULL, NULL },
- { "FUTEX_TRYLOCK_PI", NULL, NULL, NULL, NULL },
- { "FUTEX_WAIT_BITSET", " val=0x%08llx", " utime=0x%08llx", NULL, " val3=0x%08llx" },
- { "FUTEX_WAKE_BITSET", " val=%llu", NULL, NULL, " val3=0x%08llx" },
- { "FUTEX_WAIT_REQUEUE_PI", " val=0x%08llx", " utime=0x%08llx", " uaddr2=0x%08llx", " val3=0x%08llx" },
- { "FUTEX_CMP_REQUEUE_PI", " val=%llu", " val2=%llu", " uaddr2=0x%08llx", " val3=0x%08llx" },
-};
-
-
-static void futex_print(struct trace_seq *s, const struct futex_args *args,
- const struct futex_op *fop)
-{
- trace_seq_printf(s, " uaddr=0x%08llx", args->uaddr);
-
- if (fop->fmt_val)
- trace_seq_printf(s, fop->fmt_val, args->val);
-
- if (fop->fmt_utime)
- trace_seq_printf(s,fop->fmt_utime, args->utime);
-
- if (fop->fmt_uaddr2)
- trace_seq_printf(s, fop->fmt_uaddr2, args->uaddr2);
-
- if (fop->fmt_val3)
- trace_seq_printf(s, fop->fmt_val3, args->val3);
-}
-
-static int futex_handler(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- const struct futex_op *fop;
- struct futex_args args;
- unsigned long long cmd;
-
- if (tep_get_field_val(s, event, "uaddr", record, &args.uaddr, 1))
- return 1;
-
- if (tep_get_field_val(s, event, "op", record, &args.op, 1))
- return 1;
-
- if (tep_get_field_val(s, event, "val", record, &args.val, 1))
- return 1;
-
- if (tep_get_field_val(s, event, "utime", record, &args.utime, 1))
- return 1;
-
- if (tep_get_field_val(s, event, "uaddr2", record, &args.uaddr2, 1))
- return 1;
-
- if (tep_get_field_val(s, event, "val3", record, &args.val3, 1))
- return 1;
-
- cmd = args.op & FUTEX_CMD_MASK;
- if (cmd >= ARRAY_SIZE(futex_op_tbl))
- return 1;
-
- fop = &futex_op_tbl[cmd];
-
- trace_seq_printf(s, "op=%s", fop->name);
-
- if (args.op & FUTEX_PRIVATE_FLAG)
- trace_seq_puts(s, "|FUTEX_PRIVATE_FLAG");
-
- if (args.op & FUTEX_CLOCK_REALTIME)
- trace_seq_puts(s, "|FUTEX_CLOCK_REALTIME");
-
- futex_print(s, &args, fop);
- return 0;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_event_handler(tep, -1, "syscalls", "sys_enter_futex",
- futex_handler, NULL);
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_event_handler(tep, -1, "syscalls", "sys_enter_futex",
- futex_handler, NULL);
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- * Copyright (C) 2009 Johannes Berg <johannes@sipsolutions.net>
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include "event-parse.h"
-#include "trace-seq.h"
-
-static int timer_expire_handler(struct trace_seq *s,
- struct tep_record *record,
- struct tep_event *event, void *context)
-{
- trace_seq_printf(s, "hrtimer=");
-
- if (tep_print_num_field(s, "0x%llx", event, "timer",
- record, 0) == -1)
- tep_print_num_field(s, "0x%llx", event, "hrtimer",
- record, 1);
-
- trace_seq_printf(s, " now=");
-
- tep_print_num_field(s, "%llu", event, "now", record, 1);
-
- tep_print_func_field(s, " function=%s", event, "function",
- record, 0);
- return 0;
-}
-
-static int timer_start_handler(struct trace_seq *s,
- struct tep_record *record,
- struct tep_event *event, void *context)
-{
- trace_seq_printf(s, "hrtimer=");
-
- if (tep_print_num_field(s, "0x%llx", event, "timer",
- record, 0) == -1)
- tep_print_num_field(s, "0x%llx", event, "hrtimer",
- record, 1);
-
- tep_print_func_field(s, " function=%s", event, "function",
- record, 0);
-
- trace_seq_printf(s, " expires=");
- tep_print_num_field(s, "%llu", event, "expires", record, 1);
-
- trace_seq_printf(s, " softexpires=");
- tep_print_num_field(s, "%llu", event, "softexpires", record, 1);
- return 0;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_event_handler(tep, -1,
- "timer", "hrtimer_expire_entry",
- timer_expire_handler, NULL);
-
- tep_register_event_handler(tep, -1, "timer", "hrtimer_start",
- timer_start_handler, NULL);
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_event_handler(tep, -1,
- "timer", "hrtimer_expire_entry",
- timer_expire_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "timer", "hrtimer_start",
- timer_start_handler, NULL);
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include "event-parse.h"
-#include "trace-seq.h"
-
-#define MINORBITS 20
-#define MINORMASK ((1U << MINORBITS) - 1)
-
-#define MAJOR(dev) ((unsigned int) ((dev) >> MINORBITS))
-#define MINOR(dev) ((unsigned int) ((dev) & MINORMASK))
-
-static unsigned long long
-process_jbd2_dev_to_name(struct trace_seq *s, unsigned long long *args)
-{
- unsigned int dev = args[0];
-
- trace_seq_printf(s, "%d:%d", MAJOR(dev), MINOR(dev));
- return 0;
-}
-
-static unsigned long long
-process_jiffies_to_msecs(struct trace_seq *s, unsigned long long *args)
-{
- unsigned long long jiffies = args[0];
-
- trace_seq_printf(s, "%lld", jiffies);
- return jiffies;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_print_function(tep,
- process_jbd2_dev_to_name,
- TEP_FUNC_ARG_STRING,
- "jbd2_dev_to_name",
- TEP_FUNC_ARG_INT,
- TEP_FUNC_ARG_VOID);
-
- tep_register_print_function(tep,
- process_jiffies_to_msecs,
- TEP_FUNC_ARG_LONG,
- "jiffies_to_msecs",
- TEP_FUNC_ARG_LONG,
- TEP_FUNC_ARG_VOID);
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_print_function(tep, process_jbd2_dev_to_name,
- "jbd2_dev_to_name");
-
- tep_unregister_print_function(tep, process_jiffies_to_msecs,
- "jiffies_to_msecs");
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include "event-parse.h"
-#include "trace-seq.h"
-
-static int call_site_handler(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- struct tep_format_field *field;
- unsigned long long val, addr;
- void *data = record->data;
- const char *func;
-
- field = tep_find_field(event, "call_site");
- if (!field)
- return 1;
-
- if (tep_read_number_field(field, data, &val))
- return 1;
-
- func = tep_find_function(event->tep, val);
- if (!func)
- return 1;
-
- addr = tep_find_function_address(event->tep, val);
-
- trace_seq_printf(s, "(%s+0x%x) ", func, (int)(val - addr));
- return 1;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_event_handler(tep, -1, "kmem", "kfree",
- call_site_handler, NULL);
-
- tep_register_event_handler(tep, -1, "kmem", "kmalloc",
- call_site_handler, NULL);
-
- tep_register_event_handler(tep, -1, "kmem", "kmalloc_node",
- call_site_handler, NULL);
-
- tep_register_event_handler(tep, -1, "kmem", "kmem_cache_alloc",
- call_site_handler, NULL);
-
- tep_register_event_handler(tep, -1, "kmem",
- "kmem_cache_alloc_node",
- call_site_handler, NULL);
-
- tep_register_event_handler(tep, -1, "kmem", "kmem_cache_free",
- call_site_handler, NULL);
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_event_handler(tep, -1, "kmem", "kfree",
- call_site_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "kmem", "kmalloc",
- call_site_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "kmem", "kmalloc_node",
- call_site_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "kmem", "kmem_cache_alloc",
- call_site_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "kmem",
- "kmem_cache_alloc_node",
- call_site_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "kmem", "kmem_cache_free",
- call_site_handler, NULL);
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <stdint.h>
-
-#include "event-parse.h"
-#include "trace-seq.h"
-
-#ifdef HAVE_UDIS86
-
-#include <udis86.h>
-
-static ud_t ud;
-
-static void init_disassembler(void)
-{
- ud_init(&ud);
- ud_set_syntax(&ud, UD_SYN_ATT);
-}
-
-static const char *disassemble(unsigned char *insn, int len, uint64_t rip,
- int cr0_pe, int eflags_vm,
- int cs_d, int cs_l)
-{
- int mode;
-
- if (!cr0_pe)
- mode = 16;
- else if (eflags_vm)
- mode = 16;
- else if (cs_l)
- mode = 64;
- else if (cs_d)
- mode = 32;
- else
- mode = 16;
-
- ud_set_pc(&ud, rip);
- ud_set_mode(&ud, mode);
- ud_set_input_buffer(&ud, insn, len);
- ud_disassemble(&ud);
- return ud_insn_asm(&ud);
-}
-
-#else
-
-static void init_disassembler(void)
-{
-}
-
-static const char *disassemble(unsigned char *insn, int len, uint64_t rip,
- int cr0_pe, int eflags_vm,
- int cs_d, int cs_l)
-{
- static char out[15*3+1];
- int i;
-
- for (i = 0; i < len; ++i)
- sprintf(out + i * 3, "%02x ", insn[i]);
- out[len*3-1] = '\0';
- return out;
-}
-
-#endif
-
-
-#define VMX_EXIT_REASONS \
- _ER(EXCEPTION_NMI, 0) \
- _ER(EXTERNAL_INTERRUPT, 1) \
- _ER(TRIPLE_FAULT, 2) \
- _ER(PENDING_INTERRUPT, 7) \
- _ER(NMI_WINDOW, 8) \
- _ER(TASK_SWITCH, 9) \
- _ER(CPUID, 10) \
- _ER(HLT, 12) \
- _ER(INVD, 13) \
- _ER(INVLPG, 14) \
- _ER(RDPMC, 15) \
- _ER(RDTSC, 16) \
- _ER(VMCALL, 18) \
- _ER(VMCLEAR, 19) \
- _ER(VMLAUNCH, 20) \
- _ER(VMPTRLD, 21) \
- _ER(VMPTRST, 22) \
- _ER(VMREAD, 23) \
- _ER(VMRESUME, 24) \
- _ER(VMWRITE, 25) \
- _ER(VMOFF, 26) \
- _ER(VMON, 27) \
- _ER(CR_ACCESS, 28) \
- _ER(DR_ACCESS, 29) \
- _ER(IO_INSTRUCTION, 30) \
- _ER(MSR_READ, 31) \
- _ER(MSR_WRITE, 32) \
- _ER(MWAIT_INSTRUCTION, 36) \
- _ER(MONITOR_INSTRUCTION, 39) \
- _ER(PAUSE_INSTRUCTION, 40) \
- _ER(MCE_DURING_VMENTRY, 41) \
- _ER(TPR_BELOW_THRESHOLD, 43) \
- _ER(APIC_ACCESS, 44) \
- _ER(EOI_INDUCED, 45) \
- _ER(EPT_VIOLATION, 48) \
- _ER(EPT_MISCONFIG, 49) \
- _ER(INVEPT, 50) \
- _ER(PREEMPTION_TIMER, 52) \
- _ER(WBINVD, 54) \
- _ER(XSETBV, 55) \
- _ER(APIC_WRITE, 56) \
- _ER(INVPCID, 58) \
- _ER(PML_FULL, 62) \
- _ER(XSAVES, 63) \
- _ER(XRSTORS, 64)
-
-#define SVM_EXIT_REASONS \
- _ER(EXIT_READ_CR0, 0x000) \
- _ER(EXIT_READ_CR3, 0x003) \
- _ER(EXIT_READ_CR4, 0x004) \
- _ER(EXIT_READ_CR8, 0x008) \
- _ER(EXIT_WRITE_CR0, 0x010) \
- _ER(EXIT_WRITE_CR3, 0x013) \
- _ER(EXIT_WRITE_CR4, 0x014) \
- _ER(EXIT_WRITE_CR8, 0x018) \
- _ER(EXIT_READ_DR0, 0x020) \
- _ER(EXIT_READ_DR1, 0x021) \
- _ER(EXIT_READ_DR2, 0x022) \
- _ER(EXIT_READ_DR3, 0x023) \
- _ER(EXIT_READ_DR4, 0x024) \
- _ER(EXIT_READ_DR5, 0x025) \
- _ER(EXIT_READ_DR6, 0x026) \
- _ER(EXIT_READ_DR7, 0x027) \
- _ER(EXIT_WRITE_DR0, 0x030) \
- _ER(EXIT_WRITE_DR1, 0x031) \
- _ER(EXIT_WRITE_DR2, 0x032) \
- _ER(EXIT_WRITE_DR3, 0x033) \
- _ER(EXIT_WRITE_DR4, 0x034) \
- _ER(EXIT_WRITE_DR5, 0x035) \
- _ER(EXIT_WRITE_DR6, 0x036) \
- _ER(EXIT_WRITE_DR7, 0x037) \
- _ER(EXIT_EXCP_DE, 0x040) \
- _ER(EXIT_EXCP_DB, 0x041) \
- _ER(EXIT_EXCP_BP, 0x043) \
- _ER(EXIT_EXCP_OF, 0x044) \
- _ER(EXIT_EXCP_BR, 0x045) \
- _ER(EXIT_EXCP_UD, 0x046) \
- _ER(EXIT_EXCP_NM, 0x047) \
- _ER(EXIT_EXCP_DF, 0x048) \
- _ER(EXIT_EXCP_TS, 0x04a) \
- _ER(EXIT_EXCP_NP, 0x04b) \
- _ER(EXIT_EXCP_SS, 0x04c) \
- _ER(EXIT_EXCP_GP, 0x04d) \
- _ER(EXIT_EXCP_PF, 0x04e) \
- _ER(EXIT_EXCP_MF, 0x050) \
- _ER(EXIT_EXCP_AC, 0x051) \
- _ER(EXIT_EXCP_MC, 0x052) \
- _ER(EXIT_EXCP_XF, 0x053) \
- _ER(EXIT_INTR, 0x060) \
- _ER(EXIT_NMI, 0x061) \
- _ER(EXIT_SMI, 0x062) \
- _ER(EXIT_INIT, 0x063) \
- _ER(EXIT_VINTR, 0x064) \
- _ER(EXIT_CR0_SEL_WRITE, 0x065) \
- _ER(EXIT_IDTR_READ, 0x066) \
- _ER(EXIT_GDTR_READ, 0x067) \
- _ER(EXIT_LDTR_READ, 0x068) \
- _ER(EXIT_TR_READ, 0x069) \
- _ER(EXIT_IDTR_WRITE, 0x06a) \
- _ER(EXIT_GDTR_WRITE, 0x06b) \
- _ER(EXIT_LDTR_WRITE, 0x06c) \
- _ER(EXIT_TR_WRITE, 0x06d) \
- _ER(EXIT_RDTSC, 0x06e) \
- _ER(EXIT_RDPMC, 0x06f) \
- _ER(EXIT_PUSHF, 0x070) \
- _ER(EXIT_POPF, 0x071) \
- _ER(EXIT_CPUID, 0x072) \
- _ER(EXIT_RSM, 0x073) \
- _ER(EXIT_IRET, 0x074) \
- _ER(EXIT_SWINT, 0x075) \
- _ER(EXIT_INVD, 0x076) \
- _ER(EXIT_PAUSE, 0x077) \
- _ER(EXIT_HLT, 0x078) \
- _ER(EXIT_INVLPG, 0x079) \
- _ER(EXIT_INVLPGA, 0x07a) \
- _ER(EXIT_IOIO, 0x07b) \
- _ER(EXIT_MSR, 0x07c) \
- _ER(EXIT_TASK_SWITCH, 0x07d) \
- _ER(EXIT_FERR_FREEZE, 0x07e) \
- _ER(EXIT_SHUTDOWN, 0x07f) \
- _ER(EXIT_VMRUN, 0x080) \
- _ER(EXIT_VMMCALL, 0x081) \
- _ER(EXIT_VMLOAD, 0x082) \
- _ER(EXIT_VMSAVE, 0x083) \
- _ER(EXIT_STGI, 0x084) \
- _ER(EXIT_CLGI, 0x085) \
- _ER(EXIT_SKINIT, 0x086) \
- _ER(EXIT_RDTSCP, 0x087) \
- _ER(EXIT_ICEBP, 0x088) \
- _ER(EXIT_WBINVD, 0x089) \
- _ER(EXIT_MONITOR, 0x08a) \
- _ER(EXIT_MWAIT, 0x08b) \
- _ER(EXIT_MWAIT_COND, 0x08c) \
- _ER(EXIT_XSETBV, 0x08d) \
- _ER(EXIT_NPF, 0x400) \
- _ER(EXIT_AVIC_INCOMPLETE_IPI, 0x401) \
- _ER(EXIT_AVIC_UNACCELERATED_ACCESS, 0x402) \
- _ER(EXIT_ERR, -1)
-
-#define _ER(reason, val) { #reason, val },
-struct str_values {
- const char *str;
- int val;
-};
-
-static struct str_values vmx_exit_reasons[] = {
- VMX_EXIT_REASONS
- { NULL, -1}
-};
-
-static struct str_values svm_exit_reasons[] = {
- SVM_EXIT_REASONS
- { NULL, -1}
-};
-
-static struct isa_exit_reasons {
- unsigned isa;
- struct str_values *strings;
-} isa_exit_reasons[] = {
- { .isa = 1, .strings = vmx_exit_reasons },
- { .isa = 2, .strings = svm_exit_reasons },
- { }
-};
-
-static const char *find_exit_reason(unsigned isa, int val)
-{
- struct str_values *strings = NULL;
- int i;
-
- for (i = 0; isa_exit_reasons[i].strings; ++i)
- if (isa_exit_reasons[i].isa == isa) {
- strings = isa_exit_reasons[i].strings;
- break;
- }
- if (!strings)
- return "UNKNOWN-ISA";
- for (i = 0; strings[i].str; i++)
- if (strings[i].val == val)
- break;
-
- return strings[i].str;
-}
-
-static int print_exit_reason(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, const char *field)
-{
- unsigned long long isa;
- unsigned long long val;
- const char *reason;
-
- if (tep_get_field_val(s, event, field, record, &val, 1) < 0)
- return -1;
-
- if (tep_get_field_val(s, event, "isa", record, &isa, 0) < 0)
- isa = 1;
-
- reason = find_exit_reason(isa, val);
- if (reason)
- trace_seq_printf(s, "reason %s", reason);
- else
- trace_seq_printf(s, "reason UNKNOWN (%llu)", val);
- return 0;
-}
-
-static int kvm_exit_handler(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- unsigned long long info1 = 0, info2 = 0;
-
- if (print_exit_reason(s, record, event, "exit_reason") < 0)
- return -1;
-
- tep_print_num_field(s, " rip 0x%lx", event, "guest_rip", record, 1);
-
- if (tep_get_field_val(s, event, "info1", record, &info1, 0) >= 0
- && tep_get_field_val(s, event, "info2", record, &info2, 0) >= 0)
- trace_seq_printf(s, " info %llx %llx", info1, info2);
-
- return 0;
-}
-
-#define KVM_EMUL_INSN_F_CR0_PE (1 << 0)
-#define KVM_EMUL_INSN_F_EFL_VM (1 << 1)
-#define KVM_EMUL_INSN_F_CS_D (1 << 2)
-#define KVM_EMUL_INSN_F_CS_L (1 << 3)
-
-static int kvm_emulate_insn_handler(struct trace_seq *s,
- struct tep_record *record,
- struct tep_event *event, void *context)
-{
- unsigned long long rip, csbase, len, flags, failed;
- int llen;
- uint8_t *insn;
- const char *disasm;
-
- if (tep_get_field_val(s, event, "rip", record, &rip, 1) < 0)
- return -1;
-
- if (tep_get_field_val(s, event, "csbase", record, &csbase, 1) < 0)
- return -1;
-
- if (tep_get_field_val(s, event, "len", record, &len, 1) < 0)
- return -1;
-
- if (tep_get_field_val(s, event, "flags", record, &flags, 1) < 0)
- return -1;
-
- if (tep_get_field_val(s, event, "failed", record, &failed, 1) < 0)
- return -1;
-
- insn = tep_get_field_raw(s, event, "insn", record, &llen, 1);
- if (!insn)
- return -1;
-
- disasm = disassemble(insn, len, rip,
- flags & KVM_EMUL_INSN_F_CR0_PE,
- flags & KVM_EMUL_INSN_F_EFL_VM,
- flags & KVM_EMUL_INSN_F_CS_D,
- flags & KVM_EMUL_INSN_F_CS_L);
-
- trace_seq_printf(s, "%llx:%llx: %s%s", csbase, rip, disasm,
- failed ? " FAIL" : "");
- return 0;
-}
-
-
-static int kvm_nested_vmexit_inject_handler(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- if (print_exit_reason(s, record, event, "exit_code") < 0)
- return -1;
-
- tep_print_num_field(s, " info1 %llx", event, "exit_info1", record, 1);
- tep_print_num_field(s, " info2 %llx", event, "exit_info2", record, 1);
- tep_print_num_field(s, " int_info %llx", event, "exit_int_info", record, 1);
- tep_print_num_field(s, " int_info_err %llx", event, "exit_int_info_err", record, 1);
-
- return 0;
-}
-
-static int kvm_nested_vmexit_handler(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- tep_print_num_field(s, "rip %llx ", event, "rip", record, 1);
-
- return kvm_nested_vmexit_inject_handler(s, record, event, context);
-}
-
-union kvm_mmu_page_role {
- unsigned word;
- struct {
- unsigned level:4;
- unsigned cr4_pae:1;
- unsigned quadrant:2;
- unsigned direct:1;
- unsigned access:3;
- unsigned invalid:1;
- unsigned efer_nx:1;
- unsigned cr0_wp:1;
- unsigned smep_and_not_wp:1;
- unsigned smap_and_not_wp:1;
- unsigned pad_for_nice_hex_output:8;
- unsigned smm:8;
- };
-};
-
-static int kvm_mmu_print_role(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- unsigned long long val;
- static const char *access_str[] = {
- "---", "--x", "w--", "w-x", "-u-", "-ux", "wu-", "wux"
- };
- union kvm_mmu_page_role role;
-
- if (tep_get_field_val(s, event, "role", record, &val, 1) < 0)
- return -1;
-
- role.word = (int)val;
-
- /*
- * We can only use the structure if file is of the same
- * endianness.
- */
- if (tep_is_file_bigendian(event->tep) ==
- tep_is_local_bigendian(event->tep)) {
-
- trace_seq_printf(s, "%u q%u%s %s%s %spae %snxe %swp%s%s%s",
- role.level,
- role.quadrant,
- role.direct ? " direct" : "",
- access_str[role.access],
- role.invalid ? " invalid" : "",
- role.cr4_pae ? "" : "!",
- role.efer_nx ? "" : "!",
- role.cr0_wp ? "" : "!",
- role.smep_and_not_wp ? " smep" : "",
- role.smap_and_not_wp ? " smap" : "",
- role.smm ? " smm" : "");
- } else
- trace_seq_printf(s, "WORD: %08x", role.word);
-
- tep_print_num_field(s, " root %u ", event,
- "root_count", record, 1);
-
- if (tep_get_field_val(s, event, "unsync", record, &val, 1) < 0)
- return -1;
-
- trace_seq_printf(s, "%s%c", val ? "unsync" : "sync", 0);
- return 0;
-}
-
-static int kvm_mmu_get_page_handler(struct trace_seq *s,
- struct tep_record *record,
- struct tep_event *event, void *context)
-{
- unsigned long long val;
-
- if (tep_get_field_val(s, event, "created", record, &val, 1) < 0)
- return -1;
-
- trace_seq_printf(s, "%s ", val ? "new" : "existing");
-
- if (tep_get_field_val(s, event, "gfn", record, &val, 1) < 0)
- return -1;
-
- trace_seq_printf(s, "sp gfn %llx ", val);
- return kvm_mmu_print_role(s, record, event, context);
-}
-
-#define PT_WRITABLE_SHIFT 1
-#define PT_WRITABLE_MASK (1ULL << PT_WRITABLE_SHIFT)
-
-static unsigned long long
-process_is_writable_pte(struct trace_seq *s, unsigned long long *args)
-{
- unsigned long pte = args[0];
- return pte & PT_WRITABLE_MASK;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- init_disassembler();
-
- tep_register_event_handler(tep, -1, "kvm", "kvm_exit",
- kvm_exit_handler, NULL);
-
- tep_register_event_handler(tep, -1, "kvm", "kvm_emulate_insn",
- kvm_emulate_insn_handler, NULL);
-
- tep_register_event_handler(tep, -1, "kvm", "kvm_nested_vmexit",
- kvm_nested_vmexit_handler, NULL);
-
- tep_register_event_handler(tep, -1, "kvm", "kvm_nested_vmexit_inject",
- kvm_nested_vmexit_inject_handler, NULL);
-
- tep_register_event_handler(tep, -1, "kvmmmu", "kvm_mmu_get_page",
- kvm_mmu_get_page_handler, NULL);
-
- tep_register_event_handler(tep, -1, "kvmmmu", "kvm_mmu_sync_page",
- kvm_mmu_print_role, NULL);
-
- tep_register_event_handler(tep, -1,
- "kvmmmu", "kvm_mmu_unsync_page",
- kvm_mmu_print_role, NULL);
-
- tep_register_event_handler(tep, -1, "kvmmmu", "kvm_mmu_zap_page",
- kvm_mmu_print_role, NULL);
-
- tep_register_event_handler(tep, -1, "kvmmmu",
- "kvm_mmu_prepare_zap_page", kvm_mmu_print_role,
- NULL);
-
- tep_register_print_function(tep,
- process_is_writable_pte,
- TEP_FUNC_ARG_INT,
- "is_writable_pte",
- TEP_FUNC_ARG_LONG,
- TEP_FUNC_ARG_VOID);
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_event_handler(tep, -1, "kvm", "kvm_exit",
- kvm_exit_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "kvm", "kvm_emulate_insn",
- kvm_emulate_insn_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "kvm", "kvm_nested_vmexit",
- kvm_nested_vmexit_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "kvm", "kvm_nested_vmexit_inject",
- kvm_nested_vmexit_inject_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "kvmmmu", "kvm_mmu_get_page",
- kvm_mmu_get_page_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "kvmmmu", "kvm_mmu_sync_page",
- kvm_mmu_print_role, NULL);
-
- tep_unregister_event_handler(tep, -1,
- "kvmmmu", "kvm_mmu_unsync_page",
- kvm_mmu_print_role, NULL);
-
- tep_unregister_event_handler(tep, -1, "kvmmmu", "kvm_mmu_zap_page",
- kvm_mmu_print_role, NULL);
-
- tep_unregister_event_handler(tep, -1, "kvmmmu",
- "kvm_mmu_prepare_zap_page", kvm_mmu_print_role,
- NULL);
-
- tep_unregister_print_function(tep, process_is_writable_pte,
- "is_writable_pte");
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009 Johannes Berg <johannes@sipsolutions.net>
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include "event-parse.h"
-#include "trace-seq.h"
-
-#define INDENT 65
-
-static void print_string(struct trace_seq *s, struct tep_event *event,
- const char *name, const void *data)
-{
- struct tep_format_field *f = tep_find_field(event, name);
- int offset;
- int length;
-
- if (!f) {
- trace_seq_printf(s, "NOTFOUND:%s", name);
- return;
- }
-
- offset = f->offset;
- length = f->size;
-
- if (!strncmp(f->type, "__data_loc", 10)) {
- unsigned long long v;
- if (tep_read_number_field(f, data, &v)) {
- trace_seq_printf(s, "invalid_data_loc");
- return;
- }
- offset = v & 0xffff;
- length = v >> 16;
- }
-
- trace_seq_printf(s, "%.*s", length, (char *)data + offset);
-}
-
-#define SF(fn) tep_print_num_field(s, fn ":%d", event, fn, record, 0)
-#define SFX(fn) tep_print_num_field(s, fn ":%#x", event, fn, record, 0)
-#define SP() trace_seq_putc(s, ' ')
-
-static int drv_bss_info_changed(struct trace_seq *s,
- struct tep_record *record,
- struct tep_event *event, void *context)
-{
- void *data = record->data;
-
- print_string(s, event, "wiphy_name", data);
- trace_seq_printf(s, " vif:");
- print_string(s, event, "vif_name", data);
- tep_print_num_field(s, "(%d)", event, "vif_type", record, 1);
-
- trace_seq_printf(s, "\n%*s", INDENT, "");
- SF("assoc"); SP();
- SF("aid"); SP();
- SF("cts"); SP();
- SF("shortpre"); SP();
- SF("shortslot"); SP();
- SF("dtimper"); SP();
- trace_seq_printf(s, "\n%*s", INDENT, "");
- SF("bcnint"); SP();
- SFX("assoc_cap"); SP();
- SFX("basic_rates"); SP();
- SF("enable_beacon");
- trace_seq_printf(s, "\n%*s", INDENT, "");
- SF("ht_operation_mode");
-
- return 0;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_event_handler(tep, -1, "mac80211",
- "drv_bss_info_changed",
- drv_bss_info_changed, NULL);
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_event_handler(tep, -1, "mac80211",
- "drv_bss_info_changed",
- drv_bss_info_changed, NULL);
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include "event-parse.h"
-#include "trace-seq.h"
-
-static void write_state(struct trace_seq *s, int val)
-{
- const char states[] = "SDTtZXxW";
- int found = 0;
- int i;
-
- for (i = 0; i < (sizeof(states) - 1); i++) {
- if (!(val & (1 << i)))
- continue;
-
- if (found)
- trace_seq_putc(s, '|');
-
- found = 1;
- trace_seq_putc(s, states[i]);
- }
-
- if (!found)
- trace_seq_putc(s, 'R');
-}
-
-static void write_and_save_comm(struct tep_format_field *field,
- struct tep_record *record,
- struct trace_seq *s, int pid)
-{
- const char *comm;
- int len;
-
- comm = (char *)(record->data + field->offset);
- len = s->len;
- trace_seq_printf(s, "%.*s",
- field->size, comm);
-
- /* make sure the comm has a \0 at the end. */
- trace_seq_terminate(s);
- comm = &s->buffer[len];
-
- /* Help out the comm to ids. This will handle dups */
- tep_register_comm(field->event->tep, comm, pid);
-}
-
-static int sched_wakeup_handler(struct trace_seq *s,
- struct tep_record *record,
- struct tep_event *event, void *context)
-{
- struct tep_format_field *field;
- unsigned long long val;
-
- if (tep_get_field_val(s, event, "pid", record, &val, 1))
- return trace_seq_putc(s, '!');
-
- field = tep_find_any_field(event, "comm");
- if (field) {
- write_and_save_comm(field, record, s, val);
- trace_seq_putc(s, ':');
- }
- trace_seq_printf(s, "%lld", val);
-
- if (tep_get_field_val(s, event, "prio", record, &val, 0) == 0)
- trace_seq_printf(s, " [%lld]", val);
-
- if (tep_get_field_val(s, event, "success", record, &val, 1) == 0)
- trace_seq_printf(s, " success=%lld", val);
-
- if (tep_get_field_val(s, event, "target_cpu", record, &val, 0) == 0)
- trace_seq_printf(s, " CPU:%03llu", val);
-
- return 0;
-}
-
-static int sched_switch_handler(struct trace_seq *s,
- struct tep_record *record,
- struct tep_event *event, void *context)
-{
- struct tep_format_field *field;
- unsigned long long val;
-
- if (tep_get_field_val(s, event, "prev_pid", record, &val, 1))
- return trace_seq_putc(s, '!');
-
- field = tep_find_any_field(event, "prev_comm");
- if (field) {
- write_and_save_comm(field, record, s, val);
- trace_seq_putc(s, ':');
- }
- trace_seq_printf(s, "%lld ", val);
-
- if (tep_get_field_val(s, event, "prev_prio", record, &val, 0) == 0)
- trace_seq_printf(s, "[%d] ", (int) val);
-
- if (tep_get_field_val(s, event, "prev_state", record, &val, 0) == 0)
- write_state(s, val);
-
- trace_seq_puts(s, " ==> ");
-
- if (tep_get_field_val(s, event, "next_pid", record, &val, 1))
- return trace_seq_putc(s, '!');
-
- field = tep_find_any_field(event, "next_comm");
- if (field) {
- write_and_save_comm(field, record, s, val);
- trace_seq_putc(s, ':');
- }
- trace_seq_printf(s, "%lld", val);
-
- if (tep_get_field_val(s, event, "next_prio", record, &val, 0) == 0)
- trace_seq_printf(s, " [%d]", (int) val);
-
- return 0;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_event_handler(tep, -1, "sched", "sched_switch",
- sched_switch_handler, NULL);
-
- tep_register_event_handler(tep, -1, "sched", "sched_wakeup",
- sched_wakeup_handler, NULL);
-
- tep_register_event_handler(tep, -1, "sched", "sched_wakeup_new",
- sched_wakeup_handler, NULL);
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_event_handler(tep, -1, "sched", "sched_switch",
- sched_switch_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "sched", "sched_wakeup",
- sched_wakeup_handler, NULL);
-
- tep_unregister_event_handler(tep, -1, "sched", "sched_wakeup_new",
- sched_wakeup_handler, NULL);
-}
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-#include <stdio.h>
-#include <string.h>
-#include <inttypes.h>
-#include "event-parse.h"
-#include "trace-seq.h"
-
-typedef unsigned long sector_t;
-typedef uint64_t u64;
-typedef unsigned int u32;
-
-/*
- * SCSI opcodes
- */
-#define TEST_UNIT_READY 0x00
-#define REZERO_UNIT 0x01
-#define REQUEST_SENSE 0x03
-#define FORMAT_UNIT 0x04
-#define READ_BLOCK_LIMITS 0x05
-#define REASSIGN_BLOCKS 0x07
-#define INITIALIZE_ELEMENT_STATUS 0x07
-#define READ_6 0x08
-#define WRITE_6 0x0a
-#define SEEK_6 0x0b
-#define READ_REVERSE 0x0f
-#define WRITE_FILEMARKS 0x10
-#define SPACE 0x11
-#define INQUIRY 0x12
-#define RECOVER_BUFFERED_DATA 0x14
-#define MODE_SELECT 0x15
-#define RESERVE 0x16
-#define RELEASE 0x17
-#define COPY 0x18
-#define ERASE 0x19
-#define MODE_SENSE 0x1a
-#define START_STOP 0x1b
-#define RECEIVE_DIAGNOSTIC 0x1c
-#define SEND_DIAGNOSTIC 0x1d
-#define ALLOW_MEDIUM_REMOVAL 0x1e
-
-#define READ_FORMAT_CAPACITIES 0x23
-#define SET_WINDOW 0x24
-#define READ_CAPACITY 0x25
-#define READ_10 0x28
-#define WRITE_10 0x2a
-#define SEEK_10 0x2b
-#define POSITION_TO_ELEMENT 0x2b
-#define WRITE_VERIFY 0x2e
-#define VERIFY 0x2f
-#define SEARCH_HIGH 0x30
-#define SEARCH_EQUAL 0x31
-#define SEARCH_LOW 0x32
-#define SET_LIMITS 0x33
-#define PRE_FETCH 0x34
-#define READ_POSITION 0x34
-#define SYNCHRONIZE_CACHE 0x35
-#define LOCK_UNLOCK_CACHE 0x36
-#define READ_DEFECT_DATA 0x37
-#define MEDIUM_SCAN 0x38
-#define COMPARE 0x39
-#define COPY_VERIFY 0x3a
-#define WRITE_BUFFER 0x3b
-#define READ_BUFFER 0x3c
-#define UPDATE_BLOCK 0x3d
-#define READ_LONG 0x3e
-#define WRITE_LONG 0x3f
-#define CHANGE_DEFINITION 0x40
-#define WRITE_SAME 0x41
-#define UNMAP 0x42
-#define READ_TOC 0x43
-#define READ_HEADER 0x44
-#define GET_EVENT_STATUS_NOTIFICATION 0x4a
-#define LOG_SELECT 0x4c
-#define LOG_SENSE 0x4d
-#define XDWRITEREAD_10 0x53
-#define MODE_SELECT_10 0x55
-#define RESERVE_10 0x56
-#define RELEASE_10 0x57
-#define MODE_SENSE_10 0x5a
-#define PERSISTENT_RESERVE_IN 0x5e
-#define PERSISTENT_RESERVE_OUT 0x5f
-#define VARIABLE_LENGTH_CMD 0x7f
-#define REPORT_LUNS 0xa0
-#define SECURITY_PROTOCOL_IN 0xa2
-#define MAINTENANCE_IN 0xa3
-#define MAINTENANCE_OUT 0xa4
-#define MOVE_MEDIUM 0xa5
-#define EXCHANGE_MEDIUM 0xa6
-#define READ_12 0xa8
-#define SERVICE_ACTION_OUT_12 0xa9
-#define WRITE_12 0xaa
-#define SERVICE_ACTION_IN_12 0xab
-#define WRITE_VERIFY_12 0xae
-#define VERIFY_12 0xaf
-#define SEARCH_HIGH_12 0xb0
-#define SEARCH_EQUAL_12 0xb1
-#define SEARCH_LOW_12 0xb2
-#define SECURITY_PROTOCOL_OUT 0xb5
-#define READ_ELEMENT_STATUS 0xb8
-#define SEND_VOLUME_TAG 0xb6
-#define WRITE_LONG_2 0xea
-#define EXTENDED_COPY 0x83
-#define RECEIVE_COPY_RESULTS 0x84
-#define ACCESS_CONTROL_IN 0x86
-#define ACCESS_CONTROL_OUT 0x87
-#define READ_16 0x88
-#define WRITE_16 0x8a
-#define READ_ATTRIBUTE 0x8c
-#define WRITE_ATTRIBUTE 0x8d
-#define VERIFY_16 0x8f
-#define SYNCHRONIZE_CACHE_16 0x91
-#define WRITE_SAME_16 0x93
-#define SERVICE_ACTION_BIDIRECTIONAL 0x9d
-#define SERVICE_ACTION_IN_16 0x9e
-#define SERVICE_ACTION_OUT_16 0x9f
-/* values for service action in */
-#define SAI_READ_CAPACITY_16 0x10
-#define SAI_GET_LBA_STATUS 0x12
-/* values for VARIABLE_LENGTH_CMD service action codes
- * see spc4r17 Section D.3.5, table D.7 and D.8 */
-#define VLC_SA_RECEIVE_CREDENTIAL 0x1800
-/* values for maintenance in */
-#define MI_REPORT_IDENTIFYING_INFORMATION 0x05
-#define MI_REPORT_TARGET_PGS 0x0a
-#define MI_REPORT_ALIASES 0x0b
-#define MI_REPORT_SUPPORTED_OPERATION_CODES 0x0c
-#define MI_REPORT_SUPPORTED_TASK_MANAGEMENT_FUNCTIONS 0x0d
-#define MI_REPORT_PRIORITY 0x0e
-#define MI_REPORT_TIMESTAMP 0x0f
-#define MI_MANAGEMENT_PROTOCOL_IN 0x10
-/* value for MI_REPORT_TARGET_PGS ext header */
-#define MI_EXT_HDR_PARAM_FMT 0x20
-/* values for maintenance out */
-#define MO_SET_IDENTIFYING_INFORMATION 0x06
-#define MO_SET_TARGET_PGS 0x0a
-#define MO_CHANGE_ALIASES 0x0b
-#define MO_SET_PRIORITY 0x0e
-#define MO_SET_TIMESTAMP 0x0f
-#define MO_MANAGEMENT_PROTOCOL_OUT 0x10
-/* values for variable length command */
-#define XDREAD_32 0x03
-#define XDWRITE_32 0x04
-#define XPWRITE_32 0x06
-#define XDWRITEREAD_32 0x07
-#define READ_32 0x09
-#define VERIFY_32 0x0a
-#define WRITE_32 0x0b
-#define WRITE_SAME_32 0x0d
-
-#define SERVICE_ACTION16(cdb) (cdb[1] & 0x1f)
-#define SERVICE_ACTION32(cdb) ((cdb[8] << 8) | cdb[9])
-
-static const char *
-scsi_trace_misc(struct trace_seq *, unsigned char *, int);
-
-static const char *
-scsi_trace_rw6(struct trace_seq *p, unsigned char *cdb, int len)
-{
- const char *ret = p->buffer + p->len;
- sector_t lba = 0, txlen = 0;
-
- lba |= ((cdb[1] & 0x1F) << 16);
- lba |= (cdb[2] << 8);
- lba |= cdb[3];
- txlen = cdb[4];
-
- trace_seq_printf(p, "lba=%llu txlen=%llu",
- (unsigned long long)lba, (unsigned long long)txlen);
- trace_seq_putc(p, 0);
- return ret;
-}
-
-static const char *
-scsi_trace_rw10(struct trace_seq *p, unsigned char *cdb, int len)
-{
- const char *ret = p->buffer + p->len;
- sector_t lba = 0, txlen = 0;
-
- lba |= (cdb[2] << 24);
- lba |= (cdb[3] << 16);
- lba |= (cdb[4] << 8);
- lba |= cdb[5];
- txlen |= (cdb[7] << 8);
- txlen |= cdb[8];
-
- trace_seq_printf(p, "lba=%llu txlen=%llu protect=%u",
- (unsigned long long)lba, (unsigned long long)txlen,
- cdb[1] >> 5);
-
- if (cdb[0] == WRITE_SAME)
- trace_seq_printf(p, " unmap=%u", cdb[1] >> 3 & 1);
-
- trace_seq_putc(p, 0);
- return ret;
-}
-
-static const char *
-scsi_trace_rw12(struct trace_seq *p, unsigned char *cdb, int len)
-{
- const char *ret = p->buffer + p->len;
- sector_t lba = 0, txlen = 0;
-
- lba |= (cdb[2] << 24);
- lba |= (cdb[3] << 16);
- lba |= (cdb[4] << 8);
- lba |= cdb[5];
- txlen |= (cdb[6] << 24);
- txlen |= (cdb[7] << 16);
- txlen |= (cdb[8] << 8);
- txlen |= cdb[9];
-
- trace_seq_printf(p, "lba=%llu txlen=%llu protect=%u",
- (unsigned long long)lba, (unsigned long long)txlen,
- cdb[1] >> 5);
- trace_seq_putc(p, 0);
- return ret;
-}
-
-static const char *
-scsi_trace_rw16(struct trace_seq *p, unsigned char *cdb, int len)
-{
- const char *ret = p->buffer + p->len;
- sector_t lba = 0, txlen = 0;
-
- lba |= ((u64)cdb[2] << 56);
- lba |= ((u64)cdb[3] << 48);
- lba |= ((u64)cdb[4] << 40);
- lba |= ((u64)cdb[5] << 32);
- lba |= (cdb[6] << 24);
- lba |= (cdb[7] << 16);
- lba |= (cdb[8] << 8);
- lba |= cdb[9];
- txlen |= (cdb[10] << 24);
- txlen |= (cdb[11] << 16);
- txlen |= (cdb[12] << 8);
- txlen |= cdb[13];
-
- trace_seq_printf(p, "lba=%llu txlen=%llu protect=%u",
- (unsigned long long)lba, (unsigned long long)txlen,
- cdb[1] >> 5);
-
- if (cdb[0] == WRITE_SAME_16)
- trace_seq_printf(p, " unmap=%u", cdb[1] >> 3 & 1);
-
- trace_seq_putc(p, 0);
- return ret;
-}
-
-static const char *
-scsi_trace_rw32(struct trace_seq *p, unsigned char *cdb, int len)
-{
- const char *ret = p->buffer + p->len, *cmd;
- sector_t lba = 0, txlen = 0;
- u32 ei_lbrt = 0;
-
- switch (SERVICE_ACTION32(cdb)) {
- case READ_32:
- cmd = "READ";
- break;
- case VERIFY_32:
- cmd = "VERIFY";
- break;
- case WRITE_32:
- cmd = "WRITE";
- break;
- case WRITE_SAME_32:
- cmd = "WRITE_SAME";
- break;
- default:
- trace_seq_printf(p, "UNKNOWN");
- goto out;
- }
-
- lba |= ((u64)cdb[12] << 56);
- lba |= ((u64)cdb[13] << 48);
- lba |= ((u64)cdb[14] << 40);
- lba |= ((u64)cdb[15] << 32);
- lba |= (cdb[16] << 24);
- lba |= (cdb[17] << 16);
- lba |= (cdb[18] << 8);
- lba |= cdb[19];
- ei_lbrt |= (cdb[20] << 24);
- ei_lbrt |= (cdb[21] << 16);
- ei_lbrt |= (cdb[22] << 8);
- ei_lbrt |= cdb[23];
- txlen |= (cdb[28] << 24);
- txlen |= (cdb[29] << 16);
- txlen |= (cdb[30] << 8);
- txlen |= cdb[31];
-
- trace_seq_printf(p, "%s_32 lba=%llu txlen=%llu protect=%u ei_lbrt=%u",
- cmd, (unsigned long long)lba,
- (unsigned long long)txlen, cdb[10] >> 5, ei_lbrt);
-
- if (SERVICE_ACTION32(cdb) == WRITE_SAME_32)
- trace_seq_printf(p, " unmap=%u", cdb[10] >> 3 & 1);
-
-out:
- trace_seq_putc(p, 0);
- return ret;
-}
-
-static const char *
-scsi_trace_unmap(struct trace_seq *p, unsigned char *cdb, int len)
-{
- const char *ret = p->buffer + p->len;
- unsigned int regions = cdb[7] << 8 | cdb[8];
-
- trace_seq_printf(p, "regions=%u", (regions - 8) / 16);
- trace_seq_putc(p, 0);
- return ret;
-}
-
-static const char *
-scsi_trace_service_action_in(struct trace_seq *p, unsigned char *cdb, int len)
-{
- const char *ret = p->buffer + p->len, *cmd;
- sector_t lba = 0;
- u32 alloc_len = 0;
-
- switch (SERVICE_ACTION16(cdb)) {
- case SAI_READ_CAPACITY_16:
- cmd = "READ_CAPACITY_16";
- break;
- case SAI_GET_LBA_STATUS:
- cmd = "GET_LBA_STATUS";
- break;
- default:
- trace_seq_printf(p, "UNKNOWN");
- goto out;
- }
-
- lba |= ((u64)cdb[2] << 56);
- lba |= ((u64)cdb[3] << 48);
- lba |= ((u64)cdb[4] << 40);
- lba |= ((u64)cdb[5] << 32);
- lba |= (cdb[6] << 24);
- lba |= (cdb[7] << 16);
- lba |= (cdb[8] << 8);
- lba |= cdb[9];
- alloc_len |= (cdb[10] << 24);
- alloc_len |= (cdb[11] << 16);
- alloc_len |= (cdb[12] << 8);
- alloc_len |= cdb[13];
-
- trace_seq_printf(p, "%s lba=%llu alloc_len=%u", cmd,
- (unsigned long long)lba, alloc_len);
-
-out:
- trace_seq_putc(p, 0);
- return ret;
-}
-
-static const char *
-scsi_trace_varlen(struct trace_seq *p, unsigned char *cdb, int len)
-{
- switch (SERVICE_ACTION32(cdb)) {
- case READ_32:
- case VERIFY_32:
- case WRITE_32:
- case WRITE_SAME_32:
- return scsi_trace_rw32(p, cdb, len);
- default:
- return scsi_trace_misc(p, cdb, len);
- }
-}
-
-static const char *
-scsi_trace_misc(struct trace_seq *p, unsigned char *cdb, int len)
-{
- const char *ret = p->buffer + p->len;
-
- trace_seq_printf(p, "-");
- trace_seq_putc(p, 0);
- return ret;
-}
-
-const char *
-scsi_trace_parse_cdb(struct trace_seq *p, unsigned char *cdb, int len)
-{
- switch (cdb[0]) {
- case READ_6:
- case WRITE_6:
- return scsi_trace_rw6(p, cdb, len);
- case READ_10:
- case VERIFY:
- case WRITE_10:
- case WRITE_SAME:
- return scsi_trace_rw10(p, cdb, len);
- case READ_12:
- case VERIFY_12:
- case WRITE_12:
- return scsi_trace_rw12(p, cdb, len);
- case READ_16:
- case VERIFY_16:
- case WRITE_16:
- case WRITE_SAME_16:
- return scsi_trace_rw16(p, cdb, len);
- case UNMAP:
- return scsi_trace_unmap(p, cdb, len);
- case SERVICE_ACTION_IN_16:
- return scsi_trace_service_action_in(p, cdb, len);
- case VARIABLE_LENGTH_CMD:
- return scsi_trace_varlen(p, cdb, len);
- default:
- return scsi_trace_misc(p, cdb, len);
- }
-}
-
-unsigned long long process_scsi_trace_parse_cdb(struct trace_seq *s,
- unsigned long long *args)
-{
- scsi_trace_parse_cdb(s, (unsigned char *) (unsigned long) args[1], args[2]);
- return 0;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_print_function(tep,
- process_scsi_trace_parse_cdb,
- TEP_FUNC_ARG_STRING,
- "scsi_trace_parse_cdb",
- TEP_FUNC_ARG_PTR,
- TEP_FUNC_ARG_PTR,
- TEP_FUNC_ARG_INT,
- TEP_FUNC_ARG_VOID);
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_print_function(tep, process_scsi_trace_parse_cdb,
- "scsi_trace_parse_cdb");
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2015 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- */
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include "event-parse.h"
-
-enum tlb_flush_reason {
- TLB_FLUSH_ON_TASK_SWITCH,
- TLB_REMOTE_SHOOTDOWN,
- TLB_LOCAL_SHOOTDOWN,
- TLB_LOCAL_MM_SHOOTDOWN,
- NR_TLB_FLUSH_REASONS,
-};
-
-static int tlb_flush_handler(struct trace_seq *s, struct tep_record *record,
- struct tep_event *event, void *context)
-{
- unsigned long long val;
-
- trace_seq_printf(s, "pages=");
-
- tep_print_num_field(s, "%ld", event, "pages", record, 1);
-
- if (tep_get_field_val(s, event, "reason", record, &val, 1) < 0)
- return -1;
-
- trace_seq_puts(s, " reason=");
-
- switch (val) {
- case TLB_FLUSH_ON_TASK_SWITCH:
- trace_seq_puts(s, "flush on task switch");
- break;
- case TLB_REMOTE_SHOOTDOWN:
- trace_seq_puts(s, "remote shootdown");
- break;
- case TLB_LOCAL_SHOOTDOWN:
- trace_seq_puts(s, "local shootdown");
- break;
- case TLB_LOCAL_MM_SHOOTDOWN:
- trace_seq_puts(s, "local mm shootdown");
- break;
- }
-
- trace_seq_printf(s, " (%lld)", val);
-
- return 0;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_event_handler(tep, -1, "tlb", "tlb_flush",
- tlb_flush_handler, NULL);
-
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_event_handler(tep, -1,
- "tlb", "tlb_flush",
- tlb_flush_handler, NULL);
-}
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include "event-parse.h"
-#include "trace-seq.h"
-
-#define __HYPERVISOR_set_trap_table 0
-#define __HYPERVISOR_mmu_update 1
-#define __HYPERVISOR_set_gdt 2
-#define __HYPERVISOR_stack_switch 3
-#define __HYPERVISOR_set_callbacks 4
-#define __HYPERVISOR_fpu_taskswitch 5
-#define __HYPERVISOR_sched_op_compat 6
-#define __HYPERVISOR_dom0_op 7
-#define __HYPERVISOR_set_debugreg 8
-#define __HYPERVISOR_get_debugreg 9
-#define __HYPERVISOR_update_descriptor 10
-#define __HYPERVISOR_memory_op 12
-#define __HYPERVISOR_multicall 13
-#define __HYPERVISOR_update_va_mapping 14
-#define __HYPERVISOR_set_timer_op 15
-#define __HYPERVISOR_event_channel_op_compat 16
-#define __HYPERVISOR_xen_version 17
-#define __HYPERVISOR_console_io 18
-#define __HYPERVISOR_physdev_op_compat 19
-#define __HYPERVISOR_grant_table_op 20
-#define __HYPERVISOR_vm_assist 21
-#define __HYPERVISOR_update_va_mapping_otherdomain 22
-#define __HYPERVISOR_iret 23 /* x86 only */
-#define __HYPERVISOR_vcpu_op 24
-#define __HYPERVISOR_set_segment_base 25 /* x86/64 only */
-#define __HYPERVISOR_mmuext_op 26
-#define __HYPERVISOR_acm_op 27
-#define __HYPERVISOR_nmi_op 28
-#define __HYPERVISOR_sched_op 29
-#define __HYPERVISOR_callback_op 30
-#define __HYPERVISOR_xenoprof_op 31
-#define __HYPERVISOR_event_channel_op 32
-#define __HYPERVISOR_physdev_op 33
-#define __HYPERVISOR_hvm_op 34
-#define __HYPERVISOR_tmem_op 38
-
-/* Architecture-specific hypercall definitions. */
-#define __HYPERVISOR_arch_0 48
-#define __HYPERVISOR_arch_1 49
-#define __HYPERVISOR_arch_2 50
-#define __HYPERVISOR_arch_3 51
-#define __HYPERVISOR_arch_4 52
-#define __HYPERVISOR_arch_5 53
-#define __HYPERVISOR_arch_6 54
-#define __HYPERVISOR_arch_7 55
-
-#define N(x) [__HYPERVISOR_##x] = "("#x")"
-static const char *xen_hypercall_names[] = {
- N(set_trap_table),
- N(mmu_update),
- N(set_gdt),
- N(stack_switch),
- N(set_callbacks),
- N(fpu_taskswitch),
- N(sched_op_compat),
- N(dom0_op),
- N(set_debugreg),
- N(get_debugreg),
- N(update_descriptor),
- N(memory_op),
- N(multicall),
- N(update_va_mapping),
- N(set_timer_op),
- N(event_channel_op_compat),
- N(xen_version),
- N(console_io),
- N(physdev_op_compat),
- N(grant_table_op),
- N(vm_assist),
- N(update_va_mapping_otherdomain),
- N(iret),
- N(vcpu_op),
- N(set_segment_base),
- N(mmuext_op),
- N(acm_op),
- N(nmi_op),
- N(sched_op),
- N(callback_op),
- N(xenoprof_op),
- N(event_channel_op),
- N(physdev_op),
- N(hvm_op),
-
-/* Architecture-specific hypercall definitions. */
- N(arch_0),
- N(arch_1),
- N(arch_2),
- N(arch_3),
- N(arch_4),
- N(arch_5),
- N(arch_6),
- N(arch_7),
-};
-#undef N
-
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
-
-static const char *xen_hypercall_name(unsigned op)
-{
- if (op < ARRAY_SIZE(xen_hypercall_names) &&
- xen_hypercall_names[op] != NULL)
- return xen_hypercall_names[op];
-
- return "";
-}
-
-unsigned long long process_xen_hypercall_name(struct trace_seq *s,
- unsigned long long *args)
-{
- unsigned int op = args[0];
-
- trace_seq_printf(s, "%s", xen_hypercall_name(op));
- return 0;
-}
-
-int TEP_PLUGIN_LOADER(struct tep_handle *tep)
-{
- tep_register_print_function(tep,
- process_xen_hypercall_name,
- TEP_FUNC_ARG_STRING,
- "xen_hypercall_name",
- TEP_FUNC_ARG_INT,
- TEP_FUNC_ARG_VOID);
- return 0;
-}
-
-void TEP_PLUGIN_UNLOADER(struct tep_handle *tep)
-{
- tep_unregister_print_function(tep, process_xen_hypercall_name,
- "xen_hypercall_name");
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-#undef _GNU_SOURCE
-#include <string.h>
-#include <stdio.h>
-
-#include "event-parse.h"
-
-#undef _PE
-#define _PE(code, str) str
-static const char * const tep_error_str[] = {
- TEP_ERRORS
-};
-#undef _PE
-
-/*
- * The tools so far have been using the strerror_r() GNU variant, that returns
- * a string, be it the buffer passed or something else.
- *
- * But that, besides being tricky in cases where we expect that the function
- * using strerror_r() returns the error formatted in a provided buffer (we have
- * to check if it returned something else and copy that instead), breaks the
- * build on systems not using glibc, like Alpine Linux, where musl libc is
- * used.
- *
- * So, introduce yet another wrapper, str_error_r(), that has the GNU
- * interface, but uses the portable XSI variant of strerror_r(), so that users
- * rest asured that the provided buffer is used and it is what is returned.
- */
-int tep_strerror(struct tep_handle *tep __maybe_unused,
- enum tep_errno errnum, char *buf, size_t buflen)
-{
- const char *msg;
- int idx;
-
- if (!buflen)
- return 0;
-
- if (errnum >= 0) {
- int err = strerror_r(errnum, buf, buflen);
- buf[buflen - 1] = 0;
- return err;
- }
-
- if (errnum <= __TEP_ERRNO__START ||
- errnum >= __TEP_ERRNO__END)
- return -1;
-
- idx = errnum - __TEP_ERRNO__START - 1;
- msg = tep_error_str[idx];
- snprintf(buf, buflen, "%s", msg);
-
- return 0;
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-#include "trace-seq.h"
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <stdarg.h>
-
-#include <asm/bug.h>
-#include "event-parse.h"
-#include "event-utils.h"
-
-/*
- * The TRACE_SEQ_POISON is to catch the use of using
- * a trace_seq structure after it was destroyed.
- */
-#define TRACE_SEQ_POISON ((void *)0xdeadbeef)
-#define TRACE_SEQ_CHECK(s) \
-do { \
- if (WARN_ONCE((s)->buffer == TRACE_SEQ_POISON, \
- "Usage of trace_seq after it was destroyed")) \
- (s)->state = TRACE_SEQ__BUFFER_POISONED; \
-} while (0)
-
-#define TRACE_SEQ_CHECK_RET_N(s, n) \
-do { \
- TRACE_SEQ_CHECK(s); \
- if ((s)->state != TRACE_SEQ__GOOD) \
- return n; \
-} while (0)
-
-#define TRACE_SEQ_CHECK_RET(s) TRACE_SEQ_CHECK_RET_N(s, )
-#define TRACE_SEQ_CHECK_RET0(s) TRACE_SEQ_CHECK_RET_N(s, 0)
-
-/**
- * trace_seq_init - initialize the trace_seq structure
- * @s: a pointer to the trace_seq structure to initialize
- */
-void trace_seq_init(struct trace_seq *s)
-{
- s->len = 0;
- s->readpos = 0;
- s->buffer_size = TRACE_SEQ_BUF_SIZE;
- s->buffer = malloc(s->buffer_size);
- if (s->buffer != NULL)
- s->state = TRACE_SEQ__GOOD;
- else
- s->state = TRACE_SEQ__MEM_ALLOC_FAILED;
-}
-
-/**
- * trace_seq_reset - re-initialize the trace_seq structure
- * @s: a pointer to the trace_seq structure to reset
- */
-void trace_seq_reset(struct trace_seq *s)
-{
- if (!s)
- return;
- TRACE_SEQ_CHECK(s);
- s->len = 0;
- s->readpos = 0;
-}
-
-/**
- * trace_seq_destroy - free up memory of a trace_seq
- * @s: a pointer to the trace_seq to free the buffer
- *
- * Only frees the buffer, not the trace_seq struct itself.
- */
-void trace_seq_destroy(struct trace_seq *s)
-{
- if (!s)
- return;
- TRACE_SEQ_CHECK_RET(s);
- free(s->buffer);
- s->buffer = TRACE_SEQ_POISON;
-}
-
-static void expand_buffer(struct trace_seq *s)
-{
- char *buf;
-
- buf = realloc(s->buffer, s->buffer_size + TRACE_SEQ_BUF_SIZE);
- if (WARN_ONCE(!buf, "Can't allocate trace_seq buffer memory")) {
- s->state = TRACE_SEQ__MEM_ALLOC_FAILED;
- return;
- }
-
- s->buffer = buf;
- s->buffer_size += TRACE_SEQ_BUF_SIZE;
-}
-
-/**
- * trace_seq_printf - sequence printing of trace information
- * @s: trace sequence descriptor
- * @fmt: printf format string
- *
- * It returns 0 if the trace oversizes the buffer's free
- * space, the number of characters printed, or a negative
- * value in case of an error.
- *
- * The tracer may use either sequence operations or its own
- * copy to user routines. To simplify formating of a trace
- * trace_seq_printf is used to store strings into a special
- * buffer (@s). Then the output may be either used by
- * the sequencer or pulled into another buffer.
- */
-int
-trace_seq_printf(struct trace_seq *s, const char *fmt, ...)
-{
- va_list ap;
- int len;
- int ret;
-
- try_again:
- TRACE_SEQ_CHECK_RET0(s);
-
- len = (s->buffer_size - 1) - s->len;
-
- va_start(ap, fmt);
- ret = vsnprintf(s->buffer + s->len, len, fmt, ap);
- va_end(ap);
-
- if (ret >= len) {
- expand_buffer(s);
- goto try_again;
- }
-
- if (ret > 0)
- s->len += ret;
-
- return ret;
-}
-
-/**
- * trace_seq_vprintf - sequence printing of trace information
- * @s: trace sequence descriptor
- * @fmt: printf format string
- *
- * It returns 0 if the trace oversizes the buffer's free
- * space, the number of characters printed, or a negative
- * value in case of an error.
- * *
- * The tracer may use either sequence operations or its own
- * copy to user routines. To simplify formating of a trace
- * trace_seq_printf is used to store strings into a special
- * buffer (@s). Then the output may be either used by
- * the sequencer or pulled into another buffer.
- */
-int
-trace_seq_vprintf(struct trace_seq *s, const char *fmt, va_list args)
-{
- int len;
- int ret;
-
- try_again:
- TRACE_SEQ_CHECK_RET0(s);
-
- len = (s->buffer_size - 1) - s->len;
-
- ret = vsnprintf(s->buffer + s->len, len, fmt, args);
-
- if (ret >= len) {
- expand_buffer(s);
- goto try_again;
- }
-
- if (ret > 0)
- s->len += ret;
-
- return ret;
-}
-
-/**
- * trace_seq_puts - trace sequence printing of simple string
- * @s: trace sequence descriptor
- * @str: simple string to record
- *
- * The tracer may use either the sequence operations or its own
- * copy to user routines. This function records a simple string
- * into a special buffer (@s) for later retrieval by a sequencer
- * or other mechanism.
- */
-int trace_seq_puts(struct trace_seq *s, const char *str)
-{
- int len;
-
- TRACE_SEQ_CHECK_RET0(s);
-
- len = strlen(str);
-
- while (len > ((s->buffer_size - 1) - s->len))
- expand_buffer(s);
-
- TRACE_SEQ_CHECK_RET0(s);
-
- memcpy(s->buffer + s->len, str, len);
- s->len += len;
-
- return len;
-}
-
-int trace_seq_putc(struct trace_seq *s, unsigned char c)
-{
- TRACE_SEQ_CHECK_RET0(s);
-
- while (s->len >= (s->buffer_size - 1))
- expand_buffer(s);
-
- TRACE_SEQ_CHECK_RET0(s);
-
- s->buffer[s->len++] = c;
-
- return 1;
-}
-
-void trace_seq_terminate(struct trace_seq *s)
-{
- TRACE_SEQ_CHECK_RET(s);
-
- /* There's always one character left on the buffer */
- s->buffer[s->len] = 0;
-}
-
-int trace_seq_do_fprintf(struct trace_seq *s, FILE *fp)
-{
- TRACE_SEQ_CHECK(s);
-
- switch (s->state) {
- case TRACE_SEQ__GOOD:
- return fprintf(fp, "%.*s", s->len, s->buffer);
- case TRACE_SEQ__BUFFER_POISONED:
- fprintf(fp, "%s\n", "Usage of trace_seq after it was destroyed");
- break;
- case TRACE_SEQ__MEM_ALLOC_FAILED:
- fprintf(fp, "%s\n", "Can't allocate trace_seq buffer memory");
- break;
- }
- return -1;
-}
-
-int trace_seq_do_printf(struct trace_seq *s)
-{
- return trace_seq_do_fprintf(s, stdout);
-}
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-/*
- * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
- *
- */
-
-#ifndef _TRACE_SEQ_H
-#define _TRACE_SEQ_H
-
-#include <stdarg.h>
-#include <stdio.h>
-
-/* ----------------------- trace_seq ----------------------- */
-
-#ifndef TRACE_SEQ_BUF_SIZE
-#define TRACE_SEQ_BUF_SIZE 4096
-#endif
-
-enum trace_seq_fail {
- TRACE_SEQ__GOOD,
- TRACE_SEQ__BUFFER_POISONED,
- TRACE_SEQ__MEM_ALLOC_FAILED,
-};
-
-/*
- * Trace sequences are used to allow a function to call several other functions
- * to create a string of data to use (up to a max of PAGE_SIZE).
- */
-
-struct trace_seq {
- char *buffer;
- unsigned int buffer_size;
- unsigned int len;
- unsigned int readpos;
- enum trace_seq_fail state;
-};
-
-void trace_seq_init(struct trace_seq *s);
-void trace_seq_reset(struct trace_seq *s);
-void trace_seq_destroy(struct trace_seq *s);
-
-extern int trace_seq_printf(struct trace_seq *s, const char *fmt, ...)
- __attribute__ ((format (printf, 2, 3)));
-extern int trace_seq_vprintf(struct trace_seq *s, const char *fmt, va_list args)
- __attribute__ ((format (printf, 2, 0)));
-
-extern int trace_seq_puts(struct trace_seq *s, const char *str);
-extern int trace_seq_putc(struct trace_seq *s, unsigned char c);
-
-extern void trace_seq_terminate(struct trace_seq *s);
-
-extern int trace_seq_do_fprintf(struct trace_seq *s, FILE *fp);
-extern int trace_seq_do_printf(struct trace_seq *s);
-
-#endif /* _TRACE_SEQ_H */
--- /dev/null
+objtool-y += decode.o
+objtool-y += special.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <objtool/check.h>
+#include <objtool/elf.h>
+#include <objtool/arch.h>
+#include <objtool/warn.h>
+#include <objtool/builtin.h>
+#include <objtool/endianness.h>
+
+int arch_ftrace_match(char *name)
+{
+ return !strcmp(name, "_mcount");
+}
+
+unsigned long arch_dest_reloc_offset(int addend)
+{
+ return addend;
+}
+
+bool arch_callee_saved_reg(unsigned char reg)
+{
+ return false;
+}
+
+int arch_decode_hint_reg(u8 sp_reg, int *base)
+{
+ exit(-1);
+}
+
+const char *arch_nop_insn(int len)
+{
+ exit(-1);
+}
+
+const char *arch_ret_insn(int len)
+{
+ exit(-1);
+}
+
+int arch_decode_instruction(struct objtool_file *file, const struct section *sec,
+ unsigned long offset, unsigned int maxlen,
+ unsigned int *len, enum insn_type *type,
+ unsigned long *immediate,
+ struct list_head *ops_list)
+{
+ unsigned int opcode;
+ enum insn_type typ;
+ unsigned long imm;
+ u32 insn;
+
+ insn = bswap_if_needed(file->elf, *(u32 *)(sec->data->d_buf + offset));
+ opcode = insn >> 26;
+ typ = INSN_OTHER;
+ imm = 0;
+
+ switch (opcode) {
+ case 18: /* b[l][a] */
+ if ((insn & 3) == 1) /* bl */
+ typ = INSN_CALL;
+
+ imm = insn & 0x3fffffc;
+ if (imm & 0x2000000)
+ imm -= 0x4000000;
+ break;
+ }
+
+ if (opcode == 1)
+ *len = 8;
+ else
+ *len = 4;
+
+ *type = typ;
+ *immediate = imm;
+
+ return 0;
+}
+
+unsigned long arch_jump_destination(struct instruction *insn)
+{
+ return insn->offset + insn->immediate;
+}
+
+bool arch_pc_relative_reloc(struct reloc *reloc)
+{
+ /*
+ * The powerpc build only allows certain relocation types, see
+ * relocs_check.sh, and none of those accepted are PC relative.
+ */
+ return false;
+}
+
+void arch_initial_func_cfi_state(struct cfi_init_state *state)
+{
+ int i;
+
+ for (i = 0; i < CFI_NUM_REGS; i++) {
+ state->regs[i].base = CFI_UNDEFINED;
+ state->regs[i].offset = 0;
+ }
+
+ /* initial CFA (call frame address) */
+ state->cfa.base = CFI_SP;
+ state->cfa.offset = 0;
+
+ /* initial LR (return address) */
+ state->regs[CFI_RA].base = CFI_CFA;
+ state->regs[CFI_RA].offset = 0;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef _OBJTOOL_CFI_REGS_H
+#define _OBJTOOL_CFI_REGS_H
+
+#define CFI_BP 1
+#define CFI_SP CFI_BP
+#define CFI_RA 32
+#define CFI_NUM_REGS 33
+
+#endif
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef _OBJTOOL_ARCH_ELF
+#define _OBJTOOL_ARCH_ELF
+
+#define R_NONE R_PPC_NONE
+#define R_ABS64 R_PPC64_ADDR64
+#define R_ABS32 R_PPC_ADDR32
+
+#endif /* _OBJTOOL_ARCH_ELF */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef _PPC_ARCH_SPECIAL_H
+#define _PPC_ARCH_SPECIAL_H
+
+#define EX_ENTRY_SIZE 8
+#define EX_ORIG_OFFSET 0
+#define EX_NEW_OFFSET 4
+
+#define JUMP_ENTRY_SIZE 16
+#define JUMP_ORIG_OFFSET 0
+#define JUMP_NEW_OFFSET 4
+#define JUMP_KEY_OFFSET 8
+
+#define ALT_ENTRY_SIZE 12
+#define ALT_ORIG_OFFSET 0
+#define ALT_NEW_OFFSET 4
+#define ALT_FEATURE_OFFSET 8
+#define ALT_ORIG_LEN_OFFSET 10
+#define ALT_NEW_LEN_OFFSET 11
+
+#endif /* _PPC_ARCH_SPECIAL_H */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-or-later
+#include <string.h>
+#include <stdlib.h>
+#include <objtool/special.h>
+#include <objtool/builtin.h>
+
+
+bool arch_support_alt_relocation(struct special_alt *special_alt,
+ struct instruction *insn,
+ struct reloc *reloc)
+{
+ exit(-1);
+}
+
+struct reloc *arch_find_switch_table(struct objtool_file *file,
+ struct instruction *insn)
+{
+ exit(-1);
+}
#include <objtool/builtin.h>
#include <arch/elf.h>
+int arch_ftrace_match(char *name)
+{
+ return !strcmp(name, "__fentry__");
+}
+
static int is_x86_64(const struct elf *elf)
{
switch (elf->ehdr.e_machine) {
#define _OBJTOOL_ARCH_ELF
#define R_NONE R_X86_64_NONE
+#define R_ABS64 R_X86_64_64
+#define R_ABS32 R_X86_64_32
#endif /* _OBJTOOL_ARCH_ELF */
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-#ifndef _ARCH_ENDIANNESS_H
-#define _ARCH_ENDIANNESS_H
-
-#include <endian.h>
-
-#define __TARGET_BYTE_ORDER __LITTLE_ENDIAN
-
-#endif /* _ARCH_ENDIANNESS_H */
OPT_BOOLEAN(0, "dry-run", &opts.dryrun, "don't write modifications"),
OPT_BOOLEAN(0, "link", &opts.link, "object is a linked object"),
OPT_BOOLEAN(0, "module", &opts.module, "object is part of a kernel module"),
+ OPT_BOOLEAN(0, "mnop", &opts.mnop, "nop out mcount call sites"),
OPT_BOOLEAN(0, "no-unreachable", &opts.no_unreachable, "skip 'unreachable instruction' warnings"),
OPT_BOOLEAN(0, "sec-address", &opts.sec_address, "print section addresses in warnings"),
OPT_BOOLEAN(0, "stats", &opts.stats, "print statistics"),
return false;
}
+static bool mnop_opts_valid(void)
+{
+ if (opts.mnop && !opts.mcount) {
+ ERROR("--mnop requires --mcount");
+ return false;
+ }
+
+ return true;
+}
+
static bool link_opts_valid(struct objtool_file *file)
{
if (opts.link)
if (!file)
return 1;
+ if (!mnop_opts_valid())
+ return 1;
+
if (!link_opts_valid(file))
return 1;
return false;
insn = find_insn(file, func->sec, func->offset);
- if (!insn_func(insn))
+ if (!insn || !insn_func(insn))
return false;
func_for_each_insn(file, func, insn) {
static int create_mcount_loc_sections(struct objtool_file *file)
{
- struct section *sec;
- unsigned long *loc;
+ int addrsize = elf_class_addrsize(file->elf);
struct instruction *insn;
+ struct section *sec;
int idx;
sec = find_section_by_name(file->elf, "__mcount_loc");
list_for_each_entry(insn, &file->mcount_loc_list, call_node)
idx++;
- sec = elf_create_section(file->elf, "__mcount_loc", 0, sizeof(unsigned long), idx);
+ sec = elf_create_section(file->elf, "__mcount_loc", 0, addrsize, idx);
if (!sec)
return -1;
+ sec->sh.sh_addralign = addrsize;
+
idx = 0;
list_for_each_entry(insn, &file->mcount_loc_list, call_node) {
+ void *loc;
- loc = (unsigned long *)sec->data->d_buf + idx;
- memset(loc, 0, sizeof(unsigned long));
+ loc = sec->data->d_buf + idx;
+ memset(loc, 0, addrsize);
- if (elf_add_reloc_to_insn(file->elf, sec,
- idx * sizeof(unsigned long),
- R_X86_64_64,
+ if (elf_add_reloc_to_insn(file->elf, sec, idx,
+ addrsize == sizeof(u64) ? R_ABS64 : R_ABS32,
insn->sec, insn->offset))
return -1;
- idx++;
+ idx += addrsize;
}
return 0;
if (opts.mcount && sym->fentry) {
if (sibling)
WARN_FUNC("Tail call to __fentry__ !?!?", insn->sec, insn->offset);
+ if (opts.mnop) {
+ if (reloc) {
+ reloc->type = R_NONE;
+ elf_write_reloc(file->elf, reloc);
+ }
- if (reloc) {
- reloc->type = R_NONE;
- elf_write_reloc(file->elf, reloc);
- }
-
- elf_write_insn(file->elf, insn->sec,
- insn->offset, insn->len,
- arch_nop_insn(insn->len));
+ elf_write_insn(file->elf, insn->sec,
+ insn->offset, insn->len,
+ arch_nop_insn(insn->len));
- insn->type = INSN_NOP;
+ insn->type = INSN_NOP;
+ }
list_add_tail(&insn->call_node, &file->mcount_loc_list);
return;
return -1;
}
- cfi.cfa.offset = bswap_if_needed(hint->sp_offset);
+ cfi.cfa.offset = bswap_if_needed(file->elf, hint->sp_offset);
cfi.type = hint->type;
cfi.end = hint->end;
if (arch_is_rethunk(func))
func->return_thunk = true;
- if (!strcmp(func->name, "__fentry__"))
+ if (arch_ftrace_match(func->name))
func->fentry = true;
if (is_profiling_func(func->name))
* Must be before add_jump_destinations(), which depends on 'func'
* being set for alternatives, to enable proper sibling call detection.
*/
- ret = add_special_section_alts(file);
- if (ret)
- return ret;
+ if (opts.stackval || opts.orc || opts.uaccess || opts.noinstr) {
+ ret = add_special_section_alts(file);
+ if (ret)
+ return ret;
+ }
ret = add_jump_destinations(file);
if (ret)
{
char *relocname;
struct section *sec;
+ int addrsize = elf_class_addrsize(elf);
relocname = malloc(strlen(base->name) + strlen(".rela") + 1);
if (!relocname) {
strcpy(relocname, ".rela");
strcat(relocname, base->name);
- sec = elf_create_section(elf, relocname, 0, sizeof(GElf_Rela), 0);
+ if (addrsize == sizeof(u32))
+ sec = elf_create_section(elf, relocname, 0, sizeof(Elf32_Rela), 0);
+ else
+ sec = elf_create_section(elf, relocname, 0, sizeof(GElf_Rela), 0);
free(relocname);
if (!sec)
return NULL;
sec->base = base;
sec->sh.sh_type = SHT_RELA;
- sec->sh.sh_addralign = 8;
+ sec->sh.sh_addralign = addrsize;
sec->sh.sh_link = find_section_by_name(elf, ".symtab")->idx;
sec->sh.sh_info = base->idx;
sec->sh.sh_flags = SHF_INFO_LINK;
struct instruction;
+int arch_ftrace_match(char *name);
+
void arch_initial_func_cfi_state(struct cfi_init_state *state);
int arch_decode_instruction(struct objtool_file *file, const struct section *sec,
bool backup;
bool dryrun;
bool link;
+ bool mnop;
bool module;
bool no_unreachable;
bool sec_address;
return elf->num_files > 1;
}
+static inline int elf_class_addrsize(struct elf *elf)
+{
+ if (elf->ehdr.e_ident[EI_CLASS] == ELFCLASS32)
+ return sizeof(u32);
+ else
+ return sizeof(u64);
+}
+
struct elf *elf_open_read(const char *name, int flags);
struct section *elf_create_section(struct elf *elf, const char *name, unsigned int sh_flags, size_t entsize, int nr);
#ifndef _OBJTOOL_ENDIANNESS_H
#define _OBJTOOL_ENDIANNESS_H
-#include <arch/endianness.h>
#include <linux/kernel.h>
#include <endian.h>
-
-#ifndef __TARGET_BYTE_ORDER
-#error undefined arch __TARGET_BYTE_ORDER
-#endif
-
-#if __BYTE_ORDER != __TARGET_BYTE_ORDER
-#define __NEED_BSWAP 1
-#else
-#define __NEED_BSWAP 0
-#endif
+#include <objtool/elf.h>
/*
- * Does a byte swap if target endianness doesn't match the host, i.e. cross
+ * Does a byte swap if target file endianness doesn't match the host, i.e. cross
* compilation for little endian on big endian and vice versa.
* To be used for multi-byte values conversion, which are read from / about
* to be written to a target native endianness ELF file.
*/
-#define bswap_if_needed(val) \
+static inline bool need_bswap(struct elf *elf)
+{
+ return (__BYTE_ORDER == __LITTLE_ENDIAN) ^
+ (elf->ehdr.e_ident[EI_DATA] == ELFDATA2LSB);
+}
+
+#define bswap_if_needed(elf, val) \
({ \
__typeof__(val) __ret; \
+ bool __need_bswap = need_bswap(elf); \
switch (sizeof(val)) { \
- case 8: __ret = __NEED_BSWAP ? bswap_64(val) : (val); break; \
- case 4: __ret = __NEED_BSWAP ? bswap_32(val) : (val); break; \
- case 2: __ret = __NEED_BSWAP ? bswap_16(val) : (val); break; \
+ case 8: \
+ __ret = __need_bswap ? bswap_64(val) : (val); break; \
+ case 4: \
+ __ret = __need_bswap ? bswap_32(val) : (val); break; \
+ case 2: \
+ __ret = __need_bswap ? bswap_16(val) : (val); break; \
default: \
BUILD_BUG(); break; \
} \
GElf_Rela rela;
GElf_Sym sym;
Elf_Data *data, *symtab = NULL, *rela_orc_ip = NULL;
+ struct elf dummy_elf = {};
objname = _objname;
return -1;
}
+ if (!elf64_getehdr(elf)) {
+ WARN_ELF("elf64_getehdr");
+ return -1;
+ }
+ memcpy(&dummy_elf.ehdr, elf64_getehdr(elf), sizeof(dummy_elf.ehdr));
+
if (elf_getshdrnum(elf, &nr_sections)) {
WARN_ELF("elf_getshdrnum");
return -1;
printf(" sp:");
- print_reg(orc[i].sp_reg, bswap_if_needed(orc[i].sp_offset));
+ print_reg(orc[i].sp_reg, bswap_if_needed(&dummy_elf, orc[i].sp_offset));
printf(" bp:");
- print_reg(orc[i].bp_reg, bswap_if_needed(orc[i].bp_offset));
+ print_reg(orc[i].bp_reg, bswap_if_needed(&dummy_elf, orc[i].bp_offset));
printf(" type:%s end:%d\n",
orc_type_name(orc[i].type), orc[i].end);
/* populate ORC data */
orc = (struct orc_entry *)orc_sec->data->d_buf + idx;
memcpy(orc, o, sizeof(*orc));
- orc->sp_offset = bswap_if_needed(orc->sp_offset);
- orc->bp_offset = bswap_if_needed(orc->bp_offset);
+ orc->sp_offset = bswap_if_needed(elf, orc->sp_offset);
+ orc->bp_offset = bswap_if_needed(elf, orc->bp_offset);
/* populate reloc for ip */
if (elf_add_reloc_to_insn(elf, ip_sec, idx * sizeof(int), R_X86_64_PC32,
if (entry->feature) {
unsigned short feature;
- feature = bswap_if_needed(*(unsigned short *)(sec->data->d_buf +
+ feature = bswap_if_needed(elf,
+ *(unsigned short *)(sec->data->d_buf +
offset +
entry->feature));
arch_handle_alternative(feature, alt);
pmu-events/pmu-events.c
pmu-events/jevents
feature/
+libapi/
libbpf/
+libperf/
+libsubcmd/
+libsymbol/
+libtraceevent/
+libtraceevent_plugins/
fixdep
-libtraceevent-dynamic-list
Documentation/doc.dep
perf-y += builtin-evlist.o
perf-y += builtin-ftrace.o
perf-y += builtin-help.o
-perf-y += builtin-sched.o
perf-y += builtin-buildid-list.o
perf-y += builtin-buildid-cache.o
perf-y += builtin-kallsyms.o
perf-y += builtin-record.o
perf-y += builtin-report.o
perf-y += builtin-stat.o
-perf-y += builtin-timechart.o
perf-y += builtin-top.o
perf-y += builtin-script.o
-perf-y += builtin-kmem.o
-perf-y += builtin-lock.o
perf-y += builtin-kvm.o
perf-y += builtin-inject.o
perf-y += builtin-mem.o
perf-y += builtin-version.o
perf-y += builtin-c2c.o
perf-y += builtin-daemon.o
-perf-y += builtin-kwork.o
-perf-$(CONFIG_TRACE) += builtin-trace.o
+perf-$(CONFIG_LIBTRACEEVENT) += builtin-kmem.o
+perf-$(CONFIG_LIBTRACEEVENT) += builtin-kwork.o
+perf-$(CONFIG_LIBTRACEEVENT) += builtin-lock.o
+perf-$(CONFIG_LIBTRACEEVENT) += builtin-sched.o
+perf-$(CONFIG_LIBTRACEEVENT) += builtin-timechart.o
+
+ifeq ($(CONFIG_LIBTRACEEVENT),y)
+ perf-$(CONFIG_TRACE) += builtin-trace.o
+ perf-$(CONFIG_TRACE) += trace/beauty/
+endif
+
perf-$(CONFIG_LIBELF) += builtin-probe.o
perf-y += bench/
perf-y += util/
perf-y += arch/
perf-y += ui/
-perf-y += scripts/
-perf-$(CONFIG_TRACE) += trace/beauty/
+perf-$(CONFIG_LIBTRACEEVENT) += scripts/
gtk-y += ui/gtk/
-q::
--quiet::
- Do not show any message. (Suppress -v)
+ Do not show any warnings or messages. (Suppress -v)
-n::
--show-nr-samples::
-q::
--quiet::
- Do not show any message. (Suppress -v)
+ Do not show any warnings or messages. (Suppress -v)
-f::
--force::
unpack the raw data for power events and PTWRITE. The script also displays
branches, and supports 2 additional modes selected by option:
- --insn-trace - instruction trace
- --src-trace - source trace
+ - --insn-trace - instruction trace
+ - --src-trace - source trace
+
+The intel-pt-events.py script also has options:
+
+ - --all-switch-events - display all switch events, not only the last consecutive.
+ - --interleave [<n>] - interleave sample output for the same timestamp so that
+ no more than n samples for a CPU are displayed in a row. 'n' defaults to 4.
+ Note this only affects the order of output, and only when the timestamp is the
+ same.
As mentioned above, it is easy to capture too much data. One way to limit the
data captured is to use 'snapshot' mode which is explained further below.
--deprecated::
Print deprecated events. By default the deprecated events are hidden.
---cputype::
-Print events applying cpu with this type for hybrid platform
-(e.g. --cputype core or --cputype atom)
+--unit::
+Print PMU events and metrics limited to the specific PMU name.
+(e.g. --unit cpu, --unit msr, --unit cpu_core, --unit cpu_atom)
+
+-j::
+--json::
+Output in JSON format.
[[EVENT_MODIFIERS]]
EVENT MODIFIERS
-q::
--quiet::
- Do not show any message. (Suppress -v)
+ Do not show any warnings or messages. (Suppress -v)
-D::
--dump-raw-trace::
--entries=<value>::
Display this many entries.
+-l::
+--lock-addr::
+ Show lock contention stat by address
+
SEE ALSO
--------
-q::
--quiet::
- Be quiet (do not show any messages including errors).
+ Do not show any warnings or messages.
Can not use with -v.
-a::
Also, by adding a comma, the number of mmap pages for AUX
area tracing can be specified.
---group::
- Put all events in a single event group. This precedes the --event
- option and remains only for backward compatibility. See --event.
-
-g::
Enables call-graph (stack chain/backtrace) recording for both
kernel space and user space.
-q::
--quiet::
- Don't print any message, useful for scripting.
+ Don't print any warnings or messages, useful for scripting.
-v::
--verbose::
- any_call: any function call or system call
- any_ret: any function return or system call return
- ind_call: any indirect branch
+ - ind_jmp: any indirect jump
- call: direct calls, including far (to/from kernel) calls
- u: only when the branch target is at the user level
- k: only when the branch target is in the kernel
- no_tx: only when the target is not in a hardware transaction
- abort_tx: only when the target is a hardware transaction abort
- cond: conditional branches
+ - call_stack: save call stack
+ - no_flags: don't save branch flags e.g prediction, misprediction etc
+ - no_cycles: don't save branch cycles
+ - hw_index: save branch hardware index
- save_type: save branch type during sampling in case binary is not available later
For the platforms with Intel Arch LBR support (12th-Gen+ client or
4th-Gen Xeon+ server), the save branch type is unconditionally enabled
-q::
--quiet::
- Do not show any message. (Suppress -v)
+ Do not show any warnings or messages. (Suppress -v)
-n::
--show-nr-samples::
may be used to increase accuracy in this case.
--quiet::
-Don't print output. This is useful with perf stat record below to only
-write data to the perf.data file.
+Don't print output, warnings or messages. This is useful with perf stat
+record below to only write data to the perf.data file.
STAT RECORD
-----------
--count-filter=<count>::
Only display functions with more events than this.
---group::
- Put the counters into a counter group.
-
--group-sort-idx::
Sort the output by the event at the index n in group. If n is invalid,
sort by the first event. It can support multiple groups with different
perf top -e cycles,probe:icmp_rcv --switch-on=probe:icmp_rcv
- Alternatively one can ask for --group and then two overhead columns
+ Alternatively one can ask for a group and then two overhead columns
will appear, the first for cycles and the second for the switch-on event.
- perf top --group -e cycles,probe:icmp_rcv --switch-on=probe:icmp_rcv
+ perf top -e '{cycles,probe:icmp_rcv}' --switch-on=probe:icmp_rcv
This may be interesting to measure a workload only after some initialization
phase is over, i.e. insert a perf probe at that point and use the above
tools/scripts
tools/build
tools/include
-tools/lib/traceevent
tools/lib/api
tools/lib/bpf
tools/lib/subcmd
tools/lib/hweight.c
tools/lib/rbtree.c
tools/lib/string.c
-tools/lib/symbol/kallsyms.c
-tools/lib/symbol/kallsyms.h
+tools/lib/symbol
tools/lib/find_bit.c
tools/lib/bitmap.c
tools/lib/list_sort.c
# (To override it, run 'make JOBS=1' and similar.)
#
ifeq ($(JOBS),)
- JOBS := $(shell (getconf _NPROCESSORS_ONLN || egrep -c '^processor|^CPU[0-9]' /proc/cpuinfo) 2>/dev/null)
+ JOBS := $(shell (getconf _NPROCESSORS_ONLN || grep -E -c '^processor|^CPU[0-9]' /proc/cpuinfo) 2>/dev/null)
ifeq ($(JOBS),0)
JOBS := 1
endif
CORE_CFLAGS += -funwind-tables
CORE_CFLAGS += -Wall
CORE_CFLAGS += -Wextra
-CORE_CFLAGS += -std=gnu99
+CORE_CFLAGS += -std=gnu11
CXXFLAGS += -std=gnu++14 -fno-exceptions -fno-rtti
CXXFLAGS += -Wall
endif
endif
-INC_FLAGS += -I$(srctree)/tools/lib/perf/include
INC_FLAGS += -I$(src-perf)/util/include
INC_FLAGS += -I$(src-perf)/arch/$(SRCARCH)/include
INC_FLAGS += -I$(srctree)/tools/include/
INC_FLAGS += -I$(src-perf)/util
INC_FLAGS += -I$(src-perf)
-INC_FLAGS += -I$(srctree)/tools/lib/
CORE_CFLAGS += -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE
EXTLIBS += $(EXTLIBS_LIBUNWIND)
endif
-ifeq ($(NO_SYSCALL_TABLE),0)
- $(call detected,CONFIG_TRACE)
-else
- ifndef NO_LIBAUDIT
- $(call feature_check,libaudit)
- ifneq ($(feature-libaudit), 1)
- msg := $(warning No libaudit.h found, disables 'trace' tool, please install audit-libs-devel or libaudit-dev);
- NO_LIBAUDIT := 1
- else
- CFLAGS += -DHAVE_LIBAUDIT_SUPPORT
- EXTLIBS += -laudit
- $(call detected,CONFIG_TRACE)
+ifneq ($(NO_LIBTRACEEVENT),1)
+ ifeq ($(NO_SYSCALL_TABLE),0)
+ $(call detected,CONFIG_TRACE)
+ else
+ ifndef NO_LIBAUDIT
+ $(call feature_check,libaudit)
+ ifneq ($(feature-libaudit), 1)
+ msg := $(warning No libaudit.h found, disables 'trace' tool, please install audit-libs-devel or libaudit-dev);
+ NO_LIBAUDIT := 1
+ else
+ CFLAGS += -DHAVE_LIBAUDIT_SUPPORT
+ EXTLIBS += -laudit
+ $(call detected,CONFIG_TRACE)
+ endif
endif
endif
endif
NO_LIBPYTHON := 1
endef
+PYTHON_EXTENSION_SUFFIX := '.so'
ifdef NO_LIBPYTHON
$(call disable-python,Python support disabled by user)
else
else
LDFLAGS += $(PYTHON_EMBED_LDFLAGS)
EXTLIBS += $(PYTHON_EMBED_LIBADD)
- LANG_BINDINGS += $(obj-perf)python/perf.so
+ PYTHON_EXTENSION_SUFFIX := $(shell $(PYTHON) -c 'from importlib import machinery; print(machinery.EXTENSION_SUFFIXES[0])')
+ LANG_BINDINGS += $(obj-perf)python/perf$(PYTHON_EXTENSION_SUFFIX)
CFLAGS += -DHAVE_LIBPYTHON_SUPPORT
$(call detected,CONFIG_LIBPYTHON)
endif
endif
endif
-ifdef LIBTRACEEVENT_DYNAMIC
+# libtraceevent is a recommended dependency picked up from the system.
+ifneq ($(NO_LIBTRACEEVENT),1)
$(call feature_check,libtraceevent)
ifeq ($(feature-libtraceevent), 1)
+ CFLAGS += -DHAVE_LIBTRACEEVENT
EXTLIBS += -ltraceevent
LIBTRACEEVENT_VERSION := $(shell $(PKG_CONFIG) --modversion libtraceevent)
LIBTRACEEVENT_VERSION_1 := $(word 1, $(subst ., ,$(LIBTRACEEVENT_VERSION)))
LIBTRACEEVENT_VERSION_3 := $(word 3, $(subst ., ,$(LIBTRACEEVENT_VERSION)))
LIBTRACEEVENT_VERSION_CPP := $(shell expr $(LIBTRACEEVENT_VERSION_1) \* 255 \* 255 + $(LIBTRACEEVENT_VERSION_2) \* 255 + $(LIBTRACEEVENT_VERSION_3))
CFLAGS += -DLIBTRACEEVENT_VERSION=$(LIBTRACEEVENT_VERSION_CPP)
+ $(call detected,CONFIG_LIBTRACEEVENT)
+ LIBTRACEEVENT_VERSION_WITH_TEP_FIELD_IS_RELATIVE := $(shell expr 1 \* 255 \* 255 + 5 \* 255 + 0) # 1.5.0
+ ifeq ($(shell test $(LIBTRACEEVENT_VERSION_CPP) -gt $(LIBTRACEEVENT_VERSION_WITH_TEP_FIELD_IS_RELATIVE); echo $$?),0)
+ CFLAGS += -DHAVE_LIBTRACEEVENT_TEP_FIELD_IS_RELATIVE
+ endif
else
- dummy := $(error Error: No libtraceevent devel library found, please install libtraceevent-devel);
+ dummy := $(warning Warning: libtraceevent is missing limiting functionality, please install libtraceevent-dev/libtraceevent-devel)
endif
-endif
-ifdef LIBTRACEFS_DYNAMIC
$(call feature_check,libtracefs)
ifeq ($(feature-libtracefs), 1)
EXTLIBS += -ltracefs
LIBTRACEFS_VERSION_3 := $(word 3, $(subst ., ,$(LIBTRACEFS_VERSION)))
LIBTRACEFS_VERSION_CPP := $(shell expr $(LIBTRACEFS_VERSION_1) \* 255 \* 255 + $(LIBTRACEFS_VERSION_2) \* 255 + $(LIBTRACEFS_VERSION_3))
CFLAGS += -DLIBTRACEFS_VERSION=$(LIBTRACEFS_VERSION_CPP)
- else
- dummy := $(error Error: No libtracefs devel library found, please install libtracefs-dev);
endif
endif
# Among the variables below, these:
# perfexecdir
-# perf_include_dir
+# libbpf_include_dir
# perf_examples_dir
# template_dir
# mandir
mandir = share/man
infodir = share/info
perfexecdir = libexec/perf-core
-perf_include_dir = lib/perf/include
+# FIXME: system's libbpf header directory, where we expect to find bpf/bpf_helpers.h, for instance
+libbpf_include_dir = /usr/include
perf_examples_dir = lib/perf/examples
sharedir = $(prefix)/share
template_dir = share/perf-core/templates
mandir_SQ = $(subst ','\'',$(mandir))
infodir_SQ = $(subst ','\'',$(infodir))
perfexecdir_SQ = $(subst ','\'',$(perfexecdir))
-perf_include_dir_SQ = $(subst ','\'',$(perf_include_dir))
+libbpf_include_dir_SQ = $(subst ','\'',$(libbpf_include_dir))
perf_examples_dir_SQ = $(subst ','\'',$(perf_examples_dir))
template_dir_SQ = $(subst ','\'',$(template_dir))
htmldir_SQ = $(subst ','\'',$(htmldir))
ifneq ($(filter /%,$(firstword $(perfexecdir))),)
perfexec_instdir = $(perfexecdir)
-perf_include_instdir = $(perf_include_dir)
+perf_include_instdir = $(libbpf_include_dir)
perf_examples_instdir = $(perf_examples_dir)
STRACE_GROUPS_INSTDIR = $(STRACE_GROUPS_DIR)
tip_instdir = $(tipdir)
else
perfexec_instdir = $(prefix)/$(perfexecdir)
-perf_include_instdir = $(prefix)/$(perf_include_dir)
+perf_include_instdir = $(prefix)/$(libbpf_include_dir)
perf_examples_instdir = $(prefix)/$(perf_examples_dir)
STRACE_GROUPS_INSTDIR = $(prefix)/$(STRACE_GROUPS_DIR)
tip_instdir = $(prefix)/$(tipdir)
$(call detected_var,STRACE_GROUPS_DIR_SQ)
$(call detected_var,prefix_SQ)
$(call detected_var,perfexecdir_SQ)
-$(call detected_var,perf_include_dir_SQ)
+$(call detected_var,libbpf_include_dir_SQ)
$(call detected_var,perf_examples_dir_SQ)
$(call detected_var,tipdir_SQ)
$(call detected_var,srcdir_SQ)
#
# Define BUILD_BPF_SKEL to enable BPF skeletons
#
-# Define LIBTRACEEVENT_DYNAMIC to enable libtraceevent dynamic linking
-#
-# Define LIBTRACEFS_DYNAMIC to enable libtracefs dynamic linking
-#
# As per kernel Makefile, avoid funny character set dependencies
unexport LC_ALL
else # force_fixdep
-LIB_DIR = $(srctree)/tools/lib/api/
-TRACE_EVENT_DIR = $(srctree)/tools/lib/traceevent/
+LIBAPI_DIR = $(srctree)/tools/lib/api/
LIBBPF_DIR = $(srctree)/tools/lib/bpf/
-SUBCMD_DIR = $(srctree)/tools/lib/subcmd/
+LIBSUBCMD_DIR = $(srctree)/tools/lib/subcmd/
+LIBSYMBOL_DIR = $(srctree)/tools/lib/symbol/
LIBPERF_DIR = $(srctree)/tools/lib/perf/
DOC_DIR = $(srctree)/tools/perf/Documentation/
strip-libs = $(filter-out -l%,$(1))
ifneq ($(OUTPUT),)
- TE_PATH=$(OUTPUT)
- PLUGINS_PATH=$(OUTPUT)
- SUBCMD_PATH=$(OUTPUT)
- LIBPERF_PATH=$(OUTPUT)
-ifneq ($(subdir),)
- API_PATH=$(OUTPUT)/../lib/api/
+ LIBAPI_OUTPUT = $(abspath $(OUTPUT))/libapi
else
- API_PATH=$(OUTPUT)
+ LIBAPI_OUTPUT = $(CURDIR)/libapi
endif
-else
- TE_PATH=$(TRACE_EVENT_DIR)
- PLUGINS_PATH=$(TRACE_EVENT_DIR)plugins/
- API_PATH=$(LIB_DIR)
- SUBCMD_PATH=$(SUBCMD_DIR)
- LIBPERF_PATH=$(LIBPERF_DIR)
-endif
-
-LIBTRACEEVENT = $(TE_PATH)libtraceevent.a
-export LIBTRACEEVENT
-LIBTRACEEVENT_DYNAMIC_LIST = $(PLUGINS_PATH)libtraceevent-dynamic-list
-
-#
-# The static build has no dynsym table, so this does not work for
-# static build. Looks like linker starts to scream about that now
-# (in Fedora 26) so we need to switch it off for static build.
-DYNAMIC_LIST_LDFLAGS = -Xlinker --dynamic-list=$(LIBTRACEEVENT_DYNAMIC_LIST)
-LIBTRACEEVENT_DYNAMIC_LIST_LDFLAGS = $(if $(findstring -static,$(LDFLAGS)),,$(DYNAMIC_LIST_LDFLAGS))
-
-LIBAPI = $(API_PATH)libapi.a
+LIBAPI_DESTDIR = $(LIBAPI_OUTPUT)
+LIBAPI_INCLUDE = $(LIBAPI_DESTDIR)/include
+LIBAPI = $(LIBAPI_OUTPUT)/libapi.a
export LIBAPI
+CFLAGS += -I$(LIBAPI_OUTPUT)/include
ifneq ($(OUTPUT),)
LIBBPF_OUTPUT = $(abspath $(OUTPUT))/libbpf
LIBBPF_DESTDIR = $(LIBBPF_OUTPUT)
LIBBPF_INCLUDE = $(LIBBPF_DESTDIR)/include
LIBBPF = $(LIBBPF_OUTPUT)/libbpf.a
+CFLAGS += -I$(LIBBPF_OUTPUT)/include
-LIBSUBCMD = $(SUBCMD_PATH)libsubcmd.a
+ifneq ($(OUTPUT),)
+ LIBSUBCMD_OUTPUT = $(abspath $(OUTPUT))/libsubcmd
+else
+ LIBSUBCMD_OUTPUT = $(CURDIR)/libsubcmd
+endif
+LIBSUBCMD_DESTDIR = $(LIBSUBCMD_OUTPUT)
+LIBSUBCMD_INCLUDE = $(LIBSUBCMD_DESTDIR)/include
+LIBSUBCMD = $(LIBSUBCMD_OUTPUT)/libsubcmd.a
+CFLAGS += -I$(LIBSUBCMD_OUTPUT)/include
+
+ifneq ($(OUTPUT),)
+ LIBSYMBOL_OUTPUT = $(abspath $(OUTPUT))/libsymbol
+else
+ LIBSYMBOL_OUTPUT = $(CURDIR)/libsymbol
+endif
+LIBSYMBOL_DESTDIR = $(LIBSYMBOL_OUTPUT)
+LIBSYMBOL_INCLUDE = $(LIBSYMBOL_DESTDIR)/include
+LIBSYMBOL = $(LIBSYMBOL_OUTPUT)/libsymbol.a
+CFLAGS += -I$(LIBSYMBOL_OUTPUT)/include
-LIBPERF = $(LIBPERF_PATH)libperf.a
+ifneq ($(OUTPUT),)
+ LIBPERF_OUTPUT = $(abspath $(OUTPUT))/libperf
+else
+ LIBPERF_OUTPUT = $(CURDIR)/libperf
+endif
+LIBPERF_DESTDIR = $(LIBPERF_OUTPUT)
+LIBPERF_INCLUDE = $(LIBPERF_DESTDIR)/include
+LIBPERF = $(LIBPERF_OUTPUT)/libperf.a
export LIBPERF
+CFLAGS += -I$(LIBPERF_OUTPUT)/include
# python extension build directories
PYTHON_EXTBUILD := $(OUTPUT)python_ext_build/
python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT)python/perf*.so
-PYTHON_EXT_SRCS := $(shell grep -v ^\# util/python-ext-sources)
-PYTHON_EXT_DEPS := util/python-ext-sources util/setup.py $(LIBTRACEEVENT) $(LIBAPI)
+ifeq ($(CONFIG_LIBTRACEEVENT),y)
+ PYTHON_EXT_SRCS := $(shell grep -v ^\# util/python-ext-sources)
+else
+ PYTHON_EXT_SRCS := $(shell grep -v '^\#\|util/trace-event.c' util/python-ext-sources)
+endif
+
+PYTHON_EXT_DEPS := util/python-ext-sources util/setup.py $(LIBAPI)
SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH))
export PERL_PATH
-PERFLIBS = $(LIBAPI) $(LIBSUBCMD) $(LIBPERF)
+PERFLIBS = $(LIBAPI) $(LIBPERF) $(LIBSUBCMD) $(LIBSYMBOL)
ifndef NO_LIBBPF
ifndef LIBBPF_DYNAMIC
PERFLIBS += $(LIBBPF)
endif
endif
-ifndef LIBTRACEEVENT_DYNAMIC
- PERFLIBS += $(LIBTRACEEVENT)
-endif
# We choose to avoid "if .. else if .. else .. endif endif"
# because maintaining the nesting to match is a pain. If
# Create python binding output directory if not already present
_dummy := $(shell [ -d '$(OUTPUT)python' ] || mkdir -p '$(OUTPUT)python')
-$(OUTPUT)python/perf.so: $(PYTHON_EXT_SRCS) $(PYTHON_EXT_DEPS) $(LIBTRACEEVENT_DYNAMIC_LIST) $(LIBPERF)
+$(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX): $(PYTHON_EXT_SRCS) $(PYTHON_EXT_DEPS) $(LIBPERF)
$(QUIET_GEN)LDSHARED="$(CC) -pthread -shared" \
- CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS) $(LIBTRACEEVENT_DYNAMIC_LIST_LDFLAGS)' \
+ CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' \
$(PYTHON_WORD) util/setup.py \
--quiet build_ext; \
cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/
$(PERF_IN): prepare FORCE
$(Q)$(MAKE) $(build)=perf
-$(PMU_EVENTS_IN): FORCE
+$(PMU_EVENTS_IN): FORCE prepare
$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events
-$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN) $(LIBTRACEEVENT_DYNAMIC_LIST)
- $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) $(LIBTRACEEVENT_DYNAMIC_LIST_LDFLAGS) \
+$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN)
+ $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) \
$(PERF_IN) $(PMU_EVENTS_IN) $(LIBS) -o $@
-$(GTK_IN): FORCE
+$(GTK_IN): FORCE prepare
$(Q)$(MAKE) $(build)=gtk
$(OUTPUT)libperf-gtk.so: $(GTK_IN) $(PERFLIBS)
$(rename_flags_array) \
$(arch_errno_name_array) \
$(sync_file_range_arrays) \
+ $(LIBAPI) \
+ $(LIBBPF) \
+ $(LIBPERF) \
+ $(LIBSUBCMD) \
+ $(LIBSYMBOL) \
bpf-skel
$(OUTPUT)%.o: %.c prepare FORCE
$(patsubst perf-%,%.o,$(PROGRAMS)): $(wildcard */*.h)
-LIBTRACEEVENT_FLAGS += plugin_dir=$(plugindir_SQ) 'EXTRA_CFLAGS=$(EXTRA_CFLAGS)' 'LDFLAGS=$(filter-out -static,$(LDFLAGS))'
-
-$(LIBTRACEEVENT): FORCE
- $(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)libtraceevent.a
-
-libtraceevent_plugins: FORCE
- $(Q)$(MAKE) -C $(TRACE_EVENT_DIR)plugins $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) plugins
-
-$(LIBTRACEEVENT_DYNAMIC_LIST): libtraceevent_plugins
- $(Q)$(MAKE) -C $(TRACE_EVENT_DIR)plugins $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)libtraceevent-dynamic-list
-
-$(LIBTRACEEVENT)-clean:
- $(call QUIET_CLEAN, libtraceevent)
- $(Q)$(MAKE) -C $(TRACE_EVENT_DIR) O=$(OUTPUT) clean >/dev/null
-
-install-traceevent-plugins: libtraceevent_plugins
- $(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) install_plugins
-
-$(LIBAPI): FORCE
- $(Q)$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) $(OUTPUT)libapi.a
+$(LIBAPI): FORCE | $(LIBAPI_OUTPUT)
+ $(Q)$(MAKE) -C $(LIBAPI_DIR) O=$(LIBAPI_OUTPUT) \
+ DESTDIR=$(LIBAPI_DESTDIR) prefix= \
+ $@ install_headers
$(LIBAPI)-clean:
$(call QUIET_CLEAN, libapi)
- $(Q)$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) clean >/dev/null
+ $(Q)$(RM) -r -- $(LIBAPI_OUTPUT)
$(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
$(Q)$(MAKE) -C $(LIBBPF_DIR) FEATURES_DUMP=$(FEATURE_DUMP_EXPORT) \
$(call QUIET_CLEAN, libbpf)
$(Q)$(RM) -r -- $(LIBBPF_OUTPUT)
-$(LIBPERF): FORCE
- $(Q)$(MAKE) -C $(LIBPERF_DIR) EXTRA_CFLAGS="$(LIBPERF_CFLAGS)" O=$(OUTPUT) $(OUTPUT)libperf.a
+$(LIBPERF): FORCE | $(LIBPERF_OUTPUT)
+ $(Q)$(MAKE) -C $(LIBPERF_DIR) O=$(LIBPERF_OUTPUT) \
+ DESTDIR=$(LIBPERF_DESTDIR) prefix= \
+ $@ install_headers
$(LIBPERF)-clean:
$(call QUIET_CLEAN, libperf)
- $(Q)$(MAKE) -C $(LIBPERF_DIR) O=$(OUTPUT) clean >/dev/null
+ $(Q)$(RM) -r -- $(LIBPERF_OUTPUT)
-$(LIBSUBCMD): FORCE
- $(Q)$(MAKE) -C $(SUBCMD_DIR) O=$(OUTPUT) $(OUTPUT)libsubcmd.a
+$(LIBSUBCMD): FORCE | $(LIBSUBCMD_OUTPUT)
+ $(Q)$(MAKE) -C $(LIBSUBCMD_DIR) O=$(LIBSUBCMD_OUTPUT) \
+ DESTDIR=$(LIBSUBCMD_DESTDIR) prefix= \
+ $@ install_headers
$(LIBSUBCMD)-clean:
- $(Q)$(MAKE) -C $(SUBCMD_DIR) O=$(OUTPUT) clean
+ $(call QUIET_CLEAN, libsubcmd)
+ $(Q)$(RM) -r -- $(LIBSUBCMD_OUTPUT)
+
+$(LIBSYMBOL): FORCE | $(LIBSYMBOL_OUTPUT)
+ $(Q)$(MAKE) -C $(LIBSYMBOL_DIR) O=$(LIBSYMBOL_OUTPUT) \
+ DESTDIR=$(LIBSYMBOL_DESTDIR) prefix= \
+ $@ install_headers
+
+$(LIBSYMBOL)-clean:
+ $(call QUIET_CLEAN, libsymbol)
+ $(Q)$(RM) -r -- $(LIBSYMBOL_OUTPUT)
help:
@echo 'Perf make targets:'
$(call QUIET_INSTALL, libexec) \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)'
ifndef NO_LIBBPF
- $(call QUIET_INSTALL, bpf-headers) \
- $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf'; \
- $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf/linux'; \
- $(INSTALL) include/bpf/*.h -m 644 -t '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf'; \
- $(INSTALL) include/bpf/linux/*.h -m 644 -t '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf/linux'
$(call QUIET_INSTALL, bpf-examples) \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perf_examples_instdir_SQ)/bpf'; \
$(INSTALL) examples/bpf/*.c -m 644 -t '$(DESTDIR_SQ)$(perf_examples_instdir_SQ)/bpf'
$(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
$(Q)$(MAKE) -C tests/shell/coresight install-tests
-install-bin: install-tools install-tests install-traceevent-plugins
+install-bin: install-tools install-tests
install: install-bin try-install-man
SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_OUT)/lock_contention.skel.h
SKELETONS += $(SKEL_OUT)/kwork_trace.skel.h
-$(SKEL_TMP_OUT) $(LIBBPF_OUTPUT):
+$(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_OUTPUT) $(LIBSYMBOL_OUTPUT):
$(Q)$(MKDIR) -p $@
ifdef BUILD_BPF_SKEL
bpf-skel-clean:
$(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS)
-clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean
+clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBSYMBOL)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean
$(call QUIET_CLEAN, core-objs) $(RM) $(LIBPERF_A) $(OUTPUT)perf-archive $(OUTPUT)perf-iostat $(LANG_BINDINGS)
$(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
$(Q)$(RM) $(OUTPUT).config-detected
.PHONY: all install clean config-clean strip install-gtk
.PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell
.PHONY: .FORCE-PERF-VERSION-FILE TAGS tags cscope FORCE prepare
-.PHONY: libtraceevent_plugins archheaders
+.PHONY: archheaders
endif # force_fixdep
#include <elfutils/libdwfl.h>
#include "../../../util/unwind-libdw.h"
#include "../../../util/perf_regs.h"
-#include "../../../util/event.h"
+#include "../../../util/sample.h"
bool libdw__arch_set_initial_registers(Dwfl_Thread *thread, void *arg)
{
$gcc -E -dM -x c -I $incpath/include/uapi $input \
|sed -ne 's/^#define __NR_//p' \
- |sort -t' ' -k2 -nu \
+ |sort -t' ' -k2 -n \
|create_table
perf-y += perf_regs.o
perf-y += tsc.o
perf-y += pmu.o
-perf-y += kvm-stat.o
+perf-$(CONFIG_LIBTRACEEVENT) += kvm-stat.o
perf-$(CONFIG_DWARF) += dwarf-regs.o
perf-$(CONFIG_LOCAL_LIBUNWIND) += unwind-libunwind.o
perf-$(CONFIG_LIBDW_DWARF_UNWIND) += unwind-libdw.o
#include "symbol.h"
#include "callchain.h"
#include "record.h"
+#include "util/perf_regs.h"
void arch__add_leaf_frame_record_opts(struct record_opts *opts)
{
// SPDX-License-Identifier: GPL-2.0
+#include <internal/cpumap.h>
#include "../../../util/cpumap.h"
#include "../../../util/pmu.h"
#include <elfutils/libdwfl.h>
#include "../../../util/unwind-libdw.h"
#include "../../../util/perf_regs.h"
-#include "../../../util/event.h"
+#include "../../../util/sample.h"
bool libdw__arch_set_initial_registers(Dwfl_Thread *thread, void *arg)
{
perf-y += header.o
-perf-y += kvm-stat.o
+perf-$(CONFIG_LIBTRACEEVENT) += kvm-stat.o
perf-y += perf_regs.o
perf-y += mem-events.o
perf-y += sym-handling.o
#include "../../../util/tool.h"
#include "../../../util/map.h"
#include "../../../util/debug.h"
+#include "../../../util/sample.h"
void arch_perf_parse_sample_weight(struct perf_sample *data,
const __u64 *array, u64 type)
#include <linux/kernel.h>
#include "../../../util/unwind-libdw.h"
#include "../../../util/perf_regs.h"
-#include "../../../util/event.h"
+#include "../../../util/sample.h"
/* See backends/ppc_initreg.c and backends/ppc_regs.c in elfutils. */
static const int special_regs[3][2] = {
perf-y += perf_regs.o
+perf-y += header.o
perf-$(CONFIG_DWARF) += dwarf-regs.o
perf-$(CONFIG_LIBDW_DWARF_UNWIND) += unwind-libdw.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Implementation of get_cpuid().
+ *
+ * Author: Nikita Shubin <n.shubin@yadro.com>
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <api/fs/fs.h>
+#include <errno.h>
+#include "../../util/debug.h"
+#include "../../util/header.h"
+
+#define CPUINFO_MVEN "mvendorid"
+#define CPUINFO_MARCH "marchid"
+#define CPUINFO_MIMP "mimpid"
+#define CPUINFO "/proc/cpuinfo"
+
+static char *_get_field(const char *line)
+{
+ char *line2, *nl;
+
+ line2 = strrchr(line, ' ');
+ if (!line2)
+ return NULL;
+
+ line2++;
+ nl = strrchr(line, '\n');
+ if (!nl)
+ return NULL;
+
+ return strndup(line2, nl - line2);
+}
+
+static char *_get_cpuid(void)
+{
+ char *line = NULL;
+ char *mvendorid = NULL;
+ char *marchid = NULL;
+ char *mimpid = NULL;
+ char *cpuid = NULL;
+ int read;
+ unsigned long line_sz;
+ FILE *cpuinfo;
+
+ cpuinfo = fopen(CPUINFO, "r");
+ if (cpuinfo == NULL)
+ return cpuid;
+
+ while ((read = getline(&line, &line_sz, cpuinfo)) != -1) {
+ if (!strncmp(line, CPUINFO_MVEN, strlen(CPUINFO_MVEN))) {
+ mvendorid = _get_field(line);
+ if (!mvendorid)
+ goto free;
+ } else if (!strncmp(line, CPUINFO_MARCH, strlen(CPUINFO_MARCH))) {
+ marchid = _get_field(line);
+ if (!marchid)
+ goto free;
+ } else if (!strncmp(line, CPUINFO_MIMP, strlen(CPUINFO_MIMP))) {
+ mimpid = _get_field(line);
+ if (!mimpid)
+ goto free;
+
+ break;
+ }
+ }
+
+ if (!mvendorid || !marchid || !mimpid)
+ goto free;
+
+ if (asprintf(&cpuid, "%s-%s-%s", mvendorid, marchid, mimpid) < 0)
+ cpuid = NULL;
+
+free:
+ fclose(cpuinfo);
+ free(mvendorid);
+ free(marchid);
+ free(mimpid);
+
+ return cpuid;
+}
+
+int get_cpuid(char *buffer, size_t sz)
+{
+ char *cpuid = _get_cpuid();
+ int ret = 0;
+
+ if (sz < strlen(cpuid)) {
+ ret = -EINVAL;
+ goto free;
+ }
+
+ scnprintf(buffer, sz, "%s", cpuid);
+free:
+ free(cpuid);
+ return ret;
+}
+
+char *
+get_cpuid_str(struct perf_pmu *pmu __maybe_unused)
+{
+ return _get_cpuid();
+}
perf-y += header.o
-perf-y += kvm-stat.o
+perf-$(CONFIG_LIBTRACEEVENT) += kvm-stat.o
perf-y += perf_regs.o
perf-$(CONFIG_DWARF) += dwarf-regs.o
#include "../../util/unwind-libdw.h"
#include "../../util/perf_regs.h"
#include "../../util/event.h"
+#include "../../util/sample.h"
#include "dwarf-regs-table.h"
int test__rdpmc(struct test_suite *test, int subtest);
int test__insn_x86(struct test_suite *test, int subtest);
int test__intel_pt_pkt_decoder(struct test_suite *test, int subtest);
+int test__intel_pt_hybrid_compat(struct test_suite *test, int subtest);
int test__bp_modify(struct test_suite *test, int subtest);
int test__x86_sample_parsing(struct test_suite *test, int subtest);
perf-y += arch-tests.o
perf-y += sample-parsing.o
-perf-$(CONFIG_AUXTRACE) += insn-x86.o intel-pt-pkt-decoder-test.o
+perf-$(CONFIG_AUXTRACE) += insn-x86.o intel-pt-test.o
perf-$(CONFIG_X86_64) += bp-modify.o
#ifdef HAVE_AUXTRACE_SUPPORT
DEFINE_SUITE("x86 instruction decoder - new instructions", insn_x86);
-DEFINE_SUITE("Intel PT packet decoder", intel_pt_pkt_decoder);
+
+static struct test_case intel_pt_tests[] = {
+ TEST_CASE("Intel PT packet decoder", intel_pt_pkt_decoder),
+ TEST_CASE("Intel PT hybrid CPU compatibility", intel_pt_hybrid_compat),
+ { .name = NULL, }
+};
+
+struct test_suite suite__intel_pt = {
+ .desc = "Intel PT",
+ .test_cases = intel_pt_tests,
+};
+
#endif
#if defined(__x86_64__)
DEFINE_SUITE("x86 bp modify", bp_modify);
#endif
#ifdef HAVE_AUXTRACE_SUPPORT
&suite__insn_x86,
- &suite__intel_pt_pkt_decoder,
+ &suite__intel_pt,
#endif
#if defined(__x86_64__)
&suite__bp_modify,
// SPDX-License-Identifier: GPL-2.0
+#include <linux/compiler.h>
+#include <linux/bits.h>
#include <string.h>
+#include <cpuid.h>
+#include <sched.h>
#include "intel-pt-decoder/intel-pt-pkt-decoder.h"
#include "debug.h"
#include "tests/tests.h"
#include "arch-tests.h"
+#include "cpumap.h"
/**
* struct test_data - Test data.
return TEST_OK;
}
+
+static int setaffinity(int cpu)
+{
+ cpu_set_t cpu_set;
+
+ CPU_ZERO(&cpu_set);
+ CPU_SET(cpu, &cpu_set);
+ if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
+ pr_debug("sched_setaffinity() failed for CPU %d\n", cpu);
+ return -1;
+ }
+ return 0;
+}
+
+#define INTEL_PT_ADDR_FILT_CNT_MASK GENMASK(2, 0)
+#define INTEL_PT_SUBLEAF_CNT 2
+#define CPUID_REG_CNT 4
+
+struct cpuid_result {
+ union {
+ struct {
+ unsigned int eax;
+ unsigned int ebx;
+ unsigned int ecx;
+ unsigned int edx;
+ };
+ unsigned int reg[CPUID_REG_CNT];
+ };
+};
+
+struct pt_caps {
+ struct cpuid_result subleaf[INTEL_PT_SUBLEAF_CNT];
+};
+
+static int get_pt_caps(int cpu, struct pt_caps *caps)
+{
+ struct cpuid_result r;
+ int i;
+
+ if (setaffinity(cpu))
+ return -1;
+
+ memset(caps, 0, sizeof(*caps));
+
+ for (i = 0; i < INTEL_PT_SUBLEAF_CNT; i++) {
+ __get_cpuid_count(20, i, &r.eax, &r.ebx, &r.ecx, &r.edx);
+ pr_debug("CPU %d CPUID leaf 20 subleaf %d\n", cpu, i);
+ pr_debug("eax = 0x%08x\n", r.eax);
+ pr_debug("ebx = 0x%08x\n", r.ebx);
+ pr_debug("ecx = 0x%08x\n", r.ecx);
+ pr_debug("edx = 0x%08x\n", r.edx);
+ caps->subleaf[i] = r;
+ }
+
+ return 0;
+}
+
+static bool is_hydrid(void)
+{
+ unsigned int eax, ebx, ecx, edx = 0;
+ bool result;
+
+ __get_cpuid_count(7, 0, &eax, &ebx, &ecx, &edx);
+ result = edx & BIT(15);
+ pr_debug("Is %shybrid : CPUID leaf 7 subleaf 0 edx %#x (bit-15 indicates hybrid)\n",
+ result ? "" : "not ", edx);
+ return result;
+}
+
+static int compare_caps(int cpu, struct pt_caps *caps, struct pt_caps *caps0)
+{
+ struct pt_caps mask = { /* Mask of bits to check*/
+ .subleaf = {
+ [0] = {
+ .ebx = GENMASK(8, 0),
+ .ecx = GENMASK(3, 0),
+ },
+ [1] = {
+ .eax = GENMASK(31, 16),
+ .ebx = GENMASK(31, 0),
+ }
+ }
+ };
+ unsigned int m, reg, reg0;
+ int ret = 0;
+ int i, j;
+
+ for (i = 0; i < INTEL_PT_SUBLEAF_CNT; i++) {
+ for (j = 0; j < CPUID_REG_CNT; j++) {
+ m = mask.subleaf[i].reg[j];
+ reg = m & caps->subleaf[i].reg[j];
+ reg0 = m & caps0->subleaf[i].reg[j];
+ if ((reg & reg0) != reg0) {
+ pr_debug("CPU %d subleaf %d reg %d FAIL %#x vs %#x\n",
+ cpu, i, j, reg, reg0);
+ ret = -1;
+ }
+ }
+ }
+
+ m = INTEL_PT_ADDR_FILT_CNT_MASK;
+ reg = m & caps->subleaf[1].eax;
+ reg0 = m & caps0->subleaf[1].eax;
+ if (reg < reg0) {
+ pr_debug("CPU %d subleaf 1 reg 0 FAIL address filter count %#x vs %#x\n",
+ cpu, reg, reg0);
+ ret = -1;
+ }
+
+ if (!ret)
+ pr_debug("CPU %d OK\n", cpu);
+
+ return ret;
+}
+
+int test__intel_pt_hybrid_compat(struct test_suite *test, int subtest)
+{
+ int max_cpu = cpu__max_cpu().cpu;
+ struct pt_caps last_caps;
+ struct pt_caps caps0;
+ int ret = TEST_OK;
+ int cpu;
+
+ if (!is_hydrid()) {
+ test->test_cases[subtest].skip_reason = "not hybrid";
+ return TEST_SKIP;
+ }
+
+ if (get_pt_caps(0, &caps0))
+ return TEST_FAIL;
+
+ for (cpu = 1, last_caps = caps0; cpu < max_cpu; cpu++) {
+ struct pt_caps caps;
+
+ if (get_pt_caps(cpu, &caps)) {
+ pr_debug("CPU %d not found\n", cpu);
+ continue;
+ }
+ if (!memcmp(&caps, &last_caps, sizeof(caps))) {
+ pr_debug("CPU %d same caps as previous CPU\n", cpu);
+ continue;
+ }
+ if (compare_caps(cpu, &caps, &caps0))
+ ret = TEST_FAIL;
+ last_caps = caps;
+ }
+
+ return ret;
+}
#include "event.h"
#include "evsel.h"
#include "debug.h"
+#include "util/sample.h"
#include "util/synthetic-events.h"
#include "tests/tests.h"
perf-y += header.o
perf-y += tsc.o
perf-y += pmu.o
-perf-y += kvm-stat.o
+perf-$(CONFIG_LIBTRACEEVENT) += kvm-stat.o
perf-y += perf_regs.o
perf-y += topdown.o
perf-y += machine.o
#include <linux/types.h>
#include <linux/string.h>
#include <linux/zalloc.h>
+#include <stdlib.h>
#include "../../../util/event.h"
#include "../../../util/synthetic-events.h"
#include "../../../util/tool.h"
#include "../../../util/map.h"
#include "../../../util/debug.h"
+#include "util/sample.h"
#if defined(__x86_64__)
return 0;
}
+#ifdef HAVE_LIBTRACEEVENT
static int intel_pt_track_switches(struct evlist *evlist)
{
const char *sched_switch = "sched:sched_switch";
return 0;
}
+#endif
static void intel_pt_valid_str(char *str, size_t len, u64 valid)
{
ptr->have_sched_switch = 2;
}
} else {
+#ifdef HAVE_LIBTRACEEVENT
err = intel_pt_track_switches(evlist);
if (err == -EPERM)
pr_debug2("Unable to select sched:sched_switch\n");
return err;
else
ptr->have_sched_switch = 1;
+#endif
}
}
void iostat_print_counters(struct evlist *evlist,
struct perf_stat_config *config, struct timespec *ts,
- char *prefix, iostat_print_counter_t print_cnt_cb)
+ char *prefix, iostat_print_counter_t print_cnt_cb, void *arg)
{
void *perf_device = NULL;
struct evsel *counter = evlist__first(evlist);
iostat_prefix(evlist, config, prefix, ts);
fprintf(config->output, "\n%s", prefix);
}
- print_cnt_cb(config, counter, prefix);
+ print_cnt_cb(config, counter, arg);
}
fputc('\n', config->output);
}
#include <linux/types.h>
#include <math.h>
#include <string.h>
+#include <stdlib.h>
#include "../../../util/debug.h"
#include "../../../util/tsc.h"
#include <elfutils/libdwfl.h>
#include "../../../util/unwind-libdw.h"
#include "../../../util/perf_regs.h"
-#include "../../../util/event.h"
+#include "util/sample.h"
bool libdw__arch_set_initial_registers(Dwfl_Thread *thread, void *arg)
{
* The madvise transparent hugepage constants were added in glibc
* 2.13. For compatibility with older versions of glibc, define these
* tokens if they are not already defined.
- *
- * PA-RISC uses different madvise values from other architectures and
- * needs to be special-cased.
*/
-#ifdef __hppa__
-# ifndef MADV_HUGEPAGE
-# define MADV_HUGEPAGE 67
-# endif
-# ifndef MADV_NOHUGEPAGE
-# define MADV_NOHUGEPAGE 68
-# endif
-#else
# ifndef MADV_HUGEPAGE
# define MADV_HUGEPAGE 14
# endif
# ifndef MADV_NOHUGEPAGE
# define MADV_NOHUGEPAGE 15
# endif
-#endif
int bench_numa(int argc, const char **argv);
int bench_sched_messaging(int argc, const char **argv);
#include "util/data.h"
#include "util/stat.h"
#include "util/debug.h"
-#include "util/event.h"
#include "util/symbol.h"
#include "util/session.h"
#include "util/build-id.h"
+#include "util/sample.h"
#include "util/synthetic-events.h"
#define MMAP_DEV_MAJOR 8
#include <sched.h>
#include <stdio.h>
#include <assert.h>
+#include <debug.h>
#include <malloc.h>
#include <signal.h>
#include <stdlib.h>
long bytes_thread;
int nr_tasks;
- bool show_quiet;
bool show_convergence;
bool measure_convergence;
OPT_BOOLEAN('c', "show_convergence", &p0.show_convergence, "show convergence details, "
"convergence is reached when each process (all its threads) is running on a single NUMA node."),
OPT_BOOLEAN('m', "measure_convergence", &p0.measure_convergence, "measure convergence latency"),
- OPT_BOOLEAN('q', "quiet" , &p0.show_quiet, "quiet mode"),
+ OPT_BOOLEAN('q', "quiet" , &quiet,
+ "quiet mode (do not show any warnings or messages)"),
OPT_BOOLEAN('S', "serialize-startup", &p0.serialize_startup,"serialize thread startup"),
/* Special option string parsing callbacks: */
/* char array in count_process_nodes(): */
BUG_ON(g->p.nr_nodes < 0);
- if (g->p.show_quiet && !g->p.show_details)
+ if (quiet && !g->p.show_details)
g->p.show_details = -1;
/* Some memory should be specified: */
if (!name)
name = "main,";
- if (!g->p.show_quiet)
+ if (!quiet)
printf(" %-30s %15.3f, %-15s %s\n", name, val, txt_unit, txt_short);
else
printf(" %14.3f %s\n", val, txt_long);
.namespaces = perf_event__process_namespaces,
.attr = perf_event__process_attr,
.build_id = perf_event__process_build_id,
+#ifdef HAVE_LIBTRACEEVENT
.tracing_data = perf_event__process_tracing_data,
+#endif
.id_index = perf_event__process_id_index,
.auxtrace_info = perf_event__process_auxtrace_info,
.auxtrace = perf_event__process_auxtrace,
OPT_BOOLEAN('f', "force", &data.force, "don't complain, do it"),
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show symbol address, etc)"),
- OPT_BOOLEAN('q', "quiet", &quiet, "do now show any message"),
+ OPT_BOOLEAN('q', "quiet", &quiet, "do now show any warnings or messages"),
OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
"dump raw trace in ASCII"),
#ifdef HAVE_GTK2_SUPPORT
};
static const char * const daemon_usage[] = {
- "perf daemon start [<options>]",
+ "perf daemon {start|signal|stop|ping} [<options>]",
"perf daemon [<options>]",
NULL
};
-static bool done;
+static volatile sig_atomic_t done;
static void sig_handler(int sig __maybe_unused)
{
return bt_convert__perf2json(input_name, to_json, &opts);
if (to_ctf) {
-#ifdef HAVE_LIBBABELTRACE_SUPPORT
+#if defined(HAVE_LIBBABELTRACE_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
return bt_convert__perf2ctf(input_name, to_ctf, &opts);
#else
pr_err("The libbabeltrace support is not compiled in. perf should be "
"compiled with environment variables LIBBABELTRACE=1 and "
- "LIBBABELTRACE_DIR=/path/to/libbabeltrace/\n");
+ "LIBBABELTRACE_DIR=/path/to/libbabeltrace/.\n"
+ "Check also if libbtraceevent devel files are available.\n");
return -1;
#endif
}
static const struct option options[] = {
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show symbol address, etc)"),
- OPT_BOOLEAN('q', "quiet", &quiet, "Do not show any message"),
+ OPT_BOOLEAN('q', "quiet", &quiet, "Do not show any warnings or messages"),
OPT_BOOLEAN('b', "baseline-only", &show_baseline_only,
"Show only items with match in baseline"),
OPT_CALLBACK('c', "compute", &compute,
#define DEFAULT_TRACER "function_graph"
-static volatile int workload_exec_errno;
-static bool done;
+static volatile sig_atomic_t workload_exec_errno;
+static volatile sig_atomic_t done;
static void sig_handler(int sig __maybe_unused)
{
return err;
}
+#ifdef HAVE_LIBTRACEEVENT
static int perf_event__repipe_tracing_data(struct perf_session *session,
union perf_event *event)
{
return perf_event__process_tracing_data(session, event);
}
+#endif
static int dso__read_build_id(struct dso *dso)
{
return 0;
}
+#ifdef HAVE_LIBTRACEEVENT
static int perf_inject__sched_stat(struct perf_tool *tool,
union perf_event *event __maybe_unused,
struct perf_sample *sample,
build_id__mark_dso_hit(tool, event_sw, &sample_sw, evsel, machine);
return perf_event__repipe(tool, event_sw, &sample_sw, machine);
}
+#endif
static struct guest_vcpu *guest_session__vcpu(struct guest_session *gs, u32 vcpu)
{
inject->tool.mmap = perf_event__repipe_mmap;
inject->tool.mmap2 = perf_event__repipe_mmap2;
inject->tool.fork = perf_event__repipe_fork;
+#ifdef HAVE_LIBTRACEEVENT
inject->tool.tracing_data = perf_event__repipe_tracing_data;
+#endif
}
output_data_offset = perf_session__data_offset(session->evlist);
evsel->handler = perf_inject__sched_switch;
} else if (!strcmp(name, "sched:sched_process_exit"))
evsel->handler = perf_inject__sched_process_exit;
+#ifdef HAVE_LIBTRACEEVENT
else if (!strncmp(name, "sched:sched_stat_", 17))
evsel->handler = perf_inject__sched_stat;
+#endif
}
} else if (inject->itrace_synth_opts.vm_time_correlation) {
session->itrace_synth_opts = &inject->itrace_synth_opts;
#include <regex.h>
#include <linux/ctype.h>
+#include <traceevent/event-parse.h>
static int kmem_slab;
static int kmem_page;
return filename;
}
-#ifdef HAVE_KVM_STAT_SUPPORT
+#if defined(HAVE_KVM_STAT_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
void exit_event_get_key(struct evsel *evsel,
struct perf_sample *sample,
pr_info("\nLost events: %" PRIu64 "\n\n", kvm->lost_events);
}
-#ifdef HAVE_TIMERFD_SUPPORT
+#if defined(HAVE_TIMERFD_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
static int process_lost_event(struct perf_tool *tool,
union perf_event *event __maybe_unused,
struct perf_sample *sample __maybe_unused,
return true;
}
-#ifdef HAVE_TIMERFD_SUPPORT
+#if defined(HAVE_TIMERFD_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
/* keeping the max events to a modest level to keep
* the processing of samples per mmap smooth.
*/
return kvm_events_report_vcpu(kvm);
}
-#ifdef HAVE_TIMERFD_SUPPORT
+#if defined(HAVE_TIMERFD_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
static struct evlist *kvm_live_event_list(void)
{
struct evlist *evlist;
if (strlen(argv[1]) > 2 && strstarts("report", argv[1]))
return kvm_events_report(&kvm, argc - 1 , argv + 1);
-#ifdef HAVE_TIMERFD_SUPPORT
+#if defined(HAVE_TIMERFD_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
if (!strncmp(argv[1], "live", 4))
return kvm_events_live(&kvm, argc - 1 , argv + 1);
#endif
return cmd_top(argc, argv);
else if (strlen(argv[0]) > 2 && strstarts("buildid-list", argv[0]))
return __cmd_buildid_list(file_name, argc, argv);
-#ifdef HAVE_KVM_STAT_SUPPORT
+#if defined(HAVE_KVM_STAT_SUPPORT) && defined(HAVE_LIBTRACEEVENT)
else if (strlen(argv[0]) > 2 && strstarts("stat", argv[0]))
return kvm_cmd_stat(file_name, argc, argv);
#endif
*/
#include "builtin.h"
+#include "perf.h"
#include "util/data.h"
+#include "util/evlist.h"
+#include "util/evsel.h"
+#include "util/header.h"
#include "util/kwork.h"
#include "util/debug.h"
+#include "util/session.h"
#include "util/symbol.h"
#include "util/thread.h"
#include "util/string2.h"
#include <subcmd/pager.h>
#include <subcmd/parse-options.h>
+#include <traceevent/event-parse.h>
#include <errno.h>
#include <inttypes.h>
+#include <signal.h>
#include <linux/err.h>
#include <linux/time64.h>
#include <linux/zalloc.h>
#include "util/pmu-hybrid.h"
#include "util/debug.h"
#include "util/metricgroup.h"
+#include "util/string2.h"
+#include "util/strlist.h"
+#include "util/strbuf.h"
#include <subcmd/pager.h>
#include <subcmd/parse-options.h>
+#include <linux/zalloc.h>
+#include <stdarg.h>
#include <stdio.h>
-static bool desc_flag = true;
-static bool details_flag;
-static const char *hybrid_type;
+/**
+ * struct print_state - State and configuration passed to the default_print
+ * functions.
+ */
+struct print_state {
+ /**
+ * @pmu_glob: Optionally restrict PMU and metric matching to PMU or
+ * debugfs subsystem name.
+ */
+ char *pmu_glob;
+ /** @event_glob: Optional pattern matching glob. */
+ char *event_glob;
+ /** @name_only: Print event or metric names only. */
+ bool name_only;
+ /** @desc: Print the event or metric description. */
+ bool desc;
+ /** @long_desc: Print longer event or metric description. */
+ bool long_desc;
+ /** @deprecated: Print deprecated events or metrics. */
+ bool deprecated;
+ /**
+ * @detailed: Print extra information on the perf event such as names
+ * and expressions used internally by events.
+ */
+ bool detailed;
+ /** @metrics: Controls printing of metric and metric groups. */
+ bool metrics;
+ /** @metricgroups: Controls printing of metric and metric groups. */
+ bool metricgroups;
+ /** @last_topic: The last printed event topic. */
+ char *last_topic;
+ /** @last_metricgroups: The last printed metric group. */
+ char *last_metricgroups;
+ /** @visited_metrics: Metrics that are printed to avoid duplicates. */
+ struct strlist *visited_metrics;
+};
+
+static void default_print_start(void *ps)
+{
+ struct print_state *print_state = ps;
+
+ if (!print_state->name_only && pager_in_use())
+ printf("\nList of pre-defined events (to be used in -e or -M):\n\n");
+}
+
+static void default_print_end(void *print_state __maybe_unused) {}
+
+static void wordwrap(const char *s, int start, int max, int corr)
+{
+ int column = start;
+ int n;
+ bool saw_newline = false;
+
+ while (*s) {
+ int wlen = strcspn(s, " \t\n");
+
+ if ((column + wlen >= max && column > start) || saw_newline) {
+ printf("\n%*s", start, "");
+ column = start + corr;
+ }
+ n = printf("%s%.*s", column > start ? " " : "", wlen, s);
+ if (n <= 0)
+ break;
+ saw_newline = s[wlen] == '\n';
+ s += wlen;
+ column += n;
+ s = skip_spaces(s);
+ }
+}
+
+static void default_print_event(void *ps, const char *pmu_name, const char *topic,
+ const char *event_name, const char *event_alias,
+ const char *scale_unit __maybe_unused,
+ bool deprecated, const char *event_type_desc,
+ const char *desc, const char *long_desc,
+ const char *encoding_desc,
+ const char *metric_name, const char *metric_expr)
+{
+ struct print_state *print_state = ps;
+ int pos;
+
+ if (deprecated && !print_state->deprecated)
+ return;
+
+ if (print_state->pmu_glob && pmu_name && !strglobmatch(pmu_name, print_state->pmu_glob))
+ return;
+
+ if (print_state->event_glob &&
+ (!event_name || !strglobmatch(event_name, print_state->event_glob)) &&
+ (!event_alias || !strglobmatch(event_alias, print_state->event_glob)) &&
+ (!topic || !strglobmatch_nocase(topic, print_state->event_glob)))
+ return;
+
+ if (print_state->name_only) {
+ if (event_alias && strlen(event_alias))
+ printf("%s ", event_alias);
+ else
+ printf("%s ", event_name);
+ return;
+ }
+
+ if (strcmp(print_state->last_topic, topic ?: "")) {
+ if (topic)
+ printf("\n%s:\n", topic);
+ free(print_state->last_topic);
+ print_state->last_topic = strdup(topic ?: "");
+ }
+
+ if (event_alias && strlen(event_alias))
+ pos = printf(" %s OR %s", event_name, event_alias);
+ else
+ pos = printf(" %s", event_name);
+
+ if (!topic && event_type_desc) {
+ for (; pos < 53; pos++)
+ putchar(' ');
+ printf("[%s]\n", event_type_desc);
+ } else
+ putchar('\n');
+
+ if (desc && print_state->desc) {
+ printf("%*s", 8, "[");
+ wordwrap(desc, 8, pager_get_columns(), 0);
+ printf("]\n");
+ }
+ long_desc = long_desc ?: desc;
+ if (long_desc && print_state->long_desc) {
+ printf("%*s", 8, "[");
+ wordwrap(long_desc, 8, pager_get_columns(), 0);
+ printf("]\n");
+ }
+
+ if (print_state->detailed && encoding_desc) {
+ printf("%*s", 8, "");
+ wordwrap(encoding_desc, 8, pager_get_columns(), 0);
+ if (metric_name)
+ printf(" MetricName: %s", metric_name);
+ if (metric_expr)
+ printf(" MetricExpr: %s", metric_expr);
+ putchar('\n');
+ }
+}
+
+static void default_print_metric(void *ps,
+ const char *group,
+ const char *name,
+ const char *desc,
+ const char *long_desc,
+ const char *expr,
+ const char *unit __maybe_unused)
+{
+ struct print_state *print_state = ps;
+
+ if (print_state->event_glob &&
+ (!print_state->metrics || !name || !strglobmatch(name, print_state->event_glob)) &&
+ (!print_state->metricgroups || !group || !strglobmatch(group, print_state->event_glob)))
+ return;
+
+ if (!print_state->name_only && !print_state->last_metricgroups) {
+ if (print_state->metricgroups) {
+ printf("\nMetric Groups:\n");
+ if (!print_state->metrics)
+ putchar('\n');
+ } else {
+ printf("\nMetrics:\n\n");
+ }
+ }
+ if (!print_state->last_metricgroups ||
+ strcmp(print_state->last_metricgroups, group ?: "")) {
+ if (group && print_state->metricgroups) {
+ if (print_state->name_only)
+ printf("%s ", group);
+ else if (print_state->metrics)
+ printf("\n%s:\n", group);
+ else
+ printf("%s\n", group);
+ }
+ free(print_state->last_metricgroups);
+ print_state->last_metricgroups = strdup(group ?: "");
+ }
+ if (!print_state->metrics)
+ return;
+
+ if (print_state->name_only) {
+ if (print_state->metrics &&
+ !strlist__has_entry(print_state->visited_metrics, name)) {
+ printf("%s ", name);
+ strlist__add(print_state->visited_metrics, name);
+ }
+ return;
+ }
+ printf(" %s\n", name);
+
+ if (desc && print_state->desc) {
+ printf("%*s", 8, "[");
+ wordwrap(desc, 8, pager_get_columns(), 0);
+ printf("]\n");
+ }
+ if (long_desc && print_state->long_desc) {
+ printf("%*s", 8, "[");
+ wordwrap(long_desc, 8, pager_get_columns(), 0);
+ printf("]\n");
+ }
+ if (expr && print_state->detailed) {
+ printf("%*s", 8, "[");
+ wordwrap(expr, 8, pager_get_columns(), 0);
+ printf("]\n");
+ }
+}
+
+struct json_print_state {
+ /** Should a separator be printed prior to the next item? */
+ bool need_sep;
+};
+
+static void json_print_start(void *print_state __maybe_unused)
+{
+ printf("[\n");
+}
+
+static void json_print_end(void *ps)
+{
+ struct json_print_state *print_state = ps;
+
+ printf("%s]\n", print_state->need_sep ? "\n" : "");
+}
+
+static void fix_escape_printf(struct strbuf *buf, const char *fmt, ...)
+{
+ va_list args;
+
+ va_start(args, fmt);
+ strbuf_setlen(buf, 0);
+ for (size_t fmt_pos = 0; fmt_pos < strlen(fmt); fmt_pos++) {
+ switch (fmt[fmt_pos]) {
+ case '%':
+ fmt_pos++;
+ switch (fmt[fmt_pos]) {
+ case 's': {
+ const char *s = va_arg(args, const char*);
+
+ strbuf_addstr(buf, s);
+ break;
+ }
+ case 'S': {
+ const char *s = va_arg(args, const char*);
+
+ for (size_t s_pos = 0; s_pos < strlen(s); s_pos++) {
+ switch (s[s_pos]) {
+ case '\n':
+ strbuf_addstr(buf, "\\n");
+ break;
+ case '\\':
+ __fallthrough;
+ case '\"':
+ strbuf_addch(buf, '\\');
+ __fallthrough;
+ default:
+ strbuf_addch(buf, s[s_pos]);
+ break;
+ }
+ }
+ break;
+ }
+ default:
+ pr_err("Unexpected format character '%c'\n", fmt[fmt_pos]);
+ strbuf_addch(buf, '%');
+ strbuf_addch(buf, fmt[fmt_pos]);
+ }
+ break;
+ default:
+ strbuf_addch(buf, fmt[fmt_pos]);
+ break;
+ }
+ }
+ va_end(args);
+ fputs(buf->buf, stdout);
+}
+
+static void json_print_event(void *ps, const char *pmu_name, const char *topic,
+ const char *event_name, const char *event_alias,
+ const char *scale_unit,
+ bool deprecated, const char *event_type_desc,
+ const char *desc, const char *long_desc,
+ const char *encoding_desc,
+ const char *metric_name, const char *metric_expr)
+{
+ struct json_print_state *print_state = ps;
+ bool need_sep = false;
+ struct strbuf buf;
+
+ strbuf_init(&buf, 0);
+ printf("%s{\n", print_state->need_sep ? ",\n" : "");
+ print_state->need_sep = true;
+ if (pmu_name) {
+ fix_escape_printf(&buf, "\t\"Unit\": \"%S\"", pmu_name);
+ need_sep = true;
+ }
+ if (topic) {
+ fix_escape_printf(&buf, "%s\t\"Topic\": \"%S\"", need_sep ? ",\n" : "", topic);
+ need_sep = true;
+ }
+ if (event_name) {
+ fix_escape_printf(&buf, "%s\t\"EventName\": \"%S\"", need_sep ? ",\n" : "",
+ event_name);
+ need_sep = true;
+ }
+ if (event_alias && strlen(event_alias)) {
+ fix_escape_printf(&buf, "%s\t\"EventAlias\": \"%S\"", need_sep ? ",\n" : "",
+ event_alias);
+ need_sep = true;
+ }
+ if (scale_unit && strlen(scale_unit)) {
+ fix_escape_printf(&buf, "%s\t\"ScaleUnit\": \"%S\"", need_sep ? ",\n" : "",
+ scale_unit);
+ need_sep = true;
+ }
+ if (event_type_desc) {
+ fix_escape_printf(&buf, "%s\t\"EventType\": \"%S\"", need_sep ? ",\n" : "",
+ event_type_desc);
+ need_sep = true;
+ }
+ if (deprecated) {
+ fix_escape_printf(&buf, "%s\t\"Deprecated\": \"%S\"", need_sep ? ",\n" : "",
+ deprecated ? "1" : "0");
+ need_sep = true;
+ }
+ if (desc) {
+ fix_escape_printf(&buf, "%s\t\"BriefDescription\": \"%S\"", need_sep ? ",\n" : "",
+ desc);
+ need_sep = true;
+ }
+ if (long_desc) {
+ fix_escape_printf(&buf, "%s\t\"PublicDescription\": \"%S\"", need_sep ? ",\n" : "",
+ long_desc);
+ need_sep = true;
+ }
+ if (encoding_desc) {
+ fix_escape_printf(&buf, "%s\t\"Encoding\": \"%S\"", need_sep ? ",\n" : "",
+ encoding_desc);
+ need_sep = true;
+ }
+ if (metric_name) {
+ fix_escape_printf(&buf, "%s\t\"MetricName\": \"%S\"", need_sep ? ",\n" : "",
+ metric_name);
+ need_sep = true;
+ }
+ if (metric_expr) {
+ fix_escape_printf(&buf, "%s\t\"MetricExpr\": \"%S\"", need_sep ? ",\n" : "",
+ metric_expr);
+ need_sep = true;
+ }
+ printf("%s}", need_sep ? "\n" : "");
+ strbuf_release(&buf);
+}
+
+static void json_print_metric(void *ps __maybe_unused, const char *group,
+ const char *name, const char *desc,
+ const char *long_desc, const char *expr,
+ const char *unit)
+{
+ struct json_print_state *print_state = ps;
+ bool need_sep = false;
+ struct strbuf buf;
+
+ strbuf_init(&buf, 0);
+ printf("%s{\n", print_state->need_sep ? ",\n" : "");
+ print_state->need_sep = true;
+ if (group) {
+ fix_escape_printf(&buf, "\t\"MetricGroup\": \"%S\"", group);
+ need_sep = true;
+ }
+ if (name) {
+ fix_escape_printf(&buf, "%s\t\"MetricName\": \"%S\"", need_sep ? ",\n" : "", name);
+ need_sep = true;
+ }
+ if (expr) {
+ fix_escape_printf(&buf, "%s\t\"MetricExpr\": \"%S\"", need_sep ? ",\n" : "", expr);
+ need_sep = true;
+ }
+ if (unit) {
+ fix_escape_printf(&buf, "%s\t\"ScaleUnit\": \"%S\"", need_sep ? ",\n" : "", unit);
+ need_sep = true;
+ }
+ if (desc) {
+ fix_escape_printf(&buf, "%s\t\"BriefDescription\": \"%S\"", need_sep ? ",\n" : "",
+ desc);
+ need_sep = true;
+ }
+ if (long_desc) {
+ fix_escape_printf(&buf, "%s\t\"PublicDescription\": \"%S\"", need_sep ? ",\n" : "",
+ long_desc);
+ need_sep = true;
+ }
+ printf("%s}", need_sep ? "\n" : "");
+ strbuf_release(&buf);
+}
int cmd_list(int argc, const char **argv)
{
int i, ret = 0;
- bool raw_dump = false;
- bool long_desc_flag = false;
- bool deprecated = false;
- char *pmu_name = NULL;
+ struct print_state default_ps = {};
+ struct print_state json_ps = {};
+ void *ps = &default_ps;
+ struct print_callbacks print_cb = {
+ .print_start = default_print_start,
+ .print_end = default_print_end,
+ .print_event = default_print_event,
+ .print_metric = default_print_metric,
+ };
+ const char *hybrid_name = NULL;
+ const char *unit_name = NULL;
+ bool json = false;
struct option list_options[] = {
- OPT_BOOLEAN(0, "raw-dump", &raw_dump, "Dump raw events"),
- OPT_BOOLEAN('d', "desc", &desc_flag,
+ OPT_BOOLEAN(0, "raw-dump", &default_ps.name_only, "Dump raw events"),
+ OPT_BOOLEAN('j', "json", &json, "JSON encode events and metrics"),
+ OPT_BOOLEAN('d', "desc", &default_ps.desc,
"Print extra event descriptions. --no-desc to not print."),
- OPT_BOOLEAN('v', "long-desc", &long_desc_flag,
+ OPT_BOOLEAN('v', "long-desc", &default_ps.long_desc,
"Print longer event descriptions."),
- OPT_BOOLEAN(0, "details", &details_flag,
+ OPT_BOOLEAN(0, "details", &default_ps.detailed,
"Print information on the perf event names and expressions used internally by events."),
- OPT_BOOLEAN(0, "deprecated", &deprecated,
+ OPT_BOOLEAN(0, "deprecated", &default_ps.deprecated,
"Print deprecated events."),
- OPT_STRING(0, "cputype", &hybrid_type, "hybrid cpu type",
- "Print events applying cpu with this type for hybrid platform "
- "(e.g. core or atom)"),
+ OPT_STRING(0, "cputype", &hybrid_name, "hybrid cpu type",
+ "Limit PMU or metric printing to the given hybrid PMU (e.g. core or atom)."),
+ OPT_STRING(0, "unit", &unit_name, "PMU name",
+ "Limit PMU or metric printing to the specified PMU."),
OPT_INCR(0, "debug", &verbose,
"Enable debugging output"),
OPT_END()
};
set_option_flag(list_options, 0, "raw-dump", PARSE_OPT_HIDDEN);
+ /* Hide hybrid flag for the more generic 'unit' flag. */
+ set_option_flag(list_options, 0, "cputype", PARSE_OPT_HIDDEN);
argc = parse_options(argc, argv, list_options, list_usage,
PARSE_OPT_STOP_AT_NON_OPTION);
setup_pager();
- if (!raw_dump && pager_in_use())
- printf("\nList of pre-defined events (to be used in -e or -M):\n\n");
+ if (!default_ps.name_only)
+ setup_pager();
- if (hybrid_type) {
- pmu_name = perf_pmu__hybrid_type_to_pmu(hybrid_type);
- if (!pmu_name)
- pr_warning("WARNING: hybrid cputype is not supported!\n");
+ if (json) {
+ print_cb = (struct print_callbacks){
+ .print_start = json_print_start,
+ .print_end = json_print_end,
+ .print_event = json_print_event,
+ .print_metric = json_print_metric,
+ };
+ ps = &json_ps;
+ } else {
+ default_ps.desc = !default_ps.long_desc;
+ default_ps.last_topic = strdup("");
+ assert(default_ps.last_topic);
+ default_ps.visited_metrics = strlist__new(NULL, NULL);
+ assert(default_ps.visited_metrics);
+ if (unit_name)
+ default_ps.pmu_glob = strdup(unit_name);
+ else if (hybrid_name) {
+ default_ps.pmu_glob = perf_pmu__hybrid_type_to_pmu(hybrid_name);
+ if (!default_ps.pmu_glob)
+ pr_warning("WARNING: hybrid cputype is not supported!\n");
+ }
}
+ print_cb.print_start(ps);
if (argc == 0) {
- print_events(NULL, raw_dump, !desc_flag, long_desc_flag,
- details_flag, deprecated, pmu_name);
+ default_ps.metrics = true;
+ default_ps.metricgroups = true;
+ print_events(&print_cb, ps);
goto out;
}
char *sep, *s;
if (strcmp(argv[i], "tracepoint") == 0)
- print_tracepoint_events(NULL, NULL, raw_dump);
+ print_tracepoint_events(&print_cb, ps);
else if (strcmp(argv[i], "hw") == 0 ||
strcmp(argv[i], "hardware") == 0)
- print_symbol_events(NULL, PERF_TYPE_HARDWARE,
- event_symbols_hw, PERF_COUNT_HW_MAX, raw_dump);
+ print_symbol_events(&print_cb, ps, PERF_TYPE_HARDWARE,
+ event_symbols_hw, PERF_COUNT_HW_MAX);
else if (strcmp(argv[i], "sw") == 0 ||
strcmp(argv[i], "software") == 0) {
- print_symbol_events(NULL, PERF_TYPE_SOFTWARE,
- event_symbols_sw, PERF_COUNT_SW_MAX, raw_dump);
- print_tool_events(NULL, raw_dump);
+ print_symbol_events(&print_cb, ps, PERF_TYPE_SOFTWARE,
+ event_symbols_sw, PERF_COUNT_SW_MAX);
+ print_tool_events(&print_cb, ps);
} else if (strcmp(argv[i], "cache") == 0 ||
strcmp(argv[i], "hwcache") == 0)
- print_hwcache_events(NULL, raw_dump);
+ print_hwcache_events(&print_cb, ps);
else if (strcmp(argv[i], "pmu") == 0)
- print_pmu_events(NULL, raw_dump, !desc_flag,
- long_desc_flag, details_flag,
- deprecated, pmu_name);
+ print_pmu_events(&print_cb, ps);
else if (strcmp(argv[i], "sdt") == 0)
- print_sdt_events(NULL, NULL, raw_dump);
- else if (strcmp(argv[i], "metric") == 0 || strcmp(argv[i], "metrics") == 0)
- metricgroup__print(true, false, NULL, raw_dump, details_flag, pmu_name);
- else if (strcmp(argv[i], "metricgroup") == 0 || strcmp(argv[i], "metricgroups") == 0)
- metricgroup__print(false, true, NULL, raw_dump, details_flag, pmu_name);
- else if ((sep = strchr(argv[i], ':')) != NULL) {
- int sep_idx;
-
- sep_idx = sep - argv[i];
- s = strdup(argv[i]);
- if (s == NULL) {
+ print_sdt_events(&print_cb, ps);
+ else if (strcmp(argv[i], "metric") == 0 || strcmp(argv[i], "metrics") == 0) {
+ default_ps.metricgroups = false;
+ default_ps.metrics = true;
+ metricgroup__print(&print_cb, ps);
+ } else if (strcmp(argv[i], "metricgroup") == 0 ||
+ strcmp(argv[i], "metricgroups") == 0) {
+ default_ps.metricgroups = true;
+ default_ps.metrics = false;
+ metricgroup__print(&print_cb, ps);
+ } else if ((sep = strchr(argv[i], ':')) != NULL) {
+ char *old_pmu_glob = default_ps.pmu_glob;
+
+ default_ps.event_glob = strdup(argv[i]);
+ if (!default_ps.event_glob) {
ret = -1;
goto out;
}
- s[sep_idx] = '\0';
- print_tracepoint_events(s, s + sep_idx + 1, raw_dump);
- print_sdt_events(s, s + sep_idx + 1, raw_dump);
- metricgroup__print(true, true, s, raw_dump, details_flag, pmu_name);
- free(s);
+ print_tracepoint_events(&print_cb, ps);
+ print_sdt_events(&print_cb, ps);
+ default_ps.metrics = true;
+ default_ps.metricgroups = true;
+ metricgroup__print(&print_cb, ps);
+ zfree(&default_ps.event_glob);
+ default_ps.pmu_glob = old_pmu_glob;
} else {
if (asprintf(&s, "*%s*", argv[i]) < 0) {
printf("Critical: Not enough memory! Trying to continue...\n");
continue;
}
- print_symbol_events(s, PERF_TYPE_HARDWARE,
- event_symbols_hw, PERF_COUNT_HW_MAX, raw_dump);
- print_symbol_events(s, PERF_TYPE_SOFTWARE,
- event_symbols_sw, PERF_COUNT_SW_MAX, raw_dump);
- print_tool_events(s, raw_dump);
- print_hwcache_events(s, raw_dump);
- print_pmu_events(s, raw_dump, !desc_flag,
- long_desc_flag,
- details_flag,
- deprecated,
- pmu_name);
- print_tracepoint_events(NULL, s, raw_dump);
- print_sdt_events(NULL, s, raw_dump);
- metricgroup__print(true, true, s, raw_dump, details_flag, pmu_name);
+ default_ps.event_glob = s;
+ print_symbol_events(&print_cb, ps, PERF_TYPE_HARDWARE,
+ event_symbols_hw, PERF_COUNT_HW_MAX);
+ print_symbol_events(&print_cb, ps, PERF_TYPE_SOFTWARE,
+ event_symbols_sw, PERF_COUNT_SW_MAX);
+ print_tool_events(&print_cb, ps);
+ print_hwcache_events(&print_cb, ps);
+ print_pmu_events(&print_cb, ps);
+ print_tracepoint_events(&print_cb, ps);
+ print_sdt_events(&print_cb, ps);
+ default_ps.metrics = true;
+ default_ps.metricgroups = true;
+ metricgroup__print(&print_cb, ps);
free(s);
}
}
out:
- free(pmu_name);
+ print_cb.print_end(ps);
+ free(default_ps.pmu_glob);
+ free(default_ps.last_topic);
+ free(default_ps.last_metricgroups);
+ strlist__delete(default_ps.visited_metrics);
return ret;
}
#include "util/target.h"
#include "util/callchain.h"
#include "util/lock-contention.h"
+#include "util/bpf_skel/lock_data.h"
#include <subcmd/pager.h>
#include <subcmd/parse-options.h>
#include "util/data.h"
#include "util/string2.h"
#include "util/map.h"
+#include "util/util.h"
#include <sys/types.h>
#include <sys/prctl.h>
static bool combine_locks;
static bool show_thread_stats;
+static bool show_lock_addrs;
static bool use_bpf;
static unsigned long bpf_map_entries = 10240;
static int max_stack_depth = CONTENTION_STACK_DEPTH;
static int stack_skip = CONTENTION_STACK_SKIP;
static int print_nr_entries = INT_MAX / 2;
-static enum {
- LOCK_AGGR_ADDR,
- LOCK_AGGR_TASK,
- LOCK_AGGR_CALLER,
-} aggr_mode = LOCK_AGGR_ADDR;
-
-static u64 sched_text_start;
-static u64 sched_text_end;
-static u64 lock_text_start;
-static u64 lock_text_end;
+static enum lock_aggr_mode aggr_mode = LOCK_AGGR_ADDR;
static struct thread_stat *thread_stat_find(u32 tid)
{
return 0;
}
-bool is_lock_function(struct machine *machine, u64 addr)
-{
- if (!sched_text_start) {
- struct map *kmap;
- struct symbol *sym;
-
- sym = machine__find_kernel_symbol_by_name(machine,
- "__sched_text_start",
- &kmap);
- if (!sym) {
- /* to avoid retry */
- sched_text_start = 1;
- return false;
- }
-
- sched_text_start = kmap->unmap_ip(kmap, sym->start);
-
- /* should not fail from here */
- sym = machine__find_kernel_symbol_by_name(machine,
- "__sched_text_end",
- &kmap);
- sched_text_end = kmap->unmap_ip(kmap, sym->start);
-
- sym = machine__find_kernel_symbol_by_name(machine,
- "__lock_text_start",
- &kmap);
- lock_text_start = kmap->unmap_ip(kmap, sym->start);
-
- sym = machine__find_kernel_symbol_by_name(machine,
- "__lock_text_end",
- &kmap);
- lock_text_end = kmap->unmap_ip(kmap, sym->start);
- }
-
- /* failed to get kernel symbols */
- if (sched_text_start == 1)
- return false;
-
- /* mutex and rwsem functions are in sched text section */
- if (sched_text_start <= addr && addr < sched_text_end)
- return true;
-
- /* spinlock functions are in lock text section */
- if (lock_text_start <= addr && addr < lock_text_end)
- return true;
-
- return false;
-}
-
static int get_symbol_name_offset(struct map *map, struct symbol *sym, u64 ip,
char *buf, int size)
{
goto next;
sym = node->ms.sym;
- if (sym && !is_lock_function(machine, node->ip)) {
+ if (sym && !machine__is_lock_function(machine, node->ip)) {
get_symbol_name_offset(node->ms.map, sym, node->ip,
buf, size);
return 0;
if (++skip <= stack_skip)
goto next;
- if (node->ms.sym && is_lock_function(machine, node->ip))
+ if (node->ms.sym && machine__is_lock_function(machine, node->ip))
goto next;
hash ^= hash_long((unsigned long)node->ip, 64);
ls = lock_stat_find(key);
if (!ls) {
char buf[128];
- const char *caller = buf;
+ const char *name = "";
unsigned int flags = evsel__intval(evsel, sample, "flags");
+ struct machine *machine = &session->machines.host;
+ struct map *kmap;
+ struct symbol *sym;
+
+ switch (aggr_mode) {
+ case LOCK_AGGR_ADDR:
+ /* make sure it loads the kernel map to find lock symbols */
+ map__load(machine__kernel_map(machine));
- if (lock_contention_caller(evsel, sample, buf, sizeof(buf)) < 0)
- caller = "Unknown";
+ sym = machine__find_kernel_symbol(machine, key, &kmap);
+ if (sym)
+ name = sym->name;
+ break;
+ case LOCK_AGGR_CALLER:
+ name = buf;
+ if (lock_contention_caller(evsel, sample, buf, sizeof(buf)) < 0)
+ name = "Unknown";
+ break;
+ case LOCK_AGGR_TASK:
+ default:
+ break;
+ }
- ls = lock_stat_findnew(key, caller, flags);
+ ls = lock_stat_findnew(key, name, flags);
if (!ls)
return -ENOMEM;
return rc;
}
+static const struct evsel_str_handler lock_tracepoints[] = {
+ { "lock:lock_acquire", evsel__process_lock_acquire, }, /* CONFIG_LOCKDEP */
+ { "lock:lock_acquired", evsel__process_lock_acquired, }, /* CONFIG_LOCKDEP, CONFIG_LOCK_STAT */
+ { "lock:lock_contended", evsel__process_lock_contended, }, /* CONFIG_LOCKDEP, CONFIG_LOCK_STAT */
+ { "lock:lock_release", evsel__process_lock_release, }, /* CONFIG_LOCKDEP */
+};
+
+static const struct evsel_str_handler contention_tracepoints[] = {
+ { "lock:contention_begin", evsel__process_contention_begin, },
+ { "lock:contention_end", evsel__process_contention_end, },
+};
+
+static int process_event_update(struct perf_tool *tool,
+ union perf_event *event,
+ struct evlist **pevlist)
+{
+ int ret;
+
+ ret = perf_event__process_event_update(tool, event, pevlist);
+ if (ret < 0)
+ return ret;
+
+ /* this can return -EEXIST since we call it for each evsel */
+ perf_session__set_tracepoints_handlers(session, lock_tracepoints);
+ perf_session__set_tracepoints_handlers(session, contention_tracepoints);
+ return 0;
+}
+
typedef int (*tracepoint_handler)(struct evsel *evsel,
struct perf_sample *sample);
list_for_each_entry(key, &lock_keys, list)
pr_info("%*s ", key->len, key->header);
- if (show_thread_stats)
+ switch (aggr_mode) {
+ case LOCK_AGGR_TASK:
pr_info(" %10s %s\n\n", "pid", "comm");
- else
+ break;
+ case LOCK_AGGR_CALLER:
pr_info(" %10s %s\n\n", "type", "caller");
+ break;
+ case LOCK_AGGR_ADDR:
+ pr_info(" %16s %s\n\n", "address", "symbol");
+ break;
+ default:
+ break;
+ }
}
bad = total = printed = 0;
bad = bad_hist[BROKEN_CONTENDED];
while ((st = pop_from_result())) {
+ struct thread *t;
+ int pid;
+
total += use_bpf ? st->nr_contended : 1;
if (st->broken)
bad++;
pr_info(" ");
}
- if (show_thread_stats) {
- struct thread *t;
- int pid = st->addr;
-
- /* st->addr contains tid of thread */
+ switch (aggr_mode) {
+ case LOCK_AGGR_CALLER:
+ pr_info(" %10s %s\n", get_type_str(st), st->name);
+ break;
+ case LOCK_AGGR_TASK:
+ pid = st->addr;
t = perf_session__findnew(session, pid);
pr_info(" %10d %s\n", pid, thread__comm_str(t));
- goto next;
+ break;
+ case LOCK_AGGR_ADDR:
+ pr_info(" %016llx %s\n", (unsigned long long)st->addr,
+ st->name ? : "");
+ break;
+ default:
+ break;
}
- pr_info(" %10s %s\n", get_type_str(st), st->name);
- if (verbose) {
+ if (aggr_mode == LOCK_AGGR_CALLER && verbose) {
struct map *kmap;
struct symbol *sym;
char buf[128];
}
}
-next:
if (++printed >= print_nr_entries)
break;
}
print_bad_events(bad, total);
}
-static const struct evsel_str_handler lock_tracepoints[] = {
- { "lock:lock_acquire", evsel__process_lock_acquire, }, /* CONFIG_LOCKDEP */
- { "lock:lock_acquired", evsel__process_lock_acquired, }, /* CONFIG_LOCKDEP, CONFIG_LOCK_STAT */
- { "lock:lock_contended", evsel__process_lock_contended, }, /* CONFIG_LOCKDEP, CONFIG_LOCK_STAT */
- { "lock:lock_release", evsel__process_lock_release, }, /* CONFIG_LOCKDEP */
-};
-
-static const struct evsel_str_handler contention_tracepoints[] = {
- { "lock:contention_begin", evsel__process_contention_begin, },
- { "lock:contention_end", evsel__process_contention_end, },
-};
-
static bool force;
static int __cmd_report(bool display_info)
{
int err = -EINVAL;
struct perf_tool eops = {
+ .attr = perf_event__process_attr,
+ .event_update = process_event_update,
.sample = process_sample_event,
.comm = perf_event__process_comm,
.mmap = perf_event__process_mmap,
.namespaces = perf_event__process_namespaces,
+ .tracing_data = perf_event__process_tracing_data,
.ordered_events = true,
};
struct perf_data data = {
symbol_conf.sort_by_name = true;
symbol__init(&session->header.env);
- if (!perf_session__has_traces(session, "lock record"))
- goto out_delete;
+ if (!data.is_pipe) {
+ if (!perf_session__has_traces(session, "lock record"))
+ goto out_delete;
- if (perf_session__set_tracepoints_handlers(session, lock_tracepoints)) {
- pr_err("Initializing perf session tracepoint handlers failed\n");
- goto out_delete;
- }
+ if (perf_session__set_tracepoints_handlers(session, lock_tracepoints)) {
+ pr_err("Initializing perf session tracepoint handlers failed\n");
+ goto out_delete;
+ }
- if (perf_session__set_tracepoints_handlers(session, contention_tracepoints)) {
- pr_err("Initializing perf session tracepoint handlers failed\n");
- goto out_delete;
+ if (perf_session__set_tracepoints_handlers(session, contention_tracepoints)) {
+ pr_err("Initializing perf session tracepoint handlers failed\n");
+ goto out_delete;
+ }
}
if (setup_output_field(false, output_fields))
{
int err = -EINVAL;
struct perf_tool eops = {
+ .attr = perf_event__process_attr,
+ .event_update = process_event_update,
.sample = process_sample_event,
.comm = perf_event__process_comm,
.mmap = perf_event__process_mmap,
+ .tracing_data = perf_event__process_tracing_data,
.ordered_events = true,
};
struct perf_data data = {
con.machine = &session->machines.host;
+ con.aggr_mode = aggr_mode = show_thread_stats ? LOCK_AGGR_TASK :
+ show_lock_addrs ? LOCK_AGGR_ADDR : LOCK_AGGR_CALLER;
+
/* for lock function check */
symbol_conf.sort_by_name = true;
symbol__init(&session->header.env);
pr_err("lock contention BPF setup failed\n");
goto out_delete;
}
- } else {
+ } else if (!data.is_pipe) {
if (!perf_session__has_traces(session, "lock record"))
goto out_delete;
if (select_key(true))
goto out_delete;
- if (show_thread_stats)
- aggr_mode = LOCK_AGGR_TASK;
- else
- aggr_mode = LOCK_AGGR_CALLER;
-
if (use_bpf) {
lock_contention_start();
if (argc)
return 0;
}
+static int parse_max_stack(const struct option *opt, const char *str,
+ int unset __maybe_unused)
+{
+ unsigned long *len = (unsigned long *)opt->value;
+ long val;
+ char *endptr;
+
+ errno = 0;
+ val = strtol(str, &endptr, 0);
+ if (*endptr != '\0' || errno != 0) {
+ pr_err("invalid max stack depth: %s\n", str);
+ return -1;
+ }
+
+ if (val < 0 || val > sysctl__max_stack()) {
+ pr_err("invalid max stack depth: %ld\n", val);
+ return -1;
+ }
+
+ *len = val;
+ return 0;
+}
+
int cmd_lock(int argc, const char **argv)
{
const struct option lock_options[] = {
"file", "vmlinux pathname"),
OPT_STRING(0, "kallsyms", &symbol_conf.kallsyms_name,
"file", "kallsyms pathname"),
- OPT_BOOLEAN('q', "quiet", &quiet, "Do not show any message"),
+ OPT_BOOLEAN('q', "quiet", &quiet, "Do not show any warnings or messages"),
OPT_END()
};
"Trace on existing thread id (exclusive to --pid)"),
OPT_CALLBACK(0, "map-nr-entries", &bpf_map_entries, "num",
"Max number of BPF map entries", parse_map_entry),
- OPT_INTEGER(0, "max-stack", &max_stack_depth,
- "Set the maximum stack depth when collecting lock contention, "
- "Default: " __stringify(CONTENTION_STACK_DEPTH)),
+ OPT_CALLBACK(0, "max-stack", &max_stack_depth, "num",
+ "Set the maximum stack depth when collecting lopck contention, "
+ "Default: " __stringify(CONTENTION_STACK_DEPTH), parse_max_stack),
OPT_INTEGER(0, "stack-skip", &stack_skip,
"Set the number of stack depth to skip when finding a lock caller, "
"Default: " __stringify(CONTENTION_STACK_SKIP)),
OPT_INTEGER('E', "entries", &print_nr_entries, "display this many functions"),
+ OPT_BOOLEAN('l', "lock-addr", &show_lock_addrs, "show lock stats by address"),
OPT_PARENT(lock_options)
};
argc = parse_options(argc, argv, contention_options,
contention_usage, 0);
}
+
+ if (show_thread_stats && show_lock_addrs) {
+ pr_err("Cannot use thread and addr mode together\n");
+ parse_options_usage(contention_usage, contention_options,
+ "threads", 0);
+ parse_options_usage(NULL, contention_options,
+ "lock-addr", 0);
+ return -1;
+ }
+
rc = __cmd_contention(argc, argv);
} else {
usage_with_options(lock_usage, lock_options);
#include "util/symbol.h"
#include "util/pmu.h"
#include "util/pmu-hybrid.h"
+#include "util/sample.h"
#include "util/string2.h"
#include <linux/err.h>
int command; /* Command short_name */
bool list_events;
bool uprobes;
- bool quiet;
bool target_used;
int nevents;
struct perf_probe_event events[MAX_PROBES];
struct option options[] = {
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show parsed arguments, etc)"),
- OPT_BOOLEAN('q', "quiet", ¶ms.quiet,
- "be quiet (do not show any messages)"),
+ OPT_BOOLEAN('q', "quiet", &quiet,
+ "be quiet (do not show any warnings or messages)"),
OPT_CALLBACK_DEFAULT('l', "list", NULL, "[GROUP:]EVENT",
"list up probe events",
opt_set_filter_with_command, DEFAULT_LIST_FILTER),
if (ret)
return ret;
- if (params.quiet) {
+ if (quiet) {
if (verbose != 0) {
pr_err(" Error: -v and -q are exclusive.\n");
return -EINVAL;
return record__write(rec, map, bf, size);
}
-static volatile int signr = -1;
-static volatile int child_finished;
+static volatile sig_atomic_t signr = -1;
+static volatile sig_atomic_t child_finished;
#ifdef HAVE_EVENTFD_SUPPORT
-static volatile int done_fd = -1;
+static volatile sig_atomic_t done_fd = -1;
#endif
static void sig_handler(int sig)
if (rec->no_buildid)
perf_header__clear_feat(&session->header, HEADER_BUILD_ID);
+#ifdef HAVE_LIBTRACEEVENT
if (!have_tracepoints(&rec->evlist->core.entries))
perf_header__clear_feat(&session->header, HEADER_TRACING_DATA);
+#endif
if (!rec->opts.branch_stack)
perf_header__clear_feat(&session->header, HEADER_BRANCH_STACK);
}
-static volatile int workload_exec_errno;
+static volatile sig_atomic_t workload_exec_errno;
/*
* evlist__prepare_workload will send a SIGUSR1
OPT_CALLBACK(0, "mmap-flush", &record.opts, "number",
"Minimal number of bytes that is extracted from mmap data pages (default: 1)",
record__mmap_flush_parse),
- OPT_BOOLEAN(0, "group", &record.opts.group,
- "put the counters into a counter group"),
OPT_CALLBACK_NOOPT('g', NULL, &callchain_param,
NULL, "enables call-graph recording" ,
&record_callchain_opt),
&record_parse_callchain_opt),
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show counter open errors, etc)"),
- OPT_BOOLEAN('q', "quiet", &quiet, "don't print any message"),
+ OPT_BOOLEAN('q', "quiet", &quiet, "don't print any warnings or messages"),
OPT_BOOLEAN('s', "stat", &record.opts.inherit_stat,
"per thread counts"),
OPT_BOOLEAN('d', "data", &record.opts.sample_address, "Record the sample addresses"),
#include <unistd.h>
#include <linux/mman.h>
+#ifdef HAVE_LIBTRACEEVENT
+#include <traceevent/event-parse.h>
+#endif
+
struct report {
struct perf_tool tool;
struct perf_session *session;
.lost = perf_event__process_lost,
.read = process_read_event,
.attr = process_attr,
+#ifdef HAVE_LIBTRACEEVENT
.tracing_data = perf_event__process_tracing_data,
+#endif
.build_id = perf_event__process_build_id,
.id_index = perf_event__process_id_index,
.auxtrace_info = perf_event__process_auxtrace_info,
"input file name"),
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show symbol address, etc)"),
- OPT_BOOLEAN('q', "quiet", &quiet, "Do not show any message"),
+ OPT_BOOLEAN('q', "quiet", &quiet, "Do not show any warnings or messages"),
OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
"dump raw trace in ASCII"),
OPT_BOOLEAN(0, "stats", &report.stats_mode, "Display event stats"),
report.range_num);
}
+#ifdef HAVE_LIBTRACEEVENT
if (session->tevent.pevent &&
tep_set_function_resolver(session->tevent.pevent,
machine__resolve_kernel_addr,
__func__);
return -1;
}
-
+#endif
sort__setup_elide(stdout);
ret = __cmd_report(&report);
#include "perf.h"
#include <linux/ctype.h>
+#ifdef HAVE_LIBTRACEEVENT
+#include <traceevent/event-parse.h>
+#endif
static char const *script_name;
static char const *generate_script_lang;
u64 val;
if (!evsel->stats)
- evlist__alloc_stats(script->session->evlist, false);
+ evlist__alloc_stats(&stat_config, script->session->evlist, /*alloc_raw=*/false);
if (evsel_script(leader)->gnum++ == 0)
perf_stat__reset_shadow_stats();
val = sample->period * evsel->scale;
perf_sample__fprintf_bts(sample, evsel, thread, al, addr_al, machine, fp);
return;
}
-
+#ifdef HAVE_LIBTRACEEVENT
if (PRINT_FIELD(TRACE) && sample->raw_data) {
event_format__fprintf(evsel->tp_format, sample->cpu,
sample->raw_data, sample->raw_size, fp);
}
-
+#endif
if (attr->type == PERF_TYPE_SYNTH && PRINT_FIELD(SYNTH))
perf_sample__fprintf_synth(sample, evsel, fp);
static void setup_scripting(void)
{
+#ifdef HAVE_LIBTRACEEVENT
setup_perl_scripting();
setup_python_scripting();
+#endif
}
static int flush_scripting(void)
perf_evlist__set_maps(&evlist->core, script->cpus, script->threads);
- if (evlist__alloc_stats(evlist, true))
+ if (evlist__alloc_stats(&stat_config, evlist, /*alloc_raw=*/true))
return -ENOMEM;
script->allocated = true;
.fork = perf_event__process_fork,
.attr = process_attr,
.event_update = perf_event__process_event_update,
+#ifdef HAVE_LIBTRACEEVENT
.tracing_data = perf_event__process_tracing_data,
+#endif
.feature = process_feature_event,
.build_id = perf_event__process_build_id,
.id_index = perf_event__process_id_index,
else
symbol_conf.use_callchain = false;
+#ifdef HAVE_LIBTRACEEVENT
if (session->tevent.pevent &&
tep_set_function_resolver(session->tevent.pevent,
machine__resolve_kernel_addr,
err = -1;
goto out_delete;
}
-
+#endif
if (generate_script_lang) {
struct stat perf_stat;
int input;
err = -ENOENT;
goto out_delete;
}
-
+#ifdef HAVE_LIBTRACEEVENT
err = scripting_ops->generate_script(session->tevent.pevent,
"perf-script");
+#else
+ err = scripting_ops->generate_script(NULL, "perf-script");
+#endif
goto out_delete;
}
#include <linux/ctype.h>
#include <perf/evlist.h>
+#include <internal/threadmap.h>
#define DEFAULT_SEPARATOR " "
#define FREEZE_ON_SMI_PATH "devices/cpu/freeze_on_smi"
#define METRIC_ONLY_LEN 20
-static volatile pid_t child_pid = -1;
+static volatile sig_atomic_t child_pid = -1;
static int detailed_run = 0;
static bool transaction_run;
static bool topdown_run = false;
static bool smi_cost = false;
static bool smi_reset = false;
static int big_num_opt = -1;
-static bool group = false;
static const char *pre_cmd = NULL;
static const char *post_cmd = NULL;
static bool sync_run = false;
static struct perf_stat perf_stat;
#define STAT_RECORD perf_stat.record
-static volatile int done = 0;
+static volatile sig_atomic_t done = 0;
static struct perf_stat_config stat_config = {
.aggr_mode = AGGR_GLOBAL,
return 0;
}
-static void read_counters(struct timespec *rs)
+static int read_counters(struct timespec *rs)
{
- struct evsel *counter;
-
if (!stat_config.stop_read_counter) {
if (read_bpf_map_counters() ||
read_affinity_counters(rs))
- return;
+ return -1;
}
+ return 0;
+}
+
+static void process_counters(void)
+{
+ struct evsel *counter;
evlist__for_each_entry(evsel_list, counter) {
if (counter->err)
pr_warning("failed to process counter %s\n", counter->name);
counter->err = 0;
}
+
+ perf_stat_merge_counters(&stat_config, evsel_list);
+ perf_stat_process_percore(&stat_config, evsel_list);
+ perf_stat_process_shadow_stats(&stat_config, evsel_list);
}
static void process_interval(void)
diff_timespec(&rs, &ts, &ref_time);
perf_stat__reset_shadow_per_stat(&rt_stat);
- read_counters(&rs);
+ evlist__reset_aggr_stats(evsel_list);
+
+ if (read_counters(&rs) == 0)
+ process_counters();
if (STAT_RECORD) {
if (WRITE_STAT_ROUND_EVENT(rs.tv_sec * NSEC_PER_SEC + rs.tv_nsec, INTERVAL))
return err;
}
- if (stat_config.initial_delay < 0) {
- pr_info(EVLIST_DISABLED_MSG);
- return 0;
- }
-
- if (stat_config.initial_delay > 0) {
- pr_info(EVLIST_DISABLED_MSG);
- usleep(stat_config.initial_delay * USEC_PER_MSEC);
- }
-
/*
* We need to enable counters only if:
* - we don't have tracee (attaching to task or cpu)
* - we have initial delay configured
*/
- if (!target__none(&target) || stat_config.initial_delay) {
+ if (!target__none(&target)) {
if (!all_counters_use_bpf)
evlist__enable(evsel_list);
- if (stat_config.initial_delay > 0)
- pr_info(EVLIST_ENABLED_MSG);
}
return 0;
}
}
}
-static volatile int workload_exec_errno;
+static volatile sig_atomic_t workload_exec_errno;
/*
* evlist__prepare_workload will send a SIGUSR1
child_pid = evsel_list->workload.pid;
}
- if (group)
- evlist__set_leader(evsel_list);
-
if (!cpu_map__is_dummy(evsel_list->core.user_requested_cpus)) {
if (affinity__setup(&saved_affinity) < 0)
return -1;
return err;
}
- err = enable_counters();
- if (err)
- return -1;
+ if (stat_config.initial_delay) {
+ pr_info(EVLIST_DISABLED_MSG);
+ } else {
+ err = enable_counters();
+ if (err)
+ return -1;
+ }
/* Exec the command, if any */
if (forks)
evlist__start_workload(evsel_list);
+ if (stat_config.initial_delay > 0) {
+ usleep(stat_config.initial_delay * USEC_PER_MSEC);
+ err = enable_counters();
+ if (err)
+ return -1;
+
+ pr_info(EVLIST_ENABLED_MSG);
+ }
+
t0 = rdclock();
clock_gettime(CLOCK_MONOTONIC, &ref_time);
init_stats(&walltime_nsecs_stats);
update_stats(&walltime_nsecs_stats, t1 - t0);
- if (stat_config.aggr_mode == AGGR_GLOBAL)
- evlist__save_aggr_prev_raw_counts(evsel_list);
-
evlist__copy_prev_raw_counts(evsel_list);
evlist__reset_prev_raw_counts(evsel_list);
+ evlist__reset_aggr_stats(evsel_list);
perf_stat__reset_shadow_per_stat(&rt_stat);
} else {
update_stats(&walltime_nsecs_stats, t1 - t0);
* avoid arbitrary skew, we must read all counters before closing any
* group leaders.
*/
- read_counters(&(struct timespec) { .tv_nsec = t1-t0 });
+ if (read_counters(&(struct timespec) { .tv_nsec = t1-t0 }) == 0)
+ process_counters();
/*
* We need to keep evsel_list alive, because it's processed
/* Do not print anything if we record to the pipe. */
if (STAT_RECORD && perf_stat.data.is_pipe)
return;
- if (stat_config.quiet)
+ if (quiet)
return;
evlist__print_counters(evsel_list, &stat_config, &target, ts, argc, argv);
}
-static volatile int signr = -1;
+static volatile sig_atomic_t signr = -1;
static void skip_signal(int signo)
{
#endif
OPT_BOOLEAN('a', "all-cpus", &target.system_wide,
"system-wide collection from all CPUs"),
- OPT_BOOLEAN('g', "group", &group,
- "put the counters into a counter group"),
OPT_BOOLEAN(0, "scale", &stat_config.scale,
"Use --no-scale to disable counter scaling for multiplexing"),
OPT_INCR('v', "verbose", &verbose,
"print summary for interval mode"),
OPT_BOOLEAN(0, "no-csv-summary", &stat_config.no_csv_summary,
"don't print 'summary' for CSV summary output"),
- OPT_BOOLEAN(0, "quiet", &stat_config.quiet,
- "don't print output (useful with record)"),
+ OPT_BOOLEAN(0, "quiet", &quiet,
+ "don't print any output, messages or warnings (useful with record)"),
OPT_CALLBACK(0, "cputype", &evsel_list, "hybrid cpu type",
"Only enable events on applying cpu with this type "
"for hybrid platform (e.g. core or atom)",
return aggr_cpu_id__node(cpu, /*data=*/NULL);
}
+static struct aggr_cpu_id perf_stat__get_global(struct perf_stat_config *config __maybe_unused,
+ struct perf_cpu cpu)
+{
+ return aggr_cpu_id__global(cpu, /*data=*/NULL);
+}
+
+static struct aggr_cpu_id perf_stat__get_cpu(struct perf_stat_config *config __maybe_unused,
+ struct perf_cpu cpu)
+{
+ return aggr_cpu_id__cpu(cpu, /*data=*/NULL);
+}
+
static struct aggr_cpu_id perf_stat__get_aggr(struct perf_stat_config *config,
aggr_get_id_t get_id, struct perf_cpu cpu)
{
- struct aggr_cpu_id id = aggr_cpu_id__empty();
+ struct aggr_cpu_id id;
+
+ /* per-process mode - should use global aggr mode */
+ if (cpu.cpu == -1)
+ return get_id(config, cpu);
if (aggr_cpu_id__is_empty(&config->cpus_aggr_map->map[cpu.cpu]))
config->cpus_aggr_map->map[cpu.cpu] = get_id(config, cpu);
return perf_stat__get_aggr(config, perf_stat__get_node, cpu);
}
-static bool term_percore_set(void)
+static struct aggr_cpu_id perf_stat__get_global_cached(struct perf_stat_config *config,
+ struct perf_cpu cpu)
{
- struct evsel *counter;
-
- evlist__for_each_entry(evsel_list, counter) {
- if (counter->percore)
- return true;
- }
+ return perf_stat__get_aggr(config, perf_stat__get_global, cpu);
+}
- return false;
+static struct aggr_cpu_id perf_stat__get_cpu_cached(struct perf_stat_config *config,
+ struct perf_cpu cpu)
+{
+ return perf_stat__get_aggr(config, perf_stat__get_cpu, cpu);
}
static aggr_cpu_id_get_t aggr_mode__get_aggr(enum aggr_mode aggr_mode)
case AGGR_NODE:
return aggr_cpu_id__node;
case AGGR_NONE:
- if (term_percore_set())
- return aggr_cpu_id__core;
-
- return NULL;
+ return aggr_cpu_id__cpu;
case AGGR_GLOBAL:
+ return aggr_cpu_id__global;
case AGGR_THREAD:
case AGGR_UNSET:
case AGGR_MAX:
case AGGR_NODE:
return perf_stat__get_node_cached;
case AGGR_NONE:
- if (term_percore_set()) {
- return perf_stat__get_core_cached;
- }
- return NULL;
+ return perf_stat__get_cpu_cached;
case AGGR_GLOBAL:
+ return perf_stat__get_global_cached;
case AGGR_THREAD:
case AGGR_UNSET:
case AGGR_MAX:
aggr_cpu_id_get_t get_id = aggr_mode__get_aggr(stat_config.aggr_mode);
if (get_id) {
+ bool needs_sort = stat_config.aggr_mode != AGGR_NONE;
stat_config.aggr_map = cpu_aggr_map__new(evsel_list->core.user_requested_cpus,
- get_id, /*data=*/NULL);
+ get_id, /*data=*/NULL, needs_sort);
if (!stat_config.aggr_map) {
pr_err("cannot build %s map", aggr_mode__string[stat_config.aggr_mode]);
return -1;
stat_config.aggr_get_id = aggr_mode__get_id(stat_config.aggr_mode);
}
+ if (stat_config.aggr_mode == AGGR_THREAD) {
+ nr = perf_thread_map__nr(evsel_list->core.threads);
+ stat_config.aggr_map = cpu_aggr_map__empty_new(nr);
+ if (stat_config.aggr_map == NULL)
+ return -ENOMEM;
+
+ for (int s = 0; s < nr; s++) {
+ struct aggr_cpu_id id = aggr_cpu_id__empty();
+
+ id.thread_idx = s;
+ stat_config.aggr_map->map[s] = id;
+ }
+ return 0;
+ }
+
/*
* The evsel_list->cpus is the base we operate on,
* taking the highest cpu number to be the size of
return id;
}
+static struct aggr_cpu_id perf_env__get_cpu_aggr_by_cpu(struct perf_cpu cpu, void *data)
+{
+ struct perf_env *env = data;
+ struct aggr_cpu_id id = aggr_cpu_id__empty();
+
+ if (cpu.cpu != -1) {
+ /*
+ * core_id is relative to socket and die,
+ * we need a global id. So we set
+ * socket, die id and core id
+ */
+ id.socket = env->cpu[cpu.cpu].socket_id;
+ id.die = env->cpu[cpu.cpu].die_id;
+ id.core = env->cpu[cpu.cpu].core_id;
+ id.cpu = cpu;
+ }
+
+ return id;
+}
+
static struct aggr_cpu_id perf_env__get_node_aggr_by_cpu(struct perf_cpu cpu, void *data)
{
struct aggr_cpu_id id = aggr_cpu_id__empty();
return id;
}
+static struct aggr_cpu_id perf_env__get_global_aggr_by_cpu(struct perf_cpu cpu __maybe_unused,
+ void *data __maybe_unused)
+{
+ struct aggr_cpu_id id = aggr_cpu_id__empty();
+
+ /* it always aggregates to the cpu 0 */
+ id.cpu = (struct perf_cpu){ .cpu = 0 };
+ return id;
+}
+
static struct aggr_cpu_id perf_stat__get_socket_file(struct perf_stat_config *config __maybe_unused,
struct perf_cpu cpu)
{
return perf_env__get_core_aggr_by_cpu(cpu, &perf_stat.session->header.env);
}
+static struct aggr_cpu_id perf_stat__get_cpu_file(struct perf_stat_config *config __maybe_unused,
+ struct perf_cpu cpu)
+{
+ return perf_env__get_cpu_aggr_by_cpu(cpu, &perf_stat.session->header.env);
+}
+
static struct aggr_cpu_id perf_stat__get_node_file(struct perf_stat_config *config __maybe_unused,
struct perf_cpu cpu)
{
return perf_env__get_node_aggr_by_cpu(cpu, &perf_stat.session->header.env);
}
+static struct aggr_cpu_id perf_stat__get_global_file(struct perf_stat_config *config __maybe_unused,
+ struct perf_cpu cpu)
+{
+ return perf_env__get_global_aggr_by_cpu(cpu, &perf_stat.session->header.env);
+}
+
static aggr_cpu_id_get_t aggr_mode__get_aggr_file(enum aggr_mode aggr_mode)
{
switch (aggr_mode) {
return perf_env__get_core_aggr_by_cpu;
case AGGR_NODE:
return perf_env__get_node_aggr_by_cpu;
- case AGGR_NONE:
case AGGR_GLOBAL:
+ return perf_env__get_global_aggr_by_cpu;
+ case AGGR_NONE:
+ return perf_env__get_cpu_aggr_by_cpu;
case AGGR_THREAD:
case AGGR_UNSET:
case AGGR_MAX:
return perf_stat__get_core_file;
case AGGR_NODE:
return perf_stat__get_node_file;
- case AGGR_NONE:
case AGGR_GLOBAL:
+ return perf_stat__get_global_file;
+ case AGGR_NONE:
+ return perf_stat__get_cpu_file;
case AGGR_THREAD:
case AGGR_UNSET:
case AGGR_MAX:
{
struct perf_env *env = &st->session->header.env;
aggr_cpu_id_get_t get_id = aggr_mode__get_aggr_file(stat_config.aggr_mode);
+ bool needs_sort = stat_config.aggr_mode != AGGR_NONE;
+
+ if (stat_config.aggr_mode == AGGR_THREAD) {
+ int nr = perf_thread_map__nr(evsel_list->core.threads);
+
+ stat_config.aggr_map = cpu_aggr_map__empty_new(nr);
+ if (stat_config.aggr_map == NULL)
+ return -ENOMEM;
+
+ for (int s = 0; s < nr; s++) {
+ struct aggr_cpu_id id = aggr_cpu_id__empty();
+
+ id.thread_idx = s;
+ stat_config.aggr_map->map[s] = id;
+ }
+ return 0;
+ }
if (!get_id)
return 0;
- stat_config.aggr_map = cpu_aggr_map__new(evsel_list->core.user_requested_cpus, get_id, env);
+ stat_config.aggr_map = cpu_aggr_map__new(evsel_list->core.user_requested_cpus,
+ get_id, env, needs_sort);
if (!stat_config.aggr_map) {
pr_err("cannot build %s map", aggr_mode__string[stat_config.aggr_mode]);
return -1;
union perf_event *event)
{
struct perf_record_stat_round *stat_round = &event->stat_round;
- struct evsel *counter;
struct timespec tsh, *ts = NULL;
const char **argv = session->header.env.cmdline_argv;
int argc = session->header.env.nr_cmdline;
- evlist__for_each_entry(evsel_list, counter)
- perf_stat_process_counter(&stat_config, counter);
+ process_counters();
if (stat_round->type == PERF_STAT_ROUND_TYPE__FINAL)
update_stats(&walltime_nsecs_stats, stat_round->time);
if (perf_cpu_map__empty(st->cpus)) {
if (st->aggr_mode != AGGR_UNSET)
pr_warning("warning: processing task data, aggregation mode not set\n");
- return 0;
- }
-
- if (st->aggr_mode != AGGR_UNSET)
+ } else if (st->aggr_mode != AGGR_UNSET) {
stat_config.aggr_mode = st->aggr_mode;
+ }
if (perf_stat.data.is_pipe)
perf_stat_init_aggr_mode();
else
perf_stat_init_aggr_mode_file(st);
+ if (stat_config.aggr_map) {
+ int nr_aggr = stat_config.aggr_map->nr;
+
+ if (evlist__alloc_aggr_stats(session->evlist, nr_aggr) < 0) {
+ pr_err("cannot allocate aggr counts\n");
+ return -1;
+ }
+ }
return 0;
}
perf_evlist__set_maps(&evsel_list->core, st->cpus, st->threads);
- if (evlist__alloc_stats(evsel_list, true))
+ if (evlist__alloc_stats(&stat_config, evsel_list, /*alloc_raw=*/true))
return -ENOMEM;
st->maps_allocated = true;
goto out;
}
- if (!output && !stat_config.quiet) {
+ if (!output && !quiet) {
struct timespec tm;
mode = append_file ? "a" : "w";
}
}
+ if (stat_config.interval_clear && !isatty(fileno(output))) {
+ fprintf(stderr, "--interval-clear does not work with output\n");
+ parse_options_usage(stat_usage, stat_options, "o", 1);
+ parse_options_usage(NULL, stat_options, "log-fd", 0);
+ parse_options_usage(NULL, stat_options, "interval-clear", 0);
+ return -1;
+ }
+
stat_config.output = output;
/*
goto out;
}
- if (evlist__alloc_stats(evsel_list, interval))
+ if (perf_stat_init_aggr_mode())
goto out;
- if (perf_stat_init_aggr_mode())
+ if (evlist__alloc_stats(&stat_config, evsel_list, interval))
goto out;
/*
#include "util/string2.h"
#include "util/tracepoint.h"
#include <linux/err.h>
+#include <traceevent/event-parse.h>
#ifdef LACKS_OPEN_MEMSTREAM_PROTOTYPE
FILE *open_memstream(char **ptr, size_t *sizeloc);
#include <linux/ctype.h>
#include <perf/mmap.h>
-static volatile int done;
-static volatile int resize;
+static volatile sig_atomic_t done;
+static volatile sig_atomic_t resize;
#define HEADER_LINE_NR 5
"dump the symbol table used for profiling"),
OPT_INTEGER('f', "count-filter", &top.count_filter,
"only display functions with more events than this"),
- OPT_BOOLEAN(0, "group", &opts->group,
- "put the counters into a counter group"),
OPT_BOOLEAN('i', "no-inherit", &opts->no_inherit,
"child tasks do not inherit counters"),
OPT_STRING(0, "sym-annotate", &top.sym_filter, "symbol name",
*/
#include "util/record.h"
-#include <traceevent/event-parse.h>
#include <api/fs/tracing_path.h>
#include <bpf/bpf.h>
#include "util/bpf_map.h"
#include <linux/ctype.h>
#include <perf/mmap.h>
+#ifdef HAVE_LIBTRACEEVENT
+#include <traceevent/event-parse.h>
+#endif
+
#ifndef O_CLOEXEC
# define O_CLOEXEC 02000000
#endif
# define F_LINUX_SPECIFIC_BASE 1024
#endif
+#define RAW_SYSCALL_ARGS_NUM 6
+
/*
* strtoul: Go from a string to a value, i.e. for msr: MSR_FS_BASE to 0xc0000100
*/
const char *sys_enter,
*sys_exit;
} bpf_prog_name;
- struct syscall_arg_fmt arg[6];
+ struct syscall_arg_fmt arg[RAW_SYSCALL_ARGS_NUM];
u8 nr_args;
bool errpid;
bool timeout;
struct syscalltbl *sctbl;
struct {
struct syscall *table;
- struct bpf_map *map;
struct { // per syscall BPF_MAP_TYPE_PROG_ARRAY
struct bpf_map *sys_enter,
*sys_exit;
.arg = { [0] = { .scnprintf = SCA_PTR, /* brk */ }, }, },
{ .name = "clock_gettime",
.arg = { [0] = STRARRAY(clk_id, clockid), }, },
+ { .name = "clock_nanosleep",
+ .arg = { [2] = { .scnprintf = SCA_TIMESPEC, /* rqtp */ }, }, },
{ .name = "clone", .errpid = true, .nr_args = 5,
.arg = { [0] = { .name = "flags", .scnprintf = SCA_CLONE_FLAGS, },
[1] = { .name = "child_stack", .scnprintf = SCA_HEX, },
.arg = { [0] = { .scnprintf = SCA_FDAT, /* dfd */ },
[2] = { .scnprintf = SCA_OPEN_FLAGS, /* flags */ }, }, },
{ .name = "perf_event_open",
- .arg = { [2] = { .scnprintf = SCA_INT, /* cpu */ },
+ .arg = { [0] = { .scnprintf = SCA_PERF_ATTR, /* attr */ },
+ [2] = { .scnprintf = SCA_INT, /* cpu */ },
[3] = { .scnprintf = SCA_FD, /* group_fd */ },
[4] = { .scnprintf = SCA_PERF_FLAGS, /* flags */ }, }, },
{ .name = "pipe2",
};
/*
- * Must match what is in the BPF program:
- *
- * tools/perf/examples/bpf/augmented_raw_syscalls.c
- */
-struct bpf_map_syscall_entry {
- bool enabled;
- u16 string_args_len[6];
-};
-
-/*
* We need to have this 'calculated' boolean because in some cases we really
* don't know what is the duration of a syscall, for instance, when we start
* a session and some threads are waiting for a syscall to finish, say 'poll',
}
static pid_t workload_pid = -1;
-static bool done = false;
-static bool interrupted = false;
+static volatile sig_atomic_t done = false;
+static volatile sig_atomic_t interrupted = false;
static void sighandler_interrupt(int sig __maybe_unused)
{
{
int idx;
- if (nr_args == 6 && sc->fmt && sc->fmt->nr_args != 0)
+ if (nr_args == RAW_SYSCALL_ARGS_NUM && sc->fmt && sc->fmt->nr_args != 0)
nr_args = sc->fmt->nr_args;
sc->arg_fmt = calloc(nr_args, sizeof(*sc->arg_fmt));
len >= 2 && strcmp(field->name + len - 2, "fd") == 0) {
/*
* /sys/kernel/tracing/events/syscalls/sys_enter*
- * egrep 'field:.*fd;' .../format|sed -r 's/.*field:([a-z ]+) [a-z_]*fd.+/\1/g'|sort|uniq -c
+ * grep -E 'field:.*fd;' .../format|sed -r 's/.*field:([a-z ]+) [a-z_]*fd.+/\1/g'|sort|uniq -c
* 65 int
* 23 unsigned int
* 7 unsigned long
#endif
sc = trace->syscalls.table + id;
if (sc->nonexistent)
- return 0;
+ return -EEXIST;
if (name == NULL) {
sc->nonexistent = true;
- return 0;
+ return -EEXIST;
}
sc->name = name;
sc->tp_format = trace_event__tp_format("syscalls", tp_name);
}
- if (syscall__alloc_arg_fmts(sc, IS_ERR(sc->tp_format) ? 6 : sc->tp_format->format.nr_fields))
- return -ENOMEM;
-
- if (IS_ERR(sc->tp_format))
+ /*
+ * Fails to read trace point format via sysfs node, so the trace point
+ * doesn't exist. Set the 'nonexistent' flag as true.
+ */
+ if (IS_ERR(sc->tp_format)) {
+ sc->nonexistent = true;
return PTR_ERR(sc->tp_format);
+ }
+
+ if (syscall__alloc_arg_fmts(sc, IS_ERR(sc->tp_format) ?
+ RAW_SYSCALL_ARGS_NUM : sc->tp_format->format.nr_fields))
+ return -ENOMEM;
sc->args = sc->tp_format->format.fields;
/*
(err = trace__read_syscall_info(trace, id)) != 0)
goto out_cant_read;
- if (trace->syscalls.table[id].name == NULL) {
- if (trace->syscalls.table[id].nonexistent)
- return NULL;
+ if (trace->syscalls.table && trace->syscalls.table[id].nonexistent)
goto out_cant_read;
- }
return &trace->syscalls.table[id];
offset = format_field__intval(field, sample, evsel->needs_swap);
syscall_arg.len = offset >> 16;
offset &= 0xffff;
+#ifdef HAVE_LIBTRACEEVENT_TEP_FIELD_IS_RELATIVE
if (field->flags & TEP_FIELD_IS_RELATIVE)
offset += field->offset + field->size;
+#endif
}
val = (uintptr_t)(sample->raw_data + offset);
static void trace__set_bpf_map_syscalls(struct trace *trace)
{
- trace->syscalls.map = trace__find_bpf_map_by_name(trace, "syscalls");
trace->syscalls.prog_array.sys_enter = trace__find_bpf_map_by_name(trace, "syscalls_sys_enter");
trace->syscalls.prog_array.sys_exit = trace__find_bpf_map_by_name(trace, "syscalls_sys_exit");
}
return sc ? bpf_program__fd(sc->bpf_prog.sys_exit) : bpf_program__fd(trace->syscalls.unaugmented_prog);
}
-static void trace__init_bpf_map_syscall_args(struct trace *trace, int id, struct bpf_map_syscall_entry *entry)
-{
- struct syscall *sc = trace__syscall_info(trace, NULL, id);
- int arg = 0;
-
- if (sc == NULL)
- goto out;
-
- for (; arg < sc->nr_args; ++arg) {
- entry->string_args_len[arg] = 0;
- if (sc->arg_fmt[arg].scnprintf == SCA_FILENAME) {
- /* Should be set like strace -s strsize */
- entry->string_args_len[arg] = PATH_MAX;
- }
- }
-out:
- for (; arg < 6; ++arg)
- entry->string_args_len[arg] = 0;
-}
-static int trace__set_ev_qualifier_bpf_filter(struct trace *trace)
-{
- int fd = bpf_map__fd(trace->syscalls.map);
- struct bpf_map_syscall_entry value = {
- .enabled = !trace->not_ev_qualifier,
- };
- int err = 0;
- size_t i;
-
- for (i = 0; i < trace->ev_qualifier_ids.nr; ++i) {
- int key = trace->ev_qualifier_ids.entries[i];
-
- if (value.enabled) {
- trace__init_bpf_map_syscall_args(trace, key, &value);
- trace__init_syscall_bpf_progs(trace, key);
- }
-
- err = bpf_map_update_elem(fd, &key, &value, BPF_EXIST);
- if (err)
- break;
- }
-
- return err;
-}
-
-static int __trace__init_syscalls_bpf_map(struct trace *trace, bool enabled)
-{
- int fd = bpf_map__fd(trace->syscalls.map);
- struct bpf_map_syscall_entry value = {
- .enabled = enabled,
- };
- int err = 0, key;
-
- for (key = 0; key < trace->sctbl->syscalls.nr_entries; ++key) {
- if (enabled)
- trace__init_bpf_map_syscall_args(trace, key, &value);
-
- err = bpf_map_update_elem(fd, &key, &value, BPF_ANY);
- if (err)
- break;
- }
-
- return err;
-}
-
-static int trace__init_syscalls_bpf_map(struct trace *trace)
-{
- bool enabled = true;
-
- if (trace->ev_qualifier_ids.nr)
- enabled = trace->not_ev_qualifier;
-
- return __trace__init_syscalls_bpf_map(trace, enabled);
-}
-
static struct bpf_program *trace__find_usable_bpf_prog_entry(struct trace *trace, struct syscall *sc)
{
struct tep_format_field *field, *candidate_field;
{
}
-static int trace__set_ev_qualifier_bpf_filter(struct trace *trace __maybe_unused)
-{
- return 0;
-}
-
-static int trace__init_syscalls_bpf_map(struct trace *trace __maybe_unused)
-{
- return 0;
-}
-
static struct bpf_program *trace__find_bpf_program_by_title(struct trace *trace __maybe_unused,
const char *name __maybe_unused)
{
static int trace__set_ev_qualifier_filter(struct trace *trace)
{
- if (trace->syscalls.map)
- return trace__set_ev_qualifier_bpf_filter(trace);
if (trace->syscalls.events.sys_enter)
return trace__set_ev_qualifier_tp_filter(trace);
return 0;
if (err < 0)
goto out_error_mem;
- if (trace->syscalls.map)
- trace__init_syscalls_bpf_map(trace);
-
if (trace->syscalls.prog_array.sys_enter)
trace__init_syscalls_bpf_prog_array_maps(trace);
}
trace->multiple_threads = perf_thread_map__pid(evlist->core.threads, 0) == -1 ||
- evlist->core.threads->nr > 1 ||
- evlist__first(evlist)->core.attr.inherit;
+ perf_thread_map__nr(evlist->core.threads) > 1 ||
+ evlist__first(evlist)->core.attr.inherit;
/*
* Now that we already used evsel->core.attr to ask the kernel to setup the
STATUS(HAVE_AIO_SUPPORT, aio);
STATUS(HAVE_ZSTD_SUPPORT, zstd);
STATUS(HAVE_LIBPFM, libpfm4);
+ STATUS(HAVE_LIBTRACEEVENT, libtraceevent);
}
int cmd_version(int argc, const char **argv)
Copyright (C) 2018 Red Hat, Inc., Arnaldo Carvalho de Melo <acme@redhat.com>
*/
-#include <bpf.h>
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
#define NSEC_PER_SEC 1000000000L
-int probe(hrtimer_nanosleep, rqtp)(void *ctx, int err, long long sec)
+SEC("hrtimer_nanosleep=hrtimer_nanosleep rqtp")
+int hrtimer_nanosleep(void *ctx, int err, long long sec)
{
return sec / NSEC_PER_SEC == 5ULL;
}
-license(GPL);
+char _license[] SEC("license") = "GPL";
* code that will combine entry/exit in a strace like way.
*/
-#include <unistd.h>
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
#include <linux/limits.h>
-#include <linux/socket.h>
-#include <pid_filter.h>
-/* bpf-output associated map */
-bpf_map(__augmented_syscalls__, PERF_EVENT_ARRAY, int, u32, __NR_CPUS__);
+// FIXME: These should come from system headers
+typedef char bool;
+typedef int pid_t;
+typedef long long int __s64;
+typedef __s64 time64_t;
-/*
- * string_args_len: one per syscall arg, 0 means not a string or don't copy it,
- * PATH_MAX for copying everything, any other value to limit
- * it a la 'strace -s strsize'.
- */
-struct syscall {
- bool enabled;
- u16 string_args_len[6];
+struct timespec64 {
+ time64_t tv_sec;
+ long int tv_nsec;
};
-bpf_map(syscalls, ARRAY, int, struct syscall, 512);
+/* bpf-output associated map */
+struct __augmented_syscalls__ {
+ __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);
+ __type(key, int);
+ __type(value, __u32);
+ __uint(max_entries, __NR_CPUS__);
+} __augmented_syscalls__ SEC(".maps");
/*
* What to augment at entry?
*
* Pointer arg payloads (filenames, etc) passed from userspace to the kernel
*/
-bpf_map(syscalls_sys_enter, PROG_ARRAY, u32, u32, 512);
+struct syscalls_sys_enter {
+ __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
+ __type(key, __u32);
+ __type(value, __u32);
+ __uint(max_entries, 512);
+} syscalls_sys_enter SEC(".maps");
/*
* What to augment at exit?
*
* Pointer arg payloads returned from the kernel (struct stat, etc) to userspace.
*/
-bpf_map(syscalls_sys_exit, PROG_ARRAY, u32, u32, 512);
+struct syscalls_sys_exit {
+ __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
+ __type(key, __u32);
+ __type(value, __u32);
+ __uint(max_entries, 512);
+} syscalls_sys_exit SEC(".maps");
struct syscall_enter_args {
unsigned long long common_tp_fields;
char value[PATH_MAX];
};
-pid_filter(pids_filtered);
+struct pids_filtered {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, pid_t);
+ __type(value, bool);
+ __uint(max_entries, 64);
+} pids_filtered SEC(".maps");
+
+/*
+ * Desired design of maximum size and alignment (see RFC2553)
+ */
+#define SS_MAXSIZE 128 /* Implementation specific max size */
+
+typedef unsigned short sa_family_t;
+
+/*
+ * FIXME: Should come from system headers
+ *
+ * The definition uses anonymous union and struct in order to control the
+ * default alignment.
+ */
+struct sockaddr_storage {
+ union {
+ struct {
+ sa_family_t ss_family; /* address family */
+ /* Following field(s) are implementation specific */
+ char __data[SS_MAXSIZE - sizeof(unsigned short)];
+ /* space to achieve desired size, */
+ /* _SS_MAXSIZE value minus size of ss_family */
+ };
+ void *__align; /* implementation specific desired alignment */
+ };
+};
struct augmented_args_payload {
struct syscall_enter_args args;
struct augmented_arg arg, arg2;
};
struct sockaddr_storage saddr;
+ char __data[sizeof(struct augmented_arg)];
};
};
// We need more tmp space than the BPF stack can give us
-bpf_map(augmented_args_tmp, PERCPU_ARRAY, int, struct augmented_args_payload, 1);
+struct augmented_args_tmp {
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __type(key, int);
+ __type(value, struct augmented_args_payload);
+ __uint(max_entries, 1);
+} augmented_args_tmp SEC(".maps");
static inline struct augmented_args_payload *augmented_args_payload(void)
{
static inline int augmented__output(void *ctx, struct augmented_args_payload *args, int len)
{
/* If perf_event_output fails, return non-zero so that it gets recorded unaugmented */
- return perf_event_output(ctx, &__augmented_syscalls__, BPF_F_CURRENT_CPU, args, len);
+ return bpf_perf_event_output(ctx, &__augmented_syscalls__, BPF_F_CURRENT_CPU, args, len);
}
static inline
unsigned int augmented_arg__read_str(struct augmented_arg *augmented_arg, const void *arg, unsigned int arg_len)
{
unsigned int augmented_len = sizeof(*augmented_arg);
- int string_len = probe_read_str(&augmented_arg->value, arg_len, arg);
+ int string_len = bpf_probe_read_str(&augmented_arg->value, arg_len, arg);
augmented_arg->size = augmented_arg->err = 0;
/*
if (socklen > sizeof(augmented_args->saddr))
socklen = sizeof(augmented_args->saddr);
- probe_read(&augmented_args->saddr, socklen, sockaddr_arg);
+ bpf_probe_read(&augmented_args->saddr, socklen, sockaddr_arg);
return augmented__output(args, augmented_args, len + socklen);
}
if (socklen > sizeof(augmented_args->saddr))
socklen = sizeof(augmented_args->saddr);
- probe_read(&augmented_args->saddr, socklen, sockaddr_arg);
+ bpf_probe_read(&augmented_args->saddr, socklen, sockaddr_arg);
return augmented__output(args, augmented_args, len + socklen);
}
return augmented__output(args, augmented_args, len);
}
+#define PERF_ATTR_SIZE_VER0 64 /* sizeof first published struct */
+
+// we need just the start, get the size to then copy it
+struct perf_event_attr_size {
+ __u32 type;
+ /*
+ * Size of the attr structure, for fwd/bwd compat.
+ */
+ __u32 size;
+};
+
+SEC("!syscalls:sys_enter_perf_event_open")
+int sys_enter_perf_event_open(struct syscall_enter_args *args)
+{
+ struct augmented_args_payload *augmented_args = augmented_args_payload();
+ const struct perf_event_attr_size *attr = (const struct perf_event_attr_size *)args->args[0], *attr_read;
+ unsigned int len = sizeof(augmented_args->args);
+
+ if (augmented_args == NULL)
+ goto failure;
+
+ if (bpf_probe_read(&augmented_args->__data, sizeof(*attr), attr) < 0)
+ goto failure;
+
+ attr_read = (const struct perf_event_attr_size *)augmented_args->__data;
+
+ __u32 size = attr_read->size;
+
+ if (!size)
+ size = PERF_ATTR_SIZE_VER0;
+
+ if (size > sizeof(augmented_args->__data))
+ goto failure;
+
+ // Now that we read attr->size and tested it against the size limits, read it completely
+ if (bpf_probe_read(&augmented_args->__data, size, attr) < 0)
+ goto failure;
+
+ return augmented__output(args, augmented_args, len + size);
+failure:
+ return 1; /* Failure: don't filter */
+}
+
+SEC("!syscalls:sys_enter_clock_nanosleep")
+int sys_enter_clock_nanosleep(struct syscall_enter_args *args)
+{
+ struct augmented_args_payload *augmented_args = augmented_args_payload();
+ const void *rqtp_arg = (const void *)args->args[2];
+ unsigned int len = sizeof(augmented_args->args);
+ __u32 size = sizeof(struct timespec64);
+
+ if (augmented_args == NULL)
+ goto failure;
+
+ if (size > sizeof(augmented_args->__data))
+ goto failure;
+
+ bpf_probe_read(&augmented_args->__data, size, rqtp_arg);
+
+ return augmented__output(args, augmented_args, len + size);
+failure:
+ return 1; /* Failure: don't filter */
+}
+
+static pid_t getpid(void)
+{
+ return bpf_get_current_pid_tgid();
+}
+
+static bool pid_filter__has(struct pids_filtered *pids, pid_t pid)
+{
+ return bpf_map_lookup_elem(pids, &pid) != NULL;
+}
+
SEC("raw_syscalls:sys_enter")
int sys_enter(struct syscall_enter_args *args)
{
* initial, non-augmented raw_syscalls:sys_enter payload.
*/
unsigned int len = sizeof(augmented_args->args);
- struct syscall *syscall;
if (pid_filter__has(&pids_filtered, getpid()))
return 0;
if (augmented_args == NULL)
return 1;
- probe_read(&augmented_args->args, sizeof(augmented_args->args), args);
+ bpf_probe_read(&augmented_args->args, sizeof(augmented_args->args), args);
/*
* Jump to syscall specific augmenter, even if the default one,
if (pid_filter__has(&pids_filtered, getpid()))
return 0;
- probe_read(&exit_args, sizeof(exit_args), args);
+ bpf_probe_read(&exit_args, sizeof(exit_args), args);
/*
* Jump to syscall specific return augmenter, even if the default one,
* "!raw_syscalls:unaugmented" that will just return 1 to return the
return 0;
}
-license(GPL);
+char _license[] SEC("license") = "GPL";
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Augment syscalls with the contents of the pointer arguments.
- *
- * Test it with:
- *
- * perf trace -e tools/perf/examples/bpf/augmented_syscalls.c cat /etc/passwd > /dev/null
- *
- * It'll catch some openat syscalls related to the dynamic linked and
- * the last one should be the one for '/etc/passwd'.
- *
- * This matches what is marshalled into the raw_syscall:sys_enter payload
- * expected by the 'perf trace' beautifiers, and can be used by them, that will
- * check if perf_sample->raw_data is more than what is expected for each
- * syscalls:sys_{enter,exit}_SYSCALL tracepoint, uing the extra data as the
- * contents of pointer arguments.
- */
-
-#include <stdio.h>
-#include <linux/socket.h>
-
-/* bpf-output associated map */
-bpf_map(__augmented_syscalls__, PERF_EVENT_ARRAY, int, u32, __NR_CPUS__);
-
-struct syscall_exit_args {
- unsigned long long common_tp_fields;
- long syscall_nr;
- long ret;
-};
-
-struct augmented_filename {
- unsigned int size;
- int reserved;
- char value[256];
-};
-
-#define augmented_filename_syscall(syscall) \
-struct augmented_enter_##syscall##_args { \
- struct syscall_enter_##syscall##_args args; \
- struct augmented_filename filename; \
-}; \
-int syscall_enter(syscall)(struct syscall_enter_##syscall##_args *args) \
-{ \
- struct augmented_enter_##syscall##_args augmented_args = { .filename.reserved = 0, }; \
- unsigned int len = sizeof(augmented_args); \
- probe_read(&augmented_args.args, sizeof(augmented_args.args), args); \
- augmented_args.filename.size = probe_read_str(&augmented_args.filename.value, \
- sizeof(augmented_args.filename.value), \
- args->filename_ptr); \
- if (augmented_args.filename.size < sizeof(augmented_args.filename.value)) { \
- len -= sizeof(augmented_args.filename.value) - augmented_args.filename.size; \
- len &= sizeof(augmented_args.filename.value) - 1; \
- } \
- /* If perf_event_output fails, return non-zero so that it gets recorded unaugmented */ \
- return perf_event_output(args, &__augmented_syscalls__, BPF_F_CURRENT_CPU, \
- &augmented_args, len); \
-} \
-int syscall_exit(syscall)(struct syscall_exit_args *args) \
-{ \
- return 1; /* 0 as soon as we start copying data returned by the kernel, e.g. 'read' */ \
-}
-
-struct syscall_enter_openat_args {
- unsigned long long common_tp_fields;
- long syscall_nr;
- long dfd;
- char *filename_ptr;
- long flags;
- long mode;
-};
-
-augmented_filename_syscall(openat);
-
-struct syscall_enter_open_args {
- unsigned long long common_tp_fields;
- long syscall_nr;
- char *filename_ptr;
- long flags;
- long mode;
-};
-
-augmented_filename_syscall(open);
-
-struct syscall_enter_inotify_add_watch_args {
- unsigned long long common_tp_fields;
- long syscall_nr;
- long fd;
- char *filename_ptr;
- long mask;
-};
-
-augmented_filename_syscall(inotify_add_watch);
-
-struct statbuf;
-
-struct syscall_enter_newstat_args {
- unsigned long long common_tp_fields;
- long syscall_nr;
- char *filename_ptr;
- struct stat *statbuf;
-};
-
-augmented_filename_syscall(newstat);
-
-#ifndef _K_SS_MAXSIZE
-#define _K_SS_MAXSIZE 128
-#endif
-
-#define augmented_sockaddr_syscall(syscall) \
-struct augmented_enter_##syscall##_args { \
- struct syscall_enter_##syscall##_args args; \
- struct sockaddr_storage addr; \
-}; \
-int syscall_enter(syscall)(struct syscall_enter_##syscall##_args *args) \
-{ \
- struct augmented_enter_##syscall##_args augmented_args; \
- unsigned long addrlen = sizeof(augmented_args.addr); \
- probe_read(&augmented_args.args, sizeof(augmented_args.args), args); \
-/* FIXME_CLANG_OPTIMIZATION_THAT_ACCESSES_USER_CONTROLLED_ADDRLEN_DESPITE_THIS_CHECK */ \
-/* if (addrlen > augmented_args.args.addrlen) */ \
-/* addrlen = augmented_args.args.addrlen; */ \
-/* */ \
- probe_read(&augmented_args.addr, addrlen, args->addr_ptr); \
- /* If perf_event_output fails, return non-zero so that it gets recorded unaugmented */ \
- return perf_event_output(args, &__augmented_syscalls__, BPF_F_CURRENT_CPU, \
- &augmented_args, \
- sizeof(augmented_args) - sizeof(augmented_args.addr) + addrlen);\
-} \
-int syscall_exit(syscall)(struct syscall_exit_args *args) \
-{ \
- return 1; /* 0 as soon as we start copying data returned by the kernel, e.g. 'read' */ \
-}
-
-struct sockaddr;
-
-struct syscall_enter_bind_args {
- unsigned long long common_tp_fields;
- long syscall_nr;
- long fd;
- struct sockaddr *addr_ptr;
- unsigned long addrlen;
-};
-
-augmented_sockaddr_syscall(bind);
-
-struct syscall_enter_connect_args {
- unsigned long long common_tp_fields;
- long syscall_nr;
- long fd;
- struct sockaddr *addr_ptr;
- unsigned long addrlen;
-};
-
-augmented_sockaddr_syscall(connect);
-
-struct syscall_enter_sendto_args {
- unsigned long long common_tp_fields;
- long syscall_nr;
- long fd;
- void *buff;
- long len;
- unsigned long flags;
- struct sockaddr *addr_ptr;
- long addr_len;
-};
-
-augmented_sockaddr_syscall(sendto);
-
-license(GPL);
-#include <bpf/bpf.h>
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
-license(GPL);
+struct syscall_enter_args;
+
+SEC("raw_syscalls:sys_enter")
+int sys_enter(struct syscall_enter_args *args)
+{
+ return 0;
+}
+char _license[] SEC("license") = "GPL";
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Augment the filename syscalls with the contents of the filename pointer argument
- * filtering only those that do not start with /etc/.
- *
- * Test it with:
- *
- * perf trace -e tools/perf/examples/bpf/augmented_syscalls.c cat /etc/passwd > /dev/null
- *
- * It'll catch some openat syscalls related to the dynamic linked and
- * the last one should be the one for '/etc/passwd'.
- *
- * This matches what is marshalled into the raw_syscall:sys_enter payload
- * expected by the 'perf trace' beautifiers, and can be used by them unmodified,
- * which will be done as that feature is implemented in the next csets, for now
- * it will appear in a dump done by the default tracepoint handler in 'perf trace',
- * that uses bpf_output__fprintf() to just dump those contents, as done with
- * the bpf-output event associated with the __bpf_output__ map declared in
- * tools/perf/include/bpf/stdio.h.
- */
-
-#include <stdio.h>
-
-/* bpf-output associated map */
-bpf_map(__augmented_syscalls__, PERF_EVENT_ARRAY, int, u32, __NR_CPUS__);
-
-struct augmented_filename {
- int size;
- int reserved;
- char value[64];
-};
-
-#define augmented_filename_syscall_enter(syscall) \
-struct augmented_enter_##syscall##_args { \
- struct syscall_enter_##syscall##_args args; \
- struct augmented_filename filename; \
-}; \
-int syscall_enter(syscall)(struct syscall_enter_##syscall##_args *args) \
-{ \
- char etc[6] = "/etc/"; \
- struct augmented_enter_##syscall##_args augmented_args = { .filename.reserved = 0, }; \
- probe_read(&augmented_args.args, sizeof(augmented_args.args), args); \
- augmented_args.filename.size = probe_read_str(&augmented_args.filename.value, \
- sizeof(augmented_args.filename.value), \
- args->filename_ptr); \
- if (__builtin_memcmp(augmented_args.filename.value, etc, 4) != 0) \
- return 0; \
- /* If perf_event_output fails, return non-zero so that it gets recorded unaugmented */ \
- return perf_event_output(args, &__augmented_syscalls__, BPF_F_CURRENT_CPU, \
- &augmented_args, \
- (sizeof(augmented_args) - sizeof(augmented_args.filename.value) + \
- augmented_args.filename.size)); \
-}
-
-struct syscall_enter_openat_args {
- unsigned long long common_tp_fields;
- long syscall_nr;
- long dfd;
- char *filename_ptr;
- long flags;
- long mode;
-};
-
-augmented_filename_syscall_enter(openat);
-
-struct syscall_enter_open_args {
- unsigned long long common_tp_fields;
- long syscall_nr;
- char *filename_ptr;
- long flags;
- long mode;
-};
-
-augmented_filename_syscall_enter(open);
-
-license(GPL);
-#include <stdio.h>
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
-int syscall_enter(openat)(void *args)
+struct __bpf_stdout__ {
+ __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);
+ __type(key, int);
+ __type(value, __u32);
+ __uint(max_entries, __NR_CPUS__);
+} __bpf_stdout__ SEC(".maps");
+
+#define puts(from) \
+ ({ const int __len = sizeof(from); \
+ char __from[sizeof(from)] = from; \
+ bpf_perf_event_output(args, &__bpf_stdout__, BPF_F_CURRENT_CPU, \
+ &__from, __len & (sizeof(from) - 1)); })
+
+struct syscall_enter_args;
+
+SEC("raw_syscalls:sys_enter")
+int sys_enter(struct syscall_enter_args *args)
{
puts("Hello, world\n");
return 0;
}
-license(GPL);
+char _license[] SEC("license") = "GPL";
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-#ifndef _PERF_BPF_H
-#define _PERF_BPF_H
-
-#include <uapi/linux/bpf.h>
-
-/*
- * A helper structure used by eBPF C program to describe map attributes to
- * elf_bpf loader, taken from tools/testing/selftests/bpf/bpf_helpers.h:
- */
-struct bpf_map {
- unsigned int type;
- unsigned int key_size;
- unsigned int value_size;
- unsigned int max_entries;
- unsigned int map_flags;
- unsigned int inner_map_idx;
- unsigned int numa_node;
-};
-
-#define bpf_map(name, _type, type_key, type_val, _max_entries) \
-struct bpf_map SEC("maps") name = { \
- .type = BPF_MAP_TYPE_##_type, \
- .key_size = sizeof(type_key), \
- .value_size = sizeof(type_val), \
- .max_entries = _max_entries, \
-}; \
-struct ____btf_map_##name { \
- type_key key; \
- type_val value; \
-}; \
-struct ____btf_map_##name __attribute__((section(".maps." #name), used)) \
- ____btf_map_##name = { }
-
-/*
- * FIXME: this should receive .max_entries as a parameter, as careful
- * tuning of these limits is needed to avoid hitting limits that
- * prevents other BPF constructs, such as tracepoint handlers,
- * to get installed, with cryptic messages from libbpf, etc.
- * For the current need, 'perf trace --filter-pids', 64 should
- * be good enough, but this surely needs to be revisited.
- */
-#define pid_map(name, value_type) bpf_map(name, HASH, pid_t, value_type, 64)
-
-static int (*bpf_map_update_elem)(struct bpf_map *map, void *key, void *value, u64 flags) = (void *)BPF_FUNC_map_update_elem;
-static void *(*bpf_map_lookup_elem)(struct bpf_map *map, void *key) = (void *)BPF_FUNC_map_lookup_elem;
-
-static void (*bpf_tail_call)(void *ctx, void *map, int index) = (void *)BPF_FUNC_tail_call;
-
-#define SEC(NAME) __attribute__((section(NAME), used))
-
-#define probe(function, vars) \
- SEC(#function "=" #function " " #vars) function
-
-#define syscall_enter(name) \
- SEC("syscalls:sys_enter_" #name) syscall_enter_ ## name
-
-#define syscall_exit(name) \
- SEC("syscalls:sys_exit_" #name) syscall_exit_ ## name
-
-#define license(name) \
-char _license[] SEC("license") = #name; \
-int _version SEC("version") = LINUX_VERSION_CODE;
-
-static int (*probe_read)(void *dst, int size, const void *unsafe_addr) = (void *)BPF_FUNC_probe_read;
-static int (*probe_read_str)(void *dst, int size, const void *unsafe_addr) = (void *)BPF_FUNC_probe_read_str;
-
-static int (*perf_event_output)(void *, struct bpf_map *, int, void *, unsigned long) = (void *)BPF_FUNC_perf_event_output;
-
-#endif /* _PERF_BPF_H */
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
-#ifndef _UAPI_LINUX_SOCKET_H
-#define _UAPI_LINUX_SOCKET_H
-
-/*
- * Desired design of maximum size and alignment (see RFC2553)
- */
-#define _K_SS_MAXSIZE 128 /* Implementation specific max size */
-#define _K_SS_ALIGNSIZE (__alignof__ (struct sockaddr *))
- /* Implementation specific desired alignment */
-
-typedef unsigned short __kernel_sa_family_t;
-
-struct __kernel_sockaddr_storage {
- __kernel_sa_family_t ss_family; /* address family */
- /* Following field(s) are implementation specific */
- char __data[_K_SS_MAXSIZE - sizeof(unsigned short)];
- /* space to achieve desired size, */
- /* _SS_MAXSIZE value minus size of ss_family */
-} __attribute__ ((aligned(_K_SS_ALIGNSIZE))); /* force desired alignment */
-
-#define sockaddr_storage __kernel_sockaddr_storage
-
-#endif /* _UAPI_LINUX_SOCKET_H */
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-
-#ifndef _PERF_BPF_PID_FILTER_
-#define _PERF_BPF_PID_FILTER_
-
-#include <bpf.h>
-
-#define pid_filter(name) pid_map(name, bool)
-
-static int pid_filter__add(struct bpf_map *pids, pid_t pid)
-{
- bool value = true;
- return bpf_map_update_elem(pids, &pid, &value, BPF_NOEXIST);
-}
-
-static bool pid_filter__has(struct bpf_map *pids, pid_t pid)
-{
- return bpf_map_lookup_elem(pids, &pid) != NULL;
-}
-
-#endif // _PERF_BPF_PID_FILTER_
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-
-#include <bpf.h>
-
-struct bpf_map SEC("maps") __bpf_stdout__ = {
- .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(u32),
- .max_entries = __NR_CPUS__,
-};
-
-#define puts(from) \
- ({ const int __len = sizeof(from); \
- char __from[__len] = from; \
- perf_event_output(args, &__bpf_stdout__, BPF_F_CURRENT_CPU, \
- &__from, __len & (sizeof(from) - 1)); })
+++ /dev/null
-// SPDX-License-Identifier: LGPL-2.1
-
-#include <bpf.h>
-
-static int (*bpf_get_current_pid_tgid)(void) = (void *)BPF_FUNC_get_current_pid_tgid;
-
-static pid_t getpid(void)
-{
- return bpf_get_current_pid_tgid();
-}
{ "report", cmd_report, 0 },
{ "bench", cmd_bench, 0 },
{ "stat", cmd_stat, 0 },
+#ifdef HAVE_LIBTRACEEVENT
{ "timechart", cmd_timechart, 0 },
+#endif
{ "top", cmd_top, 0 },
{ "annotate", cmd_annotate, 0 },
{ "version", cmd_version, 0 },
{ "script", cmd_script, 0 },
+#ifdef HAVE_LIBTRACEEVENT
{ "sched", cmd_sched, 0 },
+#endif
#ifdef HAVE_LIBELF_SUPPORT
{ "probe", cmd_probe, 0 },
#endif
+#ifdef HAVE_LIBTRACEEVENT
{ "kmem", cmd_kmem, 0 },
{ "lock", cmd_lock, 0 },
+#endif
{ "kvm", cmd_kvm, 0 },
{ "test", cmd_test, 0 },
-#if defined(HAVE_LIBAUDIT_SUPPORT) || defined(HAVE_SYSCALL_TABLE_SUPPORT)
+#if defined(HAVE_LIBTRACEEVENT) && (defined(HAVE_LIBAUDIT_SUPPORT) || defined(HAVE_SYSCALL_TABLE_SUPPORT))
{ "trace", cmd_trace, 0 },
#endif
{ "inject", cmd_inject, 0 },
{ "data", cmd_data, 0 },
{ "ftrace", cmd_ftrace, 0 },
{ "daemon", cmd_daemon, 0 },
+#ifdef HAVE_LIBTRACEEVENT
{ "kwork", cmd_kwork, 0 },
+#endif
};
struct pager_config {
argv[0] = cmd;
}
if (strstarts(cmd, "trace")) {
-#if defined(HAVE_LIBAUDIT_SUPPORT) || defined(HAVE_SYSCALL_TABLE_SUPPORT)
- setup_path();
- argv[0] = "trace";
- return cmd_trace(argc, argv);
-#else
+#ifndef HAVE_LIBTRACEEVENT
+ fprintf(stderr,
+ "trace command not available: missing libtraceevent devel package at build time.\n");
+ goto out;
+#elif !defined(HAVE_LIBAUDIT_SUPPORT) && !defined(HAVE_SYSCALL_TABLE_SUPPORT)
fprintf(stderr,
"trace command not available: missing audit-libs devel package at build time.\n");
goto out;
+#else
+ setup_path();
+ argv[0] = "trace";
+ return cmd_trace(argc, argv);
#endif
}
/* Look for flags.. */
$(call rule_mkdir)
$(Q)$(call echo-cmd,gen)cp $< $@
else
-$(OUTPUT)pmu-events/pmu-events.c: $(JSON) $(JSON_TEST) $(JEVENTS_PY)
+$(OUTPUT)pmu-events/pmu-events.c: $(JSON) $(JSON_TEST) $(JEVENTS_PY) pmu-events/metric.py
$(call rule_mkdir)
$(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) pmu-events/arch $@
endif
0x00000000410fd460,v1,arm/cortex-a510,core
0x00000000410fd470,v1,arm/cortex-a710,core
0x00000000410fd480,v1,arm/cortex-x2,core
-0x00000000410fd490,v1,arm/neoverse-n2,core
+0x00000000410fd490,v1,arm/neoverse-n2-v2,core
+0x00000000410fd4f0,v1,arm/neoverse-n2-v2,core
0x00000000420f5160,v1,cavium/thunderx2,core
0x00000000430f0af0,v1,cavium/thunderx2,core
0x00000000460f0010,v1,fujitsu/a64fx,core
--- /dev/null
+# Format:
+# MVENDORID-MARCHID-MIMPID,Version,JSON/file/pathname,Type
+#
+# where
+# MVENDORID JEDEC code of the core provider
+# MARCHID base microarchitecture of the hart
+# MIMPID unique encoding of the version
+# of the processor implementation
+# Version could be used to track version of JSON file
+# but currently unused.
+# JSON/file/pathname is the path to JSON file, relative
+# to tools/perf/pmu-events/arch/riscv/.
+# Type is core, uncore etc
+#
+#
+#MVENDORID-MARCHID-MIMPID,Version,Filename,EventType
+0x489-0x8000000000000007-0x[[:xdigit:]]+,v1,sifive/u74,core
--- /dev/null
+[
+ {
+ "PublicDescription": "Misaligned load trap",
+ "ConfigCode": "0x8000000000000000",
+ "EventName": "FW_MISALIGNED_LOAD",
+ "BriefDescription": "Misaligned load trap event"
+ },
+ {
+ "PublicDescription": "Misaligned store trap",
+ "ConfigCode": "0x8000000000000001",
+ "EventName": "FW_MISALIGNED_STORE",
+ "BriefDescription": "Misaligned store trap event"
+ },
+ {
+ "PublicDescription": "Load access trap",
+ "ConfigCode": "0x8000000000000002",
+ "EventName": "FW_ACCESS_LOAD",
+ "BriefDescription": "Load access trap event"
+ },
+ {
+ "PublicDescription": "Store access trap",
+ "ConfigCode": "0x8000000000000003",
+ "EventName": "FW_ACCESS_STORE",
+ "BriefDescription": "Store access trap event"
+ },
+ {
+ "PublicDescription": "Illegal instruction trap",
+ "ConfigCode": "0x8000000000000004",
+ "EventName": "FW_ILLEGAL_INSN",
+ "BriefDescription": "Illegal instruction trap event"
+ },
+ {
+ "PublicDescription": "Set timer event",
+ "ConfigCode": "0x8000000000000005",
+ "EventName": "FW_SET_TIMER",
+ "BriefDescription": "Set timer event"
+ },
+ {
+ "PublicDescription": "Sent IPI to other HART event",
+ "ConfigCode": "0x8000000000000006",
+ "EventName": "FW_IPI_SENT",
+ "BriefDescription": "Sent IPI to other HART event"
+ },
+ {
+ "PublicDescription": "Received IPI from other HART event",
+ "ConfigCode": "0x8000000000000007",
+ "EventName": "FW_IPI_RECEIVED",
+ "BriefDescription": "Received IPI from other HART event"
+ },
+ {
+ "PublicDescription": "Sent FENCE.I request to other HART event",
+ "ConfigCode": "0x8000000000000008",
+ "EventName": "FW_FENCE_I_SENT",
+ "BriefDescription": "Sent FENCE.I request to other HART event"
+ },
+ {
+ "PublicDescription": "Received FENCE.I request from other HART event",
+ "ConfigCode": "0x8000000000000009",
+ "EventName": "FW_FENCE_I_RECEIVED",
+ "BriefDescription": "Received FENCE.I request from other HART event"
+ },
+ {
+ "PublicDescription": "Sent SFENCE.VMA request to other HART event",
+ "ConfigCode": "0x800000000000000a",
+ "EventName": "FW_SFENCE_VMA_SENT",
+ "BriefDescription": "Sent SFENCE.VMA request to other HART event"
+ },
+ {
+ "PublicDescription": "Received SFENCE.VMA request from other HART event",
+ "ConfigCode": "0x800000000000000b",
+ "EventName": "FW_SFENCE_VMA_RECEIVED",
+ "BriefDescription": "Received SFENCE.VMA request from other HART event"
+ },
+ {
+ "PublicDescription": "Sent SFENCE.VMA with ASID request to other HART event",
+ "ConfigCode": "0x800000000000000c",
+ "EventName": "FW_SFENCE_VMA_RECEIVED",
+ "BriefDescription": "Sent SFENCE.VMA with ASID request to other HART event"
+ },
+ {
+ "PublicDescription": "Received SFENCE.VMA with ASID request from other HART event",
+ "ConfigCode": "0x800000000000000d",
+ "EventName": "FW_SFENCE_VMA_ASID_RECEIVED",
+ "BriefDescription": "Received SFENCE.VMA with ASID request from other HART event"
+ },
+ {
+ "PublicDescription": "Sent HFENCE.GVMA request to other HART event",
+ "ConfigCode": "0x800000000000000e",
+ "EventName": "FW_HFENCE_GVMA_SENT",
+ "BriefDescription": "Sent HFENCE.GVMA request to other HART event"
+ },
+ {
+ "PublicDescription": "Received HFENCE.GVMA request from other HART event",
+ "ConfigCode": "0x800000000000000f",
+ "EventName": "FW_HFENCE_GVMA_RECEIVED",
+ "BriefDescription": "Received HFENCE.GVMA request from other HART event"
+ },
+ {
+ "PublicDescription": "Sent HFENCE.GVMA with VMID request to other HART event",
+ "ConfigCode": "0x8000000000000010",
+ "EventName": "FW_HFENCE_GVMA_VMID_SENT",
+ "BriefDescription": "Sent HFENCE.GVMA with VMID request to other HART event"
+ },
+ {
+ "PublicDescription": "Received HFENCE.GVMA with VMID request from other HART event",
+ "ConfigCode": "0x8000000000000011",
+ "EventName": "FW_HFENCE_GVMA_VMID_RECEIVED",
+ "BriefDescription": "Received HFENCE.GVMA with VMID request from other HART event"
+ },
+ {
+ "PublicDescription": "Sent HFENCE.VVMA request to other HART event",
+ "ConfigCode": "0x8000000000000012",
+ "EventName": "FW_HFENCE_VVMA_SENT",
+ "BriefDescription": "Sent HFENCE.VVMA request to other HART event"
+ },
+ {
+ "PublicDescription": "Received HFENCE.VVMA request from other HART event",
+ "ConfigCode": "0x8000000000000013",
+ "EventName": "FW_HFENCE_VVMA_RECEIVED",
+ "BriefDescription": "Received HFENCE.VVMA request from other HART event"
+ },
+ {
+ "PublicDescription": "Sent HFENCE.VVMA with ASID request to other HART event",
+ "ConfigCode": "0x8000000000000014",
+ "EventName": "FW_HFENCE_VVMA_ASID_SENT",
+ "BriefDescription": "Sent HFENCE.VVMA with ASID request to other HART event"
+ },
+ {
+ "PublicDescription": "Received HFENCE.VVMA with ASID request from other HART event",
+ "ConfigCode": "0x8000000000000015",
+ "EventName": "FW_HFENCE_VVMA_ASID_RECEIVED",
+ "BriefDescription": "Received HFENCE.VVMA with ASID request from other HART event"
+ }
+]
--- /dev/null
+[
+ {
+ "ArchStdEvent": "FW_MISALIGNED_LOAD"
+ },
+ {
+ "ArchStdEvent": "FW_MISALIGNED_STORE"
+ },
+ {
+ "ArchStdEvent": "FW_ACCESS_LOAD"
+ },
+ {
+ "ArchStdEvent": "FW_ACCESS_STORE"
+ },
+ {
+ "ArchStdEvent": "FW_ILLEGAL_INSN"
+ },
+ {
+ "ArchStdEvent": "FW_SET_TIMER"
+ },
+ {
+ "ArchStdEvent": "FW_IPI_SENT"
+ },
+ {
+ "ArchStdEvent": "FW_IPI_RECEIVED"
+ },
+ {
+ "ArchStdEvent": "FW_FENCE_I_SENT"
+ },
+ {
+ "ArchStdEvent": "FW_FENCE_I_RECEIVED"
+ },
+ {
+ "ArchStdEvent": "FW_SFENCE_VMA_SENT"
+ },
+ {
+ "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
+ },
+ {
+ "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
+ },
+ {
+ "ArchStdEvent": "FW_SFENCE_VMA_ASID_RECEIVED"
+ },
+ {
+ "ArchStdEvent": "FW_HFENCE_GVMA_SENT"
+ },
+ {
+ "ArchStdEvent": "FW_HFENCE_GVMA_RECEIVED"
+ },
+ {
+ "ArchStdEvent": "FW_HFENCE_GVMA_VMID_SENT"
+ },
+ {
+ "ArchStdEvent": "FW_HFENCE_GVMA_VMID_RECEIVED"
+ },
+ {
+ "ArchStdEvent": "FW_HFENCE_VVMA_SENT"
+ },
+ {
+ "ArchStdEvent": "FW_HFENCE_VVMA_RECEIVED"
+ },
+ {
+ "ArchStdEvent": "FW_HFENCE_VVMA_ASID_SENT"
+ },
+ {
+ "ArchStdEvent": "FW_HFENCE_VVMA_ASID_RECEIVED"
+ }
+]
--- /dev/null
+[
+ {
+ "EventName": "EXCEPTION_TAKEN",
+ "EventCode": "0x0000100",
+ "BriefDescription": "Exception taken"
+ },
+ {
+ "EventName": "INTEGER_LOAD_RETIRED",
+ "EventCode": "0x0000200",
+ "BriefDescription": "Integer load instruction retired"
+ },
+ {
+ "EventName": "INTEGER_STORE_RETIRED",
+ "EventCode": "0x0000400",
+ "BriefDescription": "Integer store instruction retired"
+ },
+ {
+ "EventName": "ATOMIC_MEMORY_RETIRED",
+ "EventCode": "0x0000800",
+ "BriefDescription": "Atomic memory operation retired"
+ },
+ {
+ "EventName": "SYSTEM_INSTRUCTION_RETIRED",
+ "EventCode": "0x0001000",
+ "BriefDescription": "System instruction retired"
+ },
+ {
+ "EventName": "INTEGER_ARITHMETIC_RETIRED",
+ "EventCode": "0x0002000",
+ "BriefDescription": "Integer arithmetic instruction retired"
+ },
+ {
+ "EventName": "CONDITIONAL_BRANCH_RETIRED",
+ "EventCode": "0x0004000",
+ "BriefDescription": "Conditional branch retired"
+ },
+ {
+ "EventName": "JAL_INSTRUCTION_RETIRED",
+ "EventCode": "0x0008000",
+ "BriefDescription": "JAL instruction retired"
+ },
+ {
+ "EventName": "JALR_INSTRUCTION_RETIRED",
+ "EventCode": "0x0010000",
+ "BriefDescription": "JALR instruction retired"
+ },
+ {
+ "EventName": "INTEGER_MULTIPLICATION_RETIRED",
+ "EventCode": "0x0020000",
+ "BriefDescription": "Integer multiplication instruction retired"
+ },
+ {
+ "EventName": "INTEGER_DIVISION_RETIRED",
+ "EventCode": "0x0040000",
+ "BriefDescription": "Integer division instruction retired"
+ },
+ {
+ "EventName": "FP_LOAD_RETIRED",
+ "EventCode": "0x0080000",
+ "BriefDescription": "Floating-point load instruction retired"
+ },
+ {
+ "EventName": "FP_STORE_RETIRED",
+ "EventCode": "0x0100000",
+ "BriefDescription": "Floating-point store instruction retired"
+ },
+ {
+ "EventName": "FP_ADDITION_RETIRED",
+ "EventCode": "0x0200000",
+ "BriefDescription": "Floating-point addition retired"
+ },
+ {
+ "EventName": "FP_MULTIPLICATION_RETIRED",
+ "EventCode": "0x0400000",
+ "BriefDescription": "Floating-point multiplication retired"
+ },
+ {
+ "EventName": "FP_FUSEDMADD_RETIRED",
+ "EventCode": "0x0800000",
+ "BriefDescription": "Floating-point fused multiply-add retired"
+ },
+ {
+ "EventName": "FP_DIV_SQRT_RETIRED",
+ "EventCode": "0x1000000",
+ "BriefDescription": "Floating-point division or square-root retired"
+ },
+ {
+ "EventName": "OTHER_FP_RETIRED",
+ "EventCode": "0x2000000",
+ "BriefDescription": "Other floating-point instruction retired"
+ }
+]
\ No newline at end of file
--- /dev/null
+[
+ {
+ "EventName": "ICACHE_RETIRED",
+ "EventCode": "0x0000102",
+ "BriefDescription": "Instruction cache miss"
+ },
+ {
+ "EventName": "DCACHE_MISS_MMIO_ACCESSES",
+ "EventCode": "0x0000202",
+ "BriefDescription": "Data cache miss or memory-mapped I/O access"
+ },
+ {
+ "EventName": "DCACHE_WRITEBACK",
+ "EventCode": "0x0000402",
+ "BriefDescription": "Data cache write-back"
+ },
+ {
+ "EventName": "INST_TLB_MISS",
+ "EventCode": "0x0000802",
+ "BriefDescription": "Instruction TLB miss"
+ },
+ {
+ "EventName": "DATA_TLB_MISS",
+ "EventCode": "0x0001002",
+ "BriefDescription": "Data TLB miss"
+ },
+ {
+ "EventName": "UTLB_MISS",
+ "EventCode": "0x0002002",
+ "BriefDescription": "UTLB miss"
+ }
+]
\ No newline at end of file
--- /dev/null
+[
+ {
+ "EventName": "ADDRESSGEN_INTERLOCK",
+ "EventCode": "0x0000101",
+ "BriefDescription": "Address-generation interlock"
+ },
+ {
+ "EventName": "LONGLAT_INTERLOCK",
+ "EventCode": "0x0000201",
+ "BriefDescription": "Long-latency interlock"
+ },
+ {
+ "EventName": "CSR_READ_INTERLOCK",
+ "EventCode": "0x0000401",
+ "BriefDescription": "CSR read interlock"
+ },
+ {
+ "EventName": "ICACHE_ITIM_BUSY",
+ "EventCode": "0x0000801",
+ "BriefDescription": "Instruction cache/ITIM busy"
+ },
+ {
+ "EventName": "DCACHE_DTIM_BUSY",
+ "EventCode": "0x0001001",
+ "BriefDescription": "Data cache/DTIM busy"
+ },
+ {
+ "EventName": "BRANCH_DIRECTION_MISPREDICTION",
+ "EventCode": "0x0002001",
+ "BriefDescription": "Branch direction misprediction"
+ },
+ {
+ "EventName": "BRANCH_TARGET_MISPREDICTION",
+ "EventCode": "0x0004001",
+ "BriefDescription": "Branch/jump target misprediction"
+ },
+ {
+ "EventName": "PIPE_FLUSH_CSR_WRITE",
+ "EventCode": "0x0008001",
+ "BriefDescription": "Pipeline flush from CSR write"
+ },
+ {
+ "EventName": "PIPE_FLUSH_OTHER_EVENT",
+ "EventCode": "0x0010001",
+ "BriefDescription": "Pipeline flush from other event"
+ },
+ {
+ "EventName": "INTEGER_MULTIPLICATION_INTERLOCK",
+ "EventCode": "0x0020001",
+ "BriefDescription": "Integer multiplication interlock"
+ },
+ {
+ "EventName": "FP_INTERLOCK",
+ "EventCode": "0x0040001",
+ "BriefDescription": "Floating-point interlock"
+ }
+]
\ No newline at end of file
},
{
"BriefDescription": "Average CPU Utilization",
- "MetricExpr": "CPU_CLK_UNHALTED.REF_TSC / msr@tsc@",
+ "MetricExpr": "CPU_CLK_UNHALTED.REF_TSC / TSC",
"MetricGroup": "HPC;Summary",
"MetricName": "CPU_Utilization",
"Unit": "cpu_core"
},
{
"BriefDescription": "Measured Average Frequency for unhalted processors [GHz]",
- "MetricExpr": "Turbo_Utilization * msr@tsc@ / 1000000000 / duration_time",
+ "MetricExpr": "Turbo_Utilization * TSC / 1000000000 / duration_time",
"MetricGroup": "Power;Summary",
"MetricName": "Average_Frequency",
"Unit": "cpu_core"
},
{
"BriefDescription": "Average external Memory Bandwidth Use for reads and writes [GB / sec]",
- "MetricExpr": "64 * (arb@event\\=0x81\\,umask\\=0x1@ + arb@event\\=0x84\\,umask\\=0x1@) / 1000000 / duration_time / 1000",
+ "MetricExpr": "64 * (UNC_ARB_TRK_REQUESTS.ALL + UNC_ARB_COH_TRK_REQUESTS.ALL) / 1000000 / duration_time / 1000",
"MetricGroup": "HPC;Mem;MemoryBW;SoC",
"MetricName": "DRAM_BW_Use",
"Unit": "cpu_core"
},
{
"BriefDescription": "Average number of parallel requests to external memory. Accounts for all requests",
- "MetricExpr": "UNC_ARB_TRK_OCCUPANCY.ALL / arb@event\\=0x81\\,umask\\=0x1@",
+ "MetricExpr": "UNC_ARB_TRK_OCCUPANCY.ALL / UNC_ARB_TRK_REQUESTS.ALL",
"MetricGroup": "Mem;SoC",
"MetricName": "MEM_Parallel_Requests",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Socket actual clocks when any core is active on that socket",
+ "MetricExpr": "UNC_CLOCK.SOCKET",
+ "MetricGroup": "SoC",
+ "MetricName": "Socket_CLKS",
+ "Unit": "cpu_core"
+ },
+ {
"BriefDescription": "Instructions per Far Branch ( Far Branches apply upon transition from application to operating system, handling interrupts, exceptions) [lower number means higher occurrence rate]",
"MetricExpr": "INST_RETIRED.ANY / BR_INST_RETIRED.FAR_BRANCH:u",
"MetricGroup": "Branches;OS",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Uncore frequency per die [GHZ]",
+ "MetricExpr": "Socket_CLKS / #num_dies / duration_time / 1000000000",
+ "MetricGroup": "SoC",
+ "MetricName": "UNCORE_FREQ"
+ },
+ {
"BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to frontend stalls.",
"MetricExpr": "TOPDOWN_FE_BOUND.ALL / SLOTS",
"MetricGroup": "TopdownL1",
},
{
"BriefDescription": "Average CPU Utilization",
- "MetricExpr": "CPU_CLK_UNHALTED.REF_TSC / msr@tsc@",
+ "MetricExpr": "CPU_CLK_UNHALTED.REF_TSC / TSC",
"MetricName": "CPU_Utilization",
"Unit": "cpu_atom"
},
},
{
"BriefDescription": "C1 residency percent per core",
- "MetricExpr": "(cstate_core@c1\\-residency@ / msr@tsc@) * 100",
+ "MetricExpr": "cstate_core@c1\\-residency@ / TSC",
"MetricGroup": "Power",
- "MetricName": "C1_Core_Residency"
+ "MetricName": "C1_Core_Residency",
+ "ScaleUnit": "100%"
},
{
"BriefDescription": "C6 residency percent per core",
- "MetricExpr": "(cstate_core@c6\\-residency@ / msr@tsc@) * 100",
+ "MetricExpr": "cstate_core@c6\\-residency@ / TSC",
"MetricGroup": "Power",
- "MetricName": "C6_Core_Residency"
+ "MetricName": "C6_Core_Residency",
+ "ScaleUnit": "100%"
},
{
"BriefDescription": "C7 residency percent per core",
- "MetricExpr": "(cstate_core@c7\\-residency@ / msr@tsc@) * 100",
+ "MetricExpr": "cstate_core@c7\\-residency@ / TSC",
"MetricGroup": "Power",
- "MetricName": "C7_Core_Residency"
+ "MetricName": "C7_Core_Residency",
+ "ScaleUnit": "100%"
},
{
"BriefDescription": "C2 residency percent per package",
- "MetricExpr": "(cstate_pkg@c2\\-residency@ / msr@tsc@) * 100",
+ "MetricExpr": "cstate_pkg@c2\\-residency@ / TSC",
"MetricGroup": "Power",
- "MetricName": "C2_Pkg_Residency"
+ "MetricName": "C2_Pkg_Residency",
+ "ScaleUnit": "100%"
},
{
"BriefDescription": "C3 residency percent per package",
- "MetricExpr": "(cstate_pkg@c3\\-residency@ / msr@tsc@) * 100",
+ "MetricExpr": "cstate_pkg@c3\\-residency@ / TSC",
"MetricGroup": "Power",
- "MetricName": "C3_Pkg_Residency"
+ "MetricName": "C3_Pkg_Residency",
+ "ScaleUnit": "100%"
},
{
"BriefDescription": "C6 residency percent per package",
- "MetricExpr": "(cstate_pkg@c6\\-residency@ / msr@tsc@) * 100",
+ "MetricExpr": "cstate_pkg@c6\\-residency@ / TSC",
"MetricGroup": "Power",
- "MetricName": "C6_Pkg_Residency"
+ "MetricName": "C6_Pkg_Residency",
+ "ScaleUnit": "100%"
},
{
"BriefDescription": "C7 residency percent per package",
- "MetricExpr": "(cstate_pkg@c7\\-residency@ / msr@tsc@) * 100",
+ "MetricExpr": "cstate_pkg@c7\\-residency@ / TSC",
"MetricGroup": "Power",
- "MetricName": "C7_Pkg_Residency"
+ "MetricName": "C7_Pkg_Residency",
+ "ScaleUnit": "100%"
},
{
"BriefDescription": "C8 residency percent per package",
- "MetricExpr": "(cstate_pkg@c8\\-residency@ / msr@tsc@) * 100",
+ "MetricExpr": "cstate_pkg@c8\\-residency@ / TSC",
"MetricGroup": "Power",
- "MetricName": "C8_Pkg_Residency"
+ "MetricName": "C8_Pkg_Residency",
+ "ScaleUnit": "100%"
},
{
"BriefDescription": "C9 residency percent per package",
- "MetricExpr": "(cstate_pkg@c9\\-residency@ / msr@tsc@) * 100",
+ "MetricExpr": "cstate_pkg@c9\\-residency@ / TSC",
"MetricGroup": "Power",
- "MetricName": "C9_Pkg_Residency"
+ "MetricName": "C9_Pkg_Residency",
+ "ScaleUnit": "100%"
},
{
"BriefDescription": "C10 residency percent per package",
- "MetricExpr": "(cstate_pkg@c10\\-residency@ / msr@tsc@) * 100",
+ "MetricExpr": "cstate_pkg@c10\\-residency@ / TSC",
"MetricGroup": "Power",
- "MetricName": "C10_Pkg_Residency"
+ "MetricName": "C10_Pkg_Residency",
+ "ScaleUnit": "100%"
}
]
[
{
- "BriefDescription": "Counts the number of cacheable memory requests that miss in the LLC. Counts on a per core basis.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x2e",
- "EventName": "LONGEST_LAT_CACHE.MISS",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x41",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cacheable memory requests that access the LLC. Counts on a per core basis.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x2e",
- "EventName": "LONGEST_LAT_CACHE.REFERENCE",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x4f",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in the L2, LLC, DRAM or MMIO (Non-DRAM).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x34",
- "EventName": "MEM_BOUND_STALLS.IFETCH",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x38",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in DRAM or MMIO (Non-DRAM).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x34",
- "EventName": "MEM_BOUND_STALLS.IFETCH_DRAM_HIT",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x20",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in the L2 cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x34",
- "EventName": "MEM_BOUND_STALLS.IFETCH_L2_HIT",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x8",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in the LLC or other core with HITE/F/M.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x34",
- "EventName": "MEM_BOUND_STALLS.IFETCH_LLC_HIT",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x10",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load miss which hit in the L2, LLC, DRAM or MMIO (Non-DRAM).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x34",
- "EventName": "MEM_BOUND_STALLS.LOAD",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x7",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load miss which hit in DRAM or MMIO (Non-DRAM).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x34",
- "EventName": "MEM_BOUND_STALLS.LOAD_DRAM_HIT",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x4",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load which hit in the L2 cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x34",
- "EventName": "MEM_BOUND_STALLS.LOAD_L2_HIT",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load which hit in the LLC or other core with HITE/F/M.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x34",
- "EventName": "MEM_BOUND_STALLS.LOAD_LLC_HIT",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of load uops retired that hit in DRAM.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "Data_LA": "1",
- "EventCode": "0xd1",
- "EventName": "MEM_LOAD_UOPS_RETIRED.DRAM_HIT",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0x80",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of load uops retired that hit in the L2 cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "Data_LA": "1",
- "EventCode": "0xd1",
- "EventName": "MEM_LOAD_UOPS_RETIRED.L2_HIT",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of load uops retired that hit in the L3 cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "Data_LA": "1",
- "EventCode": "0xd1",
- "EventName": "MEM_LOAD_UOPS_RETIRED.L3_HIT",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0x4",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles that uops are blocked for any of the following reasons: load buffer, store buffer or RSV full.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x04",
- "EventName": "MEM_SCHEDULER_BLOCK.ALL",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "20003",
- "Speculative": "1",
- "UMask": "0x7",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles that uops are blocked due to a load buffer full condition.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x04",
- "EventName": "MEM_SCHEDULER_BLOCK.LD_BUF",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "20003",
- "Speculative": "1",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles that uops are blocked due to an RSV full condition.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x04",
- "EventName": "MEM_SCHEDULER_BLOCK.RSV",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "20003",
- "Speculative": "1",
- "UMask": "0x4",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles that uops are blocked due to a store buffer full condition.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x04",
- "EventName": "MEM_SCHEDULER_BLOCK.ST_BUF",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "20003",
- "Speculative": "1",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of load uops retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.ALL_LOADS",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0x81",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of store uops retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.ALL_STORES",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0x82",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 128 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_128",
- "L1_Hit_Indication": "1",
- "MSRIndex": "0x3F6",
- "MSRValue": "0x80",
- "PEBS": "2",
- "PEBScounters": "0,1",
- "SampleAfterValue": "1000003",
- "TakenAlone": "1",
- "UMask": "0x5",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 16 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_16",
- "L1_Hit_Indication": "1",
- "MSRIndex": "0x3F6",
- "MSRValue": "0x10",
- "PEBS": "2",
- "PEBScounters": "0,1",
- "SampleAfterValue": "1000003",
- "TakenAlone": "1",
- "UMask": "0x5",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 256 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_256",
- "L1_Hit_Indication": "1",
- "MSRIndex": "0x3F6",
- "MSRValue": "0x100",
- "PEBS": "2",
- "PEBScounters": "0,1",
- "SampleAfterValue": "1000003",
- "TakenAlone": "1",
- "UMask": "0x5",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 32 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_32",
- "L1_Hit_Indication": "1",
- "MSRIndex": "0x3F6",
- "MSRValue": "0x20",
- "PEBS": "2",
- "PEBScounters": "0,1",
- "SampleAfterValue": "1000003",
- "TakenAlone": "1",
- "UMask": "0x5",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 4 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_4",
- "L1_Hit_Indication": "1",
- "MSRIndex": "0x3F6",
- "MSRValue": "0x4",
- "PEBS": "2",
- "PEBScounters": "0,1",
- "SampleAfterValue": "1000003",
- "TakenAlone": "1",
- "UMask": "0x5",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 512 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_512",
- "L1_Hit_Indication": "1",
- "MSRIndex": "0x3F6",
- "MSRValue": "0x200",
- "PEBS": "2",
- "PEBScounters": "0,1",
- "SampleAfterValue": "1000003",
- "TakenAlone": "1",
- "UMask": "0x5",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 64 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_64",
- "L1_Hit_Indication": "1",
- "MSRIndex": "0x3F6",
- "MSRValue": "0x40",
- "PEBS": "2",
- "PEBScounters": "0,1",
- "SampleAfterValue": "1000003",
- "TakenAlone": "1",
- "UMask": "0x5",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 8 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_8",
- "L1_Hit_Indication": "1",
- "MSRIndex": "0x3F6",
- "MSRValue": "0x8",
- "PEBS": "2",
- "PEBScounters": "0,1",
- "SampleAfterValue": "1000003",
- "TakenAlone": "1",
- "UMask": "0x5",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of retired split load uops.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.SPLIT_LOADS",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0x41",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of stores uops retired. Counts with or without PEBS enabled.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "Data_LA": "1",
- "EventCode": "0xd0",
- "EventName": "MEM_UOPS_RETIRED.STORE_LATENCY",
- "L1_Hit_Indication": "1",
- "PEBS": "2",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "UMask": "0x6",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand data reads that were supplied by the L3 cache.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_DATA_RD.L3_HIT",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x3F803C0001",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand data reads that were supplied by the L3 cache where a snoop was sent, the snoop hit, and modified data was forwarded.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HITM",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x10003C0001",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand data reads that were supplied by the L3 cache where a snoop was sent, the snoop hit, but no data was forwarded.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HIT_NO_FWD",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x4003C0001",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand data reads that were supplied by the L3 cache where a snoop was sent, the snoop hit, and non-modified data was forwarded.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HIT_WITH_FWD",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x8003C0001",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were supplied by the L3 cache.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_RFO.L3_HIT",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x3F803C0002",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were supplied by the L3 cache where a snoop was sent, the snoop hit, and modified data was forwarded.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_RFO.L3_HIT.SNOOP_HITM",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x10003C0002",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to instruction cache misses.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.ICACHE",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x20",
- "Unit": "cpu_atom"
- },
- {
"BriefDescription": "L1D.HWPF_MISS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x51",
"EventName": "L1D.HWPF_MISS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts the number of cache lines replaced in L1 data cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x51",
"EventName": "L1D.REPLACEMENT",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts L1D data line replacements including opportunistic replacements, and replacements that require stall-for-replace or block-for-replace.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of cycles a demand request has waited due to L1D Fill Buffer (FB) unavailability.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x48",
"EventName": "L1D_PEND_MISS.FB_FULL",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts number of cycles a demand request has waited due to L1D Fill Buffer (FB) unavailability. Demand requests include cacheable/uncacheable demand load, store, lock or SW prefetch accesses.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
- "BriefDescription": "Number of phases a demand request has waited due to L1D Fill Buffer (FB) unavailablability.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Number of phases a demand request has waited due to L1D Fill Buffer (FB) unavailability.",
"CounterMask": "1",
"EdgeDetect": "1",
"EventCode": "0x48",
"EventName": "L1D_PEND_MISS.FB_FULL_PERIODS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts number of phases a demand request has waited due to L1D Fill Buffer (FB) unavailability. Demand requests include cacheable/uncacheable demand load, store, lock or SW prefetch accesses.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "This event is deprecated. Refer to new event L1D_PEND_MISS.L2_STALLS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "Deprecated": "1",
"EventCode": "0x48",
"EventName": "L1D_PEND_MISS.L2_STALL",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of cycles a demand request has waited due to L1D due to lack of L2 resources.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x48",
"EventName": "L1D_PEND_MISS.L2_STALLS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts number of cycles a demand request has waited due to L1D due to lack of L2 resources. Demand requests include cacheable/uncacheable demand load, store, lock or SW prefetch accesses.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of L1D misses that are outstanding",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x48",
"EventName": "L1D_PEND_MISS.PENDING",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts number of L1D misses that are outstanding in each cycle, that is each cycle the number of Fill Buffers (FB) outstanding required by Demand Reads. FB either is held by demand loads, or it is held by non-demand loads and gets hit at least once by demand. The valid outstanding interval is defined until the FB deallocation by one of the following ways: from FB allocation, if FB is allocated by demand from the demand Hit FB, if it is allocated by hardware or software prefetch. Note: In the L1D, a Demand Read contains cacheable or noncacheable demand loads, including ones causing cache-line splits and reads due to page walks resulted from any request type.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles with L1D load Misses outstanding.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EventCode": "0x48",
"EventName": "L1D_PEND_MISS.PENDING_CYCLES",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts duration of L1D miss outstanding in cycles.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "L2 cache lines filling L2",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x25",
"EventName": "L2_LINES_IN.ALL",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of L2 cache lines filling the L2. Counting does not cover rejects.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x1f",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cache lines that have been L2 hardware prefetched but not used by demand accesses",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x26",
"EventName": "L2_LINES_OUT.USELESS_HWPF",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of cache lines that have been prefetched by the L2 hardware prefetcher but not used by demand access when evicted from the L2 cache",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "All accesses to L2 cache[This event is alias to L2_RQSTS.REFERENCES]",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_REQUEST.ALL",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts all requests that were hit or true misses in L2 cache. True-miss excludes misses that were merged with ongoing L2 misses.[This event is alias to L2_RQSTS.REFERENCES]",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0xff",
"Unit": "cpu_core"
},
{
"BriefDescription": "Read requests with true-miss in L2 cache.[This event is alias to L2_RQSTS.MISS]",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_REQUEST.MISS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts read requests of any type with true-miss in the L2 cache. True-miss excludes L2 misses that were merged with ongoing L2 misses.[This event is alias to L2_RQSTS.MISS]",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x3f",
"Unit": "cpu_core"
},
{
"BriefDescription": "L2 code requests",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.ALL_CODE_RD",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the total number of L2 code requests.",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0xe4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Demand Data Read access L2 cache",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.ALL_DEMAND_DATA_RD",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts Demand Data Read requests accessing the L2 cache. These requests may hit or miss L2 cache. True-miss exclude misses that were merged with ongoing L2 misses. An access is counted once.",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0xe1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Demand requests that miss L2 cache",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.ALL_DEMAND_MISS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts demand requests that miss L2 cache.",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x27",
"Unit": "cpu_core"
},
{
"BriefDescription": "L2_RQSTS.ALL_HWPF",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.ALL_HWPF",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0xf0",
"Unit": "cpu_core"
},
{
"BriefDescription": "RFO requests to L2 cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.ALL_RFO",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the total number of RFO (read for ownership) requests to L2 cache. L2 RFO requests include both L1D demand RFO misses as well as L1D RFO prefetches.",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0xe2",
"Unit": "cpu_core"
},
{
"BriefDescription": "L2 cache hits when fetching instructions, code reads.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.CODE_RD_HIT",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts L2 cache hits when fetching instructions, code reads.",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0xc4",
"Unit": "cpu_core"
},
{
"BriefDescription": "L2 cache misses when fetching instructions",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.CODE_RD_MISS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts L2 cache misses when fetching instructions.",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x24",
"Unit": "cpu_core"
},
{
"BriefDescription": "Demand Data Read requests that hit L2 cache",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.DEMAND_DATA_RD_HIT",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of demand Data Read requests initiated by load instructions that hit L2 cache.",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0xc1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Demand Data Read miss L2 cache",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.DEMAND_DATA_RD_MISS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts demand Data Read requests with true-miss in the L2 cache. True-miss excludes misses that were merged with ongoing L2 misses. An access is counted once.",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x21",
"Unit": "cpu_core"
},
{
"BriefDescription": "L2_RQSTS.HWPF_MISS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.HWPF_MISS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x30",
"Unit": "cpu_core"
},
{
"BriefDescription": "Read requests with true-miss in L2 cache.[This event is alias to L2_REQUEST.MISS]",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x24",
"EventName": "L2_RQSTS.MISS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts read requests of any type with true-miss in the L2 cache. True-miss excludes L2 misses that were merged with ongoing L2 misses.[This event is alias to L2_REQUEST.MISS]",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x3f",
"Unit": "cpu_core"
},
{
- "BriefDescription": "All accesses to L2 cache[This event is alias to L2_REQUEST.ALL]",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
- "EventCode": "0x24",
- "EventName": "L2_RQSTS.REFERENCES",
- "PEBScounters": "0,1,2,3",
+ "BriefDescription": "All accesses to L2 cache[This event is alias to L2_REQUEST.ALL]",
+ "EventCode": "0x24",
+ "EventName": "L2_RQSTS.REFERENCES",
+ "PublicDescription": "Counts all requests that were hit or true misses in L2 cache. True-miss excludes misses that were merged with ongoing L2 misses.[This event is alias to L2_REQUEST.ALL]",
+ "SampleAfterValue": "200003",
+ "UMask": "0xff",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "RFO requests that hit L2 cache.",
+ "EventCode": "0x24",
+ "EventName": "L2_RQSTS.RFO_HIT",
+ "PublicDescription": "Counts the RFO (Read-for-Ownership) requests that hit L2 cache.",
+ "SampleAfterValue": "200003",
+ "UMask": "0xc2",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "RFO requests that miss L2 cache",
+ "EventCode": "0x24",
+ "EventName": "L2_RQSTS.RFO_MISS",
+ "PublicDescription": "Counts the RFO (Read-for-Ownership) requests that miss L2 cache.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x22",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "SW prefetch requests that hit L2 cache.",
+ "EventCode": "0x24",
+ "EventName": "L2_RQSTS.SWPF_HIT",
+ "PublicDescription": "Counts Software prefetch requests that hit the L2 cache. Accounts for PREFETCHNTA and PREFETCHT0/1/2 instructions when FB is not full.",
+ "SampleAfterValue": "200003",
+ "UMask": "0xc8",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "SW prefetch requests that miss L2 cache.",
+ "EventCode": "0x24",
+ "EventName": "L2_RQSTS.SWPF_MISS",
+ "PublicDescription": "Counts Software prefetch requests that miss the L2 cache. Accounts for PREFETCHNTA and PREFETCHT0/1/2 instructions when FB is not full.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x28",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Counts the number of cacheable memory requests that miss in the LLC. Counts on a per core basis.",
+ "EventCode": "0x2e",
+ "EventName": "LONGEST_LAT_CACHE.MISS",
+ "PublicDescription": "Counts the number of cacheable memory requests that miss in the Last Level Cache (LLC). Requests include demand loads, reads for ownership (RFO), instruction fetches and L1 HW prefetches. If the platform has an L3 cache, the LLC is the L3 cache, otherwise it is the L2 cache. Counts on a per core basis.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x41",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Core-originated cacheable requests that missed L3 (Except hardware prefetches to the L3)",
+ "EventCode": "0x2e",
+ "EventName": "LONGEST_LAT_CACHE.MISS",
+ "PublicDescription": "Counts core-originated cacheable requests that miss the L3 cache (Longest Latency cache). Requests include data and code reads, Reads-for-Ownership (RFOs), speculative accesses and hardware prefetches to the L1 and L2. It does not include hardware prefetches to the L3, and may not count other types of requests to the L3.",
+ "SampleAfterValue": "100003",
+ "UMask": "0x41",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Counts the number of cacheable memory requests that access the LLC. Counts on a per core basis.",
+ "EventCode": "0x2e",
+ "EventName": "LONGEST_LAT_CACHE.REFERENCE",
+ "PublicDescription": "Counts the number of cacheable memory requests that access the Last Level Cache (LLC). Requests include demand loads, reads for ownership (RFO), instruction fetches and L1 HW prefetches. If the platform has an L3 cache, the LLC is the L3 cache, otherwise it is the L2 cache. Counts on a per core basis.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x4f",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Core-originated cacheable requests that refer to L3 (Except hardware prefetches to the L3)",
+ "EventCode": "0x2e",
+ "EventName": "LONGEST_LAT_CACHE.REFERENCE",
+ "PublicDescription": "Counts core-originated cacheable requests to the L3 cache (Longest Latency cache). Requests include data and code reads, Reads-for-Ownership (RFOs), speculative accesses and hardware prefetches to the L1 and L2. It does not include hardware prefetches to the L3, and may not count other types of requests to the L3.",
+ "SampleAfterValue": "100003",
+ "UMask": "0x4f",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in the L2, LLC, DRAM or MMIO (Non-DRAM).",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.IFETCH",
+ "PublicDescription": "Counts the number of cycles the core is stalled due to an instruction cache or translation lookaside buffer (TLB) miss which hit in the L2, LLC, DRAM or MMIO (Non-DRAM).",
+ "SampleAfterValue": "200003",
+ "UMask": "0x38",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in DRAM or MMIO (Non-DRAM).",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.IFETCH_DRAM_HIT",
+ "PublicDescription": "Counts the number of cycles the core is stalled due to an instruction cache or translation lookaside buffer (TLB) miss which hit in DRAM or MMIO (non-DRAM).",
"SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0xff",
- "Unit": "cpu_core"
+ "UMask": "0x20",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "RFO requests that hit L2 cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
- "EventCode": "0x24",
- "EventName": "L2_RQSTS.RFO_HIT",
- "PEBScounters": "0,1,2,3",
+ "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in the L2 cache.",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.IFETCH_L2_HIT",
+ "PublicDescription": "Counts the number of cycles the core is stalled due to an instruction cache or Translation Lookaside Buffer (TLB) miss which hit in the L2 cache.",
"SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0xc2",
- "Unit": "cpu_core"
+ "UMask": "0x8",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "RFO requests that miss L2 cache",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
- "EventCode": "0x24",
- "EventName": "L2_RQSTS.RFO_MISS",
- "PEBScounters": "0,1,2,3",
+ "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in the LLC or other core with HITE/F/M.",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.IFETCH_LLC_HIT",
+ "PublicDescription": "Counts the number of cycles the core is stalled due to an instruction cache or Translation Lookaside Buffer (TLB) miss which hit in the Last Level Cache (LLC) or other core with HITE/F/M.",
"SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x22",
- "Unit": "cpu_core"
+ "UMask": "0x10",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "SW prefetch requests that hit L2 cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
- "EventCode": "0x24",
- "EventName": "L2_RQSTS.SWPF_HIT",
- "PEBScounters": "0,1,2,3",
+ "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load miss which hit in the L2, LLC, DRAM or MMIO (Non-DRAM).",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.LOAD",
"SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0xc8",
- "Unit": "cpu_core"
+ "UMask": "0x7",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "SW prefetch requests that miss L2 cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
- "EventCode": "0x24",
- "EventName": "L2_RQSTS.SWPF_MISS",
- "PEBScounters": "0,1,2,3",
+ "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load miss which hit in DRAM or MMIO (Non-DRAM).",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.LOAD_DRAM_HIT",
"SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x28",
- "Unit": "cpu_core"
+ "UMask": "0x4",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Core-originated cacheable requests that missed L3 (Except hardware prefetches to the L3)",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
- "EventCode": "0x2e",
- "EventName": "LONGEST_LAT_CACHE.MISS",
- "PEBScounters": "0,1,2,3,4,5,6,7",
- "SampleAfterValue": "100003",
- "Speculative": "1",
- "UMask": "0x41",
- "Unit": "cpu_core"
+ "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load which hit in the L2 cache.",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.LOAD_L2_HIT",
+ "SampleAfterValue": "200003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Core-originated cacheable requests that refer to L3 (Except hardware prefetches to the L3)",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
- "EventCode": "0x2e",
- "EventName": "LONGEST_LAT_CACHE.REFERENCE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
- "SampleAfterValue": "100003",
- "Speculative": "1",
- "UMask": "0x4f",
- "Unit": "cpu_core"
+ "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load which hit in the LLC or other core with HITE/F/M.",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.LOAD_LLC_HIT",
+ "PublicDescription": "Counts the number of cycles the core is stalled due to a demand load which hit in the Last Level Cache (LLC) or other core with HITE/F/M.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
},
{
"BriefDescription": "Retired load instructions.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd0",
"EventName": "MEM_INST_RETIRED.ALL_LOADS",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts all retired load instructions. This event accounts for SW prefetch instructions of PREFETCHNTA or PREFETCHT0/1/2 or PREFETCHW.",
"SampleAfterValue": "1000003",
"UMask": "0x81",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired store instructions.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd0",
"EventName": "MEM_INST_RETIRED.ALL_STORES",
- "L1_Hit_Indication": "1",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts all retired store instructions.",
"SampleAfterValue": "1000003",
"UMask": "0x82",
"Unit": "cpu_core"
},
{
"BriefDescription": "All retired memory instructions.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd0",
"EventName": "MEM_INST_RETIRED.ANY",
- "L1_Hit_Indication": "1",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts all retired memory instructions - loads and stores.",
"SampleAfterValue": "1000003",
"UMask": "0x83",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired load instructions with locked access.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd0",
"EventName": "MEM_INST_RETIRED.LOCK_LOADS",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts retired load instructions with locked access.",
"SampleAfterValue": "100007",
"UMask": "0x21",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired load instructions that split across a cacheline boundary.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd0",
"EventName": "MEM_INST_RETIRED.SPLIT_LOADS",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts retired load instructions that split across a cacheline boundary.",
"SampleAfterValue": "100003",
"UMask": "0x41",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired store instructions that split across a cacheline boundary.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd0",
"EventName": "MEM_INST_RETIRED.SPLIT_STORES",
- "L1_Hit_Indication": "1",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts retired store instructions that split across a cacheline boundary.",
"SampleAfterValue": "100003",
"UMask": "0x42",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired load instructions that miss the STLB.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd0",
"EventName": "MEM_INST_RETIRED.STLB_MISS_LOADS",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Number of retired load instructions that (start a) miss in the 2nd-level TLB (STLB).",
"SampleAfterValue": "100003",
"UMask": "0x11",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired store instructions that miss the STLB.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd0",
"EventName": "MEM_INST_RETIRED.STLB_MISS_STORES",
- "L1_Hit_Indication": "1",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Number of retired store instructions that (start a) miss in the 2nd-level TLB (STLB).",
"SampleAfterValue": "100003",
"UMask": "0x12",
"Unit": "cpu_core"
},
{
"BriefDescription": "Completed demand load uops that miss the L1 d-cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x43",
"EventName": "MEM_LOAD_COMPLETED.L1_MISS_ANY",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Number of completed demand load requests that missed the L1 data cache including shadow misses (FB hits, merge to an ongoing L1D miss)",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0xfd",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired load instructions whose data sources were HitM responses from shared L3",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd2",
"EventName": "MEM_LOAD_L3_HIT_RETIRED.XSNP_FWD",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts retired load instructions whose data sources were HitM responses from shared L3.",
"SampleAfterValue": "20011",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired load instructions whose data sources were L3 and cross-core snoop hits in on-pkg core cache",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd2",
"EventName": "MEM_LOAD_L3_HIT_RETIRED.XSNP_HIT",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts retired load instructions whose data sources were L3 and cross-core snoop hits in on-pkg core cache.",
"SampleAfterValue": "20011",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired load instructions whose data sources were HitM responses from shared L3",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Data_LA": "1",
"EventCode": "0xd2",
"EventName": "MEM_LOAD_L3_HIT_RETIRED.XSNP_HITM",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts retired load instructions whose data sources were HitM responses from shared L3.",
+ "SampleAfterValue": "20011",
+ "UMask": "0x4",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Retired load instructions whose data sources were L3 hit and cross-core snoop missed in on-pkg core cache.",
+ "Data_LA": "1",
+ "EventCode": "0xd2",
+ "EventName": "MEM_LOAD_L3_HIT_RETIRED.XSNP_MISS",
+ "PEBS": "1",
+ "PublicDescription": "Counts the retired load instructions whose data sources were L3 hit and cross-core snoop missed in on-pkg core cache.",
+ "SampleAfterValue": "20011",
+ "UMask": "0x1",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Retired load instructions whose data sources were hits in L3 without snoops required",
+ "Data_LA": "1",
+ "EventCode": "0xd2",
+ "EventName": "MEM_LOAD_L3_HIT_RETIRED.XSNP_NONE",
+ "PEBS": "1",
+ "PublicDescription": "Counts retired load instructions whose data sources were hits in L3 without snoops required.",
+ "SampleAfterValue": "100003",
+ "UMask": "0x8",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Retired load instructions whose data sources were L3 and cross-core snoop hits in on-pkg core cache",
+ "Data_LA": "1",
+ "EventCode": "0xd2",
+ "EventName": "MEM_LOAD_L3_HIT_RETIRED.XSNP_NO_FWD",
+ "PEBS": "1",
+ "PublicDescription": "Counts retired load instructions whose data sources were L3 and cross-core snoop hits in on-pkg core cache.",
"SampleAfterValue": "20011",
+ "UMask": "0x2",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Retired load instructions which data sources missed L3 but serviced from local dram",
+ "Data_LA": "1",
+ "EventCode": "0xd3",
+ "EventName": "MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM",
+ "PEBS": "1",
+ "PublicDescription": "Retired load instructions which data sources missed L3 but serviced from local DRAM.",
+ "SampleAfterValue": "100007",
+ "UMask": "0x1",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Retired instructions with at least 1 uncacheable load or lock.",
+ "Data_LA": "1",
+ "EventCode": "0xd4",
+ "EventName": "MEM_LOAD_MISC_RETIRED.UC",
+ "PEBS": "1",
+ "PublicDescription": "Retired instructions with at least one load to uncacheable memory-type, or at least one cache-line split locked access (Bus Lock).",
+ "SampleAfterValue": "100007",
+ "UMask": "0x4",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Number of completed demand load requests that missed the L1, but hit the FB(fill buffer), because a preceding miss to the same cacheline initiated the line to be brought into L1, but data is not yet ready in L1.",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_RETIRED.FB_HIT",
+ "PEBS": "1",
+ "PublicDescription": "Counts retired load instructions with at least one uop was load missed in L1 but hit FB (Fill Buffers) due to preceding miss to the same cache line with data not ready.",
+ "SampleAfterValue": "100007",
+ "UMask": "0x40",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Retired load instructions with L1 cache hits as data sources",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_RETIRED.L1_HIT",
+ "PEBS": "1",
+ "PublicDescription": "Counts retired load instructions with at least one uop that hit in the L1 data cache. This event includes all SW prefetches and lock instructions regardless of the data source.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Retired load instructions missed L1 cache as data sources",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_RETIRED.L1_MISS",
+ "PEBS": "1",
+ "PublicDescription": "Counts retired load instructions with at least one uop that missed in the L1 cache.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x8",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Retired load instructions with L2 cache hits as data sources",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_RETIRED.L2_HIT",
+ "PEBS": "1",
+ "PublicDescription": "Counts retired load instructions with L2 cache hits as data sources.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x2",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Retired load instructions missed L2 cache as data sources",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_RETIRED.L2_MISS",
+ "PEBS": "1",
+ "PublicDescription": "Counts retired load instructions missed L2 cache as data sources.",
+ "SampleAfterValue": "100021",
+ "UMask": "0x10",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Retired load instructions with L3 cache hits as data sources",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_RETIRED.L3_HIT",
+ "PEBS": "1",
+ "PublicDescription": "Counts retired load instructions with at least one uop that hit in the L3 cache.",
+ "SampleAfterValue": "100021",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
- "BriefDescription": "Retired load instructions whose data sources were L3 hit and cross-core snoop missed in on-pkg core cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
- "Data_LA": "1",
- "EventCode": "0xd2",
- "EventName": "MEM_LOAD_L3_HIT_RETIRED.XSNP_MISS",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "20011",
+ "BriefDescription": "Retired load instructions missed L3 cache as data sources",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_RETIRED.L3_MISS",
+ "PEBS": "1",
+ "PublicDescription": "Counts retired load instructions with at least one uop that missed in the L3 cache.",
+ "SampleAfterValue": "50021",
+ "UMask": "0x20",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Counts the number of load uops retired that hit in DRAM.",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_UOPS_RETIRED.DRAM_HIT",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x80",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of load uops retired that hit in the L2 cache.",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_UOPS_RETIRED.L2_HIT",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of load uops retired that hit in the L3 cache.",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_UOPS_RETIRED.L3_HIT",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x4",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that uops are blocked for any of the following reasons: load buffer, store buffer or RSV full.",
+ "EventCode": "0x04",
+ "EventName": "MEM_SCHEDULER_BLOCK.ALL",
+ "SampleAfterValue": "20003",
+ "UMask": "0x7",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that uops are blocked due to a load buffer full condition.",
+ "EventCode": "0x04",
+ "EventName": "MEM_SCHEDULER_BLOCK.LD_BUF",
+ "SampleAfterValue": "20003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that uops are blocked due to an RSV full condition.",
+ "EventCode": "0x04",
+ "EventName": "MEM_SCHEDULER_BLOCK.RSV",
+ "SampleAfterValue": "20003",
+ "UMask": "0x4",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that uops are blocked due to a store buffer full condition.",
+ "EventCode": "0x04",
+ "EventName": "MEM_SCHEDULER_BLOCK.ST_BUF",
+ "SampleAfterValue": "20003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "MEM_STORE_RETIRED.L2_HIT",
+ "EventCode": "0x44",
+ "EventName": "MEM_STORE_RETIRED.L2_HIT",
+ "SampleAfterValue": "200003",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
- "BriefDescription": "Retired load instructions whose data sources were hits in L3 without snoops required",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of load uops retired.",
"Data_LA": "1",
- "EventCode": "0xd2",
- "EventName": "MEM_LOAD_L3_HIT_RETIRED.XSNP_NONE",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.ALL_LOADS",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "100003",
- "UMask": "0x8",
- "Unit": "cpu_core"
+ "PublicDescription": "Counts the total number of load uops retired.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x81",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Retired load instructions whose data sources were L3 and cross-core snoop hits in on-pkg core cache",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of store uops retired.",
"Data_LA": "1",
- "EventCode": "0xd2",
- "EventName": "MEM_LOAD_L3_HIT_RETIRED.XSNP_NO_FWD",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.ALL_STORES",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "20011",
- "UMask": "0x2",
- "Unit": "cpu_core"
+ "PublicDescription": "Counts the total number of store uops retired.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x82",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Retired load instructions which data sources missed L3 but serviced from local dram",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 128 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
"Data_LA": "1",
- "EventCode": "0xd3",
- "EventName": "MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "100007",
- "UMask": "0x1",
- "Unit": "cpu_core"
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_128",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x80",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 128 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Retired instructions with at least 1 uncacheable load or lock.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 16 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
"Data_LA": "1",
- "EventCode": "0xd4",
- "EventName": "MEM_LOAD_MISC_RETIRED.UC",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "100007",
- "UMask": "0x4",
- "Unit": "cpu_core"
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_16",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x10",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 16 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Number of completed demand load requests that missed the L1, but hit the FB(fill buffer), because a preceding miss to the same cacheline initiated the line to be brought into L1, but data is not yet ready in L1.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 256 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
"Data_LA": "1",
- "EventCode": "0xd1",
- "EventName": "MEM_LOAD_RETIRED.FB_HIT",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "100007",
- "UMask": "0x40",
- "Unit": "cpu_core"
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_256",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x100",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 256 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Retired load instructions with L1 cache hits as data sources",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 32 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
"Data_LA": "1",
- "EventCode": "0xd1",
- "EventName": "MEM_LOAD_RETIRED.L1_HIT",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_32",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x20",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 32 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
"SampleAfterValue": "1000003",
- "UMask": "0x1",
- "Unit": "cpu_core"
+ "UMask": "0x5",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Retired load instructions missed L1 cache as data sources",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 4 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
"Data_LA": "1",
- "EventCode": "0xd1",
- "EventName": "MEM_LOAD_RETIRED.L1_MISS",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "200003",
- "UMask": "0x8",
- "Unit": "cpu_core"
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_4",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x4",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 4 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Retired load instructions with L2 cache hits as data sources",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 512 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
"Data_LA": "1",
- "EventCode": "0xd1",
- "EventName": "MEM_LOAD_RETIRED.L2_HIT",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "200003",
- "UMask": "0x2",
- "Unit": "cpu_core"
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_512",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x200",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 512 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Retired load instructions missed L2 cache as data sources",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 64 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
"Data_LA": "1",
- "EventCode": "0xd1",
- "EventName": "MEM_LOAD_RETIRED.L2_MISS",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "100021",
- "UMask": "0x10",
- "Unit": "cpu_core"
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_64",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x40",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 64 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Retired load instructions with L3 cache hits as data sources",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 8 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
"Data_LA": "1",
- "EventCode": "0xd1",
- "EventName": "MEM_LOAD_RETIRED.L3_HIT",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "100021",
- "UMask": "0x4",
- "Unit": "cpu_core"
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_8",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x8",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 8 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Retired load instructions missed L3 cache as data sources",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "BriefDescription": "Counts the number of retired split load uops.",
"Data_LA": "1",
- "EventCode": "0xd1",
- "EventName": "MEM_LOAD_RETIRED.L3_MISS",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.SPLIT_LOADS",
"PEBS": "1",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "50021",
- "UMask": "0x20",
- "Unit": "cpu_core"
+ "SampleAfterValue": "200003",
+ "UMask": "0x41",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "MEM_STORE_RETIRED.L2_HIT",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
- "EventCode": "0x44",
- "EventName": "MEM_STORE_RETIRED.L2_HIT",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "200003",
- "UMask": "0x1",
- "Unit": "cpu_core"
+ "BriefDescription": "Counts the number of stores uops retired. Counts with or without PEBS enabled.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.STORE_LATENCY",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of stores uops retired. Counts with or without PEBS enabled. If PEBS is enabled and a PEBS record is generated, will populate PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x6",
+ "Unit": "cpu_atom"
},
{
"BriefDescription": "Retired memory uops for any access",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xe5",
"EventName": "MEM_UOP_RETIRED.ANY",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of retired micro-operations (uops) for load or store memory accesses",
"SampleAfterValue": "1000003",
"UMask": "0x3",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts demand data reads that were supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_HIT",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F803C0001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that were supplied by the L3 cache where a snoop was sent, the snoop hit, and modified data was forwarded.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HITM",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10003C0001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Counts demand data reads that resulted in a snoop hit in another cores caches, data forwarding is required as the data is modified.",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x2A,0x2B",
"EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HITM",
"MSRIndex": "0x1a6,0x1a7",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts demand data reads that were supplied by the L3 cache where a snoop was sent, the snoop hit, but no data was forwarded.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HIT_NO_FWD",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x4003C0001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that were supplied by the L3 cache where a snoop was sent, the snoop hit, and non-modified data was forwarded.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HIT_WITH_FWD",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x8003C0001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HIT_WITH_FWD",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x2A,0x2B",
"EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HIT_WITH_FWD",
"MSRIndex": "0x1a6,0x1a7",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_RFO.L3_HIT",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F803C0002",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were supplied by the L3 cache where a snoop was sent, the snoop hit, and modified data was forwarded.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_RFO.L3_HIT.SNOOP_HITM",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10003C0002",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Counts demand read for ownership (RFO) requests and software prefetches for exclusive ownership (PREFETCHW) that resulted in a snoop hit in another cores caches, data forwarding is required as the data is modified.",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x2A,0x2B",
"EventName": "OCR.DEMAND_RFO.L3_HIT.SNOOP_HITM",
"MSRIndex": "0x1a6,0x1a7",
},
{
"BriefDescription": "OFFCORE_REQUESTS.ALL_REQUESTS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x21",
"EventName": "OFFCORE_REQUESTS.ALL_REQUESTS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x80",
"Unit": "cpu_core"
},
{
"BriefDescription": "Demand and prefetch data reads",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x21",
"EventName": "OFFCORE_REQUESTS.DATA_RD",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the demand and prefetch data reads. All Core Data Reads include cacheable 'Demands' and L2 prefetchers (not L3 prefetchers). Counting also covers reads due to page walks resulted from any request type.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Demand Data Read requests sent to uncore",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x21",
"EventName": "OFFCORE_REQUESTS.DEMAND_DATA_RD",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the Demand Data Read requests sent to uncore. Use it in conjunction with OFFCORE_REQUESTS_OUTSTANDING to determine average latency in the uncore.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "This event is deprecated. Refer to new event OFFCORE_REQUESTS_OUTSTANDING.DATA_RD",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
+ "Deprecated": "1",
"Errata": "ADL038",
"EventCode": "0x20",
"EventName": "OFFCORE_REQUESTS_OUTSTANDING.ALL_DATA_RD",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_DATA_RD",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"Errata": "ADL038",
"EventCode": "0x20",
"EventName": "OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_DATA_RD",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "For every cycle where the core is waiting on at least 1 outstanding Demand RFO request, increments by 1.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EventCode": "0x20",
"EventName": "OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_DEMAND_RFO",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_DEMAND_RFO",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "OFFCORE_REQUESTS_OUTSTANDING.DATA_RD",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"Errata": "ADL038",
"EventCode": "0x20",
"EventName": "OFFCORE_REQUESTS_OUTSTANDING.DATA_RD",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of PREFETCHNTA instructions executed.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x40",
"EventName": "SW_PREFETCH_ACCESS.NTA",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of PREFETCHNTA instructions executed.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of PREFETCHW instructions executed.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x40",
"EventName": "SW_PREFETCH_ACCESS.PREFETCHW",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of PREFETCHW instructions executed.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of PREFETCHT0 instructions executed.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x40",
"EventName": "SW_PREFETCH_ACCESS.T0",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of PREFETCHT0 instructions executed.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of PREFETCHT1 or PREFETCHT2 instructions executed.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x40",
"EventName": "SW_PREFETCH_ACCESS.T1_T2",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of PREFETCHT1 or PREFETCHT2 instructions executed.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to instruction cache misses.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.ICACHE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x20",
+ "Unit": "cpu_atom"
}
]
[
{
- "BriefDescription": "Counts the number of floating point operations retired that required microcode assist.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc3",
- "EventName": "MACHINE_CLEARS.FP_ASSIST",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "20003",
- "Speculative": "1",
- "UMask": "0x4",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of floating point divide uops retired (x87 and SSE, including x87 sqrt).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc2",
- "EventName": "UOPS_RETIRED.FPDIV",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "UMask": "0x8",
- "Unit": "cpu_atom"
- },
- {
"BriefDescription": "ARITH.FPDIV_ACTIVE",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EventCode": "0xb0",
"EventName": "ARITH.FPDIV_ACTIVE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts all microcode FP assists.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc1",
"EventName": "ASSISTS.FP",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts all microcode Floating Point assists.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "ASSISTS.SSE_AVX_MIX",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc1",
"EventName": "ASSISTS.SSE_AVX_MIX",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "FP_ARITH_DISPATCHED.PORT_0",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb3",
"EventName": "FP_ARITH_DISPATCHED.PORT_0",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "FP_ARITH_DISPATCHED.PORT_1",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb3",
"EventName": "FP_ARITH_DISPATCHED.PORT_1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "FP_ARITH_DISPATCHED.PORT_5",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb3",
"EventName": "FP_ARITH_DISPATCHED.PORT_5",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts number of SSE/AVX computational 128-bit packed double precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 2 computation operations, one for each element. Applies to SSE* and AVX* packed double precision floating-point instructions: ADD SUB HADD HSUB SUBADD MUL DIV MIN MAX SQRT DPP FM(N)ADD/SUB. DPP and FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc7",
"EventName": "FP_ARITH_INST_RETIRED.128B_PACKED_DOUBLE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of SSE/AVX computational 128-bit packed double precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 2 computation operations, one for each element. Applies to SSE* and AVX* packed double precision floating-point instructions: ADD SUB HADD HSUB SUBADD MUL DIV MIN MAX SQRT DPP FM(N)ADD/SUB. DPP and FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element. The DAZ and FTZ flags in the MXCSR register need to be set when using these events.",
"SampleAfterValue": "100003",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of SSE/AVX computational 128-bit packed single precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 4 computation operations, one for each element. Applies to SSE* and AVX* packed single precision floating-point instructions: ADD SUB MUL DIV MIN MAX RCP14 RSQRT14 SQRT DPP FM(N)ADD/SUB. DPP and FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc7",
"EventName": "FP_ARITH_INST_RETIRED.128B_PACKED_SINGLE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of SSE/AVX computational 128-bit packed single precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 4 computation operations, one for each element. Applies to SSE* and AVX* packed single precision floating-point instructions: ADD SUB HADD HSUB SUBADD MUL DIV MIN MAX SQRT RSQRT RCP DPP FM(N)ADD/SUB. DPP and FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element. The DAZ and FTZ flags in the MXCSR register need to be set when using these events.",
"SampleAfterValue": "100003",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts number of SSE/AVX computational 256-bit packed double precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 4 computation operations, one for each element. Applies to SSE* and AVX* packed double precision floating-point instructions: ADD SUB HADD HSUB SUBADD MUL DIV MIN MAX SQRT FM(N)ADD/SUB. FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc7",
"EventName": "FP_ARITH_INST_RETIRED.256B_PACKED_DOUBLE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of SSE/AVX computational 256-bit packed double precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 4 computation operations, one for each element. Applies to SSE* and AVX* packed double precision floating-point instructions: ADD SUB HADD HSUB SUBADD MUL DIV MIN MAX SQRT FM(N)ADD/SUB. FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element. The DAZ and FTZ flags in the MXCSR register need to be set when using these events.",
"SampleAfterValue": "100003",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts number of SSE/AVX computational 256-bit packed single precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 8 computation operations, one for each element. Applies to SSE* and AVX* packed single precision floating-point instructions: ADD SUB HADD HSUB SUBADD MUL DIV MIN MAX SQRT RSQRT RCP DPP FM(N)ADD/SUB. DPP and FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc7",
"EventName": "FP_ARITH_INST_RETIRED.256B_PACKED_SINGLE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of SSE/AVX computational 256-bit packed single precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 8 computation operations, one for each element. Applies to SSE* and AVX* packed single precision floating-point instructions: ADD SUB HADD HSUB SUBADD MUL DIV MIN MAX SQRT RSQRT RCP DPP FM(N)ADD/SUB. DPP and FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element. The DAZ and FTZ flags in the MXCSR register need to be set when using these events.",
"SampleAfterValue": "100003",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts number of SSE/AVX computational scalar double precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 1 computational operation. Applies to SSE* and AVX* scalar double precision floating-point instructions: ADD SUB MUL DIV MIN MAX SQRT FM(N)ADD/SUB. FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc7",
"EventName": "FP_ARITH_INST_RETIRED.SCALAR_DOUBLE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of SSE/AVX computational scalar double precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 1 computational operation. Applies to SSE* and AVX* scalar double precision floating-point instructions: ADD SUB MUL DIV MIN MAX SQRT FM(N)ADD/SUB. FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element. The DAZ and FTZ flags in the MXCSR register need to be set when using these events.",
"SampleAfterValue": "100003",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts number of SSE/AVX computational scalar single precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 1 computational operation. Applies to SSE* and AVX* scalar single precision floating-point instructions: ADD SUB MUL DIV MIN MAX SQRT RSQRT RCP FM(N)ADD/SUB. FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc7",
"EventName": "FP_ARITH_INST_RETIRED.SCALAR_SINGLE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of SSE/AVX computational scalar single precision floating-point instructions retired; some instructions will count twice as noted below. Each count represents 1 computational operation. Applies to SSE* and AVX* scalar single precision floating-point instructions: ADD SUB MUL DIV MIN MAX SQRT RSQRT RCP FM(N)ADD/SUB. FM(N)ADD/SUB instructions count twice as they perform 2 calculations per element. The DAZ and FTZ flags in the MXCSR register need to be set when using these events.",
"SampleAfterValue": "100003",
"UMask": "0x2",
"Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Counts the number of floating point operations retired that required microcode assist.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.FP_ASSIST",
+ "PublicDescription": "Counts the number of floating point operations retired that required microcode assist, which is not a reflection of the number of FP operations, instructions or uops.",
+ "SampleAfterValue": "20003",
+ "UMask": "0x4",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of floating point divide uops retired (x87 and SSE, including x87 sqrt).",
+ "EventCode": "0xc2",
+ "EventName": "UOPS_RETIRED.FPDIV",
+ "PEBS": "1",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x8",
+ "Unit": "cpu_atom"
}
]
[
{
"BriefDescription": "Counts the total number of BACLEARS due to all branch types including conditional and unconditional jumps, returns, and indirect branches.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
"EventCode": "0xe6",
"EventName": "BACLEARS.ANY",
- "PEBScounters": "0,1,2,3,4,5",
+ "PublicDescription": "Counts the total number of BACLEARS, which occur when the Branch Target Buffer (BTB) prediction or lack thereof, was corrected by a later branch predictor in the frontend. Includes BACLEARS due to all branch types including conditional and unconditional jumps, returns, and indirect branches.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_atom"
},
{
- "BriefDescription": "Counts the number of requests to the instruction cache for one or more bytes of a cache line.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x80",
- "EventName": "ICACHE.ACCESSES",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x3",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of instruction cache misses.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x80",
- "EventName": "ICACHE.MISSES",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
"BriefDescription": "Stalls caused by changing prefix length of the instruction.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x87",
"EventName": "DECODE.LCP",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts cycles that the Instruction Length decoder (ILD) stalls occurred due to dynamically changing prefix length of the decoded instruction (by operand size prefix instruction 0x66, address size prefix instruction 0x67 or REX.W for Intel64). Count is proportional to the number of prefixes in a 16B-line. This may result in a three-cycle penalty for each LCP (Length changing prefix) in a 16-byte chunk.",
"SampleAfterValue": "500009",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles the Microcode Sequencer is busy.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x87",
"EventName": "DECODE.MS_BUSY",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "500009",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "DSB-to-MITE switch true penalty cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x61",
"EventName": "DSB2MITE_SWITCHES.PENALTY_CYCLES",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Decode Stream Buffer (DSB) is a Uop-cache that holds translations of previously fetched instructions that were decoded by the legacy x86 decode pipeline (MITE). This event counts fetch penalty cycles when a transition occurs from DSB to MITE.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired Instructions who experienced DSB miss.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.ANY_DSB_MISS",
"MSRIndex": "0x3F7",
"MSRValue": "0x1",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired Instructions that experienced DSB (Decode stream buffer i.e. the decoded instruction-cache) miss.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired Instructions who experienced a critical DSB miss.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.DSB_MISS",
"MSRIndex": "0x3F7",
"MSRValue": "0x11",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of retired Instructions that experienced a critical DSB (Decode stream buffer i.e. the decoded instruction-cache) miss. Critical means stalls were exposed to the back-end as a result of the DSB miss.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired Instructions who experienced iTLB true miss.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.ITLB_MISS",
"MSRIndex": "0x3F7",
"MSRValue": "0x14",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired Instructions that experienced iTLB (Instruction TLB) true miss.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired Instructions who experienced Instruction L1 Cache true miss.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.L1I_MISS",
"MSRIndex": "0x3F7",
"MSRValue": "0x12",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired Instructions who experienced Instruction L1 Cache true miss.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired Instructions who experienced Instruction L2 Cache true miss.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.L2_MISS",
"MSRIndex": "0x3F7",
"MSRValue": "0x13",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired Instructions who experienced Instruction L2 Cache true miss.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions after front-end starvation of at least 1 cycle",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_1",
"MSRIndex": "0x3F7",
"MSRValue": "0x600106",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Retired instructions that are fetched after an interval where the front-end delivered no uops for a period of at least 1 cycle which was not interrupted by a back-end stall.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 128 cycles which was not interrupted by a back-end stall.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_128",
"MSRIndex": "0x3F7",
"MSRValue": "0x608006",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 128 cycles which was not interrupted by a back-end stall.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 16 cycles which was not interrupted by a back-end stall.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_16",
"MSRIndex": "0x3F7",
"MSRValue": "0x601006",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired instructions that are delivered to the back-end after a front-end stall of at least 16 cycles. During this period the front-end delivered no uops.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions after front-end starvation of at least 2 cycles",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_2",
"MSRIndex": "0x3F7",
"MSRValue": "0x600206",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Retired instructions that are fetched after an interval where the front-end delivered no uops for a period of at least 2 cycles which was not interrupted by a back-end stall.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 256 cycles which was not interrupted by a back-end stall.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_256",
"MSRIndex": "0x3F7",
"MSRValue": "0x610006",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 256 cycles which was not interrupted by a back-end stall.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions that are fetched after an interval where the front-end had at least 1 bubble-slot for a period of 2 cycles which was not interrupted by a back-end stall.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_2_BUBBLES_GE_1",
"MSRIndex": "0x3F7",
"MSRValue": "0x100206",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired instructions that are delivered to the back-end after the front-end had at least 1 bubble-slot for a period of 2 cycles. A bubble-slot is an empty issue-pipeline slot while there was no RAT stall.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 32 cycles which was not interrupted by a back-end stall.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_32",
"MSRIndex": "0x3F7",
"MSRValue": "0x602006",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired instructions that are delivered to the back-end after a front-end stall of at least 32 cycles. During this period the front-end delivered no uops.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 4 cycles which was not interrupted by a back-end stall.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_4",
"MSRIndex": "0x3F7",
"MSRValue": "0x600406",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 4 cycles which was not interrupted by a back-end stall.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 512 cycles which was not interrupted by a back-end stall.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_512",
"MSRIndex": "0x3F7",
"MSRValue": "0x620006",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 512 cycles which was not interrupted by a back-end stall.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 64 cycles which was not interrupted by a back-end stall.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_64",
"MSRIndex": "0x3F7",
"MSRValue": "0x604006",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 64 cycles which was not interrupted by a back-end stall.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired instructions that are fetched after an interval where the front-end delivered no uops for a period of 8 cycles which was not interrupted by a back-end stall.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.LATENCY_GE_8",
"MSRIndex": "0x3F7",
"MSRValue": "0x600806",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired instructions that are delivered to the back-end after a front-end stall of at least 8 cycles. During this period the front-end delivered no uops.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "FRONTEND_RETIRED.MS_FLOWS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.MS_FLOWS",
"MSRIndex": "0x3F7",
"MSRValue": "0x8",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired Instructions who experienced STLB (2nd level TLB) true miss.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.STLB_MISS",
"MSRIndex": "0x3F7",
"MSRValue": "0x15",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired Instructions that experienced STLB (2nd level TLB) true miss.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "FRONTEND_RETIRED.UNKNOWN_BRANCH",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc6",
"EventName": "FRONTEND_RETIRED.UNKNOWN_BRANCH",
"MSRIndex": "0x3F7",
"MSRValue": "0x17",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of requests to the instruction cache for one or more bytes of a cache line.",
+ "EventCode": "0x80",
+ "EventName": "ICACHE.ACCESSES",
+ "PublicDescription": "Counts the total number of requests to the instruction cache. The event only counts new cache line accesses, so that multiple back to back fetches to the exact same cache line or byte chunk count as one. Specifically, the event counts when accesses from sequential code crosses the cache line boundary, or when a branch target is moved to a new line or to a non-sequential byte chunk of the same line.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x3",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of instruction cache misses.",
+ "EventCode": "0x80",
+ "EventName": "ICACHE.MISSES",
+ "PublicDescription": "Counts the number of missed requests to the instruction cache. The event only counts new cache line accesses, so that multiple back to back fetches to the exact same cache line and byte chunk count as one. Specifically, the event counts when accesses from sequential code crosses the cache line boundary, or when a branch target is moved to a new line or to a non-sequential byte chunk of the same line.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Cycles where a code fetch is stalled due to L1 instruction cache miss.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x80",
"EventName": "ICACHE_DATA.STALLS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts cycles where a code line fetch is stalled due to an L1 instruction cache miss. The decode pipeline works at a 32 Byte granularity.",
"SampleAfterValue": "500009",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles where a code fetch is stalled due to L1 instruction cache tag miss.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x83",
"EventName": "ICACHE_TAG.STALLS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts cycles where a code fetch is stalled due to L1 instruction cache tag miss.",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles Decode Stream Buffer (DSB) is delivering any Uop",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EventCode": "0x79",
"EventName": "IDQ.DSB_CYCLES_ANY",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of cycles uops were delivered to Instruction Decode Queue (IDQ) from the Decode Stream Buffer (DSB) path.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles DSB is delivering optimal number of Uops",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "6",
"EventCode": "0x79",
"EventName": "IDQ.DSB_CYCLES_OK",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of cycles where optimal number of uops was delivered to the Instruction Decode Queue (IDQ) from the MITE (legacy decode pipeline) path. During these cycles uops are not being delivered from the Decode Stream Buffer (DSB).",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops delivered to Instruction Decode Queue (IDQ) from the Decode Stream Buffer (DSB) path",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x79",
"EventName": "IDQ.DSB_UOPS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of uops delivered to Instruction Decode Queue (IDQ) from the Decode Stream Buffer (DSB) path.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles MITE is delivering any Uop",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EventCode": "0x79",
"EventName": "IDQ.MITE_CYCLES_ANY",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of cycles uops were delivered to the Instruction Decode Queue (IDQ) from the MITE (legacy decode pipeline) path. During these cycles uops are not being delivered from the Decode Stream Buffer (DSB).",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles MITE is delivering optimal number of Uops",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "6",
"EventCode": "0x79",
"EventName": "IDQ.MITE_CYCLES_OK",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of cycles where optimal number of uops was delivered to the Instruction Decode Queue (IDQ) from the MITE (legacy decode pipeline) path. During these cycles uops are not being delivered from the Decode Stream Buffer (DSB).",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops delivered to Instruction Decode Queue (IDQ) from MITE path",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x79",
"EventName": "IDQ.MITE_UOPS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of uops delivered to Instruction Decode Queue (IDQ) from the MITE path. This also means that uops are not being delivered from the Decode Stream Buffer (DSB).",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles when uops are being delivered to IDQ while MS is busy",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EventCode": "0x79",
"EventName": "IDQ.MS_CYCLES_ANY",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts cycles during which uops are being delivered to Instruction Decode Queue (IDQ) while the Microcode Sequencer (MS) is busy. Uops maybe initiated by Decode Stream Buffer (DSB) or MITE.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of switches from DSB or MITE to the MS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EdgeDetect": "1",
"EventCode": "0x79",
"EventName": "IDQ.MS_SWITCHES",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Number of switches from DSB (Decode Stream Buffer) or MITE (legacy decode pipeline) to the Microcode Sequencer.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops delivered to IDQ while MS is busy",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x79",
"EventName": "IDQ.MS_UOPS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the total number of uops delivered by the Microcode Sequencer (MS).",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops not delivered by IDQ when backend of the machine is not stalled",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x9c",
"EventName": "IDQ_UOPS_NOT_DELIVERED.CORE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of uops not delivered to by the Instruction Decode Queue (IDQ) to the back-end of the pipeline when there was no back-end stalls. This event counts for one SMT thread in a given cycle.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles when no uops are not delivered by the IDQ when backend of the machine is not stalled",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "6",
"EventCode": "0x9c",
"EventName": "IDQ_UOPS_NOT_DELIVERED.CYCLES_0_UOPS_DELIV.CORE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of cycles when no uops were delivered by the Instruction Decode Queue (IDQ) to the back-end of the pipeline when there was no back-end stalls. This event counts for one SMT thread in a given cycle.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles when optimal number of uops was delivered to the back-end when the back-end is not stalled",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EventCode": "0x9c",
"EventName": "IDQ_UOPS_NOT_DELIVERED.CYCLES_FE_WAS_OK",
"Invert": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of cycles when the optimal number of uops were delivered by the Instruction Decode Queue (IDQ) to the back-end of the pipeline when there was no back-end stalls. This event counts for one SMT thread in a given cycle.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
}
[
{
+ "BriefDescription": "Execution stalls while L3 cache miss demand load is outstanding.",
+ "CounterMask": "6",
+ "EventCode": "0xa3",
+ "EventName": "CYCLE_ACTIVITY.STALLS_L3_MISS",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x6",
+ "Unit": "cpu_core"
+ },
+ {
"BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer is stalled due to any number of reasons, including an L1 miss, WCB full, pagewalk, store address block or store data block, on a load that retires.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
"EventCode": "0x05",
"EventName": "LD_HEAD.ANY_AT_RET",
- "PEBScounters": "0,1,2,3,4,5",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0xff",
"Unit": "cpu_atom"
},
{
"BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer is stalled due to a core bound stall including a store address match, a DTLB miss or a page walk that detains the load from retiring.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
"EventCode": "0x05",
"EventName": "LD_HEAD.L1_BOUND_AT_RET",
- "PEBScounters": "0,1,2,3,4,5",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0xf4",
"Unit": "cpu_atom"
},
{
"BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to other block cases.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
"EventCode": "0x05",
"EventName": "LD_HEAD.OTHER_AT_RET",
- "PEBScounters": "0,1,2,3,4,5",
+ "PublicDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to other block cases such as pipeline conflicts, fences, etc.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0xc0",
"Unit": "cpu_atom"
},
{
"BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to a pagewalk.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
"EventCode": "0x05",
"EventName": "LD_HEAD.PGWALK_AT_RET",
- "PEBScounters": "0,1,2,3,4,5",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0xa0",
"Unit": "cpu_atom"
},
{
"BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to a store address match.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
"EventCode": "0x05",
"EventName": "LD_HEAD.ST_ADDR_AT_RET",
- "PEBScounters": "0,1,2,3,4,5",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x84",
"Unit": "cpu_atom"
},
{
"BriefDescription": "Counts the number of machine clears due to memory ordering caused by a snoop from an external agent. Does not count internally generated machine clears such as those due to memory disambiguation.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
"EventCode": "0xc3",
"EventName": "MACHINE_CLEARS.MEMORY_ORDERING",
- "PEBScounters": "0,1,2,3,4,5",
"SampleAfterValue": "20003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_atom"
},
{
- "BriefDescription": "Counts demand data reads that were not supplied by the L3 cache.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_DATA_RD.L3_MISS",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x3F84400001",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand data reads that were not supplied by the L3 cache.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_DATA_RD.L3_MISS_LOCAL",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x3F84400001",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were not supplied by the L3 cache.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_RFO.L3_MISS",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x3F84400002",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were not supplied by the L3 cache.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_RFO.L3_MISS_LOCAL",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x3F84400002",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Execution stalls while L3 cache miss demand load is outstanding.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
- "CounterMask": "6",
- "EventCode": "0xa3",
- "EventName": "CYCLE_ACTIVITY.STALLS_L3_MISS",
- "PEBScounters": "0,1,2,3",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x6",
- "Unit": "cpu_core"
- },
- {
"BriefDescription": "Number of machine clears due to memory ordering conflicts.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc3",
"EventName": "MACHINE_CLEARS.MEMORY_ORDERING",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of Machine Clears detected dye to memory ordering. Memory Ordering Machine Clears may apply when a memory read may not conform to the memory ordering rules of the x86 architecture",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles while L1 cache miss demand load is outstanding.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "2",
"EventCode": "0x47",
"EventName": "MEMORY_ACTIVITY.CYCLES_L1D_MISS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Execution stalls while L1 cache miss demand load is outstanding.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "3",
"EventCode": "0x47",
"EventName": "MEMORY_ACTIVITY.STALLS_L1D_MISS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x3",
"Unit": "cpu_core"
},
{
"BriefDescription": "MEMORY_ACTIVITY.STALLS_L2_MISS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "5",
"EventCode": "0x47",
"EventName": "MEMORY_ACTIVITY.STALLS_L2_MISS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x5",
"Unit": "cpu_core"
},
{
"BriefDescription": "MEMORY_ACTIVITY.STALLS_L3_MISS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "9",
"EventCode": "0x47",
"EventName": "MEMORY_ACTIVITY.STALLS_L3_MISS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x9",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 128 cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "1,2,3,4,5,6,7",
"Data_LA": "1",
"EventCode": "0xcd",
"EventName": "MEM_TRANS_RETIRED.LOAD_LATENCY_GT_128",
"MSRIndex": "0x3F6",
"MSRValue": "0x80",
"PEBS": "2",
- "PEBScounters": "1,2,3,4,5,6,7",
+ "PublicDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 128 cycles. Reported latency may be longer than just the memory latency.",
"SampleAfterValue": "1009",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 16 cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "1,2,3,4,5,6,7",
"Data_LA": "1",
"EventCode": "0xcd",
"EventName": "MEM_TRANS_RETIRED.LOAD_LATENCY_GT_16",
"MSRIndex": "0x3F6",
"MSRValue": "0x10",
"PEBS": "2",
- "PEBScounters": "1,2,3,4,5,6,7",
+ "PublicDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 16 cycles. Reported latency may be longer than just the memory latency.",
"SampleAfterValue": "20011",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 256 cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "1,2,3,4,5,6,7",
"Data_LA": "1",
"EventCode": "0xcd",
"EventName": "MEM_TRANS_RETIRED.LOAD_LATENCY_GT_256",
"MSRIndex": "0x3F6",
"MSRValue": "0x100",
"PEBS": "2",
- "PEBScounters": "1,2,3,4,5,6,7",
+ "PublicDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 256 cycles. Reported latency may be longer than just the memory latency.",
"SampleAfterValue": "503",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 32 cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "1,2,3,4,5,6,7",
"Data_LA": "1",
"EventCode": "0xcd",
"EventName": "MEM_TRANS_RETIRED.LOAD_LATENCY_GT_32",
"MSRIndex": "0x3F6",
"MSRValue": "0x20",
"PEBS": "2",
- "PEBScounters": "1,2,3,4,5,6,7",
+ "PublicDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 32 cycles. Reported latency may be longer than just the memory latency.",
"SampleAfterValue": "100007",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 4 cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "1,2,3,4,5,6,7",
"Data_LA": "1",
"EventCode": "0xcd",
"EventName": "MEM_TRANS_RETIRED.LOAD_LATENCY_GT_4",
"MSRIndex": "0x3F6",
"MSRValue": "0x4",
"PEBS": "2",
- "PEBScounters": "1,2,3,4,5,6,7",
+ "PublicDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 4 cycles. Reported latency may be longer than just the memory latency.",
"SampleAfterValue": "100003",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 512 cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "1,2,3,4,5,6,7",
"Data_LA": "1",
"EventCode": "0xcd",
"EventName": "MEM_TRANS_RETIRED.LOAD_LATENCY_GT_512",
"MSRIndex": "0x3F6",
"MSRValue": "0x200",
"PEBS": "2",
- "PEBScounters": "1,2,3,4,5,6,7",
+ "PublicDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 512 cycles. Reported latency may be longer than just the memory latency.",
"SampleAfterValue": "101",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 64 cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "1,2,3,4,5,6,7",
"Data_LA": "1",
"EventCode": "0xcd",
"EventName": "MEM_TRANS_RETIRED.LOAD_LATENCY_GT_64",
"MSRIndex": "0x3F6",
"MSRValue": "0x40",
"PEBS": "2",
- "PEBScounters": "1,2,3,4,5,6,7",
+ "PublicDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 64 cycles. Reported latency may be longer than just the memory latency.",
"SampleAfterValue": "2003",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 8 cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "1,2,3,4,5,6,7",
"Data_LA": "1",
"EventCode": "0xcd",
"EventName": "MEM_TRANS_RETIRED.LOAD_LATENCY_GT_8",
"MSRIndex": "0x3F6",
"MSRValue": "0x8",
"PEBS": "2",
- "PEBScounters": "1,2,3,4,5,6,7",
+ "PublicDescription": "Counts randomly selected loads when the latency from first dispatch to completion is greater than 8 cycles. Reported latency may be longer than just the memory latency.",
"SampleAfterValue": "50021",
- "TakenAlone": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired memory store access operations. A PDist event for PEBS Store Latency Facility.",
- "CollectPEBSRecord": "2",
"Data_LA": "1",
"EventCode": "0xcd",
"EventName": "MEM_TRANS_RETIRED.STORE_SAMPLE",
"PEBS": "2",
+ "PublicDescription": "Counts Retired memory accesses with at least 1 store operation. This PEBS event is the precisely-distributed (PDist) trigger covering all stores uops for sampling by the PEBS Store Latency Facility. The facility is described in Intel SDM Volume 3 section 19.9.8",
"SampleAfterValue": "1000003",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts demand data reads that were not supplied by the L3 cache.",
- "Counter": "0,1,2,3,4,5,6,7",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_MISS",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F84400001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that were not supplied by the L3 cache.",
"EventCode": "0x2A,0x2B",
"EventName": "OCR.DEMAND_DATA_RD.L3_MISS",
"MSRIndex": "0x1a6,0x1a7",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts demand data reads that were not supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_MISS_LOCAL",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F84400001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were not supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_RFO.L3_MISS",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F84400002",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Counts demand read for ownership (RFO) requests and software prefetches for exclusive ownership (PREFETCHW) that were not supplied by the L3 cache.",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x2A,0x2B",
"EventName": "OCR.DEMAND_RFO.L3_MISS",
"MSRIndex": "0x1a6,0x1a7",
"SampleAfterValue": "100003",
"UMask": "0x1",
"Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were not supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_RFO.L3_MISS_LOCAL",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F84400002",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Demand Data Read requests who miss L3 cache",
+ "EventCode": "0x21",
+ "EventName": "OFFCORE_REQUESTS.L3_MISS_DEMAND_DATA_RD",
+ "PublicDescription": "Demand Data Read requests who miss L3 cache.",
+ "SampleAfterValue": "100003",
+ "UMask": "0x10",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "For every cycle, increments by the number of demand data read requests pending that are known to have missed the L3 cache.",
+ "EventCode": "0x20",
+ "EventName": "OFFCORE_REQUESTS_OUTSTANDING.L3_MISS_DEMAND_DATA_RD",
+ "PublicDescription": "For every cycle, increments by the number of demand data read requests pending that are known to have missed the L3 cache. Note that this does not capture all elapsed cycles while requests are outstanding - only cycles from when the requests were known by the requesting core to have missed the L3 cache.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x10",
+ "Unit": "cpu_core"
}
]
[
{
- "BriefDescription": "Counts modified writebacks from L1 cache and L2 cache that have any type of response.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.COREWB_M.ANY_RESPONSE",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x10008",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand data reads that have any type of response.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_DATA_RD.ANY_RESPONSE",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x10001",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that have any type of response.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.DEMAND_RFO.ANY_RESPONSE",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x10002",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts streaming stores that have any type of response.",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xB7",
- "EventName": "OCR.STREAMING_WR.ANY_RESPONSE",
- "MSRIndex": "0x1a6,0x1a7",
- "MSRValue": "0x10800",
- "SampleAfterValue": "100003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
"BriefDescription": "ASSISTS.HARDWARE",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc1",
"EventName": "ASSISTS.HARDWARE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "ASSISTS.PAGE_FAULT",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc1",
"EventName": "ASSISTS.PAGE_FAULT",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "CORE_POWER.LICENSE_1",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x28",
"EventName": "CORE_POWER.LICENSE_1",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "CORE_POWER.LICENSE_2",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x28",
"EventName": "CORE_POWER.LICENSE_2",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "CORE_POWER.LICENSE_3",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x28",
"EventName": "CORE_POWER.LICENSE_3",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "200003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts modified writebacks from L1 cache and L2 cache that have any type of response.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.COREWB_M.ANY_RESPONSE",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10008",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that have any type of response.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.ANY_RESPONSE",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Counts demand data reads that have any type of response.",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x2A,0x2B",
"EventName": "OCR.DEMAND_DATA_RD.ANY_RESPONSE",
"MSRIndex": "0x1a6,0x1a7",
},
{
"BriefDescription": "Counts demand data reads that were supplied by DRAM.",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x2A,0x2B",
"EventName": "OCR.DEMAND_DATA_RD.DRAM",
"MSRIndex": "0x1a6,0x1a7",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that have any type of response.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_RFO.ANY_RESPONSE",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10002",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Counts demand read for ownership (RFO) requests and software prefetches for exclusive ownership (PREFETCHW) that have any type of response.",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x2A,0x2B",
"EventName": "OCR.DEMAND_RFO.ANY_RESPONSE",
"MSRIndex": "0x1a6,0x1a7",
},
{
"BriefDescription": "Counts streaming stores that have any type of response.",
- "Counter": "0,1,2,3,4,5,6,7",
+ "EventCode": "0xB7",
+ "EventName": "OCR.STREAMING_WR.ANY_RESPONSE",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10800",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts streaming stores that have any type of response.",
"EventCode": "0x2A,0x2B",
"EventName": "OCR.STREAMING_WR.ANY_RESPONSE",
"MSRIndex": "0x1a6,0x1a7",
},
{
"BriefDescription": "Cycles when Reservation Station (RS) is empty for the thread.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa5",
"EventName": "RS.EMPTY",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles during which the reservation station (RS) is empty for this logical processor. This is usually caused when the front-end pipeline runs into starvation periods (e.g. branch mispredictions or i-cache misses)",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x7",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts end of periods where the Reservation Station (RS) was empty.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EdgeDetect": "1",
"EventCode": "0xa5",
"EventName": "RS.EMPTY_COUNT",
"Invert": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts end of periods where the Reservation Station (RS) was empty. Could be useful to closely sample on front-end latency issues (see the FRONTEND_RETIRED event of designated precise events)",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x7",
"Unit": "cpu_core"
},
{
"BriefDescription": "This event is deprecated. Refer to new event RS.EMPTY_COUNT",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
+ "Deprecated": "1",
"EdgeDetect": "1",
"EventCode": "0xa5",
"EventName": "RS_EMPTY.COUNT",
"Invert": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x7",
"Unit": "cpu_core"
},
{
"BriefDescription": "This event is deprecated. Refer to new event RS.EMPTY",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
+ "Deprecated": "1",
"EventCode": "0xa5",
"EventName": "RS_EMPTY.CYCLES",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x7",
"Unit": "cpu_core"
},
{
"BriefDescription": "XQ.FULL_CYCLES",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EventCode": "0x2d",
"EventName": "XQ.FULL_CYCLES",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
}
[
{
- "BriefDescription": "Counts the total number of branch instructions retired for all branch types.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.ALL_BRANCHES",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.NEAR_CALL",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.CALL",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xf9",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of retired JCC (Jump on Conditional Code) branch instructions retired, includes both taken and not taken branches.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.COND",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0x7e",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of taken JCC (Jump on Conditional Code) branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.COND_TAKEN",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xfe",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of far branch instructions retired, includes far jump, far call and return, and interrupt call and return.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.FAR_BRANCH",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xbf",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of near indirect JMP and near indirect CALL branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.INDIRECT",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xeb",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of near indirect CALL branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.INDIRECT_CALL",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xfb",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.INDIRECT_CALL",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.IND_CALL",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xfb",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.COND",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.JCC",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0x7e",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of near CALL branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.NEAR_CALL",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xf9",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of near RET branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.NEAR_RETURN",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xf7",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.INDIRECT",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.NON_RETURN_IND",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xeb",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of near relative CALL branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.REL_CALL",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xfd",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.NEAR_RETURN",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.RETURN",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xf7",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.COND_TAKEN",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc4",
- "EventName": "BR_INST_RETIRED.TAKEN_JCC",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xfe",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the total number of mispredicted branch instructions retired for all branch types.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc5",
- "EventName": "BR_MISP_RETIRED.ALL_BRANCHES",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of mispredicted JCC (Jump on Conditional Code) branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc5",
- "EventName": "BR_MISP_RETIRED.COND",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0x7e",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of mispredicted taken JCC (Jump on Conditional Code) branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc5",
- "EventName": "BR_MISP_RETIRED.COND_TAKEN",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xfe",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of mispredicted near indirect JMP and near indirect CALL branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc5",
- "EventName": "BR_MISP_RETIRED.INDIRECT",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xeb",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of mispredicted near indirect CALL branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc5",
- "EventName": "BR_MISP_RETIRED.INDIRECT_CALL",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xfb",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.INDIRECT_CALL",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc5",
- "EventName": "BR_MISP_RETIRED.IND_CALL",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xfb",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.COND",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc5",
- "EventName": "BR_MISP_RETIRED.JCC",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0x7e",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.INDIRECT",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc5",
- "EventName": "BR_MISP_RETIRED.NON_RETURN_IND",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xeb",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of mispredicted near RET branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc5",
- "EventName": "BR_MISP_RETIRED.RETURN",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xf7",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.COND_TAKEN",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc5",
- "EventName": "BR_MISP_RETIRED.TAKEN_JCC",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "UMask": "0xfe",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of unhalted core clock cycles. (Fixed event)",
- "CollectPEBSRecord": "2",
- "Counter": "Fixed counter 1",
- "EventName": "CPU_CLK_UNHALTED.CORE",
- "PEBScounters": "33",
- "SampleAfterValue": "2000003",
- "Speculative": "1",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of unhalted core clock cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x3c",
- "EventName": "CPU_CLK_UNHALTED.CORE_P",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "Speculative": "1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of unhalted reference clock cycles at TSC frequency. (Fixed event)",
- "CollectPEBSRecord": "2",
- "Counter": "Fixed counter 2",
- "EventName": "CPU_CLK_UNHALTED.REF_TSC",
- "PEBScounters": "34",
- "SampleAfterValue": "2000003",
- "Speculative": "1",
- "UMask": "0x3",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of unhalted reference clock cycles at TSC frequency.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x3c",
- "EventName": "CPU_CLK_UNHALTED.REF_TSC_P",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "Speculative": "1",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of unhalted core clock cycles. (Fixed event)",
- "CollectPEBSRecord": "2",
- "Counter": "Fixed counter 1",
- "EventName": "CPU_CLK_UNHALTED.THREAD",
- "PEBScounters": "33",
- "SampleAfterValue": "2000003",
- "Speculative": "1",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of unhalted core clock cycles.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x3c",
- "EventName": "CPU_CLK_UNHALTED.THREAD_P",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "Speculative": "1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the total number of instructions retired. (Fixed event)",
- "CollectPEBSRecord": "2",
- "Counter": "Fixed counter 0",
- "EventName": "INST_RETIRED.ANY",
- "PEBS": "1",
- "PEBScounters": "32",
- "SampleAfterValue": "2000003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the total number of instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc0",
- "EventName": "INST_RETIRED.ANY_P",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "This event is deprecated. Refer to new event LD_BLOCKS.ADDRESS_ALIAS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x03",
- "EventName": "LD_BLOCKS.4K_ALIAS",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "UMask": "0x4",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of retired loads that are blocked because it initially appears to be store forward blocked, but subsequently is shown not to be blocked based on 4K alias check.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x03",
- "EventName": "LD_BLOCKS.ADDRESS_ALIAS",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "UMask": "0x4",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of retired loads that are blocked because its address exactly matches an older store whose data is not ready.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x03",
- "EventName": "LD_BLOCKS.DATA_UNKNOWN",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of machine clears due to memory ordering in which an internal load passes an older store within the same CPU.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc3",
- "EventName": "MACHINE_CLEARS.DISAMBIGUATION",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "20003",
- "Speculative": "1",
- "UMask": "0x8",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of machines clears due to memory renaming.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc3",
- "EventName": "MACHINE_CLEARS.MRN_NUKE",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x80",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of machine clears due to a page fault. Counts both I-Side and D-Side (Loads/Stores) page faults. A page fault occurs when either the page is not present, or an access violation occurs.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc3",
- "EventName": "MACHINE_CLEARS.PAGE_FAULT",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "20003",
- "Speculative": "1",
- "UMask": "0x20",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of machine clears that flush the pipeline and restart the machine with the use of microcode due to SMC, MEMORY_ORDERING, FP_ASSISTS, PAGE_FAULT, DISAMBIGUATION, and FPC_VIRTUAL_TRAP.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc3",
- "EventName": "MACHINE_CLEARS.SLOW",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "20003",
- "Speculative": "1",
- "UMask": "0x6f",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of machine clears due to program modifying data (self modifying code) within 1K of a recently fetched code page.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc3",
- "EventName": "MACHINE_CLEARS.SMC",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "20003",
- "Speculative": "1",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots not consumed by the backend due to a micro-sequencer (MS) scoreboard, which stalls the front-end from issuing from the UROM until a specified older uop retires.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x75",
- "EventName": "SERIALIZATION.NON_C01_MS_SCB",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a mispredicted jump or a machine clear.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x73",
- "EventName": "TOPDOWN_BAD_SPECULATION.ALL",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to fast nukes such as memory ordering and memory disambiguation machine clears.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x73",
- "EventName": "TOPDOWN_BAD_SPECULATION.FASTNUKE",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a machine clear (nuke) of any kind including memory ordering and memory disambiguation.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x73",
- "EventName": "TOPDOWN_BAD_SPECULATION.MACHINE_CLEARS",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x3",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to branch mispredicts.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x73",
- "EventName": "TOPDOWN_BAD_SPECULATION.MISPREDICT",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x4",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to a machine clear (nuke).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x73",
- "EventName": "TOPDOWN_BAD_SPECULATION.NUKE",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the total number of issue slots every cycle that were not consumed by the backend due to backend stalls.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x74",
- "EventName": "TOPDOWN_BE_BOUND.ALL",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to certain allocation restrictions.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x74",
- "EventName": "TOPDOWN_BE_BOUND.ALLOC_RESTRICTIONS",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to memory reservation stalls in which a scheduler is not able to accept uops.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x74",
- "EventName": "TOPDOWN_BE_BOUND.MEM_SCHEDULER",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to IEC or FPC RAT stalls, which can be due to FIQ or IEC reservation stalls in which the integer, floating point or SIMD scheduler is not able to accept uops.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x74",
- "EventName": "TOPDOWN_BE_BOUND.NON_MEM_SCHEDULER",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x8",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to the physical register file unable to accept an entry (marble stalls).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x74",
- "EventName": "TOPDOWN_BE_BOUND.REGISTER",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x20",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to the reorder buffer being full (ROB stalls).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x74",
- "EventName": "TOPDOWN_BE_BOUND.REORDER_BUFFER",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x40",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to scoreboards from the instruction queue (IQ), jump execution unit (JEU), or microcode sequencer (MS).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x74",
- "EventName": "TOPDOWN_BE_BOUND.SERIALIZATION",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x10",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the total number of issue slots every cycle that were not consumed by the backend due to frontend stalls.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.ALL",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to BACLEARS.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.BRANCH_DETECT",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to BTCLEARS.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.BRANCH_RESTEER",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x40",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to the microcode sequencer (MS).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.CISC",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to decode stalls.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.DECODE",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x8",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to frontend bandwidth restrictions due to decode, predecode, cisc, and other limitations.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.FRONTEND_BANDWIDTH",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x8d",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to a latency related stalls including BACLEARs, BTCLEARs, ITLB misses, and ICache misses.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.FRONTEND_LATENCY",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x72",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to ITLB misses.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.ITLB",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x10",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to other common frontend stalls not categorized.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.OTHER",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x80",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to wrong predecodes.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x71",
- "EventName": "TOPDOWN_FE_BOUND.PREDECODE",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x4",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the total number of consumed retirement slots.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc2",
- "EventName": "TOPDOWN_RETIRING.ALL",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the total number of uops retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc2",
- "EventName": "UOPS_RETIRED.ALL",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of integer divide uops retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc2",
- "EventName": "UOPS_RETIRED.IDIV",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "UMask": "0x10",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of uops that are from complex flows issued by the micro-sequencer (MS).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc2",
- "EventName": "UOPS_RETIRED.MS",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of x87 uops retired, includes those in MS flows.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0xc2",
- "EventName": "UOPS_RETIRED.X87",
- "PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "UMask": "0x2",
- "Unit": "cpu_atom"
- },
- {
"BriefDescription": "This event is deprecated. Refer to new event ARITH.DIV_ACTIVE",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
+ "Deprecated": "1",
"EventCode": "0xb0",
"EventName": "ARITH.DIVIDER_ACTIVE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x9",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles when divide unit is busy executing divide or square root operations.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EventCode": "0xb0",
"EventName": "ARITH.DIV_ACTIVE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles when divide unit is busy executing divide or square root operations. Accounts for integer and floating-point operations.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x9",
"Unit": "cpu_core"
},
{
"BriefDescription": "This event is deprecated. Refer to new event ARITH.FPDIV_ACTIVE",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
+ "Deprecated": "1",
"EventCode": "0xb0",
"EventName": "ARITH.FP_DIVIDER_ACTIVE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "This event counts the cycles the integer divider is busy.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb0",
"EventName": "ARITH.IDIV_ACTIVE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "This event is deprecated. Refer to new event ARITH.IDIV_ACTIVE",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
+ "Deprecated": "1",
"EventCode": "0xb0",
"EventName": "ARITH.INT_DIVIDER_ACTIVE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of occurrences where a microcode assist is invoked by hardware.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc1",
"EventName": "ASSISTS.ANY",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of occurrences where a microcode assist is invoked by hardware. Examples include AD (page Access Dirty), FP and AVX related assists.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x1b",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the total number of branch instructions retired for all branch types.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.ALL_BRANCHES",
+ "PEBS": "1",
+ "PublicDescription": "Counts the total number of instructions in which the instruction pointer (IP) of the processor is resteered due to a branch instruction and the branch instruction successfully retires. All branch type instructions are accounted for.",
+ "SampleAfterValue": "200003",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "All branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc4",
"EventName": "BR_INST_RETIRED.ALL_BRANCHES",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts all branch instructions retired.",
"SampleAfterValue": "400009",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.NEAR_CALL",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xf9",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of retired JCC (Jump on Conditional Code) branch instructions retired, includes both taken and not taken branches.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.COND",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x7e",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Conditional branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc4",
"EventName": "BR_INST_RETIRED.COND",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts conditional branch instructions retired.",
"SampleAfterValue": "400009",
"UMask": "0x11",
"Unit": "cpu_core"
},
{
"BriefDescription": "Not taken branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc4",
"EventName": "BR_INST_RETIRED.COND_NTAKEN",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts not taken branch instructions retired.",
"SampleAfterValue": "400009",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of taken JCC (Jump on Conditional Code) branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.COND_TAKEN",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfe",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Taken conditional branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc4",
"EventName": "BR_INST_RETIRED.COND_TAKEN",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts taken conditional branch instructions retired.",
"SampleAfterValue": "400009",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of far branch instructions retired, includes far jump, far call and return, and interrupt call and return.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.FAR_BRANCH",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xbf",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Far branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc4",
"EventName": "BR_INST_RETIRED.FAR_BRANCH",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts far branch instructions retired.",
"SampleAfterValue": "100007",
"UMask": "0x40",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of near indirect JMP and near indirect CALL branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.INDIRECT",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xeb",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Indirect near branch instructions retired (excluding returns)",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc4",
"EventName": "BR_INST_RETIRED.INDIRECT",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts near indirect branch instructions retired excluding returns. TSX abort is an indirect branch.",
"SampleAfterValue": "100003",
"UMask": "0x80",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of near indirect CALL branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.INDIRECT_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfb",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.INDIRECT_CALL",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.IND_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfb",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.COND",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.JCC",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x7e",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of near CALL branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.NEAR_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xf9",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Direct and indirect near call instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc4",
"EventName": "BR_INST_RETIRED.NEAR_CALL",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts both direct and indirect near call instructions retired.",
"SampleAfterValue": "100007",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of near RET branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.NEAR_RETURN",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xf7",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Return instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc4",
"EventName": "BR_INST_RETIRED.NEAR_RETURN",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts return instructions retired.",
"SampleAfterValue": "100007",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Taken branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc4",
"EventName": "BR_INST_RETIRED.NEAR_TAKEN",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts taken branch instructions retired.",
"SampleAfterValue": "400009",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.INDIRECT",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.NON_RETURN_IND",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xeb",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of near relative CALL branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.REL_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfd",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.NEAR_RETURN",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.RETURN",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xf7",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.COND_TAKEN",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.TAKEN_JCC",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfe",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the total number of mispredicted branch instructions retired for all branch types.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.ALL_BRANCHES",
+ "PEBS": "1",
+ "PublicDescription": "Counts the total number of mispredicted branch instructions retired. All branch type instructions are accounted for. Prediction of the branch target address enables the processor to begin executing instructions before the non-speculative execution path is known. The branch prediction unit (BPU) predicts the target address based on the instruction pointer (IP) of the branch and on the execution path through which execution reached this IP. A branch misprediction occurs when the prediction is wrong, and results in discarding all instructions executed in the speculative path and re-fetching from the correct path.",
+ "SampleAfterValue": "200003",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "All mispredicted branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc5",
"EventName": "BR_MISP_RETIRED.ALL_BRANCHES",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts all the retired branch instructions that were mispredicted by the processor. A branch misprediction occurs when the processor incorrectly predicts the destination of the branch. When the misprediction is discovered at execution, all the instructions executed in the wrong (speculative) path must be discarded, and the processor must start fetching from the correct path.",
"SampleAfterValue": "400009",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of mispredicted JCC (Jump on Conditional Code) branch instructions retired.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.COND",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x7e",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Mispredicted conditional branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc5",
"EventName": "BR_MISP_RETIRED.COND",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts mispredicted conditional branch instructions retired.",
"SampleAfterValue": "400009",
"UMask": "0x11",
"Unit": "cpu_core"
},
{
"BriefDescription": "Mispredicted non-taken conditional branch instructions retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc5",
"EventName": "BR_MISP_RETIRED.COND_NTAKEN",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of conditional branch instructions retired that were mispredicted and the branch direction was not taken.",
"SampleAfterValue": "400009",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of mispredicted taken JCC (Jump on Conditional Code) branch instructions retired.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.COND_TAKEN",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfe",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "number of branch instructions retired that were mispredicted and taken.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc5",
"EventName": "BR_MISP_RETIRED.COND_TAKEN",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts taken conditional mispredicted branch instructions retired.",
"SampleAfterValue": "400009",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of mispredicted near indirect JMP and near indirect CALL branch instructions retired.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.INDIRECT",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xeb",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of mispredicted near indirect CALL branch instructions retired.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.INDIRECT_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfb",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Mispredicted indirect CALL retired.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc5",
"EventName": "BR_MISP_RETIRED.INDIRECT_CALL",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts retired mispredicted indirect (near taken) CALL instructions, including both register and memory indirect.",
"SampleAfterValue": "400009",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.INDIRECT_CALL",
+ "Deprecated": "1",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.IND_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfb",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.COND",
+ "Deprecated": "1",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.JCC",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x7e",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Number of near branch instructions retired that were mispredicted and taken.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc5",
"EventName": "BR_MISP_RETIRED.NEAR_TAKEN",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts number of near branch instructions retired that were mispredicted and taken.",
"SampleAfterValue": "400009",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.INDIRECT",
+ "Deprecated": "1",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.NON_RETURN_IND",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xeb",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "This event counts the number of mispredicted ret instructions retired. Non PEBS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc5",
"EventName": "BR_MISP_RETIRED.RET",
"PEBS": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "This is a non-precise version (that is, does not use PEBS) of the event that counts mispredicted return instructions retired.",
"SampleAfterValue": "100007",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of mispredicted near RET branch instructions retired.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.RETURN",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xf7",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.COND_TAKEN",
+ "Deprecated": "1",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.TAKEN_JCC",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfe",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Core clocks when the thread is in the C0.1 light-weight slower wakeup time but more power saving optimized state.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xec",
"EventName": "CPU_CLK_UNHALTED.C01",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts core clocks when the thread is in the C0.1 light-weight slower wakeup time but more power saving optimized state. This state can be entered via the TPAUSE or UMWAIT instructions.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "Core clocks when the thread is in the C0.2 light-weight faster wakeup time but less power saving optimized state.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xec",
"EventName": "CPU_CLK_UNHALTED.C02",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts core clocks when the thread is in the C0.2 light-weight faster wakeup time but less power saving optimized state. This state can be entered via the TPAUSE or UMWAIT instructions.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Core clocks when the thread is in the C0.1 or C0.2 or running a PAUSE in C0 ACPI state.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xec",
"EventName": "CPU_CLK_UNHALTED.C0_WAIT",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts core clocks when the thread is in the C0.1 or C0.2 power saving optimized states (TPAUSE or UMWAIT instructions) or running the PAUSE instruction.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x70",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of unhalted core clock cycles. (Fixed event)",
+ "EventName": "CPU_CLK_UNHALTED.CORE",
+ "PublicDescription": "Counts the number of core cycles while the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. The core frequency may change from time to time. For this reason this event may have a changing ratio with regards to time. This event uses fixed counter 1.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of unhalted core clock cycles.",
+ "EventCode": "0x3c",
+ "EventName": "CPU_CLK_UNHALTED.CORE_P",
+ "PublicDescription": "Counts the number of core cycles while the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. The core frequency may change from time to time. For this reason this event may have a changing ratio with regards to time. This event uses a programmable general purpose performance counter.",
+ "SampleAfterValue": "2000003",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Cycle counts are evenly distributed between active threads in the Core.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xec",
"EventName": "CPU_CLK_UNHALTED.DISTRIBUTED",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "This event distributes cycle counts between active hyperthreads, i.e., those in C0. A hyperthread becomes inactive when it executes the HLT or MWAIT instructions. If all other hyperthreads are inactive (or disabled or do not exist), all counts are attributed to this hyperthread. To obtain the full count when the Core is active, sum the counts from each hyperthread.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Core crystal clock cycles when this thread is unhalted and the other thread is halted.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x3c",
"EventName": "CPU_CLK_UNHALTED.ONE_THREAD_ACTIVE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts Core crystal clock cycles when current thread is unhalted and the other thread is halted.",
"SampleAfterValue": "25003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "CPU_CLK_UNHALTED.PAUSE",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xec",
"EventName": "CPU_CLK_UNHALTED.PAUSE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x40",
"Unit": "cpu_core"
},
{
"BriefDescription": "CPU_CLK_UNHALTED.PAUSE_INST",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EdgeDetect": "1",
"EventCode": "0xec",
"EventName": "CPU_CLK_UNHALTED.PAUSE_INST",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x40",
"Unit": "cpu_core"
},
{
"BriefDescription": "Core crystal clock cycles. Cycle counts are evenly distributed between active threads in the Core.",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x3c",
"EventName": "CPU_CLK_UNHALTED.REF_DISTRIBUTED",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "This event distributes Core crystal clock cycle counts between active hyperthreads, i.e., those in C0 sleep-state. A hyperthread becomes inactive when it executes the HLT or MWAIT instructions. If one thread is active in a core, all counts are attributed to this hyperthread. To obtain the full count when the Core is active, sum the counts from each hyperthread.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of unhalted reference clock cycles at TSC frequency. (Fixed event)",
+ "EventName": "CPU_CLK_UNHALTED.REF_TSC",
+ "PublicDescription": "Counts the number of reference cycles that the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. This event is not affected by core frequency changes and increments at a fixed frequency that is also used for the Time Stamp Counter (TSC). This event uses fixed counter 2.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x3",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Reference cycles when the core is not in halt state.",
- "CollectPEBSRecord": "2",
- "Counter": "Fixed counter 2",
"EventName": "CPU_CLK_UNHALTED.REF_TSC",
- "PEBScounters": "34",
+ "PublicDescription": "Counts the number of reference cycles when the core is not in a halt state. The core enters the halt state when it is running the HLT instruction or the MWAIT instruction. This event is not affected by core frequency changes (for example, P states, TM2 transitions) but has the same incrementing frequency as the time stamp counter. This event can approximate elapsed time while the core was not in a halt state. It is counted on a dedicated fixed counter, leaving the eight programmable counters available for other events. Note: On all current platforms this event stops counting during 'throttling (TM)' states duty off periods the processor is 'halted'. The counter update is done at a lower clock rate then the core clock the overflow status bit for this counter may appear 'sticky'. After the counter has overflowed and software clears the overflow status bit and resets the counter to less than MAX. The reset value to the counter is not clocked immediately so the overflow status bit will flip 'high (1)' and generate another PMI (if enabled) after which the reset value gets clocked into the counter. Therefore, software will get the interrupt, read the overflow status bit '1 for bit 34 while the counter value is less than MAX. Software should ignore this case.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x3",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of unhalted reference clock cycles at TSC frequency.",
+ "EventCode": "0x3c",
+ "EventName": "CPU_CLK_UNHALTED.REF_TSC_P",
+ "PublicDescription": "Counts the number of reference cycles that the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. This event is not affected by core frequency changes and increments at a fixed frequency that is also used for the Time Stamp Counter (TSC). This event uses a programmable general purpose performance counter.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Reference cycles when the core is not in halt state.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x3c",
"EventName": "CPU_CLK_UNHALTED.REF_TSC_P",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of reference cycles when the core is not in a halt state. The core enters the halt state when it is running the HLT instruction or the MWAIT instruction. This event is not affected by core frequency changes (for example, P states, TM2 transitions) but has the same incrementing frequency as the time stamp counter. This event can approximate elapsed time while the core was not in a halt state. It is counted on a dedicated fixed counter, leaving the four (eight when Hyperthreading is disabled) programmable counters available for other events. Note: On all current platforms this event stops counting during 'throttling (TM)' states duty off periods the processor is 'halted'. The counter update is done at a lower clock rate then the core clock the overflow status bit for this counter may appear 'sticky'. After the counter has overflowed and software clears the overflow status bit and resets the counter to less than MAX. The reset value to the counter is not clocked immediately so the overflow status bit will flip 'high (1)' and generate another PMI (if enabled) after which the reset value gets clocked into the counter. Therefore, software will get the interrupt, read the overflow status bit '1 for bit 34 while the counter value is less than MAX. Software should ignore this case.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of unhalted core clock cycles. (Fixed event)",
+ "EventName": "CPU_CLK_UNHALTED.THREAD",
+ "PublicDescription": "Counts the number of core cycles while the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. The core frequency may change from time to time. For this reason this event may have a changing ratio with regards to time. This event uses fixed counter 1.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Core cycles when the thread is not in halt state",
- "CollectPEBSRecord": "2",
- "Counter": "Fixed counter 1",
"EventName": "CPU_CLK_UNHALTED.THREAD",
- "PEBScounters": "33",
+ "PublicDescription": "Counts the number of core cycles while the thread is not in a halt state. The thread enters the halt state when it is running the HLT instruction. This event is a component in many key event ratios. The core frequency may change from time to time due to transitions associated with Enhanced Intel SpeedStep Technology or TM2. For this reason this event may have a changing ratio with regards to time. When the core frequency is constant, this event can approximate elapsed time while the core was not in the halt state. It is counted on a dedicated fixed counter, leaving the eight programmable counters available for other events.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of unhalted core clock cycles.",
+ "EventCode": "0x3c",
+ "EventName": "CPU_CLK_UNHALTED.THREAD_P",
+ "PublicDescription": "Counts the number of core cycles while the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. The core frequency may change from time to time. For this reason this event may have a changing ratio with regards to time. This event uses a programmable general purpose performance counter.",
+ "SampleAfterValue": "2000003",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Thread cycles when thread is not in halt state",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0x3c",
"EventName": "CPU_CLK_UNHALTED.THREAD_P",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "This is an architectural event that counts the number of thread cycles while the thread is not in a halt state. The thread enters the halt state when it is running the HLT instruction. The core frequency may change from time to time due to power or thermal throttling. For this reason, this event may have a changing ratio with regards to wall clock time.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles while L1 cache miss demand load is outstanding.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "8",
"EventCode": "0xa3",
"EventName": "CYCLE_ACTIVITY.CYCLES_L1D_MISS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles while L2 cache miss demand load is outstanding.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EventCode": "0xa3",
"EventName": "CYCLE_ACTIVITY.CYCLES_L2_MISS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles while memory subsystem has an outstanding load.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "16",
"EventCode": "0xa3",
"EventName": "CYCLE_ACTIVITY.CYCLES_MEM_ANY",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "Execution stalls while L1 cache miss demand load is outstanding.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "12",
"EventCode": "0xa3",
"EventName": "CYCLE_ACTIVITY.STALLS_L1D_MISS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0xc",
"Unit": "cpu_core"
},
{
"BriefDescription": "Execution stalls while L2 cache miss demand load is outstanding.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "5",
"EventCode": "0xa3",
"EventName": "CYCLE_ACTIVITY.STALLS_L2_MISS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x5",
"Unit": "cpu_core"
},
{
"BriefDescription": "Total execution stalls.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "4",
"EventCode": "0xa3",
"EventName": "CYCLE_ACTIVITY.STALLS_TOTAL",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles total of 1 uop is executed on all ports and Reservation Station was not empty.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa6",
"EventName": "EXE_ACTIVITY.1_PORTS_UTIL",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles during which a total of 1 uop was executed on all ports and Reservation Station (RS) was not empty.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles total of 2 uops are executed on all ports and Reservation Station was not empty.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa6",
"EventName": "EXE_ACTIVITY.2_PORTS_UTIL",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles during which a total of 2 uops were executed on all ports and Reservation Station (RS) was not empty.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles total of 3 uops are executed on all ports and Reservation Station was not empty.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa6",
"EventName": "EXE_ACTIVITY.3_PORTS_UTIL",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Cycles total of 3 uops are executed on all ports and Reservation Station (RS) was not empty.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles total of 4 uops are executed on all ports and Reservation Station was not empty.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa6",
"EventName": "EXE_ACTIVITY.4_PORTS_UTIL",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Cycles total of 4 uops are executed on all ports and Reservation Station (RS) was not empty.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "Execution stalls while memory subsystem has an outstanding load.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "5",
"EventCode": "0xa6",
"EventName": "EXE_ACTIVITY.BOUND_ON_LOADS",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x21",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles where the Store Buffer was full and no loads caused an execution stall.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "2",
"EventCode": "0xa6",
"EventName": "EXE_ACTIVITY.BOUND_ON_STORES",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles where the Store Buffer was full and no loads caused an execution stall.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x40",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles no uop executed while RS was not empty, the SB was not full and there was no outstanding load.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa6",
"EventName": "EXE_ACTIVITY.EXE_BOUND_0_PORTS",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of cycles total of 0 uops executed on all ports, Reservation Station (RS) was not empty, the Store Buffer (SB) was not full and there was no outstanding load.",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x80",
"Unit": "cpu_core"
},
{
"BriefDescription": "Instruction decoders utilized in a cycle",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x75",
"EventName": "INST_DECODED.DECODERS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Number of decoders utilized in a cycle when the MITE (legacy decode pipeline) fetches instructions.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the total number of instructions retired. (Fixed event)",
+ "EventName": "INST_RETIRED.ANY",
+ "PEBS": "1",
+ "PublicDescription": "Counts the total number of instructions that retired. For instructions that consist of multiple uops, this event counts the retirement of the last uop of the instruction. This event continues counting during hardware interrupts, traps, and inside interrupt handlers. This event uses fixed counter 0.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Number of instructions retired. Fixed Counter - architectural event",
- "CollectPEBSRecord": "2",
- "Counter": "Fixed counter 0",
"EventName": "INST_RETIRED.ANY",
"PEBS": "1",
- "PEBScounters": "32",
+ "PublicDescription": "Counts the number of X86 instructions retired - an Architectural PerfMon event. Counting continues during hardware interrupts, traps, and inside interrupt handlers. Notes: INST_RETIRED.ANY is counted by a designated fixed counter freeing up programmable counters to count other events. INST_RETIRED.ANY_P is counted by a programmable counter.",
"SampleAfterValue": "2000003",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the total number of instructions retired.",
+ "EventCode": "0xc0",
+ "EventName": "INST_RETIRED.ANY_P",
+ "PEBS": "1",
+ "PublicDescription": "Counts the total number of instructions that retired. For instructions that consist of multiple uops, this event counts the retirement of the last uop of the instruction. This event continues counting during hardware interrupts, traps, and inside interrupt handlers. This event uses a programmable general purpose performance counter.",
+ "SampleAfterValue": "2000003",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Number of instructions retired. General Counter - architectural event",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc0",
"EventName": "INST_RETIRED.ANY_P",
"PEBS": "1",
- "PEBScounters": "1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of X86 instructions retired - an Architectural PerfMon event. Counting continues during hardware interrupts, traps, and inside interrupt handlers. Notes: INST_RETIRED.ANY is counted by a designated fixed counter freeing up programmable counters to count other events. INST_RETIRED.ANY_P is counted by a programmable counter.",
"SampleAfterValue": "2000003",
"Unit": "cpu_core"
},
{
"BriefDescription": "INST_RETIRED.MACRO_FUSED",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc0",
"EventName": "INST_RETIRED.MACRO_FUSED",
- "PEBScounters": "1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired NOP instructions.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc0",
"EventName": "INST_RETIRED.NOP",
- "PEBScounters": "1,2,3,4,5,6,7",
+ "PublicDescription": "Counts all retired NOP or ENDBR32/64 instructions",
"SampleAfterValue": "2000003",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Precise instruction retired with PEBS precise-distribution",
- "CollectPEBSRecord": "2",
- "Counter": "Fixed counter 0",
"EventName": "INST_RETIRED.PREC_DIST",
"PEBS": "1",
- "PEBScounters": "32",
+ "PublicDescription": "A version of INST_RETIRED that allows for a precise distribution of samples across instructions retired. It utilizes the Precise Distribution of Instructions Retired (PDIR++) feature to fix bias in how retired instructions get sampled. Use on Fixed Counter 0.",
"SampleAfterValue": "2000003",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "INST_RETIRED.REP_ITERATION",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc0",
"EventName": "INST_RETIRED.REP_ITERATION",
- "PEBScounters": "1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts cycles after recovery from a branch misprediction or machine clear till the first uop is issued from the resteered path.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xad",
"EventName": "INT_MISC.CLEAR_RESTEER_CYCLES",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Cycles after recovery from a branch misprediction or machine clear till the first uop is issued from the resteered path.",
"SampleAfterValue": "500009",
- "Speculative": "1",
"UMask": "0x80",
"Unit": "cpu_core"
},
{
"BriefDescription": "Core cycles the allocator was stalled due to recovery from earlier clear event for this thread",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xad",
"EventName": "INT_MISC.RECOVERY_CYCLES",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts core cycles when the Resource allocator was stalled due to recovery from an earlier branch misprediction or machine clear event.",
"SampleAfterValue": "500009",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "INT_MISC.UNKNOWN_BRANCH_CYCLES",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xad",
"EventName": "INT_MISC.UNKNOWN_BRANCH_CYCLES",
"MSRIndex": "0x3F7",
"MSRValue": "0x7",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
- "Speculative": "1",
- "TakenAlone": "1",
"UMask": "0x40",
"Unit": "cpu_core"
},
{
"BriefDescription": "TMA slots where uops got dropped",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xad",
"EventName": "INT_MISC.UOP_DROPPING",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Estimated number of Top-down Microarchitecture Analysis slots that got dropped due to non front-end reasons",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "INT_VEC_RETIRED.128BIT",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xe7",
"EventName": "INT_VEC_RETIRED.128BIT",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
"UMask": "0x13",
"Unit": "cpu_core"
},
{
"BriefDescription": "INT_VEC_RETIRED.256BIT",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xe7",
"EventName": "INT_VEC_RETIRED.256BIT",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
"UMask": "0xac",
"Unit": "cpu_core"
},
{
"BriefDescription": "integer ADD, SUB, SAD 128-bit vector instructions.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xe7",
"EventName": "INT_VEC_RETIRED.ADD_128",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of retired integer ADD/SUB (regular or horizontal), SAD 128-bit vector instructions.",
"SampleAfterValue": "1000003",
"UMask": "0x3",
"Unit": "cpu_core"
},
{
"BriefDescription": "integer ADD, SUB, SAD 256-bit vector instructions.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xe7",
"EventName": "INT_VEC_RETIRED.ADD_256",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of retired integer ADD/SUB (regular or horizontal), SAD 256-bit vector instructions.",
"SampleAfterValue": "1000003",
"UMask": "0xc",
"Unit": "cpu_core"
},
{
"BriefDescription": "INT_VEC_RETIRED.MUL_256",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xe7",
"EventName": "INT_VEC_RETIRED.MUL_256",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
"UMask": "0x80",
"Unit": "cpu_core"
},
{
"BriefDescription": "INT_VEC_RETIRED.SHUFFLES",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xe7",
"EventName": "INT_VEC_RETIRED.SHUFFLES",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
"UMask": "0x40",
"Unit": "cpu_core"
},
{
"BriefDescription": "INT_VEC_RETIRED.VNNI_128",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xe7",
"EventName": "INT_VEC_RETIRED.VNNI_128",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "INT_VEC_RETIRED.VNNI_256",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xe7",
"EventName": "INT_VEC_RETIRED.VNNI_256",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "This event is deprecated. Refer to new event LD_BLOCKS.ADDRESS_ALIAS",
+ "Deprecated": "1",
+ "EventCode": "0x03",
+ "EventName": "LD_BLOCKS.4K_ALIAS",
+ "PEBS": "1",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x4",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of retired loads that are blocked because it initially appears to be store forward blocked, but subsequently is shown not to be blocked based on 4K alias check.",
+ "EventCode": "0x03",
+ "EventName": "LD_BLOCKS.ADDRESS_ALIAS",
+ "PEBS": "1",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x4",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "False dependencies in MOB due to partial compare on address.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x03",
"EventName": "LD_BLOCKS.ADDRESS_ALIAS",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of times a load got blocked due to false dependencies in MOB due to partial compare on address.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of retired loads that are blocked because its address exactly matches an older store whose data is not ready.",
+ "EventCode": "0x03",
+ "EventName": "LD_BLOCKS.DATA_UNKNOWN",
+ "PEBS": "1",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "The number of times that split load operations are temporarily blocked because all resources for handling the split accesses are in use.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x03",
"EventName": "LD_BLOCKS.NO_SR",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of times that split load operations are temporarily blocked because all resources for handling the split accesses are in use.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x88",
"Unit": "cpu_core"
},
{
"BriefDescription": "Loads blocked due to overlapping with a preceding store that cannot be forwarded.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x03",
"EventName": "LD_BLOCKS.STORE_FORWARD",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of times where store forwarding was prevented for a load operation. The most common case is a load blocked due to the address of memory access (partially) overlapping with a preceding uncompleted store. Note: See the table of not supported store forwards in the Optimization Guide.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x82",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts the number of demand load dispatches that hit L1D fill buffer (FB) allocated for software prefetch.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x4c",
"EventName": "LOAD_HIT_PREFETCH.SWPF",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts all not software-prefetch load dispatches that hit the fill buffer (FB) allocated for the software prefetch. It can also be incremented by some lock instructions. So it should only be used with profiling so that the locks can be excluded by ASM (Assembly File) inspection of the nearby instructions.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles Uops delivered by the LSD, but didn't come from the decoder.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EventCode": "0xa8",
"EventName": "LSD.CYCLES_ACTIVE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the cycles when at least one uop is delivered by the LSD (Loop-stream detector).",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles optimal number of Uops delivered by the LSD, but did not come from the decoder.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "6",
"EventCode": "0xa8",
"EventName": "LSD.CYCLES_OK",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the cycles when optimal number of uops is delivered by the LSD (Loop-stream detector).",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
- "BriefDescription": "Number of Uops delivered by the LSD.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
- "EventCode": "0xa8",
- "EventName": "LSD.UOPS",
- "PEBScounters": "0,1,2,3,4,5,6,7",
- "SampleAfterValue": "2000003",
- "Speculative": "1",
- "UMask": "0x1",
- "Unit": "cpu_core"
+ "BriefDescription": "Number of Uops delivered by the LSD.",
+ "EventCode": "0xa8",
+ "EventName": "LSD.UOPS",
+ "PublicDescription": "Counts the number of uops delivered to the back-end by the LSD(Loop Stream Detector).",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x1",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Number of machine clears (nukes) of any type.",
+ "CounterMask": "1",
+ "EdgeDetect": "1",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.COUNT",
+ "PublicDescription": "Counts the number of machine clears (nukes) of any type.",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1",
+ "Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears due to memory ordering in which an internal load passes an older store within the same CPU.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.DISAMBIGUATION",
+ "SampleAfterValue": "20003",
+ "UMask": "0x8",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of machines clears due to memory renaming.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.MRN_NUKE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x80",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears due to a page fault. Counts both I-Side and D-Side (Loads/Stores) page faults. A page fault occurs when either the page is not present, or an access violation occurs.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.PAGE_FAULT",
+ "SampleAfterValue": "20003",
+ "UMask": "0x20",
+ "Unit": "cpu_atom"
},
{
- "BriefDescription": "Number of machine clears (nukes) of any type.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
- "CounterMask": "1",
- "EdgeDetect": "1",
+ "BriefDescription": "Counts the number of machine clears that flush the pipeline and restart the machine with the use of microcode due to SMC, MEMORY_ORDERING, FP_ASSISTS, PAGE_FAULT, DISAMBIGUATION, and FPC_VIRTUAL_TRAP.",
"EventCode": "0xc3",
- "EventName": "MACHINE_CLEARS.COUNT",
- "PEBScounters": "0,1,2,3,4,5,6,7",
- "SampleAfterValue": "100003",
- "Speculative": "1",
+ "EventName": "MACHINE_CLEARS.SLOW",
+ "SampleAfterValue": "20003",
+ "UMask": "0x6f",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears due to program modifying data (self modifying code) within 1K of a recently fetched code page.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.SMC",
+ "SampleAfterValue": "20003",
"UMask": "0x1",
- "Unit": "cpu_core"
+ "Unit": "cpu_atom"
},
{
"BriefDescription": "Self-modifying code (SMC) detected.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc3",
"EventName": "MACHINE_CLEARS.SMC",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts self-modifying code (SMC) detected, which causes a machine clear.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "MISC2_RETIRED.LFENCE",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xe0",
"EventName": "MISC2_RETIRED.LFENCE",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "400009",
- "Speculative": "1",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Increments whenever there is an update to the LBR array.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xcc",
"EventName": "MISC_RETIRED.LBR_INSERTS",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Increments when an entry is added to the Last Branch Record (LBR) array (or removed from the array in case of RETURNs in call stack mode). The event requires LBR enable via IA32_DEBUGCTL MSR and branch type selection via MSR_LBR_SELECT.",
"SampleAfterValue": "100003",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles stalled due to no store buffers available. (not including draining form sync).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa2",
"EventName": "RESOURCE_STALLS.SB",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts allocation stall cycles caused by the store buffer (SB) being full. This counts cycles that the pipeline back-end blocked uop delivery from the front-end.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts cycles where the pipeline is stalled due to serializing operations.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa2",
"EventName": "RESOURCE_STALLS.SCOREBOARD",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of issue slots not consumed by the backend due to a micro-sequencer (MS) scoreboard, which stalls the front-end from issuing from the UROM until a specified older uop retires.",
+ "EventCode": "0x75",
+ "EventName": "SERIALIZATION.NON_C01_MS_SCB",
+ "PublicDescription": "Counts the number of issue slots not consumed by the backend due to a micro-sequencer (MS) scoreboard, which stalls the front-end from issuing from the UROM until a specified older uop retires. The most commonly executed instruction with an MS scoreboard is PAUSE.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "TMA slots where no uops were being issued due to lack of back-end resources.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa4",
"EventName": "TOPDOWN.BACKEND_BOUND_SLOTS",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of slots in TMA method where no micro-operations were being issued from front-end to back-end of the machine due to lack of back-end resources.",
"SampleAfterValue": "10000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "TMA slots wasted due to incorrect speculations.",
- "CollectPEBSRecord": "2",
"EventCode": "0xa4",
"EventName": "TOPDOWN.BAD_SPEC_SLOTS",
+ "PublicDescription": "Number of slots of TMA method that were wasted due to incorrect speculation. It covers all types of control-flow or data-related mis-speculations.",
"SampleAfterValue": "10000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "TMA slots wasted due to incorrect speculation by branch mispredictions",
- "CollectPEBSRecord": "2",
"EventCode": "0xa4",
"EventName": "TOPDOWN.BR_MISPREDICT_SLOTS",
+ "PublicDescription": "Number of TMA slots that were wasted due to incorrect speculation by (any type of) branch mispredictions. This event estimates number of specualtive operations that were issued but not retired as well as the out-of-order engine recovery past a branch misprediction.",
"SampleAfterValue": "10000003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "TOPDOWN.MEMORY_BOUND_SLOTS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa4",
"EventName": "TOPDOWN.MEMORY_BOUND_SLOTS",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "10000003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "TMA slots available for an unhalted logical processor. Fixed counter - architectural event",
- "CollectPEBSRecord": "2",
- "Counter": "Fixed counter 3",
"EventName": "TOPDOWN.SLOTS",
- "PEBScounters": "35",
+ "PublicDescription": "Number of available slots for an unhalted logical processor. The event increments by machine-width of the narrowest pipeline as employed by the Top-down Microarchitecture Analysis method (TMA). The count is distributed among unhalted logical processors (hyper-threads) who share the same physical core. Software can use this event as the denominator for the top-level metrics of the TMA method. This architectural event is counted on a designated fixed counter (Fixed Counter 3).",
"SampleAfterValue": "10000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "TMA slots available for an unhalted logical processor. General counter - architectural event",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xa4",
"EventName": "TOPDOWN.SLOTS_P",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of available slots for an unhalted logical processor. The event increments by machine-width of the narrowest pipeline as employed by the Top-down Microarchitecture Analysis method. The count is distributed among unhalted logical processors (hyper-threads) who share the same physical core.",
"SampleAfterValue": "10000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a mispredicted jump or a machine clear.",
+ "EventCode": "0x73",
+ "EventName": "TOPDOWN_BAD_SPECULATION.ALL",
+ "PublicDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a mispredicted jump or a machine clear. Only issue slots wasted due to fast nukes such as memory ordering nukes are counted. Other nukes are not accounted for. Counts all issue slots blocked during this recovery window including relevant microcode flows and while uops are not yet available in the instruction queue (IQ) even if an FE_bound event occurs during this period. Also includes the issue slots that were consumed by the backend but were thrown away because they were younger than the mispredict or machine clear.",
+ "SampleAfterValue": "1000003",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to fast nukes such as memory ordering and memory disambiguation machine clears.",
+ "EventCode": "0x73",
+ "EventName": "TOPDOWN_BAD_SPECULATION.FASTNUKE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a machine clear (nuke) of any kind including memory ordering and memory disambiguation.",
+ "EventCode": "0x73",
+ "EventName": "TOPDOWN_BAD_SPECULATION.MACHINE_CLEARS",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x3",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to branch mispredicts.",
+ "EventCode": "0x73",
+ "EventName": "TOPDOWN_BAD_SPECULATION.MISPREDICT",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x4",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to a machine clear (nuke).",
+ "EventCode": "0x73",
+ "EventName": "TOPDOWN_BAD_SPECULATION.NUKE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots every cycle that were not consumed by the backend due to backend stalls.",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.ALL",
+ "SampleAfterValue": "1000003",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to certain allocation restrictions.",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.ALLOC_RESTRICTIONS",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to memory reservation stalls in which a scheduler is not able to accept uops.",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.MEM_SCHEDULER",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to IEC or FPC RAT stalls, which can be due to FIQ or IEC reservation stalls in which the integer, floating point or SIMD scheduler is not able to accept uops.",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.NON_MEM_SCHEDULER",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x8",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to the physical register file unable to accept an entry (marble stalls).",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.REGISTER",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x20",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to the reorder buffer being full (ROB stalls).",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.REORDER_BUFFER",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x40",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to scoreboards from the instruction queue (IQ), jump execution unit (JEU), or microcode sequencer (MS).",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.SERIALIZATION",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x10",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots every cycle that were not consumed by the backend due to frontend stalls.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.ALL",
+ "SampleAfterValue": "1000003",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to BACLEARS.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.BRANCH_DETECT",
+ "PublicDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to BACLEARS, which occurs when the Branch Target Buffer (BTB) prediction or lack thereof, was corrected by a later branch predictor in the frontend. Includes BACLEARS due to all branch types including conditional and unconditional jumps, returns, and indirect branches.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to BTCLEARS.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.BRANCH_RESTEER",
+ "PublicDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to BTCLEARS, which occurs when the Branch Target Buffer (BTB) predicts a taken branch.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x40",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to the microcode sequencer (MS).",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.CISC",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to decode stalls.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.DECODE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x8",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to frontend bandwidth restrictions due to decode, predecode, cisc, and other limitations.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.FRONTEND_BANDWIDTH",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x8d",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to a latency related stalls including BACLEARs, BTCLEARs, ITLB misses, and ICache misses.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.FRONTEND_LATENCY",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x72",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to ITLB misses.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.ITLB",
+ "PublicDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to Instruction Table Lookaside Buffer (ITLB) misses.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x10",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to other common frontend stalls not categorized.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.OTHER",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x80",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to wrong predecodes.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.PREDECODE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x4",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the total number of consumed retirement slots.",
+ "EventCode": "0xc2",
+ "EventName": "TOPDOWN_RETIRING.ALL",
+ "PEBS": "1",
+ "SampleAfterValue": "1000003",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "UOPS_DECODED.DEC0_UOPS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x76",
"EventName": "UOPS_DECODED.DEC0_UOPS",
- "PEBScounters": "0,1,2,3",
"SampleAfterValue": "1000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops executed on port 0",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb2",
"EventName": "UOPS_DISPATCHED.PORT_0",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of uops dispatch to execution port 0.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops executed on port 1",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb2",
"EventName": "UOPS_DISPATCHED.PORT_1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of uops dispatch to execution port 1.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops executed on ports 2, 3 and 10",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb2",
"EventName": "UOPS_DISPATCHED.PORT_2_3_10",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of uops dispatch to execution ports 2, 3 and 10",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops executed on ports 4 and 9",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb2",
"EventName": "UOPS_DISPATCHED.PORT_4_9",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of uops dispatch to execution ports 4 and 9",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops executed on ports 5 and 11",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb2",
"EventName": "UOPS_DISPATCHED.PORT_5_11",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of uops dispatch to execution ports 5 and 11",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops executed on port 6",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb2",
"EventName": "UOPS_DISPATCHED.PORT_6",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of uops dispatch to execution port 6.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x40",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops executed on ports 7 and 8",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb2",
"EventName": "UOPS_DISPATCHED.PORT_7_8",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Number of uops dispatch to execution ports 7 and 8.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x80",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles at least 1 micro-op is executed from any thread on physical core.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.CORE_CYCLES_GE_1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles when at least 1 micro-op is executed from any thread on physical core.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles at least 2 micro-op is executed from any thread on physical core.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "2",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.CORE_CYCLES_GE_2",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles when at least 2 micro-ops are executed from any thread on physical core.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles at least 3 micro-op is executed from any thread on physical core.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "3",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.CORE_CYCLES_GE_3",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles when at least 3 micro-ops are executed from any thread on physical core.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles at least 4 micro-op is executed from any thread on physical core.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "4",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.CORE_CYCLES_GE_4",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles when at least 4 micro-ops are executed from any thread on physical core.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles where at least 1 uop was executed per-thread",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.CYCLES_GE_1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Cycles where at least 1 uop was executed per-thread.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles where at least 2 uops were executed per-thread",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "2",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.CYCLES_GE_2",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Cycles where at least 2 uops were executed per-thread.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles where at least 3 uops were executed per-thread",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "3",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.CYCLES_GE_3",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Cycles where at least 3 uops were executed per-thread.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles where at least 4 uops were executed per-thread",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "4",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.CYCLES_GE_4",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Cycles where at least 4 uops were executed per-thread.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts number of cycles no uops were dispatched to be executed on this thread.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.STALLS",
"Invert": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles during which no uops were dispatched from the Reservation Station (RS) per thread.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "This event is deprecated. Refer to new event UOPS_EXECUTED.STALLS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
+ "Deprecated": "1",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.STALL_CYCLES",
"Invert": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts the number of uops to be executed per-thread each cycle.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.THREAD",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
"BriefDescription": "Counts the number of x87 uops dispatched.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xb1",
"EventName": "UOPS_EXECUTED.X87",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of x87 uops executed.",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "Uops that RAT issues to RS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xae",
"EventName": "UOPS_ISSUED.ANY",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of uops that the Resource Allocation Table (RAT) issues to the Reservation Station (RS).",
"SampleAfterValue": "2000003",
- "Speculative": "1",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the total number of uops retired.",
+ "EventCode": "0xc2",
+ "EventName": "UOPS_RETIRED.ALL",
+ "PEBS": "1",
+ "SampleAfterValue": "2000003",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Cycles with retired uop(s).",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EventCode": "0xc2",
"EventName": "UOPS_RETIRED.CYCLES",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts cycles where at least one uop has retired.",
"SampleAfterValue": "1000003",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retired uops except the last uop of each instruction.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc2",
"EventName": "UOPS_RETIRED.HEAVY",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the number of retired micro-operations (uops) except the last uop of each instruction. An instruction that is decoded into less than two uops does not contribute to the count.",
"SampleAfterValue": "2000003",
"UMask": "0x1",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of integer divide uops retired.",
+ "EventCode": "0xc2",
+ "EventName": "UOPS_RETIRED.IDIV",
+ "PEBS": "1",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x10",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of uops that are from complex flows issued by the micro-sequencer (MS).",
+ "EventCode": "0xc2",
+ "EventName": "UOPS_RETIRED.MS",
+ "PEBS": "1",
+ "PublicDescription": "Counts the number of uops that are from complex flows issued by the Microcode Sequencer (MS). This includes uops from flows due to complex instructions, faults, assists, and inserted flows.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "UOPS_RETIRED.MS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc2",
"EventName": "UOPS_RETIRED.MS",
"MSRIndex": "0x3F7",
"MSRValue": "0x8",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "2000003",
- "TakenAlone": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Retirement slots used.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"EventCode": "0xc2",
"EventName": "UOPS_RETIRED.SLOTS",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "Counts the retirement slots used each cycle.",
"SampleAfterValue": "2000003",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles without actually retired uops.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
"EventCode": "0xc2",
"EventName": "UOPS_RETIRED.STALLS",
"Invert": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
+ "PublicDescription": "This event counts cycles without actually retired uops.",
"SampleAfterValue": "1000003",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "This event is deprecated. Refer to new event UOPS_RETIRED.STALLS",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5,6,7",
"CounterMask": "1",
+ "Deprecated": "1",
"EventCode": "0xc2",
"EventName": "UOPS_RETIRED.STALL_CYCLES",
"Invert": "1",
- "PEBScounters": "0,1,2,3,4,5,6,7",
"SampleAfterValue": "1000003",
"UMask": "0x2",
"Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Counts the number of x87 uops retired, includes those in MS flows.",
+ "EventCode": "0xc2",
+ "EventName": "UOPS_RETIRED.X87",
+ "PEBS": "1",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x2",
+ "Unit": "cpu_atom"
}
]
[
{
- "BriefDescription": "Number of clocks",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x01",
- "EventName": "UNC_M_CLOCKTICKS",
+ "BriefDescription": "Counts every 64B read request entering the Memory Controller 0 to DRAM (sum of all channels).",
+ "EventName": "UNC_MC0_RDCAS_COUNT_FREERUN",
"PerPkg": "1",
+ "PublicDescription": "Counts every 64B read request entering the Memory Controller 0 to DRAM (sum of all channels).",
"Unit": "iMC"
},
{
- "BriefDescription": "Incoming VC0 read request",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x02",
- "EventName": "UNC_M_VC0_REQUESTS_RD",
+ "BriefDescription": "Counts every 64B write request entering the Memory Controller 0 to DRAM (sum of all channels). Each write request counts as a new request incrementing this counter. However, same cache line write requests (both full and partial) are combined to a single 64 byte data transfer to DRAM.",
+ "EventName": "UNC_MC0_WRCAS_COUNT_FREERUN",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "Incoming VC0 write request",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x03",
- "EventName": "UNC_M_VC0_REQUESTS_WR",
+ "BriefDescription": "Counts every 64B read request entering the Memory Controller 1 to DRAM (sum of all channels).",
+ "EventName": "UNC_MC1_RDCAS_COUNT_FREERUN",
"PerPkg": "1",
+ "PublicDescription": "Counts every 64B read entering the Memory Controller 1 to DRAM (sum of all channels).",
"Unit": "iMC"
},
{
- "BriefDescription": "Incoming VC1 read request",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x04",
- "EventName": "UNC_M_VC1_REQUESTS_RD",
+ "BriefDescription": "Counts every 64B write request entering the Memory Controller 1 to DRAM (sum of all channels). Each write request counts as a new request incrementing this counter. However, same cache line write requests (both full and partial) are combined to a single 64 byte data transfer to DRAM.",
+ "EventName": "UNC_MC1_WRCAS_COUNT_FREERUN",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "Incoming VC1 write request",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x05",
- "EventName": "UNC_M_VC1_REQUESTS_WR",
+ "BriefDescription": "ACT command for a read request sent to DRAM",
+ "EventCode": "0x24",
+ "EventName": "UNC_M_ACT_COUNT_RD",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "Incoming read prefetch request from IA",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x0A",
- "EventName": "UNC_M_PREFETCH_RD",
+ "BriefDescription": "ACT command sent to DRAM",
+ "EventCode": "0x26",
+ "EventName": "UNC_M_ACT_COUNT_TOTAL",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "Any Rank at Hot state",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x19",
- "EventName": "UNC_M_DRAM_THERMAL_HOT",
+ "BriefDescription": "ACT command for a write request sent to DRAM",
+ "EventCode": "0x25",
+ "EventName": "UNC_M_ACT_COUNT_WR",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "Any Rank at Warm state",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x1A",
- "EventName": "UNC_M_DRAM_THERMAL_WARM",
+ "BriefDescription": "Read CAS command sent to DRAM",
+ "EventCode": "0x22",
+ "EventName": "UNC_M_CAS_COUNT_RD",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "incoming read request page status is Page Hit",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x1C",
- "EventName": "UNC_M_DRAM_PAGE_HIT_RD",
+ "BriefDescription": "Write CAS command sent to DRAM",
+ "EventCode": "0x23",
+ "EventName": "UNC_M_CAS_COUNT_WR",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Number of clocks",
+ "EventCode": "0x01",
+ "EventName": "UNC_M_CLOCKTICKS",
"PerPkg": "1",
"Unit": "iMC"
},
{
"BriefDescription": "incoming read request page status is Page Empty",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
"EventCode": "0x1D",
"EventName": "UNC_M_DRAM_PAGE_EMPTY_RD",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "incoming read request page status is Page Miss",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x1E",
- "EventName": "UNC_M_DRAM_PAGE_MISS_RD",
+ "BriefDescription": "incoming write request page status is Page Empty",
+ "EventCode": "0x20",
+ "EventName": "UNC_M_DRAM_PAGE_EMPTY_WR",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "incoming read request page status is Page Hit",
+ "EventCode": "0x1C",
+ "EventName": "UNC_M_DRAM_PAGE_HIT_RD",
"PerPkg": "1",
"Unit": "iMC"
},
{
"BriefDescription": "incoming write request page status is Page Hit",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
"EventCode": "0x1F",
"EventName": "UNC_M_DRAM_PAGE_HIT_WR",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "incoming write request page status is Page Empty",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x20",
- "EventName": "UNC_M_DRAM_PAGE_EMPTY_WR",
+ "BriefDescription": "incoming read request page status is Page Miss",
+ "EventCode": "0x1E",
+ "EventName": "UNC_M_DRAM_PAGE_MISS_RD",
"PerPkg": "1",
"Unit": "iMC"
},
{
"BriefDescription": "incoming write request page status is Page Miss",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
"EventCode": "0x21",
"EventName": "UNC_M_DRAM_PAGE_MISS_WR",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "Read CAS command sent to DRAM",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x22",
- "EventName": "UNC_M_CAS_COUNT_RD",
- "PerPkg": "1",
- "Unit": "iMC"
- },
- {
- "BriefDescription": "Write CAS command sent to DRAM",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x23",
- "EventName": "UNC_M_CAS_COUNT_WR",
+ "BriefDescription": "Any Rank at Hot state",
+ "EventCode": "0x19",
+ "EventName": "UNC_M_DRAM_THERMAL_HOT",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "ACT command for a read request sent to DRAM",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x24",
- "EventName": "UNC_M_ACT_COUNT_RD",
+ "BriefDescription": "Any Rank at Warm state",
+ "EventCode": "0x1A",
+ "EventName": "UNC_M_DRAM_THERMAL_WARM",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "ACT command for a write request sent to DRAM",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x25",
- "EventName": "UNC_M_ACT_COUNT_WR",
+ "BriefDescription": "Incoming read prefetch request from IA.",
+ "EventCode": "0x0A",
+ "EventName": "UNC_M_PREFETCH_RD",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "ACT command sent to DRAM",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x26",
- "EventName": "UNC_M_ACT_COUNT_TOTAL",
+ "BriefDescription": "PRE command sent to DRAM due to page table idle timer expiration",
+ "EventCode": "0x28",
+ "EventName": "UNC_M_PRE_COUNT_IDLE",
"PerPkg": "1",
"Unit": "iMC"
},
{
"BriefDescription": "PRE command sent to DRAM for a read/write request",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
"EventCode": "0x27",
"EventName": "UNC_M_PRE_COUNT_PAGE_MISS",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "PRE command sent to DRAM due to page table idle timer expiration",
- "Counter": "0,1,2,3,4",
- "CounterType": "PGMABLE",
- "EventCode": "0x28",
- "EventName": "UNC_M_PRE_COUNT_IDLE",
- "PerPkg": "1",
- "Unit": "iMC"
- },
- {
- "BriefDescription": "Counts every 64B read request entering the Memory Controller 0 to DRAM (sum of all channels)",
- "CounterType": "FREERUN",
- "EventName": "UNC_MC0_RDCAS_COUNT_FREERUN",
+ "BriefDescription": "Incoming VC0 read request",
+ "EventCode": "0x02",
+ "EventName": "UNC_M_VC0_REQUESTS_RD",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "Counts every 64B read request entering the Memory Controller 1 to DRAM (sum of all channels)",
- "Counter": "3",
- "CounterType": "FREERUN",
- "EventName": "UNC_MC1_RDCAS_COUNT_FREERUN",
+ "BriefDescription": "Incoming VC0 write request",
+ "EventCode": "0x03",
+ "EventName": "UNC_M_VC0_REQUESTS_WR",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "Counts every 64B write request entering the Memory Controller 0 to DRAM (sum of all channels). Each write request counts as a new request incrementing this counter. However, same cache line write requests (both full and partial) are combined to a single 64 byte data transfer to DRAM",
- "Counter": "1",
- "CounterType": "FREERUN",
- "EventName": "UNC_MC0_WRCAS_COUNT_FREERUN",
+ "BriefDescription": "Incoming VC1 read request",
+ "EventCode": "0x04",
+ "EventName": "UNC_M_VC1_REQUESTS_RD",
"PerPkg": "1",
"Unit": "iMC"
},
{
- "BriefDescription": "Counts every 64B write request entering the Memory Controller 1 to DRAM (sum of all channels). Each write request counts as a new request incrementing this counter. However, same cache line write requests (both full and partial) are combined to a single 64 byte data transfer to DRAM",
- "Counter": "4",
- "CounterType": "FREERUN",
- "EventName": "UNC_MC1_WRCAS_COUNT_FREERUN",
+ "BriefDescription": "Incoming VC1 write request",
+ "EventCode": "0x05",
+ "EventName": "UNC_M_VC1_REQUESTS_WR",
"PerPkg": "1",
"Unit": "iMC"
}
[
{
- "BriefDescription": "This 48-bit fixed counter counts the UCLK cycles",
- "Counter": "Fixed",
- "CounterType": "PGMABLE",
- "EventCode": "0xff",
- "EventName": "UNC_CLOCK.SOCKET",
+ "BriefDescription": "Number of requests allocated in Coherency Tracker.",
+ "EventCode": "0x84",
+ "EventName": "UNC_ARB_COH_TRK_REQUESTS.ALL",
"PerPkg": "1",
- "Unit": "CLOCK"
+ "UMask": "0x1",
+ "Unit": "ARB"
},
{
- "BriefDescription": "Counts the number of coherent and in-coherent requests initiated by IA cores, processor graphic units, or LLC",
- "Counter": "0,1",
- "CounterType": "PGMABLE",
+ "BriefDescription": "Each cycle counts number of any coherent request at memory controller that were issued by any core.",
+ "EventCode": "0x85",
+ "EventName": "UNC_ARB_DAT_OCCUPANCY.ALL",
+ "PerPkg": "1",
+ "UMask": "0x1",
+ "Unit": "ARB"
+ },
+ {
+ "BriefDescription": "Each cycle counts number of coherent reads pending on data return from memory controller that were issued by any core.",
+ "EventCode": "0x85",
+ "EventName": "UNC_ARB_DAT_OCCUPANCY.RD",
+ "PerPkg": "1",
+ "UMask": "0x2",
+ "Unit": "ARB"
+ },
+ {
+ "BriefDescription": "Number of coherent read requests sent to memory controller that were issued by any core.",
"EventCode": "0x81",
- "EventName": "UNC_ARB_TRK_REQUESTS.ALL",
+ "EventName": "UNC_ARB_DAT_REQUESTS.RD",
"PerPkg": "1",
- "UMask": "0x01",
+ "UMask": "0x2",
"Unit": "ARB"
},
{
- "BriefDescription": "Number of requests allocated in Coherency Tracker",
- "Counter": "0,1",
- "CounterType": "PGMABLE",
- "EventCode": "0x84",
- "EventName": "UNC_ARB_COH_TRK_REQUESTS.ALL",
+ "BriefDescription": "This event is deprecated. Refer to new event UNC_ARB_DAT_OCCUPANCY.ALL",
+ "EventCode": "0x85",
+ "EventName": "UNC_ARB_IFA_OCCUPANCY.ALL",
"PerPkg": "1",
- "UMask": "0x01",
+ "UMask": "0x1",
"Unit": "ARB"
},
{
- "BriefDescription": "Each cycle counts number of all outgoing valid entries in ReqTrk. Such entry is defined as valid from its allocation in ReqTrk till deallocation. Accounts for Coherent and non-coherent traffic",
- "CounterType": "PGMABLE",
+ "BriefDescription": "This event is deprecated. Refer to new event UNC_ARB_DAT_REQUESTS.RD",
+ "EventCode": "0x81",
+ "EventName": "UNC_ARB_REQ_TRK_REQUEST.DRD",
+ "PerPkg": "1",
+ "UMask": "0x2",
+ "Unit": "ARB"
+ },
+ {
+ "BriefDescription": "Each cycle counts number of all outgoing valid entries in ReqTrk. Such entry is defined as valid from its allocation in ReqTrk till deallocation. Accounts for Coherent and non-coherent traffic.",
"EventCode": "0x80",
"EventName": "UNC_ARB_TRK_OCCUPANCY.ALL",
"PerPkg": "1",
- "UMask": "0x01",
+ "UMask": "0x1",
"Unit": "ARB"
+ },
+ {
+ "BriefDescription": "Counts the number of coherent and in-coherent requests initiated by IA cores, processor graphic units, or LLC.",
+ "EventCode": "0x81",
+ "EventName": "UNC_ARB_TRK_REQUESTS.ALL",
+ "PerPkg": "1",
+ "UMask": "0x1",
+ "Unit": "ARB"
+ },
+ {
+ "BriefDescription": "This 48-bit fixed counter counts the UCLK cycles.",
+ "EventCode": "0xff",
+ "EventName": "UNC_CLOCK.SOCKET",
+ "PerPkg": "1",
+ "Unit": "CLOCK"
}
]
[
{
- "BriefDescription": "Counts the number of page walks completed due to load DTLB misses to any page size.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x08",
- "EventName": "DTLB_LOAD_MISSES.WALK_COMPLETED",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0xe",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of page walks completed due to store DTLB misses to any page size.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x49",
- "EventName": "DTLB_STORE_MISSES.WALK_COMPLETED",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "Speculative": "1",
- "UMask": "0xe",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of page walks initiated by a instruction fetch that missed the first and second level TLBs.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x85",
- "EventName": "ITLB_MISSES.MISS_CAUSED_WALK",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x1",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of page walks due to an instruction fetch that miss the PDE (Page Directory Entry) cache.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x85",
- "EventName": "ITLB_MISSES.PDE_CACHE_MISS",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "2000003",
- "Speculative": "1",
- "UMask": "0x80",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of page walks completed due to instruction fetch misses to any page size.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x85",
- "EventName": "ITLB_MISSES.WALK_COMPLETED",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "200003",
- "Speculative": "1",
- "UMask": "0xe",
- "Unit": "cpu_atom"
- },
- {
- "BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to a DTLB miss.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3,4,5",
- "EventCode": "0x05",
- "EventName": "LD_HEAD.DTLB_MISS_AT_RET",
- "PEBScounters": "0,1,2,3,4,5",
- "SampleAfterValue": "1000003",
- "Speculative": "1",
- "UMask": "0x90",
- "Unit": "cpu_atom"
- },
- {
"BriefDescription": "Loads that miss the DTLB and hit the STLB.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x12",
"EventName": "DTLB_LOAD_MISSES.STLB_HIT",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts loads that miss the DTLB (Data TLB) and hit the STLB (Second level TLB).",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles when at least one PMH is busy with a page walk for a demand load.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EventCode": "0x12",
"EventName": "DTLB_LOAD_MISSES.WALK_ACTIVE",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts cycles when at least one PMH (Page Miss Handler) is busy with a page walk for a demand load.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of page walks completed due to load DTLB misses to any page size.",
+ "EventCode": "0x08",
+ "EventName": "DTLB_LOAD_MISSES.WALK_COMPLETED",
+ "PublicDescription": "Counts the number of page walks completed due to loads (including SW prefetches) whose address translations missed in all Translation Lookaside Buffer (TLB) levels and were mapped to any page size. Includes page walks that page fault.",
+ "SampleAfterValue": "200003",
+ "UMask": "0xe",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Load miss in all TLB levels causes a page walk that completes. (All page sizes)",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x12",
"EventName": "DTLB_LOAD_MISSES.WALK_COMPLETED",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (all page sizes) caused by demand data loads. This implies it missed in the DTLB and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0xe",
"Unit": "cpu_core"
},
{
"BriefDescription": "Page walks completed due to a demand data load to a 1G page.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x12",
"EventName": "DTLB_LOAD_MISSES.WALK_COMPLETED_1G",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (1G sizes) caused by demand data loads. This implies address translations missed in the DTLB and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Page walks completed due to a demand data load to a 2M/4M page.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x12",
"EventName": "DTLB_LOAD_MISSES.WALK_COMPLETED_2M_4M",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (2M/4M sizes) caused by demand data loads. This implies address translations missed in the DTLB and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Page walks completed due to a demand data load to a 4K page.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x12",
"EventName": "DTLB_LOAD_MISSES.WALK_COMPLETED_4K",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (4K sizes) caused by demand data loads. This implies address translations missed in the DTLB and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of page walks outstanding for a demand load in the PMH each cycle.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x12",
"EventName": "DTLB_LOAD_MISSES.WALK_PENDING",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of page walks outstanding for a demand load in the PMH (Page Miss Handler) each cycle.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
"BriefDescription": "Stores that miss the DTLB and hit the STLB.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x13",
"EventName": "DTLB_STORE_MISSES.STLB_HIT",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts stores that miss the DTLB (Data TLB) and hit the STLB (2nd Level TLB).",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles when at least one PMH is busy with a page walk for a store.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EventCode": "0x13",
"EventName": "DTLB_STORE_MISSES.WALK_ACTIVE",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts cycles when at least one PMH (Page Miss Handler) is busy with a page walk for a store.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of page walks completed due to store DTLB misses to any page size.",
+ "EventCode": "0x49",
+ "EventName": "DTLB_STORE_MISSES.WALK_COMPLETED",
+ "PublicDescription": "Counts the number of page walks completed due to stores whose address translations missed in all Translation Lookaside Buffer (TLB) levels and were mapped to any page size. Includes page walks that page fault.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0xe",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Store misses in all TLB levels causes a page walk that completes. (All page sizes)",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x13",
"EventName": "DTLB_STORE_MISSES.WALK_COMPLETED",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (all page sizes) caused by demand data stores. This implies it missed in the DTLB and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0xe",
"Unit": "cpu_core"
},
{
"BriefDescription": "Page walks completed due to a demand data store to a 1G page.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x13",
"EventName": "DTLB_STORE_MISSES.WALK_COMPLETED_1G",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (1G sizes) caused by demand data stores. This implies address translations missed in the DTLB and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x8",
"Unit": "cpu_core"
},
{
"BriefDescription": "Page walks completed due to a demand data store to a 2M/4M page.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x13",
"EventName": "DTLB_STORE_MISSES.WALK_COMPLETED_2M_4M",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (2M/4M sizes) caused by demand data stores. This implies address translations missed in the DTLB and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Page walks completed due to a demand data store to a 4K page.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x13",
"EventName": "DTLB_STORE_MISSES.WALK_COMPLETED_4K",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (4K sizes) caused by demand data stores. This implies address translations missed in the DTLB and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of page walks outstanding for a store in the PMH each cycle.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x13",
"EventName": "DTLB_STORE_MISSES.WALK_PENDING",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of page walks outstanding for a store in the PMH (Page Miss Handler) each cycle.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of page walks initiated by a instruction fetch that missed the first and second level TLBs.",
+ "EventCode": "0x85",
+ "EventName": "ITLB_MISSES.MISS_CAUSED_WALK",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1",
+ "Unit": "cpu_atom"
+ },
+ {
+ "BriefDescription": "Counts the number of page walks due to an instruction fetch that miss the PDE (Page Directory Entry) cache.",
+ "EventCode": "0x85",
+ "EventName": "ITLB_MISSES.PDE_CACHE_MISS",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x80",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Instruction fetch requests that miss the ITLB and hit the STLB.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x11",
"EventName": "ITLB_MISSES.STLB_HIT",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts instruction fetch requests that miss the ITLB (Instruction TLB) and hit the STLB (Second-level TLB).",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x20",
"Unit": "cpu_core"
},
{
"BriefDescription": "Cycles when at least one PMH is busy with a page walk for code (instruction fetch) request.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"CounterMask": "1",
"EventCode": "0x11",
"EventName": "ITLB_MISSES.WALK_ACTIVE",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts cycles when at least one PMH (Page Miss Handler) is busy with a page walk for a code (instruction fetch) request.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
},
{
+ "BriefDescription": "Counts the number of page walks completed due to instruction fetch misses to any page size.",
+ "EventCode": "0x85",
+ "EventName": "ITLB_MISSES.WALK_COMPLETED",
+ "PublicDescription": "Counts the number of page walks completed due to instruction fetches whose address translations missed in all Translation Lookaside Buffer (TLB) levels and were mapped to any page size. Includes page walks that page fault.",
+ "SampleAfterValue": "200003",
+ "UMask": "0xe",
+ "Unit": "cpu_atom"
+ },
+ {
"BriefDescription": "Code miss in all TLB levels causes a page walk that completes. (All page sizes)",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x11",
"EventName": "ITLB_MISSES.WALK_COMPLETED",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (all page sizes) caused by a code fetch. This implies it missed in the ITLB (Instruction TLB) and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0xe",
"Unit": "cpu_core"
},
{
"BriefDescription": "Code miss in all TLB levels causes a page walk that completes. (2M/4M)",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x11",
"EventName": "ITLB_MISSES.WALK_COMPLETED_2M_4M",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (2M/4M page sizes) caused by a code fetch. This implies it missed in the ITLB (Instruction TLB) and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x4",
"Unit": "cpu_core"
},
{
"BriefDescription": "Code miss in all TLB levels causes a page walk that completes. (4K)",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x11",
"EventName": "ITLB_MISSES.WALK_COMPLETED_4K",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts completed page walks (4K page sizes) caused by a code fetch. This implies it missed in the ITLB (Instruction TLB) and further levels of TLB. The page walk can end with or without a fault.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x2",
"Unit": "cpu_core"
},
{
"BriefDescription": "Number of page walks outstanding for an outstanding code request in the PMH each cycle.",
- "CollectPEBSRecord": "2",
- "Counter": "0,1,2,3",
"EventCode": "0x11",
"EventName": "ITLB_MISSES.WALK_PENDING",
- "PEBScounters": "0,1,2,3",
+ "PublicDescription": "Counts the number of page walks outstanding for an outstanding code (instruction fetch) request in the PMH (Page Miss Handler) each cycle.",
"SampleAfterValue": "100003",
- "Speculative": "1",
"UMask": "0x10",
"Unit": "cpu_core"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to a DTLB miss.",
+ "EventCode": "0x05",
+ "EventName": "LD_HEAD.DTLB_MISS_AT_RET",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x90",
+ "Unit": "cpu_atom"
}
]
--- /dev/null
+[
+ {
+ "BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to frontend stalls.",
+ "MetricExpr": "TOPDOWN_FE_BOUND.ALL / SLOTS",
+ "MetricGroup": "TopdownL1",
+ "MetricName": "tma_frontend_bound",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not delivered by the frontend due to frontend bandwidth restrictions due to decode, predecode, cisc, and other limitations.",
+ "MetricExpr": "TOPDOWN_FE_BOUND.FRONTEND_LATENCY / SLOTS",
+ "MetricGroup": "TopdownL2;tma_frontend_bound_group",
+ "MetricName": "tma_frontend_latency",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not delivered by the frontend due to instruction cache misses.",
+ "MetricExpr": "TOPDOWN_FE_BOUND.ICACHE / SLOTS",
+ "MetricGroup": "TopdownL3;tma_frontend_latency_group",
+ "MetricName": "tma_icache",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not delivered by the frontend due to Instruction Table Lookaside Buffer (ITLB) misses.",
+ "MetricExpr": "TOPDOWN_FE_BOUND.ITLB / SLOTS",
+ "MetricGroup": "TopdownL3;tma_frontend_latency_group",
+ "MetricName": "tma_itlb",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not delivered by the frontend due to BACLEARS, which occurs when the Branch Target Buffer (BTB) prediction or lack thereof, was corrected by a later branch predictor in the frontend",
+ "MetricExpr": "TOPDOWN_FE_BOUND.BRANCH_DETECT / SLOTS",
+ "MetricGroup": "TopdownL3;tma_frontend_latency_group",
+ "MetricName": "tma_branch_detect",
+ "PublicDescription": "Counts the number of issue slots that were not delivered by the frontend due to BACLEARS, which occurs when the Branch Target Buffer (BTB) prediction or lack thereof, was corrected by a later branch predictor in the frontend. Includes BACLEARS due to all branch types including conditional and unconditional jumps, returns, and indirect branches.",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not delivered by the frontend due to BTCLEARS, which occurs when the Branch Target Buffer (BTB) predicts a taken branch.",
+ "MetricExpr": "TOPDOWN_FE_BOUND.BRANCH_RESTEER / SLOTS",
+ "MetricGroup": "TopdownL3;tma_frontend_latency_group",
+ "MetricName": "tma_branch_resteer",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not delivered by the frontend due to frontend bandwidth restrictions due to decode, predecode, cisc, and other limitations.",
+ "MetricExpr": "TOPDOWN_FE_BOUND.FRONTEND_BANDWIDTH / SLOTS",
+ "MetricGroup": "TopdownL2;tma_frontend_bound_group",
+ "MetricName": "tma_frontend_bandwidth",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not delivered by the frontend due to the microcode sequencer (MS).",
+ "MetricExpr": "TOPDOWN_FE_BOUND.CISC / SLOTS",
+ "MetricGroup": "TopdownL3;tma_frontend_bandwidth_group",
+ "MetricName": "tma_cisc",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not delivered by the frontend due to decode stalls.",
+ "MetricExpr": "TOPDOWN_FE_BOUND.DECODE / SLOTS",
+ "MetricGroup": "TopdownL3;tma_frontend_bandwidth_group",
+ "MetricName": "tma_decode",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not delivered by the frontend due to wrong predecodes.",
+ "MetricExpr": "TOPDOWN_FE_BOUND.PREDECODE / SLOTS",
+ "MetricGroup": "TopdownL3;tma_frontend_bandwidth_group",
+ "MetricName": "tma_predecode",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not delivered by the frontend due to other common frontend stalls not categorized.",
+ "MetricExpr": "TOPDOWN_FE_BOUND.OTHER / SLOTS",
+ "MetricGroup": "TopdownL3;tma_frontend_bandwidth_group",
+ "MetricName": "tma_other_fb",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a mispredicted jump or a machine clear",
+ "MetricExpr": "(SLOTS - (TOPDOWN_FE_BOUND.ALL + TOPDOWN_BE_BOUND.ALL + TOPDOWN_RETIRING.ALL)) / SLOTS",
+ "MetricGroup": "TopdownL1",
+ "MetricName": "tma_bad_speculation",
+ "PublicDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a mispredicted jump or a machine clear. Only issue slots wasted due to fast nukes such as memory ordering nukes are counted. Other nukes are not accounted for. Counts all issue slots blocked during this recovery window including relevant microcode flows and while uops are not yet available in the instruction queue (IQ). Also includes the issue slots that were consumed by the backend but were thrown away because they were younger than the mispredict or machine clear.",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to branch mispredicts.",
+ "MetricExpr": "TOPDOWN_BAD_SPECULATION.MISPREDICT / SLOTS",
+ "MetricGroup": "TopdownL2;tma_bad_speculation_group",
+ "MetricName": "tma_branch_mispredicts",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a machine clear (nuke) of any kind including memory ordering and memory disambiguation.",
+ "MetricExpr": "TOPDOWN_BAD_SPECULATION.MACHINE_CLEARS / SLOTS",
+ "MetricGroup": "TopdownL2;tma_bad_speculation_group",
+ "MetricName": "tma_machine_clears",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to a machine clear (slow nuke).",
+ "MetricExpr": "TOPDOWN_BAD_SPECULATION.NUKE / SLOTS",
+ "MetricGroup": "TopdownL3;tma_machine_clears_group",
+ "MetricName": "tma_nuke",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears relative to the number of nuke slots due to SMC. ",
+ "MetricExpr": "tma_nuke * (MACHINE_CLEARS.SMC / MACHINE_CLEARS.SLOW)",
+ "MetricGroup": "TopdownL4;tma_nuke_group",
+ "MetricName": "tma_smc",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears relative to the number of nuke slots due to memory ordering. ",
+ "MetricExpr": "tma_nuke * (MACHINE_CLEARS.MEMORY_ORDERING / MACHINE_CLEARS.SLOW)",
+ "MetricGroup": "TopdownL4;tma_nuke_group",
+ "MetricName": "tma_memory_ordering",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears relative to the number of nuke slots due to FP assists. ",
+ "MetricExpr": "tma_nuke * (MACHINE_CLEARS.FP_ASSIST / MACHINE_CLEARS.SLOW)",
+ "MetricGroup": "TopdownL4;tma_nuke_group",
+ "MetricName": "tma_fp_assist",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears relative to the number of nuke slots due to memory disambiguation. ",
+ "MetricExpr": "tma_nuke * (MACHINE_CLEARS.DISAMBIGUATION / MACHINE_CLEARS.SLOW)",
+ "MetricGroup": "TopdownL4;tma_nuke_group",
+ "MetricName": "tma_disambiguation",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears relative to the number of nuke slots due to page faults. ",
+ "MetricExpr": "tma_nuke * (MACHINE_CLEARS.PAGE_FAULT / MACHINE_CLEARS.SLOW)",
+ "MetricGroup": "TopdownL4;tma_nuke_group",
+ "MetricName": "tma_page_fault",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to a machine clear classified as a fast nuke due to memory ordering, memory disambiguation and memory renaming.",
+ "MetricExpr": "TOPDOWN_BAD_SPECULATION.FASTNUKE / SLOTS",
+ "MetricGroup": "TopdownL3;tma_machine_clears_group",
+ "MetricName": "tma_fast_nuke",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend due to backend stalls",
+ "MetricExpr": "TOPDOWN_BE_BOUND.ALL / SLOTS",
+ "MetricGroup": "TopdownL1",
+ "MetricName": "tma_backend_bound",
+ "PublicDescription": "Counts the total number of issue slots that were not consumed by the backend due to backend stalls. Note that uops must be available for consumption in order for this event to count. If a uop is not available (IQ is empty), this event will not count. The rest of these subevents count backend stalls, in cycles, due to an outstanding request which is memory bound vs core bound. The subevents are not slot based events and therefore can not be precisely added or subtracted from the Backend_Bound_Aux subevents which are slot based.",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles due to backend bound stalls that are core execution bound and not attributed to outstanding demand load or store stalls. ",
+ "MetricExpr": "max(0, tma_backend_bound - tma_load_store_bound)",
+ "MetricGroup": "TopdownL2;tma_backend_bound_group",
+ "MetricName": "tma_core_bound",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to stores or loads. ",
+ "MetricExpr": "min((TOPDOWN_BE_BOUND.ALL / SLOTS), (LD_HEAD.ANY_AT_RET / CLKS) + tma_store_bound)",
+ "MetricGroup": "TopdownL2;tma_backend_bound_group",
+ "MetricName": "tma_load_store_bound",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to store buffer full.",
+ "MetricExpr": "tma_mem_scheduler * (MEM_SCHEDULER_BLOCK.ST_BUF / MEM_SCHEDULER_BLOCK.ALL)",
+ "MetricGroup": "TopdownL3;tma_load_store_bound_group",
+ "MetricName": "tma_store_bound",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the oldest load of the load buffer is stalled at retirement due to a load block.",
+ "MetricExpr": "LD_HEAD.L1_BOUND_AT_RET / CLKS",
+ "MetricGroup": "TopdownL3;tma_load_store_bound_group",
+ "MetricName": "tma_l1_bound",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the oldest load of the load buffer is stalled at retirement due to a store forward block.",
+ "MetricExpr": "LD_HEAD.ST_ADDR_AT_RET / CLKS",
+ "MetricGroup": "TopdownL4;tma_l1_bound_group",
+ "MetricName": "tma_store_fwd",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the oldest load of the load buffer is stalled at retirement due to a first level TLB miss.",
+ "MetricExpr": "LD_HEAD.DTLB_MISS_AT_RET / CLKS",
+ "MetricGroup": "TopdownL4;tma_l1_bound_group",
+ "MetricName": "tma_stlb_hit",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the oldest load of the load buffer is stalled at retirement due to a second level TLB miss requiring a page walk.",
+ "MetricExpr": "LD_HEAD.PGWALK_AT_RET / CLKS",
+ "MetricGroup": "TopdownL4;tma_l1_bound_group",
+ "MetricName": "tma_stlb_miss",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the oldest load of the load buffer is stalled at retirement due to a number of other load blocks.",
+ "MetricExpr": "LD_HEAD.OTHER_AT_RET / CLKS",
+ "MetricGroup": "TopdownL4;tma_l1_bound_group",
+ "MetricName": "tma_other_l1",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles a core is stalled due to a demand load which hit in the L2 Cache.",
+ "MetricExpr": "(MEM_BOUND_STALLS.LOAD_L2_HIT / CLKS) - (MEM_BOUND_STALLS_AT_RET_CORRECTION * MEM_BOUND_STALLS.LOAD_L2_HIT / MEM_BOUND_STALLS.LOAD)",
+ "MetricGroup": "TopdownL3;tma_load_store_bound_group",
+ "MetricName": "tma_l2_bound",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles a core is stalled due to a demand load which hit in the Last Level Cache (LLC) or other core with HITE/F/M.",
+ "MetricExpr": "(MEM_BOUND_STALLS.LOAD_LLC_HIT / CLKS) - (MEM_BOUND_STALLS_AT_RET_CORRECTION * MEM_BOUND_STALLS.LOAD_LLC_HIT / MEM_BOUND_STALLS.LOAD)",
+ "MetricGroup": "TopdownL3;tma_load_store_bound_group",
+ "MetricName": "tma_l3_bound",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load miss which hit in DRAM or MMIO (Non-DRAM).",
+ "MetricExpr": "(MEM_BOUND_STALLS.LOAD_DRAM_HIT / CLKS) - (MEM_BOUND_STALLS_AT_RET_CORRECTION * MEM_BOUND_STALLS.LOAD_DRAM_HIT / MEM_BOUND_STALLS.LOAD)",
+ "MetricGroup": "TopdownL3;tma_load_store_bound_group",
+ "MetricName": "tma_dram_bound",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load miss which hits in the L2, LLC, DRAM or MMIO (Non-DRAM) but could not be correctly attributed or cycles in which the load miss is waiting on a request buffer.",
+ "MetricExpr": "max(0, tma_load_store_bound - (tma_store_bound + tma_l1_bound + tma_l2_bound + tma_l3_bound + tma_dram_bound))",
+ "MetricGroup": "TopdownL3;tma_load_store_bound_group",
+ "MetricName": "tma_other_load_store",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend due to backend stalls",
+ "MetricExpr": "tma_backend_bound",
+ "MetricGroup": "TopdownL1",
+ "MetricName": "tma_backend_bound_aux",
+ "PublicDescription": "Counts the total number of issue slots that were not consumed by the backend due to backend stalls. Note that UOPS must be available for consumption in order for this event to count. If a uop is not available (IQ is empty), this event will not count. All of these subevents count backend stalls, in slots, due to a resource limitation. These are not cycle based events and therefore can not be precisely added or subtracted from the Backend_Bound subevents which are cycle based. These subevents are supplementary to Backend_Bound and can be used to analyze results from a resource perspective at allocation. ",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend due to backend stalls",
+ "MetricExpr": "tma_backend_bound",
+ "MetricGroup": "TopdownL2;tma_backend_bound_aux_group",
+ "MetricName": "tma_resource_bound",
+ "PublicDescription": "Counts the total number of issue slots that were not consumed by the backend due to backend stalls. Note that uops must be available for consumption in order for this event to count. If a uop is not available (IQ is empty), this event will not count. ",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to memory reservation stalls in which a scheduler is not able to accept uops.",
+ "MetricExpr": "TOPDOWN_BE_BOUND.MEM_SCHEDULER / SLOTS",
+ "MetricGroup": "TopdownL3;tma_resource_bound_group",
+ "MetricName": "tma_mem_scheduler",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles, relative to the number of mem_scheduler slots, in which uops are blocked due to store buffer full",
+ "MetricExpr": "tma_mem_scheduler * (MEM_SCHEDULER_BLOCK.ST_BUF / MEM_SCHEDULER_BLOCK.ALL)",
+ "MetricGroup": "TopdownL4;tma_mem_scheduler_group",
+ "MetricName": "tma_st_buffer",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles, relative to the number of mem_scheduler slots, in which uops are blocked due to load buffer full",
+ "MetricExpr": "tma_mem_scheduler * MEM_SCHEDULER_BLOCK.LD_BUF / MEM_SCHEDULER_BLOCK.ALL",
+ "MetricGroup": "TopdownL4;tma_mem_scheduler_group",
+ "MetricName": "tma_ld_buffer",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles, relative to the number of mem_scheduler slots, in which uops are blocked due to RSV full relative ",
+ "MetricExpr": "tma_mem_scheduler * MEM_SCHEDULER_BLOCK.RSV / MEM_SCHEDULER_BLOCK.ALL",
+ "MetricGroup": "TopdownL4;tma_mem_scheduler_group",
+ "MetricName": "tma_rsv",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to IEC or FPC RAT stalls, which can be due to FIQ or IEC reservation stalls in which the integer, floating point or SIMD scheduler is not able to accept uops.",
+ "MetricExpr": "TOPDOWN_BE_BOUND.NON_MEM_SCHEDULER / SLOTS",
+ "MetricGroup": "TopdownL3;tma_resource_bound_group",
+ "MetricName": "tma_non_mem_scheduler",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to the physical register file unable to accept an entry (marble stalls).",
+ "MetricExpr": "TOPDOWN_BE_BOUND.REGISTER / SLOTS",
+ "MetricGroup": "TopdownL3;tma_resource_bound_group",
+ "MetricName": "tma_register",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to the reorder buffer being full (ROB stalls).",
+ "MetricExpr": "TOPDOWN_BE_BOUND.REORDER_BUFFER / SLOTS",
+ "MetricGroup": "TopdownL3;tma_resource_bound_group",
+ "MetricName": "tma_reorder_buffer",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to certain allocation restrictions.",
+ "MetricExpr": "TOPDOWN_BE_BOUND.ALLOC_RESTRICTIONS / SLOTS",
+ "MetricGroup": "TopdownL3;tma_resource_bound_group",
+ "MetricName": "tma_alloc_restriction",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots that were not consumed by the backend due to scoreboards from the instruction queue (IQ), jump execution unit (JEU), or microcode sequencer (MS).",
+ "MetricExpr": "TOPDOWN_BE_BOUND.SERIALIZATION / SLOTS",
+ "MetricGroup": "TopdownL3;tma_resource_bound_group",
+ "MetricName": "tma_serialization",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the numer of issue slots that result in retirement slots. ",
+ "MetricExpr": "TOPDOWN_RETIRING.ALL / SLOTS",
+ "MetricGroup": "TopdownL1",
+ "MetricName": "tma_retiring",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of uops that are not from the microsequencer. ",
+ "MetricExpr": "(TOPDOWN_RETIRING.ALL - UOPS_RETIRED.MS) / SLOTS",
+ "MetricGroup": "TopdownL2;tma_retiring_group",
+ "MetricName": "tma_base",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of floating point operations per uop with all default weighting.",
+ "MetricExpr": "UOPS_RETIRED.FPDIV / SLOTS",
+ "MetricGroup": "TopdownL3;tma_base_group",
+ "MetricName": "tma_fp_uops",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of uops retired excluding ms and fp div uops.",
+ "MetricExpr": "(TOPDOWN_RETIRING.ALL - UOPS_RETIRED.MS - UOPS_RETIRED.FPDIV) / SLOTS",
+ "MetricGroup": "TopdownL3;tma_base_group",
+ "MetricName": "tma_other_ret",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "Counts the number of uops that are from the complex flows issued by the micro-sequencer (MS)",
+ "MetricExpr": "UOPS_RETIRED.MS / SLOTS",
+ "MetricGroup": "TopdownL2;tma_retiring_group",
+ "MetricName": "tma_ms_uops",
+ "PublicDescription": "Counts the number of uops that are from the complex flows issued by the micro-sequencer (MS). This includes uops from flows due to complex instructions, faults, assists, and inserted flows.",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "",
+ "MetricExpr": "CPU_CLK_UNHALTED.CORE",
+ "MetricName": "CLKS"
+ },
+ {
+ "BriefDescription": "",
+ "MetricExpr": "CPU_CLK_UNHALTED.CORE_P",
+ "MetricName": "CLKS_P"
+ },
+ {
+ "BriefDescription": "",
+ "MetricExpr": "5 * CLKS",
+ "MetricName": "SLOTS"
+ },
+ {
+ "BriefDescription": "Instructions Per Cycle",
+ "MetricExpr": "INST_RETIRED.ANY / CLKS",
+ "MetricName": "IPC"
+ },
+ {
+ "BriefDescription": "Cycles Per Instruction",
+ "MetricExpr": "CLKS / INST_RETIRED.ANY",
+ "MetricName": "CPI"
+ },
+ {
+ "BriefDescription": "Uops Per Instruction",
+ "MetricExpr": "UOPS_RETIRED.ALL / INST_RETIRED.ANY",
+ "MetricName": "UPI"
+ },
+ {
+ "BriefDescription": "Percentage of total non-speculative loads with a store forward or unknown store address block",
+ "MetricExpr": "100 * LD_BLOCKS.DATA_UNKNOWN / MEM_UOPS_RETIRED.ALL_LOADS",
+ "MetricName": "Store_Fwd_Blocks"
+ },
+ {
+ "BriefDescription": "Percentage of total non-speculative loads with a address aliasing block",
+ "MetricExpr": "100 * LD_BLOCKS.4K_ALIAS / MEM_UOPS_RETIRED.ALL_LOADS",
+ "MetricName": "Address_Alias_Blocks"
+ },
+ {
+ "BriefDescription": "Percentage of total non-speculative loads that are splits",
+ "MetricExpr": "100 * MEM_UOPS_RETIRED.SPLIT_LOADS / MEM_UOPS_RETIRED.ALL_LOADS",
+ "MetricName": "Load_Splits"
+ },
+ {
+ "BriefDescription": "Instructions per Branch (lower number means higher occurrence rate)",
+ "MetricExpr": "INST_RETIRED.ANY / BR_INST_RETIRED.ALL_BRANCHES",
+ "MetricName": "IpBranch"
+ },
+ {
+ "BriefDescription": "Instruction per (near) call (lower number means higher occurrence rate)",
+ "MetricExpr": "INST_RETIRED.ANY / BR_INST_RETIRED.CALL",
+ "MetricName": "IpCall"
+ },
+ {
+ "BriefDescription": "Instructions per Load",
+ "MetricExpr": "INST_RETIRED.ANY / MEM_UOPS_RETIRED.ALL_LOADS",
+ "MetricName": "IpLoad"
+ },
+ {
+ "BriefDescription": "Instructions per Store",
+ "MetricExpr": "INST_RETIRED.ANY / MEM_UOPS_RETIRED.ALL_STORES",
+ "MetricName": "IpStore"
+ },
+ {
+ "BriefDescription": "Number of Instructions per non-speculative Branch Misprediction",
+ "MetricExpr": "INST_RETIRED.ANY / BR_MISP_RETIRED.ALL_BRANCHES",
+ "MetricName": "IpMispredict"
+ },
+ {
+ "BriefDescription": "Instructions per Far Branch",
+ "MetricExpr": "INST_RETIRED.ANY / (BR_INST_RETIRED.FAR_BRANCH / 2)",
+ "MetricName": "IpFarBranch"
+ },
+ {
+ "BriefDescription": "Ratio of all branches which mispredict",
+ "MetricExpr": "BR_MISP_RETIRED.ALL_BRANCHES / BR_INST_RETIRED.ALL_BRANCHES",
+ "MetricName": "Branch_Mispredict_Ratio"
+ },
+ {
+ "BriefDescription": "Ratio between Mispredicted branches and unknown branches",
+ "MetricExpr": "BR_MISP_RETIRED.ALL_BRANCHES / BACLEARS.ANY",
+ "MetricName": "Branch_Mispredict_to_Unknown_Branch_Ratio"
+ },
+ {
+ "BriefDescription": "Percentage of all uops which are ucode ops",
+ "MetricExpr": "100 * UOPS_RETIRED.MS / UOPS_RETIRED.ALL",
+ "MetricName": "Microcode_Uop_Ratio"
+ },
+ {
+ "BriefDescription": "Percentage of all uops which are FPDiv uops",
+ "MetricExpr": "100 * UOPS_RETIRED.FPDIV / UOPS_RETIRED.ALL",
+ "MetricName": "FPDiv_Uop_Ratio"
+ },
+ {
+ "BriefDescription": "Percentage of all uops which are IDiv uops",
+ "MetricExpr": "100 * UOPS_RETIRED.IDIV / UOPS_RETIRED.ALL",
+ "MetricName": "IDiv_Uop_Ratio"
+ },
+ {
+ "BriefDescription": "Percentage of all uops which are x87 uops",
+ "MetricExpr": "100 * UOPS_RETIRED.X87 / UOPS_RETIRED.ALL",
+ "MetricName": "X87_Uop_Ratio"
+ },
+ {
+ "BriefDescription": "Average Frequency Utilization relative nominal frequency",
+ "MetricExpr": "CLKS / CPU_CLK_UNHALTED.REF_TSC",
+ "MetricName": "Turbo_Utilization"
+ },
+ {
+ "BriefDescription": "Fraction of cycles spent in Kernel mode",
+ "MetricExpr": "cpu@CPU_CLK_UNHALTED.CORE@k / CPU_CLK_UNHALTED.CORE",
+ "MetricName": "Kernel_Utilization"
+ },
+ {
+ "BriefDescription": "Average CPU Utilization",
+ "MetricExpr": "CPU_CLK_UNHALTED.REF_TSC / TSC",
+ "MetricName": "CPU_Utilization"
+ },
+ {
+ "BriefDescription": "Cycle cost per L2 hit",
+ "MetricExpr": "MEM_BOUND_STALLS.LOAD_L2_HIT / MEM_LOAD_UOPS_RETIRED.L2_HIT",
+ "MetricName": "Cycles_per_Demand_Load_L2_Hit"
+ },
+ {
+ "BriefDescription": "Cycle cost per LLC hit",
+ "MetricExpr": "MEM_BOUND_STALLS.LOAD_LLC_HIT / MEM_LOAD_UOPS_RETIRED.L3_HIT",
+ "MetricName": "Cycles_per_Demand_Load_L3_Hit"
+ },
+ {
+ "BriefDescription": "Cycle cost per DRAM hit",
+ "MetricExpr": "MEM_BOUND_STALLS.LOAD_DRAM_HIT / MEM_LOAD_UOPS_RETIRED.DRAM_HIT",
+ "MetricName": "Cycles_per_Demand_Load_DRAM_Hit"
+ },
+ {
+ "BriefDescription": "Percent of instruction miss cost that hit in the L2",
+ "MetricExpr": "100 * MEM_BOUND_STALLS.IFETCH_L2_HIT / (MEM_BOUND_STALLS.IFETCH)",
+ "MetricName": "Inst_Miss_Cost_L2Hit_Percent"
+ },
+ {
+ "BriefDescription": "Percent of instruction miss cost that hit in the L3",
+ "MetricExpr": "100 * MEM_BOUND_STALLS.IFETCH_LLC_HIT / (MEM_BOUND_STALLS.IFETCH)",
+ "MetricName": "Inst_Miss_Cost_L3Hit_Percent"
+ },
+ {
+ "BriefDescription": "Percent of instruction miss cost that hit in DRAM",
+ "MetricExpr": "100 * MEM_BOUND_STALLS.IFETCH_DRAM_HIT / (MEM_BOUND_STALLS.IFETCH)",
+ "MetricName": "Inst_Miss_Cost_DRAMHit_Percent"
+ },
+ {
+ "BriefDescription": "load ops retired per 1000 instruction",
+ "MetricExpr": "1000 * MEM_UOPS_RETIRED.ALL_LOADS / INST_RETIRED.ANY",
+ "MetricName": "MemLoadPKI"
+ },
+ {
+ "BriefDescription": "C1 residency percent per core",
+ "MetricExpr": "cstate_core@c1\\-residency@ / TSC",
+ "MetricGroup": "Power",
+ "MetricName": "C1_Core_Residency",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "C6 residency percent per core",
+ "MetricExpr": "cstate_core@c6\\-residency@ / TSC",
+ "MetricGroup": "Power",
+ "MetricName": "C6_Core_Residency",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "C7 residency percent per core",
+ "MetricExpr": "cstate_core@c7\\-residency@ / TSC",
+ "MetricGroup": "Power",
+ "MetricName": "C7_Core_Residency",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "C2 residency percent per package",
+ "MetricExpr": "cstate_pkg@c2\\-residency@ / TSC",
+ "MetricGroup": "Power",
+ "MetricName": "C2_Pkg_Residency",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "C3 residency percent per package",
+ "MetricExpr": "cstate_pkg@c3\\-residency@ / TSC",
+ "MetricGroup": "Power",
+ "MetricName": "C3_Pkg_Residency",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "C6 residency percent per package",
+ "MetricExpr": "cstate_pkg@c6\\-residency@ / TSC",
+ "MetricGroup": "Power",
+ "MetricName": "C6_Pkg_Residency",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "C7 residency percent per package",
+ "MetricExpr": "cstate_pkg@c7\\-residency@ / TSC",
+ "MetricGroup": "Power",
+ "MetricName": "C7_Pkg_Residency",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "C8 residency percent per package",
+ "MetricExpr": "cstate_pkg@c8\\-residency@ / TSC",
+ "MetricGroup": "Power",
+ "MetricName": "C8_Pkg_Residency",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "C9 residency percent per package",
+ "MetricExpr": "cstate_pkg@c9\\-residency@ / TSC",
+ "MetricGroup": "Power",
+ "MetricName": "C9_Pkg_Residency",
+ "ScaleUnit": "100%"
+ },
+ {
+ "BriefDescription": "C10 residency percent per package",
+ "MetricExpr": "cstate_pkg@c10\\-residency@ / TSC",
+ "MetricGroup": "Power",
+ "MetricName": "C10_Pkg_Residency",
+ "ScaleUnit": "100%"
+ }
+]
--- /dev/null
+[
+ {
+ "BriefDescription": "Counts the number of cacheable memory requests that miss in the LLC. Counts on a per core basis.",
+ "EventCode": "0x2e",
+ "EventName": "LONGEST_LAT_CACHE.MISS",
+ "PublicDescription": "Counts the number of cacheable memory requests that miss in the Last Level Cache (LLC). Requests include demand loads, reads for ownership (RFO), instruction fetches and L1 HW prefetches. If the platform has an L3 cache, the LLC is the L3 cache, otherwise it is the L2 cache. Counts on a per core basis.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x41"
+ },
+ {
+ "BriefDescription": "Counts the number of cacheable memory requests that access the LLC. Counts on a per core basis.",
+ "EventCode": "0x2e",
+ "EventName": "LONGEST_LAT_CACHE.REFERENCE",
+ "PublicDescription": "Counts the number of cacheable memory requests that access the Last Level Cache (LLC). Requests include demand loads, reads for ownership (RFO), instruction fetches and L1 HW prefetches. If the platform has an L3 cache, the LLC is the L3 cache, otherwise it is the L2 cache. Counts on a per core basis.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x4f"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in the L2, LLC, DRAM or MMIO (Non-DRAM).",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.IFETCH",
+ "PublicDescription": "Counts the number of cycles the core is stalled due to an instruction cache or translation lookaside buffer (TLB) miss which hit in the L2, LLC, DRAM or MMIO (Non-DRAM).",
+ "SampleAfterValue": "200003",
+ "UMask": "0x38"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in DRAM or MMIO (Non-DRAM).",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.IFETCH_DRAM_HIT",
+ "PublicDescription": "Counts the number of cycles the core is stalled due to an instruction cache or translation lookaside buffer (TLB) miss which hit in DRAM or MMIO (non-DRAM).",
+ "SampleAfterValue": "200003",
+ "UMask": "0x20"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in the L2 cache.",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.IFETCH_L2_HIT",
+ "PublicDescription": "Counts the number of cycles the core is stalled due to an instruction cache or Translation Lookaside Buffer (TLB) miss which hit in the L2 cache.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x8"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to an instruction cache or TLB miss which hit in the LLC or other core with HITE/F/M.",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.IFETCH_LLC_HIT",
+ "PublicDescription": "Counts the number of cycles the core is stalled due to an instruction cache or Translation Lookaside Buffer (TLB) miss which hit in the Last Level Cache (LLC) or other core with HITE/F/M.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x10"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load miss which hit in the L2, LLC, DRAM or MMIO (Non-DRAM).",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.LOAD",
+ "SampleAfterValue": "200003",
+ "UMask": "0x7"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load miss which hit in DRAM or MMIO (Non-DRAM).",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.LOAD_DRAM_HIT",
+ "SampleAfterValue": "200003",
+ "UMask": "0x4"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load which hit in the L2 cache.",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.LOAD_L2_HIT",
+ "SampleAfterValue": "200003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles the core is stalled due to a demand load which hit in the LLC or other core with HITE/F/M.",
+ "EventCode": "0x34",
+ "EventName": "MEM_BOUND_STALLS.LOAD_LLC_HIT",
+ "PublicDescription": "Counts the number of cycles the core is stalled due to a demand load which hit in the Last Level Cache (LLC) or other core with HITE/F/M.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x2"
+ },
+ {
+ "BriefDescription": "Counts the number of load uops retired that hit in DRAM.",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_UOPS_RETIRED.DRAM_HIT",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x80"
+ },
+ {
+ "BriefDescription": "Counts the number of load uops retired that hit in the L2 cache.",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_UOPS_RETIRED.L2_HIT",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x2"
+ },
+ {
+ "BriefDescription": "Counts the number of load uops retired that hit in the L3 cache.",
+ "Data_LA": "1",
+ "EventCode": "0xd1",
+ "EventName": "MEM_LOAD_UOPS_RETIRED.L3_HIT",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x4"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that uops are blocked for any of the following reasons: load buffer, store buffer or RSV full.",
+ "EventCode": "0x04",
+ "EventName": "MEM_SCHEDULER_BLOCK.ALL",
+ "SampleAfterValue": "20003",
+ "UMask": "0x7"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that uops are blocked due to a load buffer full condition.",
+ "EventCode": "0x04",
+ "EventName": "MEM_SCHEDULER_BLOCK.LD_BUF",
+ "SampleAfterValue": "20003",
+ "UMask": "0x2"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that uops are blocked due to an RSV full condition.",
+ "EventCode": "0x04",
+ "EventName": "MEM_SCHEDULER_BLOCK.RSV",
+ "SampleAfterValue": "20003",
+ "UMask": "0x4"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that uops are blocked due to a store buffer full condition.",
+ "EventCode": "0x04",
+ "EventName": "MEM_SCHEDULER_BLOCK.ST_BUF",
+ "SampleAfterValue": "20003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of load uops retired.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.ALL_LOADS",
+ "PEBS": "1",
+ "PublicDescription": "Counts the total number of load uops retired.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x81"
+ },
+ {
+ "BriefDescription": "Counts the number of store uops retired.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.ALL_STORES",
+ "PEBS": "1",
+ "PublicDescription": "Counts the total number of store uops retired.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x82"
+ },
+ {
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 128 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_128",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x80",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 128 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5"
+ },
+ {
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 16 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_16",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x10",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 16 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5"
+ },
+ {
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 256 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_256",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x100",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 256 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5"
+ },
+ {
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 32 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_32",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x20",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 32 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5"
+ },
+ {
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 4 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_4",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x4",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 4 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5"
+ },
+ {
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 512 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_512",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x200",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 512 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5"
+ },
+ {
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 64 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_64",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x40",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 64 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5"
+ },
+ {
+ "BriefDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 8 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.LOAD_LATENCY_GT_8",
+ "MSRIndex": "0x3F6",
+ "MSRValue": "0x8",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of tagged loads with an instruction latency that exceeds or equals the threshold of 8 cycles as defined in MEC_CR_PEBS_LD_LAT_THRESHOLD (3F6H). Only counts with PEBS enabled. If a PEBS record is generated, will populate the PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x5"
+ },
+ {
+ "BriefDescription": "Counts the number of retired split load uops.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.SPLIT_LOADS",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x41"
+ },
+ {
+ "BriefDescription": "Counts the number of stores uops retired. Counts with or without PEBS enabled.",
+ "Data_LA": "1",
+ "EventCode": "0xd0",
+ "EventName": "MEM_UOPS_RETIRED.STORE_LATENCY",
+ "PEBS": "2",
+ "PublicDescription": "Counts the number of stores uops retired. Counts with or without PEBS enabled. If PEBS is enabled and a PEBS record is generated, will populate PEBS Latency and PEBS Data Source fields accordingly.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x6"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that were supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_HIT",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F803C0001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that were supplied by the L3 cache where a snoop was sent, the snoop hit, and modified data was forwarded.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HITM",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10003C0001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that were supplied by the L3 cache where a snoop was sent, the snoop hit, but no data was forwarded.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HIT_NO_FWD",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x4003C0001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that were supplied by the L3 cache where a snoop was sent, the snoop hit, and non-modified data was forwarded.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_HIT.SNOOP_HIT_WITH_FWD",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x8003C0001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_RFO.L3_HIT",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F803C0002",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were supplied by the L3 cache where a snoop was sent, the snoop hit, and modified data was forwarded.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_RFO.L3_HIT.SNOOP_HITM",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10003C0002",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to instruction cache misses.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.ICACHE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x20"
+ }
+]
--- /dev/null
+[
+ {
+ "BriefDescription": "Counts the number of floating point operations retired that required microcode assist.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.FP_ASSIST",
+ "PublicDescription": "Counts the number of floating point operations retired that required microcode assist, which is not a reflection of the number of FP operations, instructions or uops.",
+ "SampleAfterValue": "20003",
+ "UMask": "0x4"
+ },
+ {
+ "BriefDescription": "Counts the number of floating point divide uops retired (x87 and SSE, including x87 sqrt).",
+ "EventCode": "0xc2",
+ "EventName": "UOPS_RETIRED.FPDIV",
+ "PEBS": "1",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x8"
+ }
+]
--- /dev/null
+[
+ {
+ "BriefDescription": "Counts the total number of BACLEARS due to all branch types including conditional and unconditional jumps, returns, and indirect branches.",
+ "EventCode": "0xe6",
+ "EventName": "BACLEARS.ANY",
+ "PublicDescription": "Counts the total number of BACLEARS, which occur when the Branch Target Buffer (BTB) prediction or lack thereof, was corrected by a later branch predictor in the frontend. Includes BACLEARS due to all branch types including conditional and unconditional jumps, returns, and indirect branches.",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of requests to the instruction cache for one or more bytes of a cache line.",
+ "EventCode": "0x80",
+ "EventName": "ICACHE.ACCESSES",
+ "PublicDescription": "Counts the total number of requests to the instruction cache. The event only counts new cache line accesses, so that multiple back to back fetches to the exact same cache line or byte chunk count as one. Specifically, the event counts when accesses from sequential code crosses the cache line boundary, or when a branch target is moved to a new line or to a non-sequential byte chunk of the same line.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x3"
+ },
+ {
+ "BriefDescription": "Counts the number of instruction cache misses.",
+ "EventCode": "0x80",
+ "EventName": "ICACHE.MISSES",
+ "PublicDescription": "Counts the number of missed requests to the instruction cache. The event only counts new cache line accesses, so that multiple back to back fetches to the exact same cache line and byte chunk count as one. Specifically, the event counts when accesses from sequential code crosses the cache line boundary, or when a branch target is moved to a new line or to a non-sequential byte chunk of the same line.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x2"
+ }
+]
--- /dev/null
+[
+ {
+ "BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer is stalled due to any number of reasons, including an L1 miss, WCB full, pagewalk, store address block or store data block, on a load that retires.",
+ "EventCode": "0x05",
+ "EventName": "LD_HEAD.ANY_AT_RET",
+ "SampleAfterValue": "1000003",
+ "UMask": "0xff"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer is stalled due to a core bound stall including a store address match, a DTLB miss or a page walk that detains the load from retiring.",
+ "EventCode": "0x05",
+ "EventName": "LD_HEAD.L1_BOUND_AT_RET",
+ "SampleAfterValue": "1000003",
+ "UMask": "0xf4"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to other block cases.",
+ "EventCode": "0x05",
+ "EventName": "LD_HEAD.OTHER_AT_RET",
+ "PublicDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to other block cases such as pipeline conflicts, fences, etc.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0xc0"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to a pagewalk.",
+ "EventCode": "0x05",
+ "EventName": "LD_HEAD.PGWALK_AT_RET",
+ "SampleAfterValue": "1000003",
+ "UMask": "0xa0"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to a store address match.",
+ "EventCode": "0x05",
+ "EventName": "LD_HEAD.ST_ADDR_AT_RET",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x84"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears due to memory ordering caused by a snoop from an external agent. Does not count internally generated machine clears such as those due to memory disambiguation.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.MEMORY_ORDERING",
+ "SampleAfterValue": "20003",
+ "UMask": "0x2"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that were not supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_MISS",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F84400001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that were not supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.L3_MISS_LOCAL",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F84400001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were not supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_RFO.L3_MISS",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F84400002",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that were not supplied by the L3 cache.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_RFO.L3_MISS_LOCAL",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x3F84400002",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ }
+]
--- /dev/null
+[
+ {
+ "BriefDescription": "Counts modified writebacks from L1 cache and L2 cache that have any type of response.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.COREWB_M.ANY_RESPONSE",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10008",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts demand data reads that have any type of response.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_DATA_RD.ANY_RESPONSE",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10001",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts demand reads for ownership (RFO) and software prefetches for exclusive ownership (PREFETCHW) that have any type of response.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.DEMAND_RFO.ANY_RESPONSE",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10002",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts streaming stores that have any type of response.",
+ "EventCode": "0xB7",
+ "EventName": "OCR.STREAMING_WR.ANY_RESPONSE",
+ "MSRIndex": "0x1a6,0x1a7",
+ "MSRValue": "0x10800",
+ "SampleAfterValue": "100003",
+ "UMask": "0x1"
+ }
+]
--- /dev/null
+[
+ {
+ "BriefDescription": "Counts the total number of branch instructions retired for all branch types.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.ALL_BRANCHES",
+ "PEBS": "1",
+ "PublicDescription": "Counts the total number of instructions in which the instruction pointer (IP) of the processor is resteered due to a branch instruction and the branch instruction successfully retires. All branch type instructions are accounted for.",
+ "SampleAfterValue": "200003"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.NEAR_CALL",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xf9"
+ },
+ {
+ "BriefDescription": "Counts the number of retired JCC (Jump on Conditional Code) branch instructions retired, includes both taken and not taken branches.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.COND",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x7e"
+ },
+ {
+ "BriefDescription": "Counts the number of taken JCC (Jump on Conditional Code) branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.COND_TAKEN",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfe"
+ },
+ {
+ "BriefDescription": "Counts the number of far branch instructions retired, includes far jump, far call and return, and interrupt call and return.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.FAR_BRANCH",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xbf"
+ },
+ {
+ "BriefDescription": "Counts the number of near indirect JMP and near indirect CALL branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.INDIRECT",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xeb"
+ },
+ {
+ "BriefDescription": "Counts the number of near indirect CALL branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.INDIRECT_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfb"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.INDIRECT_CALL",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.IND_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfb"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.COND",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.JCC",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x7e"
+ },
+ {
+ "BriefDescription": "Counts the number of near CALL branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.NEAR_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xf9"
+ },
+ {
+ "BriefDescription": "Counts the number of near RET branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.NEAR_RETURN",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xf7"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.INDIRECT",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.NON_RETURN_IND",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xeb"
+ },
+ {
+ "BriefDescription": "Counts the number of near relative CALL branch instructions retired.",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.REL_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfd"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.NEAR_RETURN",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.RETURN",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xf7"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_INST_RETIRED.COND_TAKEN",
+ "Deprecated": "1",
+ "EventCode": "0xc4",
+ "EventName": "BR_INST_RETIRED.TAKEN_JCC",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfe"
+ },
+ {
+ "BriefDescription": "Counts the total number of mispredicted branch instructions retired for all branch types.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.ALL_BRANCHES",
+ "PEBS": "1",
+ "PublicDescription": "Counts the total number of mispredicted branch instructions retired. All branch type instructions are accounted for. Prediction of the branch target address enables the processor to begin executing instructions before the non-speculative execution path is known. The branch prediction unit (BPU) predicts the target address based on the instruction pointer (IP) of the branch and on the execution path through which execution reached this IP. A branch misprediction occurs when the prediction is wrong, and results in discarding all instructions executed in the speculative path and re-fetching from the correct path.",
+ "SampleAfterValue": "200003"
+ },
+ {
+ "BriefDescription": "Counts the number of mispredicted JCC (Jump on Conditional Code) branch instructions retired.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.COND",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x7e"
+ },
+ {
+ "BriefDescription": "Counts the number of mispredicted taken JCC (Jump on Conditional Code) branch instructions retired.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.COND_TAKEN",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfe"
+ },
+ {
+ "BriefDescription": "Counts the number of mispredicted near indirect JMP and near indirect CALL branch instructions retired.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.INDIRECT",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xeb"
+ },
+ {
+ "BriefDescription": "Counts the number of mispredicted near indirect CALL branch instructions retired.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.INDIRECT_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfb"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.INDIRECT_CALL",
+ "Deprecated": "1",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.IND_CALL",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfb"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.COND",
+ "Deprecated": "1",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.JCC",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0x7e"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.INDIRECT",
+ "Deprecated": "1",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.NON_RETURN_IND",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xeb"
+ },
+ {
+ "BriefDescription": "Counts the number of mispredicted near RET branch instructions retired.",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.RETURN",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xf7"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event BR_MISP_RETIRED.COND_TAKEN",
+ "Deprecated": "1",
+ "EventCode": "0xc5",
+ "EventName": "BR_MISP_RETIRED.TAKEN_JCC",
+ "PEBS": "1",
+ "SampleAfterValue": "200003",
+ "UMask": "0xfe"
+ },
+ {
+ "BriefDescription": "Counts the number of unhalted core clock cycles. (Fixed event)",
+ "EventName": "CPU_CLK_UNHALTED.CORE",
+ "PublicDescription": "Counts the number of core cycles while the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. The core frequency may change from time to time. For this reason this event may have a changing ratio with regards to time. This event uses fixed counter 1.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x2"
+ },
+ {
+ "BriefDescription": "Counts the number of unhalted core clock cycles.",
+ "EventCode": "0x3c",
+ "EventName": "CPU_CLK_UNHALTED.CORE_P",
+ "PublicDescription": "Counts the number of core cycles while the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. The core frequency may change from time to time. For this reason this event may have a changing ratio with regards to time. This event uses a programmable general purpose performance counter.",
+ "SampleAfterValue": "2000003"
+ },
+ {
+ "BriefDescription": "Counts the number of unhalted reference clock cycles at TSC frequency. (Fixed event)",
+ "EventName": "CPU_CLK_UNHALTED.REF_TSC",
+ "PublicDescription": "Counts the number of reference cycles that the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. This event is not affected by core frequency changes and increments at a fixed frequency that is also used for the Time Stamp Counter (TSC). This event uses fixed counter 2.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x3"
+ },
+ {
+ "BriefDescription": "Counts the number of unhalted reference clock cycles at TSC frequency.",
+ "EventCode": "0x3c",
+ "EventName": "CPU_CLK_UNHALTED.REF_TSC_P",
+ "PublicDescription": "Counts the number of reference cycles that the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. This event is not affected by core frequency changes and increments at a fixed frequency that is also used for the Time Stamp Counter (TSC). This event uses a programmable general purpose performance counter.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of unhalted core clock cycles. (Fixed event)",
+ "EventName": "CPU_CLK_UNHALTED.THREAD",
+ "PublicDescription": "Counts the number of core cycles while the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. The core frequency may change from time to time. For this reason this event may have a changing ratio with regards to time. This event uses fixed counter 1.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x2"
+ },
+ {
+ "BriefDescription": "Counts the number of unhalted core clock cycles.",
+ "EventCode": "0x3c",
+ "EventName": "CPU_CLK_UNHALTED.THREAD_P",
+ "PublicDescription": "Counts the number of core cycles while the core is not in a halt state. The core enters the halt state when it is running the HLT instruction. The core frequency may change from time to time. For this reason this event may have a changing ratio with regards to time. This event uses a programmable general purpose performance counter.",
+ "SampleAfterValue": "2000003"
+ },
+ {
+ "BriefDescription": "Counts the total number of instructions retired. (Fixed event)",
+ "EventName": "INST_RETIRED.ANY",
+ "PEBS": "1",
+ "PublicDescription": "Counts the total number of instructions that retired. For instructions that consist of multiple uops, this event counts the retirement of the last uop of the instruction. This event continues counting during hardware interrupts, traps, and inside interrupt handlers. This event uses fixed counter 0.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the total number of instructions retired.",
+ "EventCode": "0xc0",
+ "EventName": "INST_RETIRED.ANY_P",
+ "PEBS": "1",
+ "PublicDescription": "Counts the total number of instructions that retired. For instructions that consist of multiple uops, this event counts the retirement of the last uop of the instruction. This event continues counting during hardware interrupts, traps, and inside interrupt handlers. This event uses a programmable general purpose performance counter.",
+ "SampleAfterValue": "2000003"
+ },
+ {
+ "BriefDescription": "This event is deprecated. Refer to new event LD_BLOCKS.ADDRESS_ALIAS",
+ "Deprecated": "1",
+ "EventCode": "0x03",
+ "EventName": "LD_BLOCKS.4K_ALIAS",
+ "PEBS": "1",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x4"
+ },
+ {
+ "BriefDescription": "Counts the number of retired loads that are blocked because it initially appears to be store forward blocked, but subsequently is shown not to be blocked based on 4K alias check.",
+ "EventCode": "0x03",
+ "EventName": "LD_BLOCKS.ADDRESS_ALIAS",
+ "PEBS": "1",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x4"
+ },
+ {
+ "BriefDescription": "Counts the number of retired loads that are blocked because its address exactly matches an older store whose data is not ready.",
+ "EventCode": "0x03",
+ "EventName": "LD_BLOCKS.DATA_UNKNOWN",
+ "PEBS": "1",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears due to memory ordering in which an internal load passes an older store within the same CPU.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.DISAMBIGUATION",
+ "SampleAfterValue": "20003",
+ "UMask": "0x8"
+ },
+ {
+ "BriefDescription": "Counts the number of machines clears due to memory renaming.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.MRN_NUKE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x80"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears due to a page fault. Counts both I-Side and D-Side (Loads/Stores) page faults. A page fault occurs when either the page is not present, or an access violation occurs.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.PAGE_FAULT",
+ "SampleAfterValue": "20003",
+ "UMask": "0x20"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears that flush the pipeline and restart the machine with the use of microcode due to SMC, MEMORY_ORDERING, FP_ASSISTS, PAGE_FAULT, DISAMBIGUATION, and FPC_VIRTUAL_TRAP.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.SLOW",
+ "SampleAfterValue": "20003",
+ "UMask": "0x6f"
+ },
+ {
+ "BriefDescription": "Counts the number of machine clears due to program modifying data (self modifying code) within 1K of a recently fetched code page.",
+ "EventCode": "0xc3",
+ "EventName": "MACHINE_CLEARS.SMC",
+ "SampleAfterValue": "20003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots not consumed by the backend due to a micro-sequencer (MS) scoreboard, which stalls the front-end from issuing from the UROM until a specified older uop retires.",
+ "EventCode": "0x75",
+ "EventName": "SERIALIZATION.NON_C01_MS_SCB",
+ "PublicDescription": "Counts the number of issue slots not consumed by the backend due to a micro-sequencer (MS) scoreboard, which stalls the front-end from issuing from the UROM until a specified older uop retires. The most commonly executed instruction with an MS scoreboard is PAUSE.",
+ "SampleAfterValue": "200003",
+ "UMask": "0x2"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a mispredicted jump or a machine clear.",
+ "EventCode": "0x73",
+ "EventName": "TOPDOWN_BAD_SPECULATION.ALL",
+ "PublicDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a mispredicted jump or a machine clear. Only issue slots wasted due to fast nukes such as memory ordering nukes are counted. Other nukes are not accounted for. Counts all issue slots blocked during this recovery window including relevant microcode flows and while uops are not yet available in the instruction queue (IQ) even if an FE_bound event occurs during this period. Also includes the issue slots that were consumed by the backend but were thrown away because they were younger than the mispredict or machine clear.",
+ "SampleAfterValue": "1000003"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to fast nukes such as memory ordering and memory disambiguation machine clears.",
+ "EventCode": "0x73",
+ "EventName": "TOPDOWN_BAD_SPECULATION.FASTNUKE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x2"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots that were not consumed by the backend because allocation is stalled due to a machine clear (nuke) of any kind including memory ordering and memory disambiguation.",
+ "EventCode": "0x73",
+ "EventName": "TOPDOWN_BAD_SPECULATION.MACHINE_CLEARS",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x3"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to branch mispredicts.",
+ "EventCode": "0x73",
+ "EventName": "TOPDOWN_BAD_SPECULATION.MISPREDICT",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x4"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to a machine clear (nuke).",
+ "EventCode": "0x73",
+ "EventName": "TOPDOWN_BAD_SPECULATION.NUKE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots every cycle that were not consumed by the backend due to backend stalls.",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.ALL",
+ "SampleAfterValue": "1000003"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to certain allocation restrictions.",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.ALLOC_RESTRICTIONS",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to memory reservation stalls in which a scheduler is not able to accept uops.",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.MEM_SCHEDULER",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x2"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to IEC or FPC RAT stalls, which can be due to FIQ or IEC reservation stalls in which the integer, floating point or SIMD scheduler is not able to accept uops.",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.NON_MEM_SCHEDULER",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x8"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to the physical register file unable to accept an entry (marble stalls).",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.REGISTER",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x20"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to the reorder buffer being full (ROB stalls).",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.REORDER_BUFFER",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x40"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not consumed by the backend due to scoreboards from the instruction queue (IQ), jump execution unit (JEU), or microcode sequencer (MS).",
+ "EventCode": "0x74",
+ "EventName": "TOPDOWN_BE_BOUND.SERIALIZATION",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x10"
+ },
+ {
+ "BriefDescription": "Counts the total number of issue slots every cycle that were not consumed by the backend due to frontend stalls.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.ALL",
+ "SampleAfterValue": "1000003"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to BACLEARS.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.BRANCH_DETECT",
+ "PublicDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to BACLEARS, which occurs when the Branch Target Buffer (BTB) prediction or lack thereof, was corrected by a later branch predictor in the frontend. Includes BACLEARS due to all branch types including conditional and unconditional jumps, returns, and indirect branches.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x2"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to BTCLEARS.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.BRANCH_RESTEER",
+ "PublicDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to BTCLEARS, which occurs when the Branch Target Buffer (BTB) predicts a taken branch.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x40"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to the microcode sequencer (MS).",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.CISC",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to decode stalls.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.DECODE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x8"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to frontend bandwidth restrictions due to decode, predecode, cisc, and other limitations.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.FRONTEND_BANDWIDTH",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x8d"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to a latency related stalls including BACLEARs, BTCLEARs, ITLB misses, and ICache misses.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.FRONTEND_LATENCY",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x72"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to ITLB misses.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.ITLB",
+ "PublicDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to Instruction Table Lookaside Buffer (ITLB) misses.",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x10"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to other common frontend stalls not categorized.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.OTHER",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x80"
+ },
+ {
+ "BriefDescription": "Counts the number of issue slots every cycle that were not delivered by the frontend due to wrong predecodes.",
+ "EventCode": "0x71",
+ "EventName": "TOPDOWN_FE_BOUND.PREDECODE",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x4"
+ },
+ {
+ "BriefDescription": "Counts the total number of consumed retirement slots.",
+ "EventCode": "0xc2",
+ "EventName": "TOPDOWN_RETIRING.ALL",
+ "PEBS": "1",
+ "SampleAfterValue": "1000003"
+ },
+ {
+ "BriefDescription": "Counts the total number of uops retired.",
+ "EventCode": "0xc2",
+ "EventName": "UOPS_RETIRED.ALL",
+ "PEBS": "1",
+ "SampleAfterValue": "2000003"
+ },
+ {
+ "BriefDescription": "Counts the number of integer divide uops retired.",
+ "EventCode": "0xc2",
+ "EventName": "UOPS_RETIRED.IDIV",
+ "PEBS": "1",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x10"
+ },
+ {
+ "BriefDescription": "Counts the number of uops that are from complex flows issued by the micro-sequencer (MS).",
+ "EventCode": "0xc2",
+ "EventName": "UOPS_RETIRED.MS",
+ "PEBS": "1",
+ "PublicDescription": "Counts the number of uops that are from complex flows issued by the Microcode Sequencer (MS). This includes uops from flows due to complex instructions, faults, assists, and inserted flows.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of x87 uops retired, includes those in MS flows.",
+ "EventCode": "0xc2",
+ "EventName": "UOPS_RETIRED.X87",
+ "PEBS": "1",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x2"
+ }
+]
--- /dev/null
+[
+ {
+ "BriefDescription": "Counts every 64B read request entering the Memory Controller 0 to DRAM (sum of all channels).",
+ "EventName": "UNC_MC0_RDCAS_COUNT_FREERUN",
+ "PerPkg": "1",
+ "PublicDescription": "Counts every 64B read request entering the Memory Controller 0 to DRAM (sum of all channels).",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Counts every 64B write request entering the Memory Controller 0 to DRAM (sum of all channels). Each write request counts as a new request incrementing this counter. However, same cache line write requests (both full and partial) are combined to a single 64 byte data transfer to DRAM.",
+ "EventName": "UNC_MC0_WRCAS_COUNT_FREERUN",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Counts every 64B read request entering the Memory Controller 1 to DRAM (sum of all channels).",
+ "EventName": "UNC_MC1_RDCAS_COUNT_FREERUN",
+ "PerPkg": "1",
+ "PublicDescription": "Counts every 64B read entering the Memory Controller 1 to DRAM (sum of all channels).",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Counts every 64B write request entering the Memory Controller 1 to DRAM (sum of all channels). Each write request counts as a new request incrementing this counter. However, same cache line write requests (both full and partial) are combined to a single 64 byte data transfer to DRAM.",
+ "EventName": "UNC_MC1_WRCAS_COUNT_FREERUN",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "ACT command for a read request sent to DRAM",
+ "EventCode": "0x24",
+ "EventName": "UNC_M_ACT_COUNT_RD",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "ACT command sent to DRAM",
+ "EventCode": "0x26",
+ "EventName": "UNC_M_ACT_COUNT_TOTAL",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "ACT command for a write request sent to DRAM",
+ "EventCode": "0x25",
+ "EventName": "UNC_M_ACT_COUNT_WR",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Read CAS command sent to DRAM",
+ "EventCode": "0x22",
+ "EventName": "UNC_M_CAS_COUNT_RD",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Write CAS command sent to DRAM",
+ "EventCode": "0x23",
+ "EventName": "UNC_M_CAS_COUNT_WR",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Number of clocks",
+ "EventCode": "0x01",
+ "EventName": "UNC_M_CLOCKTICKS",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "incoming read request page status is Page Empty",
+ "EventCode": "0x1D",
+ "EventName": "UNC_M_DRAM_PAGE_EMPTY_RD",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "incoming write request page status is Page Empty",
+ "EventCode": "0x20",
+ "EventName": "UNC_M_DRAM_PAGE_EMPTY_WR",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "incoming read request page status is Page Hit",
+ "EventCode": "0x1C",
+ "EventName": "UNC_M_DRAM_PAGE_HIT_RD",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "incoming write request page status is Page Hit",
+ "EventCode": "0x1F",
+ "EventName": "UNC_M_DRAM_PAGE_HIT_WR",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "incoming read request page status is Page Miss",
+ "EventCode": "0x1E",
+ "EventName": "UNC_M_DRAM_PAGE_MISS_RD",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "incoming write request page status is Page Miss",
+ "EventCode": "0x21",
+ "EventName": "UNC_M_DRAM_PAGE_MISS_WR",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Any Rank at Hot state",
+ "EventCode": "0x19",
+ "EventName": "UNC_M_DRAM_THERMAL_HOT",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Any Rank at Warm state",
+ "EventCode": "0x1A",
+ "EventName": "UNC_M_DRAM_THERMAL_WARM",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Incoming read prefetch request from IA.",
+ "EventCode": "0x0A",
+ "EventName": "UNC_M_PREFETCH_RD",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "PRE command sent to DRAM due to page table idle timer expiration",
+ "EventCode": "0x28",
+ "EventName": "UNC_M_PRE_COUNT_IDLE",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "PRE command sent to DRAM for a read/write request",
+ "EventCode": "0x27",
+ "EventName": "UNC_M_PRE_COUNT_PAGE_MISS",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Incoming VC0 read request",
+ "EventCode": "0x02",
+ "EventName": "UNC_M_VC0_REQUESTS_RD",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Incoming VC0 write request",
+ "EventCode": "0x03",
+ "EventName": "UNC_M_VC0_REQUESTS_WR",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Incoming VC1 read request",
+ "EventCode": "0x04",
+ "EventName": "UNC_M_VC1_REQUESTS_RD",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ },
+ {
+ "BriefDescription": "Incoming VC1 write request",
+ "EventCode": "0x05",
+ "EventName": "UNC_M_VC1_REQUESTS_WR",
+ "PerPkg": "1",
+ "Unit": "iMC"
+ }
+]
--- /dev/null
+[
+ {
+ "BriefDescription": "Number of requests allocated in Coherency Tracker.",
+ "EventCode": "0x84",
+ "EventName": "UNC_ARB_COH_TRK_REQUESTS.ALL",
+ "PerPkg": "1",
+ "UMask": "0x1",
+ "Unit": "ARB"
+ },
+ {
+ "BriefDescription": "Each cycle counts number of all outgoing valid entries in ReqTrk. Such entry is defined as valid from its allocation in ReqTrk till deallocation. Accounts for Coherent and non-coherent traffic.",
+ "EventCode": "0x80",
+ "EventName": "UNC_ARB_TRK_OCCUPANCY.ALL",
+ "PerPkg": "1",
+ "UMask": "0x1",
+ "Unit": "ARB"
+ },
+ {
+ "BriefDescription": "Counts the number of coherent and in-coherent requests initiated by IA cores, processor graphic units, or LLC.",
+ "EventCode": "0x81",
+ "EventName": "UNC_ARB_TRK_REQUESTS.ALL",
+ "PerPkg": "1",
+ "UMask": "0x1",
+ "Unit": "ARB"
+ },
+ {
+ "BriefDescription": "This 48-bit fixed counter counts the UCLK cycles.",
+ "EventCode": "0xff",
+ "EventName": "UNC_CLOCK.SOCKET",
+ "PerPkg": "1",
+ "Unit": "CLOCK"
+ }
+]
--- /dev/null
+[
+ {
+ "BriefDescription": "Counts the number of page walks completed due to load DTLB misses to any page size.",
+ "EventCode": "0x08",
+ "EventName": "DTLB_LOAD_MISSES.WALK_COMPLETED",
+ "PublicDescription": "Counts the number of page walks completed due to loads (including SW prefetches) whose address translations missed in all Translation Lookaside Buffer (TLB) levels and were mapped to any page size. Includes page walks that page fault.",
+ "SampleAfterValue": "200003",
+ "UMask": "0xe"
+ },
+ {
+ "BriefDescription": "Counts the number of page walks completed due to store DTLB misses to any page size.",
+ "EventCode": "0x49",
+ "EventName": "DTLB_STORE_MISSES.WALK_COMPLETED",
+ "PublicDescription": "Counts the number of page walks completed due to stores whose address translations missed in all Translation Lookaside Buffer (TLB) levels and were mapped to any page size. Includes page walks that page fault.",
+ "SampleAfterValue": "2000003",
+ "UMask": "0xe"
+ },
+ {
+ "BriefDescription": "Counts the number of page walks initiated by a instruction fetch that missed the first and second level TLBs.",
+ "EventCode": "0x85",
+ "EventName": "ITLB_MISSES.MISS_CAUSED_WALK",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x1"
+ },
+ {
+ "BriefDescription": "Counts the number of page walks due to an instruction fetch that miss the PDE (Page Directory Entry) cache.",
+ "EventCode": "0x85",
+ "EventName": "ITLB_MISSES.PDE_CACHE_MISS",
+ "SampleAfterValue": "2000003",
+ "UMask": "0x80"
+ },
+ {
+ "BriefDescription": "Counts the number of page walks completed due to instruction fetch misses to any page size.",
+ "EventCode": "0x85",
+ "EventName": "ITLB_MISSES.WALK_COMPLETED",
+ "PublicDescription": "Counts the number of page walks completed due to instruction fetches whose address translations missed in all Translation Lookaside Buffer (TLB) levels and were mapped to any page size. Includes page walks that page fault.",
+ "SampleAfterValue": "200003",
+ "UMask": "0xe"
+ },
+ {
+ "BriefDescription": "Counts the number of cycles that the head (oldest load) of the load buffer and retirement are both stalled due to a DTLB miss.",
+ "EventCode": "0x05",
+ "EventName": "LD_HEAD.DTLB_MISS_AT_RET",
+ "SampleAfterValue": "1000003",
+ "UMask": "0x90"
+ }
+]
Family-model,Version,Filename,EventType
-GenuineIntel-6-(97|9A|B7|BA|BE|BF),v1.15,alderlake,core
+GenuineIntel-6-(97|9A|B7|BA|BF),v1.16,alderlake,core
+GenuineIntel-6-BE,v1.16,alderlaken,core
GenuineIntel-6-(1C|26|27|35|36),v4,bonnell,core
GenuineIntel-6-(3D|47),v26,broadwell,core
GenuineIntel-6-56,v23,broadwellde,core
import argparse
import csv
import json
+import metric
import os
import sys
from typing import (Callable, Dict, Optional, Sequence, Set, Tuple)
self.metric_name = jd.get('MetricName')
self.metric_group = jd.get('MetricGroup')
self.metric_constraint = jd.get('MetricConstraint')
- self.metric_expr = jd.get('MetricExpr')
- if self.metric_expr:
- self.metric_expr = self.metric_expr.replace('\\', '\\\\')
+ self.metric_expr = None
+ if 'MetricExpr' in jd:
+ self.metric_expr = metric.ParsePerfJson(jd['MetricExpr']).Simplify()
+
arch_std = jd.get('ArchStdEvent')
if precise and self.desc and '(Precise Event)' not in self.desc:
extra_desc += ' (Must be precise)' if precise == '2' else (' (Precise '
s = ''
for attr in _json_event_attributes:
x = getattr(self, attr)
+ if x and attr == 'metric_expr':
+ # Convert parsed metric expressions into a string. Slashes
+ # must be doubled in the file.
+ x = x.ToPerfJson().replace('\\', '\\\\')
s += f'{x}\\000' if x else '\\000'
return s
--- /dev/null
+# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+"""Parse or generate representations of perf metrics."""
+import ast
+import decimal
+import json
+import re
+from typing import Dict, List, Optional, Set, Union
+
+
+class Expression:
+ """Abstract base class of elements in a metric expression."""
+
+ def ToPerfJson(self) -> str:
+ """Returns a perf json file encoded representation."""
+ raise NotImplementedError()
+
+ def ToPython(self) -> str:
+ """Returns a python expr parseable representation."""
+ raise NotImplementedError()
+
+ def Simplify(self):
+ """Returns a simplified version of self."""
+ raise NotImplementedError()
+
+ def Equals(self, other) -> bool:
+ """Returns true when two expressions are the same."""
+ raise NotImplementedError()
+
+ def __str__(self) -> str:
+ return self.ToPerfJson()
+
+ def __or__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('|', self, other)
+
+ def __ror__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('|', other, self)
+
+ def __xor__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('^', self, other)
+
+ def __and__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('&', self, other)
+
+ def __lt__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('<', self, other)
+
+ def __gt__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('>', self, other)
+
+ def __add__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('+', self, other)
+
+ def __radd__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('+', other, self)
+
+ def __sub__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('-', self, other)
+
+ def __rsub__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('-', other, self)
+
+ def __mul__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('*', self, other)
+
+ def __rmul__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('*', other, self)
+
+ def __truediv__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('/', self, other)
+
+ def __rtruediv__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('/', other, self)
+
+ def __mod__(self, other: Union[int, float, 'Expression']) -> 'Operator':
+ return Operator('%', self, other)
+
+
+def _Constify(val: Union[bool, int, float, Expression]) -> Expression:
+ """Used to ensure that the nodes in the expression tree are all Expression."""
+ if isinstance(val, bool):
+ return Constant(1 if val else 0)
+ if isinstance(val, (int, float)):
+ return Constant(val)
+ return val
+
+
+# Simple lookup for operator precedence, used to avoid unnecessary
+# brackets. Precedence matches that of python and the simple expression parser.
+_PRECEDENCE = {
+ '|': 0,
+ '^': 1,
+ '&': 2,
+ '<': 3,
+ '>': 3,
+ '+': 4,
+ '-': 4,
+ '*': 5,
+ '/': 5,
+ '%': 5,
+}
+
+
+class Operator(Expression):
+ """Represents a binary operator in the parse tree."""
+
+ def __init__(self, operator: str, lhs: Union[int, float, Expression],
+ rhs: Union[int, float, Expression]):
+ self.operator = operator
+ self.lhs = _Constify(lhs)
+ self.rhs = _Constify(rhs)
+
+ def Bracket(self,
+ other: Expression,
+ other_str: str,
+ rhs: bool = False) -> str:
+ """If necessary brackets the given other value.
+
+ If ``other`` is an operator then a bracket is necessary when
+ this/self operator has higher precedence. Consider: '(a + b) * c',
+ ``other_str`` will be 'a + b'. A bracket is necessary as without
+ the bracket 'a + b * c' will evaluate 'b * c' first. However, '(a
+ * b) + c' doesn't need a bracket as 'a * b' will always be
+ evaluated first. For 'a / (b * c)' (ie the same precedence level
+ operations) then we add the bracket to best match the original
+ input, but not for '(a / b) * c' where the bracket is unnecessary.
+
+ Args:
+ other (Expression): is a lhs or rhs operator
+ other_str (str): ``other`` in the appropriate string form
+ rhs (bool): is ``other`` on the RHS
+
+ Returns:
+ str: possibly bracketed other_str
+ """
+ if isinstance(other, Operator):
+ if _PRECEDENCE.get(self.operator, -1) > _PRECEDENCE.get(
+ other.operator, -1):
+ return f'({other_str})'
+ if rhs and _PRECEDENCE.get(self.operator, -1) == _PRECEDENCE.get(
+ other.operator, -1):
+ return f'({other_str})'
+ return other_str
+
+ def ToPerfJson(self):
+ return (f'{self.Bracket(self.lhs, self.lhs.ToPerfJson())} {self.operator} '
+ f'{self.Bracket(self.rhs, self.rhs.ToPerfJson(), True)}')
+
+ def ToPython(self):
+ return (f'{self.Bracket(self.lhs, self.lhs.ToPython())} {self.operator} '
+ f'{self.Bracket(self.rhs, self.rhs.ToPython(), True)}')
+
+ def Simplify(self) -> Expression:
+ lhs = self.lhs.Simplify()
+ rhs = self.rhs.Simplify()
+ if isinstance(lhs, Constant) and isinstance(rhs, Constant):
+ return Constant(ast.literal_eval(lhs + self.operator + rhs))
+
+ if isinstance(self.lhs, Constant):
+ if self.operator in ('+', '|') and lhs.value == '0':
+ return rhs
+
+ # Simplify multiplication by 0 except for the slot event which
+ # is deliberately introduced using this pattern.
+ if self.operator == '*' and lhs.value == '0' and (
+ not isinstance(rhs, Event) or 'slots' not in rhs.name.lower()):
+ return Constant(0)
+
+ if self.operator == '*' and lhs.value == '1':
+ return rhs
+
+ if isinstance(rhs, Constant):
+ if self.operator in ('+', '|') and rhs.value == '0':
+ return lhs
+
+ if self.operator == '*' and rhs.value == '0':
+ return Constant(0)
+
+ if self.operator == '*' and self.rhs.value == '1':
+ return lhs
+
+ return Operator(self.operator, lhs, rhs)
+
+ def Equals(self, other: Expression) -> bool:
+ if isinstance(other, Operator):
+ return self.operator == other.operator and self.lhs.Equals(
+ other.lhs) and self.rhs.Equals(other.rhs)
+ return False
+
+
+class Select(Expression):
+ """Represents a select ternary in the parse tree."""
+
+ def __init__(self, true_val: Union[int, float, Expression],
+ cond: Union[int, float, Expression],
+ false_val: Union[int, float, Expression]):
+ self.true_val = _Constify(true_val)
+ self.cond = _Constify(cond)
+ self.false_val = _Constify(false_val)
+
+ def ToPerfJson(self):
+ true_str = self.true_val.ToPerfJson()
+ cond_str = self.cond.ToPerfJson()
+ false_str = self.false_val.ToPerfJson()
+ return f'({true_str} if {cond_str} else {false_str})'
+
+ def ToPython(self):
+ return (f'Select({self.true_val.ToPython()}, {self.cond.ToPython()}, '
+ f'{self.false_val.ToPython()})')
+
+ def Simplify(self) -> Expression:
+ cond = self.cond.Simplify()
+ true_val = self.true_val.Simplify()
+ false_val = self.false_val.Simplify()
+ if isinstance(cond, Constant):
+ return false_val if cond.value == '0' else true_val
+
+ if true_val.Equals(false_val):
+ return true_val
+
+ return Select(true_val, cond, false_val)
+
+ def Equals(self, other: Expression) -> bool:
+ if isinstance(other, Select):
+ return self.cond.Equals(other.cond) and self.false_val.Equals(
+ other.false_val) and self.true_val.Equals(other.true_val)
+ return False
+
+
+class Function(Expression):
+ """A function in an expression like min, max, d_ratio."""
+
+ def __init__(self,
+ fn: str,
+ lhs: Union[int, float, Expression],
+ rhs: Optional[Union[int, float, Expression]] = None):
+ self.fn = fn
+ self.lhs = _Constify(lhs)
+ self.rhs = _Constify(rhs)
+
+ def ToPerfJson(self):
+ if self.rhs:
+ return f'{self.fn}({self.lhs.ToPerfJson()}, {self.rhs.ToPerfJson()})'
+ return f'{self.fn}({self.lhs.ToPerfJson()})'
+
+ def ToPython(self):
+ if self.rhs:
+ return f'{self.fn}({self.lhs.ToPython()}, {self.rhs.ToPython()})'
+ return f'{self.fn}({self.lhs.ToPython()})'
+
+ def Simplify(self) -> Expression:
+ lhs = self.lhs.Simplify()
+ rhs = self.rhs.Simplify() if self.rhs else None
+ if isinstance(lhs, Constant) and isinstance(rhs, Constant):
+ if self.fn == 'd_ratio':
+ if rhs.value == '0':
+ return Constant(0)
+ Constant(ast.literal_eval(f'{lhs} / {rhs}'))
+ return Constant(ast.literal_eval(f'{self.fn}({lhs}, {rhs})'))
+
+ return Function(self.fn, lhs, rhs)
+
+ def Equals(self, other: Expression) -> bool:
+ if isinstance(other, Function):
+ return self.fn == other.fn and self.lhs.Equals(
+ other.lhs) and self.rhs.Equals(other.rhs)
+ return False
+
+
+def _FixEscapes(s: str) -> str:
+ s = re.sub(r'([^\\]),', r'\1\\,', s)
+ return re.sub(r'([^\\])=', r'\1\\=', s)
+
+
+class Event(Expression):
+ """An event in an expression."""
+
+ def __init__(self, name: str, legacy_name: str = ''):
+ self.name = _FixEscapes(name)
+ self.legacy_name = _FixEscapes(legacy_name)
+
+ def ToPerfJson(self):
+ result = re.sub('/', '@', self.name)
+ return result
+
+ def ToPython(self):
+ return f'Event(r"{self.name}")'
+
+ def Simplify(self) -> Expression:
+ return self
+
+ def Equals(self, other: Expression) -> bool:
+ return isinstance(other, Event) and self.name == other.name
+
+
+class Constant(Expression):
+ """A constant within the expression tree."""
+
+ def __init__(self, value: Union[float, str]):
+ ctx = decimal.Context()
+ ctx.prec = 20
+ dec = ctx.create_decimal(repr(value) if isinstance(value, float) else value)
+ self.value = dec.normalize().to_eng_string()
+ self.value = self.value.replace('+', '')
+ self.value = self.value.replace('E', 'e')
+
+ def ToPerfJson(self):
+ return self.value
+
+ def ToPython(self):
+ return f'Constant({self.value})'
+
+ def Simplify(self) -> Expression:
+ return self
+
+ def Equals(self, other: Expression) -> bool:
+ return isinstance(other, Constant) and self.value == other.value
+
+
+class Literal(Expression):
+ """A runtime literal within the expression tree."""
+
+ def __init__(self, value: str):
+ self.value = value
+
+ def ToPerfJson(self):
+ return self.value
+
+ def ToPython(self):
+ return f'Literal({self.value})'
+
+ def Simplify(self) -> Expression:
+ return self
+
+ def Equals(self, other: Expression) -> bool:
+ return isinstance(other, Literal) and self.value == other.value
+
+
+def min(lhs: Union[int, float, Expression], rhs: Union[int, float,
+ Expression]) -> Function:
+ # pylint: disable=redefined-builtin
+ # pylint: disable=invalid-name
+ return Function('min', lhs, rhs)
+
+
+def max(lhs: Union[int, float, Expression], rhs: Union[int, float,
+ Expression]) -> Function:
+ # pylint: disable=redefined-builtin
+ # pylint: disable=invalid-name
+ return Function('max', lhs, rhs)
+
+
+def d_ratio(lhs: Union[int, float, Expression],
+ rhs: Union[int, float, Expression]) -> Function:
+ # pylint: disable=redefined-builtin
+ # pylint: disable=invalid-name
+ return Function('d_ratio', lhs, rhs)
+
+
+def source_count(event: Event) -> Function:
+ # pylint: disable=redefined-builtin
+ # pylint: disable=invalid-name
+ return Function('source_count', event)
+
+
+class Metric:
+ """An individual metric that will specifiable on the perf command line."""
+ groups: Set[str]
+ expr: Expression
+ scale_unit: str
+ constraint: bool
+
+ def __init__(self,
+ name: str,
+ description: str,
+ expr: Expression,
+ scale_unit: str,
+ constraint: bool = False):
+ self.name = name
+ self.description = description
+ self.expr = expr.Simplify()
+ # Workraound valid_only_metric hiding certain metrics based on unit.
+ scale_unit = scale_unit.replace('/sec', ' per sec')
+ if scale_unit[0].isdigit():
+ self.scale_unit = scale_unit
+ else:
+ self.scale_unit = f'1{scale_unit}'
+ self.constraint = constraint
+ self.groups = set()
+
+ def __lt__(self, other):
+ """Sort order."""
+ return self.name < other.name
+
+ def AddToMetricGroup(self, group):
+ """Callback used when being added to a MetricGroup."""
+ self.groups.add(group.name)
+
+ def Flatten(self) -> Set['Metric']:
+ """Return a leaf metric."""
+ return set([self])
+
+ def ToPerfJson(self) -> Dict[str, str]:
+ """Return as dictionary for Json generation."""
+ result = {
+ 'MetricName': self.name,
+ 'MetricGroup': ';'.join(sorted(self.groups)),
+ 'BriefDescription': self.description,
+ 'MetricExpr': self.expr.ToPerfJson(),
+ 'ScaleUnit': self.scale_unit
+ }
+ if self.constraint:
+ result['MetricConstraint'] = 'NO_NMI_WATCHDOG'
+
+ return result
+
+
+class _MetricJsonEncoder(json.JSONEncoder):
+ """Special handling for Metric objects."""
+
+ def default(self, o):
+ if isinstance(o, Metric):
+ return o.ToPerfJson()
+ return json.JSONEncoder.default(self, o)
+
+
+class MetricGroup:
+ """A group of metrics.
+
+ Metric groups may be specificd on the perf command line, but within
+ the json they aren't encoded. Metrics may be in multiple groups
+ which can facilitate arrangements similar to trees.
+ """
+
+ def __init__(self, name: str, metric_list: List[Union[Metric,
+ 'MetricGroup']]):
+ self.name = name
+ self.metric_list = metric_list
+ for metric in metric_list:
+ metric.AddToMetricGroup(self)
+
+ def AddToMetricGroup(self, group):
+ """Callback used when a MetricGroup is added into another."""
+ for metric in self.metric_list:
+ metric.AddToMetricGroup(group)
+
+ def Flatten(self) -> Set[Metric]:
+ """Returns a set of all leaf metrics."""
+ result = set()
+ for x in self.metric_list:
+ result = result.union(x.Flatten())
+
+ return result
+
+ def ToPerfJson(self) -> str:
+ return json.dumps(sorted(self.Flatten()), indent=2, cls=_MetricJsonEncoder)
+
+ def __str__(self) -> str:
+ return self.ToPerfJson()
+
+
+class _RewriteIfExpToSelect(ast.NodeTransformer):
+
+ def visit_IfExp(self, node):
+ # pylint: disable=invalid-name
+ self.generic_visit(node)
+ call = ast.Call(
+ func=ast.Name(id='Select', ctx=ast.Load()),
+ args=[node.body, node.test, node.orelse],
+ keywords=[])
+ ast.copy_location(call, node.test)
+ return call
+
+
+def ParsePerfJson(orig: str) -> Expression:
+ """A simple json metric expression decoder.
+
+ Converts a json encoded metric expression by way of python's ast and
+ eval routine. First tokens are mapped to Event calls, then
+ accidentally converted keywords or literals are mapped to their
+ appropriate calls. Python's ast is used to match if-else that can't
+ be handled via operator overloading. Finally the ast is evaluated.
+
+ Args:
+ orig (str): String to parse.
+
+ Returns:
+ Expression: The parsed string.
+ """
+ # pylint: disable=eval-used
+ py = orig.strip()
+ py = re.sub(r'([a-zA-Z][^-+/\* \\\(\),]*(?:\\.[^-+/\* \\\(\),]*)*)',
+ r'Event(r"\1")', py)
+ py = re.sub(r'#Event\(r"([^"]*)"\)', r'Literal("#\1")', py)
+ py = re.sub(r'([0-9]+)Event\(r"(e[0-9]+)"\)', r'\1\2', py)
+ keywords = ['if', 'else', 'min', 'max', 'd_ratio', 'source_count']
+ for kw in keywords:
+ py = re.sub(rf'Event\(r"{kw}"\)', kw, py)
+
+ parsed = ast.parse(py, mode='eval')
+ _RewriteIfExpToSelect().visit(parsed)
+ parsed = ast.fix_missing_locations(parsed)
+ return _Constify(eval(compile(parsed, orig, 'eval')))
--- /dev/null
+# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+import unittest
+from metric import Constant
+from metric import Event
+from metric import ParsePerfJson
+
+
+class TestMetricExpressions(unittest.TestCase):
+
+ def test_Operators(self):
+ a = Event('a')
+ b = Event('b')
+ self.assertEqual((a | b).ToPerfJson(), 'a | b')
+ self.assertEqual((a ^ b).ToPerfJson(), 'a ^ b')
+ self.assertEqual((a & b).ToPerfJson(), 'a & b')
+ self.assertEqual((a < b).ToPerfJson(), 'a < b')
+ self.assertEqual((a > b).ToPerfJson(), 'a > b')
+ self.assertEqual((a + b).ToPerfJson(), 'a + b')
+ self.assertEqual((a - b).ToPerfJson(), 'a - b')
+ self.assertEqual((a * b).ToPerfJson(), 'a * b')
+ self.assertEqual((a / b).ToPerfJson(), 'a / b')
+ self.assertEqual((a % b).ToPerfJson(), 'a % b')
+ one = Constant(1)
+ self.assertEqual((a + one).ToPerfJson(), 'a + 1')
+
+ def test_Brackets(self):
+ a = Event('a')
+ b = Event('b')
+ c = Event('c')
+ self.assertEqual((a * b + c).ToPerfJson(), 'a * b + c')
+ self.assertEqual((a + b * c).ToPerfJson(), 'a + b * c')
+ self.assertEqual(((a + a) + a).ToPerfJson(), 'a + a + a')
+ self.assertEqual(((a + b) * c).ToPerfJson(), '(a + b) * c')
+ self.assertEqual((a + (b * c)).ToPerfJson(), 'a + b * c')
+ self.assertEqual(((a / b) * c).ToPerfJson(), 'a / b * c')
+ self.assertEqual((a / (b * c)).ToPerfJson(), 'a / (b * c)')
+
+ def test_ParsePerfJson(self):
+ # Based on an example of a real metric.
+ before = '(a + b + c + d) / (2 * e)'
+ after = before
+ self.assertEqual(ParsePerfJson(before).ToPerfJson(), after)
+
+ # Parsing should handle events with '-' in their name. Note, in
+ # the json file the '\' are doubled to '\\'.
+ before = r'topdown\-fe\-bound / topdown\-slots - 1'
+ after = before
+ self.assertEqual(ParsePerfJson(before).ToPerfJson(), after)
+
+ # Parsing should handle escaped modifiers. Note, in the json file
+ # the '\' are doubled to '\\'.
+ before = r'arb@event\=0x81\,umask\=0x1@ + arb@event\=0x84\,umask\=0x1@'
+ after = before
+ self.assertEqual(ParsePerfJson(before).ToPerfJson(), after)
+
+ # Parsing should handle exponents in numbers.
+ before = r'a + 1e12 + b'
+ after = before
+ self.assertEqual(ParsePerfJson(before).ToPerfJson(), after)
+
+ def test_IfElseTests(self):
+ # if-else needs rewriting to Select and back.
+ before = r'Event1 if #smt_on else Event2'
+ after = f'({before})'
+ self.assertEqual(ParsePerfJson(before).ToPerfJson(), after)
+
+ before = r'Event1 if 0 else Event2'
+ after = f'({before})'
+ self.assertEqual(ParsePerfJson(before).ToPerfJson(), after)
+
+ before = r'Event1 if 1 else Event2'
+ after = f'({before})'
+ self.assertEqual(ParsePerfJson(before).ToPerfJson(), after)
+
+ # Ensure the select is evaluate last.
+ before = r'Event1 + 1 if Event2 < 2 else Event3 + 3'
+ after = (r'Select(Event(r"Event1") + Constant(1), Event(r"Event2") < '
+ r'Constant(2), Event(r"Event3") + Constant(3))')
+ self.assertEqual(ParsePerfJson(before).ToPython(), after)
+
+ before = r'Event1 > 1 if Event2 < 2 else Event3 > 3'
+ after = (r'Select(Event(r"Event1") > Constant(1), Event(r"Event2") < '
+ r'Constant(2), Event(r"Event3") > Constant(3))')
+ self.assertEqual(ParsePerfJson(before).ToPython(), after)
+
+ before = r'min(a + b if c > 1 else c + d, e + f)'
+ after = r'min((a + b if c > 1 else c + d), e + f)'
+ self.assertEqual(ParsePerfJson(before).ToPerfJson(), after)
+
+ before =3D r'a if b else c if d else e'
+ after =3D r'(a if b else (c if d else e))'
+ self.assertEqual(ParsePerfJson(before).ToPerfJson(), after)
+
+ def test_ToPython(self):
+ # pylint: disable=eval-used
+ # Based on an example of a real metric.
+ before = '(a + b + c + d) / (2 * e)'
+ py = ParsePerfJson(before).ToPython()
+ after = eval(py).ToPerfJson()
+ self.assertEqual(before, after)
+
+ def test_Simplify(self):
+ before = '1 + 2 + 3'
+ after = '6'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = 'a + 0'
+ after = 'a'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = '0 + a'
+ after = 'a'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = 'a | 0'
+ after = 'a'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = '0 | a'
+ after = 'a'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = 'a * 0'
+ after = '0'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = '0 * a'
+ after = '0'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = 'a * 1'
+ after = 'a'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = '1 * a'
+ after = 'a'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = 'a if 0 else b'
+ after = 'b'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = 'a if 1 else b'
+ after = 'a'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ before = 'a if b else a'
+ after = 'a'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+ # Pattern used to add a slots event to metrics that require it.
+ before = '0 * SLOTS'
+ after = '0 * SLOTS'
+ self.assertEqual(ParsePerfJson(before).Simplify().ToPerfJson(), after)
+
+if __name__ == '__main__':
+ unittest.main()
-perf-y += Context.o
+perf-$(CONFIG_LIBTRACEEVENT) += Context.o
CFLAGS_Context.o += $(PYTHON_EMBED_CCOPTS) -Wno-redundant-decls -Wno-strict-prototypes -Wno-unused-parameter -Wno-nested-externs
--- /dev/null
+#!/bin/bash
+perf record -e sched:sched_switch -e sched:sched_migrate_task "$@"
--- /dev/null
+#!/bin/bash
+# description: analyze timings of tasks
+perf script -s "$PERF_EXEC_PATH"/scripts/python/task-analyzer.py -- "$@"
from __future__ import print_function
+import io
import os
import sys
import struct
import argparse
+import contextlib
from libxed import LibXED
from ctypes import create_string_buffer, addressof
glb_source_file_name = None
glb_line_number = None
glb_dso = None
+glb_stash_dict = {}
+glb_output = None
+glb_output_pos = 0
+glb_cpu = -1
+glb_time = 0
def get_optional_null(perf_dict, field):
if field in perf_dict:
ap.add_argument("--insn-trace", action='store_true')
ap.add_argument("--src-trace", action='store_true')
ap.add_argument("--all-switch-events", action='store_true')
+ ap.add_argument("--interleave", type=int, nargs='?', const=4, default=0)
global glb_args
global glb_insn
global glb_src
perf_set_itrace_options(perf_script_context, itrace)
def trace_end():
+ if glb_args.interleave:
+ flush_stashed_output()
print("End")
def trace_unhandled(event_name, context, event_fields_dict):
print(' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())]))
+def stash_output():
+ global glb_stash_dict
+ global glb_output_pos
+ output_str = glb_output.getvalue()[glb_output_pos:]
+ n = len(output_str)
+ if n:
+ glb_output_pos += n
+ if glb_cpu not in glb_stash_dict:
+ glb_stash_dict[glb_cpu] = []
+ glb_stash_dict[glb_cpu].append(output_str)
+
+def flush_stashed_output():
+ global glb_stash_dict
+ while glb_stash_dict:
+ cpus = list(glb_stash_dict.keys())
+ # Output at most glb_args.interleave output strings per cpu
+ for cpu in cpus:
+ items = glb_stash_dict[cpu]
+ countdown = glb_args.interleave
+ while len(items) and countdown:
+ sys.stdout.write(items[0])
+ del items[0]
+ countdown -= 1
+ if not items:
+ del glb_stash_dict[cpu]
+
def print_ptwrite(raw_buf):
data = struct.unpack_from("<IQ", raw_buf)
flags = data[0]
print_common_start(comm, sample, name)
print_common_ip(param_dict, sample, symbol, dso)
+def interleave_events(param_dict):
+ global glb_cpu
+ global glb_time
+ global glb_output
+ global glb_output_pos
+
+ sample = param_dict["sample"]
+ glb_cpu = sample["cpu"]
+ ts = sample["time"]
+
+ if glb_time != ts:
+ glb_time = ts
+ flush_stashed_output()
+
+ glb_output_pos = 0
+ with contextlib.redirect_stdout(io.StringIO()) as glb_output:
+ do_process_event(param_dict)
+
+ stash_output()
+
def process_event(param_dict):
try:
- do_process_event(param_dict)
+ if glb_args.interleave:
+ interleave_events(param_dict)
+ else:
+ do_process_event(param_dict)
except broken_pipe_exception:
# Stop python printing broken pipe errors and traceback
sys.stdout = open(os.devnull, 'w')
sys.exit(1)
def auxtrace_error(typ, code, cpu, pid, tid, ip, ts, msg, cpumode, *x):
+ if glb_args.interleave:
+ flush_stashed_output()
if len(x) >= 2 and x[0]:
machine_pid = x[0]
vcpu = x[1]
sys.exit(1)
def context_switch(ts, cpu, pid, tid, np_pid, np_tid, machine_pid, out, out_preempt, *x):
+ if glb_args.interleave:
+ flush_stashed_output()
if out:
out_str = "Switch out "
else:
--- /dev/null
+# task-analyzer.py - comprehensive perf tasks analysis
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022, Hagen Paul Pfeifer <hagen@jauu.net>
+# Licensed under the terms of the GNU GPL License version 2
+#
+# Usage:
+#
+# perf record -e sched:sched_switch -a -- sleep 10
+# perf script report task-analyzer
+#
+
+from __future__ import print_function
+import sys
+import os
+import string
+import argparse
+import decimal
+
+
+sys.path.append(
+ os.environ["PERF_EXEC_PATH"] + "/scripts/python/Perf-Trace-Util/lib/Perf/Trace"
+)
+from perf_trace_context import *
+from Core import *
+
+# Definition of possible ASCII color codes
+_COLORS = {
+ "grey": "\033[90m",
+ "red": "\033[91m",
+ "green": "\033[92m",
+ "yellow": "\033[93m",
+ "blue": "\033[94m",
+ "violet": "\033[95m",
+ "reset": "\033[0m",
+}
+
+# Columns will have a static size to align everything properly
+# Support of 116 days of active update with nano precision
+LEN_SWITCHED_IN = len("9999999.999999999") # 17
+LEN_SWITCHED_OUT = len("9999999.999999999") # 17
+LEN_CPU = len("000")
+LEN_PID = len("maxvalue") # 8
+LEN_TID = len("maxvalue") # 8
+LEN_COMM = len("max-comms-length") # 16
+LEN_RUNTIME = len("999999.999") # 10
+# Support of 3.45 hours of timespans
+LEN_OUT_IN = len("99999999999.999") # 15
+LEN_OUT_OUT = len("99999999999.999") # 15
+LEN_IN_IN = len("99999999999.999") # 15
+LEN_IN_OUT = len("99999999999.999") # 15
+
+
+# py2/py3 compatibility layer, see PEP469
+try:
+ dict.iteritems
+except AttributeError:
+ # py3
+ def itervalues(d):
+ return iter(d.values())
+
+ def iteritems(d):
+ return iter(d.items())
+
+else:
+ # py2
+ def itervalues(d):
+ return d.itervalues()
+
+ def iteritems(d):
+ return d.iteritems()
+
+
+def _check_color():
+ global _COLORS
+ """user enforced no-color or if stdout is no tty we disable colors"""
+ if sys.stdout.isatty() and args.stdio_color != "never":
+ return
+ _COLORS = {
+ "grey": "",
+ "red": "",
+ "green": "",
+ "yellow": "",
+ "blue": "",
+ "violet": "",
+ "reset": "",
+ }
+
+
+def _parse_args():
+ global args
+ parser = argparse.ArgumentParser(description="Analyze tasks behavior")
+ parser.add_argument(
+ "--time-limit",
+ default=[],
+ help=
+ "print tasks only in time[s] window e.g"
+ " --time-limit 123.111:789.222(print all between 123.111 and 789.222)"
+ " --time-limit 123: (print all from 123)"
+ " --time-limit :456 (print all until incl. 456)",
+ )
+ parser.add_argument(
+ "--summary", action="store_true", help="print addtional runtime information"
+ )
+ parser.add_argument(
+ "--summary-only", action="store_true", help="print only summary without traces"
+ )
+ parser.add_argument(
+ "--summary-extended",
+ action="store_true",
+ help="print the summary with additional information of max inter task times"
+ " relative to the prev task",
+ )
+ parser.add_argument(
+ "--ns", action="store_true", help="show timestamps in nanoseconds"
+ )
+ parser.add_argument(
+ "--ms", action="store_true", help="show timestamps in miliseconds"
+ )
+ parser.add_argument(
+ "--extended-times",
+ action="store_true",
+ help="Show the elapsed times between schedule in/schedule out"
+ " of this task and the schedule in/schedule out of previous occurrence"
+ " of the same task",
+ )
+ parser.add_argument(
+ "--filter-tasks",
+ default=[],
+ help="filter out unneeded tasks by tid, pid or processname."
+ " E.g --filter-task 1337,/sbin/init ",
+ )
+ parser.add_argument(
+ "--limit-to-tasks",
+ default=[],
+ help="limit output to selected task by tid, pid, processname."
+ " E.g --limit-to-tasks 1337,/sbin/init",
+ )
+ parser.add_argument(
+ "--highlight-tasks",
+ default="",
+ help="colorize special tasks by their pid/tid/comm."
+ " E.g. --highlight-tasks 1:red,mutt:yellow"
+ " Colors available: red,grey,yellow,blue,violet,green",
+ )
+ parser.add_argument(
+ "--rename-comms-by-tids",
+ default="",
+ help="rename task names by using tid (<tid>:<newname>,<tid>:<newname>)"
+ " This option is handy for inexpressive processnames like python interpreted"
+ " process. E.g --rename 1337:my-python-app",
+ )
+ parser.add_argument(
+ "--stdio-color",
+ default="auto",
+ choices=["always", "never", "auto"],
+ help="always, never or auto, allowing configuring color output"
+ " via the command line",
+ )
+ parser.add_argument(
+ "--csv",
+ default="",
+ help="Write trace to file selected by user. Options, like --ns or --extended"
+ "-times are used.",
+ )
+ parser.add_argument(
+ "--csv-summary",
+ default="",
+ help="Write summary to file selected by user. Options, like --ns or"
+ " --summary-extended are used.",
+ )
+ args = parser.parse_args()
+ args.tid_renames = dict()
+
+ _argument_filter_sanity_check()
+ _argument_prepare_check()
+
+
+def time_uniter(unit):
+ picker = {
+ "s": 1,
+ "ms": 1e3,
+ "us": 1e6,
+ "ns": 1e9,
+ }
+ return picker[unit]
+
+
+def _init_db():
+ global db
+ db = dict()
+ db["running"] = dict()
+ db["cpu"] = dict()
+ db["tid"] = dict()
+ db["global"] = []
+ if args.summary or args.summary_extended or args.summary_only:
+ db["task_info"] = dict()
+ db["runtime_info"] = dict()
+ # min values for summary depending on the header
+ db["task_info"]["pid"] = len("PID")
+ db["task_info"]["tid"] = len("TID")
+ db["task_info"]["comm"] = len("Comm")
+ db["runtime_info"]["runs"] = len("Runs")
+ db["runtime_info"]["acc"] = len("Accumulated")
+ db["runtime_info"]["max"] = len("Max")
+ db["runtime_info"]["max_at"] = len("Max At")
+ db["runtime_info"]["min"] = len("Min")
+ db["runtime_info"]["mean"] = len("Mean")
+ db["runtime_info"]["median"] = len("Median")
+ if args.summary_extended:
+ db["inter_times"] = dict()
+ db["inter_times"]["out_in"] = len("Out-In")
+ db["inter_times"]["inter_at"] = len("At")
+ db["inter_times"]["out_out"] = len("Out-Out")
+ db["inter_times"]["in_in"] = len("In-In")
+ db["inter_times"]["in_out"] = len("In-Out")
+
+
+def _median(numbers):
+ """phython3 hat statistics module - we have nothing"""
+ n = len(numbers)
+ index = n // 2
+ if n % 2:
+ return sorted(numbers)[index]
+ return sum(sorted(numbers)[index - 1 : index + 1]) / 2
+
+
+def _mean(numbers):
+ return sum(numbers) / len(numbers)
+
+
+class Timespans(object):
+ """
+ The elapsed time between two occurrences of the same task is being tracked with the
+ help of this class. There are 4 of those Timespans Out-Out, In-Out, Out-In and
+ In-In.
+ The first half of the name signals the first time point of the
+ first task. The second half of the name represents the second
+ timepoint of the second task.
+ """
+
+ def __init__(self):
+ self._last_start = None
+ self._last_finish = None
+ self.out_out = -1
+ self.in_out = -1
+ self.out_in = -1
+ self.in_in = -1
+ if args.summary_extended:
+ self._time_in = -1
+ self.max_out_in = -1
+ self.max_at = -1
+ self.max_in_out = -1
+ self.max_in_in = -1
+ self.max_out_out = -1
+
+ def feed(self, task):
+ """
+ Called for every recorded trace event to find process pair and calculate the
+ task timespans. Chronological ordering, feed does not do reordering
+ """
+ if not self._last_finish:
+ self._last_start = task.time_in(time_unit)
+ self._last_finish = task.time_out(time_unit)
+ return
+ self._time_in = task.time_in()
+ time_in = task.time_in(time_unit)
+ time_out = task.time_out(time_unit)
+ self.in_in = time_in - self._last_start
+ self.out_in = time_in - self._last_finish
+ self.in_out = time_out - self._last_start
+ self.out_out = time_out - self._last_finish
+ if args.summary_extended:
+ self._update_max_entries()
+ self._last_finish = task.time_out(time_unit)
+ self._last_start = task.time_in(time_unit)
+
+ def _update_max_entries(self):
+ if self.in_in > self.max_in_in:
+ self.max_in_in = self.in_in
+ if self.out_out > self.max_out_out:
+ self.max_out_out = self.out_out
+ if self.in_out > self.max_in_out:
+ self.max_in_out = self.in_out
+ if self.out_in > self.max_out_in:
+ self.max_out_in = self.out_in
+ self.max_at = self._time_in
+
+
+
+class Summary(object):
+ """
+ Primary instance for calculating the summary output. Processes the whole trace to
+ find and memorize relevant data such as mean, max et cetera. This instance handles
+ dynamic alignment aspects for summary output.
+ """
+
+ def __init__(self):
+ self._body = []
+
+ class AlignmentHelper:
+ """
+ Used to calculated the alignment for the output of the summary.
+ """
+ def __init__(self, pid, tid, comm, runs, acc, mean,
+ median, min, max, max_at):
+ self.pid = pid
+ self.tid = tid
+ self.comm = comm
+ self.runs = runs
+ self.acc = acc
+ self.mean = mean
+ self.median = median
+ self.min = min
+ self.max = max
+ self.max_at = max_at
+ if args.summary_extended:
+ self.out_in = None
+ self.inter_at = None
+ self.out_out = None
+ self.in_in = None
+ self.in_out = None
+
+ def _print_header(self):
+ '''
+ Output is trimmed in _format_stats thus additional adjustment in the header
+ is needed, depending on the choice of timeunit. The adjustment corresponds
+ to the amount of column titles being adjusted in _column_titles.
+ '''
+ decimal_precision = 6 if not args.ns else 9
+ fmt = " {{:^{}}}".format(sum(db["task_info"].values()))
+ fmt += " {{:^{}}}".format(
+ sum(db["runtime_info"].values()) - 2 * decimal_precision
+ )
+ _header = ("Task Information", "Runtime Information")
+
+ if args.summary_extended:
+ fmt += " {{:^{}}}".format(
+ sum(db["inter_times"].values()) - 4 * decimal_precision
+ )
+ _header += ("Max Inter Task Times",)
+ fd_sum.write(fmt.format(*_header) + "\n")
+
+ def _column_titles(self):
+ """
+ Cells are being processed and displayed in different way so an alignment adjust
+ is implemented depeding on the choice of the timeunit. The positions of the max
+ values are being displayed in grey. Thus in their format two additional {},
+ are placed for color set and reset.
+ """
+ separator, fix_csv_align = _prepare_fmt_sep()
+ decimal_precision, time_precision = _prepare_fmt_precision()
+ fmt = "{{:>{}}}".format(db["task_info"]["pid"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, db["task_info"]["tid"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, db["task_info"]["comm"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, db["runtime_info"]["runs"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, db["runtime_info"]["acc"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, db["runtime_info"]["mean"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(
+ separator, db["runtime_info"]["median"] * fix_csv_align
+ )
+ fmt += "{}{{:>{}}}".format(
+ separator, (db["runtime_info"]["min"] - decimal_precision) * fix_csv_align
+ )
+ fmt += "{}{{:>{}}}".format(
+ separator, (db["runtime_info"]["max"] - decimal_precision) * fix_csv_align
+ )
+ fmt += "{}{{}}{{:>{}}}{{}}".format(
+ separator, (db["runtime_info"]["max_at"] - time_precision) * fix_csv_align
+ )
+
+ column_titles = ("PID", "TID", "Comm")
+ column_titles += ("Runs", "Accumulated", "Mean", "Median", "Min", "Max")
+ column_titles += (_COLORS["grey"], "Max At", _COLORS["reset"])
+
+ if args.summary_extended:
+ fmt += "{}{{:>{}}}".format(
+ separator,
+ (db["inter_times"]["out_in"] - decimal_precision) * fix_csv_align
+ )
+ fmt += "{}{{}}{{:>{}}}{{}}".format(
+ separator,
+ (db["inter_times"]["inter_at"] - time_precision) * fix_csv_align
+ )
+ fmt += "{}{{:>{}}}".format(
+ separator,
+ (db["inter_times"]["out_out"] - decimal_precision) * fix_csv_align
+ )
+ fmt += "{}{{:>{}}}".format(
+ separator,
+ (db["inter_times"]["in_in"] - decimal_precision) * fix_csv_align
+ )
+ fmt += "{}{{:>{}}}".format(
+ separator,
+ (db["inter_times"]["in_out"] - decimal_precision) * fix_csv_align
+ )
+
+ column_titles += ("Out-In", _COLORS["grey"], "Max At", _COLORS["reset"],
+ "Out-Out", "In-In", "In-Out")
+
+ fd_sum.write(fmt.format(*column_titles) + "\n")
+
+
+ def _task_stats(self):
+ """calculates the stats of every task and constructs the printable summary"""
+ for tid in sorted(db["tid"]):
+ color_one_sample = _COLORS["grey"]
+ color_reset = _COLORS["reset"]
+ no_executed = 0
+ runtimes = []
+ time_in = []
+ timespans = Timespans()
+ for task in db["tid"][tid]:
+ pid = task.pid
+ comm = task.comm
+ no_executed += 1
+ runtimes.append(task.runtime(time_unit))
+ time_in.append(task.time_in())
+ timespans.feed(task)
+ if len(runtimes) > 1:
+ color_one_sample = ""
+ color_reset = ""
+ time_max = max(runtimes)
+ time_min = min(runtimes)
+ max_at = time_in[runtimes.index(max(runtimes))]
+
+ # The size of the decimal after sum,mean and median varies, thus we cut
+ # the decimal number, by rounding it. It has no impact on the output,
+ # because we have a precision of the decimal points at the output.
+ time_sum = round(sum(runtimes), 3)
+ time_mean = round(_mean(runtimes), 3)
+ time_median = round(_median(runtimes), 3)
+
+ align_helper = self.AlignmentHelper(pid, tid, comm, no_executed, time_sum,
+ time_mean, time_median, time_min, time_max, max_at)
+ self._body.append([pid, tid, comm, no_executed, time_sum, color_one_sample,
+ time_mean, time_median, time_min, time_max,
+ _COLORS["grey"], max_at, _COLORS["reset"], color_reset])
+ if args.summary_extended:
+ self._body[-1].extend([timespans.max_out_in,
+ _COLORS["grey"], timespans.max_at,
+ _COLORS["reset"], timespans.max_out_out,
+ timespans.max_in_in,
+ timespans.max_in_out])
+ align_helper.out_in = timespans.max_out_in
+ align_helper.inter_at = timespans.max_at
+ align_helper.out_out = timespans.max_out_out
+ align_helper.in_in = timespans.max_in_in
+ align_helper.in_out = timespans.max_in_out
+ self._calc_alignments_summary(align_helper)
+
+ def _format_stats(self):
+ separator, fix_csv_align = _prepare_fmt_sep()
+ decimal_precision, time_precision = _prepare_fmt_precision()
+ len_pid = db["task_info"]["pid"] * fix_csv_align
+ len_tid = db["task_info"]["tid"] * fix_csv_align
+ len_comm = db["task_info"]["comm"] * fix_csv_align
+ len_runs = db["runtime_info"]["runs"] * fix_csv_align
+ len_acc = db["runtime_info"]["acc"] * fix_csv_align
+ len_mean = db["runtime_info"]["mean"] * fix_csv_align
+ len_median = db["runtime_info"]["median"] * fix_csv_align
+ len_min = (db["runtime_info"]["min"] - decimal_precision) * fix_csv_align
+ len_max = (db["runtime_info"]["max"] - decimal_precision) * fix_csv_align
+ len_max_at = (db["runtime_info"]["max_at"] - time_precision) * fix_csv_align
+ if args.summary_extended:
+ len_out_in = (
+ db["inter_times"]["out_in"] - decimal_precision
+ ) * fix_csv_align
+ len_inter_at = (
+ db["inter_times"]["inter_at"] - time_precision
+ ) * fix_csv_align
+ len_out_out = (
+ db["inter_times"]["out_out"] - decimal_precision
+ ) * fix_csv_align
+ len_in_in = (db["inter_times"]["in_in"] - decimal_precision) * fix_csv_align
+ len_in_out = (
+ db["inter_times"]["in_out"] - decimal_precision
+ ) * fix_csv_align
+
+ fmt = "{{:{}d}}".format(len_pid)
+ fmt += "{}{{:{}d}}".format(separator, len_tid)
+ fmt += "{}{{:>{}}}".format(separator, len_comm)
+ fmt += "{}{{:{}d}}".format(separator, len_runs)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_acc, time_precision)
+ fmt += "{}{{}}{{:{}.{}f}}".format(separator, len_mean, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_median, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_min, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_max, time_precision)
+ fmt += "{}{{}}{{:{}.{}f}}{{}}{{}}".format(
+ separator, len_max_at, decimal_precision
+ )
+ if args.summary_extended:
+ fmt += "{}{{:{}.{}f}}".format(separator, len_out_in, time_precision)
+ fmt += "{}{{}}{{:{}.{}f}}{{}}".format(
+ separator, len_inter_at, decimal_precision
+ )
+ fmt += "{}{{:{}.{}f}}".format(separator, len_out_out, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_in_in, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_in_out, time_precision)
+ return fmt
+
+
+ def _calc_alignments_summary(self, align_helper):
+ # Length is being cut in 3 groups so that further addition is easier to handle.
+ # The length of every argument from the alignment helper is being checked if it
+ # is longer than the longest until now. In that case the length is being saved.
+ for key in db["task_info"]:
+ if len(str(getattr(align_helper, key))) > db["task_info"][key]:
+ db["task_info"][key] = len(str(getattr(align_helper, key)))
+ for key in db["runtime_info"]:
+ if len(str(getattr(align_helper, key))) > db["runtime_info"][key]:
+ db["runtime_info"][key] = len(str(getattr(align_helper, key)))
+ if args.summary_extended:
+ for key in db["inter_times"]:
+ if len(str(getattr(align_helper, key))) > db["inter_times"][key]:
+ db["inter_times"][key] = len(str(getattr(align_helper, key)))
+
+
+ def print(self):
+ self._task_stats()
+ fmt = self._format_stats()
+
+ if not args.csv_summary:
+ print("\nSummary")
+ self._print_header()
+ self._column_titles()
+ for i in range(len(self._body)):
+ fd_sum.write(fmt.format(*tuple(self._body[i])) + "\n")
+
+
+
+class Task(object):
+ """ The class is used to handle the information of a given task."""
+
+ def __init__(self, id, tid, cpu, comm):
+ self.id = id
+ self.tid = tid
+ self.cpu = cpu
+ self.comm = comm
+ self.pid = None
+ self._time_in = None
+ self._time_out = None
+
+ def schedule_in_at(self, time):
+ """set the time where the task was scheduled in"""
+ self._time_in = time
+
+ def schedule_out_at(self, time):
+ """set the time where the task was scheduled out"""
+ self._time_out = time
+
+ def time_out(self, unit="s"):
+ """return time where a given task was scheduled out"""
+ factor = time_uniter(unit)
+ return self._time_out * decimal.Decimal(factor)
+
+ def time_in(self, unit="s"):
+ """return time where a given task was scheduled in"""
+ factor = time_uniter(unit)
+ return self._time_in * decimal.Decimal(factor)
+
+ def runtime(self, unit="us"):
+ factor = time_uniter(unit)
+ return (self._time_out - self._time_in) * decimal.Decimal(factor)
+
+ def update_pid(self, pid):
+ self.pid = pid
+
+
+def _task_id(pid, cpu):
+ """returns a "unique-enough" identifier, please do not change"""
+ return "{}-{}".format(pid, cpu)
+
+
+def _filter_non_printable(unfiltered):
+ """comm names may contain loony chars like '\x00000'"""
+ filtered = ""
+ for char in unfiltered:
+ if char not in string.printable:
+ continue
+ filtered += char
+ return filtered
+
+
+def _fmt_header():
+ separator, fix_csv_align = _prepare_fmt_sep()
+ fmt = "{{:>{}}}".format(LEN_SWITCHED_IN*fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, LEN_SWITCHED_OUT*fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, LEN_CPU*fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, LEN_PID*fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, LEN_TID*fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, LEN_COMM*fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, LEN_RUNTIME*fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, LEN_OUT_IN*fix_csv_align)
+ if args.extended_times:
+ fmt += "{}{{:>{}}}".format(separator, LEN_OUT_OUT*fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, LEN_IN_IN*fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, LEN_IN_OUT*fix_csv_align)
+ return fmt
+
+
+def _fmt_body():
+ separator, fix_csv_align = _prepare_fmt_sep()
+ decimal_precision, time_precision = _prepare_fmt_precision()
+ fmt = "{{}}{{:{}.{}f}}".format(LEN_SWITCHED_IN*fix_csv_align, decimal_precision)
+ fmt += "{}{{:{}.{}f}}".format(
+ separator, LEN_SWITCHED_OUT*fix_csv_align, decimal_precision
+ )
+ fmt += "{}{{:{}d}}".format(separator, LEN_CPU*fix_csv_align)
+ fmt += "{}{{:{}d}}".format(separator, LEN_PID*fix_csv_align)
+ fmt += "{}{{}}{{:{}d}}{{}}".format(separator, LEN_TID*fix_csv_align)
+ fmt += "{}{{}}{{:>{}}}".format(separator, LEN_COMM*fix_csv_align)
+ fmt += "{}{{:{}.{}f}}".format(separator, LEN_RUNTIME*fix_csv_align, time_precision)
+ if args.extended_times:
+ fmt += "{}{{:{}.{}f}}".format(separator, LEN_OUT_IN*fix_csv_align, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, LEN_OUT_OUT*fix_csv_align, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, LEN_IN_IN*fix_csv_align, time_precision)
+ fmt += "{}{{:{}.{}f}}{{}}".format(
+ separator, LEN_IN_OUT*fix_csv_align, time_precision
+ )
+ else:
+ fmt += "{}{{:{}.{}f}}{{}}".format(
+ separator, LEN_OUT_IN*fix_csv_align, time_precision
+ )
+ return fmt
+
+
+def _print_header():
+ fmt = _fmt_header()
+ header = ("Switched-In", "Switched-Out", "CPU", "PID", "TID", "Comm", "Runtime",
+ "Time Out-In")
+ if args.extended_times:
+ header += ("Time Out-Out", "Time In-In", "Time In-Out")
+ fd_task.write(fmt.format(*header) + "\n")
+
+
+
+def _print_task_finish(task):
+ """calculating every entry of a row and printing it immediately"""
+ c_row_set = ""
+ c_row_reset = ""
+ out_in = -1
+ out_out = -1
+ in_in = -1
+ in_out = -1
+ fmt = _fmt_body()
+ # depending on user provided highlight option we change the color
+ # for particular tasks
+ if str(task.tid) in args.highlight_tasks_map:
+ c_row_set = _COLORS[args.highlight_tasks_map[str(task.tid)]]
+ c_row_reset = _COLORS["reset"]
+ if task.comm in args.highlight_tasks_map:
+ c_row_set = _COLORS[args.highlight_tasks_map[task.comm]]
+ c_row_reset = _COLORS["reset"]
+ # grey-out entries if PID == TID, they
+ # are identical, no threaded model so the
+ # thread id (tid) do not matter
+ c_tid_set = ""
+ c_tid_reset = ""
+ if task.pid == task.tid:
+ c_tid_set = _COLORS["grey"]
+ c_tid_reset = _COLORS["reset"]
+ if task.tid in db["tid"]:
+ # get last task of tid
+ last_tid_task = db["tid"][task.tid][-1]
+ # feed the timespan calculate, last in tid db
+ # and second the current one
+ timespan_gap_tid = Timespans()
+ timespan_gap_tid.feed(last_tid_task)
+ timespan_gap_tid.feed(task)
+ out_in = timespan_gap_tid.out_in
+ out_out = timespan_gap_tid.out_out
+ in_in = timespan_gap_tid.in_in
+ in_out = timespan_gap_tid.in_out
+
+
+ if args.extended_times:
+ line_out = fmt.format(c_row_set, task.time_in(), task.time_out(), task.cpu,
+ task.pid, c_tid_set, task.tid, c_tid_reset, c_row_set, task.comm,
+ task.runtime(time_unit), out_in, out_out, in_in, in_out,
+ c_row_reset) + "\n"
+ else:
+ line_out = fmt.format(c_row_set, task.time_in(), task.time_out(), task.cpu,
+ task.pid, c_tid_set, task.tid, c_tid_reset, c_row_set, task.comm,
+ task.runtime(time_unit), out_in, c_row_reset) + "\n"
+ try:
+ fd_task.write(line_out)
+ except(IOError):
+ # don't mangle the output if user SIGINT this script
+ sys.exit()
+
+def _record_cleanup(_list):
+ """
+ no need to store more then one element if --summarize
+ is not enabled
+ """
+ if not args.summary and len(_list) > 1:
+ _list = _list[len(_list) - 1 :]
+
+
+def _record_by_tid(task):
+ tid = task.tid
+ if tid not in db["tid"]:
+ db["tid"][tid] = []
+ db["tid"][tid].append(task)
+ _record_cleanup(db["tid"][tid])
+
+
+def _record_by_cpu(task):
+ cpu = task.cpu
+ if cpu not in db["cpu"]:
+ db["cpu"][cpu] = []
+ db["cpu"][cpu].append(task)
+ _record_cleanup(db["cpu"][cpu])
+
+
+def _record_global(task):
+ """record all executed task, ordered by finish chronological"""
+ db["global"].append(task)
+ _record_cleanup(db["global"])
+
+
+def _handle_task_finish(tid, cpu, time, perf_sample_dict):
+ if tid == 0:
+ return
+ _id = _task_id(tid, cpu)
+ if _id not in db["running"]:
+ # may happen, if we missed the switch to
+ # event. Seen in combination with --exclude-perf
+ # where the start is filtered out, but not the
+ # switched in. Probably a bug in exclude-perf
+ # option.
+ return
+ task = db["running"][_id]
+ task.schedule_out_at(time)
+
+ # record tid, during schedule in the tid
+ # is not available, update now
+ pid = int(perf_sample_dict["sample"]["pid"])
+
+ task.update_pid(pid)
+ del db["running"][_id]
+
+ # print only tasks which are not being filtered and no print of trace
+ # for summary only, but record every task.
+ if not _limit_filtered(tid, pid, task.comm) and not args.summary_only:
+ _print_task_finish(task)
+ _record_by_tid(task)
+ _record_by_cpu(task)
+ _record_global(task)
+
+
+def _handle_task_start(tid, cpu, comm, time):
+ if tid == 0:
+ return
+ if tid in args.tid_renames:
+ comm = args.tid_renames[tid]
+ _id = _task_id(tid, cpu)
+ if _id in db["running"]:
+ # handle corner cases where already running tasks
+ # are switched-to again - saw this via --exclude-perf
+ # recorded traces. We simple ignore this "second start"
+ # event.
+ return
+ assert _id not in db["running"]
+ task = Task(_id, tid, cpu, comm)
+ task.schedule_in_at(time)
+ db["running"][_id] = task
+
+
+def _time_to_internal(time_ns):
+ """
+ To prevent float rounding errors we use Decimal internally
+ """
+ return decimal.Decimal(time_ns) / decimal.Decimal(1e9)
+
+
+def _limit_filtered(tid, pid, comm):
+ if args.filter_tasks:
+ if str(tid) in args.filter_tasks or comm in args.filter_tasks:
+ return True
+ else:
+ return False
+ if args.limit_to_tasks:
+ if str(tid) in args.limit_to_tasks or comm in args.limit_to_tasks:
+ return False
+ else:
+ return True
+
+
+def _argument_filter_sanity_check():
+ if args.limit_to_tasks and args.filter_tasks:
+ sys.exit("Error: Filter and Limit at the same time active.")
+ if args.extended_times and args.summary_only:
+ sys.exit("Error: Summary only and extended times active.")
+ if args.time_limit and ":" not in args.time_limit:
+ sys.exit(
+ "Error: No bound set for time limit. Please set bound by ':' e.g :123."
+ )
+ if args.time_limit and (args.summary or args.summary_only or args.summary_extended):
+ sys.exit("Error: Cannot set time limit and print summary")
+ if args.csv_summary:
+ args.summary = True
+ if args.csv == args.csv_summary:
+ sys.exit("Error: Chosen files for csv and csv summary are the same")
+ if args.csv and (args.summary_extended or args.summary) and not args.csv_summary:
+ sys.exit("Error: No file chosen to write summary to. Choose with --csv-summary "
+ "<file>")
+ if args.csv and args.summary_only:
+ sys.exit("Error: --csv chosen and --summary-only. Standard task would not be"
+ "written to csv file.")
+
+def _argument_prepare_check():
+ global time_unit, fd_task, fd_sum
+ if args.filter_tasks:
+ args.filter_tasks = args.filter_tasks.split(",")
+ if args.limit_to_tasks:
+ args.limit_to_tasks = args.limit_to_tasks.split(",")
+ if args.time_limit:
+ args.time_limit = args.time_limit.split(":")
+ for rename_tuple in args.rename_comms_by_tids.split(","):
+ tid_name = rename_tuple.split(":")
+ if len(tid_name) != 2:
+ continue
+ args.tid_renames[int(tid_name[0])] = tid_name[1]
+ args.highlight_tasks_map = dict()
+ for highlight_tasks_tuple in args.highlight_tasks.split(","):
+ tasks_color_map = highlight_tasks_tuple.split(":")
+ # default highlight color to red if no color set by user
+ if len(tasks_color_map) == 1:
+ tasks_color_map.append("red")
+ if args.highlight_tasks and tasks_color_map[1].lower() not in _COLORS:
+ sys.exit(
+ "Error: Color not defined, please choose from grey,red,green,yellow,blue,"
+ "violet"
+ )
+ if len(tasks_color_map) != 2:
+ continue
+ args.highlight_tasks_map[tasks_color_map[0]] = tasks_color_map[1]
+ time_unit = "us"
+ if args.ns:
+ time_unit = "ns"
+ elif args.ms:
+ time_unit = "ms"
+
+
+ fd_task = sys.stdout
+ if args.csv:
+ args.stdio_color = "never"
+ fd_task = open(args.csv, "w")
+ print("generating csv at",args.csv,)
+
+ fd_sum = sys.stdout
+ if args.csv_summary:
+ args.stdio_color = "never"
+ fd_sum = open(args.csv_summary, "w")
+ print("generating csv summary at",args.csv_summary)
+ if not args.csv:
+ args.summary_only = True
+
+
+def _is_within_timelimit(time):
+ """
+ Check if a time limit was given by parameter, if so ignore the rest. If not,
+ process the recorded trace in its entirety.
+ """
+ if not args.time_limit:
+ return True
+ lower_time_limit = args.time_limit[0]
+ upper_time_limit = args.time_limit[1]
+ # check for upper limit
+ if upper_time_limit == "":
+ if time >= decimal.Decimal(lower_time_limit):
+ return True
+ # check for lower limit
+ if lower_time_limit == "":
+ if time <= decimal.Decimal(upper_time_limit):
+ return True
+ # quit if time exceeds upper limit. Good for big datasets
+ else:
+ quit()
+ if lower_time_limit != "" and upper_time_limit != "":
+ if (time >= decimal.Decimal(lower_time_limit) and
+ time <= decimal.Decimal(upper_time_limit)):
+ return True
+ # quit if time exceeds upper limit. Good for big datasets
+ elif time > decimal.Decimal(upper_time_limit):
+ quit()
+
+def _prepare_fmt_precision():
+ decimal_precision = 6
+ time_precision = 3
+ if args.ns:
+ decimal_precision = 9
+ time_precision = 0
+ return decimal_precision, time_precision
+
+def _prepare_fmt_sep():
+ separator = " "
+ fix_csv_align = 1
+ if args.csv or args.csv_summary:
+ separator = ";"
+ fix_csv_align = 0
+ return separator, fix_csv_align
+
+def trace_unhandled(event_name, context, event_fields_dict, perf_sample_dict):
+ pass
+
+
+def trace_begin():
+ _parse_args()
+ _check_color()
+ _init_db()
+ if not args.summary_only:
+ _print_header()
+
+def trace_end():
+ if args.summary or args.summary_extended or args.summary_only:
+ Summary().print()
+
+def sched__sched_switch(event_name, context, common_cpu, common_secs, common_nsecs,
+ common_pid, common_comm, common_callchain, prev_comm,
+ prev_pid, prev_prio, prev_state, next_comm, next_pid,
+ next_prio, perf_sample_dict):
+ # ignore common_secs & common_nsecs cause we need
+ # high res timestamp anyway, using the raw value is
+ # faster
+ time = _time_to_internal(perf_sample_dict["sample"]["time"])
+ if not _is_within_timelimit(time):
+ # user specific --time-limit a:b set
+ return
+
+ next_comm = _filter_non_printable(next_comm)
+ _handle_task_finish(prev_pid, common_cpu, time, perf_sample_dict)
+ _handle_task_start(next_pid, common_cpu, next_comm, time)
perf-y += dso-data.o
perf-y += attr.o
perf-y += vmlinux-kallsyms.o
-perf-y += openat-syscall.o
-perf-y += openat-syscall-all-cpus.o
-perf-y += openat-syscall-tp-fields.o
-perf-y += mmap-basic.o
+perf-$(CONFIG_LIBTRACEEVENT) += openat-syscall.o
+perf-$(CONFIG_LIBTRACEEVENT) += openat-syscall-all-cpus.o
+perf-$(CONFIG_LIBTRACEEVENT) += openat-syscall-tp-fields.o
+perf-$(CONFIG_LIBTRACEEVENT) += mmap-basic.o
perf-y += perf-record.o
perf-y += evsel-roundtrip-name.o
-perf-y += evsel-tp-sched.o
+perf-$(CONFIG_LIBTRACEEVENT) += evsel-tp-sched.o
perf-y += fdarray.o
perf-y += pmu.o
perf-y += pmu-events.o
perf-y += sw-clock.o
perf-y += mmap-thread-lookup.o
perf-y += thread-maps-share.o
-perf-y += switch-tracking.o
+perf-$(CONFIG_LIBTRACEEVENT) += switch-tracking.o
perf-y += keep-tracking.o
perf-y += code-reading.o
perf-y += sample-parsing.o
perf-y += perf-time-to-tsc.o
perf-y += dlfilter-test.o
perf-y += sigtrap.o
+perf-y += event_groups.o
$(OUTPUT)tests/llvm-src-base.c: tests/bpf-script-example.c tests/Build
$(call rule_mkdir)
CFLAGS_attr.o += -DBINDIR="BUILD_STR($(bindir_SQ))" -DPYTHON="BUILD_STR($(PYTHON_WORD))"
CFLAGS_python-use.o += -DPYTHONPATH="BUILD_STR($(OUTPUT)python)" -DPYTHON="BUILD_STR($(PYTHON_WORD))"
CFLAGS_dwarf-unwind.o += -fno-optimize-sibling-calls
+
+perf-y += workloads/
import sys
import glob
import optparse
+import platform
import tempfile
import logging
+import re
import shutil
+import subprocess
try:
import configparser
if not data_equal(self[t], other[t]):
log.warning("expected %s=%s, got %s" % (t, self[t], other[t]))
+def parse_version(version):
+ if not version:
+ return None
+ return [int(v) for v in version.split(".")[0:2]]
+
# Test file description needs to have following sections:
# [config]
# - just single instance in file
# - needs to specify:
# 'command' - perf command name
# 'args' - special command arguments
-# 'ret' - expected command return value (0 by default)
+# 'ret' - Skip test if Perf doesn't exit with this value (0 by default)
+# 'test_ret'- If set to 'true', fail test instead of skipping for 'ret' argument
# 'arch' - architecture specific test (optional)
# comma separated list, ! at the beginning
# negates it.
-#
+# 'auxv' - Truthy statement that is evaled in the scope of the auxv map. When false,
+# the test is skipped. For example 'auxv["AT_HWCAP"] == 10'. (optional)
+# 'kernel_since' - Inclusive kernel version from which the test will start running. Only the
+# first two values are supported, for example "6.1" (optional)
+# 'kernel_until' - Exclusive kernel version from which the test will stop running. (optional)
# [eventX:base]
# - one or multiple instances in file
# - expected values assignments
except:
self.ret = 0
+ self.test_ret = parser.getboolean('config', 'test_ret', fallback=False)
+
try:
self.arch = parser.get('config', 'arch')
log.warning("test limitation '%s'" % self.arch)
except:
self.arch = ''
+ self.auxv = parser.get('config', 'auxv', fallback=None)
+ self.kernel_since = parse_version(parser.get('config', 'kernel_since', fallback=None))
+ self.kernel_until = parse_version(parser.get('config', 'kernel_until', fallback=None))
self.expect = {}
self.result = {}
log.debug(" loading expected events");
else:
return True
- def skip_test(self, myarch):
+ def skip_test_kernel_since(self):
+ if not self.kernel_since:
+ return False
+ return not self.kernel_since <= parse_version(platform.release())
+
+ def skip_test_kernel_until(self):
+ if not self.kernel_until:
+ return False
+ return not parse_version(platform.release()) < self.kernel_until
+
+ def skip_test_auxv(self):
+ def new_auxv(a, pattern):
+ items = list(filter(None, pattern.split(a)))
+ # AT_HWCAP is hex but doesn't have a prefix, so special case it
+ if items[0] == "AT_HWCAP":
+ value = int(items[-1], 16)
+ else:
+ try:
+ value = int(items[-1], 0)
+ except:
+ value = items[-1]
+ return (items[0], value)
+
+ if not self.auxv:
+ return False
+ auxv = subprocess.check_output("LD_SHOW_AUXV=1 sleep 0", shell=True) \
+ .decode(sys.stdout.encoding)
+ pattern = re.compile(r"[: ]+")
+ auxv = dict([new_auxv(a, pattern) for a in auxv.splitlines()])
+ return not eval(self.auxv)
+
+ def skip_test_arch(self, myarch):
# If architecture not set always run test
if self.arch == '':
# log.warning("test for arch %s is ok" % myarch)
def run_cmd(self, tempdir):
junk1, junk2, junk3, junk4, myarch = (os.uname())
- if self.skip_test(myarch):
+ if self.skip_test_arch(myarch):
raise Notest(self, myarch)
+ if self.skip_test_auxv():
+ raise Notest(self, "auxv skip")
+
+ if self.skip_test_kernel_since():
+ raise Notest(self, "old kernel skip")
+
+ if self.skip_test_kernel_until():
+ raise Notest(self, "new kernel skip")
+
cmd = "PERF_TEST_ATTR=%s %s %s -o %s/perf.data %s" % (tempdir,
self.perf, self.command, tempdir, self.args)
ret = os.WEXITSTATUS(os.system(cmd))
log.info(" '%s' ret '%s', expected '%s'" % (cmd, str(ret), str(self.ret)))
if not data_equal(str(ret), str(self.ret)):
- raise Unsup(self)
+ if self.test_ret:
+ raise Fail(self, "Perf exit code failure")
+ else:
+ raise Unsup(self)
def compare(self, expect, result):
match = {}
perf record --call-graph dwarf kill (test-record-graph-dwarf)
perf record --call-graph fp kill (test-record-graph-fp)
perf record --call-graph fp kill (test-record-graph-fp-aarch64)
- perf record --group -e cycles,instructions kill (test-record-group)
perf record -e '{cycles,instructions}' kill (test-record-group1)
perf record -e '{cycles/period=1/,instructions/period=2/}:S' kill (test-record-group2)
perf record -D kill (test-record-no-delay)
perf stat -d kill (test-stat-detailed-1)
perf stat -dd kill (test-stat-detailed-2)
perf stat -ddd kill (test-stat-detailed-3)
- perf stat --group -e cycles,instructions kill (test-stat-group)
perf stat -e '{cycles,instructions}' kill (test-stat-group1)
perf stat -i -e cycles kill (test-stat-no-inherit)
+++ /dev/null
-[config]
-command = record
-args = --no-bpf-event --group -e cycles,instructions kill >/dev/null 2>&1
-ret = 1
-
-[event-1:base-record]
-fd=1
-group_fd=-1
-sample_type=327
-read_format=4|20
-
-[event-2:base-record]
-fd=2
-group_fd=1
-config=1
-sample_type=327
-read_format=4|20
-mmap=0
-comm=0
-task=0
-enable_on_exec=0
-disabled=0
--- /dev/null
+# Test that asking for VG fails if the system doesn't support SVE. This
+# applies both before and after the feature was added in 6.1
+[config]
+command = record
+args = --no-bpf-event --user-regs=vg kill >/dev/null 2>&1
+ret = 129
+test_ret = true
+arch = aarch64
+auxv = auxv["AT_HWCAP"] & 0x200000 == 0
--- /dev/null
+# Test that asking for VG always fails on old kernels because it was
+# added in 6.1. This applies to systems that either support or don't
+# support SVE.
+[config]
+command = record
+args = --no-bpf-event --user-regs=vg kill >/dev/null 2>&1
+ret = 129
+test_ret = true
+arch = aarch64
+kernel_until = 6.1
--- /dev/null
+# Test that asking for VG works if the system has SVE and after the
+# feature was added in 6.1
+[config]
+command = record
+args = --no-bpf-event --user-regs=vg kill >/dev/null 2>&1
+ret = 1
+test_ret = true
+arch = aarch64
+auxv = auxv["AT_HWCAP"] & 0x200000 == 0x200000
+kernel_since = 6.1
+
+[event:base-record]
+sample_type=4359
+sample_regs_user=70368744177664
+++ /dev/null
-[config]
-command = stat
-args = --group -e cycles,instructions kill >/dev/null 2>&1
-ret = 1
-
-[event-1:base-stat]
-fd=1
-group_fd=-1
-read_format=3|15
-
-[event-2:base-stat]
-fd=2
-group_fd=1
-config=1
-disabled=0
-enable_on_exec=0
-read_format=3|15
static struct test_suite *generic_tests[] = {
&suite__vmlinux_matches_kallsyms,
+#ifdef HAVE_LIBTRACEEVENT
&suite__openat_syscall_event,
&suite__openat_syscall_event_on_all_cpus,
&suite__basic_mmap,
+#endif
&suite__mem,
&suite__parse_events,
&suite__expr,
&suite__dso_data_cache,
&suite__dso_data_reopen,
&suite__perf_evsel__roundtrip_name_test,
+#ifdef HAVE_LIBTRACEEVENT
&suite__perf_evsel__tp_sched_test,
&suite__syscall_openat_tp_fields,
+#endif
&suite__attr,
&suite__hists_link,
&suite__python_use,
&suite__thread_maps_share,
&suite__hists_output,
&suite__hists_cumulate,
+#ifdef HAVE_LIBTRACEEVENT
&suite__switch_tracking,
+#endif
&suite__fdarray__filter,
&suite__fdarray__add,
&suite__kmod_path__parse,
&suite__perf_time_to_tsc,
&suite__dlfilter,
&suite__sigtrap,
+ &suite__event_groups,
NULL,
};
arch_tests,
};
+static struct test_workload *workloads[] = {
+ &workload__noploop,
+ &workload__thloop,
+ &workload__leafloop,
+ &workload__sqrtloop,
+ &workload__brstack,
+ &workload__datasym,
+};
+
static int num_subtests(const struct test_suite *t)
{
int num;
return 0;
}
+static int run_workload(const char *work, int argc, const char **argv)
+{
+ unsigned int i = 0;
+ struct test_workload *twl;
+
+ for (i = 0; i < ARRAY_SIZE(workloads); i++) {
+ twl = workloads[i];
+ if (!strcmp(twl->name, work))
+ return twl->func(argc, argv);
+ }
+
+ pr_info("No workload found: %s\n", work);
+ return -1;
+}
+
int cmd_test(int argc, const char **argv)
{
const char *test_usage[] = {
NULL,
};
const char *skip = NULL;
+ const char *workload = NULL;
const struct option test_options[] = {
OPT_STRING('s', "skip", &skip, "tests", "tests to skip"),
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show symbol address, etc)"),
OPT_BOOLEAN('F', "dont-fork", &dont_fork,
"Do not fork for testcase"),
+ OPT_STRING('w', "workload", &workload, "work", "workload to run for testing"),
OPT_END()
};
const char * const test_subcommands[] = { "list", NULL };
if (argc >= 1 && !strcmp(argv[0], "list"))
return perf_test__list(argc - 1, argv + 1);
+ if (workload)
+ return run_workload(workload, argc, argv);
+
symbol_conf.priv_size = sizeof(int);
symbol_conf.sort_by_name = true;
symbol_conf.try_vmlinux_path = true;
#include "dso.h"
#include "env.h"
#include "parse-events.h"
-#include "trace-event.h"
#include "evlist.h"
#include "evsel.h"
#include "thread_map.h"
#include "util/mmap.h"
#include "util/string2.h"
#include "util/synthetic-events.h"
+#include "util/util.h"
#include "thread.h"
#include "tests.h"
* see disassemble_bytes() at binutils/objdump.c for details
* how objdump chooses display endian)
*/
- if (bytes_read > 1 && !bigendian()) {
+ if (bytes_read > 1 && !host_is_bigendian()) {
unsigned char *chunk_end = chunk_start + bytes_read - 1;
unsigned char tmp;
#include "util/synthetic-events.h"
#include <string.h>
#include <linux/bitops.h>
-#include <perf/cpumap.h>
+#include <internal/cpumap.h>
#include "debug.h"
struct machine;
#include "archinsn.h"
#include "dlfilter.h"
#include "tests.h"
+#include "util/sample.h"
#define MAP_START 0x400000
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <string.h>
+#include <unistd.h>
+#include <stdio.h>
+#include "linux/perf_event.h"
+#include "tests.h"
+#include "debug.h"
+#include "pmu.h"
+#include "pmus.h"
+#include "header.h"
+#include "../perf-sys.h"
+
+/* hw: cycles, sw: context-switch, uncore: [arch dependent] */
+static int types[] = {0, 1, -1};
+static unsigned long configs[] = {0, 3, 0};
+
+#define NR_UNCORE_PMUS 5
+
+/* Uncore pmus that support more than 3 counters */
+static struct uncore_pmus {
+ const char *name;
+ __u64 config;
+} uncore_pmus[NR_UNCORE_PMUS] = {
+ { "amd_l3", 0x0 },
+ { "amd_df", 0x0 },
+ { "uncore_imc_0", 0x1 }, /* Intel */
+ { "core_imc", 0x318 }, /* PowerPC: core_imc/CPM_STCX_FIN/ */
+ { "hv_24x7", 0x22000000003 }, /* PowerPC: hv_24x7/CPM_STCX_FIN/ */
+};
+
+static int event_open(int type, unsigned long config, int group_fd)
+{
+ struct perf_event_attr attr;
+
+ memset(&attr, 0, sizeof(struct perf_event_attr));
+ attr.type = type;
+ attr.size = sizeof(struct perf_event_attr);
+ attr.config = config;
+ /*
+ * When creating an event group, typically the group leader is
+ * initialized with disabled set to 1 and any child events are
+ * initialized with disabled set to 0. Despite disabled being 0,
+ * the child events will not start until the group leader is
+ * enabled.
+ */
+ attr.disabled = group_fd == -1 ? 1 : 0;
+
+ return sys_perf_event_open(&attr, -1, 0, group_fd, 0);
+}
+
+static int setup_uncore_event(void)
+{
+ struct perf_pmu *pmu;
+ int i, fd;
+
+ if (list_empty(&pmus))
+ perf_pmu__scan(NULL);
+
+ perf_pmus__for_each_pmu(pmu) {
+ for (i = 0; i < NR_UNCORE_PMUS; i++) {
+ if (!strcmp(uncore_pmus[i].name, pmu->name)) {
+ pr_debug("Using %s for uncore pmu event\n", pmu->name);
+ types[2] = pmu->type;
+ configs[2] = uncore_pmus[i].config;
+ /*
+ * Check if the chosen uncore pmu event can be
+ * used in the test. For example, incase of accessing
+ * hv_24x7 pmu counters, partition should have
+ * additional permissions. If not, event open will
+ * fail. So check if the event open succeeds
+ * before proceeding.
+ */
+ fd = event_open(types[2], configs[2], -1);
+ if (fd < 0)
+ return -1;
+ close(fd);
+ return 0;
+ }
+ }
+ }
+ return -1;
+}
+
+static int run_test(int i, int j, int k)
+{
+ int erroneous = ((((1 << i) | (1 << j) | (1 << k)) & 5) == 5);
+ int group_fd, sibling_fd1, sibling_fd2;
+
+ group_fd = event_open(types[i], configs[i], -1);
+ if (group_fd == -1)
+ return -1;
+
+ sibling_fd1 = event_open(types[j], configs[j], group_fd);
+ if (sibling_fd1 == -1) {
+ close(group_fd);
+ return erroneous ? 0 : -1;
+ }
+
+ sibling_fd2 = event_open(types[k], configs[k], group_fd);
+ if (sibling_fd2 == -1) {
+ close(sibling_fd1);
+ close(group_fd);
+ return erroneous ? 0 : -1;
+ }
+
+ close(sibling_fd2);
+ close(sibling_fd1);
+ close(group_fd);
+ return erroneous ? -1 : 0;
+}
+
+static int test__event_groups(struct test_suite *text __maybe_unused, int subtest __maybe_unused)
+{
+ int i, j, k;
+ int ret;
+ int r;
+
+ ret = setup_uncore_event();
+ if (ret || types[2] == -1)
+ return TEST_SKIP;
+
+ ret = TEST_OK;
+ for (i = 0; i < 3; i++) {
+ for (j = 0; j < 3; j++) {
+ for (k = 0; k < 3; k++) {
+ r = run_test(i, j, k);
+ if (r)
+ ret = TEST_FAIL;
+
+ pr_debug("0x%x 0x%lx, 0x%x 0x%lx, 0x%x 0x%lx: %s\n",
+ types[i], configs[i], types[j], configs[j],
+ types[k], configs[k], r ? "Fail" : "Pass");
+ }
+ }
+ }
+ return ret;
+}
+
+DEFINE_SUITE("Event groups", event_groups);
#include "util/cputopo.h"
#include "util/debug.h"
#include "util/expr.h"
+#include "util/hashmap.h"
#include "util/header.h"
#include "util/smt.h"
#include "tests.h"
PARALLEL_OPT=
ifeq ($(SET_PARALLEL),1)
ifeq ($(JOBS),)
- cores := $(shell (getconf _NPROCESSORS_ONLN || egrep -c '^processor|^CPU[0-9]' /proc/cpuinfo) 2>/dev/null)
+ cores := $(shell (getconf _NPROCESSORS_ONLN || grep -E -c '^processor|^CPU[0-9]' /proc/cpuinfo) 2>/dev/null)
ifeq ($(cores),0)
cores := 1
endif
#include <perf/cpumap.h>
#include "debug.h"
+#include "event.h"
#include "evlist.h"
#include "evsel.h"
#include "thread_map.h"
#include "tests.h"
#include "util/mmap.h"
+#include "util/sample.h"
#include <linux/err.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include "util/mmap.h"
#include <errno.h>
#include <perf/mmap.h>
+#include "util/sample.h"
#ifndef O_DIRECTORY
#define O_DIRECTORY 00200000
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
+#include <unistd.h>
#include "thread_map.h"
#include "evsel.h"
#include "debug.h"
#define PERF_TP_SAMPLE_TYPE (PERF_SAMPLE_RAW | PERF_SAMPLE_TIME | \
PERF_SAMPLE_CPU | PERF_SAMPLE_PERIOD)
+#ifdef HAVE_LIBTRACEEVENT
+
#if defined(__s390x__)
/* Return true if kvm module is available and loaded. Test this
* and return success when trace point kvm_s390_create_vm
}
return TEST_OK;
}
+#endif /* HAVE_LIBTRACEEVENT */
static int test__checkevent_raw(struct evlist *evlist)
{
return TEST_OK;
}
+#ifdef HAVE_LIBTRACEEVENT
static int test__checkevent_tracepoint_modifier(struct evlist *evlist)
{
struct evsel *evsel = evlist__first(evlist);
return test__checkevent_tracepoint_multi(evlist);
}
+#endif /* HAVE_LIBTRACEEVENT */
static int test__checkevent_raw_modifier(struct evlist *evlist)
{
return TEST_OK;
}
+#ifdef HAVE_LIBTRACEEVENT
static int test__checkevent_list(struct evlist *evlist)
{
struct evsel *evsel = evlist__first(evlist);
return TEST_OK;
}
+#endif
static int test__checkevent_pmu_name(struct evlist *evlist)
{
return TEST_OK;
}
+#ifdef HAVE_LIBTRACEEVENT
static int test__group3(struct evlist *evlist __maybe_unused)
{
struct evsel *evsel, *leader;
return TEST_OK;
}
+#endif
static int test__group4(struct evlist *evlist __maybe_unused)
{
return TEST_OK;
}
+#ifdef HAVE_LIBTRACEEVENT
static int count_tracepoints(void)
{
struct dirent *events_ent;
return test__checkevent_tracepoint_multi(evlist);
}
+#endif /* HAVE_LIBTRACEVENT */
static int test__hybrid_hw_event_with_pmu(struct evlist *evlist)
{
};
static const struct evlist_test test__events[] = {
+#ifdef HAVE_LIBTRACEEVENT
{
.name = "syscalls:sys_enter_openat",
.check = test__checkevent_tracepoint,
.check = test__checkevent_tracepoint_multi,
/* 1 */
},
+#endif
{
.name = "r1a",
.check = test__checkevent_raw,
.check = test__checkevent_breakpoint_w,
/* 1 */
},
+#ifdef HAVE_LIBTRACEEVENT
{
.name = "syscalls:sys_enter_openat:k",
.check = test__checkevent_tracepoint_modifier,
.check = test__checkevent_tracepoint_multi_modifier,
/* 3 */
},
+#endif
{
.name = "r1a:kp",
.check = test__checkevent_raw_modifier,
.check = test__checkevent_breakpoint_w_modifier,
/* 2 */
},
+#ifdef HAVE_LIBTRACEEVENT
{
.name = "r1,syscalls:sys_enter_openat:k,1:1:hp",
.check = test__checkevent_list,
/* 3 */
},
+#endif
{
.name = "instructions:G",
.check = test__checkevent_exclude_host_modifier,
.check = test__group2,
/* 9 */
},
+#ifdef HAVE_LIBTRACEEVENT
{
.name = "group1{syscalls:sys_enter_openat:H,cycles:kppp},group2{cycles,1:3}:G,instructions:u",
.check = test__group3,
/* 0 */
},
+#endif
{
.name = "{cycles:u,instructions:kp}:p",
.check = test__group4,
.check = test__group5,
/* 2 */
},
+#ifdef HAVE_LIBTRACEEVENT
{
.name = "*:*",
.check = test__all_tracepoints,
/* 3 */
},
+#endif
{
.name = "{cycles,cache-misses:G}:H",
.check = test__group_gh1,
.check = test__checkevent_breakpoint_len_rw_modifier,
/* 4 */
},
-#if defined(__s390x__)
+#if defined(__s390x__) && defined(HAVE_LIBTRACEEVENT)
{
.name = "kvm-s390:kvm_s390_create_vm",
.check = test__checkevent_tracepoint,
pr_debug("Test PMU event failed for '%s'", name);
ret = combine_test_results(ret, test_ret);
}
+ /*
+ * Names containing '-' are recognized as prefixes and suffixes
+ * due to '-' being a legacy PMU separator. This fails when the
+ * prefix or suffix collides with an existing legacy token. For
+ * example, branch-brs has a prefix (branch) that collides with
+ * a PE_NAME_CACHE_TYPE token causing a parse error as a suffix
+ * isn't expected after this. As event names in the config
+ * slashes are allowed a '-' in the name we check this works
+ * above.
+ */
+ if (strchr(ent->d_name, '-'))
+ continue;
+
snprintf(name, sizeof(name), "%s:u,cpu/event=%s/u", ent->d_name, ent->d_name);
e.name = name;
e.check = test__checkevent_pmu_events_mix;
if (err)
goto out;
- err = evlist__alloc_stats(evlist, false);
+ err = evlist__alloc_stats(/*config=*/NULL, evlist, /*alloc_raw=*/false);
if (err)
goto out;
#include "evlist.h"
#include "header.h"
#include "debug.h"
+#include "util/sample.h"
static int process_event(struct evlist **pevlist, union perf_event *event)
{
#include <sched.h>
#include <perf/mmap.h>
+#include "event.h"
#include "evlist.h"
#include "evsel.h"
#include "debug.h"
#include "record.h"
#include "tests.h"
#include "util/mmap.h"
+#include "util/sample.h"
static int sched__get_first_possible_cpu(pid_t pid, cpu_set_t *maskp)
{
#include "tsc.h"
#include "mmap.h"
#include "tests.h"
+#include "util/sample.h"
/*
* Except x86_64/i386 and Arm64, other archs don't support TSC in perf. Just
#include <perf/evlist.h>
#include "util/evlist.h"
#include "util/expr.h"
+#include "util/hashmap.h"
#include "util/parse-events.h"
#include "metricgroup.h"
#include "stat.h"
goto out_err;
}
- err = evlist__alloc_stats(evlist, false);
+ err = evlist__alloc_stats(/*config=*/NULL, evlist, /*alloc_raw=*/false);
if (err)
goto out_err;
/*
#include "evsel.h"
#include "debug.h"
#include "util/synthetic-events.h"
-#include "util/trace-event.h"
+#include "util/util.h"
#include "tests.h"
COMP(branch_stack->hw_idx);
for (i = 0; i < s1->branch_stack->nr; i++) {
if (needs_swap)
- return ((tep_is_bigendian()) ?
+ return ((host_is_bigendian()) ?
(FLAG(s2).value == BS_EXPECTED_BE) :
(FLAG(s2).value == BS_EXPECTED_LE));
else
add_probe_vfs_getname() {
local verbose=$1
if [ $had_vfs_getname -eq 1 ] ; then
- line=$(perf probe -L getname_flags 2>&1 | egrep 'result.*=.*filename;' | sed -r 's/[[:space:]]+([[:digit:]]+)[[:space:]]+result->uptr.*/\1/')
+ line=$(perf probe -L getname_flags 2>&1 | grep -E 'result.*=.*filename;' | sed -r 's/[[:space:]]+([[:digit:]]+)[[:space:]]+result->uptr.*/\1/')
perf probe -q "vfs_getname=getname_flags:${line} pathname=result->name:string" || \
perf probe $verbose "vfs_getname=getname_flags:${line} pathname=filename:ustring"
fi
}
skip_if_no_debuginfo() {
- add_probe_vfs_getname -v 2>&1 | egrep -q "^(Failed to find the path for the kernel|Debuginfo-analysis is not supported)|(file has no debug information)" && return 2
+ add_probe_vfs_getname -v 2>&1 | grep -E -q "^(Failed to find the path for the kernel|Debuginfo-analysis is not supported)|(file has no debug information)" && return 2
return 1
}
if ! perf lock con -b true > /dev/null 2>&1 ; then
echo "[Skip] No BPF support"
- exit
+ return
fi
# the perf lock contention output goes to the stderr
fi
}
+test_record_concurrent()
+{
+ echo "Testing perf lock record and perf lock contention at the same time"
+ perf lock record -o- -- perf bench sched messaging 2> /dev/null | \
+ perf lock contention -i- -E 1 -q 2> ${result}
+ if [ $(cat "${result}" | wc -l) != "1" ]; then
+ echo "[Fail] Recorded result count is not 1:" $(cat "${result}" | wc -l)
+ err=1
+ exit
+ fi
+}
+
+test_aggr_task()
+{
+ echo "Testing perf lock contention --threads"
+ perf lock contention -i ${perfdata} -t -E 1 -q 2> ${result}
+ if [ $(cat "${result}" | wc -l) != "1" ]; then
+ echo "[Fail] Recorded result count is not 1:" $(cat "${result}" | wc -l)
+ err=1
+ exit
+ fi
+
+ if ! perf lock con -b true > /dev/null 2>&1 ; then
+ return
+ fi
+
+ # the perf lock contention output goes to the stderr
+ perf lock con -a -b -t -E 1 -q -- perf bench sched messaging > /dev/null 2> ${result}
+ if [ $(cat "${result}" | wc -l) != "1" ]; then
+ echo "[Fail] BPF result count is not 1:" $(cat "${result}" | wc -l)
+ err=1
+ exit
+ fi
+}
+
+test_aggr_addr()
+{
+ echo "Testing perf lock contention --lock-addr"
+ perf lock contention -i ${perfdata} -l -E 1 -q 2> ${result}
+ if [ $(cat "${result}" | wc -l) != "1" ]; then
+ echo "[Fail] Recorded result count is not 1:" $(cat "${result}" | wc -l)
+ err=1
+ exit
+ fi
+
+ if ! perf lock con -b true > /dev/null 2>&1 ; then
+ return
+ fi
+
+ # the perf lock contention output goes to the stderr
+ perf lock con -a -b -t -E 1 -q -- perf bench sched messaging > /dev/null 2> ${result}
+ if [ $(cat "${result}" | wc -l) != "1" ]; then
+ echo "[Fail] BPF result count is not 1:" $(cat "${result}" | wc -l)
+ err=1
+ exit
+ fi
+}
+
check
test_record
test_bpf
+test_record_concurrent
+test_aggr_task
+test_aggr_addr
exit ${err}
# perf pipe recording and injection test
# SPDX-License-Identifier: GPL-2.0
-# skip if there's no compiler
-if ! [ -x "$(command -v cc)" ]; then
- echo "failed: no compiler, install gcc"
- exit 2
-fi
-
-file=$(mktemp /tmp/test.file.XXXXXX)
data=$(mktemp /tmp/perf.data.XXXXXX)
+prog="perf test -w noploop"
+task="perf"
+sym="noploop"
-cat <<EOF | cc -o ${file} -x c -
-#include <signal.h>
-#include <stdlib.h>
-#include <unistd.h>
-
-volatile int done;
-
-void sigalrm(int sig) {
- done = 1;
-}
-
-__attribute__((noinline)) void noploop(void) {
- while (!done)
- continue;
-}
-
-int main(int argc, char *argv[]) {
- int sec = 1;
-
- if (argc > 1)
- sec = atoi(argv[1]);
-
- signal(SIGALRM, sigalrm);
- alarm(sec);
-
- noploop();
- return 0;
-}
-EOF
-
-
-if ! perf record -e task-clock:u -o - ${file} | perf report -i - --task | grep test.file; then
+if ! perf record -e task-clock:u -o - ${prog} | perf report -i - --task | grep ${task}; then
echo "cannot find the test file in the perf report"
exit 1
fi
-if ! perf record -e task-clock:u -o - ${file} | perf inject -b | perf report -i - | grep noploop; then
+if ! perf record -e task-clock:u -o - ${prog} | perf inject -b | perf report -i - | grep ${sym}; then
echo "cannot find noploop function in pipe #1"
exit 1
fi
-perf record -e task-clock:u -o - ${file} | perf inject -b -o ${data}
-if ! perf report -i ${data} | grep noploop; then
+perf record -e task-clock:u -o - ${prog} | perf inject -b -o ${data}
+if ! perf report -i ${data} | grep ${sym}; then
echo "cannot find noploop function in pipe #2"
exit 1
fi
-perf record -e task-clock:u -o ${data} ${file}
-if ! perf inject -b -i ${data} | perf report -i - | grep noploop; then
+perf record -e task-clock:u -o ${data} ${prog}
+if ! perf inject -b -i ${data} | perf report -i - | grep ${sym}; then
echo "cannot find noploop function in pipe #3"
exit 1
fi
-rm -f ${file} ${data} ${data}.old
+rm -f ${data} ${data}.old
exit 0
while read line <&3 && read -r pattern <&4; do
[ -z "$pattern" ] && break
echo $line
- echo "$line" | egrep -q "$pattern"
+ echo "$line" | grep -E -q "$pattern"
if [ $? -ne 0 ] ; then
printf "FAIL: expected backtrace entry \"%s\" got \"%s\"\n" "$pattern" "$line"
return 1
perf_script_filenames() {
echo "Looking at perf.data file for vfs_getname records for the file we touched:"
perf script -i ${perfdata} | \
- egrep " +touch +[0-9]+ +\[[0-9]+\] +[0-9]+\.[0-9]+: +probe:vfs_getname[_0-9]*: +\([[:xdigit:]]+\) +pathname=\"${file}\""
+ grep -E " +touch +[0-9]+ +\[[0-9]+\] +[0-9]+\.[0-9]+: +probe:vfs_getname[_0-9]*: +\([[:xdigit:]]+\) +pathname=\"${file}\""
}
add_probe_vfs_getname || skip_if_no_debuginfo
set -e
+shelldir=$(dirname "$0")
+. "${shelldir}"/lib/waiting.sh
+
err=0
perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
+testprog="perf test -w thloop"
+testsym="test_loop"
cleanup() {
- rm -f ${perfdata}
- rm -f ${perfdata}.old
- trap - exit term int
+ rm -rf "${perfdata}"
+ rm -rf "${perfdata}".old
+
+ trap - EXIT TERM INT
}
trap_cleanup() {
cleanup
exit 1
}
-trap trap_cleanup exit term int
+trap trap_cleanup EXIT TERM INT
test_per_thread() {
echo "Basic --per-thread mode test"
- if ! perf record -e instructions:u -o ${perfdata} --quiet true 2> /dev/null
+ if ! perf record -o /dev/null --quiet ${testprog} 2> /dev/null
then
- echo "Per-thread record [Skipped instructions:u not supported]"
- if [ $err -ne 1 ]
- then
- err=2
- fi
+ echo "Per-thread record [Skipped event not supported]"
return
fi
- if ! perf record -e instructions:u --per-thread -o ${perfdata} true 2> /dev/null
+ if ! perf record --per-thread -o "${perfdata}" ${testprog} 2> /dev/null
then
- echo "Per-thread record of instructions:u [Failed]"
+ echo "Per-thread record [Failed record]"
err=1
return
fi
- if ! perf report -i ${perfdata} -q | egrep -q true
+ if ! perf report -i "${perfdata}" -q | grep -q "${testsym}"
then
echo "Per-thread record [Failed missing output]"
err=1
return
fi
+
+ # run the test program in background (for 30 seconds)
+ ${testprog} 30 &
+ TESTPID=$!
+
+ rm -f "${perfdata}"
+
+ wait_for_threads ${TESTPID} 2
+ perf record -p "${TESTPID}" --per-thread -o "${perfdata}" sleep 1 2> /dev/null
+ kill ${TESTPID}
+
+ if [ ! -e "${perfdata}" ]
+ then
+ echo "Per-thread record [Failed record -p]"
+ err=1
+ return
+ fi
+ if ! perf report -i "${perfdata}" -q | grep -q "${testsym}"
+ then
+ echo "Per-thread record [Failed -p missing output]"
+ err=1
+ return
+ fi
+
echo "Basic --per-thread mode test [Success]"
}
test_register_capture() {
echo "Register capture test"
- if ! perf list | egrep -q 'br_inst_retired.near_call'
+ if ! perf list | grep -q 'br_inst_retired.near_call'
then
- echo "Register capture test [Skipped missing instruction]"
- if [ $err -ne 1 ]
- then
- err=2
- fi
+ echo "Register capture test [Skipped missing event]"
return
fi
- if ! perf record --intr-regs=\? 2>&1 | egrep -q 'available registers: AX BX CX DX SI DI BP SP IP FLAGS CS SS R8 R9 R10 R11 R12 R13 R14 R15'
+ if ! perf record --intr-regs=\? 2>&1 | grep -q 'available registers: AX BX CX DX SI DI BP SP IP FLAGS CS SS R8 R9 R10 R11 R12 R13 R14 R15'
then
echo "Register capture test [Skipped missing registers]"
return
fi
- if ! perf record -o - --intr-regs=di,r8,dx,cx -e br_inst_retired.near_call:p \
- -c 1000 --per-thread true 2> /dev/null \
+ if ! perf record -o - --intr-regs=di,r8,dx,cx -e br_inst_retired.near_call \
+ -c 1000 --per-thread ${testprog} 2> /dev/null \
| perf script -F ip,sym,iregs -i - 2> /dev/null \
- | egrep -q "DI:"
+ | grep -q "DI:"
then
echo "Register capture test [Failed missing output]"
err=1
echo "Register capture test [Success]"
}
+test_system_wide() {
+ echo "Basic --system-wide mode test"
+ if ! perf record -aB --synth=no -o "${perfdata}" ${testprog} 2> /dev/null
+ then
+ echo "System-wide record [Skipped not supported]"
+ return
+ fi
+ if ! perf report -i "${perfdata}" -q | grep -q "${testsym}"
+ then
+ echo "System-wide record [Failed missing output]"
+ err=1
+ return
+ fi
+ if ! perf record -aB --synth=no -e cpu-clock,cs --threads=cpu \
+ -o "${perfdata}" ${testprog} 2> /dev/null
+ then
+ echo "System-wide record [Failed record --threads option]"
+ err=1
+ return
+ fi
+ if ! perf report -i "${perfdata}" -q | grep -q "${testsym}"
+ then
+ echo "System-wide record [Failed --threads missing output]"
+ err=1
+ return
+ fi
+ echo "Basic --system-wide mode test [Success]"
+}
+
+test_workload() {
+ echo "Basic target workload test"
+ if ! perf record -o "${perfdata}" ${testprog} 2> /dev/null
+ then
+ echo "Workload record [Failed record]"
+ err=1
+ return
+ fi
+ if ! perf report -i "${perfdata}" -q | grep -q "${testsym}"
+ then
+ echo "Workload record [Failed missing output]"
+ err=1
+ return
+ fi
+ if ! perf record -e cpu-clock,cs --threads=package \
+ -o "${perfdata}" ${testprog} 2> /dev/null
+ then
+ echo "Workload record [Failed record --threads option]"
+ err=1
+ return
+ fi
+ if ! perf report -i "${perfdata}" -q | grep -q "${testsym}"
+ then
+ echo "Workload record [Failed --threads missing output]"
+ err=1
+ return
+ fi
+ echo "Basic target workload test [Success]"
+}
+
test_per_thread
test_register_capture
+test_system_wide
+test_workload
cleanup
exit $err
err=1
return
fi
- if ! perf report -i ${perfdata} -q --percent-limit=90 | egrep -q sleep
+ if ! perf report -i ${perfdata} -q --percent-limit=90 | grep -E -q sleep
then
echo "Basic off-cpu test [Failed missing output]"
err=1
err=0
test_default_stat() {
echo "Basic stat command test"
- if ! perf stat true 2>&1 | egrep -q "Performance counter stats for 'true':"
+ if ! perf stat true 2>&1 | grep -E -q "Performance counter stats for 'true':"
then
echo "Basic stat command test [Failed]"
err=1
test_stat_record_report() {
echo "stat record and report test"
if ! perf stat record -o - true | perf stat report -i - 2>&1 | \
- egrep -q "Performance counter stats for 'pipe':"
+ grep -E -q "Performance counter stats for 'pipe':"
then
echo "stat record and report test [Failed]"
err=1
echo "Topdown event group test [Skipped event parsing failed]"
return
fi
- if perf stat -e '{slots,topdown-retiring}' true 2>&1 | egrep -q "<not supported>"
+ if perf stat -e '{slots,topdown-retiring}' true 2>&1 | grep -E -q "<not supported>"
then
echo "Topdown event group test [Failed events not supported]"
err=1
return
fi
- if perf stat -e '{topdown-retiring,slots}' true 2>&1 | egrep -q "<not supported>"
+ if perf stat -e '{topdown-retiring,slots}' true 2>&1 | grep -E -q "<not supported>"
then
echo "Topdown event group test [Failed slots not reordered first]"
err=1
return
fi
group_needs_break="{slots,topdown-bad-spec,topdown-be-bound,topdown-fe-bound,topdown-retiring,branch-instructions,branch-misses,bus-cycles,cache-misses,cache-references,cpu-cycles,instructions,mem-loads,mem-stores,ref-cycles,cache-misses,cache-references}:W"
- if perf stat --no-merge -e "$group_needs_break" true 2>&1 | egrep -q "<not supported>"
+ if perf stat --no-merge -e "$group_needs_break" true 2>&1 | grep -E -q "<not supported>"
then
echo "Topdown weak groups test [Failed events not supported]"
err=1
lscpu | grep -q "aarch64" || exit 2
-if ! [ -x "$(command -v cc)" ]; then
- echo "failed: no compiler, install gcc"
- exit 2
-fi
-
PERF_DATA=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
-TEST_PROGRAM_SOURCE=$(mktemp /tmp/test_program.XXXXX.c)
-TEST_PROGRAM=$(mktemp /tmp/test_program.XXXXX)
+TEST_PROGRAM="perf test -w leafloop"
cleanup_files()
{
rm -f $PERF_DATA
- rm -f $TEST_PROGRAM_SOURCE
- rm -f $TEST_PROGRAM
}
trap cleanup_files exit term int
-cat << EOF > $TEST_PROGRAM_SOURCE
-int a = 0;
-void leaf(void) {
- for (;;)
- a += a;
-}
-void parent(void) {
- leaf();
-}
-int main(void) {
- parent();
- return 0;
-}
-EOF
-
-echo " + Compiling test program ($TEST_PROGRAM)..."
-
-CFLAGS="-g -O0 -fno-inline -fno-omit-frame-pointer"
-cc $CFLAGS $TEST_PROGRAM_SOURCE -o $TEST_PROGRAM || exit 1
-
# Add a 1 second delay to skip samples that are not in the leaf() function
perf record -o $PERF_DATA --call-graph fp -e cycles//u -D 1000 --user-callchains -- $TEST_PROGRAM 2> /dev/null &
PID=$!
# program
# 728 leaf
# 753 parent
-# 76c main
+# 76c leafloop
# ...
perf script -i $PERF_DATA -F comm,ip,sym | head -n4
perf script -i $PERF_DATA -F comm,ip,sym | head -n4 | \
awk '{ if ($2 != "") sym[i++] = $2 } END { if (sym[0] != "leaf" ||
sym[1] != "parent" ||
- sym[2] != "main") exit 1 }'
+ sym[2] != "leafloop") exit 1 }'
# touch 6512 1 branches:u: ffffb22082e0 strcmp+0xa0 (/lib/aarch64-linux-gnu/ld-2.27.so)
# touch 6512 1 branches:u: ffffb2208320 strcmp+0xe0 (/lib/aarch64-linux-gnu/ld-2.27.so)
perf script -F,-time -i ${perfdata} 2>&1 | \
- egrep " +$1 +[0-9]+ .* +branches:(.*:)? +" > /dev/null 2>&1
+ grep -E " +$1 +[0-9]+ .* +branches:(.*:)? +" > /dev/null 2>&1
}
perf_report_branch_samples() {
# 7.71% 7.71% touch libc-2.27.so [.] getenv
# 2.59% 2.59% touch ld-2.27.so [.] strcmp
perf report --stdio -i ${perfdata} 2>&1 | \
- egrep " +[0-9]+\.[0-9]+% +[0-9]+\.[0-9]+% +$1 " > /dev/null 2>&1
+ grep -E " +[0-9]+\.[0-9]+% +[0-9]+\.[0-9]+% +$1 " > /dev/null 2>&1
}
perf_report_instruction_samples() {
# 5.80% touch libc-2.27.so [.] getenv
# 4.35% touch ld-2.27.so [.] _dl_fixup
perf report --itrace=i20i --stdio -i ${perfdata} 2>&1 | \
- egrep " +[0-9]+\.[0-9]+% +$1" > /dev/null 2>&1
+ grep -E " +[0-9]+\.[0-9]+% +$1" > /dev/null 2>&1
}
arm_cs_report() {
# If the node of "enable_sink" is existed under the device path, this
# means the device is a sink device. Need to exclude 'tpiu' since it
# cannot support perf PMU.
- echo "$1" | egrep -q -v "tpiu"
+ echo "$1" | grep -E -q -v "tpiu"
if [ $? -eq 0 -a -e "$1/enable_sink" ]; then
# German Gomez <german.gomez@arm.com>, 2021
skip_if_no_arm_spe_event() {
- perf list | egrep -q 'arm_spe_[0-9]+//' && return 0
+ perf list | grep -E -q 'arm_spe_[0-9]+//' && return 0
# arm_spe event doesn't exist
return 2
# dd 3048 [002] 1 tlb-access: ffffaa64999c __GI___libc_write+0x3c (/lib/aarch64-linux-gnu/libc-2.27.so)
# dd 3048 [002] 1 memory: ffffaa64999c __GI___libc_write+0x3c (/lib/aarch64-linux-gnu/libc-2.27.so)
perf script -F,-time -i ${perfdata} 2>&1 | \
- egrep " +$1 +[0-9]+ .* +${events}:(.*:)? +" > /dev/null 2>&1
+ grep -E " +$1 +[0-9]+ .* +${events}:(.*:)? +" > /dev/null 2>&1
}
perf_report_samples() {
# 7.71% 7.71% dd libc-2.27.so [.] getenv
# 2.59% 2.59% dd ld-2.27.so [.] strcmp
perf report --stdio -i ${perfdata} 2>&1 | \
- egrep " +[0-9]+\.[0-9]+% +[0-9]+\.[0-9]+% +$1 " > /dev/null 2>&1
+ grep -E " +[0-9]+\.[0-9]+% +[0-9]+\.[0-9]+% +$1 " > /dev/null 2>&1
}
arm_spe_snapshot_test() {
# German Gomez <german.gomez@arm.com>, 2022
skip_if_no_arm_spe_event() {
- perf list | egrep -q 'arm_spe_[0-9]+//' && return 0
+ perf list | grep -E -q 'arm_spe_[0-9]+//' && return 0
return 2
}
skip_if_no_arm_spe_event || exit 2
-# skip if there's no compiler
-if ! [ -x "$(command -v cc)" ]; then
- echo "failed: no compiler, install gcc"
- exit 2
-fi
-
-TEST_PROGRAM_SOURCE=$(mktemp /tmp/__perf_test.program.XXXXX.c)
-TEST_PROGRAM=$(mktemp /tmp/__perf_test.program.XXXXX)
+TEST_PROGRAM="perf test -w sqrtloop 10"
PERF_DATA=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
PERF_RECORD_LOG=$(mktemp /tmp/__perf_test.log.XXXXX)
echo "Cleaning up files..."
rm -f ${PERF_RECORD_LOG}
rm -f ${PERF_DATA}
- rm -f ${TEST_PROGRAM_SOURCE}
- rm -f ${TEST_PROGRAM}
}
trap cleanup_files exit term int
-# compile test program
-cat << EOF > $TEST_PROGRAM_SOURCE
-#include <math.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <unistd.h>
-#include <sys/wait.h>
-
-int workload() {
- while (1)
- sqrt(rand());
- return 0;
-}
-
-int main() {
- switch (fork()) {
- case 0:
- return workload();
- case -1:
- return 1;
- default:
- wait(NULL);
- }
- return 0;
-}
-EOF
-
-echo "Compiling test program..."
-CFLAGS="-lm"
-cc $TEST_PROGRAM_SOURCE $CFLAGS -o $TEST_PROGRAM || exit 1
-
echo "Recording workload..."
perf record -o ${PERF_DATA} -e arm_spe/period=65536/ -vvv -- $TEST_PROGRAM > ${PERF_RECORD_LOG} 2>&1 &
PERFPID=$!
kill $PERFPID
wait $PERFPID
-# test program may leave an orphan process running the workload
-killall $(basename $TEST_PROGRAM)
if [ "$log0" = "$log1" ];
then
# SPDX-License-Identifier: GPL-2.0
# German Gomez <german.gomez@arm.com>, 2022
-# we need a C compiler to build the test programs
-# so bail if none is found
-if ! [ -x "$(command -v cc)" ]; then
- echo "failed: no compiler, install gcc"
- exit 2
-fi
-
# skip the test if the hardware doesn't support branch stack sampling
# and if the architecture doesn't support filter types: any,save_type,u
if ! perf record -o- --no-buildid --branch-filter any,save_type,u -- true > /dev/null 2>&1 ; then
fi
TMPDIR=$(mktemp -d /tmp/__perf_test.program.XXXXX)
+TESTPROG="perf test -w brstack"
cleanup() {
rm -rf $TMPDIR
trap cleanup exit term int
-gen_test_program() {
- # generate test program
- cat << EOF > $1
-#define BENCH_RUNS 999999
-int cnt;
-void bar(void) {
-} /* return */
-void foo(void) {
- bar(); /* call */
-} /* return */
-void bench(void) {
- void (*foo_ind)(void) = foo;
- if ((cnt++) % 3) /* branch (cond) */
- foo(); /* call */
- bar(); /* call */
- foo_ind(); /* call (ind) */
-}
-int main(void)
-{
- int cnt = 0;
- while (1) {
- if ((cnt++) > BENCH_RUNS)
- break;
- bench(); /* call */
- } /* branch (uncond) */
- return 0;
-}
-EOF
-}
-
test_user_branches() {
echo "Testing user branch stack sampling"
- gen_test_program "$TEMPDIR/program.c"
- cc -fno-inline -g "$TEMPDIR/program.c" -o $TMPDIR/a.out
-
- perf record -o $TMPDIR/perf.data --branch-filter any,save_type,u -- $TMPDIR/a.out > /dev/null 2>&1
+ perf record -o $TMPDIR/perf.data --branch-filter any,save_type,u -- ${TESTPROG} > /dev/null 2>&1
perf script -i $TMPDIR/perf.data --fields brstacksym | xargs -n1 > $TMPDIR/perf.script
# example of branch entries:
- # foo+0x14/bar+0x40/P/-/-/0/CALL
+ # brstack_foo+0x14/brstack_bar+0x40/P/-/-/0/CALL
set -x
- egrep -m1 "^bench\+[^ ]*/foo\+[^ ]*/IND_CALL$" $TMPDIR/perf.script
- egrep -m1 "^foo\+[^ ]*/bar\+[^ ]*/CALL$" $TMPDIR/perf.script
- egrep -m1 "^bench\+[^ ]*/foo\+[^ ]*/CALL$" $TMPDIR/perf.script
- egrep -m1 "^bench\+[^ ]*/bar\+[^ ]*/CALL$" $TMPDIR/perf.script
- egrep -m1 "^bar\+[^ ]*/foo\+[^ ]*/RET$" $TMPDIR/perf.script
- egrep -m1 "^foo\+[^ ]*/bench\+[^ ]*/RET$" $TMPDIR/perf.script
- egrep -m1 "^bench\+[^ ]*/bench\+[^ ]*/COND$" $TMPDIR/perf.script
- egrep -m1 "^main\+[^ ]*/main\+[^ ]*/UNCOND$" $TMPDIR/perf.script
+ grep -E -m1 "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/IND_CALL$" $TMPDIR/perf.script
+ grep -E -m1 "^brstack_foo\+[^ ]*/brstack_bar\+[^ ]*/CALL$" $TMPDIR/perf.script
+ grep -E -m1 "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/CALL$" $TMPDIR/perf.script
+ grep -E -m1 "^brstack_bench\+[^ ]*/brstack_bar\+[^ ]*/CALL$" $TMPDIR/perf.script
+ grep -E -m1 "^brstack_bar\+[^ ]*/brstack_foo\+[^ ]*/RET$" $TMPDIR/perf.script
+ grep -E -m1 "^brstack_foo\+[^ ]*/brstack_bench\+[^ ]*/RET$" $TMPDIR/perf.script
+ grep -E -m1 "^brstack_bench\+[^ ]*/brstack_bench\+[^ ]*/COND$" $TMPDIR/perf.script
+ grep -E -m1 "^brstack\+[^ ]*/brstack\+[^ ]*/UNCOND$" $TMPDIR/perf.script
set +x
# some branch types are still not being tested:
echo "Testing branch stack filtering permutation ($filter,$expect)"
- gen_test_program "$TEMPDIR/program.c"
- cc -fno-inline -g "$TEMPDIR/program.c" -o $TMPDIR/a.out
-
- perf record -o $TMPDIR/perf.data --branch-filter $filter,save_type,u -- $TMPDIR/a.out > /dev/null 2>&1
+ perf record -o $TMPDIR/perf.data --branch-filter $filter,save_type,u -- ${TESTPROG} > /dev/null 2>&1
perf script -i $TMPDIR/perf.data --fields brstack | xargs -n1 > $TMPDIR/perf.script
# fail if we find any branch type that doesn't match any of the expected ones
# also consider UNKNOWN branch types (-)
- if egrep -vm1 "^[^ ]*/($expect|-|( *))$" $TMPDIR/perf.script; then
+ if grep -E -vm1 "^[^ ]*/($expect|-|( *))$" $TMPDIR/perf.script; then
return 1
fi
}
# Leo Yan <leo.yan@linaro.org>, 2022
skip_if_no_mem_event() {
- perf mem record -e list 2>&1 | egrep -q 'available' && return 0
+ perf mem record -e list 2>&1 | grep -E -q 'available' && return 0
return 2
}
skip_if_no_mem_event || exit 2
-# skip if there's no compiler
-if ! [ -x "$(command -v cc)" ]; then
- echo "skip: no compiler, install gcc"
- exit 2
-fi
-
-TEST_PROGRAM=$(mktemp /tmp/__perf_test.program.XXXXX)
+TEST_PROGRAM="perf test -w datasym"
PERF_DATA=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
check_result() {
{
echo "Cleaning up files..."
rm -f ${PERF_DATA}
- rm -f ${TEST_PROGRAM}
}
trap cleanup_files exit term int
-# compile test program
-echo "Compiling test program..."
-cat << EOF | cc -o ${TEST_PROGRAM} -x c -
-typedef struct _buf {
- char data1;
- char reserved[55];
- char data2;
-} buf __attribute__((aligned(64)));
-
-static buf buf1;
-
-int main(void) {
- for (;;) {
- buf1.data1++;
- buf1.data2 += buf1.data1;
- }
- return 0;
-}
-EOF
-
echo "Recording workload..."
# perf mem/c2c internally uses IBS PMU on AMD CPU which doesn't support
# user/kernel filtering and per-process monitoring, spin program on
# specific CPU and test in per-CPU mode.
-is_amd=$(egrep -c 'vendor_id.*AuthenticAMD' /proc/cpuinfo)
+is_amd=$(grep -E -c 'vendor_id.*AuthenticAMD' /proc/cpuinfo)
if (($is_amd >= 1)); then
perf mem record -o ${PERF_DATA} -C 0 -- taskset -c 0 $TEST_PROGRAM &
else
# 8.18% jshell jitted-50116-29.so [.] Interpreter
# 0.75% Thread-1 jitted-83602-1670.so [.] jdk.internal.jimage.BasicImageReader.getString(int)
perf report --stdio -i ${PERF_INJ_DATA} 2>&1 | \
- egrep " +[0-9]+\.[0-9]+% .* (Interpreter|jdk\.internal).*" > /dev/null 2>&1
+ grep -E " +[0-9]+\.[0-9]+% .* (Interpreter|jdk\.internal).*" > /dev/null 2>&1
if [ $? -ne 0 ]; then
echo "Fail to find java symbols"
--- /dev/null
+#!/bin/bash
+# perf script task-analyzer tests
+# SPDX-License-Identifier: GPL-2.0
+
+tmpdir=$(mktemp -d /tmp/perf-script-task-analyzer-XXXXX)
+err=0
+
+cleanup() {
+ rm -f perf.data
+ rm -f perf.data.old
+ rm -f csv
+ rm -f csvsummary
+ rm -rf $tmpdir
+ trap - exit term int
+}
+
+trap_cleanup() {
+ cleanup
+ exit 1
+}
+trap trap_cleanup exit term int
+
+report() {
+ if [ $1 = 0 ]; then
+ echo "PASS: \"$2\""
+ else
+ echo "FAIL: \"$2\" Error message: \"$3\""
+ err=1
+ fi
+}
+
+check_exec_0() {
+ if [ $? != 0 ]; then
+ report 1 "invokation of ${$1} command failed"
+ fi
+}
+
+find_str_or_fail() {
+ grep -q "$1" $2
+ if [ $? != 0 ]; then
+ report 1 $3 "Failed to find required string:'${1}'."
+ else
+ report 0 $3
+ fi
+}
+
+prepare_perf_data() {
+ # 1s should be sufficient to catch at least some switches
+ perf record -e sched:sched_switch -a -- sleep 1 > /dev/null 2>&1
+}
+
+# check standard inkvokation with no arguments
+test_basic() {
+ out="$tmpdir/perf.out"
+ perf script report task-analyzer > $out
+ check_exec_0 "perf"
+ find_str_or_fail "Comm" $out ${FUNCNAME[0]}
+}
+
+test_ns_rename(){
+ out="$tmpdir/perf.out"
+ perf script report task-analyzer --ns --rename-comms-by-tids 0:random > $out
+ check_exec_0 "perf"
+ find_str_or_fail "Comm" $out ${FUNCNAME[0]}
+}
+
+test_ms_filtertasks_highlight(){
+ out="$tmpdir/perf.out"
+ perf script report task-analyzer --ms --filter-tasks perf --highlight-tasks perf \
+ > $out
+ check_exec_0 "perf"
+ find_str_or_fail "Comm" $out ${FUNCNAME[0]}
+}
+
+test_extended_times_timelimit_limittasks() {
+ out="$tmpdir/perf.out"
+ perf script report task-analyzer --extended-times --time-limit :99999 \
+ --limit-to-tasks perf > $out
+ check_exec_0 "perf"
+ find_str_or_fail "Out-Out" $out ${FUNCNAME[0]}
+}
+
+test_summary() {
+ out="$tmpdir/perf.out"
+ perf script report task-analyzer --summary > $out
+ check_exec_0 "perf"
+ find_str_or_fail "Summary" $out ${FUNCNAME[0]}
+}
+
+test_summaryextended() {
+ out="$tmpdir/perf.out"
+ perf script report task-analyzer --summary-extended > $out
+ check_exec_0 "perf"
+ find_str_or_fail "Inter Task Times" $out ${FUNCNAME[0]}
+}
+
+test_summaryonly() {
+ out="$tmpdir/perf.out"
+ perf script report task-analyzer --summary-only > $out
+ check_exec_0 "perf"
+ find_str_or_fail "Summary" $out ${FUNCNAME[0]}
+}
+
+test_extended_times_summary_ns() {
+ out="$tmpdir/perf.out"
+ perf script report task-analyzer --extended-times --summary --ns > $out
+ check_exec_0 "perf"
+ find_str_or_fail "Out-Out" $out ${FUNCNAME[0]}
+ find_str_or_fail "Summary" $out ${FUNCNAME[0]}
+}
+
+test_csv() {
+ perf script report task-analyzer --csv csv > /dev/null
+ check_exec_0 "perf"
+ find_str_or_fail "Comm;" csv ${FUNCNAME[0]}
+}
+
+test_csv_extended_times() {
+ perf script report task-analyzer --csv csv --extended-times > /dev/null
+ check_exec_0 "perf"
+ find_str_or_fail "Out-Out;" csv ${FUNCNAME[0]}
+}
+
+test_csvsummary() {
+ perf script report task-analyzer --csv-summary csvsummary > /dev/null
+ check_exec_0 "perf"
+ find_str_or_fail "Comm;" csvsummary ${FUNCNAME[0]}
+}
+
+test_csvsummary_extended() {
+ perf script report task-analyzer --csv-summary csvsummary --summary-extended \
+ >/dev/null
+ check_exec_0 "perf"
+ find_str_or_fail "Out-Out;" csvsummary ${FUNCNAME[0]}
+}
+
+prepare_perf_data
+test_basic
+test_ns_rename
+test_ms_filtertasks_highlight
+test_extended_times_timelimit_limittasks
+test_summary
+test_summaryextended
+test_summaryonly
+test_extended_times_summary_ns
+test_csv
+test_csvsummary
+test_csv_extended_times
+test_csvsummary_extended
+cleanup
+exit $err
. $(dirname $0)/lib/probe_vfs_getname.sh
trace_open_vfs_getname() {
- evts=$(echo $(perf list syscalls:sys_enter_open* 2>/dev/null | egrep 'open(at)? ' | sed -r 's/.*sys_enter_([a-z]+) +\[.*$/\1/') | sed 's/ /,/')
+ evts=$(echo $(perf list syscalls:sys_enter_open* 2>/dev/null | grep -E 'open(at)? ' | sed -r 's/.*sys_enter_([a-z]+) +\[.*$/\1/') | sed 's/ /,/')
perf trace -e $evts touch $file 2>&1 | \
- egrep " +[0-9]+\.[0-9]+ +\( +[0-9]+\.[0-9]+ ms\): +touch\/[0-9]+ open(at)?\((dfd: +CWD, +)?filename: +${file}, +flags: CREAT\|NOCTTY\|NONBLOCK\|WRONLY, +mode: +IRUGO\|IWUGO\) += +[0-9]+$"
+ grep -E " +[0-9]+\.[0-9]+ +\( +[0-9]+\.[0-9]+ ms\): +touch\/[0-9]+ open(at)?\((dfd: +CWD, +)?filename: +${file}, +flags: CREAT\|NOCTTY\|NONBLOCK\|WRONLY, +mode: +IRUGO\|IWUGO\) += +[0-9]+$"
}
#include "util/evlist.h"
#include "util/cpumap.h"
#include "util/mmap.h"
+#include "util/sample.h"
#include "util/thread_map.h"
#include <perf/evlist.h>
#include <perf/mmap.h>
#include "record.h"
#include "tests.h"
#include "util/mmap.h"
+#include "util/sample.h"
#include "pmu.h"
static int spin_sleep(void)
DECLARE_SUITE(perf_time_to_tsc);
DECLARE_SUITE(dlfilter);
DECLARE_SUITE(sigtrap);
+DECLARE_SUITE(event_groups);
/*
* PowerPC and S390 do not support creation of instruction breakpoints using the
DECLARE_SUITE(vectors_page);
#endif
+/*
+ * Define test workloads to be used in test suites.
+ */
+typedef int (*workload_fnptr)(int argc, const char **argv);
+
+struct test_workload {
+ const char *name;
+ workload_fnptr func;
+};
+
+#define DECLARE_WORKLOAD(work) \
+ extern struct test_workload workload__##work
+
+#define DEFINE_WORKLOAD(work) \
+struct test_workload workload__##work = { \
+ .name = #work, \
+ .func = work, \
+}
+
+/* The list of test workloads */
+DECLARE_WORKLOAD(noploop);
+DECLARE_WORKLOAD(thloop);
+DECLARE_WORKLOAD(leafloop);
+DECLARE_WORKLOAD(sqrtloop);
+DECLARE_WORKLOAD(brstack);
+DECLARE_WORKLOAD(datasym);
+
#endif /* TESTS_H */
#include "util/synthetic-events.h"
#include <linux/zalloc.h>
#include <perf/event.h>
+#include <internal/threadmap.h>
struct perf_sample;
struct perf_tool;
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+
+perf-y += noploop.o
+perf-y += thloop.o
+perf-y += leafloop.o
+perf-y += sqrtloop.o
+perf-y += brstack.o
+perf-y += datasym.o
+
+CFLAGS_sqrtloop.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE
+CFLAGS_leafloop.o = -g -O0 -fno-inline -fno-omit-frame-pointer -U_FORTIFY_SOURCE
+CFLAGS_brstack.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE
+CFLAGS_datasym.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <stdlib.h>
+#include "../tests.h"
+
+#define BENCH_RUNS 999999
+
+static volatile int cnt;
+
+static void brstack_bar(void) {
+} /* return */
+
+static void brstack_foo(void) {
+ brstack_bar(); /* call */
+} /* return */
+
+static void brstack_bench(void) {
+ void (*brstack_foo_ind)(void) = brstack_foo;
+
+ if ((cnt++) % 3) /* branch (cond) */
+ brstack_foo(); /* call */
+ brstack_bar(); /* call */
+ brstack_foo_ind(); /* call (ind) */
+}
+
+static int brstack(int argc, const char **argv)
+{
+ int num_loops = BENCH_RUNS;
+
+ if (argc > 0)
+ num_loops = atoi(argv[0]);
+
+ while (1) {
+ if ((cnt++) > num_loops)
+ break;
+ brstack_bench();/* call */
+ } /* branch (uncond) */
+ return 0;
+}
+
+DEFINE_WORKLOAD(brstack);
--- /dev/null
+#include <linux/compiler.h>
+#include "../tests.h"
+
+typedef struct _buf {
+ char data1;
+ char reserved[55];
+ char data2;
+} buf __attribute__((aligned(64)));
+
+static buf buf1 = {
+ /* to have this in the data section */
+ .reserved[0] = 1,
+};
+
+static int datasym(int argc __maybe_unused, const char **argv __maybe_unused)
+{
+ for (;;) {
+ buf1.data1++;
+ buf1.data2 += buf1.data1;
+ }
+ return 0;
+}
+
+DEFINE_WORKLOAD(datasym);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <stdlib.h>
+#include <linux/compiler.h>
+#include "../tests.h"
+
+/* We want to check these symbols in perf script */
+noinline void leaf(volatile int b);
+noinline void parent(volatile int b);
+
+static volatile int a;
+
+noinline void leaf(volatile int b)
+{
+ for (;;)
+ a += b;
+}
+
+noinline void parent(volatile int b)
+{
+ leaf(b);
+}
+
+static int leafloop(int argc, const char **argv)
+{
+ int c = 1;
+
+ if (argc > 0)
+ c = atoi(argv[0]);
+
+ parent(c);
+ return 0;
+}
+
+DEFINE_WORKLOAD(leafloop);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <stdlib.h>
+#include <signal.h>
+#include <unistd.h>
+#include <linux/compiler.h>
+#include "../tests.h"
+
+static volatile sig_atomic_t done;
+
+static void sighandler(int sig __maybe_unused)
+{
+ done = 1;
+}
+
+static int noploop(int argc, const char **argv)
+{
+ int sec = 1;
+
+ if (argc > 0)
+ sec = atoi(argv[0]);
+
+ signal(SIGINT, sighandler);
+ signal(SIGALRM, sighandler);
+ alarm(sec);
+
+ while (!done)
+ continue;
+
+ return 0;
+}
+
+DEFINE_WORKLOAD(noploop);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <math.h>
+#include <signal.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <linux/compiler.h>
+#include <sys/wait.h>
+#include "../tests.h"
+
+static volatile sig_atomic_t done;
+
+static void sighandler(int sig __maybe_unused)
+{
+ done = 1;
+}
+
+static int __sqrtloop(int sec)
+{
+ signal(SIGALRM, sighandler);
+ alarm(sec);
+
+ while (!done)
+ (void)sqrt(rand());
+ return 0;
+}
+
+static int sqrtloop(int argc, const char **argv)
+{
+ int sec = 1;
+
+ if (argc > 0)
+ sec = atoi(argv[0]);
+
+ switch (fork()) {
+ case 0:
+ return __sqrtloop(sec);
+ case -1:
+ return -1;
+ default:
+ wait(NULL);
+ }
+ return 0;
+}
+
+DEFINE_WORKLOAD(sqrtloop);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <pthread.h>
+#include <stdlib.h>
+#include <signal.h>
+#include <unistd.h>
+#include <linux/compiler.h>
+#include "../tests.h"
+
+static volatile sig_atomic_t done;
+static volatile unsigned count;
+
+/* We want to check this symbol in perf report */
+noinline void test_loop(void);
+
+static void sighandler(int sig __maybe_unused)
+{
+ done = 1;
+}
+
+noinline void test_loop(void)
+{
+ while (!done)
+ count++;
+}
+
+static void *thfunc(void *arg)
+{
+ void (*loop_fn)(void) = arg;
+
+ loop_fn();
+ return NULL;
+}
+
+static int thloop(int argc, const char **argv)
+{
+ int sec = 1;
+ pthread_t th;
+
+ if (argc > 0)
+ sec = atoi(argv[0]);
+
+ signal(SIGINT, sighandler);
+ signal(SIGALRM, sighandler);
+ alarm(sec);
+
+ pthread_create(&th, NULL, thfunc, test_loop);
+ test_loop();
+ pthread_join(th, NULL);
+
+ return 0;
+}
+
+DEFINE_WORKLOAD(thloop);
get__perf_event_attr(&attr, wp_type, wp_addr, wp_len);
fd = sys_perf_event_open(&attr, 0, -1, -1,
perf_event_open_cloexec_flag());
- if (fd < 0)
+ if (fd < 0) {
+ fd = -errno;
pr_debug("failed opening event %x\n", attr.bp_type);
+ }
return fd;
}
fd = __event(HW_BREAKPOINT_R, (void *)&data1, sizeof(data1));
if (fd < 0)
- return -1;
+ return fd == -ENODEV ? TEST_SKIP : -1;
tmp = data1;
WP_TEST_ASSERT_VAL(fd, "RO watchpoint", 1);
fd = __event(HW_BREAKPOINT_W, (void *)&data1, sizeof(data1));
if (fd < 0)
- return -1;
+ return fd == -ENODEV ? TEST_SKIP : -1;
tmp = data1;
WP_TEST_ASSERT_VAL(fd, "WO watchpoint", 0);
fd = __event(HW_BREAKPOINT_R | HW_BREAKPOINT_W, (void *)&data1,
sizeof(data1));
if (fd < 0)
- return -1;
+ return fd == -ENODEV ? TEST_SKIP : -1;
tmp = data1;
WP_TEST_ASSERT_VAL(fd, "RW watchpoint", 1);
fd = __event(HW_BREAKPOINT_W, (void *)&data1, sizeof(data1));
if (fd < 0)
- return -1;
+ return fd == -ENODEV ? TEST_SKIP : -1;
data1 = tmp;
WP_TEST_ASSERT_VAL(fd, "Modify watchpoint", 1);
perf-y += socket.o
perf-y += statx.o
perf-y += sync_file_range.o
+perf-y += timespec.o
perf-y += tracepoints/
size_t syscall_arg__scnprintf_sync_file_range_flags(char *bf, size_t size, struct syscall_arg *arg);
#define SCA_SYNC_FILE_RANGE_FLAGS syscall_arg__scnprintf_sync_file_range_flags
+size_t syscall_arg__scnprintf_timespec(char *bf, size_t size, struct syscall_arg *arg);
+#define SCA_TIMESPEC syscall_arg__scnprintf_timespec
+
size_t open__scnprintf_flags(unsigned long flags, char *bf, size_t size, bool show_prefix);
void syscall_arg__set_ret_scnprintf(struct syscall_arg *arg,
printf "static const char *fadvise_advices[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+POSIX_FADV_(\w+)[[:space:]]+([[:digit:]]+)[[:space:]]+.*'
-egrep $regex ${header_dir}/fadvise.h | \
+grep -E $regex ${header_dir}/fadvise.h | \
sed -r "s/$regex/\2 \1/g" | \
sort | xargs printf "\t[%s] = \"%s\",\n" | \
grep -v "[6].*DONTNEED" | grep -v "[7].*NOREUSE"
printf "static const char *fsmount_attr_flags[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+MOUNT_ATTR_([[:alnum:]][[:alnum:]_]+)[[:space:]]+(0x[[:xdigit:]]+)[[:space:]]*.*'
-egrep $regex ${linux_mount} | grep -v MOUNT_ATTR_RELATIME | \
+grep -E $regex ${linux_mount} | grep -v MOUNT_ATTR_RELATIME | \
sed -r "s/$regex/\2 \1/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n"
printf "};\n"
printf "static const char *fspick_flags[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+FSPICK_([[:alnum:]_]+)[[:space:]]+(0x[[:xdigit:]]+)[[:space:]]*.*'
-egrep $regex ${linux_mount} | \
+grep -E $regex ${linux_mount} | \
sed -r "s/$regex/\2 \1/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n"
printf "};\n"
printf "static const char *kcmp_types[] = {\n"
regex='^[[:space:]]+(KCMP_(\w+)),'
-egrep $regex ${header_dir}/kcmp.h | grep -v KCMP_TYPES, | \
+grep -E $regex ${header_dir}/kcmp.h | grep -v KCMP_TYPES, | \
sed -r "s/$regex/\1 \2/g" | \
xargs printf "\t[%s]\t= \"%s\",\n"
printf "};\n"
printf "static const char *kvm_ioctl_cmds[] = {\n"
regex='^#[[:space:]]*define[[:space:]]+KVM_(\w+)[[:space:]]+_IO[RW]*\([[:space:]]*KVMIO[[:space:]]*,[[:space:]]*(0x[[:xdigit:]]+).*'
-egrep $regex ${header_dir}/kvm.h | \
+grep -E $regex ${header_dir}/kvm.h | \
sed -r "s/$regex/\2 \1/g" | \
- egrep -v " ((ARM|PPC|S390)_|[GS]ET_(DEBUGREGS|PIT2|XSAVE|TSC_KHZ)|CREATE_SPAPR_TCE_64)" | \
+ grep -E -v " ((ARM|PPC|S390)_|[GS]ET_(DEBUGREGS|PIT2|XSAVE|TSC_KHZ)|CREATE_SPAPR_TCE_64)" | \
sort | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n"
printf "static const char *madvise_advices[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+MADV_([[:alnum:]_]+)[[:space:]]+([[:digit:]]+)[[:space:]]*.*'
-egrep $regex ${header_dir}/mman-common.h | \
+grep -E $regex ${header_dir}/mman-common.h | \
sed -r "s/$regex/\2 \1/g" | \
sort -n | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n"
linux_mman=${linux_header_dir}/mman.h
arch_mman=${arch_header_dir}/mman.h
-# those in egrep -vw are flags, we want just the bits
+# those in grep -E -vw are flags, we want just the bits
printf "static const char *mmap_flags[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+MAP_([[:alnum:]_]+)[[:space:]]+(0x[[:xdigit:]]+)[[:space:]]*.*'
-egrep -q $regex ${arch_mman} && \
-(egrep $regex ${arch_mman} | \
+grep -E -q $regex ${arch_mman} && \
+(grep -E $regex ${arch_mman} | \
sed -r "s/$regex/\2 \1 \1 \1 \2/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n#ifndef MAP_%s\n#define MAP_%s %s\n#endif\n")
-egrep -q $regex ${linux_mman} && \
-(egrep $regex ${linux_mman} | \
- egrep -vw 'MAP_(UNINITIALIZED|TYPE|SHARED_VALIDATE)' | \
+grep -E -q $regex ${linux_mman} && \
+(grep -E $regex ${linux_mman} | \
+ grep -E -vw 'MAP_(UNINITIALIZED|TYPE|SHARED_VALIDATE)' | \
sed -r "s/$regex/\2 \1 \1 \1 \2/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n#ifndef MAP_%s\n#define MAP_%s %s\n#endif\n")
-([ ! -f ${arch_mman} ] || egrep -q '#[[:space:]]*include[[:space:]]+.*uapi/asm-generic/mman.*' ${arch_mman}) &&
-(egrep $regex ${header_dir}/mman-common.h | \
- egrep -vw 'MAP_(UNINITIALIZED|TYPE|SHARED_VALIDATE)' | \
+([ ! -f ${arch_mman} ] || grep -E -q '#[[:space:]]*include[[:space:]]+.*uapi/asm-generic/mman.*' ${arch_mman}) &&
+(grep -E $regex ${header_dir}/mman-common.h | \
+ grep -E -vw 'MAP_(UNINITIALIZED|TYPE|SHARED_VALIDATE)' | \
sed -r "s/$regex/\2 \1 \1 \1 \2/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n#ifndef MAP_%s\n#define MAP_%s %s\n#endif\n")
-([ ! -f ${arch_mman} ] || egrep -q '#[[:space:]]*include[[:space:]]+.*uapi/asm-generic/mman.h>.*' ${arch_mman}) &&
-(egrep $regex ${header_dir}/mman.h | \
+([ ! -f ${arch_mman} ] || grep -E -q '#[[:space:]]*include[[:space:]]+.*uapi/asm-generic/mman.h>.*' ${arch_mman}) &&
+(grep -E $regex ${header_dir}/mman.h | \
sed -r "s/$regex/\2 \1 \1 \1 \2/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n#ifndef MAP_%s\n#define MAP_%s %s\n#endif\n")
printf "};\n"
printf "static const char *mmap_prot[] = {\n"
regex=`printf '^[[:space:]]*#[[:space:]]*define[[:space:]]+%s_([[:alnum:]_]+)[[:space:]]+(0x[[:xdigit:]]+)[[:space:]]*.*' ${prefix}`
-([ ! -f ${arch_mman} ] || egrep -q '#[[:space:]]*include[[:space:]]+.*uapi/asm-generic/mman.*' ${arch_mman}) &&
-(egrep $regex ${common_mman} | \
- egrep -vw PROT_NONE | \
+([ ! -f ${arch_mman} ] || grep -E -q '#[[:space:]]*include[[:space:]]+.*uapi/asm-generic/mman.*' ${arch_mman}) &&
+(grep -E $regex ${common_mman} | \
+ grep -E -vw PROT_NONE | \
sed -r "s/$regex/\2 \1 \1 \1 \2/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n#ifndef ${prefix}_%s\n#define ${prefix}_%s %s\n#endif\n")
-[ -f ${arch_mman} ] && egrep -q $regex ${arch_mman} &&
-(egrep $regex ${arch_mman} | \
- egrep -vw PROT_NONE | \
+[ -f ${arch_mman} ] && grep -E -q $regex ${arch_mman} &&
+(grep -E $regex ${arch_mman} | \
+ grep -E -vw PROT_NONE | \
sed -r "s/$regex/\2 \1 \1 \1 \2/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n#ifndef ${prefix}_%s\n#define ${prefix}_%s %s\n#endif\n")
printf "};\n"
printf "static const char *mount_flags[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+MS_([[:alnum:]_]+)[[:space:]]+([[:digit:]]+)[[:space:]]*.*'
-egrep $regex ${header_dir}/mount.h | egrep -v '(MSK|VERBOSE|MGC_VAL)\>' | \
+grep -E $regex ${header_dir}/mount.h | grep -E -v '(MSK|VERBOSE|MGC_VAL)\>' | \
sed -r "s/$regex/\2 \2 \1/g" | sort -n | \
xargs printf "\t[%s ? (ilog2(%s) + 1) : 0] = \"%s\",\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+MS_([[:alnum:]_]+)[[:space:]]+\(1<<([[:digit:]]+)\)[[:space:]]*.*'
-egrep $regex ${header_dir}/mount.h | \
+grep -E $regex ${header_dir}/mount.h | \
sed -r "s/$regex/\2 \1/g" | \
xargs printf "\t[%s + 1] = \"%s\",\n"
printf "};\n"
printf "static const char *move_mount_flags[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+MOVE_MOUNT_([^_]+_[[:alnum:]_]+)[[:space:]]+(0x[[:xdigit:]]+)[[:space:]]*.*'
-egrep $regex ${linux_mount} | \
+grep -E $regex ${linux_mount} | \
sed -r "s/$regex/\2 \1/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n"
printf "};\n"
printf "static const char *mremap_flags[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+MREMAP_([[:alnum:]_]+)[[:space:]]+((0x)?[[:xdigit:]]+)[[:space:]]*.*'
-egrep -q $regex ${linux_mman} && \
-(egrep $regex ${linux_mman} | \
+grep -E -q $regex ${linux_mman} && \
+(grep -E $regex ${linux_mman} | \
sed -r "s/$regex/\2 \1 \1 \1 \2/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n#ifndef MREMAP_%s\n#define MREMAP_%s %s\n#endif\n")
printf "};\n"
}
#define SCA_PERF_FLAGS syscall_arg__scnprintf_perf_flags
+
+struct attr_fprintf_args {
+ size_t size, printed;
+ char *bf;
+ bool first;
+};
+
+static int attr__fprintf(FILE *fp __maybe_unused, const char *name, const char *val, void *priv)
+{
+ struct attr_fprintf_args *args = priv;
+ size_t printed = scnprintf(args->bf + args->printed , args->size - args->printed, "%s%s: %s", args->first ? "" : ", ", name, val);
+
+ args->first = false;
+ args->printed += printed;
+ return printed;
+}
+
+static size_t perf_event_attr___scnprintf(struct perf_event_attr *attr, char *bf, size_t size, bool show_zeros __maybe_unused)
+{
+ struct attr_fprintf_args args = {
+ .printed = scnprintf(bf, size, "{ "),
+ .size = size,
+ .first = true,
+ .bf = bf,
+ };
+
+ perf_event_attr__fprintf(stdout, attr, attr__fprintf, &args);
+ return args.printed + scnprintf(bf + args.printed, size - args.printed, " }");
+}
+
+static size_t syscall_arg__scnprintf_augmented_perf_event_attr(struct syscall_arg *arg, char *bf, size_t size)
+{
+ return perf_event_attr___scnprintf((void *)arg->augmented.args, bf, size, arg->trace->show_zeros);
+}
+
+static size_t syscall_arg__scnprintf_perf_event_attr(char *bf, size_t size, struct syscall_arg *arg)
+{
+ if (arg->augmented.args)
+ return syscall_arg__scnprintf_augmented_perf_event_attr(arg, bf, size);
+
+ return scnprintf(bf, size, "%#lx", arg->val);
+}
+
+#define SCA_PERF_ATTR syscall_arg__scnprintf_perf_event_attr
printf "static const char *perf_ioctl_cmds[] = {\n"
regex='^#[[:space:]]*define[[:space:]]+PERF_EVENT_IOC_(\w+)[[:space:]]+_IO[RW]*[[:space:]]*\([[:space:]]*.\$.[[:space:]]*,[[:space:]]*([[:digit:]]+).*'
-egrep $regex ${header_dir}/perf_event.h | \
+grep -E $regex ${header_dir}/perf_event.h | \
sed -r "s/$regex/\2 \1/g" | \
sort | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n"
printf "static const char *pkey_alloc_access_rights[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+PKEY_([[:alnum:]_]+)[[:space:]]+(0x[[:xdigit:]]+)[[:space:]]*'
-egrep $regex ${header_dir}/mman-common.h | \
+grep -E $regex ${header_dir}/mman-common.h | \
sed -r "s/$regex/\2 \2 \1/g" | \
sort | xargs printf "\t[%s ? (ilog2(%s) + 1) : 0] = \"%s\",\n"
printf "};\n"
printf "static const char *prctl_options[] = {\n"
regex='^#define[[:space:]]{1}PR_(\w+)[[:space:]]*([[:xdigit:]]+)([[:space:]]*\/.*)?$'
-egrep $regex ${header_dir}/prctl.h | grep -v PR_SET_PTRACER | \
+grep -E $regex ${header_dir}/prctl.h | grep -v PR_SET_PTRACER | \
sed -r "s/$regex/\2 \1/g" | \
sort -n | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n"
printf "static const char *prctl_set_mm_options[] = {\n"
regex='^#[[:space:]]+define[[:space:]]+PR_SET_MM_(\w+)[[:space:]]*([[:digit:]]+).*'
-egrep $regex ${header_dir}/prctl.h | \
+grep -E $regex ${header_dir}/prctl.h | \
sed -r "s/$regex/\2 \1/g" | \
sort -n | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n"
printf "static const char *rename_flags[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+RENAME_([[:alnum:]_]+)[[:space:]]+\(1[[:space:]]*<<[[:space:]]*([[:xdigit:]]+)[[:space:]]*\)[[:space:]]*.*'
-egrep -q $regex ${fs_header} && \
-(egrep $regex ${fs_header} | \
+grep -E -q $regex ${fs_header} && \
+(grep -E $regex ${fs_header} | \
sed -r "s/$regex/\2 \1/g" | \
xargs printf "\t[%d + 1] = \"%s\",\n")
printf "};\n"
# #define AF_LOCAL 1 /* POSIX name for AF_UNIX */
regex='^#define[[:space:]]+AF_(\w+)[[:space:]]+([[:digit:]]+).*'
-egrep $regex ${header_dir}/socket.h | \
+grep -E $regex ${header_dir}/socket.h | \
sed -r "s/$regex/\2 \1/g" | \
xargs printf "\t[%s] = \"%s\",\n" | \
- egrep -v "\"(UNIX|MAX)\""
+ grep -E -v "\"(UNIX|MAX)\""
printf "};\n"
printf "static const char *socket_ipproto[] = {\n"
ipproto_regex='^[[:space:]]+IPPROTO_(\w+)[[:space:]]+=[[:space:]]+([[:digit:]]+),.*'
-egrep $ipproto_regex ${uapi_header_dir}/in.h | \
+grep -E $ipproto_regex ${uapi_header_dir}/in.h | \
sed -r "s/$ipproto_regex/\2 \1/g" | \
sort -n | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n\n"
printf "static const char *socket_level[] = {\n"
socket_level_regex='^#define[[:space:]]+SOL_(\w+)[[:space:]]+([[:digit:]]+)([[:space:]]+\/.*)?'
-egrep $socket_level_regex ${beauty_header_dir}/socket.h | \
+grep -E $socket_level_regex ${beauty_header_dir}/socket.h | \
sed -r "s/$socket_level_regex/\2 \1/g" | \
sort -n | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n\n"
printf "static const char *sync_file_range_flags[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+SYNC_FILE_RANGE_([[:alnum:]_]+)[[:space:]]+([[:xdigit:]]+)[[:space:]]*.*'
-egrep $regex ${linux_fs} | \
+grep -E $regex ${linux_fs} | \
sed -r "s/$regex/\2 \1/g" | \
xargs printf "\t[ilog2(%s) + 1] = \"%s\",\n"
printf "};\n"
--- /dev/null
+// SPDX-License-Identifier: LGPL-2.1
+// Copyright (C) 2022, Red Hat Inc, Arnaldo Carvalho de Melo <acme@redhat.com>
+
+#include "trace/beauty/beauty.h"
+#include <inttypes.h>
+#include <time.h>
+
+static size_t syscall_arg__scnprintf_augmented_timespec(struct syscall_arg *arg, char *bf, size_t size)
+{
+ struct timespec *ts = (struct timespec *)arg->augmented.args;
+
+ return scnprintf(bf, size, "{ .tv_sec: %" PRIu64 ", .tv_nsec: %" PRIu64 " }", ts->tv_sec, ts->tv_nsec);
+}
+
+size_t syscall_arg__scnprintf_timespec(char *bf, size_t size, struct syscall_arg *arg)
+{
+ if (arg->augmented.args)
+ return syscall_arg__scnprintf_augmented_timespec(arg, bf, size);
+
+ return scnprintf(bf, size, "%#lx", arg->val);
+}
# the time of writing of this script was: IRQ_MOVE_CLEANUP_VECTOR.
first_external_regex='^#define[[:space:]]+FIRST_EXTERNAL_VECTOR[[:space:]]+(0x[[:xdigit:]]+)$'
-first_external_vector=$(egrep ${first_external_regex} ${x86_irq_vectors} | sed -r "s/${first_external_regex}/\1/g")
+first_external_vector=$(grep -E ${first_external_regex} ${x86_irq_vectors} | sed -r "s/${first_external_regex}/\1/g")
printf "static const char *x86_irq_vectors[] = {\n"
regex='^#define[[:space:]]+([[:alnum:]_]+)_VECTOR[[:space:]]+(0x[[:xdigit:]]+)$'
sed -r "s/FIRST_EXTERNAL_VECTOR/${first_external_vector}/g" ${x86_irq_vectors} | \
-egrep ${regex} | \
+grep -E ${regex} | \
sed -r "s/${regex}/\2 \1/g" | sort -n | \
xargs printf "\t[%s] = \"%s\",\n"
printf "};\n\n"
printf "static const char *x86_MSRs[] = {\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+MSR_([[:alnum:]][[:alnum:]_]+)[[:space:]]+(0x00000[[:xdigit:]]+)[[:space:]]*.*'
-egrep $regex ${x86_msr_index} | egrep -v 'MSR_(ATOM|P[46]|IA32_(TSC_DEADLINE|UCODE_REV)|IDT_FCR4)' | \
+grep -E $regex ${x86_msr_index} | grep -E -v 'MSR_(ATOM|P[46]|IA32_(TSC_DEADLINE|UCODE_REV)|IDT_FCR4)' | \
sed -r "s/$regex/\2 \1/g" | sort -n | \
xargs printf "\t[%s] = \"%s\",\n"
printf "};\n\n"
# Remove MSR_K6_WHCR, clashes with MSR_LSTAR
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+MSR_([[:alnum:]][[:alnum:]_]+)[[:space:]]+(0xc0000[[:xdigit:]]+)[[:space:]]*.*'
printf "#define x86_64_specific_MSRs_offset "
-egrep $regex ${x86_msr_index} | sed -r "s/$regex/\2/g" | sort -n | head -1
+grep -E $regex ${x86_msr_index} | sed -r "s/$regex/\2/g" | sort -n | head -1
printf "static const char *x86_64_specific_MSRs[] = {\n"
-egrep $regex ${x86_msr_index} | \
- sed -r "s/$regex/\2 \1/g" | egrep -vw 'K6_WHCR' | sort -n | \
+grep -E $regex ${x86_msr_index} | \
+ sed -r "s/$regex/\2 \1/g" | grep -E -vw 'K6_WHCR' | sort -n | \
xargs printf "\t[%s - x86_64_specific_MSRs_offset] = \"%s\",\n"
printf "};\n\n"
regex='^[[:space:]]*#[[:space:]]*define[[:space:]]+MSR_([[:alnum:]][[:alnum:]_]+)[[:space:]]+(0xc0010[[:xdigit:]]+)[[:space:]]*.*'
printf "#define x86_AMD_V_KVM_MSRs_offset "
-egrep $regex ${x86_msr_index} | sed -r "s/$regex/\2/g" | sort -n | head -1
+grep -E $regex ${x86_msr_index} | sed -r "s/$regex/\2/g" | sort -n | head -1
printf "static const char *x86_AMD_V_KVM_MSRs[] = {\n"
-egrep $regex ${x86_msr_index} | \
+grep -E $regex ${x86_msr_index} | \
sed -r "s/$regex/\2 \1/g" | sort -n | \
xargs printf "\t[%s - x86_AMD_V_KVM_MSRs_offset] = \"%s\",\n"
printf "};\n"
printf "static const char *usbdevfs_ioctl_cmds[] = {\n"
regex="^#[[:space:]]*define[[:space:]]+USBDEVFS_(\w+)(\(\w+\))?[[:space:]]+_IO[CWR]{0,2}\([[:space:]]*(_IOC_\w+,[[:space:]]*)?'U'[[:space:]]*,[[:space:]]*([[:digit:]]+).*"
-egrep "$regex" ${header_dir}/usbdevice_fs.h | egrep -v 'USBDEVFS_\w+32[[:space:]]' | \
+grep -E "$regex" ${header_dir}/usbdevice_fs.h | grep -E -v 'USBDEVFS_\w+32[[:space:]]' | \
sed -r "s/$regex/\4 \1/g" | \
sort | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n\n"
printf "#if 0\n"
printf "static const char *usbdevfs_ioctl_32_cmds[] = {\n"
regex="^#[[:space:]]*define[[:space:]]+USBDEVFS_(\w+)[[:space:]]+_IO[WR]{0,2}\([[:space:]]*'U'[[:space:]]*,[[:space:]]*([[:digit:]]+).*"
-egrep $regex ${header_dir}/usbdevice_fs.h | egrep 'USBDEVFS_\w+32[[:space:]]' | \
+grep -E $regex ${header_dir}/usbdevice_fs.h | grep -E 'USBDEVFS_\w+32[[:space:]]' | \
sed -r "s/$regex/\2 \1/g" | \
sort | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n"
printf "static const char *vhost_virtio_ioctl_cmds[] = {\n"
regex='^#[[:space:]]*define[[:space:]]+VHOST_(\w+)[[:space:]]+_IOW?\([[:space:]]*VHOST_VIRTIO[[:space:]]*,[[:space:]]*(0x[[:xdigit:]]+).*'
-egrep $regex ${header_dir}/vhost.h | \
+grep -E $regex ${header_dir}/vhost.h | \
sed -r "s/$regex/\2 \1/g" | \
sort | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n"
printf "static const char *vhost_virtio_ioctl_read_cmds[] = {\n"
regex='^#[[:space:]]*define[[:space:]]+VHOST_(\w+)[[:space:]]+_IOW?R\([[:space:]]*VHOST_VIRTIO[[:space:]]*,[[:space:]]*(0x[[:xdigit:]]+).*'
-egrep $regex ${header_dir}/vhost.h | \
+grep -E $regex ${header_dir}/vhost.h | \
sed -r "s/$regex/\2 \1/g" | \
sort | xargs printf "\t[%s] = \"%s\",\n"
printf "};\n"
printf "static const char *x86_arch_prctl_codes_%d[] = {\n" $idx
regex=`printf '^[[:space:]]*#[[:space:]]*define[[:space:]]+ARCH_([[:alnum:]_]+)[[:space:]]+(%s[[:xdigit:]]+).*' ${prefix}`
fmt="\t[%#x - ${first_entry}]= \"%s\",\n"
- egrep -q $regex ${prctl_arch_header} && \
- (egrep $regex ${prctl_arch_header} | \
+ grep -E -q $regex ${prctl_arch_header} && \
+ (grep -E $regex ${prctl_arch_header} | \
sed -r "s/$regex/\2 \1/g" | \
xargs printf "$fmt")
printf "};\n\n"
static int perf_stdio__warning(const char *format, va_list args)
{
+ if (quiet)
+ return 0;
+
fprintf(stderr, "Warning:\n");
vfprintf(stderr, format, args);
return 0;
{
int ret;
va_list args;
+ if (quiet)
+ return 0;
va_start(args, format);
ret = perf_eops->warning(format, args);
perf-y += evswitch.o
perf-y += find_bit.o
perf-y += get_current_dir_name.o
-perf-y += kallsyms.o
perf-y += levenshtein.o
perf-y += llvm-utils.o
perf-y += mmap.o
perf-y += comm.o
perf-y += thread.o
perf-y += thread_map.o
-perf-y += trace-event-parse.o
perf-y += parse-events-flex.o
perf-y += parse-events-bison.o
perf-y += pmu.o
+perf-y += pmus.o
perf-y += pmu-flex.o
perf-y += pmu-bison.o
perf-y += pmu-hybrid.o
-perf-y += trace-event-read.o
-perf-y += trace-event-info.o
-perf-y += trace-event-scripting.o
-perf-y += trace-event.o
perf-y += svghelper.o
+perf-$(CONFIG_LIBTRACEEVENT) += trace-event-info.o
+perf-$(CONFIG_LIBTRACEEVENT) += trace-event-scripting.o
+perf-$(CONFIG_LIBTRACEEVENT) += trace-event.o
+perf-$(CONFIG_LIBTRACEEVENT) += trace-event-parse.o
+perf-$(CONFIG_LIBTRACEEVENT) += trace-event-read.o
perf-y += sort.o
perf-y += hist.o
perf-y += util.o
perf-$(CONFIG_AUXTRACE) += cs-etm.o
perf-$(CONFIG_AUXTRACE) += cs-etm-decoder/
endif
+perf-$(CONFIG_AUXTRACE) += cs-etm-base.o
perf-y += parse-branch-options.o
perf-y += dump-insn.o
perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter_cgroup.o
perf-$(CONFIG_PERF_BPF_SKEL) += bpf_ftrace.o
perf-$(CONFIG_PERF_BPF_SKEL) += bpf_off_cpu.o
-perf-$(CONFIG_PERF_BPF_SKEL) += bpf_kwork.o
perf-$(CONFIG_PERF_BPF_SKEL) += bpf_lock_contention.o
+
+ifeq ($(CONFIG_LIBTRACEEVENT),y)
+ perf-$(CONFIG_PERF_BPF_SKEL) += bpf_kwork.o
+endif
+
perf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o
perf-$(CONFIG_LIBELF) += symbol-elf.o
perf-$(CONFIG_LIBELF) += probe-file.o
perf-$(CONFIG_LIBUNWIND_X86) += libunwind/x86_32.o
perf-$(CONFIG_LIBUNWIND_AARCH64) += libunwind/arm64.o
-perf-$(CONFIG_LIBBABELTRACE) += data-convert-bt.o
+ifeq ($(CONFIG_LIBTRACEEVENT),y)
+ perf-$(CONFIG_LIBBABELTRACE) += data-convert-bt.o
+endif
+
perf-y += data-convert-json.o
perf-y += scripting-engines/
perf-$(CONFIG_LIBPFM4) += pfm.o
CFLAGS_config.o += -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))"
-CFLAGS_llvm-utils.o += -DPERF_INCLUDE_DIR="BUILD_STR($(perf_include_dir_SQ))"
+CFLAGS_llvm-utils.o += -DLIBBPF_INCLUDE_DIR="BUILD_STR($(libbpf_include_dir_SQ))"
# avoid compiler warnings in 32-bit mode
CFLAGS_genelf_debug.o += -Wno-packed
CFLAGS_header.o += -include $(OUTPUT)PERF-VERSION-FILE
CFLAGS_arm-spe.o += -I$(srctree)/tools/arch/arm64/include/
-$(OUTPUT)util/kallsyms.o: ../lib/symbol/kallsyms.c FORCE
- $(call rule_mkdir)
- $(call if_changed_dep,cc_o_c)
-
$(OUTPUT)util/argv_split.o: ../lib/argv_split.c FORCE
$(call rule_mkdir)
$(call if_changed_dep,cc_o_c)
#include "evlist.h"
#include "sample-raw.h"
#include "pmu-events/pmu-events.h"
+#include "util/sample.h"
static u32 cpu_family, cpu_model, ibs_fetch_type, ibs_op_type;
static bool zen4_ibs_extensions;
#ifndef __PERF_ARM_FRAME_POINTER_UNWIND_SUPPORT_H
#define __PERF_ARM_FRAME_POINTER_UNWIND_SUPPORT_H
-#include "event.h"
-#include "thread.h"
+#include <linux/types.h>
+
+struct perf_sample;
+struct thread;
u64 get_leaf_frame_caller_aarch64(struct perf_sample *sample, struct thread *thread, int user_idx);
#include <linux/ctype.h>
#include "symbol/kallsyms.h"
#include <internal/lib.h>
+#include "util/sample.h"
/*
* Make a group from 'leader' to 'last', requiring that the events were not
#include <linux/list.h>
#include <linux/perf_event.h>
#include <linux/types.h>
-#include <internal/cpumap.h>
+#include <perf/cpumap.h>
#include <asm/bitsperlong.h>
#include <asm/barrier.h>
#include "util.h"
#include "llvm-utils.h"
#include "c++/clang-c.h"
-#ifdef HAVE_LIBBPF_SUPPORT
-#include <bpf/hashmap.h>
-#else
#include "util/hashmap.h"
-#endif
#include "asm/bug.h"
#include <internal/xyarray.h>
#ifndef __BPF_PROLOGUE_H
#define __BPF_PROLOGUE_H
-#include <linux/compiler.h>
-#include <linux/filter.h>
-#include "probe-event.h"
+struct probe_trace_arg;
+struct bpf_insn;
#define BPF_PROLOGUE_MAX_ARGS 3
#define BPF_PROLOGUE_START_ARG_REG BPF_REG_3
struct bpf_insn *new_prog, size_t *new_cnt,
size_t cnt_space);
#else
+#include <linux/compiler.h>
#include <errno.h>
static inline int
if (filter_type == BPERF_FILTER_PID ||
filter_type == BPERF_FILTER_TGID)
- key = evsel->core.threads->map[i].pid;
+ key = perf_thread_map__pid(evsel->core.threads, i);
else if (filter_type == BPERF_FILTER_CPU)
key = evsel->core.cpus->map[i].cpu;
else
#include <time.h>
#include <fcntl.h>
+#include <signal.h>
#include <stdio.h>
#include <unistd.h>
#include <linux/time64.h>
#include "util/debug.h"
+#include "util/evsel.h"
#include "util/kwork.h"
#include <bpf/bpf.h>
+#include <perf/cpumap.h>
#include "util/bpf_skel/kwork_trace.skel.h"
#include "util/map.h"
#include "util/symbol.h"
#include "util/target.h"
+#include "util/thread.h"
#include "util/thread_map.h"
#include "util/lock-contention.h"
#include <linux/zalloc.h>
#include <bpf/bpf.h>
#include "bpf_skel/lock_contention.skel.h"
+#include "bpf_skel/lock_data.h"
static struct lock_contention_bpf *skel;
-struct lock_contention_data {
- u64 total_time;
- u64 min_time;
- u64 max_time;
- u32 count;
- u32 flags;
-};
-
int lock_contention_prepare(struct lock_contention *con)
{
int i, fd;
}
bpf_map__set_value_size(skel->maps.stacks, con->max_stack * sizeof(u64));
- bpf_map__set_max_entries(skel->maps.stacks, con->map_nr_entries);
bpf_map__set_max_entries(skel->maps.lock_stat, con->map_nr_entries);
+ bpf_map__set_max_entries(skel->maps.tstamp, con->map_nr_entries);
+
+ if (con->aggr_mode == LOCK_AGGR_TASK) {
+ bpf_map__set_max_entries(skel->maps.task_data, con->map_nr_entries);
+ bpf_map__set_max_entries(skel->maps.stacks, 1);
+ } else {
+ bpf_map__set_max_entries(skel->maps.task_data, 1);
+ bpf_map__set_max_entries(skel->maps.stacks, con->map_nr_entries);
+ }
if (target__has_cpu(target))
ncpus = perf_cpu_map__nr(evlist->core.user_requested_cpus);
bpf_map_update_elem(fd, &pid, &val, BPF_ANY);
}
+ /* these don't work well if in the rodata section */
skel->bss->stack_skip = con->stack_skip;
+ skel->bss->aggr_mode = con->aggr_mode;
lock_contention_bpf__attach(skel);
return 0;
int lock_contention_read(struct lock_contention *con)
{
- int fd, stack;
- s32 prev_key, key;
- struct lock_contention_data data;
- struct lock_stat *st;
+ int fd, stack, task_fd, err = 0;
+ struct contention_key *prev_key, key;
+ struct contention_data data = {};
+ struct lock_stat *st = NULL;
struct machine *machine = con->machine;
- u64 stack_trace[con->max_stack];
+ u64 *stack_trace;
+ size_t stack_size = con->max_stack * sizeof(*stack_trace);
fd = bpf_map__fd(skel->maps.lock_stat);
stack = bpf_map__fd(skel->maps.stacks);
+ task_fd = bpf_map__fd(skel->maps.task_data);
con->lost = skel->bss->lost;
- prev_key = 0;
- while (!bpf_map_get_next_key(fd, &prev_key, &key)) {
+ stack_trace = zalloc(stack_size);
+ if (stack_trace == NULL)
+ return -1;
+
+ if (con->aggr_mode == LOCK_AGGR_TASK) {
+ struct thread *idle = __machine__findnew_thread(machine,
+ /*pid=*/0,
+ /*tid=*/0);
+ thread__set_comm(idle, "swapper", /*timestamp=*/0);
+ }
+
+ /* make sure it loads the kernel map */
+ map__load(maps__first(machine->kmaps));
+
+ prev_key = NULL;
+ while (!bpf_map_get_next_key(fd, prev_key, &key)) {
struct map *kmap;
struct symbol *sym;
int idx = 0;
+ s32 stack_id;
+
+ /* to handle errors in the loop body */
+ err = -1;
bpf_map_lookup_elem(fd, &key, &data);
st = zalloc(sizeof(*st));
if (st == NULL)
- return -1;
+ break;
st->nr_contended = data.count;
st->wait_time_total = data.total_time;
st->avg_wait_time = data.total_time / data.count;
st->flags = data.flags;
+ st->addr = key.aggr_key;
+
+ if (con->aggr_mode == LOCK_AGGR_TASK) {
+ struct contention_task_data task;
+ struct thread *t;
+ int pid = key.aggr_key;
+
+ /* do not update idle comm which contains CPU number */
+ if (st->addr) {
+ bpf_map_lookup_elem(task_fd, &pid, &task);
+ t = __machine__findnew_thread(machine, /*pid=*/-1, pid);
+ thread__set_comm(t, task.comm, /*timestamp=*/0);
+ }
+ goto next;
+ }
- bpf_map_lookup_elem(stack, &key, stack_trace);
+ if (con->aggr_mode == LOCK_AGGR_ADDR) {
+ sym = machine__find_kernel_symbol(machine, st->addr, &kmap);
+ if (sym)
+ st->name = strdup(sym->name);
+ goto next;
+ }
+
+ stack_id = key.aggr_key;
+ bpf_map_lookup_elem(stack, &stack_id, stack_trace);
/* skip lock internal functions */
- while (is_lock_function(machine, stack_trace[idx]) &&
+ while (machine__is_lock_function(machine, stack_trace[idx]) &&
idx < con->max_stack - 1)
idx++;
st->name = strdup(sym->name);
if (ret < 0 || st->name == NULL)
- return -1;
+ break;
} else if (asprintf(&st->name, "%#lx", (unsigned long)st->addr) < 0) {
- free(st);
- return -1;
+ break;
}
if (verbose) {
- st->callstack = memdup(stack_trace, sizeof(stack_trace));
- if (st->callstack == NULL) {
- free(st);
- return -1;
- }
+ st->callstack = memdup(stack_trace, stack_size);
+ if (st->callstack == NULL)
+ break;
}
-
+next:
hlist_add_head(&st->hash_entry, con->result);
- prev_key = key;
+ prev_key = &key;
+
+ /* we're fine now, reset the values */
+ st = NULL;
+ err = 0;
}
- return 0;
+ free(stack_trace);
+ if (st) {
+ free(st->name);
+ free(st);
+ }
+
+ return err;
}
int lock_contention_finish(void)
#define __PERF_BPF_MAP_H 1
#include <stdio.h>
-#include <linux/compiler.h>
struct bpf_map;
#ifdef HAVE_LIBBPF_SUPPORT
#else
+#include <linux/compiler.h>
+
static inline int bpf_map__fprintf(struct bpf_map *map __maybe_unused, FILE *fp __maybe_unused)
{
return 0;
const struct btf_type *t1, *t2, *t3;
u32 type_id;
- type_id = btf__find_by_name_kind(btf, "bpf_trace_sched_switch",
+ type_id = btf__find_by_name_kind(btf, "btf_trace_sched_switch",
BTF_KIND_TYPEDEF);
if ((s32)type_id < 0)
return;
#include <bpf/bpf_tracing.h>
#include <bpf/bpf_core_read.h>
-/* maximum stack trace depth */
-#define MAX_STACKS 8
+#include "lock_data.h"
/* default buffer size */
#define MAX_ENTRIES 10240
-struct contention_key {
- __s32 stack_id;
-};
-
-struct contention_data {
- __u64 total_time;
- __u64 min_time;
- __u64 max_time;
- __u32 count;
- __u32 flags;
-};
-
struct tstamp_data {
__u64 timestamp;
__u64 lock;
struct {
__uint(type, BPF_MAP_TYPE_STACK_TRACE);
__uint(key_size, sizeof(__u32));
- __uint(value_size, MAX_STACKS * sizeof(__u64));
+ __uint(value_size, sizeof(__u64));
__uint(max_entries, MAX_ENTRIES);
} stacks SEC(".maps");
/* maintain timestamp at the beginning of contention */
struct {
- __uint(type, BPF_MAP_TYPE_TASK_STORAGE);
- __uint(map_flags, BPF_F_NO_PREALLOC);
+ __uint(type, BPF_MAP_TYPE_HASH);
__type(key, int);
__type(value, struct tstamp_data);
+ __uint(max_entries, MAX_ENTRIES);
} tstamp SEC(".maps");
/* actual lock contention statistics */
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__uint(key_size, sizeof(__u32));
+ __uint(value_size, sizeof(struct contention_task_data));
+ __uint(max_entries, MAX_ENTRIES);
+} task_data SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(key_size, sizeof(__u32));
__uint(value_size, sizeof(__u8));
__uint(max_entries, 1);
} cpu_filter SEC(".maps");
int has_task;
int stack_skip;
+/* determine the key of lock stat */
+int aggr_mode;
+
/* error stat */
int lost;
return 1;
}
+static inline void update_task_data(__u32 pid)
+{
+ struct contention_task_data *p;
+
+ p = bpf_map_lookup_elem(&task_data, &pid);
+ if (p == NULL) {
+ struct contention_task_data data;
+
+ bpf_get_current_comm(data.comm, sizeof(data.comm));
+ bpf_map_update_elem(&task_data, &pid, &data, BPF_NOEXIST);
+ }
+}
+
SEC("tp_btf/contention_begin")
int contention_begin(u64 *ctx)
{
- struct task_struct *curr;
+ __u32 pid;
struct tstamp_data *pelem;
if (!enabled || !can_record())
return 0;
- curr = bpf_get_current_task_btf();
- pelem = bpf_task_storage_get(&tstamp, curr, NULL,
- BPF_LOCAL_STORAGE_GET_F_CREATE);
- if (!pelem || pelem->lock)
+ pid = bpf_get_current_pid_tgid();
+ pelem = bpf_map_lookup_elem(&tstamp, &pid);
+ if (pelem && pelem->lock)
return 0;
+ if (pelem == NULL) {
+ struct tstamp_data zero = {};
+
+ bpf_map_update_elem(&tstamp, &pid, &zero, BPF_ANY);
+ pelem = bpf_map_lookup_elem(&tstamp, &pid);
+ if (pelem == NULL) {
+ lost++;
+ return 0;
+ }
+ }
+
pelem->timestamp = bpf_ktime_get_ns();
pelem->lock = (__u64)ctx[0];
pelem->flags = (__u32)ctx[1];
- pelem->stack_id = bpf_get_stackid(ctx, &stacks, BPF_F_FAST_STACK_CMP | stack_skip);
- if (pelem->stack_id < 0)
- lost++;
+ if (aggr_mode == LOCK_AGGR_CALLER) {
+ pelem->stack_id = bpf_get_stackid(ctx, &stacks,
+ BPF_F_FAST_STACK_CMP | stack_skip);
+ if (pelem->stack_id < 0)
+ lost++;
+ }
+
return 0;
}
SEC("tp_btf/contention_end")
int contention_end(u64 *ctx)
{
- struct task_struct *curr;
+ __u32 pid;
struct tstamp_data *pelem;
struct contention_key key;
struct contention_data *data;
if (!enabled)
return 0;
- curr = bpf_get_current_task_btf();
- pelem = bpf_task_storage_get(&tstamp, curr, NULL, 0);
+ pid = bpf_get_current_pid_tgid();
+ pelem = bpf_map_lookup_elem(&tstamp, &pid);
if (!pelem || pelem->lock != ctx[0])
return 0;
duration = bpf_ktime_get_ns() - pelem->timestamp;
- key.stack_id = pelem->stack_id;
+ switch (aggr_mode) {
+ case LOCK_AGGR_CALLER:
+ key.aggr_key = pelem->stack_id;
+ break;
+ case LOCK_AGGR_TASK:
+ key.aggr_key = pid;
+ update_task_data(pid);
+ break;
+ case LOCK_AGGR_ADDR:
+ key.aggr_key = pelem->lock;
+ break;
+ default:
+ /* should not happen */
+ return 0;
+ }
+
data = bpf_map_lookup_elem(&lock_stat, &key);
if (!data) {
struct contention_data first = {
};
bpf_map_update_elem(&lock_stat, &key, &first, BPF_NOEXIST);
- pelem->lock = 0;
+ bpf_map_delete_elem(&tstamp, &pid);
return 0;
}
if (data->min_time > duration)
data->min_time = duration;
- pelem->lock = 0;
+ bpf_map_delete_elem(&tstamp, &pid);
return 0;
}
--- /dev/null
+// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+/* Data structures shared between BPF and tools. */
+#ifndef UTIL_BPF_SKEL_LOCK_DATA_H
+#define UTIL_BPF_SKEL_LOCK_DATA_H
+
+struct contention_key {
+ u64 aggr_key; /* can be stack_id, pid or lock addr */
+};
+
+#define TASK_COMM_LEN 16
+
+struct contention_task_data {
+ char comm[TASK_COMM_LEN];
+};
+
+struct contention_data {
+ u64 total_time;
+ u64 min_time;
+ u64 max_time;
+ u32 count;
+ u32 flags;
+};
+
+enum lock_aggr_mode {
+ LOCK_AGGR_ADDR = 0,
+ LOCK_AGGR_TASK,
+ LOCK_AGGR_CALLER,
+};
+
+#endif /* UTIL_BPF_SKEL_LOCK_DATA_H */
* detected in at least musl libc, used in Alpine Linux. -acme
*/
#include <stdio.h>
-#include <stdint.h>
-#include <linux/compiler.h>
-#include <linux/stddef.h>
#include <linux/perf_event.h>
#include <linux/types.h>
-#include "event.h"
+#include "util/map_symbol.h"
+#include "util/sample.h"
struct branch_flags {
union {
u64 abort:1;
u64 cycles:16;
u64 type:4;
+ u64 spec:2;
u64 new_type:4;
u64 priv:3;
- u64 reserved:33;
+ u64 reserved:31;
};
};
};
config = strdup(mkpath("%s/.perfconfig", home));
if (config == NULL) {
- pr_warning("Not enough memory to process %s/.perfconfig, ignoring it.", home);
+ pr_warning("Not enough memory to process %s/.perfconfig, ignoring it.\n", home);
return NULL;
}
goto out_free;
if (st.st_uid && (st.st_uid != geteuid())) {
- pr_warning("File %s not owned by current user or root, ignoring it.", config);
+ pr_warning("File %s not owned by current user or root, ignoring it.\n", config);
goto out_free;
}
{
xyarray__reset(counts->loaded);
xyarray__reset(counts->values);
- memset(&counts->aggr, 0, sizeof(struct perf_counts_values));
}
void evsel__reset_counts(struct evsel *evsel)
struct perf_counts {
s8 scaled;
- struct perf_counts_values aggr;
struct xyarray *values;
struct xyarray *loaded;
};
#include <linux/ctype.h>
#include <linux/zalloc.h>
+#include <internal/cpumap.h>
static struct perf_cpu max_cpu_num;
static struct perf_cpu max_present_cpu_num;
struct cpu_aggr_map *cpu_aggr_map__new(const struct perf_cpu_map *cpus,
aggr_cpu_id_get_t get_id,
- void *data)
+ void *data, bool needs_sort)
{
int idx;
struct perf_cpu cpu;
if (trimmed_c)
c = trimmed_c;
}
+
/* ensure we process id in increasing order */
- qsort(c->map, c->nr, sizeof(struct aggr_cpu_id), aggr_cpu_id__cmp);
+ if (needs_sort)
+ qsort(c->map, c->nr, sizeof(struct aggr_cpu_id), aggr_cpu_id__cmp);
return c;
return id;
}
+struct aggr_cpu_id aggr_cpu_id__global(struct perf_cpu cpu, void *data __maybe_unused)
+{
+ struct aggr_cpu_id id = aggr_cpu_id__empty();
+
+ /* it always aggregates to the cpu 0 */
+ cpu.cpu = 0;
+ id.cpu = cpu;
+ return id;
+}
+
/* setup simple routines to easily access node numbers given a cpu number */
static int get_max_num(char *path, int *max)
{
#include <stdbool.h>
#include <stdio.h>
-#include <internal/cpumap.h>
#include <perf/cpumap.h>
+#include <linux/refcount.h>
/** Identify where counts are aggregated, -1 implies not to aggregate. */
struct aggr_cpu_id {
*/
struct cpu_aggr_map *cpu_aggr_map__new(const struct perf_cpu_map *cpus,
aggr_cpu_id_get_t get_id,
- void *data);
+ void *data, bool needs_sort);
bool aggr_cpu_id__equal(const struct aggr_cpu_id *a, const struct aggr_cpu_id *b);
bool aggr_cpu_id__is_empty(const struct aggr_cpu_id *a);
* cpu. The function signature is compatible with aggr_cpu_id_get_t.
*/
struct aggr_cpu_id aggr_cpu_id__node(struct perf_cpu cpu, void *data);
-
+/**
+ * aggr_cpu_id__global - Create an aggr_cpu_id for global aggregation.
+ * The function signature is compatible with aggr_cpu_id_get_t.
+ */
+struct aggr_cpu_id aggr_cpu_id__global(struct perf_cpu cpu, void *data);
#endif /* __PERF_CPUMAP_H */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * File for any parts of the Coresight decoding that don't require
+ * OpenCSD.
+ */
+
+#include <errno.h>
+#include <inttypes.h>
+
+#include "cs-etm.h"
+
+static const char * const cs_etm_global_header_fmts[] = {
+ [CS_HEADER_VERSION] = " Header version %llx\n",
+ [CS_PMU_TYPE_CPUS] = " PMU type/num cpus %llx\n",
+ [CS_ETM_SNAPSHOT] = " Snapshot %llx\n",
+};
+
+static const char * const cs_etm_priv_fmts[] = {
+ [CS_ETM_MAGIC] = " Magic number %llx\n",
+ [CS_ETM_CPU] = " CPU %lld\n",
+ [CS_ETM_NR_TRC_PARAMS] = " NR_TRC_PARAMS %llx\n",
+ [CS_ETM_ETMCR] = " ETMCR %llx\n",
+ [CS_ETM_ETMTRACEIDR] = " ETMTRACEIDR %llx\n",
+ [CS_ETM_ETMCCER] = " ETMCCER %llx\n",
+ [CS_ETM_ETMIDR] = " ETMIDR %llx\n",
+};
+
+static const char * const cs_etmv4_priv_fmts[] = {
+ [CS_ETM_MAGIC] = " Magic number %llx\n",
+ [CS_ETM_CPU] = " CPU %lld\n",
+ [CS_ETM_NR_TRC_PARAMS] = " NR_TRC_PARAMS %llx\n",
+ [CS_ETMV4_TRCCONFIGR] = " TRCCONFIGR %llx\n",
+ [CS_ETMV4_TRCTRACEIDR] = " TRCTRACEIDR %llx\n",
+ [CS_ETMV4_TRCIDR0] = " TRCIDR0 %llx\n",
+ [CS_ETMV4_TRCIDR1] = " TRCIDR1 %llx\n",
+ [CS_ETMV4_TRCIDR2] = " TRCIDR2 %llx\n",
+ [CS_ETMV4_TRCIDR8] = " TRCIDR8 %llx\n",
+ [CS_ETMV4_TRCAUTHSTATUS] = " TRCAUTHSTATUS %llx\n",
+ [CS_ETE_TRCDEVARCH] = " TRCDEVARCH %llx\n"
+};
+
+static const char * const param_unk_fmt =
+ " Unknown parameter [%d] %"PRIx64"\n";
+static const char * const magic_unk_fmt =
+ " Magic number Unknown %"PRIx64"\n";
+
+static int cs_etm__print_cpu_metadata_v0(u64 *val, int *offset)
+{
+ int i = *offset, j, nr_params = 0, fmt_offset;
+ u64 magic;
+
+ /* check magic value */
+ magic = val[i + CS_ETM_MAGIC];
+ if ((magic != __perf_cs_etmv3_magic) &&
+ (magic != __perf_cs_etmv4_magic)) {
+ /* failure - note bad magic value */
+ fprintf(stdout, magic_unk_fmt, magic);
+ return -EINVAL;
+ }
+
+ /* print common header block */
+ fprintf(stdout, cs_etm_priv_fmts[CS_ETM_MAGIC], val[i++]);
+ fprintf(stdout, cs_etm_priv_fmts[CS_ETM_CPU], val[i++]);
+
+ if (magic == __perf_cs_etmv3_magic) {
+ nr_params = CS_ETM_NR_TRC_PARAMS_V0;
+ fmt_offset = CS_ETM_ETMCR;
+ /* after common block, offset format index past NR_PARAMS */
+ for (j = fmt_offset; j < nr_params + fmt_offset; j++, i++)
+ fprintf(stdout, cs_etm_priv_fmts[j], val[i]);
+ } else if (magic == __perf_cs_etmv4_magic) {
+ nr_params = CS_ETMV4_NR_TRC_PARAMS_V0;
+ fmt_offset = CS_ETMV4_TRCCONFIGR;
+ /* after common block, offset format index past NR_PARAMS */
+ for (j = fmt_offset; j < nr_params + fmt_offset; j++, i++)
+ fprintf(stdout, cs_etmv4_priv_fmts[j], val[i]);
+ }
+ *offset = i;
+ return 0;
+}
+
+static int cs_etm__print_cpu_metadata_v1(u64 *val, int *offset)
+{
+ int i = *offset, j, total_params = 0;
+ u64 magic;
+
+ magic = val[i + CS_ETM_MAGIC];
+ /* total params to print is NR_PARAMS + common block size for v1 */
+ total_params = val[i + CS_ETM_NR_TRC_PARAMS] + CS_ETM_COMMON_BLK_MAX_V1;
+
+ if (magic == __perf_cs_etmv3_magic) {
+ for (j = 0; j < total_params; j++, i++) {
+ /* if newer record - could be excess params */
+ if (j >= CS_ETM_PRIV_MAX)
+ fprintf(stdout, param_unk_fmt, j, val[i]);
+ else
+ fprintf(stdout, cs_etm_priv_fmts[j], val[i]);
+ }
+ } else if (magic == __perf_cs_etmv4_magic || magic == __perf_cs_ete_magic) {
+ /*
+ * ETE and ETMv4 can be printed in the same block because the number of parameters
+ * is saved and they share the list of parameter names. ETE is also only supported
+ * in V1 files.
+ */
+ for (j = 0; j < total_params; j++, i++) {
+ /* if newer record - could be excess params */
+ if (j >= CS_ETE_PRIV_MAX)
+ fprintf(stdout, param_unk_fmt, j, val[i]);
+ else
+ fprintf(stdout, cs_etmv4_priv_fmts[j], val[i]);
+ }
+ } else {
+ /* failure - note bad magic value and error out */
+ fprintf(stdout, magic_unk_fmt, magic);
+ return -EINVAL;
+ }
+ *offset = i;
+ return 0;
+}
+
+static void cs_etm__print_auxtrace_info(u64 *val, int num)
+{
+ int i, cpu = 0, version, err;
+
+ version = val[0];
+
+ for (i = 0; i < CS_HEADER_VERSION_MAX; i++)
+ fprintf(stdout, cs_etm_global_header_fmts[i], val[i]);
+
+ for (i = CS_HEADER_VERSION_MAX; cpu < num; cpu++) {
+ if (version == 0)
+ err = cs_etm__print_cpu_metadata_v0(val, &i);
+ else if (version == 1)
+ err = cs_etm__print_cpu_metadata_v1(val, &i);
+ if (err)
+ return;
+ }
+}
+
+/*
+ * Do some basic checks and print the auxtrace info header before calling
+ * into cs_etm__process_auxtrace_info_full() which requires OpenCSD to be
+ * linked in. This allows some basic debugging if OpenCSD is missing.
+ */
+int cs_etm__process_auxtrace_info(union perf_event *event,
+ struct perf_session *session)
+{
+ struct perf_record_auxtrace_info *auxtrace_info = &event->auxtrace_info;
+ int event_header_size = sizeof(struct perf_event_header);
+ int num_cpu;
+ u64 *ptr = NULL;
+ u64 hdr_version;
+
+ if (auxtrace_info->header.size < (event_header_size + INFO_HEADER_SIZE))
+ return -EINVAL;
+
+ /* First the global part */
+ ptr = (u64 *) auxtrace_info->priv;
+
+ /* Look for version of the header */
+ hdr_version = ptr[0];
+ if (hdr_version > CS_HEADER_CURRENT_VERSION) {
+ pr_err("\nCS ETM Trace: Unknown Header Version = %#" PRIx64, hdr_version);
+ pr_err(", version supported <= %x\n", CS_HEADER_CURRENT_VERSION);
+ return -EINVAL;
+ }
+
+ if (dump_trace) {
+ num_cpu = ptr[CS_PMU_TYPE_CPUS] & 0xffffffff;
+ cs_etm__print_auxtrace_info(ptr, num_cpu);
+ }
+
+ return cs_etm__process_auxtrace_info_full(event, session);
+}
return timeless_decoding;
}
-static const char * const cs_etm_global_header_fmts[] = {
- [CS_HEADER_VERSION] = " Header version %llx\n",
- [CS_PMU_TYPE_CPUS] = " PMU type/num cpus %llx\n",
- [CS_ETM_SNAPSHOT] = " Snapshot %llx\n",
-};
-
-static const char * const cs_etm_priv_fmts[] = {
- [CS_ETM_MAGIC] = " Magic number %llx\n",
- [CS_ETM_CPU] = " CPU %lld\n",
- [CS_ETM_NR_TRC_PARAMS] = " NR_TRC_PARAMS %llx\n",
- [CS_ETM_ETMCR] = " ETMCR %llx\n",
- [CS_ETM_ETMTRACEIDR] = " ETMTRACEIDR %llx\n",
- [CS_ETM_ETMCCER] = " ETMCCER %llx\n",
- [CS_ETM_ETMIDR] = " ETMIDR %llx\n",
-};
-
-static const char * const cs_etmv4_priv_fmts[] = {
- [CS_ETM_MAGIC] = " Magic number %llx\n",
- [CS_ETM_CPU] = " CPU %lld\n",
- [CS_ETM_NR_TRC_PARAMS] = " NR_TRC_PARAMS %llx\n",
- [CS_ETMV4_TRCCONFIGR] = " TRCCONFIGR %llx\n",
- [CS_ETMV4_TRCTRACEIDR] = " TRCTRACEIDR %llx\n",
- [CS_ETMV4_TRCIDR0] = " TRCIDR0 %llx\n",
- [CS_ETMV4_TRCIDR1] = " TRCIDR1 %llx\n",
- [CS_ETMV4_TRCIDR2] = " TRCIDR2 %llx\n",
- [CS_ETMV4_TRCIDR8] = " TRCIDR8 %llx\n",
- [CS_ETMV4_TRCAUTHSTATUS] = " TRCAUTHSTATUS %llx\n",
- [CS_ETE_TRCDEVARCH] = " TRCDEVARCH %llx\n"
-};
-
-static const char * const param_unk_fmt =
- " Unknown parameter [%d] %llx\n";
-static const char * const magic_unk_fmt =
- " Magic number Unknown %llx\n";
-
-static int cs_etm__print_cpu_metadata_v0(__u64 *val, int *offset)
-{
- int i = *offset, j, nr_params = 0, fmt_offset;
- __u64 magic;
-
- /* check magic value */
- magic = val[i + CS_ETM_MAGIC];
- if ((magic != __perf_cs_etmv3_magic) &&
- (magic != __perf_cs_etmv4_magic)) {
- /* failure - note bad magic value */
- fprintf(stdout, magic_unk_fmt, magic);
- return -EINVAL;
- }
-
- /* print common header block */
- fprintf(stdout, cs_etm_priv_fmts[CS_ETM_MAGIC], val[i++]);
- fprintf(stdout, cs_etm_priv_fmts[CS_ETM_CPU], val[i++]);
-
- if (magic == __perf_cs_etmv3_magic) {
- nr_params = CS_ETM_NR_TRC_PARAMS_V0;
- fmt_offset = CS_ETM_ETMCR;
- /* after common block, offset format index past NR_PARAMS */
- for (j = fmt_offset; j < nr_params + fmt_offset; j++, i++)
- fprintf(stdout, cs_etm_priv_fmts[j], val[i]);
- } else if (magic == __perf_cs_etmv4_magic) {
- nr_params = CS_ETMV4_NR_TRC_PARAMS_V0;
- fmt_offset = CS_ETMV4_TRCCONFIGR;
- /* after common block, offset format index past NR_PARAMS */
- for (j = fmt_offset; j < nr_params + fmt_offset; j++, i++)
- fprintf(stdout, cs_etmv4_priv_fmts[j], val[i]);
- }
- *offset = i;
- return 0;
-}
-
-static int cs_etm__print_cpu_metadata_v1(__u64 *val, int *offset)
-{
- int i = *offset, j, total_params = 0;
- __u64 magic;
-
- magic = val[i + CS_ETM_MAGIC];
- /* total params to print is NR_PARAMS + common block size for v1 */
- total_params = val[i + CS_ETM_NR_TRC_PARAMS] + CS_ETM_COMMON_BLK_MAX_V1;
-
- if (magic == __perf_cs_etmv3_magic) {
- for (j = 0; j < total_params; j++, i++) {
- /* if newer record - could be excess params */
- if (j >= CS_ETM_PRIV_MAX)
- fprintf(stdout, param_unk_fmt, j, val[i]);
- else
- fprintf(stdout, cs_etm_priv_fmts[j], val[i]);
- }
- } else if (magic == __perf_cs_etmv4_magic || magic == __perf_cs_ete_magic) {
- /*
- * ETE and ETMv4 can be printed in the same block because the number of parameters
- * is saved and they share the list of parameter names. ETE is also only supported
- * in V1 files.
- */
- for (j = 0; j < total_params; j++, i++) {
- /* if newer record - could be excess params */
- if (j >= CS_ETE_PRIV_MAX)
- fprintf(stdout, param_unk_fmt, j, val[i]);
- else
- fprintf(stdout, cs_etmv4_priv_fmts[j], val[i]);
- }
- } else {
- /* failure - note bad magic value and error out */
- fprintf(stdout, magic_unk_fmt, magic);
- return -EINVAL;
- }
- *offset = i;
- return 0;
-}
-
-static void cs_etm__print_auxtrace_info(__u64 *val, int num)
-{
- int i, cpu = 0, version, err;
-
- /* bail out early on bad header version */
- version = val[0];
- if (version > CS_HEADER_CURRENT_VERSION) {
- /* failure.. return */
- fprintf(stdout, " Unknown Header Version = %x, ", version);
- fprintf(stdout, "Version supported <= %x\n", CS_HEADER_CURRENT_VERSION);
- return;
- }
-
- for (i = 0; i < CS_HEADER_VERSION_MAX; i++)
- fprintf(stdout, cs_etm_global_header_fmts[i], val[i]);
-
- for (i = CS_HEADER_VERSION_MAX; cpu < num; cpu++) {
- if (version == 0)
- err = cs_etm__print_cpu_metadata_v0(val, &i);
- else if (version == 1)
- err = cs_etm__print_cpu_metadata_v1(val, &i);
- if (err)
- return;
- }
-}
-
/*
* Read a single cpu parameter block from the auxtrace_info priv block.
*
return 0;
}
-int cs_etm__process_auxtrace_info(union perf_event *event,
- struct perf_session *session)
+int cs_etm__process_auxtrace_info_full(union perf_event *event,
+ struct perf_session *session)
{
struct perf_record_auxtrace_info *auxtrace_info = &event->auxtrace_info;
struct cs_etm_auxtrace *etm = NULL;
struct int_node *inode;
- unsigned int pmu_type;
int event_header_size = sizeof(struct perf_event_header);
- int info_header_size;
int total_size = auxtrace_info->header.size;
int priv_size = 0;
int num_cpu, trcidr_idx;
int err = 0;
int i, j;
- u64 *ptr, *hdr = NULL;
+ u64 *ptr = NULL;
u64 **metadata = NULL;
- u64 hdr_version;
-
- /*
- * sizeof(auxtrace_info_event::type) +
- * sizeof(auxtrace_info_event::reserved) == 8
- */
- info_header_size = 8;
-
- if (total_size < (event_header_size + info_header_size))
- return -EINVAL;
-
- priv_size = total_size - event_header_size - info_header_size;
-
- /* First the global part */
- ptr = (u64 *) auxtrace_info->priv;
-
- /* Look for version of the header */
- hdr_version = ptr[0];
- if (hdr_version > CS_HEADER_CURRENT_VERSION) {
- /* print routine will print an error on bad version */
- if (dump_trace)
- cs_etm__print_auxtrace_info(auxtrace_info->priv, 0);
- return -EINVAL;
- }
-
- hdr = zalloc(sizeof(*hdr) * CS_HEADER_VERSION_MAX);
- if (!hdr)
- return -ENOMEM;
-
- /* Extract header information - see cs-etm.h for format */
- for (i = 0; i < CS_HEADER_VERSION_MAX; i++)
- hdr[i] = ptr[i];
- num_cpu = hdr[CS_PMU_TYPE_CPUS] & 0xffffffff;
- pmu_type = (unsigned int) ((hdr[CS_PMU_TYPE_CPUS] >> 32) &
- 0xffffffff);
/*
* Create an RB tree for traceID-metadata tuple. Since the conversion
* in anything other than a sequential array is worth doing.
*/
traceid_list = intlist__new(NULL);
- if (!traceid_list) {
- err = -ENOMEM;
- goto err_free_hdr;
- }
+ if (!traceid_list)
+ return -ENOMEM;
+ /* First the global part */
+ ptr = (u64 *) auxtrace_info->priv;
+ num_cpu = ptr[CS_PMU_TYPE_CPUS] & 0xffffffff;
metadata = zalloc(sizeof(*metadata) * num_cpu);
if (!metadata) {
err = -ENOMEM;
goto err_free_traceid_list;
}
+ /* Start parsing after the common part of the header */
+ i = CS_HEADER_VERSION_MAX;
+
/*
* The metadata is stored in the auxtrace_info section and encodes
* the configuration of the ARM embedded trace macrocell which is
* The following tests if the correct number of double words was
* present in the auxtrace info section.
*/
+ priv_size = total_size - event_header_size - INFO_HEADER_SIZE;
if (i * 8 != priv_size) {
err = -EINVAL;
goto err_free_metadata;
etm->machine = &session->machines.host;
etm->num_cpu = num_cpu;
- etm->pmu_type = pmu_type;
- etm->snapshot_mode = (hdr[CS_ETM_SNAPSHOT] != 0);
+ etm->pmu_type = (unsigned int) ((ptr[CS_PMU_TYPE_CPUS] >> 32) & 0xffffffff);
+ etm->snapshot_mode = (ptr[CS_ETM_SNAPSHOT] != 0);
etm->metadata = metadata;
etm->auxtrace_type = auxtrace_info->type;
etm->timeless_decoding = cs_etm__is_timeless_decoding(etm);
goto err_delete_thread;
}
- if (dump_trace) {
- cs_etm__print_auxtrace_info(auxtrace_info->priv, num_cpu);
- }
-
err = cs_etm__synth_events(etm, session);
if (err)
goto err_delete_thread;
zfree(&metadata);
err_free_traceid_list:
intlist__delete(traceid_list);
-err_free_hdr:
- zfree(&hdr);
- /*
- * At this point, as a minimum we have valid header. Dump the rest of
- * the info section - the print routines will error out on structural
- * issues.
- */
- if (dump_trace)
- cs_etm__print_auxtrace_info(auxtrace_info->priv, num_cpu);
return err;
}
#ifndef INCLUDE__UTIL_PERF_CS_ETM_H__
#define INCLUDE__UTIL_PERF_CS_ETM_H__
+#include "debug.h"
#include "util/event.h"
#include <linux/bits.h>
#define CS_ETMV4_PRIV_SIZE (CS_ETMV4_PRIV_MAX * sizeof(u64))
#define CS_ETE_PRIV_SIZE (CS_ETE_PRIV_MAX * sizeof(u64))
-#ifdef HAVE_CSTRACE_SUPPORT
+#define INFO_HEADER_SIZE (sizeof(((struct perf_record_auxtrace_info *)0)->type) + \
+ sizeof(((struct perf_record_auxtrace_info *)0)->reserved__))
+
int cs_etm__process_auxtrace_info(union perf_event *event,
struct perf_session *session);
+
+#ifdef HAVE_CSTRACE_SUPPORT
int cs_etm__get_cpu(u8 trace_chan_id, int *cpu);
int cs_etm__get_pid_fmt(u8 trace_chan_id, u64 *pid_fmt);
int cs_etm__etmq_set_tid(struct cs_etm_queue *etmq,
u8 trace_chan_id);
struct cs_etm_packet_queue
*cs_etm__etmq_get_packet_queue(struct cs_etm_queue *etmq, u8 trace_chan_id);
+int cs_etm__process_auxtrace_info_full(union perf_event *event __maybe_unused,
+ struct perf_session *session __maybe_unused);
#else
static inline int
-cs_etm__process_auxtrace_info(union perf_event *event __maybe_unused,
- struct perf_session *session __maybe_unused)
-{
- return -1;
-}
-
-static inline int cs_etm__get_cpu(u8 trace_chan_id __maybe_unused,
- int *cpu __maybe_unused)
+cs_etm__process_auxtrace_info_full(union perf_event *event __maybe_unused,
+ struct perf_session *session __maybe_unused)
{
+ pr_err("\nCS ETM Trace: OpenCSD is not linked in, please recompile with CORESIGHT=1\n");
return -1;
}
-
-static inline int cs_etm__etmq_set_tid(
- struct cs_etm_queue *etmq __maybe_unused,
- pid_t tid __maybe_unused,
- u8 trace_chan_id __maybe_unused)
-{
- return -1;
-}
-
-static inline bool cs_etm__etmq_is_timeless(
- struct cs_etm_queue *etmq __maybe_unused)
-{
- /* What else to return? */
- return true;
-}
-
-static inline void cs_etm__etmq_set_traceid_queue_timestamp(
- struct cs_etm_queue *etmq __maybe_unused,
- u8 trace_chan_id __maybe_unused) {}
-
-static inline struct cs_etm_packet_queue *cs_etm__etmq_get_packet_queue(
- struct cs_etm_queue *etmq __maybe_unused,
- u8 trace_chan_id __maybe_unused)
-{
- return NULL;
-}
#endif
#endif
#include <babeltrace/ctf-writer/event-fields.h>
#include <babeltrace/ctf-ir/utils.h>
#include <babeltrace/ctf/events.h>
-#include <traceevent/event-parse.h>
#include "asm/bug.h"
#include "data-convert.h"
#include "session.h"
#include <linux/time64.h>
#include "util.h"
#include "clockid.h"
+#include "util/sample.h"
+
+#ifdef HAVE_LIBTRACEEVENT
+#include <traceevent/event-parse.h>
+#endif
#define pr_N(n, fmt, ...) \
eprintf(n, debug_data_convert, fmt, ##__VA_ARGS__)
offset = tmp_val;
len = offset >> 16;
offset &= 0xffff;
+#ifdef HAVE_LIBTRACEEVENT_TEP_FIELD_IS_RELATIVE
if (flags & TEP_FIELD_IS_RELATIVE)
offset += fmtf->offset + fmtf->size;
+#endif
}
if (flags & TEP_FIELD_IS_ARRAY) {
#include "util/thread.h"
#include "util/tool.h"
+#ifdef HAVE_LIBTRACEEVENT
+#include <traceevent/event-parse.h>
+#endif
+
struct convert_json {
struct perf_tool tool;
FILE *out;
}
output_json_format(out, false, 3, "]");
+#ifdef HAVE_LIBTRACEEVENT
+ if (sample->raw_data) {
+ int i;
+ struct tep_format_field **fields;
+
+ fields = tep_event_fields(evsel->tp_format);
+ if (fields) {
+ i = 0;
+ while (fields[i]) {
+ struct trace_seq s;
+
+ trace_seq_init(&s);
+ tep_print_field(&s, sample->raw_data, fields[i]);
+ output_json_key_string(out, true, 3, fields[i]->name, s.buffer);
+
+ i++;
+ }
+ free(fields);
+ }
+ }
+#endif
output_json_format(out, false, 2, "}");
return 0;
}
.exit = perf_event__process_exit,
.fork = perf_event__process_fork,
.lost = perf_event__process_lost,
+#ifdef HAVE_LIBTRACEEVENT
.tracing_data = perf_event__process_tracing_data,
+#endif
.build_id = perf_event__process_build_id,
.id_index = perf_event__process_id_index,
.auxtrace_info = perf_event__process_auxtrace_info,
if (die_find_realfunc(cu_die, addr, &die_mem)
&& die_entrypc(&die_mem, &faddr) == 0 &&
faddr == addr) {
- *fname = dwarf_decl_file(&die_mem);
+ *fname = die_get_decl_file(&die_mem);
dwarf_decl_line(&die_mem, lineno);
goto out;
}
}
out:
- return *lineno ?: -ENOENT;
+ return (*lineno && *fname) ? *lineno : -ENOENT;
}
static int __die_find_inline_cb(Dwarf_Die *die_mem, void *data);
{
Dwarf_Attribute attr;
- if (dwarf_attr(tp_die, attr_name, &attr) == NULL ||
+ if (dwarf_attr_integrate(tp_die, attr_name, &attr) == NULL ||
dwarf_formudata(&attr, result) != 0)
return -ENOENT;
return 0;
}
-/* Get attribute and translate it as a sdata */
-static int die_get_attr_sdata(Dwarf_Die *tp_die, unsigned int attr_name,
- Dwarf_Sword *result)
-{
- Dwarf_Attribute attr;
-
- if (dwarf_attr(tp_die, attr_name, &attr) == NULL ||
- dwarf_formsdata(&attr, result) != 0)
- return -ENOENT;
-
- return 0;
-}
-
/**
* die_is_signed_type - Check whether a type DIE is signed or not
* @tp_die: a DIE of a type
/* Get the call file index number in CU DIE */
static int die_get_call_fileno(Dwarf_Die *in_die)
{
- Dwarf_Sword idx;
+ Dwarf_Word idx;
- if (die_get_attr_sdata(in_die, DW_AT_call_file, &idx) == 0)
+ if (die_get_attr_udata(in_die, DW_AT_call_file, &idx) == 0)
return (int)idx;
else
return -ENOENT;
/* Get the declared file index number in CU DIE */
static int die_get_decl_fileno(Dwarf_Die *pdie)
{
- Dwarf_Sword idx;
+ Dwarf_Word idx;
- if (die_get_attr_sdata(pdie, DW_AT_decl_file, &idx) == 0)
+ if (die_get_attr_udata(pdie, DW_AT_decl_file, &idx) == 0)
return (int)idx;
else
return -ENOENT;
}
+/* Return the file name by index */
+static const char *die_get_file_name(Dwarf_Die *dw_die, int idx)
+{
+ Dwarf_Die cu_die;
+ Dwarf_Files *files;
+
+ if (idx < 0 || !dwarf_diecu(dw_die, &cu_die, NULL, NULL) ||
+ dwarf_getsrcfiles(&cu_die, &files, NULL) != 0)
+ return NULL;
+
+ return dwarf_filesrc(files, idx, NULL, NULL);
+}
+
/**
* die_get_call_file - Get callsite file name of inlined function instance
* @in_die: a DIE of an inlined function instance
*/
const char *die_get_call_file(Dwarf_Die *in_die)
{
- Dwarf_Die cu_die;
- Dwarf_Files *files;
- int idx;
-
- idx = die_get_call_fileno(in_die);
- if (idx < 0 || !dwarf_diecu(in_die, &cu_die, NULL, NULL) ||
- dwarf_getsrcfiles(&cu_die, &files, NULL) != 0)
- return NULL;
-
- return dwarf_filesrc(files, idx, NULL, NULL);
+ return die_get_file_name(in_die, die_get_call_fileno(in_die));
}
+/**
+ * die_get_decl_file - Find the declared file name of this DIE
+ * @dw_die: a DIE for something declared.
+ *
+ * Get declared file name of @dw_die.
+ * NOTE: Since some version of clang DWARF5 implementation incorrectly uses
+ * file index 0 for DW_AT_decl_file, die_get_decl_file() will return NULL for
+ * such cases. Use this function instead.
+ */
+const char *die_get_decl_file(Dwarf_Die *dw_die)
+{
+ return die_get_file_name(dw_die, die_get_decl_fileno(dw_die));
+}
/**
* die_find_child - Generic DIE search function in DIE tree
}
if (addr) {
- fname = dwarf_decl_file(in_die);
+ fname = die_get_decl_file(in_die);
if (fname && dwarf_decl_line(in_die, &lineno) == 0) {
lw->retval = lw->callback(fname, lineno, addr, lw->data);
if (lw->retval != 0)
int lineno;
/* Handle function declaration line */
- fname = dwarf_decl_file(sp_die);
+ fname = die_get_decl_file(sp_die);
if (fname && dwarf_decl_line(sp_die, &lineno) == 0 &&
die_entrypc(sp_die, &addr) == 0) {
lw.retval = callback(fname, lineno, addr, data);
if (dwarf_tag(rt_die) != DW_TAG_compile_unit) {
cu_die = dwarf_diecu(rt_die, &die_mem, NULL, NULL);
dwarf_decl_line(rt_die, &decl);
- decf = dwarf_decl_file(rt_die);
+ decf = die_get_decl_file(rt_die);
+ if (!decf) {
+ pr_debug2("Failed to get the declared file name of %s\n",
+ dwarf_diename(rt_die));
+ return -EINVAL;
+ }
} else
cu_die = rt_die;
if (!cu_die) {
dwarf_decl_line(&die_mem, &inl);
if (inl != decl ||
- decf != dwarf_decl_file(&die_mem))
+ decf != die_get_decl_file(&die_mem))
continue;
}
}
/* Get callsite file name of inlined function instance */
const char *die_get_call_file(Dwarf_Die *in_die);
+/* Get declared file name of a DIE */
+const char *die_get_decl_file(Dwarf_Die *dw_die);
+
/* Get type die */
Dwarf_Die *die_get_type(Dwarf_Die *vr_die, Dwarf_Die *die_mem);
#include <perf/event.h>
#include <linux/types.h>
-#include "perf_regs.h"
-
struct dso;
struct machine;
struct perf_event_attr;
+struct perf_sample;
#ifdef __LP64__
/*
/* perf sample has 16 bits size limit */
#define PERF_SAMPLE_MAX_SIZE (1 << 16)
-/* number of register is bound by the number of bits in regs_dump::mask (64) */
-#define PERF_SAMPLE_REGS_CACHE_SIZE (8 * sizeof(u64))
-
-struct regs_dump {
- u64 abi;
- u64 mask;
- u64 *regs;
-
- /* Cached values/mask filled by first register access. */
- u64 cache_regs[PERF_SAMPLE_REGS_CACHE_SIZE];
- u64 cache_mask;
-};
-
-struct stack_dump {
- u16 offset;
- u64 size;
- char *data;
-};
-
-struct sample_read_value {
- u64 value;
- u64 id; /* only if PERF_FORMAT_ID */
- u64 lost; /* only if PERF_FORMAT_LOST */
-};
-
-struct sample_read {
- u64 time_enabled;
- u64 time_running;
- union {
- struct {
- u64 nr;
- struct sample_read_value *values;
- } group;
- struct sample_read_value one;
- };
-};
-
-static inline size_t sample_read_value_size(u64 read_format)
-{
- /* PERF_FORMAT_ID is forced for PERF_SAMPLE_READ */
- if (read_format & PERF_FORMAT_LOST)
- return sizeof(struct sample_read_value);
- else
- return offsetof(struct sample_read_value, lost);
-}
-
-static inline struct sample_read_value *
-next_sample_read_value(struct sample_read_value *v, u64 read_format)
-{
- return (void *)v + sample_read_value_size(read_format);
-}
-
-#define sample_read_group__for_each(v, nr, rf) \
- for (int __i = 0; __i < (int)nr; v = next_sample_read_value(v, rf), __i++)
-
struct ip_callchain {
u64 nr;
u64 ips[];
PERF_IP_FLAG_VMENTRY |\
PERF_IP_FLAG_VMEXIT)
-#define MAX_INSN 16
-
-struct aux_sample {
- u64 size;
- void *data;
-};
-
-struct perf_sample {
- u64 ip;
- u32 pid, tid;
- u64 time;
- u64 addr;
- u64 id;
- u64 stream_id;
- u64 period;
- u64 weight;
- u64 transaction;
- u64 insn_cnt;
- u64 cyc_cnt;
- u32 cpu;
- u32 raw_size;
- u64 data_src;
- u64 phys_addr;
- u64 data_page_size;
- u64 code_page_size;
- u64 cgroup;
- u32 flags;
- u32 machine_pid;
- u32 vcpu;
- u16 insn_len;
- u8 cpumode;
- u16 misc;
- u16 ins_lat;
- u16 p_stage_cyc;
- bool no_hw_idx; /* No hw_idx collected in branch_stack */
- char insn[MAX_INSN];
- void *raw_data;
- struct ip_callchain *callchain;
- struct branch_stack *branch_stack;
- struct regs_dump user_regs;
- struct regs_dump intr_regs;
- struct stack_dump user_stack;
- struct sample_read read;
- struct aux_sample aux_sample;
-};
-
#define PERF_MEM_DATA_SRC_NONE \
(PERF_MEM_S(OP, NA) |\
PERF_MEM_S(LVL, NA) |\
u64 branch_ip; /* If via_branch */
};
-/*
- * raw_data is always 4 bytes from an 8-byte boundary, so subtract 4 to get
- * 8-byte alignment.
- */
-static inline void *perf_sample__synth_ptr(struct perf_sample *sample)
-{
- return sample->raw_data - 4;
-}
-
static inline void *perf_synth__raw_data(void *p)
{
return p + 4;
struct perf_sample *sample,
struct machine *machine);
-struct addr_location;
-
-int machine__resolve(struct machine *machine, struct addr_location *al,
- struct perf_sample *sample);
-
-void addr_location__put(struct addr_location *al);
-
-struct thread;
-
bool is_bts_event(struct perf_event_attr *attr);
bool sample_addr_correlates_sym(struct perf_event_attr *attr);
-void thread__resolve(struct thread *thread, struct addr_location *al,
- struct perf_sample *sample);
const char *perf_event__name(unsigned int id);
#include "../perf.h"
#include "asm/bug.h"
#include "bpf-event.h"
+#include "util/event.h"
#include "util/string2.h"
#include "util/perf_api_probe.h"
#include "util/evsel_fprintf.h"
#include "util/evlist-hybrid.h"
#include "util/pmu.h"
+#include "util/sample.h"
#include <signal.h>
#include <unistd.h>
#include <sched.h>
return err;
}
-void evlist__set_leader(struct evlist *evlist)
+static void evlist__set_leader(struct evlist *evlist)
{
perf_evlist__set_leader(&evlist->core);
}
return evsel;
}
+#ifdef HAVE_LIBTRACEEVENT
struct evsel *evlist__add_sched_switch(struct evlist *evlist, bool system_wide)
{
struct evsel *evsel = evsel__newtp_idx("sched", "sched_switch", 0);
evlist__add(evlist, evsel);
return evsel;
-};
+}
+#endif
int evlist__add_attrs(struct evlist *evlist, struct perf_event_attr *attrs, size_t nr_attrs)
{
return NULL;
}
+#ifdef HAVE_LIBTRACEEVENT
int evlist__add_newtp(struct evlist *evlist, const char *sys, const char *name, void *handler)
{
struct evsel *evsel = evsel__newtp(sys, name);
evlist__add(evlist, evsel);
return 0;
}
+#endif
struct evlist_cpu_iterator evlist__cpu_begin(struct evlist *evlist, struct affinity *affinity)
{
{
return evlist__add_aux_dummy(evlist, true);
}
+#ifdef HAVE_LIBTRACEEVENT
struct evsel *evlist__add_sched_switch(struct evlist *evlist, bool system_wide);
+#endif
int evlist__add_sb_event(struct evlist *evlist, struct perf_event_attr *attr,
evsel__sb_cb_t cb, void *data);
int evlist__start_sb_thread(struct evlist *evlist, struct target *target);
void evlist__stop_sb_thread(struct evlist *evlist);
+#ifdef HAVE_LIBTRACEEVENT
int evlist__add_newtp(struct evlist *evlist, const char *sys, const char *name, void *handler);
+#endif
int __evlist__set_tracepoints_handlers(struct evlist *evlist,
const struct evsel_str_handler *assocs,
int evlist__create_maps(struct evlist *evlist, struct target *target);
int evlist__apply_filters(struct evlist *evlist, struct evsel **err_evsel);
-void evlist__set_leader(struct evlist *evlist);
-
u64 __evlist__combined_sample_type(struct evlist *evlist);
u64 evlist__combined_sample_type(struct evlist *evlist);
u64 evlist__combined_branch_type(struct evlist *evlist);
#include <linux/bitops.h>
#include <api/fs/fs.h>
#include <api/fs/tracing_path.h>
-#include <traceevent/event-parse.h>
#include <linux/hw_breakpoint.h>
#include <linux/perf_event.h>
#include <linux/compiler.h>
#include "string2.h"
#include "memswap.h"
#include "util.h"
-#ifdef HAVE_LIBBPF_SUPPORT
-#include <bpf/hashmap.h>
-#else
#include "util/hashmap.h"
-#endif
#include "pmu-hybrid.h"
#include "off_cpu.h"
#include "../perf-sys.h"
#include "util/parse-branch-options.h"
#include <internal/xyarray.h>
#include <internal/lib.h>
+#include <internal/threadmap.h>
#include <linux/ctype.h>
+#ifdef HAVE_LIBTRACEEVENT
+#include <traceevent/event-parse.h>
+#endif
+
struct perf_missing_features perf_missing_features;
static clockid_t clockid;
goto out_err;
}
evsel->cgrp = cgroup__get(orig->cgrp);
+#ifdef HAVE_LIBTRACEEVENT
evsel->tp_format = orig->tp_format;
+#endif
evsel->handler = orig->handler;
evsel->core.leader = orig->core.leader;
evsel->collect_stat = orig->collect_stat;
evsel->weak_group = orig->weak_group;
evsel->use_config_name = orig->use_config_name;
+ evsel->pmu = orig->pmu;
if (evsel__copy_config_terms(evsel, orig) < 0)
goto out_err;
/*
* Returns pointer with encoded error via <linux/err.h> interface.
*/
+#ifdef HAVE_LIBTRACEEVENT
struct evsel *evsel__newtp_idx(const char *sys, const char *name, int idx)
{
struct evsel *evsel = zalloc(perf_evsel__object.size);
out_err:
return ERR_PTR(err);
}
+#endif
const char *const evsel__hw_names[PERF_COUNT_HW_MAX] = {
"cycles",
if (!evsel->prev_raw_counts)
return;
- if (cpu_map_idx == -1) {
- tmp = evsel->prev_raw_counts->aggr;
- evsel->prev_raw_counts->aggr = *count;
- } else {
- tmp = *perf_counts(evsel->prev_raw_counts, cpu_map_idx, thread);
- *perf_counts(evsel->prev_raw_counts, cpu_map_idx, thread) = *count;
- }
+ tmp = *perf_counts(evsel->prev_raw_counts, cpu_map_idx, thread);
+ *perf_counts(evsel->prev_raw_counts, cpu_map_idx, thread) = *count;
count->val = count->val - tmp.val;
count->ena = count->ena - tmp.ena;
perf_missing_features.mmap2 = true;
pr_debug2_peo("switching off mmap2\n");
return true;
- } else if ((evsel->core.attr.exclude_guest || evsel->core.attr.exclude_host) &&
- (evsel->pmu == NULL || evsel->pmu->missing_features.exclude_guest)) {
- if (evsel->pmu == NULL) {
+ } else if (evsel->core.attr.exclude_guest || evsel->core.attr.exclude_host) {
+ if (evsel->pmu == NULL)
evsel->pmu = evsel__find_pmu(evsel);
- if (evsel->pmu)
- evsel->pmu->missing_features.exclude_guest = true;
- else {
- /* we cannot find PMU, disable attrs now */
- evsel->core.attr.exclude_host = false;
- evsel->core.attr.exclude_guest = false;
- }
+
+ if (evsel->pmu)
+ evsel->pmu->missing_features.exclude_guest = true;
+ else {
+ /* we cannot find PMU, disable attrs now */
+ evsel->core.attr.exclude_host = false;
+ evsel->core.attr.exclude_guest = false;
}
if (evsel->exclude_GH) {
* as it has variable bit-field sizes. Instead the
* macro takes the bit-field position/size,
* swaps it based on the host endianness.
- *
- * tep_is_bigendian() is used here instead of
- * bigendian() to avoid python test fails.
*/
- if (tep_is_bigendian()) {
+ if (host_is_bigendian()) {
new_val = bitfield_swap(value, 0, 1);
new_val |= bitfield_swap(value, 1, 1);
new_val |= bitfield_swap(value, 2, 1);
return size;
}
+#ifdef HAVE_LIBTRACEEVENT
struct tep_format_field *evsel__field(struct evsel *evsel, const char *name)
{
return tep_find_field(evsel->tp_format, name);
if (field->flags & TEP_FIELD_IS_DYNAMIC) {
offset = *(int *)(sample->raw_data + field->offset);
offset &= 0xffff;
+#ifdef HAVE_LIBTRACEEVENT_TEP_FIELD_IS_RELATIVE
if (field->flags & TEP_FIELD_IS_RELATIVE)
offset += field->offset + field->size;
+#endif
}
return sample->raw_data + offset;
return field ? format_field__intval(field, sample, evsel->needs_swap) : 0;
}
+#endif
bool evsel__fallback(struct evsel *evsel, int err, char *msg, size_t msgsize)
{
}
}
-bool evsel__is_hybrid(struct evsel *evsel)
+bool evsel__is_hybrid(const struct evsel *evsel)
{
return evsel->pmu_name && perf_pmu__is_hybrid(evsel->pmu_name);
}
#include <internal/evsel.h>
#include <perf/evsel.h>
#include "symbol_conf.h"
-#include <internal/cpumap.h>
-#include <perf/cpumap.h>
struct bpf_object;
struct cgroup;
char *name;
char *group_name;
const char *pmu_name;
+#ifdef HAVE_LIBTRACEEVENT
struct tep_event *tp_format;
+#endif
char *filter;
unsigned long max_events;
double scale;
}
struct evsel *evsel__clone(struct evsel *orig);
-struct evsel *evsel__newtp_idx(const char *sys, const char *name, int idx);
int copy_config_terms(struct list_head *dst, struct list_head *src);
void free_config_terms(struct list_head *config_terms);
+
+#ifdef HAVE_LIBTRACEEVENT
+struct evsel *evsel__newtp_idx(const char *sys, const char *name, int idx);
+
/*
* Returns pointer with encoded error via <linux/err.h> interface.
*/
{
return evsel__newtp_idx(sys, name, 0);
}
+#endif
struct evsel *evsel__new_cycles(bool precise, __u32 type, __u64 config);
+#ifdef HAVE_LIBTRACEEVENT
struct tep_event *event_format__new(const char *sys, const char *name);
+#endif
void evsel__init(struct evsel *evsel, struct perf_event_attr *attr, int idx);
void evsel__exit(struct evsel *evsel);
struct perf_sample;
+#ifdef HAVE_LIBTRACEEVENT
void *evsel__rawptr(struct evsel *evsel, struct perf_sample *sample, const char *name);
u64 evsel__intval(struct evsel *evsel, struct perf_sample *sample, const char *name);
{
return evsel__rawptr(evsel, sample, name);
}
+#endif
struct tep_format_field;
int evsel__store_ids(struct evsel *evsel, struct evlist *evlist);
void evsel__zero_per_pkg(struct evsel *evsel);
-bool evsel__is_hybrid(struct evsel *evsel);
+bool evsel__is_hybrid(const struct evsel *evsel);
struct evsel *evsel__leader(struct evsel *evsel);
bool evsel__has_leader(struct evsel *evsel, struct evsel *leader);
bool evsel__is_leader(struct evsel *evsel);
#include <inttypes.h>
#include <stdio.h>
#include <stdbool.h>
-#include <traceevent/event-parse.h>
#include "evsel.h"
#include "util/evsel_fprintf.h"
#include "util/event.h"
#include "srcline.h"
#include "dso.h"
+#ifdef HAVE_LIBTRACEEVENT
+#include <traceevent/event-parse.h>
+#endif
+
static int comma_fprintf(FILE *fp, bool *first, const char *fmt, ...)
{
va_list args;
term, (u64)evsel->core.attr.sample_freq);
}
+#ifdef HAVE_LIBTRACEEVENT
if (details->trace_fields) {
struct tep_format_field *field;
field = field->next;
}
}
+#endif
out:
fputc('\n', fp);
return ++printed;
#include "expr.h"
#include "expr-bison.h"
#include "expr-flex.h"
+#include "util/hashmap.h"
#include "smt.h"
#include "tsc.h"
#include <linux/err.h>
#ifndef PARSE_CTX_H
#define PARSE_CTX_H 1
-#ifdef HAVE_LIBBPF_SUPPORT
-#include <bpf/hashmap.h>
-#else
-#include "util/hashmap.h"
-#endif
-
+struct hashmap;
struct metric_ref;
struct expr_scanner_ctx {
#include <sys/types.h>
#include <byteswap.h>
#include <unistd.h>
+#include <regex.h>
#include <stdio.h>
#include <stdlib.h>
#include <linux/compiler.h>
#include <linux/ctype.h>
#include <internal/lib.h>
+#ifdef HAVE_LIBTRACEEVENT
+#include <traceevent/event-parse.h>
+#endif
+
/*
* magic2 = "PERFILE2"
* must be a numerical value to let the endianness
return 0;
}
+#ifdef HAVE_LIBTRACEEVENT
static int write_tracing_data(struct feat_fd *ff,
struct evlist *evlist)
{
return read_tracing_data(ff->fd, &evlist->core.entries);
}
+#endif
static int write_build_id(struct feat_fd *ff,
struct evlist *evlist __maybe_unused)
FEAT_PROCESS_STR_FUN(cpudesc, cpu_desc);
FEAT_PROCESS_STR_FUN(cpuid, cpuid);
+#ifdef HAVE_LIBTRACEEVENT
static int process_tracing_data(struct feat_fd *ff, void *data)
{
ssize_t ret = trace_report(ff->fd, data, false);
return ret < 0 ? -1 : 0;
}
+#endif
static int process_build_id(struct feat_fd *ff, void *data __maybe_unused)
{
const struct perf_header_feature_ops feat_ops[HEADER_LAST_FEATURE];
const struct perf_header_feature_ops feat_ops[HEADER_LAST_FEATURE] = {
+#ifdef HAVE_LIBTRACEEVENT
FEAT_OPN(TRACING_DATA, tracing_data, false),
+#endif
FEAT_OPN(BUILD_ID, build_id, false),
FEAT_OPR(HOSTNAME, hostname, false),
FEAT_OPR(OSRELEASE, osrelease, false),
return ret <= 0 ? -1 : 0;
}
+#ifdef HAVE_LIBTRACEEVENT
static int evsel__prepare_tracepoint_event(struct evsel *evsel, struct tep_handle *pevent)
{
struct tep_event *event;
return 0;
}
+#endif
int perf_session__read_header(struct perf_session *session, int repipe_fd)
{
lseek(fd, tmp, SEEK_SET);
}
+#ifdef HAVE_LIBTRACEEVENT
perf_header__process_sections(header, fd, &session->tevent,
perf_file_section__process);
if (evlist__prepare_tracepoint_events(session->evlist, session->tevent.pevent))
goto out_delete_evlist;
+#else
+ perf_header__process_sections(header, fd, NULL, perf_file_section__process);
+#endif
return 0;
out_errno:
return 0;
}
+#ifdef HAVE_LIBTRACEEVENT
int perf_event__process_tracing_data(struct perf_session *session,
union perf_event *event)
{
return size_read + padding;
}
+#endif
int perf_event__process_build_id(struct perf_session *session,
union perf_event *event)
union perf_event *event,
struct evlist **pevlist);
size_t perf_event__fprintf_event_update(union perf_event *event, FILE *fp);
+#ifdef HAVE_LIBTRACEEVENT
int perf_event__process_tracing_data(struct perf_session *session,
union perf_event *event);
+#endif
int perf_event__process_build_id(struct perf_session *session,
union perf_event *event);
bool is_perf_magic(u64 magic);
#include "intel-pt-insn-decoder.h"
#include "dump-insn.h"
+#include "util/sample.h"
#if INTEL_PT_INSN_BUF_SZ < MAX_INSN_SIZE || INTEL_PT_INSN_BUF_SZ > MAX_INSN
#error Instruction buffer size too small
return 1;
}
+#ifdef HAVE_LIBTRACEEVENT
static int intel_pt_process_switch(struct intel_pt *pt,
struct perf_sample *sample)
{
return machine__set_current_tid(pt->machine, cpu, -1, tid);
}
+#endif /* HAVE_LIBTRACEEVENT */
static int intel_pt_context_switch_in(struct intel_pt *pt,
struct perf_sample *sample)
return err;
}
+#ifdef HAVE_LIBTRACEEVENT
if (pt->switch_evsel && event->header.type == PERF_RECORD_SAMPLE)
err = intel_pt_process_switch(pt, sample);
- else if (event->header.type == PERF_RECORD_ITRACE_START)
+ else
+#endif
+ if (event->header.type == PERF_RECORD_ITRACE_START)
err = intel_pt_process_itrace_start(pt, event, sample);
else if (event->header.type == PERF_RECORD_AUX_OUTPUT_HW_ID)
err = intel_pt_process_aux_output_hw_id(pt, event, sample);
struct perf_stat_config *config __maybe_unused,
struct timespec *ts __maybe_unused,
char *prefix __maybe_unused,
- iostat_print_counter_t print_cnt_cb __maybe_unused)
+ iostat_print_counter_t print_cnt_cb __maybe_unused,
+ void *arg __maybe_unused)
{
}
extern enum iostat_mode_t iostat_mode;
-typedef void (*iostat_print_counter_t)(struct perf_stat_config *, struct evsel *, char *);
+typedef void (*iostat_print_counter_t)(struct perf_stat_config *, struct evsel *, void *);
int iostat_prepare(struct evlist *evlist, struct perf_stat_config *config);
int iostat_parse(const struct option *opt, const char *str,
struct perf_stat_output_ctx *out);
void iostat_print_counters(struct evlist *evlist,
struct perf_stat_config *config, struct timespec *ts,
- char *prefix, iostat_print_counter_t print_cnt_cb);
+ char *prefix, iostat_print_counter_t print_cnt_cb, void *arg);
#endif /* _IOSTAT_H */
#ifndef PERF_UTIL_KWORK_H
#define PERF_UTIL_KWORK_H
-#include "perf.h"
-
#include "util/tool.h"
-#include "util/event.h"
-#include "util/evlist.h"
-#include "util/session.h"
#include "util/time-utils.h"
-#include <linux/list.h>
#include <linux/bitmap.h>
+#include <linux/list.h>
+#include <linux/rbtree.h>
+#include <linux/types.h>
+
+struct perf_sample;
+struct perf_session;
enum kwork_class_type {
KWORK_CLASS_IRQ,
char *pipe_template = NULL;
const char *opts = llvm_param.opts;
char *command_echo = NULL, *command_out;
- char *perf_include_dir = system_path(PERF_INCLUDE_DIR);
+ char *libbpf_include_dir = system_path(LIBBPF_INCLUDE_DIR);
if (path[0] != '-' && realpath(path, abspath) == NULL) {
err = errno;
snprintf(linux_version_code_str, sizeof(linux_version_code_str),
"0x%x", kernel_version);
- if (asprintf(&perf_bpf_include_opts, "-I%s/bpf", perf_include_dir) < 0)
+ if (asprintf(&perf_bpf_include_opts, "-I%s/", libbpf_include_dir) < 0)
goto errout;
force_set_env("NR_CPUS", nr_cpus_avail_str);
force_set_env("LINUX_VERSION_CODE", linux_version_code_str);
free(kbuild_dir);
free(kbuild_include_opts);
free(perf_bpf_include_opts);
- free(perf_include_dir);
+ free(libbpf_include_dir);
if (!p_obj_buf)
free(obj_buf);
free(kbuild_include_opts);
free(obj_buf);
free(perf_bpf_include_opts);
- free(perf_include_dir);
+ free(libbpf_include_dir);
free(pipe_template);
if (p_obj_buf)
*p_obj_buf = NULL;
* Number of stack trace entries to skip when finding callers.
* The first few entries belong to the locking implementation itself.
*/
-#define CONTENTION_STACK_SKIP 3
+#define CONTENTION_STACK_SKIP 4
/*
* flags for lock:contention_begin
int lost;
int max_stack;
int stack_skip;
+ int aggr_mode;
};
#ifdef HAVE_BPF_SKEL
#endif /* HAVE_BPF_SKEL */
-bool is_lock_function(struct machine *machine, u64 addr);
-
#endif /* PERF_LOCK_CONTENTION_H */
}
return err;
}
+
+bool machine__is_lock_function(struct machine *machine, u64 addr)
+{
+ if (!machine->sched.text_start) {
+ struct map *kmap;
+ struct symbol *sym = machine__find_kernel_symbol_by_name(machine, "__sched_text_start", &kmap);
+
+ if (!sym) {
+ /* to avoid retry */
+ machine->sched.text_start = 1;
+ return false;
+ }
+
+ machine->sched.text_start = kmap->unmap_ip(kmap, sym->start);
+
+ /* should not fail from here */
+ sym = machine__find_kernel_symbol_by_name(machine, "__sched_text_end", &kmap);
+ machine->sched.text_end = kmap->unmap_ip(kmap, sym->start);
+
+ sym = machine__find_kernel_symbol_by_name(machine, "__lock_text_start", &kmap);
+ machine->lock.text_start = kmap->unmap_ip(kmap, sym->start);
+
+ sym = machine__find_kernel_symbol_by_name(machine, "__lock_text_end", &kmap);
+ machine->lock.text_end = kmap->unmap_ip(kmap, sym->start);
+ }
+
+ /* failed to get kernel symbols */
+ if (machine->sched.text_start == 1)
+ return false;
+
+ /* mutex and rwsem functions are in sched text section */
+ if (machine->sched.text_start <= addr && addr < machine->sched.text_end)
+ return true;
+
+ /* spinlock functions are in lock text section */
+ if (machine->lock.text_start <= addr && addr < machine->lock.text_end)
+ return true;
+
+ return false;
+}
struct maps *kmaps;
struct map *vmlinux_map;
u64 kernel_start;
+ struct {
+ u64 text_start;
+ u64 text_end;
+ } sched, lock;
pid_t *current_tid;
size_t current_tid_sz;
union { /* Tool specific area */
return machine ? machine->pid == HOST_KERNEL_ID : false;
}
+bool machine__is_lock_function(struct machine *machine, u64 addr);
bool machine__is(struct machine *machine, const char *arch);
bool machine__normalized_is(struct machine *machine, const char *arch);
int machine__nr_cpus_avail(struct machine *machine);
int machine__map_x86_64_entry_trampolines(struct machine *machine,
struct dso *kernel);
+int machine__resolve(struct machine *machine, struct addr_location *al,
+ struct perf_sample *sample);
+
#endif /* __PERF_MACHINE_H */
#include "strbuf.h"
#include "pmu.h"
#include "pmu-hybrid.h"
+#include "print-events.h"
#include "expr.h"
#include "rblist.h"
#include <string.h>
#include "util.h"
#include <asm/bug.h>
#include "cgroup.h"
+#include "util/hashmap.h"
struct metric_event *metricgroup__lookup(struct rblist *metric_events,
struct evsel *evsel,
match_metric(pe->metric_name, metric);
}
+/** struct mep - RB-tree node for building printing information. */
struct mep {
+ /** nd - RB-tree element. */
struct rb_node nd;
- const char *name;
- struct strlist *metrics;
+ /** @metric_group: Owned metric group name, separated others with ';'. */
+ char *metric_group;
+ const char *metric_name;
+ const char *metric_desc;
+ const char *metric_long_desc;
+ const char *metric_expr;
+ const char *metric_unit;
};
static int mep_cmp(struct rb_node *rb_node, const void *entry)
{
struct mep *a = container_of(rb_node, struct mep, nd);
struct mep *b = (struct mep *)entry;
+ int ret;
- return strcmp(a->name, b->name);
+ ret = strcmp(a->metric_group, b->metric_group);
+ if (ret)
+ return ret;
+
+ return strcmp(a->metric_name, b->metric_name);
}
-static struct rb_node *mep_new(struct rblist *rl __maybe_unused,
- const void *entry)
+static struct rb_node *mep_new(struct rblist *rl __maybe_unused, const void *entry)
{
struct mep *me = malloc(sizeof(struct mep));
if (!me)
return NULL;
+
memcpy(me, entry, sizeof(struct mep));
- me->name = strdup(me->name);
- if (!me->name)
- goto out_me;
- me->metrics = strlist__new(NULL, NULL);
- if (!me->metrics)
- goto out_name;
return &me->nd;
-out_name:
- zfree(&me->name);
-out_me:
+}
+
+static void mep_delete(struct rblist *rl __maybe_unused,
+ struct rb_node *nd)
+{
+ struct mep *me = container_of(nd, struct mep, nd);
+
+ zfree(&me->metric_group);
free(me);
- return NULL;
}
-static struct mep *mep_lookup(struct rblist *groups, const char *name)
+static struct mep *mep_lookup(struct rblist *groups, const char *metric_group,
+ const char *metric_name)
{
struct rb_node *nd;
struct mep me = {
- .name = name
+ .metric_group = strdup(metric_group),
+ .metric_name = metric_name,
};
nd = rblist__find(groups, &me);
- if (nd)
+ if (nd) {
+ free(me.metric_group);
return container_of(nd, struct mep, nd);
+ }
rblist__add_node(groups, &me);
nd = rblist__find(groups, &me);
if (nd)
return NULL;
}
-static void mep_delete(struct rblist *rl __maybe_unused,
- struct rb_node *nd)
-{
- struct mep *me = container_of(nd, struct mep, nd);
-
- strlist__delete(me->metrics);
- zfree(&me->name);
- free(me);
-}
-
-static void metricgroup__print_strlist(struct strlist *metrics, bool raw)
-{
- struct str_node *sn;
- int n = 0;
-
- strlist__for_each_entry (sn, metrics) {
- if (raw)
- printf("%s%s", n > 0 ? " " : "", sn->s);
- else
- printf(" %s\n", sn->s);
- n++;
- }
- if (raw)
- putchar('\n');
-}
-
-static int metricgroup__print_pmu_event(const struct pmu_event *pe,
- bool metricgroups, char *filter,
- bool raw, bool details,
- struct rblist *groups,
- struct strlist *metriclist)
+static int metricgroup__add_to_mep_groups(const struct pmu_event *pe,
+ struct rblist *groups)
{
const char *g;
char *omg, *mg;
- g = pe->metric_group;
- if (!g && pe->metric_name) {
- if (pe->name)
- return 0;
- g = "No_group";
- }
-
- if (!g)
- return 0;
-
- mg = strdup(g);
-
+ mg = strdup(pe->metric_group ?: "No_group");
if (!mg)
return -ENOMEM;
omg = mg;
while ((g = strsep(&mg, ";")) != NULL) {
struct mep *me;
- char *s;
g = skip_spaces(g);
- if (*g == 0)
- g = "No_group";
- if (filter && !strstr(g, filter))
- continue;
- if (raw)
- s = (char *)pe->metric_name;
- else {
- if (asprintf(&s, "%s\n%*s%s]",
- pe->metric_name, 8, "[", pe->desc) < 0)
- return -1;
- if (details) {
- if (asprintf(&s, "%s\n%*s%s]",
- s, 8, "[", pe->metric_expr) < 0)
- return -1;
- }
- }
-
- if (!s)
- continue;
+ if (strlen(g))
+ me = mep_lookup(groups, g, pe->metric_name);
+ else
+ me = mep_lookup(groups, "No_group", pe->metric_name);
- if (!metricgroups) {
- strlist__add(metriclist, s);
- } else {
- me = mep_lookup(groups, g);
- if (!me)
- continue;
- strlist__add(me->metrics, s);
+ if (me) {
+ me->metric_desc = pe->desc;
+ me->metric_long_desc = pe->long_desc;
+ me->metric_expr = pe->metric_expr;
+ me->metric_unit = pe->unit;
}
-
- if (!raw)
- free(s);
}
free(omg);
return 0;
}
-struct metricgroup_print_sys_idata {
- struct strlist *metriclist;
- char *filter;
- struct rblist *groups;
- bool metricgroups;
- bool raw;
- bool details;
-};
-
struct metricgroup_iter_data {
pmu_event_iter_fn fn;
void *data;
return d->fn(pe, table, d->data);
}
-
return 0;
}
-static int metricgroup__print_sys_event_iter(const struct pmu_event *pe,
- const struct pmu_events_table *table __maybe_unused,
- void *data)
-{
- struct metricgroup_print_sys_idata *d = data;
-
- return metricgroup__print_pmu_event(pe, d->metricgroups, d->filter, d->raw,
- d->details, d->groups, d->metriclist);
-}
-
-struct metricgroup_print_data {
- const char *pmu_name;
- struct strlist *metriclist;
- char *filter;
- struct rblist *groups;
- bool metricgroups;
- bool raw;
- bool details;
-};
-
-static int metricgroup__print_callback(const struct pmu_event *pe,
- const struct pmu_events_table *table __maybe_unused,
- void *vdata)
+static int metricgroup__add_to_mep_groups_callback(const struct pmu_event *pe,
+ const struct pmu_events_table *table __maybe_unused,
+ void *vdata)
{
- struct metricgroup_print_data *data = vdata;
+ struct rblist *groups = vdata;
- if (!pe->metric_expr)
- return 0;
-
- if (data->pmu_name && perf_pmu__is_hybrid(pe->pmu) && strcmp(data->pmu_name, pe->pmu))
+ if (!pe->metric_name)
return 0;
- return metricgroup__print_pmu_event(pe, data->metricgroups, data->filter,
- data->raw, data->details, data->groups,
- data->metriclist);
+ return metricgroup__add_to_mep_groups(pe, groups);
}
-void metricgroup__print(bool metrics, bool metricgroups, char *filter,
- bool raw, bool details, const char *pmu_name)
+void metricgroup__print(const struct print_callbacks *print_cb, void *print_state)
{
struct rblist groups;
- struct rb_node *node, *next;
- struct strlist *metriclist = NULL;
const struct pmu_events_table *table;
-
- if (!metricgroups) {
- metriclist = strlist__new(NULL, NULL);
- if (!metriclist)
- return;
- }
+ struct rb_node *node, *next;
rblist__init(&groups);
groups.node_new = mep_new;
groups.node_delete = mep_delete;
table = pmu_events_table__find();
if (table) {
- struct metricgroup_print_data data = {
- .pmu_name = pmu_name,
- .metriclist = metriclist,
- .metricgroups = metricgroups,
- .filter = filter,
- .raw = raw,
- .details = details,
- .groups = &groups,
- };
-
pmu_events_table_for_each_event(table,
- metricgroup__print_callback,
- &data);
+ metricgroup__add_to_mep_groups_callback,
+ &groups);
}
{
struct metricgroup_iter_data data = {
- .fn = metricgroup__print_sys_event_iter,
- .data = (void *) &(struct metricgroup_print_sys_idata){
- .metriclist = metriclist,
- .metricgroups = metricgroups,
- .filter = filter,
- .raw = raw,
- .details = details,
- .groups = &groups,
- },
+ .fn = metricgroup__add_to_mep_groups_callback,
+ .data = &groups,
};
-
pmu_for_each_sys_event(metricgroup__sys_event_iter, &data);
}
- if (!filter || !rblist__empty(&groups)) {
- if (metricgroups && !raw)
- printf("\nMetric Groups:\n\n");
- else if (metrics && !raw)
- printf("\nMetrics:\n\n");
- }
-
for (node = rb_first_cached(&groups.entries); node; node = next) {
struct mep *me = container_of(node, struct mep, nd);
- if (metricgroups)
- printf("%s%s%s", me->name, metrics && !raw ? ":" : "", raw ? " " : "\n");
- if (metrics)
- metricgroup__print_strlist(me->metrics, raw);
+ print_cb->print_metric(print_state,
+ me->metric_group,
+ me->metric_name,
+ me->metric_desc,
+ me->metric_long_desc,
+ me->metric_expr,
+ me->metric_unit);
next = rb_next(node);
rblist__remove_node(&groups, node);
}
- if (!metricgroups)
- metricgroup__print_strlist(metriclist, raw);
- strlist__delete(metriclist);
}
static const char *code_characters = ",-=@";
struct evlist;
struct evsel;
struct option;
+struct print_callbacks;
struct rblist;
struct cgroup;
bool metric_no_merge,
struct rblist *metric_events);
-void metricgroup__print(bool metrics, bool groups, char *filter,
- bool raw, bool details, const char *pmu_name);
+void metricgroup__print(const struct print_callbacks *print_cb, void *print_state);
bool metricgroup__has_metric(const char *metric);
int arch_get_runtimeparam(const struct pmu_event *pe __maybe_unused);
void metricgroup__rblist_exit(struct rblist *metric_events);
#define __PERF_MMAP_H 1
#include <internal/mmap.h>
-#include <linux/compiler.h>
-#include <linux/refcount.h>
#include <linux/types.h>
-#include <linux/ring_buffer.h>
#include <linux/bitops.h>
#include <perf/cpumap.h>
-#include <stdbool.h>
#ifdef HAVE_AIO_SUPPORT
#include <aio.h>
#endif
#include "auxtrace.h"
-#include "event.h"
#include "util/compress.h"
struct aiocb;
BRANCH_OPT("cond", PERF_SAMPLE_BRANCH_COND),
BRANCH_OPT("ind_jmp", PERF_SAMPLE_BRANCH_IND_JUMP),
BRANCH_OPT("call", PERF_SAMPLE_BRANCH_CALL),
+ BRANCH_OPT("no_flags", PERF_SAMPLE_BRANCH_NO_FLAGS),
+ BRANCH_OPT("no_cycles", PERF_SAMPLE_BRANCH_NO_CYCLES),
BRANCH_OPT("save_type", PERF_SAMPLE_BRANCH_TYPE_SAVE),
BRANCH_OPT("stack", PERF_SAMPLE_BRANCH_CALL_STACK),
+ BRANCH_OPT("hw_index", PERF_SAMPLE_BRANCH_HW_INDEX),
BRANCH_OPT("priv", PERF_SAMPLE_BRANCH_PRIV_SAVE),
BRANCH_END
};
evsel->core.own_cpus = perf_cpu_map__get(cpus);
evsel->core.requires_cpu = pmu ? pmu->is_uncore : false;
evsel->auto_merge_stats = auto_merge_stats;
+ evsel->pmu = pmu;
if (name)
evsel->name = strdup(name);
return ret;
}
+#ifdef HAVE_LIBTRACEEVENT
static void tracepoint_error(struct parse_events_error *e, int err,
const char *sys, const char *name)
{
closedir(events_dir);
return ret;
}
+#endif /* HAVE_LIBTRACEEVENT */
#ifdef HAVE_LIBBPF_SUPPORT
struct __add_bpf_event_param {
return config_term_common(attr, term, err);
}
+#ifdef HAVE_LIBTRACEEVENT
static int config_term_tracepoint(struct perf_event_attr *attr,
struct parse_events_term *term,
struct parse_events_error *err)
return 0;
}
+#endif
static int config_attr(struct perf_event_attr *attr,
struct list_head *head,
struct parse_events_error *err,
struct list_head *head_config)
{
+#ifdef HAVE_LIBTRACEEVENT
if (head_config) {
struct perf_event_attr attr;
else
return add_tracepoint_event(list, idx, sys, event,
err, head_config);
+#else
+ (void)list;
+ (void)idx;
+ (void)sys;
+ (void)event;
+ (void)head_config;
+ parse_events_error__handle(err, 0, strdup("unsupported tracepoint"),
+ strdup("libtraceevent is necessary for tracepoint support"));
+ return -1;
+#endif
}
int parse_events_add_numeric(struct parse_events_state *parse_state,
struct option;
struct perf_pmu;
-bool have_tracepoints(struct list_head *evlist);
bool is_event_supported(u8 type, u64 config);
const char *event_type(int type);
#include <errno.h>
#include <string.h>
#include "perf_regs.h"
-#include "event.h"
+#include "util/sample.h"
int __weak arch_sdt_arg_parse_op(char *old_op __maybe_unused,
char **new_op __maybe_unused)
#include "util/parse-events.h"
#include "util/pmu.h"
#include "util/pfm.h"
+#include "util/strbuf.h"
#include <string.h>
#include <linux/kernel.h>
};
static void
-print_attr_flags(pfm_event_attr_info_t *info)
+print_attr_flags(struct strbuf *buf, const pfm_event_attr_info_t *info)
{
- int n = 0;
+ if (info->is_dfl)
+ strbuf_addf(buf, "[default] ");
- if (info->is_dfl) {
- printf("[default] ");
- n++;
- }
-
- if (info->is_precise) {
- printf("[precise] ");
- n++;
- }
-
- if (!n)
- printf("- ");
+ if (info->is_precise)
+ strbuf_addf(buf, "[precise] ");
}
static void
-print_libpfm_events_detailed(pfm_event_info_t *info, bool long_desc)
+print_libpfm_event(const struct print_callbacks *print_cb, void *print_state,
+ const pfm_pmu_info_t *pinfo, const pfm_event_info_t *info,
+ struct strbuf *buf)
{
- pfm_event_attr_info_t ainfo;
- const char *src;
int j, ret;
+ char topic[80], name[80];
- ainfo.size = sizeof(ainfo);
+ strbuf_setlen(buf, 0);
+ snprintf(topic, sizeof(topic), "pfm %s", pinfo->name);
- printf(" %s\n", info->name);
- printf(" [%s]\n", info->desc);
- if (long_desc) {
- if (info->equiv)
- printf(" Equiv: %s\n", info->equiv);
+ snprintf(name, sizeof(name), "%s::%s", pinfo->name, info->name);
+ strbuf_addf(buf, "Code: 0x%"PRIx64"\n", info->code);
- printf(" Code : 0x%"PRIx64"\n", info->code);
- }
pfm_for_each_event_attr(j, info) {
- ret = pfm_get_event_attr_info(info->idx, j,
- PFM_OS_PERF_EVENT_EXT, &ainfo);
- if (ret != PFM_SUCCESS)
- continue;
-
- if (ainfo.type == PFM_ATTR_UMASK) {
- printf(" %s:%s\n", info->name, ainfo.name);
- printf(" [%s]\n", ainfo.desc);
- }
+ pfm_event_attr_info_t ainfo;
+ const char *src;
- if (!long_desc)
+ ainfo.size = sizeof(ainfo);
+ ret = pfm_get_event_attr_info(info->idx, j, PFM_OS_PERF_EVENT_EXT, &ainfo);
+ if (ret != PFM_SUCCESS)
continue;
if (ainfo.ctrl >= PFM_ATTR_CTRL_MAX)
src = srcs[ainfo.ctrl];
switch (ainfo.type) {
- case PFM_ATTR_UMASK:
- printf(" Umask : 0x%02"PRIx64" : %s: ",
- ainfo.code, src);
- print_attr_flags(&ainfo);
- putchar('\n');
+ case PFM_ATTR_UMASK: /* Ignore for now */
break;
case PFM_ATTR_MOD_BOOL:
- printf(" Modif : %s: [%s] : %s (boolean)\n", src,
- ainfo.name, ainfo.desc);
+ strbuf_addf(buf, " Modif: %s: [%s] : %s (boolean)\n", src,
+ ainfo.name, ainfo.desc);
break;
case PFM_ATTR_MOD_INTEGER:
- printf(" Modif : %s: [%s] : %s (integer)\n", src,
- ainfo.name, ainfo.desc);
+ strbuf_addf(buf, " Modif: %s: [%s] : %s (integer)\n", src,
+ ainfo.name, ainfo.desc);
break;
case PFM_ATTR_NONE:
case PFM_ATTR_RAW_UMASK:
case PFM_ATTR_MAX:
default:
- printf(" Attr : %s: [%s] : %s\n", src,
- ainfo.name, ainfo.desc);
+ strbuf_addf(buf, " Attr: %s: [%s] : %s\n", src,
+ ainfo.name, ainfo.desc);
}
}
-}
+ print_cb->print_event(print_state,
+ pinfo->name,
+ topic,
+ name, info->equiv,
+ /*scale_unit=*/NULL,
+ /*deprecated=*/NULL, "PFM event",
+ info->desc, /*long_desc=*/NULL,
+ /*encoding_desc=*/buf->buf,
+ /*metric_name=*/NULL, /*metric_expr=*/NULL);
-/*
- * list all pmu::event:umask, pmu::event
- * printed events may not be all valid combinations of umask for an event
- */
-static void
-print_libpfm_events_raw(pfm_pmu_info_t *pinfo, pfm_event_info_t *info)
-{
- pfm_event_attr_info_t ainfo;
- int j, ret;
- bool has_umask = false;
+ pfm_for_each_event_attr(j, info) {
+ pfm_event_attr_info_t ainfo;
+ const char *src;
- ainfo.size = sizeof(ainfo);
+ strbuf_setlen(buf, 0);
- pfm_for_each_event_attr(j, info) {
- ret = pfm_get_event_attr_info(info->idx, j,
- PFM_OS_PERF_EVENT_EXT, &ainfo);
+ ainfo.size = sizeof(ainfo);
+ ret = pfm_get_event_attr_info(info->idx, j, PFM_OS_PERF_EVENT_EXT, &ainfo);
if (ret != PFM_SUCCESS)
continue;
- if (ainfo.type != PFM_ATTR_UMASK)
- continue;
+ if (ainfo.ctrl >= PFM_ATTR_CTRL_MAX)
+ ainfo.ctrl = PFM_ATTR_CTRL_UNKNOWN;
- printf("%s::%s:%s\n", pinfo->name, info->name, ainfo.name);
- has_umask = true;
+ src = srcs[ainfo.ctrl];
+ if (ainfo.type == PFM_ATTR_UMASK) {
+ strbuf_addf(buf, "Umask: 0x%02"PRIx64" : %s: ",
+ ainfo.code, src);
+ print_attr_flags(buf, &ainfo);
+ snprintf(name, sizeof(name), "%s::%s:%s",
+ pinfo->name, info->name, ainfo.name);
+ print_cb->print_event(print_state,
+ pinfo->name,
+ topic,
+ name, /*alias=*/NULL,
+ /*scale_unit=*/NULL,
+ /*deprecated=*/NULL, "PFM event",
+ ainfo.desc, /*long_desc=*/NULL,
+ /*encoding_desc=*/buf->buf,
+ /*metric_name=*/NULL, /*metric_expr=*/NULL);
+ }
}
- if (!has_umask)
- printf("%s::%s\n", pinfo->name, info->name);
}
-void print_libpfm_events(bool name_only, bool long_desc)
+void print_libpfm_events(const struct print_callbacks *print_cb, void *print_state)
{
pfm_event_info_t info;
pfm_pmu_info_t pinfo;
- int i, p, ret;
+ int p, ret;
+ struct strbuf storage;
libpfm_initialize();
info.size = sizeof(info);
pinfo.size = sizeof(pinfo);
- if (!name_only)
- puts("\nList of pre-defined events (to be used in --pfm-events):\n");
+ strbuf_init(&storage, 2048);
pfm_for_all_pmus(p) {
- bool printed_pmu = false;
-
ret = pfm_get_pmu_info(p, &pinfo);
if (ret != PFM_SUCCESS)
continue;
if (pinfo.pmu == PFM_PMU_PERF_EVENT)
continue;
- for (i = pinfo.first_event; i != -1;
- i = pfm_get_event_next(i)) {
-
+ for (int i = pinfo.first_event; i != -1; i = pfm_get_event_next(i)) {
ret = pfm_get_event_info(i, PFM_OS_PERF_EVENT_EXT,
&info);
if (ret != PFM_SUCCESS)
continue;
- if (!name_only && !printed_pmu) {
- printf("%s:\n", pinfo.name);
- printed_pmu = true;
- }
-
- if (!name_only)
- print_libpfm_events_detailed(&info, long_desc);
- else
- print_libpfm_events_raw(&pinfo, &info);
+ print_libpfm_event(print_cb, print_state, &pinfo, &info, &storage);
}
- if (!name_only && printed_pmu)
- putchar('\n');
}
+ strbuf_release(&storage);
}
#ifndef __PERF_PFM_H
#define __PERF_PFM_H
+#include "print-events.h"
#include <subcmd/parse-options.h>
#ifdef HAVE_LIBPFM
int parse_libpfm_events_option(const struct option *opt, const char *str,
int unset);
-void print_libpfm_events(bool name_only, bool long_desc);
+void print_libpfm_events(const struct print_callbacks *print_cb, void *print_state);
#else
#include <linux/compiler.h>
return 0;
}
-static inline void print_libpfm_events(bool name_only __maybe_unused,
- bool long_desc __maybe_unused)
+static inline void print_libpfm_events(const struct print_callbacks *print_cb __maybe_unused,
+ void *print_state __maybe_unused)
{
}
#include "debug.h"
#include "evsel.h"
#include "pmu.h"
+#include "pmus.h"
#include "parse-events.h"
+#include "print-events.h"
#include "header.h"
#include "string2.h"
#include "strbuf.h"
struct perf_pmu perf_pmu__fake;
+/**
+ * struct perf_pmu_format - Values from a format file read from
+ * <sysfs>/devices/cpu/format/ held in struct perf_pmu.
+ *
+ * For example, the contents of <sysfs>/devices/cpu/format/event may be
+ * "config:0-7" and will be represented here as name="event",
+ * value=PERF_PMU_FORMAT_VALUE_CONFIG and bits 0 to 7 will be set.
+ */
struct perf_pmu_format {
+ /** @name: The modifier/file name. */
char *name;
+ /**
+ * @value : Which config value the format relates to. Supported values
+ * are from PERF_PMU_FORMAT_VALUE_CONFIG to
+ * PERF_PMU_FORMAT_VALUE_CONFIG_END.
+ */
int value;
+ /** @bits: Which config bits are set by this format value. */
DECLARE_BITMAP(bits, PERF_PMU_FORMAT_BITS);
+ /** @list: Element on list within struct perf_pmu. */
struct list_head list;
};
int perf_pmu_parse(struct list_head *list, char *name);
extern FILE *perf_pmu_in;
-static LIST_HEAD(pmus);
static bool hybrid_scanned;
/*
pmu->is_uncore = pmu_is_uncore(name);
if (pmu->is_uncore)
pmu->id = pmu_id(name);
- pmu->is_hybrid = is_hybrid;
pmu->max_precise = pmu_max_precise(name);
pmu_add_cpu_aliases(&aliases, pmu);
pmu_add_sys_aliases(&aliases, pmu);
list_splice(&aliases, &pmu->aliases);
list_add_tail(&pmu->list, &pmus);
- if (pmu->is_hybrid)
+ if (is_hybrid)
list_add_tail(&pmu->hybrid_list, &perf_pmu__hybrid_pmus);
pmu->default_config = perf_pmu__get_default_config(pmu);
{
struct perf_pmu *pmu = NULL;
+ if (evsel->pmu)
+ return evsel->pmu;
+
while ((pmu = perf_pmu__scan(pmu)) != NULL) {
if (pmu->type == evsel->core.attr.type)
break;
}
+ evsel->pmu = pmu;
return pmu;
}
return a - b;
}
-static char *format_alias(char *buf, int len, struct perf_pmu *pmu,
- struct perf_pmu_alias *alias)
+static char *format_alias(char *buf, int len, const struct perf_pmu *pmu,
+ const struct perf_pmu_alias *alias)
{
struct parse_events_term *term;
int used = snprintf(buf, len, "%s/%s", pmu->name, alias->name);
return buf;
}
-static char *format_alias_or(char *buf, int len, struct perf_pmu *pmu,
- struct perf_pmu_alias *alias)
-{
- snprintf(buf, len, "%s OR %s/%s/", alias->name, pmu->name, alias->name);
- return buf;
-}
-
+/** Struct for ordering events as output in perf list. */
struct sevent {
- char *name;
- char *desc;
- char *topic;
- char *str;
- char *pmu;
- char *metric_expr;
- char *metric_name;
- int is_cpu;
+ /** PMU for event. */
+ const struct perf_pmu *pmu;
+ /**
+ * Optional event for name, desc, etc. If not present then this is a
+ * selectable PMU and the event name is shown as "//".
+ */
+ const struct perf_pmu_alias *event;
+ /** Is the PMU for the CPU? */
+ bool is_cpu;
};
static int cmp_sevent(const void *a, const void *b)
{
const struct sevent *as = a;
const struct sevent *bs = b;
+ const char *a_pmu_name, *b_pmu_name;
+ const char *a_name = "//", *a_desc = NULL, *a_topic = "";
+ const char *b_name = "//", *b_desc = NULL, *b_topic = "";
int ret;
- /* Put extra events last */
- if (!!as->desc != !!bs->desc)
- return !!as->desc - !!bs->desc;
- if (as->topic && bs->topic) {
- int n = strcmp(as->topic, bs->topic);
-
- if (n)
- return n;
+ if (as->event) {
+ a_name = as->event->name;
+ a_desc = as->event->desc;
+ a_topic = as->event->topic ?: "";
}
-
- /* Order CPU core events to be first */
- if (as->is_cpu != bs->is_cpu)
- return bs->is_cpu - as->is_cpu;
-
- ret = strcmp(as->name, bs->name);
- if (!ret) {
- if (as->pmu && bs->pmu)
- return strcmp(as->pmu, bs->pmu);
+ if (bs->event) {
+ b_name = bs->event->name;
+ b_desc = bs->event->desc;
+ b_topic = bs->event->topic ?: "";
}
+ /* Put extra events last. */
+ if (!!a_desc != !!b_desc)
+ return !!a_desc - !!b_desc;
- return ret;
-}
+ /* Order by topics. */
+ ret = strcmp(a_topic, b_topic);
+ if (ret)
+ return ret;
-static void wordwrap(char *s, int start, int max, int corr)
-{
- int column = start;
- int n;
+ /* Order CPU core events to be first */
+ if (as->is_cpu != bs->is_cpu)
+ return as->is_cpu ? -1 : 1;
- while (*s) {
- int wlen = strcspn(s, " \t");
+ /* Order by PMU name. */
+ a_pmu_name = as->pmu->name ?: "";
+ b_pmu_name = bs->pmu->name ?: "";
+ ret = strcmp(a_pmu_name, b_pmu_name);
+ if (ret)
+ return ret;
- if (column + wlen >= max && column > start) {
- printf("\n%*s", start, "");
- column = start + corr;
- }
- n = printf("%s%.*s", column > start ? " " : "", wlen, s);
- if (n <= 0)
- break;
- s += wlen;
- column += n;
- s = skip_spaces(s);
- }
+ /* Order by event name. */
+ return strcmp(a_name, b_name);
}
bool is_pmu_core(const char *name)
static bool pmu_alias_is_duplicate(struct sevent *alias_a,
struct sevent *alias_b)
{
- /* Different names -> never duplicates */
- if (strcmp(alias_a->name, alias_b->name))
- return false;
+ const char *a_pmu_name, *b_pmu_name;
+ const char *a_name = alias_a->event ? alias_a->event->name : "//";
+ const char *b_name = alias_b->event ? alias_b->event->name : "//";
- /* Don't remove duplicates for hybrid PMUs */
- if (perf_pmu__is_hybrid(alias_a->pmu) &&
- perf_pmu__is_hybrid(alias_b->pmu))
+ /* Different names -> never duplicates */
+ if (strcmp(a_name, b_name))
return false;
- return true;
+ /* Don't remove duplicates for different PMUs */
+ a_pmu_name = alias_a->pmu->name ?: "";
+ b_pmu_name = alias_b->pmu->name ?: "";
+ return strcmp(a_pmu_name, b_pmu_name) == 0;
}
-void print_pmu_events(const char *event_glob, bool name_only, bool quiet_flag,
- bool long_desc, bool details_flag, bool deprecated,
- const char *pmu_name)
+void print_pmu_events(const struct print_callbacks *print_cb, void *print_state)
{
struct perf_pmu *pmu;
- struct perf_pmu_alias *alias;
+ struct perf_pmu_alias *event;
char buf[1024];
int printed = 0;
int len, j;
struct sevent *aliases;
- int numdesc = 0;
- int columns = pager_get_columns();
- char *topic = NULL;
pmu = NULL;
len = 0;
while ((pmu = perf_pmu__scan(pmu)) != NULL) {
- list_for_each_entry(alias, &pmu->aliases, list)
+ list_for_each_entry(event, &pmu->aliases, list)
len++;
if (pmu->selectable)
len++;
}
aliases = zalloc(sizeof(struct sevent) * len);
- if (!aliases)
- goto out_enomem;
+ if (!aliases) {
+ pr_err("FATAL: not enough memory to print PMU events\n");
+ return;
+ }
pmu = NULL;
j = 0;
while ((pmu = perf_pmu__scan(pmu)) != NULL) {
- if (pmu_name && perf_pmu__is_hybrid(pmu->name) &&
- strcmp(pmu_name, pmu->name)) {
- continue;
- }
-
- list_for_each_entry(alias, &pmu->aliases, list) {
- char *name = alias->desc ? alias->name :
- format_alias(buf, sizeof(buf), pmu, alias);
- bool is_cpu = is_pmu_core(pmu->name) ||
- perf_pmu__is_hybrid(pmu->name);
+ bool is_cpu = is_pmu_core(pmu->name) || perf_pmu__is_hybrid(pmu->name);
- if (alias->deprecated && !deprecated)
- continue;
-
- if (event_glob != NULL &&
- !(strglobmatch_nocase(name, event_glob) ||
- (!is_cpu && strglobmatch_nocase(alias->name,
- event_glob)) ||
- (alias->topic &&
- strglobmatch_nocase(alias->topic, event_glob))))
- continue;
-
- if (is_cpu && !name_only && !alias->desc)
- name = format_alias_or(buf, sizeof(buf), pmu, alias);
-
- aliases[j].name = name;
- if (is_cpu && !name_only && !alias->desc)
- aliases[j].name = format_alias_or(buf,
- sizeof(buf),
- pmu, alias);
- aliases[j].name = strdup(aliases[j].name);
- if (!aliases[j].name)
- goto out_enomem;
-
- aliases[j].desc = long_desc ? alias->long_desc :
- alias->desc;
- aliases[j].topic = alias->topic;
- aliases[j].str = alias->str;
- aliases[j].pmu = pmu->name;
- aliases[j].metric_expr = alias->metric_expr;
- aliases[j].metric_name = alias->metric_name;
+ list_for_each_entry(event, &pmu->aliases, list) {
+ aliases[j].event = event;
+ aliases[j].pmu = pmu;
aliases[j].is_cpu = is_cpu;
j++;
}
- if (pmu->selectable &&
- (event_glob == NULL || strglobmatch(pmu->name, event_glob))) {
- char *s;
- if (asprintf(&s, "%s//", pmu->name) < 0)
- goto out_enomem;
- aliases[j].name = s;
+ if (pmu->selectable) {
+ aliases[j].event = NULL;
+ aliases[j].pmu = pmu;
+ aliases[j].is_cpu = is_cpu;
j++;
}
}
len = j;
qsort(aliases, len, sizeof(struct sevent), cmp_sevent);
for (j = 0; j < len; j++) {
+ const char *name, *alias = NULL, *scale_unit = NULL,
+ *desc = NULL, *long_desc = NULL,
+ *encoding_desc = NULL, *topic = NULL,
+ *metric_name = NULL, *metric_expr = NULL;
+ bool deprecated = false;
+ size_t buf_used;
+
/* Skip duplicates */
if (j > 0 && pmu_alias_is_duplicate(&aliases[j], &aliases[j - 1]))
continue;
- if (name_only) {
- printf("%s ", aliases[j].name);
- continue;
- }
- if (aliases[j].desc && !quiet_flag) {
- if (numdesc++ == 0)
- printf("\n");
- if (aliases[j].topic && (!topic ||
- strcmp(topic, aliases[j].topic))) {
- printf("%s%s:\n", topic ? "\n" : "",
- aliases[j].topic);
- topic = aliases[j].topic;
+ if (!aliases[j].event) {
+ /* A selectable event. */
+ buf_used = snprintf(buf, sizeof(buf), "%s//", aliases[j].pmu->name) + 1;
+ name = buf;
+ } else {
+ if (aliases[j].event->desc) {
+ name = aliases[j].event->name;
+ buf_used = 0;
+ } else {
+ name = format_alias(buf, sizeof(buf), aliases[j].pmu,
+ aliases[j].event);
+ if (aliases[j].is_cpu) {
+ alias = name;
+ name = aliases[j].event->name;
+ }
+ buf_used = strlen(buf) + 1;
}
- printf(" %-50s\n", aliases[j].name);
- printf("%*s", 8, "[");
- wordwrap(aliases[j].desc, 8, columns, 0);
- printf("]\n");
- if (details_flag) {
- printf("%*s%s/%s/ ", 8, "", aliases[j].pmu, aliases[j].str);
- if (aliases[j].metric_name)
- printf(" MetricName: %s", aliases[j].metric_name);
- if (aliases[j].metric_expr)
- printf(" MetricExpr: %s", aliases[j].metric_expr);
- putchar('\n');
+ if (strlen(aliases[j].event->unit) || aliases[j].event->scale != 1.0) {
+ scale_unit = buf + buf_used;
+ buf_used += snprintf(buf + buf_used, sizeof(buf) - buf_used,
+ "%G%s", aliases[j].event->scale,
+ aliases[j].event->unit) + 1;
}
- } else
- printf(" %-50s [Kernel PMU event]\n", aliases[j].name);
- printed++;
+ desc = aliases[j].event->desc;
+ long_desc = aliases[j].event->long_desc;
+ topic = aliases[j].event->topic;
+ encoding_desc = buf + buf_used;
+ buf_used += snprintf(buf + buf_used, sizeof(buf) - buf_used,
+ "%s/%s/", aliases[j].pmu->name,
+ aliases[j].event->str) + 1;
+ metric_name = aliases[j].event->metric_name;
+ metric_expr = aliases[j].event->metric_expr;
+ deprecated = aliases[j].event->deprecated;
+ }
+ print_cb->print_event(print_state,
+ aliases[j].pmu->name,
+ topic,
+ name,
+ alias,
+ scale_unit,
+ deprecated,
+ "Kernel PMU event",
+ desc,
+ long_desc,
+ encoding_desc,
+ metric_name,
+ metric_expr);
}
if (printed && pager_in_use())
printf("\n");
-out_free:
- for (j = 0; j < len; j++)
- zfree(&aliases[j].name);
+
zfree(&aliases);
return;
-
-out_enomem:
- printf("FATAL: not enough memory to print PMU events\n");
- if (aliases)
- goto out_free;
}
bool pmu_have_event(const char *pname, const char *name)
struct evsel_config_term;
struct perf_cpu_map;
+struct print_callbacks;
enum {
PERF_PMU_FORMAT_VALUE_CONFIG,
struct list_head list;
};
+/**
+ * struct perf_pmu - hi
+ */
struct perf_pmu {
+ /** @name: The name of the PMU such as "cpu". */
char *name;
+ /**
+ * @alias_name: Optional alternate name for the PMU determined in
+ * architecture specific code.
+ */
char *alias_name;
+ /**
+ * @id: Optional PMU identifier read from
+ * <sysfs>/bus/event_source/devices/<name>/identifier.
+ */
char *id;
+ /**
+ * @type: Perf event attributed type value, read from
+ * <sysfs>/bus/event_source/devices/<name>/type.
+ */
__u32 type;
+ /**
+ * @selectable: Can the PMU name be selected as if it were an event?
+ */
bool selectable;
+ /**
+ * @is_uncore: Is the PMU not within the CPU core? Determined by the
+ * presence of <sysfs>/bus/event_source/devices/<name>/cpumask.
+ */
bool is_uncore;
- bool is_hybrid;
+ /**
+ * @auxtrace: Are events auxiliary events? Determined in architecture
+ * specific code.
+ */
bool auxtrace;
+ /**
+ * @max_precise: Number of levels of :ppp precision supported by the
+ * PMU, read from
+ * <sysfs>/bus/event_source/devices/<name>/caps/max_precise.
+ */
int max_precise;
+ /**
+ * @default_config: Optional default perf_event_attr determined in
+ * architecture specific code.
+ */
struct perf_event_attr *default_config;
+ /**
+ * @cpus: Empty or the contents of either of:
+ * <sysfs>/bus/event_source/devices/<name>/cpumask.
+ * <sysfs>/bus/event_source/devices/<cpu>/cpus.
+ */
struct perf_cpu_map *cpus;
- struct list_head format; /* HEAD struct perf_pmu_format -> list */
- struct list_head aliases; /* HEAD struct perf_pmu_alias -> list */
+ /**
+ * @format: Holds the contents of files read from
+ * <sysfs>/bus/event_source/devices/<name>/format/. The contents specify
+ * which event parameter changes what config, config1 or config2 bits.
+ */
+ struct list_head format;
+ /**
+ * @aliases: List of struct perf_pmu_alias. Each alias corresponds to an
+ * event read from <sysfs>/bus/event_source/devices/<name>/events/ or
+ * from json events in pmu-events.c.
+ */
+ struct list_head aliases;
+ /** @caps_initialized: Has the list caps been initialized? */
bool caps_initialized;
+ /** @nr_caps: The length of the list caps. */
u32 nr_caps;
- struct list_head caps; /* HEAD struct perf_pmu_caps -> list */
- struct list_head list; /* ELEM */
+ /**
+ * @caps: Holds the contents of files read from
+ * <sysfs>/bus/event_source/devices/<name>/caps/.
+ *
+ * The contents are pairs of the filename with the value of its
+ * contents, for example, max_precise (see above) may have a value of 3.
+ */
+ struct list_head caps;
+ /** @list: Element on pmus list in pmu.c. */
+ struct list_head list;
+ /** @hybrid_list: Element on perf_pmu__hybrid_pmus. */
struct list_head hybrid_list;
+ /**
+ * @missing_features: Features to inhibit when events on this PMU are
+ * opened.
+ */
struct {
+ /**
+ * @exclude_guest: Disables perf_event_attr exclude_guest and
+ * exclude_host.
+ */
bool exclude_guest;
} missing_features;
};
+/** @perf_pmu__fake: A special global PMU used for testing. */
extern struct perf_pmu perf_pmu__fake;
struct perf_pmu_info {
#define UNIT_MAX_LEN 31 /* max length for event unit name */
+/**
+ * struct perf_pmu_alias - An event either read from sysfs or builtin in
+ * pmu-events.c, created by parsing the pmu-events json files.
+ */
struct perf_pmu_alias {
+ /** @name: Name of the event like "mem-loads". */
char *name;
+ /** @desc: Optional short description of the event. */
char *desc;
+ /** @long_desc: Optional long description. */
char *long_desc;
+ /**
+ * @topic: Optional topic such as cache or pipeline, particularly for
+ * json events.
+ */
char *topic;
+ /**
+ * @str: Comma separated parameter list like
+ * "event=0xcd,umask=0x1,ldlat=0x3".
+ */
char *str;
- struct list_head terms; /* HEAD struct parse_events_term -> list */
- struct list_head list; /* ELEM */
+ /** @terms: Owned list of the original parsed parameters. */
+ struct list_head terms;
+ /** @list: List element of struct perf_pmu aliases. */
+ struct list_head list;
+ /** @unit: Units for the event, such as bytes or cache lines. */
char unit[UNIT_MAX_LEN+1];
+ /** @scale: Value to scale read counter values by. */
double scale;
+ /**
+ * @per_pkg: Does the file
+ * <sysfs>/bus/event_source/devices/<pmu_name>/events/<name>.per-pkg or
+ * equivalent json value exist and have the value 1.
+ */
bool per_pkg;
+ /**
+ * @snapshot: Does the file
+ * <sysfs>/bus/event_source/devices/<pmu_name>/events/<name>.snapshot
+ * exist and have the value 1.
+ */
bool snapshot;
+ /**
+ * @deprecated: Is the event hidden and so not shown in perf list by
+ * default.
+ */
bool deprecated;
+ /**
+ * @metric_expr: A metric expression associated with an event. Doing
+ * this makes little sense due to scale and unit applying to both.
+ */
char *metric_expr;
+ /** @metric_name: A name for the metric. unit applying to both. */
char *metric_name;
+ /** @pmu_name: The name copied from struct perf_pmu. */
char *pmu_name;
};
struct perf_pmu *perf_pmu__scan(struct perf_pmu *pmu);
bool is_pmu_core(const char *name);
-void print_pmu_events(const char *event_glob, bool name_only, bool quiet,
- bool long_desc, bool details_flag,
- bool deprecated, const char *pmu_name);
+void print_pmu_events(const struct print_callbacks *print_cb, void *print_state);
bool pmu_have_event(const char *pname, const char *name);
int perf_pmu__scan_file(struct perf_pmu *pmu, const char *name, const char *fmt, ...) __scanf(3, 4);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/list.h>
+#include <pmus.h>
+
+LIST_HEAD(pmus);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __PMUS_H
+#define __PMUS_H
+
+extern struct list_head pmus;
+
+#define perf_pmus__for_each_pmu(pmu) list_for_each_entry(pmu, &pmus, list)
+
+#endif /* __PMUS_H */
#define MAX_NAME_LEN 100
+/** Strings corresponding to enum perf_type_id. */
static const char * const event_type_descriptors[] = {
"Hardware event",
"Software event",
},
};
-static int cmp_string(const void *a, const void *b)
-{
- const char * const *as = a;
- const char * const *bs = b;
-
- return strcmp(*as, *bs);
-}
-
/*
* Print the events from <debugfs_mount_point>/tracing/events
*/
-void print_tracepoint_events(const char *subsys_glob,
- const char *event_glob, bool name_only)
+void print_tracepoint_events(const struct print_callbacks *print_cb, void *print_state)
{
- DIR *sys_dir, *evt_dir;
- struct dirent *sys_dirent, *evt_dirent;
- char evt_path[MAXPATHLEN];
- char *dir_path;
- char **evt_list = NULL;
- unsigned int evt_i = 0, evt_num = 0;
- bool evt_num_known = false;
-
-restart:
- sys_dir = tracing_events__opendir();
- if (!sys_dir)
- return;
-
- if (evt_num_known) {
- evt_list = zalloc(sizeof(char *) * evt_num);
- if (!evt_list)
- goto out_close_sys_dir;
- }
-
- for_each_subsystem(sys_dir, sys_dirent) {
- if (subsys_glob != NULL &&
- !strglobmatch(sys_dirent->d_name, subsys_glob))
+ struct dirent **sys_namelist = NULL;
+ int sys_items = tracing_events__scandir_alphasort(&sys_namelist);
+
+ for (int i = 0; i < sys_items; i++) {
+ struct dirent *sys_dirent = sys_namelist[i];
+ struct dirent **evt_namelist = NULL;
+ char *dir_path;
+ int evt_items;
+
+ if (sys_dirent->d_type != DT_DIR ||
+ !strcmp(sys_dirent->d_name, ".") ||
+ !strcmp(sys_dirent->d_name, ".."))
continue;
dir_path = get_events_file(sys_dirent->d_name);
if (!dir_path)
continue;
- evt_dir = opendir(dir_path);
- if (!evt_dir)
- goto next;
- for_each_event(dir_path, evt_dir, evt_dirent) {
- if (event_glob != NULL &&
- !strglobmatch(evt_dirent->d_name, event_glob))
+ evt_items = scandir(dir_path, &evt_namelist, NULL, alphasort);
+ for (int j = 0; j < evt_items; j++) {
+ struct dirent *evt_dirent = evt_namelist[j];
+ char evt_path[MAXPATHLEN];
+
+ if (evt_dirent->d_type != DT_DIR ||
+ !strcmp(evt_dirent->d_name, ".") ||
+ !strcmp(evt_dirent->d_name, ".."))
continue;
- if (!evt_num_known) {
- evt_num++;
+ if (tp_event_has_id(dir_path, evt_dirent) != 0)
continue;
- }
snprintf(evt_path, MAXPATHLEN, "%s:%s",
sys_dirent->d_name, evt_dirent->d_name);
-
- evt_list[evt_i] = strdup(evt_path);
- if (evt_list[evt_i] == NULL) {
- put_events_file(dir_path);
- goto out_close_evt_dir;
- }
- evt_i++;
- }
- closedir(evt_dir);
-next:
- put_events_file(dir_path);
- }
- closedir(sys_dir);
-
- if (!evt_num_known) {
- evt_num_known = true;
- goto restart;
- }
- qsort(evt_list, evt_num, sizeof(char *), cmp_string);
- evt_i = 0;
- while (evt_i < evt_num) {
- if (name_only) {
- printf("%s ", evt_list[evt_i++]);
- continue;
+ print_cb->print_event(print_state,
+ /*topic=*/NULL,
+ /*pmu_name=*/NULL,
+ evt_path,
+ /*event_alias=*/NULL,
+ /*scale_unit=*/NULL,
+ /*deprecated=*/false,
+ "Tracepoint event",
+ /*desc=*/NULL,
+ /*long_desc=*/NULL,
+ /*encoding_desc=*/NULL,
+ /*metric_name=*/NULL,
+ /*metric_expr=*/NULL);
}
- printf(" %-50s [%s]\n", evt_list[evt_i++],
- event_type_descriptors[PERF_TYPE_TRACEPOINT]);
+ free(dir_path);
+ free(evt_namelist);
}
- if (evt_num && pager_in_use())
- printf("\n");
-
-out_free:
- evt_num = evt_i;
- for (evt_i = 0; evt_i < evt_num; evt_i++)
- zfree(&evt_list[evt_i]);
- zfree(&evt_list);
- return;
-
-out_close_evt_dir:
- closedir(evt_dir);
-out_close_sys_dir:
- closedir(sys_dir);
-
- printf("FATAL: not enough memory to print %s\n",
- event_type_descriptors[PERF_TYPE_TRACEPOINT]);
- if (evt_list)
- goto out_free;
+ free(sys_namelist);
}
-void print_sdt_events(const char *subsys_glob, const char *event_glob,
- bool name_only)
+void print_sdt_events(const struct print_callbacks *print_cb, void *print_state)
{
- struct probe_cache *pcache;
- struct probe_cache_entry *ent;
struct strlist *bidlist, *sdtlist;
- struct strlist_config cfg = {.dont_dupstr = true};
- struct str_node *nd, *nd2;
- char *buf, *path, *ptr = NULL;
- bool show_detail = false;
- int ret;
-
- sdtlist = strlist__new(NULL, &cfg);
+ struct str_node *bid_nd, *sdt_name, *next_sdt_name;
+ const char *last_sdt_name = NULL;
+
+ /*
+ * The implicitly sorted sdtlist will hold the tracepoint name followed
+ * by @<buildid>. If the tracepoint name is unique (determined by
+ * looking at the adjacent nodes) the @<buildid> is dropped otherwise
+ * the executable path and buildid are added to the name.
+ */
+ sdtlist = strlist__new(NULL, NULL);
if (!sdtlist) {
pr_debug("Failed to allocate new strlist for SDT\n");
return;
pr_debug("Failed to get buildids: %d\n", errno);
return;
}
- strlist__for_each_entry(nd, bidlist) {
- pcache = probe_cache__new(nd->s, NULL);
+ strlist__for_each_entry(bid_nd, bidlist) {
+ struct probe_cache *pcache;
+ struct probe_cache_entry *ent;
+
+ pcache = probe_cache__new(bid_nd->s, NULL);
if (!pcache)
continue;
list_for_each_entry(ent, &pcache->entries, node) {
- if (!ent->sdt)
- continue;
- if (subsys_glob &&
- !strglobmatch(ent->pev.group, subsys_glob))
- continue;
- if (event_glob &&
- !strglobmatch(ent->pev.event, event_glob))
- continue;
- ret = asprintf(&buf, "%s:%s@%s", ent->pev.group,
- ent->pev.event, nd->s);
- if (ret > 0)
- strlist__add(sdtlist, buf);
+ char buf[1024];
+
+ snprintf(buf, sizeof(buf), "%s:%s@%s",
+ ent->pev.group, ent->pev.event, bid_nd->s);
+ strlist__add(sdtlist, buf);
}
probe_cache__delete(pcache);
}
strlist__delete(bidlist);
- strlist__for_each_entry(nd, sdtlist) {
- buf = strchr(nd->s, '@');
- if (buf)
- *(buf++) = '\0';
- if (name_only) {
- printf("%s ", nd->s);
- continue;
- }
- nd2 = strlist__next(nd);
- if (nd2) {
- ptr = strchr(nd2->s, '@');
- if (ptr)
- *ptr = '\0';
- if (strcmp(nd->s, nd2->s) == 0)
- show_detail = true;
+ strlist__for_each_entry(sdt_name, sdtlist) {
+ bool show_detail = false;
+ char *bid = strchr(sdt_name->s, '@');
+ char *evt_name = NULL;
+
+ if (bid)
+ *(bid++) = '\0';
+
+ if (last_sdt_name && !strcmp(last_sdt_name, sdt_name->s)) {
+ show_detail = true;
+ } else {
+ next_sdt_name = strlist__next(sdt_name);
+ if (next_sdt_name) {
+ char *bid2 = strchr(next_sdt_name->s, '@');
+
+ if (bid2)
+ *bid2 = '\0';
+ if (strcmp(sdt_name->s, next_sdt_name->s) == 0)
+ show_detail = true;
+ if (bid2)
+ *bid2 = '@';
+ }
}
+ last_sdt_name = sdt_name->s;
+
if (show_detail) {
- path = build_id_cache__origname(buf);
- ret = asprintf(&buf, "%s@%s(%.12s)", nd->s, path, buf);
- if (ret > 0) {
- printf(" %-50s [%s]\n", buf, "SDT event");
- free(buf);
+ char *path = build_id_cache__origname(bid);
+
+ if (path) {
+ if (asprintf(&evt_name, "%s@%s(%.12s)", sdt_name->s, path, bid) < 0)
+ evt_name = NULL;
+ free(path);
}
- free(path);
- } else
- printf(" %-50s [%s]\n", nd->s, "SDT event");
- if (nd2) {
- if (strcmp(nd->s, nd2->s) != 0)
- show_detail = false;
- if (ptr)
- *ptr = '@';
}
+ print_cb->print_event(print_state,
+ /*topic=*/NULL,
+ /*pmu_name=*/NULL,
+ evt_name ?: sdt_name->s,
+ /*event_alias=*/NULL,
+ /*deprecated=*/false,
+ /*scale_unit=*/NULL,
+ "SDT event",
+ /*desc=*/NULL,
+ /*long_desc=*/NULL,
+ /*encoding_desc=*/NULL,
+ /*metric_name=*/NULL,
+ /*metric_expr=*/NULL);
+
+ free(evt_name);
}
strlist__delete(sdtlist);
}
-int print_hwcache_events(const char *event_glob, bool name_only)
+int print_hwcache_events(const struct print_callbacks *print_cb, void *print_state)
{
- unsigned int type, op, i, evt_i = 0, evt_num = 0, npmus = 0;
- char name[64], new_name[128];
- char **evt_list = NULL, **evt_pmus = NULL;
- bool evt_num_known = false;
- struct perf_pmu *pmu = NULL;
-
- if (perf_pmu__has_hybrid()) {
- npmus = perf_pmu__hybrid_pmu_num();
- evt_pmus = zalloc(sizeof(char *) * npmus);
- if (!evt_pmus)
- goto out_enomem;
- }
+ struct strlist *evt_name_list = strlist__new(NULL, NULL);
+ struct str_node *nd;
-restart:
- if (evt_num_known) {
- evt_list = zalloc(sizeof(char *) * evt_num);
- if (!evt_list)
- goto out_enomem;
+ if (!evt_name_list) {
+ pr_debug("Failed to allocate new strlist for hwcache events\n");
+ return -ENOMEM;
}
-
- for (type = 0; type < PERF_COUNT_HW_CACHE_MAX; type++) {
- for (op = 0; op < PERF_COUNT_HW_CACHE_OP_MAX; op++) {
+ for (int type = 0; type < PERF_COUNT_HW_CACHE_MAX; type++) {
+ for (int op = 0; op < PERF_COUNT_HW_CACHE_OP_MAX; op++) {
/* skip invalid cache type */
if (!evsel__is_cache_op_valid(type, op))
continue;
- for (i = 0; i < PERF_COUNT_HW_CACHE_RESULT_MAX; i++) {
- unsigned int hybrid_supported = 0, j;
- bool supported;
+ for (int i = 0; i < PERF_COUNT_HW_CACHE_RESULT_MAX; i++) {
+ struct perf_pmu *pmu = NULL;
+ char name[64];
__evsel__hw_cache_type_op_res_name(type, op, i, name, sizeof(name));
- if (event_glob != NULL && !strglobmatch(name, event_glob))
- continue;
-
if (!perf_pmu__has_hybrid()) {
- if (!is_event_supported(PERF_TYPE_HW_CACHE,
- type | (op << 8) | (i << 16))) {
- continue;
- }
- } else {
- perf_pmu__for_each_hybrid_pmu(pmu) {
- if (!evt_num_known) {
- evt_num++;
- continue;
- }
-
- supported = is_event_supported(
- PERF_TYPE_HW_CACHE,
- type | (op << 8) | (i << 16) |
- ((__u64)pmu->type << PERF_PMU_TYPE_SHIFT));
- if (supported) {
- snprintf(new_name, sizeof(new_name),
- "%s/%s/", pmu->name, name);
- evt_pmus[hybrid_supported] =
- strdup(new_name);
- hybrid_supported++;
- }
- }
-
- if (hybrid_supported == 0)
- continue;
- }
-
- if (!evt_num_known) {
- evt_num++;
+ if (is_event_supported(PERF_TYPE_HW_CACHE,
+ type | (op << 8) | (i << 16)))
+ strlist__add(evt_name_list, name);
continue;
}
-
- if ((hybrid_supported == 0) ||
- (hybrid_supported == npmus)) {
- evt_list[evt_i] = strdup(name);
- if (npmus > 0) {
- for (j = 0; j < npmus; j++)
- zfree(&evt_pmus[j]);
- }
- } else {
- for (j = 0; j < hybrid_supported; j++) {
- evt_list[evt_i++] = evt_pmus[j];
- evt_pmus[j] = NULL;
+ perf_pmu__for_each_hybrid_pmu(pmu) {
+ if (is_event_supported(PERF_TYPE_HW_CACHE,
+ type | (op << 8) | (i << 16) |
+ ((__u64)pmu->type << PERF_PMU_TYPE_SHIFT))) {
+ char new_name[128];
+ snprintf(new_name, sizeof(new_name),
+ "%s/%s/", pmu->name, name);
+ strlist__add(evt_name_list, new_name);
}
- continue;
}
-
- if (evt_list[evt_i] == NULL)
- goto out_enomem;
- evt_i++;
}
}
}
- if (!evt_num_known) {
- evt_num_known = true;
- goto restart;
- }
-
- for (evt_i = 0; evt_i < evt_num; evt_i++) {
- if (!evt_list[evt_i])
- break;
+ strlist__for_each_entry(nd, evt_name_list) {
+ print_cb->print_event(print_state,
+ "cache",
+ /*pmu_name=*/NULL,
+ nd->s,
+ /*event_alias=*/NULL,
+ /*scale_unit=*/NULL,
+ /*deprecated=*/false,
+ event_type_descriptors[PERF_TYPE_HW_CACHE],
+ /*desc=*/NULL,
+ /*long_desc=*/NULL,
+ /*encoding_desc=*/NULL,
+ /*metric_name=*/NULL,
+ /*metric_expr=*/NULL);
}
-
- evt_num = evt_i;
- qsort(evt_list, evt_num, sizeof(char *), cmp_string);
- evt_i = 0;
- while (evt_i < evt_num) {
- if (name_only) {
- printf("%s ", evt_list[evt_i++]);
- continue;
- }
- printf(" %-50s [%s]\n", evt_list[evt_i++],
- event_type_descriptors[PERF_TYPE_HW_CACHE]);
- }
- if (evt_num && pager_in_use())
- printf("\n");
-
-out_free:
- evt_num = evt_i;
- for (evt_i = 0; evt_i < evt_num; evt_i++)
- zfree(&evt_list[evt_i]);
- zfree(&evt_list);
-
- for (evt_i = 0; evt_i < npmus; evt_i++)
- zfree(&evt_pmus[evt_i]);
- zfree(&evt_pmus);
- return evt_num;
-
-out_enomem:
- printf("FATAL: not enough memory to print %s\n",
- event_type_descriptors[PERF_TYPE_HW_CACHE]);
- if (evt_list)
- goto out_free;
- return evt_num;
+ strlist__delete(evt_name_list);
+ return 0;
}
-static void print_tool_event(const struct event_symbol *syms, const char *event_glob,
- bool name_only)
+void print_tool_events(const struct print_callbacks *print_cb, void *print_state)
{
- if (syms->symbol == NULL)
- return;
-
- if (event_glob && !(strglobmatch(syms->symbol, event_glob) ||
- (syms->alias && strglobmatch(syms->alias, event_glob))))
- return;
-
- if (name_only)
- printf("%s ", syms->symbol);
- else {
- char name[MAX_NAME_LEN];
-
- if (syms->alias && strlen(syms->alias))
- snprintf(name, MAX_NAME_LEN, "%s OR %s", syms->symbol, syms->alias);
- else
- strlcpy(name, syms->symbol, MAX_NAME_LEN);
- printf(" %-50s [%s]\n", name, "Tool event");
+ // Start at 1 because the first enum entry means no tool event.
+ for (int i = 1; i < PERF_TOOL_MAX; ++i) {
+ print_cb->print_event(print_state,
+ "tool",
+ /*pmu_name=*/NULL,
+ event_symbols_tool[i].symbol,
+ event_symbols_tool[i].alias,
+ /*scale_unit=*/NULL,
+ /*deprecated=*/false,
+ "Tool event",
+ /*desc=*/NULL,
+ /*long_desc=*/NULL,
+ /*encoding_desc=*/NULL,
+ /*metric_name=*/NULL,
+ /*metric_expr=*/NULL);
}
}
-void print_tool_events(const char *event_glob, bool name_only)
+void print_symbol_events(const struct print_callbacks *print_cb, void *print_state,
+ unsigned int type, const struct event_symbol *syms,
+ unsigned int max)
{
- // Start at 1 because the first enum entry means no tool event.
- for (int i = 1; i < PERF_TOOL_MAX; ++i)
- print_tool_event(event_symbols_tool + i, event_glob, name_only);
-
- if (pager_in_use())
- printf("\n");
-}
+ struct strlist *evt_name_list = strlist__new(NULL, NULL);
+ struct str_node *nd;
-void print_symbol_events(const char *event_glob, unsigned int type,
- struct event_symbol *syms, unsigned int max,
- bool name_only)
-{
- unsigned int i, evt_i = 0, evt_num = 0;
- char name[MAX_NAME_LEN];
- char **evt_list = NULL;
- bool evt_num_known = false;
-
-restart:
- if (evt_num_known) {
- evt_list = zalloc(sizeof(char *) * evt_num);
- if (!evt_list)
- goto out_enomem;
- syms -= max;
+ if (!evt_name_list) {
+ pr_debug("Failed to allocate new strlist for symbol events\n");
+ return;
}
-
- for (i = 0; i < max; i++, syms++) {
+ for (unsigned int i = 0; i < max; i++) {
/*
* New attr.config still not supported here, the latest
* example was PERF_COUNT_SW_CGROUP_SWITCHES
*/
- if (syms->symbol == NULL)
- continue;
-
- if (event_glob != NULL && !(strglobmatch(syms->symbol, event_glob) ||
- (syms->alias && strglobmatch(syms->alias, event_glob))))
+ if (syms[i].symbol == NULL)
continue;
if (!is_event_supported(type, i))
continue;
- if (!evt_num_known) {
- evt_num++;
- continue;
- }
-
- if (!name_only && strlen(syms->alias))
- snprintf(name, MAX_NAME_LEN, "%s OR %s", syms->symbol, syms->alias);
- else
- strlcpy(name, syms->symbol, MAX_NAME_LEN);
+ if (strlen(syms[i].alias)) {
+ char name[MAX_NAME_LEN];
- evt_list[evt_i] = strdup(name);
- if (evt_list[evt_i] == NULL)
- goto out_enomem;
- evt_i++;
+ snprintf(name, MAX_NAME_LEN, "%s OR %s", syms[i].symbol, syms[i].alias);
+ strlist__add(evt_name_list, name);
+ } else
+ strlist__add(evt_name_list, syms[i].symbol);
}
- if (!evt_num_known) {
- evt_num_known = true;
- goto restart;
- }
- qsort(evt_list, evt_num, sizeof(char *), cmp_string);
- evt_i = 0;
- while (evt_i < evt_num) {
- if (name_only) {
- printf("%s ", evt_list[evt_i++]);
- continue;
+ strlist__for_each_entry(nd, evt_name_list) {
+ char *alias = strstr(nd->s, " OR ");
+
+ if (alias) {
+ *alias = '\0';
+ alias += 4;
}
- printf(" %-50s [%s]\n", evt_list[evt_i++], event_type_descriptors[type]);
+ print_cb->print_event(print_state,
+ /*topic=*/NULL,
+ /*pmu_name=*/NULL,
+ nd->s,
+ alias,
+ /*scale_unit=*/NULL,
+ /*deprecated=*/false,
+ event_type_descriptors[type],
+ /*desc=*/NULL,
+ /*long_desc=*/NULL,
+ /*encoding_desc=*/NULL,
+ /*metric_name=*/NULL,
+ /*metric_expr=*/NULL);
}
- if (evt_num && pager_in_use())
- printf("\n");
-
-out_free:
- evt_num = evt_i;
- for (evt_i = 0; evt_i < evt_num; evt_i++)
- zfree(&evt_list[evt_i]);
- zfree(&evt_list);
- return;
-
-out_enomem:
- printf("FATAL: not enough memory to print %s\n", event_type_descriptors[type]);
- if (evt_list)
- goto out_free;
+ strlist__delete(evt_name_list);
}
/*
* Print the help text for the event symbols:
*/
-void print_events(const char *event_glob, bool name_only, bool quiet_flag,
- bool long_desc, bool details_flag, bool deprecated,
- const char *pmu_name)
+void print_events(const struct print_callbacks *print_cb, void *print_state)
{
- print_symbol_events(event_glob, PERF_TYPE_HARDWARE,
- event_symbols_hw, PERF_COUNT_HW_MAX, name_only);
-
- print_symbol_events(event_glob, PERF_TYPE_SOFTWARE,
- event_symbols_sw, PERF_COUNT_SW_MAX, name_only);
- print_tool_events(event_glob, name_only);
-
- print_hwcache_events(event_glob, name_only);
-
- print_pmu_events(event_glob, name_only, quiet_flag, long_desc,
- details_flag, deprecated, pmu_name);
-
- if (event_glob != NULL)
- return;
-
- if (!name_only) {
- printf(" %-50s [%s]\n",
- "rNNN",
- event_type_descriptors[PERF_TYPE_RAW]);
- printf(" %-50s [%s]\n",
- "cpu/t1=v1[,t2=v2,t3 ...]/modifier",
- event_type_descriptors[PERF_TYPE_RAW]);
- if (pager_in_use())
- printf(" (see 'man perf-list' on how to encode it)\n\n");
-
- printf(" %-50s [%s]\n",
- "mem:<addr>[/len][:access]",
- event_type_descriptors[PERF_TYPE_BREAKPOINT]);
- if (pager_in_use())
- printf("\n");
- }
-
- print_tracepoint_events(NULL, NULL, name_only);
-
- print_sdt_events(NULL, NULL, name_only);
-
- metricgroup__print(true, true, NULL, name_only, details_flag,
- pmu_name);
-
- print_libpfm_events(name_only, long_desc);
+ print_symbol_events(print_cb, print_state, PERF_TYPE_HARDWARE,
+ event_symbols_hw, PERF_COUNT_HW_MAX);
+ print_symbol_events(print_cb, print_state, PERF_TYPE_SOFTWARE,
+ event_symbols_sw, PERF_COUNT_SW_MAX);
+
+ print_tool_events(print_cb, print_state);
+
+ print_hwcache_events(print_cb, print_state);
+
+ print_pmu_events(print_cb, print_state);
+
+ print_cb->print_event(print_state,
+ /*topic=*/NULL,
+ /*pmu_name=*/NULL,
+ "rNNN",
+ /*event_alias=*/NULL,
+ /*scale_unit=*/NULL,
+ /*deprecated=*/false,
+ event_type_descriptors[PERF_TYPE_RAW],
+ /*desc=*/NULL,
+ /*long_desc=*/NULL,
+ /*encoding_desc=*/NULL,
+ /*metric_name=*/NULL,
+ /*metric_expr=*/NULL);
+
+ print_cb->print_event(print_state,
+ /*topic=*/NULL,
+ /*pmu_name=*/NULL,
+ "cpu/t1=v1[,t2=v2,t3 ...]/modifier",
+ /*event_alias=*/NULL,
+ /*scale_unit=*/NULL,
+ /*deprecated=*/false,
+ event_type_descriptors[PERF_TYPE_RAW],
+ "(see 'man perf-list' on how to encode it)",
+ /*long_desc=*/NULL,
+ /*encoding_desc=*/NULL,
+ /*metric_name=*/NULL,
+ /*metric_expr=*/NULL);
+
+ print_cb->print_event(print_state,
+ /*topic=*/NULL,
+ /*pmu_name=*/NULL,
+ "mem:<addr>[/len][:access]",
+ /*scale_unit=*/NULL,
+ /*event_alias=*/NULL,
+ /*deprecated=*/false,
+ event_type_descriptors[PERF_TYPE_BREAKPOINT],
+ /*desc=*/NULL,
+ /*long_desc=*/NULL,
+ /*encoding_desc=*/NULL,
+ /*metric_name=*/NULL,
+ /*metric_expr=*/NULL);
+
+ print_tracepoint_events(print_cb, print_state);
+
+ print_sdt_events(print_cb, print_state);
+
+ metricgroup__print(print_cb, print_state);
+
+ print_libpfm_events(print_cb, print_state);
}
#ifndef __PERF_PRINT_EVENTS_H
#define __PERF_PRINT_EVENTS_H
+#include <linux/perf_event.h>
#include <stdbool.h>
struct event_symbol;
-void print_events(const char *event_glob, bool name_only, bool quiet_flag,
- bool long_desc, bool details_flag, bool deprecated,
- const char *pmu_name);
-int print_hwcache_events(const char *event_glob, bool name_only);
-void print_sdt_events(const char *subsys_glob, const char *event_glob,
- bool name_only);
-void print_symbol_events(const char *event_glob, unsigned int type,
- struct event_symbol *syms, unsigned int max,
- bool name_only);
-void print_tool_events(const char *event_glob, bool name_only);
-void print_tracepoint_events(const char *subsys_glob, const char *event_glob,
- bool name_only);
+struct print_callbacks {
+ void (*print_start)(void *print_state);
+ void (*print_end)(void *print_state);
+ void (*print_event)(void *print_state, const char *topic,
+ const char *pmu_name,
+ const char *event_name, const char *event_alias,
+ const char *scale_unit,
+ bool deprecated, const char *event_type_desc,
+ const char *desc, const char *long_desc,
+ const char *encoding_desc,
+ const char *metric_name, const char *metric_expr);
+ void (*print_metric)(void *print_state,
+ const char *group,
+ const char *name,
+ const char *desc,
+ const char *long_desc,
+ const char *expr,
+ const char *unit);
+};
+
+/** Print all events, the default when no options are specified. */
+void print_events(const struct print_callbacks *print_cb, void *print_state);
+int print_hwcache_events(const struct print_callbacks *print_cb, void *print_state);
+void print_sdt_events(const struct print_callbacks *print_cb, void *print_state);
+void print_symbol_events(const struct print_callbacks *print_cb, void *print_state,
+ unsigned int type, const struct event_symbol *syms,
+ unsigned int max);
+void print_tool_events(const struct print_callbacks *print_cb, void *print_state);
+void print_tracepoint_events(const struct print_callbacks *print_cb, void *print_state);
#endif /* __PERF_PRINT_EVENTS_H */
/* Skip if declared file name does not match */
if (fsp->file) {
- file = dwarf_decl_file(fn_die);
+ file = die_get_decl_file(fn_die);
if (!file || strcmp(fsp->file, file) != 0)
return 0;
}
struct dwarf_callback_param *param = data;
struct probe_finder *pf = param->data;
struct perf_probe_point *pp = &pf->pev->point;
+ const char *fname;
/* Check tag and diename */
if (!die_is_func_def(sp_die) ||
return DWARF_CB_OK;
/* Check declared file */
- if (pp->file && strtailcmp(pp->file, dwarf_decl_file(sp_die)))
+ fname = die_get_decl_file(sp_die);
+ if (!fname) {
+ pr_warning("A function DIE doesn't have decl_line. Maybe broken DWARF?\n");
+ return DWARF_CB_OK;
+ }
+ if (pp->file && fname && strtailcmp(pp->file, fname))
return DWARF_CB_OK;
pr_debug("Matched function: %s [%lx]\n", dwarf_diename(sp_die),
(unsigned long)dwarf_dieoffset(sp_die));
- pf->fname = dwarf_decl_file(sp_die);
+ pf->fname = fname;
if (pp->line) { /* Function relative line */
dwarf_decl_line(sp_die, &pf->lno);
pf->lno += pp->line;
static int pubname_search_cb(Dwarf *dbg, Dwarf_Global *gl, void *data)
{
struct pubname_callback_param *param = data;
+ const char *fname;
if (dwarf_offdie(dbg, gl->die_offset, param->sp_die)) {
if (dwarf_tag(param->sp_die) != DW_TAG_subprogram)
if (!dwarf_offdie(dbg, gl->cu_offset, param->cu_die))
return DWARF_CB_OK;
- if (param->file &&
- strtailcmp(param->file, dwarf_decl_file(param->sp_die)))
- return DWARF_CB_OK;
+ if (param->file) {
+ fname = die_get_decl_file(param->sp_die);
+ if (!fname || strtailcmp(param->file, fname))
+ return DWARF_CB_OK;
+ }
param->found = 1;
return DWARF_CB_ABORT;
goto post;
}
- fname = dwarf_decl_file(&spdie);
+ fname = die_get_decl_file(&spdie);
if (addr == baseaddr) {
/* Function entry - Relative line number is 0 */
lineno = baseline;
}
}
/* Verify the lineno and baseline are in a same file */
- tmp = dwarf_decl_file(&spdie);
- if (!tmp || strcmp(tmp, fname) != 0)
+ tmp = die_get_decl_file(&spdie);
+ if (!tmp || (fname && strcmp(tmp, fname) != 0))
lineno = 0;
}
struct dwarf_callback_param *param = data;
struct line_finder *lf = param->data;
struct line_range *lr = lf->lr;
+ const char *fname;
/* Check declared file */
- if (lr->file && strtailcmp(lr->file, dwarf_decl_file(sp_die)))
- return DWARF_CB_OK;
+ if (lr->file) {
+ fname = die_get_decl_file(sp_die);
+ if (!fname || strtailcmp(lr->file, fname))
+ return DWARF_CB_OK;
+ }
if (die_match_name(sp_die, lr->function) && die_is_func_def(sp_die)) {
- lf->fname = dwarf_decl_file(sp_die);
+ lf->fname = die_get_decl_file(sp_die);
dwarf_decl_line(sp_die, &lr->offset);
pr_debug("fname: %s, lineno:%d\n", lf->fname, lr->offset);
lf->lno_s = lr->offset + lr->start;
#include <poll.h>
#include <linux/err.h>
#include <perf/cpumap.h>
+#ifdef HAVE_LIBTRACEEVENT
#include <traceevent/event-parse.h>
+#endif
#include <perf/mmap.h>
#include "evlist.h"
#include "callchain.h"
return ret;
}
+#ifdef HAVE_LIBTRACEEVENT
static bool is_tracepoint(struct pyrf_event *pevent)
{
return pevent->evsel->core.attr.type == PERF_TYPE_TRACEPOINT;
offset = val;
len = offset >> 16;
offset &= 0xffff;
+#ifdef HAVE_LIBTRACEEVENT_TEP_FIELD_IS_RELATIVE
if (field->flags & TEP_FIELD_IS_RELATIVE)
offset += field->offset + field->size;
+#endif
}
if (field->flags & TEP_FIELD_IS_STRING &&
is_printable_array(data + offset, len)) {
return tracepoint_field(pevent, field);
}
+#endif /* HAVE_LIBTRACEEVENT */
static PyObject*
pyrf_sample_event__getattro(struct pyrf_event *pevent, PyObject *attr_name)
{
PyObject *obj = NULL;
+#ifdef HAVE_LIBTRACEEVENT
if (is_tracepoint(pevent))
obj = get_tracepoint_field(pevent, attr_name);
+#endif
return obj ?: PyObject_GenericGetAttr((PyObject *) pevent, attr_name);
}
{
struct pyrf_thread_map *pthreads = (void *)obj;
- return pthreads->threads->nr;
+ return perf_thread_map__nr(pthreads->threads);
}
static PyObject *pyrf_thread_map__item(PyObject *obj, Py_ssize_t i)
{
struct pyrf_thread_map *pthreads = (void *)obj;
- if (i >= pthreads->threads->nr)
+ if (i >= perf_thread_map__nr(pthreads->threads))
return NULL;
- return Py_BuildValue("i", pthreads->threads->map[i]);
+ return Py_BuildValue("i", perf_thread_map__pid(pthreads->threads, i));
}
static PySequenceMethods pyrf_thread_map__sequence_methods = {
PyObject *args, PyObject *kwargs)
{
struct evlist *evlist = &pevlist->evlist;
- int group = 0;
- static char *kwlist[] = { "group", NULL };
-
- if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|OOii", kwlist, &group))
- return NULL;
-
- if (group)
- evlist__set_leader(evlist);
if (evlist__open(evlist) < 0) {
PyErr_SetFromErrno(PyExc_OSError);
static PyObject *pyrf__tracepoint(struct pyrf_evsel *pevsel,
PyObject *args, PyObject *kwargs)
{
+#ifndef HAVE_LIBTRACEEVENT
+ return NULL;
+#else
struct tep_event *tp_format;
static char *kwlist[] = { "sys", "name", NULL };
char *sys = NULL;
return _PyLong_FromLong(-1);
return _PyLong_FromLong(tp_format->id);
+#endif // HAVE_LIBTRACEEVENT
}
static PyMethodDef perf__methods[] = {
bool use_comm_exec;
bool sample_id = opts->sample_id;
- /*
- * Set the evsel leader links before we configure attributes,
- * since some might depend on this info.
- */
- if (opts->group)
- evlist__set_leader(evlist);
-
if (perf_cpu_map__cpu(evlist->core.user_requested_cpus, 0).cpu < 0)
opts->no_inherit = true;
struct record_opts {
struct target target;
- bool group;
bool inherit_stat;
bool no_buffering;
bool no_inherit;
#include "s390-cpumsf-kernel.h"
#include "s390-cpumcf-kernel.h"
#include "config.h"
+#include "util/sample.h"
struct s390_cpumsf {
struct auxtrace auxtrace;
#include "sample-raw.h"
#include "s390-cpumcf-kernel.h"
#include "pmu-events/pmu-events.h"
+#include "util/sample.h"
static size_t ctrset_size(struct cf_ctrset_entry *set)
{
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __PERF_SAMPLE_H
+#define __PERF_SAMPLE_H
+
+#include <linux/perf_event.h>
+#include <linux/types.h>
+
+/* number of register is bound by the number of bits in regs_dump::mask (64) */
+#define PERF_SAMPLE_REGS_CACHE_SIZE (8 * sizeof(u64))
+
+struct regs_dump {
+ u64 abi;
+ u64 mask;
+ u64 *regs;
+
+ /* Cached values/mask filled by first register access. */
+ u64 cache_regs[PERF_SAMPLE_REGS_CACHE_SIZE];
+ u64 cache_mask;
+};
+
+struct stack_dump {
+ u16 offset;
+ u64 size;
+ char *data;
+};
+
+struct sample_read_value {
+ u64 value;
+ u64 id; /* only if PERF_FORMAT_ID */
+ u64 lost; /* only if PERF_FORMAT_LOST */
+};
+
+struct sample_read {
+ u64 time_enabled;
+ u64 time_running;
+ union {
+ struct {
+ u64 nr;
+ struct sample_read_value *values;
+ } group;
+ struct sample_read_value one;
+ };
+};
+
+static inline size_t sample_read_value_size(u64 read_format)
+{
+ /* PERF_FORMAT_ID is forced for PERF_SAMPLE_READ */
+ if (read_format & PERF_FORMAT_LOST)
+ return sizeof(struct sample_read_value);
+ else
+ return offsetof(struct sample_read_value, lost);
+}
+
+static inline struct sample_read_value *next_sample_read_value(struct sample_read_value *v, u64 read_format)
+{
+ return (void *)v + sample_read_value_size(read_format);
+}
+
+#define sample_read_group__for_each(v, nr, rf) \
+ for (int __i = 0; __i < (int)nr; v = next_sample_read_value(v, rf), __i++)
+
+#define MAX_INSN 16
+
+struct aux_sample {
+ u64 size;
+ void *data;
+};
+
+struct perf_sample {
+ u64 ip;
+ u32 pid, tid;
+ u64 time;
+ u64 addr;
+ u64 id;
+ u64 stream_id;
+ u64 period;
+ u64 weight;
+ u64 transaction;
+ u64 insn_cnt;
+ u64 cyc_cnt;
+ u32 cpu;
+ u32 raw_size;
+ u64 data_src;
+ u64 phys_addr;
+ u64 data_page_size;
+ u64 code_page_size;
+ u64 cgroup;
+ u32 flags;
+ u32 machine_pid;
+ u32 vcpu;
+ u16 insn_len;
+ u8 cpumode;
+ u16 misc;
+ u16 ins_lat;
+ u16 p_stage_cyc;
+ bool no_hw_idx; /* No hw_idx collected in branch_stack */
+ char insn[MAX_INSN];
+ void *raw_data;
+ struct ip_callchain *callchain;
+ struct branch_stack *branch_stack;
+ struct regs_dump user_regs;
+ struct regs_dump intr_regs;
+ struct stack_dump user_stack;
+ struct sample_read read;
+ struct aux_sample aux_sample;
+};
+
+/*
+ * raw_data is always 4 bytes from an 8-byte boundary, so subtract 4 to get
+ * 8-byte alignment.
+ */
+static inline void *perf_sample__synth_ptr(struct perf_sample *sample)
+{
+ return sample->raw_data - 4;
+}
+
+#endif /* __PERF_SAMPLE_H */
-perf-$(CONFIG_LIBPERL) += trace-event-perl.o
-perf-$(CONFIG_LIBPYTHON) += trace-event-python.o
+ifeq ($(CONFIG_LIBTRACEEVENT),y)
+ perf-$(CONFIG_LIBPERL) += trace-event-perl.o
+ perf-$(CONFIG_LIBPYTHON) += trace-event-python.o
+endif
CFLAGS_trace-event-perl.o += $(PERL_EMBED_CCOPTS) -Wno-redundant-decls -Wno-strict-prototypes -Wno-unused-parameter -Wno-shadow -Wno-nested-externs -Wno-undef -Wno-switch-default -Wno-bad-function-cast -Wno-declaration-after-statement -Wno-switch-enum
#include <errno.h>
#include <linux/bitmap.h>
#include <linux/time64.h>
+#include <traceevent/event-parse.h>
#include <stdbool.h>
/* perl needs the following define, right after including stdbool.h */
if (field->flags & TEP_FIELD_IS_DYNAMIC) {
offset = *(int *)(data + field->offset);
offset &= 0xffff;
+#ifdef HAVE_LIBTRACEEVENT_TEP_FIELD_IS_RELATIVE
if (field->flags & TEP_FIELD_IS_RELATIVE)
offset += field->offset + field->size;
+#endif
} else
offset = field->offset;
XPUSHs(sv_2mortal(newSVpv((char *)data + offset, 0)));
#include <linux/bitmap.h>
#include <linux/compiler.h>
#include <linux/time64.h>
+#include <traceevent/event-parse.h>
#include "../build-id.h"
#include "../counts.h"
#include "print_binary.h"
#include "stat.h"
#include "mem-events.h"
+#include "util/perf_regs.h"
#if PY_MAJOR_VERSION < 3
#define _PyUnicode_FromString(arg) \
offset = val;
len = offset >> 16;
offset &= 0xffff;
+#ifdef HAVE_LIBTRACEEVENT_TEP_FIELD_IS_RELATIVE
if (field->flags & TEP_FIELD_IS_RELATIVE)
offset += field->offset + field->size;
+#endif
}
if (field->flags & TEP_FIELD_IS_STRING &&
is_printable_array(data + offset, len)) {
struct perf_cpu_map *cpus = counter->core.cpus;
int cpu, thread;
- if (config->aggr_mode == AGGR_GLOBAL) {
- process_stat(counter, (struct perf_cpu){ .cpu = -1 }, -1, tstamp,
- &counter->counts->aggr);
- return;
- }
-
- for (thread = 0; thread < threads->nr; thread++) {
+ for (thread = 0; thread < perf_thread_map__nr(threads); thread++) {
for (cpu = 0; cpu < perf_cpu_map__nr(cpus); cpu++) {
process_stat(counter, perf_cpu_map__cpu(cpus, cpu),
perf_thread_map__pid(threads, thread), tstamp,
// SPDX-License-Identifier: GPL-2.0
#include <errno.h>
+#include <signal.h>
#include <inttypes.h>
#include <linux/err.h>
#include <linux/kernel.h>
evlist__delete(session->evlist);
perf_data__close(session->data);
}
+#ifdef HAVE_LIBTRACEEVENT
trace_event__cleanup(&session->tevent);
+#endif
free(session);
}
NULL);
}
-volatile int session_done;
+volatile sig_atomic_t session_done;
static int __perf_session__process_decomp_events(struct perf_session *session);
struct auxtrace *auxtrace;
struct itrace_synth_opts *itrace_synth_opts;
struct list_head auxtrace_index;
+#ifdef HAVE_LIBTRACEEVENT
struct trace_event tevent;
+#endif
struct perf_record_time_conv time_conv;
bool repipe;
bool one_mmap;
src_feature_tests = getenv('srctree') + '/tools/build/feature'
def clang_has_option(option):
- cc_output = Popen([cc, option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines()
+ cc_output = Popen([cc.split()[0], str(cc.split()[1:]) + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines()
return [o for o in cc_output if ((b"unknown argument" in o) or (b"is not supported" in o))] == [ ]
if cc_is_clang:
ext_sources = [f.strip() for f in open('util/python-ext-sources')
if len(f.strip()) > 0 and f[0] != '#']
+extra_libraries = []
+
+if '-DHAVE_LIBTRACEEVENT' in cflags:
+ extra_libraries += [ 'traceevent' ]
+else:
+ ext_sources.remove('util/trace-event.c')
+
# use full paths with source files
ext_sources = list(map(lambda x: '%s/%s' % (src_perf, x) , ext_sources))
-extra_libraries = []
if '-DHAVE_LIBNUMA_SUPPORT' in cflags:
- extra_libraries = [ 'numa' ]
+ extra_libraries += [ 'numa' ]
if '-DHAVE_LIBCAP_SUPPORT' in cflags:
extra_libraries += [ 'cap' ]
include_dirs = ['util/include'],
libraries = extra_libraries,
extra_compile_args = cflags,
- extra_objects = [libtraceevent, libapikfs, libperf],
+ extra_objects = [ x for x in [libtraceevent, libapikfs, libperf]
+ if x is not None],
)
setup(name='perf',
#include "srcline.h"
#include "strlist.h"
#include "strbuf.h"
-#include <traceevent/event-parse.h>
#include "mem-events.h"
#include "annotate.h"
#include "event.h"
#include <linux/kernel.h>
#include <linux/string.h>
+#ifdef HAVE_LIBTRACEEVENT
+#include <traceevent/event-parse.h>
+#endif
+
regex_t parent_regex;
const char default_parent_pattern[] = "^sys_|^do_page_fault";
const char *parent_pattern = default_parent_pattern;
/* --sort trace */
+#ifdef HAVE_LIBTRACEEVENT
static char *get_trace_output(struct hist_entry *he)
{
struct trace_seq seq;
.se_snprintf = hist_entry__trace_snprintf,
.se_width_idx = HISTC_TRACE,
};
+#endif /* HAVE_LIBTRACEEVENT */
/* sort keys for branch stacks */
DIM(SORT_LOCAL_WEIGHT, "local_weight", sort_local_weight),
DIM(SORT_GLOBAL_WEIGHT, "weight", sort_global_weight),
DIM(SORT_TRANSACTION, "transaction", sort_transaction),
+#ifdef HAVE_LIBTRACEEVENT
DIM(SORT_TRACE, "trace", sort_trace),
+#endif
DIM(SORT_SYM_SIZE, "symbol_size", sort_sym_size),
DIM(SORT_DSO_SIZE, "dso_size", sort_dso_size),
DIM(SORT_CGROUP, "cgroup", sort_cgroup),
return hse->se == &sort_ ## key ; \
}
+#ifdef HAVE_LIBTRACEEVENT
MK_SORT_ENTRY_CHK(trace)
+#else
+bool perf_hpp__is_trace_entry(struct perf_hpp_fmt *fmt __maybe_unused)
+{
+ return false;
+}
+#endif
MK_SORT_ENTRY_CHK(srcline)
MK_SORT_ENTRY_CHK(srcfile)
MK_SORT_ENTRY_CHK(thread)
return 0;
}
+#ifndef HAVE_LIBTRACEEVENT
+bool perf_hpp__is_dynamic_entry(struct perf_hpp_fmt *fmt __maybe_unused)
+{
+ return false;
+}
+bool perf_hpp__defined_dynamic_entry(struct perf_hpp_fmt *fmt __maybe_unused,
+ struct hists *hists __maybe_unused)
+{
+ return false;
+}
+#else
struct hpp_dynamic_entry {
struct perf_hpp_fmt hpp;
struct evsel *evsel;
tep_read_number_field(field, a->raw_data, &dyn);
offset = dyn & 0xffff;
size = (dyn >> 16) & 0xffff;
+#ifdef HAVE_LIBTRACEEVENT_TEP_FIELD_IS_RELATIVE
if (field->flags & TEP_FIELD_IS_RELATIVE)
offset += field->offset + field->size;
-
+#endif
/* record max width for output */
if (size > hde->dynamic_len)
hde->dynamic_len = size;
return hde;
}
+#endif /* HAVE_LIBTRACEEVENT */
struct perf_hpp_fmt *perf_hpp_fmt__dup(struct perf_hpp_fmt *fmt)
{
new_hse = memdup(hse, sizeof(*hse));
if (new_hse)
new_fmt = &new_hse->hpp;
+#ifdef HAVE_LIBTRACEEVENT
} else if (perf_hpp__is_dynamic_entry(fmt)) {
struct hpp_dynamic_entry *hde, *new_hde;
new_hde = memdup(hde, sizeof(*hde));
if (new_hde)
new_fmt = &new_hde->hpp;
+#endif
} else {
new_fmt = memdup(fmt, sizeof(*fmt));
}
return evsel;
}
+#ifdef HAVE_LIBTRACEEVENT
static int __dynamic_dimension__add(struct evsel *evsel,
struct tep_format_field *field,
bool raw_trace, int level)
}
return ret;
}
+#endif /* HAVE_LIBTRACEEVENT */
static int add_dynamic_entry(struct evlist *evlist, const char *tok,
int level)
{
char *str, *event_name, *field_name, *opt_name;
struct evsel *evsel;
- struct tep_format_field *field;
bool raw_trace = symbol_conf.raw_trace;
int ret = 0;
raw_trace = true;
}
+#ifdef HAVE_LIBTRACEEVENT
if (!strcmp(field_name, "trace_fields")) {
ret = add_all_dynamic_fields(evlist, raw_trace, level);
goto out;
ret = add_all_matching_fields(evlist, field_name, raw_trace, level);
goto out;
}
+#endif
evsel = find_evsel(evlist, event_name);
if (evsel == NULL) {
goto out;
}
+#ifdef HAVE_LIBTRACEEVENT
if (!strcmp(field_name, "*")) {
ret = add_evsel_fields(evsel, raw_trace, level);
} else {
- field = tep_find_any_field(evsel->tp_format, field_name);
+ struct tep_format_field *field = tep_find_any_field(evsel->tp_format, field_name);
+
if (field == NULL) {
pr_debug("Cannot find event field for %s.%s\n",
event_name, field_name);
ret = __dynamic_dimension__add(evsel, field, raw_trace, level);
}
+#else
+ (void)level;
+ (void)raw_trace;
+#endif /* HAVE_LIBTRACEEVENT */
out:
free(str);
for (i = 0; i < ARRAY_SIZE(common_sort_dimensions); i++) {
struct sort_dimension *sd = &common_sort_dimensions[i];
- if (strncasecmp(tok, sd->name, strlen(tok)))
+ if (!sd->name || strncasecmp(tok, sd->name, strlen(tok)))
continue;
for (j = 0; j < ARRAY_SIZE(dynamic_headers); j++) {
- if (!strcmp(dynamic_headers[j], sd->name))
+ if (sd->name && !strcmp(dynamic_headers[j], sd->name))
sort_dimension_add_dynamic_header(sd);
}
for (i = 0; i < ARRAY_SIZE(bstack_sort_dimensions); i++) {
struct sort_dimension *sd = &bstack_sort_dimensions[i];
- if (strncasecmp(tok, sd->name, strlen(tok)))
+ if (!sd->name || strncasecmp(tok, sd->name, strlen(tok)))
continue;
if (sort__mode != SORT_MODE__BRANCH)
for (i = 0; i < ARRAY_SIZE(memory_sort_dimensions); i++) {
struct sort_dimension *sd = &memory_sort_dimensions[i];
- if (strncasecmp(tok, sd->name, strlen(tok)))
+ if (!sd->name || strncasecmp(tok, sd->name, strlen(tok)))
continue;
if (sort__mode != SORT_MODE__MEMORY)
for (i = 0; i < ARRAY_SIZE(common_sort_dimensions); i++) {
struct sort_dimension *sd = &common_sort_dimensions[i];
- if (strncasecmp(tok, sd->name, strlen(tok)))
+ if (!sd->name || strncasecmp(tok, sd->name, strlen(tok)))
continue;
return __sort_dimension__add_output(list, sd);
for (i = 0; i < ARRAY_SIZE(bstack_sort_dimensions); i++) {
struct sort_dimension *sd = &bstack_sort_dimensions[i];
- if (strncasecmp(tok, sd->name, strlen(tok)))
+ if (!sd->name || strncasecmp(tok, sd->name, strlen(tok)))
continue;
if (sort__mode != SORT_MODE__BRANCH)
for (i = 0; i < ARRAY_SIZE(memory_sort_dimensions); i++) {
struct sort_dimension *sd = &memory_sort_dimensions[i];
- if (strncasecmp(tok, sd->name, strlen(tok)))
+ if (!sd->name || strncasecmp(tok, sd->name, strlen(tok)))
continue;
if (sort__mode != SORT_MODE__MEMORY)
static void add_key(struct strbuf *sb, const char *str, int *llen)
{
+ if (!str)
+ return;
+
if (*llen >= 75) {
strbuf_addstr(sb, "\n\t\t\t ");
*llen = INDENT;
#define CNTR_NOT_SUPPORTED "<not supported>"
#define CNTR_NOT_COUNTED "<not counted>"
-static void print_running(struct perf_stat_config *config,
- u64 run, u64 ena)
+#define METRIC_LEN 38
+#define EVNAME_LEN 32
+#define COUNTS_LEN 18
+#define INTERVAL_LEN 16
+#define CGROUP_LEN 16
+#define COMM_LEN 16
+#define PID_LEN 7
+#define CPUS_LEN 4
+
+static int aggr_header_lens[] = {
+ [AGGR_CORE] = 18,
+ [AGGR_DIE] = 12,
+ [AGGR_SOCKET] = 6,
+ [AGGR_NODE] = 6,
+ [AGGR_NONE] = 6,
+ [AGGR_THREAD] = 16,
+ [AGGR_GLOBAL] = 0,
+};
+
+static const char *aggr_header_csv[] = {
+ [AGGR_CORE] = "core,cpus,",
+ [AGGR_DIE] = "die,cpus,",
+ [AGGR_SOCKET] = "socket,cpus,",
+ [AGGR_NONE] = "cpu,",
+ [AGGR_THREAD] = "comm-pid,",
+ [AGGR_NODE] = "node,",
+ [AGGR_GLOBAL] = ""
+};
+
+static const char *aggr_header_std[] = {
+ [AGGR_CORE] = "core",
+ [AGGR_DIE] = "die",
+ [AGGR_SOCKET] = "socket",
+ [AGGR_NONE] = "cpu",
+ [AGGR_THREAD] = "comm-pid",
+ [AGGR_NODE] = "node",
+ [AGGR_GLOBAL] = ""
+};
+
+static void print_running_std(struct perf_stat_config *config, u64 run, u64 ena)
{
+ if (run != ena)
+ fprintf(config->output, " (%.2f%%)", 100.0 * run / ena);
+}
+static void print_running_csv(struct perf_stat_config *config, u64 run, u64 ena)
+{
double enabled_percent = 100;
if (run != ena)
enabled_percent = 100 * run / ena;
- if (config->json_output)
- fprintf(config->output,
- "\"event-runtime\" : %" PRIu64 ", \"pcnt-running\" : %.2f, ",
- run, enabled_percent);
- else if (config->csv_output)
- fprintf(config->output,
- "%s%" PRIu64 "%s%.2f", config->csv_sep,
- run, config->csv_sep, enabled_percent);
- else if (run != ena)
- fprintf(config->output, " (%.2f%%)", 100.0 * run / ena);
+ fprintf(config->output, "%s%" PRIu64 "%s%.2f",
+ config->csv_sep, run, config->csv_sep, enabled_percent);
+}
+
+static void print_running_json(struct perf_stat_config *config, u64 run, u64 ena)
+{
+ double enabled_percent = 100;
+
+ if (run != ena)
+ enabled_percent = 100 * run / ena;
+ fprintf(config->output, "\"event-runtime\" : %" PRIu64 ", \"pcnt-running\" : %.2f, ",
+ run, enabled_percent);
+}
+
+static void print_running(struct perf_stat_config *config,
+ u64 run, u64 ena, bool before_metric)
+{
+ if (config->json_output) {
+ if (before_metric)
+ print_running_json(config, run, ena);
+ } else if (config->csv_output) {
+ if (before_metric)
+ print_running_csv(config, run, ena);
+ } else {
+ if (!before_metric)
+ print_running_std(config, run, ena);
+ }
+}
+
+static void print_noise_pct_std(struct perf_stat_config *config,
+ double pct)
+{
+ if (pct)
+ fprintf(config->output, " ( +-%6.2f%% )", pct);
+}
+
+static void print_noise_pct_csv(struct perf_stat_config *config,
+ double pct)
+{
+ fprintf(config->output, "%s%.2f%%", config->csv_sep, pct);
+}
+
+static void print_noise_pct_json(struct perf_stat_config *config,
+ double pct)
+{
+ fprintf(config->output, "\"variance\" : %.2f, ", pct);
}
static void print_noise_pct(struct perf_stat_config *config,
- double total, double avg)
+ double total, double avg, bool before_metric)
{
double pct = rel_stddev_stats(total, avg);
- if (config->json_output)
- fprintf(config->output, "\"variance\" : %.2f, ", pct);
- else if (config->csv_output)
- fprintf(config->output, "%s%.2f%%", config->csv_sep, pct);
- else if (pct)
- fprintf(config->output, " ( +-%6.2f%% )", pct);
+ if (config->json_output) {
+ if (before_metric)
+ print_noise_pct_json(config, pct);
+ } else if (config->csv_output) {
+ if (before_metric)
+ print_noise_pct_csv(config, pct);
+ } else {
+ if (!before_metric)
+ print_noise_pct_std(config, pct);
+ }
}
static void print_noise(struct perf_stat_config *config,
- struct evsel *evsel, double avg)
+ struct evsel *evsel, double avg, bool before_metric)
{
struct perf_stat_evsel *ps;
return;
ps = evsel->stats;
- print_noise_pct(config, stddev_stats(&ps->res_stats), avg);
+ print_noise_pct(config, stddev_stats(&ps->res_stats), avg, before_metric);
}
-static void print_cgroup(struct perf_stat_config *config, struct evsel *evsel)
+static void print_cgroup_std(struct perf_stat_config *config, const char *cgrp_name)
{
- if (nr_cgroups) {
- const char *cgrp_name = evsel->cgrp ? evsel->cgrp->name : "";
+ fprintf(config->output, " %-*s", CGROUP_LEN, cgrp_name);
+}
+
+static void print_cgroup_csv(struct perf_stat_config *config, const char *cgrp_name)
+{
+ fprintf(config->output, "%s%s", config->csv_sep, cgrp_name);
+}
+
+static void print_cgroup_json(struct perf_stat_config *config, const char *cgrp_name)
+{
+ fprintf(config->output, "\"cgroup\" : \"%s\", ", cgrp_name);
+}
+
+static void print_cgroup(struct perf_stat_config *config, struct cgroup *cgrp)
+{
+ if (nr_cgroups || config->cgroup_list) {
+ const char *cgrp_name = cgrp ? cgrp->name : "";
if (config->json_output)
- fprintf(config->output, "\"cgroup\" : \"%s\", ", cgrp_name);
+ print_cgroup_json(config, cgrp_name);
+ else if (config->csv_output)
+ print_cgroup_csv(config, cgrp_name);
else
- fprintf(config->output, "%s%s", config->csv_sep, cgrp_name);
+ print_cgroup_std(config, cgrp_name);
}
}
-
-static void aggr_printout(struct perf_stat_config *config,
- struct evsel *evsel, struct aggr_cpu_id id, int nr)
+static void print_aggr_id_std(struct perf_stat_config *config,
+ struct evsel *evsel, struct aggr_cpu_id id, int nr)
{
+ FILE *output = config->output;
+ int idx = config->aggr_mode;
+ char buf[128];
+
+ switch (config->aggr_mode) {
+ case AGGR_CORE:
+ snprintf(buf, sizeof(buf), "S%d-D%d-C%d", id.socket, id.die, id.core);
+ break;
+ case AGGR_DIE:
+ snprintf(buf, sizeof(buf), "S%d-D%d", id.socket, id.die);
+ break;
+ case AGGR_SOCKET:
+ snprintf(buf, sizeof(buf), "S%d", id.socket);
+ break;
+ case AGGR_NODE:
+ snprintf(buf, sizeof(buf), "N%d", id.node);
+ break;
+ case AGGR_NONE:
+ if (evsel->percore && !config->percore_show_thread) {
+ snprintf(buf, sizeof(buf), "S%d-D%d-C%d ",
+ id.socket, id.die, id.core);
+ fprintf(output, "%-*s ",
+ aggr_header_lens[AGGR_CORE], buf);
+ } else if (id.cpu.cpu > -1) {
+ fprintf(output, "CPU%-*d ",
+ aggr_header_lens[AGGR_NONE] - 3, id.cpu.cpu);
+ }
+ return;
+ case AGGR_THREAD:
+ fprintf(output, "%*s-%-*d ",
+ COMM_LEN, perf_thread_map__comm(evsel->core.threads, id.thread_idx),
+ PID_LEN, perf_thread_map__pid(evsel->core.threads, id.thread_idx));
+ return;
+ case AGGR_GLOBAL:
+ case AGGR_UNSET:
+ case AGGR_MAX:
+ default:
+ return;
+ }
+ fprintf(output, "%-*s %*d ", aggr_header_lens[idx], buf, 4, nr);
+}
- if (config->json_output && !config->interval)
- fprintf(config->output, "{");
+static void print_aggr_id_csv(struct perf_stat_config *config,
+ struct evsel *evsel, struct aggr_cpu_id id, int nr)
+{
+ FILE *output = config->output;
+ const char *sep = config->csv_sep;
switch (config->aggr_mode) {
case AGGR_CORE:
- if (config->json_output) {
- fprintf(config->output,
- "\"core\" : \"S%d-D%d-C%d\", \"aggregate-number\" : %d, ",
- id.socket,
- id.die,
- id.core,
- nr);
- } else {
- fprintf(config->output, "S%d-D%d-C%*d%s%*d%s",
- id.socket,
- id.die,
- config->csv_output ? 0 : -8,
- id.core,
- config->csv_sep,
- config->csv_output ? 0 : 4,
- nr,
- config->csv_sep);
- }
+ fprintf(output, "S%d-D%d-C%d%s%d%s",
+ id.socket, id.die, id.core, sep, nr, sep);
break;
case AGGR_DIE:
- if (config->json_output) {
- fprintf(config->output,
- "\"die\" : \"S%d-D%d\", \"aggregate-number\" : %d, ",
- id.socket,
- id.die,
- nr);
- } else {
- fprintf(config->output, "S%d-D%*d%s%*d%s",
- id.socket,
- config->csv_output ? 0 : -8,
- id.die,
- config->csv_sep,
- config->csv_output ? 0 : 4,
- nr,
- config->csv_sep);
- }
+ fprintf(output, "S%d-D%d%s%d%s",
+ id.socket, id.die, sep, nr, sep);
break;
case AGGR_SOCKET:
- if (config->json_output) {
- fprintf(config->output,
- "\"socket\" : \"S%d\", \"aggregate-number\" : %d, ",
- id.socket,
- nr);
- } else {
- fprintf(config->output, "S%*d%s%*d%s",
- config->csv_output ? 0 : -5,
- id.socket,
- config->csv_sep,
- config->csv_output ? 0 : 4,
- nr,
- config->csv_sep);
- }
+ fprintf(output, "S%d%s%d%s",
+ id.socket, sep, nr, sep);
break;
case AGGR_NODE:
- if (config->json_output) {
- fprintf(config->output, "\"node\" : \"N%d\", \"aggregate-number\" : %d, ",
- id.node,
- nr);
- } else {
- fprintf(config->output, "N%*d%s%*d%s",
- config->csv_output ? 0 : -5,
- id.node,
- config->csv_sep,
- config->csv_output ? 0 : 4,
- nr,
- config->csv_sep);
- }
+ fprintf(output, "N%d%s%d%s",
+ id.node, sep, nr, sep);
break;
case AGGR_NONE:
- if (config->json_output) {
- if (evsel->percore && !config->percore_show_thread) {
- fprintf(config->output, "\"core\" : \"S%d-D%d-C%d\"",
- id.socket,
- id.die,
- id.core);
- } else if (id.cpu.cpu > -1) {
- fprintf(config->output, "\"cpu\" : \"%d\", ",
- id.cpu.cpu);
- }
- } else {
- if (evsel->percore && !config->percore_show_thread) {
- fprintf(config->output, "S%d-D%d-C%*d%s",
- id.socket,
- id.die,
- config->csv_output ? 0 : -3,
- id.core, config->csv_sep);
- } else if (id.cpu.cpu > -1) {
- fprintf(config->output, "CPU%*d%s",
- config->csv_output ? 0 : -7,
- id.cpu.cpu, config->csv_sep);
- }
+ if (evsel->percore && !config->percore_show_thread) {
+ fprintf(output, "S%d-D%d-C%d%s",
+ id.socket, id.die, id.core, sep);
+ } else if (id.cpu.cpu > -1) {
+ fprintf(output, "CPU%d%s",
+ id.cpu.cpu, sep);
}
break;
case AGGR_THREAD:
- if (config->json_output) {
- fprintf(config->output, "\"thread\" : \"%s-%d\", ",
- perf_thread_map__comm(evsel->core.threads, id.thread_idx),
- perf_thread_map__pid(evsel->core.threads, id.thread_idx));
- } else {
- fprintf(config->output, "%*s-%*d%s",
- config->csv_output ? 0 : 16,
- perf_thread_map__comm(evsel->core.threads, id.thread_idx),
- config->csv_output ? 0 : -8,
- perf_thread_map__pid(evsel->core.threads, id.thread_idx),
- config->csv_sep);
+ fprintf(output, "%s-%d%s",
+ perf_thread_map__comm(evsel->core.threads, id.thread_idx),
+ perf_thread_map__pid(evsel->core.threads, id.thread_idx),
+ sep);
+ break;
+ case AGGR_GLOBAL:
+ case AGGR_UNSET:
+ case AGGR_MAX:
+ default:
+ break;
+ }
+}
+
+static void print_aggr_id_json(struct perf_stat_config *config,
+ struct evsel *evsel, struct aggr_cpu_id id, int nr)
+{
+ FILE *output = config->output;
+
+ switch (config->aggr_mode) {
+ case AGGR_CORE:
+ fprintf(output, "\"core\" : \"S%d-D%d-C%d\", \"aggregate-number\" : %d, ",
+ id.socket, id.die, id.core, nr);
+ break;
+ case AGGR_DIE:
+ fprintf(output, "\"die\" : \"S%d-D%d\", \"aggregate-number\" : %d, ",
+ id.socket, id.die, nr);
+ break;
+ case AGGR_SOCKET:
+ fprintf(output, "\"socket\" : \"S%d\", \"aggregate-number\" : %d, ",
+ id.socket, nr);
+ break;
+ case AGGR_NODE:
+ fprintf(output, "\"node\" : \"N%d\", \"aggregate-number\" : %d, ",
+ id.node, nr);
+ break;
+ case AGGR_NONE:
+ if (evsel->percore && !config->percore_show_thread) {
+ fprintf(output, "\"core\" : \"S%d-D%d-C%d\"",
+ id.socket, id.die, id.core);
+ } else if (id.cpu.cpu > -1) {
+ fprintf(output, "\"cpu\" : \"%d\", ",
+ id.cpu.cpu);
}
break;
+ case AGGR_THREAD:
+ fprintf(output, "\"thread\" : \"%s-%d\", ",
+ perf_thread_map__comm(evsel->core.threads, id.thread_idx),
+ perf_thread_map__pid(evsel->core.threads, id.thread_idx));
+ break;
case AGGR_GLOBAL:
case AGGR_UNSET:
case AGGR_MAX:
}
}
+static void aggr_printout(struct perf_stat_config *config,
+ struct evsel *evsel, struct aggr_cpu_id id, int nr)
+{
+ if (config->json_output)
+ print_aggr_id_json(config, evsel, id, nr);
+ else if (config->csv_output)
+ print_aggr_id_csv(config, evsel, id, nr);
+ else
+ print_aggr_id_std(config, evsel, id, nr);
+}
+
struct outstate {
FILE *fh;
bool newline;
+ bool first;
const char *prefix;
int nfields;
int nr;
struct aggr_cpu_id id;
struct evsel *evsel;
+ struct cgroup *cgrp;
};
-#define METRIC_LEN 35
-
static void new_line_std(struct perf_stat_config *config __maybe_unused,
void *ctx)
{
struct outstate *os)
{
fputc('\n', os->fh);
- fputs(os->prefix, os->fh);
+ if (os->prefix)
+ fputs(os->prefix, os->fh);
aggr_printout(config, os->evsel, os->id, os->nr);
if (config->aggr_mode == AGGR_NONE)
fprintf(os->fh, " ");
{
struct outstate *os = ctx;
- fputc('\n', os->fh);
+ fputs("\n{", os->fh);
if (os->prefix)
fprintf(os->fh, "%s", os->prefix);
aggr_printout(config, os->evsel, os->id, os->nr);
color_snprintf(str, sizeof(str), color ?: "", fmt, val);
fprintf(out, "%*s ", mlen, str);
+ os->first = false;
}
static void print_metric_only_csv(struct perf_stat_config *config __maybe_unused,
ends++;
*ends = 0;
fprintf(out, "%s%s", vals, config->csv_sep);
+ os->first = false;
}
static void print_metric_only_json(struct perf_stat_config *config __maybe_unused,
while (isdigit(*ends) || *ends == '.')
ends++;
*ends = 0;
- fprintf(out, "{\"metric-value\" : \"%s\"}", vals);
+ if (!unit[0] || !vals[0])
+ return;
+ fprintf(out, "%s\"%s\" : \"%s\"", os->first ? "" : ", ", unit, vals);
+ os->first = false;
}
static void new_line_metric(struct perf_stat_config *config __maybe_unused,
os->evsel->priv != os->evsel->evlist->selected->priv)
return;
- if (!valid_only_metric(unit) && !config->json_output)
+ if (os->evsel->cgrp != os->cgrp)
+ return;
+
+ if (!valid_only_metric(unit))
return;
unit = fixunit(tbuf, os->evsel, unit);
if (config->json_output)
- fprintf(os->fh, "\"unit\" : \"%s\"", unit);
+ return;
else if (config->csv_output)
fprintf(os->fh, "%s%s", unit, config->csv_sep);
else
fprintf(os->fh, "%*s ", config->metric_only_len, unit);
}
-static int first_shadow_map_idx(struct perf_stat_config *config,
- struct evsel *evsel, const struct aggr_cpu_id *id)
+static void print_counter_value_std(struct perf_stat_config *config,
+ struct evsel *evsel, double avg, bool ok)
{
- struct perf_cpu_map *cpus = evsel__cpus(evsel);
- struct perf_cpu cpu;
- int idx;
-
- if (config->aggr_mode == AGGR_NONE)
- return perf_cpu_map__idx(cpus, id->cpu);
+ FILE *output = config->output;
+ double sc = evsel->scale;
+ const char *fmt;
+ const char *bad_count = evsel->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED;
- if (config->aggr_mode == AGGR_THREAD)
- return id->thread_idx;
+ if (config->big_num)
+ fmt = floor(sc) != sc ? "%'*.2f " : "%'*.0f ";
+ else
+ fmt = floor(sc) != sc ? "%*.2f " : "%*.0f ";
- if (!config->aggr_get_id)
- return 0;
+ if (ok)
+ fprintf(output, fmt, COUNTS_LEN, avg);
+ else
+ fprintf(output, "%*s ", COUNTS_LEN, bad_count);
- perf_cpu_map__for_each_cpu(cpu, idx, cpus) {
- struct aggr_cpu_id cpu_id = config->aggr_get_id(config, cpu);
+ if (evsel->unit)
+ fprintf(output, "%-*s ", config->unit_width, evsel->unit);
- if (aggr_cpu_id__equal(&cpu_id, id))
- return idx;
- }
- return 0;
+ fprintf(output, "%-*s", EVNAME_LEN, evsel__name(evsel));
}
-static void abs_printout(struct perf_stat_config *config,
- struct aggr_cpu_id id, int nr, struct evsel *evsel, double avg)
+static void print_counter_value_csv(struct perf_stat_config *config,
+ struct evsel *evsel, double avg, bool ok)
{
FILE *output = config->output;
double sc = evsel->scale;
- const char *fmt;
+ const char *sep = config->csv_sep;
+ const char *fmt = floor(sc) != sc ? "%.2f%s" : "%.0f%s";
+ const char *bad_count = evsel->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED;
- if (config->csv_output) {
- fmt = floor(sc) != sc ? "%.2f%s" : "%.0f%s";
- } else {
- if (config->big_num)
- fmt = floor(sc) != sc ? "%'18.2f%s" : "%'18.0f%s";
- else
- fmt = floor(sc) != sc ? "%18.2f%s" : "%18.0f%s";
- }
+ if (ok)
+ fprintf(output, fmt, avg, sep);
+ else
+ fprintf(output, "%s%s", bad_count, sep);
- aggr_printout(config, evsel, id, nr);
+ if (evsel->unit)
+ fprintf(output, "%s%s", evsel->unit, sep);
- if (config->json_output)
+ fprintf(output, "%s", evsel__name(evsel));
+}
+
+static void print_counter_value_json(struct perf_stat_config *config,
+ struct evsel *evsel, double avg, bool ok)
+{
+ FILE *output = config->output;
+ const char *bad_count = evsel->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED;
+
+ if (ok)
fprintf(output, "\"counter-value\" : \"%f\", ", avg);
else
- fprintf(output, fmt, avg, config->csv_sep);
+ fprintf(output, "\"counter-value\" : \"%s\", ", bad_count);
- if (config->json_output) {
- if (evsel->unit) {
- fprintf(output, "\"unit\" : \"%s\", ",
- evsel->unit);
- }
- } else {
- if (evsel->unit)
- fprintf(output, "%-*s%s",
- config->csv_output ? 0 : config->unit_width,
- evsel->unit, config->csv_sep);
- }
+ if (evsel->unit)
+ fprintf(output, "\"unit\" : \"%s\", ", evsel->unit);
+ fprintf(output, "\"event\" : \"%s\", ", evsel__name(evsel));
+}
+
+static void print_counter_value(struct perf_stat_config *config,
+ struct evsel *evsel, double avg, bool ok)
+{
if (config->json_output)
- fprintf(output, "\"event\" : \"%s\", ", evsel__name(evsel));
+ print_counter_value_json(config, evsel, avg, ok);
+ else if (config->csv_output)
+ print_counter_value_csv(config, evsel, avg, ok);
else
- fprintf(output, "%-*s", config->csv_output ? 0 : 32, evsel__name(evsel));
+ print_counter_value_std(config, evsel, avg, ok);
+}
- print_cgroup(config, evsel);
+static void abs_printout(struct perf_stat_config *config,
+ struct aggr_cpu_id id, int nr,
+ struct evsel *evsel, double avg, bool ok)
+{
+ aggr_printout(config, evsel, id, nr);
+ print_counter_value(config, evsel, avg, ok);
+ print_cgroup(config, evsel->cgrp);
}
static bool is_mixed_hw_group(struct evsel *counter)
return false;
}
-static void printout(struct perf_stat_config *config, struct aggr_cpu_id id, int nr,
- struct evsel *counter, double uval,
- char *prefix, u64 run, u64 ena, double noise,
- struct runtime_stat *st)
+static void printout(struct perf_stat_config *config, struct outstate *os,
+ double uval, u64 run, u64 ena, double noise, int map_idx)
{
struct perf_stat_output_ctx out;
- struct outstate os = {
- .fh = config->output,
- .prefix = prefix ? prefix : "",
- .id = id,
- .nr = nr,
- .evsel = counter,
- };
print_metric_t pm;
new_line_t nl;
+ bool ok = true;
+ struct evsel *counter = os->evsel;
if (config->csv_output) {
- static const int aggr_fields[AGGR_MAX] = {
- [AGGR_NONE] = 1,
- [AGGR_GLOBAL] = 0,
- [AGGR_SOCKET] = 2,
- [AGGR_DIE] = 2,
- [AGGR_CORE] = 2,
- [AGGR_THREAD] = 1,
- [AGGR_UNSET] = 0,
- [AGGR_NODE] = 1,
- };
-
pm = config->metric_only ? print_metric_only_csv : print_metric_csv;
nl = config->metric_only ? new_line_metric : new_line_csv;
- os.nfields = 3 + aggr_fields[config->aggr_mode] + (counter->cgrp ? 1 : 0);
+ os->nfields = 4 + (counter->cgrp ? 1 : 0);
} else if (config->json_output) {
pm = config->metric_only ? print_metric_only_json : print_metric_json;
nl = config->metric_only ? new_line_metric : new_line_json;
nl = config->metric_only ? new_line_metric : new_line_std;
}
- if (!config->no_csv_summary && config->csv_output &&
- config->summary && !config->interval) {
- fprintf(config->output, "%16s%s", "summary", config->csv_sep);
- }
-
if (run == 0 || ena == 0 || counter->counts->scaled == -1) {
if (config->metric_only) {
- pm(config, &os, NULL, "", "", 0);
+ pm(config, os, NULL, "", "", 0);
return;
}
- aggr_printout(config, counter, id, nr);
- if (config->json_output) {
- fprintf(config->output, "\"counter-value\" : \"%s\", ",
- counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED);
- } else {
- fprintf(config->output, "%*s%s",
- config->csv_output ? 0 : 18,
- counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
- config->csv_sep);
- }
+ ok = false;
if (counter->supported) {
if (!evlist__has_hybrid(counter->evlist)) {
config->print_mixed_hw_group_error = 1;
}
}
-
- if (config->json_output) {
- fprintf(config->output, "\"unit\" : \"%s\", ", counter->unit);
- } else {
- fprintf(config->output, "%-*s%s",
- config->csv_output ? 0 : config->unit_width,
- counter->unit, config->csv_sep);
- }
-
- if (config->json_output) {
- fprintf(config->output, "\"event\" : \"%s\", ",
- evsel__name(counter));
- } else {
- fprintf(config->output, "%*s",
- config->csv_output ? 0 : -25, evsel__name(counter));
- }
-
- print_cgroup(config, counter);
-
- if (!config->csv_output && !config->json_output)
- pm(config, &os, NULL, NULL, "", 0);
- print_noise(config, counter, noise);
- print_running(config, run, ena);
- if (config->csv_output)
- pm(config, &os, NULL, NULL, "", 0);
- else if (config->json_output)
- pm(config, &os, NULL, NULL, "", 0);
- return;
}
- if (!config->metric_only)
- abs_printout(config, id, nr, counter, uval);
-
out.print_metric = pm;
out.new_line = nl;
- out.ctx = &os;
+ out.ctx = os;
out.force_header = false;
- if (config->csv_output && !config->metric_only) {
- print_noise(config, counter, noise);
- print_running(config, run, ena);
- } else if (config->json_output && !config->metric_only) {
- print_noise(config, counter, noise);
- print_running(config, run, ena);
- }
+ if (!config->metric_only) {
+ abs_printout(config, os->id, os->nr, counter, uval, ok);
- perf_stat__print_shadow_stats(config, counter, uval,
- first_shadow_map_idx(config, counter, &id),
- &out, &config->metric_events, st);
- if (!config->csv_output && !config->metric_only && !config->json_output) {
- print_noise(config, counter, noise);
- print_running(config, run, ena);
+ print_noise(config, counter, noise, /*before_metric=*/true);
+ print_running(config, run, ena, /*before_metric=*/true);
}
-}
-static void aggr_update_shadow(struct perf_stat_config *config,
- struct evlist *evlist)
-{
- int idx, s;
- struct perf_cpu cpu;
- struct aggr_cpu_id s2, id;
- u64 val;
- struct evsel *counter;
- struct perf_cpu_map *cpus;
+ if (ok) {
+ perf_stat__print_shadow_stats(config, counter, uval, map_idx,
+ &out, &config->metric_events, &rt_stat);
+ } else {
+ pm(config, os, /*color=*/NULL, /*format=*/NULL, /*unit=*/"", /*val=*/0);
+ }
- for (s = 0; s < config->aggr_map->nr; s++) {
- id = config->aggr_map->map[s];
- evlist__for_each_entry(evlist, counter) {
- cpus = evsel__cpus(counter);
- val = 0;
- perf_cpu_map__for_each_cpu(cpu, idx, cpus) {
- s2 = config->aggr_get_id(config, cpu);
- if (!aggr_cpu_id__equal(&s2, &id))
- continue;
- val += perf_counts(counter->counts, idx, 0)->val;
- }
- perf_stat__update_shadow_stats(counter, val,
- first_shadow_map_idx(config, counter, &id),
- &rt_stat);
- }
+ if (!config->metric_only) {
+ print_noise(config, counter, noise, /*before_metric=*/false);
+ print_running(config, run, ena, /*before_metric=*/false);
}
}
counter->name = new_name;
}
} else {
- if (perf_pmu__has_hybrid()) {
+ if (evsel__is_hybrid(counter)) {
ret = asprintf(&new_name, "%s/%s/",
counter->pmu_name, counter->name);
} else {
counter->uniquified_name = true;
}
-static void collect_all_aliases(struct perf_stat_config *config, struct evsel *counter,
- void (*cb)(struct perf_stat_config *config, struct evsel *counter, void *data,
- bool first),
- void *data)
-{
- struct evlist *evlist = counter->evlist;
- struct evsel *alias;
-
- alias = list_prepare_entry(counter, &(evlist->core.entries), core.node);
- list_for_each_entry_continue (alias, &evlist->core.entries, core.node) {
- /* Merge events with the same name, etc. but on different PMUs. */
- if (!strcmp(evsel__name(alias), evsel__name(counter)) &&
- alias->scale == counter->scale &&
- alias->cgrp == counter->cgrp &&
- !strcmp(alias->unit, counter->unit) &&
- evsel__is_clock(alias) == evsel__is_clock(counter) &&
- strcmp(alias->pmu_name, counter->pmu_name)) {
- alias->merged_stat = true;
- cb(config, alias, data, false);
- }
- }
-}
-
-static bool is_uncore(struct evsel *evsel)
+static bool hybrid_uniquify(struct evsel *evsel, struct perf_stat_config *config)
{
- struct perf_pmu *pmu = evsel__find_pmu(evsel);
-
- return pmu && pmu->is_uncore;
+ return evsel__is_hybrid(evsel) && !config->hybrid_merge;
}
-static bool hybrid_uniquify(struct evsel *evsel)
+static void uniquify_counter(struct perf_stat_config *config, struct evsel *counter)
{
- return perf_pmu__has_hybrid() && !is_uncore(evsel);
+ if (config->no_merge || hybrid_uniquify(counter, config))
+ uniquify_event_name(counter);
}
-static bool hybrid_merge(struct evsel *counter, struct perf_stat_config *config,
- bool check)
+static void print_counter_aggrdata(struct perf_stat_config *config,
+ struct evsel *counter, int s,
+ struct outstate *os)
{
- if (hybrid_uniquify(counter)) {
- if (check)
- return config && config->hybrid_merge;
- else
- return config && !config->hybrid_merge;
- }
+ FILE *output = config->output;
+ u64 ena, run, val;
+ double uval;
+ struct perf_stat_evsel *ps = counter->stats;
+ struct perf_stat_aggr *aggr = &ps->aggr[s];
+ struct aggr_cpu_id id = config->aggr_map->map[s];
+ double avg = aggr->counts.val;
+ bool metric_only = config->metric_only;
- return false;
-}
+ os->id = id;
+ os->nr = aggr->nr;
+ os->evsel = counter;
-static bool collect_data(struct perf_stat_config *config, struct evsel *counter,
- void (*cb)(struct perf_stat_config *config, struct evsel *counter, void *data,
- bool first),
- void *data)
-{
+ /* Skip already merged uncore/hybrid events */
if (counter->merged_stat)
- return false;
- cb(config, counter, data, true);
- if (config->no_merge || hybrid_merge(counter, config, false))
- uniquify_event_name(counter);
- else if (counter->auto_merge_stats || hybrid_merge(counter, config, true))
- collect_all_aliases(config, counter, cb, data);
- return true;
-}
+ return;
-struct aggr_data {
- u64 ena, run, val;
- struct aggr_cpu_id id;
- int nr;
- int cpu_map_idx;
-};
+ uniquify_counter(config, counter);
-static void aggr_cb(struct perf_stat_config *config,
- struct evsel *counter, void *data, bool first)
-{
- struct aggr_data *ad = data;
- int idx;
- struct perf_cpu cpu;
- struct perf_cpu_map *cpus;
- struct aggr_cpu_id s2;
+ val = aggr->counts.val;
+ ena = aggr->counts.ena;
+ run = aggr->counts.run;
- cpus = evsel__cpus(counter);
- perf_cpu_map__for_each_cpu(cpu, idx, cpus) {
- struct perf_counts_values *counts;
+ /*
+ * Skip value 0 when enabling --per-thread globally, otherwise it will
+ * have too many 0 output.
+ */
+ if (val == 0 && config->aggr_mode == AGGR_THREAD && config->system_wide)
+ return;
- s2 = config->aggr_get_id(config, cpu);
- if (!aggr_cpu_id__equal(&s2, &ad->id))
- continue;
- if (first)
- ad->nr++;
- counts = perf_counts(counter->counts, idx, 0);
- /*
- * When any result is bad, make them all to give
- * consistent output in interval mode.
- */
- if (counts->ena == 0 || counts->run == 0 ||
- counter->counts->scaled == -1) {
- ad->ena = 0;
- ad->run = 0;
- break;
- }
- ad->val += counts->val;
- ad->ena += counts->ena;
- ad->run += counts->run;
+ if (!metric_only) {
+ if (config->json_output)
+ fputc('{', output);
+ if (os->prefix)
+ fprintf(output, "%s", os->prefix);
+ else if (config->summary && config->csv_output &&
+ !config->no_csv_summary && !config->interval)
+ fprintf(output, "%s%s", "summary", config->csv_sep);
}
+
+ uval = val * counter->scale;
+
+ printout(config, os, uval, run, ena, avg, s);
+
+ if (!metric_only)
+ fputc('\n', output);
}
-static void print_counter_aggrdata(struct perf_stat_config *config,
- struct evsel *counter, int s,
- char *prefix, bool metric_only,
- bool *first, struct perf_cpu cpu)
+static void print_metric_begin(struct perf_stat_config *config,
+ struct evlist *evlist,
+ struct outstate *os, int aggr_idx)
{
- struct aggr_data ad;
- FILE *output = config->output;
- u64 ena, run, val;
- int nr;
+ struct perf_stat_aggr *aggr;
struct aggr_cpu_id id;
- double uval;
+ struct evsel *evsel;
- ad.id = id = config->aggr_map->map[s];
- ad.val = ad.ena = ad.run = 0;
- ad.nr = 0;
- if (!collect_data(config, counter, aggr_cb, &ad))
+ os->first = true;
+ if (!config->metric_only)
return;
- if (perf_pmu__has_hybrid() && ad.ena == 0)
- return;
+ if (config->json_output)
+ fputc('{', config->output);
+ if (os->prefix)
+ fprintf(config->output, "%s", os->prefix);
- nr = ad.nr;
- ena = ad.ena;
- run = ad.run;
- val = ad.val;
- if (*first && metric_only) {
- *first = false;
- aggr_printout(config, counter, id, nr);
- }
- if (prefix && !metric_only)
- fprintf(output, "%s", prefix);
+ evsel = evlist__first(evlist);
+ id = config->aggr_map->map[aggr_idx];
+ aggr = &evsel->stats->aggr[aggr_idx];
+ aggr_printout(config, evsel, id, aggr->nr);
- uval = val * counter->scale;
- if (cpu.cpu != -1)
- id = aggr_cpu_id__cpu(cpu, /*data=*/NULL);
+ print_cgroup(config, os->cgrp ? : evsel->cgrp);
+}
- printout(config, id, nr, counter, uval,
- prefix, run, ena, 1.0, &rt_stat);
- if (!metric_only)
- fputc('\n', output);
+static void print_metric_end(struct perf_stat_config *config, struct outstate *os)
+{
+ FILE *output = config->output;
+
+ if (!config->metric_only)
+ return;
+
+ if (config->json_output) {
+ if (os->first)
+ fputs("\"metric-value\" : \"none\"", output);
+ fputc('}', output);
+ }
+ fputc('\n', output);
}
static void print_aggr(struct perf_stat_config *config,
struct evlist *evlist,
- char *prefix)
+ struct outstate *os)
{
- bool metric_only = config->metric_only;
- FILE *output = config->output;
struct evsel *counter;
int s;
- bool first;
if (!config->aggr_map || !config->aggr_get_id)
return;
- aggr_update_shadow(config, evlist);
-
/*
* With metric_only everything is on a single line.
* Without each counter has its own line.
*/
for (s = 0; s < config->aggr_map->nr; s++) {
- if (prefix && metric_only)
- fprintf(output, "%s", prefix);
+ print_metric_begin(config, evlist, os, s);
- first = true;
evlist__for_each_entry(evlist, counter) {
- print_counter_aggrdata(config, counter, s,
- prefix, metric_only,
- &first, (struct perf_cpu){ .cpu = -1 });
+ print_counter_aggrdata(config, counter, s, os);
}
- if (metric_only)
- fputc('\n', output);
+ print_metric_end(config, os);
}
}
-static int cmp_val(const void *a, const void *b)
+static void print_aggr_cgroup(struct perf_stat_config *config,
+ struct evlist *evlist,
+ struct outstate *os)
{
- return ((struct perf_aggr_thread_value *)b)->val -
- ((struct perf_aggr_thread_value *)a)->val;
-}
-
-static struct perf_aggr_thread_value *sort_aggr_thread(
- struct evsel *counter,
- int *ret,
- struct target *_target)
-{
- int nthreads = perf_thread_map__nr(counter->core.threads);
- int i = 0;
- double uval;
- struct perf_aggr_thread_value *buf;
-
- buf = calloc(nthreads, sizeof(struct perf_aggr_thread_value));
- if (!buf)
- return NULL;
-
- for (int thread = 0; thread < nthreads; thread++) {
- int idx;
- u64 ena = 0, run = 0, val = 0;
-
- perf_cpu_map__for_each_idx(idx, evsel__cpus(counter)) {
- struct perf_counts_values *counts =
- perf_counts(counter->counts, idx, thread);
-
- val += counts->val;
- ena += counts->ena;
- run += counts->run;
- }
+ struct evsel *counter, *evsel;
+ int s;
- uval = val * counter->scale;
+ if (!config->aggr_map || !config->aggr_get_id)
+ return;
- /*
- * Skip value 0 when enabling --per-thread globally,
- * otherwise too many 0 output.
- */
- if (uval == 0.0 && target__has_per_thread(_target))
+ evlist__for_each_entry(evlist, evsel) {
+ if (os->cgrp == evsel->cgrp)
continue;
- buf[i].counter = counter;
- buf[i].id = aggr_cpu_id__empty();
- buf[i].id.thread_idx = thread;
- buf[i].uval = uval;
- buf[i].val = val;
- buf[i].run = run;
- buf[i].ena = ena;
- i++;
- }
+ os->cgrp = evsel->cgrp;
- qsort(buf, i, sizeof(struct perf_aggr_thread_value), cmp_val);
-
- if (ret)
- *ret = i;
-
- return buf;
-}
-
-static void print_aggr_thread(struct perf_stat_config *config,
- struct target *_target,
- struct evsel *counter, char *prefix)
-{
- FILE *output = config->output;
- int thread, sorted_threads;
- struct aggr_cpu_id id;
- struct perf_aggr_thread_value *buf;
-
- buf = sort_aggr_thread(counter, &sorted_threads, _target);
- if (!buf) {
- perror("cannot sort aggr thread");
- return;
- }
+ for (s = 0; s < config->aggr_map->nr; s++) {
+ print_metric_begin(config, evlist, os, s);
- for (thread = 0; thread < sorted_threads; thread++) {
- if (prefix)
- fprintf(output, "%s", prefix);
+ evlist__for_each_entry(evlist, counter) {
+ if (counter->cgrp != os->cgrp)
+ continue;
- id = buf[thread].id;
- printout(config, id, 0, buf[thread].counter, buf[thread].uval,
- prefix, buf[thread].run, buf[thread].ena, 1.0,
- &rt_stat);
- fputc('\n', output);
+ print_counter_aggrdata(config, counter, s, os);
+ }
+ print_metric_end(config, os);
+ }
}
-
- free(buf);
-}
-
-struct caggr_data {
- double avg, avg_enabled, avg_running;
-};
-
-static void counter_aggr_cb(struct perf_stat_config *config __maybe_unused,
- struct evsel *counter, void *data,
- bool first __maybe_unused)
-{
- struct caggr_data *cd = data;
- struct perf_counts_values *aggr = &counter->counts->aggr;
-
- cd->avg += aggr->val;
- cd->avg_enabled += aggr->ena;
- cd->avg_running += aggr->run;
-}
-
-/*
- * Print out the results of a single counter:
- * aggregated counts in system-wide mode
- */
-static void print_counter_aggr(struct perf_stat_config *config,
- struct evsel *counter, char *prefix)
-{
- bool metric_only = config->metric_only;
- FILE *output = config->output;
- double uval;
- struct caggr_data cd = { .avg = 0.0 };
-
- if (!collect_data(config, counter, counter_aggr_cb, &cd))
- return;
-
- if (prefix && !metric_only)
- fprintf(output, "%s", prefix);
-
- uval = cd.avg * counter->scale;
- printout(config, aggr_cpu_id__empty(), 0, counter, uval, prefix, cd.avg_running,
- cd.avg_enabled, cd.avg, &rt_stat);
- if (!metric_only)
- fprintf(output, "\n");
-}
-
-static void counter_cb(struct perf_stat_config *config __maybe_unused,
- struct evsel *counter, void *data,
- bool first __maybe_unused)
-{
- struct aggr_data *ad = data;
-
- ad->val += perf_counts(counter->counts, ad->cpu_map_idx, 0)->val;
- ad->ena += perf_counts(counter->counts, ad->cpu_map_idx, 0)->ena;
- ad->run += perf_counts(counter->counts, ad->cpu_map_idx, 0)->run;
}
-/*
- * Print out the results of a single counter:
- * does not use aggregated count in system-wide
- */
static void print_counter(struct perf_stat_config *config,
- struct evsel *counter, char *prefix)
+ struct evsel *counter, struct outstate *os)
{
- FILE *output = config->output;
- u64 ena, run, val;
- double uval;
- int idx;
- struct perf_cpu cpu;
- struct aggr_cpu_id id;
-
- perf_cpu_map__for_each_cpu(cpu, idx, evsel__cpus(counter)) {
- struct aggr_data ad = { .cpu_map_idx = idx };
-
- if (!collect_data(config, counter, counter_cb, &ad))
- return;
- val = ad.val;
- ena = ad.ena;
- run = ad.run;
-
- if (prefix)
- fprintf(output, "%s", prefix);
+ int s;
- uval = val * counter->scale;
- id = aggr_cpu_id__cpu(cpu, /*data=*/NULL);
- printout(config, id, 0, counter, uval, prefix,
- run, ena, 1.0, &rt_stat);
+ /* AGGR_THREAD doesn't have config->aggr_get_id */
+ if (!config->aggr_map)
+ return;
- fputc('\n', output);
+ for (s = 0; s < config->aggr_map->nr; s++) {
+ print_counter_aggrdata(config, counter, s, os);
}
}
static void print_no_aggr_metric(struct perf_stat_config *config,
struct evlist *evlist,
- char *prefix)
+ struct outstate *os)
{
int all_idx;
struct perf_cpu cpu;
evlist__for_each_entry(evlist, counter) {
u64 ena, run, val;
double uval;
- struct aggr_cpu_id id;
+ struct perf_stat_evsel *ps = counter->stats;
int counter_idx = perf_cpu_map__idx(evsel__cpus(counter), cpu);
if (counter_idx < 0)
continue;
- id = aggr_cpu_id__cpu(cpu, /*data=*/NULL);
+ os->evsel = counter;
+ os->id = aggr_cpu_id__cpu(cpu, /*data=*/NULL);
if (first) {
- if (prefix)
- fputs(prefix, config->output);
- aggr_printout(config, counter, id, 0);
+ print_metric_begin(config, evlist, os, counter_idx);
first = false;
}
- val = perf_counts(counter->counts, counter_idx, 0)->val;
- ena = perf_counts(counter->counts, counter_idx, 0)->ena;
- run = perf_counts(counter->counts, counter_idx, 0)->run;
+ val = ps->aggr[counter_idx].counts.val;
+ ena = ps->aggr[counter_idx].counts.ena;
+ run = ps->aggr[counter_idx].counts.run;
uval = val * counter->scale;
- printout(config, id, 0, counter, uval, prefix,
- run, ena, 1.0, &rt_stat);
+ printout(config, os, uval, run, ena, 1.0, counter_idx);
}
if (!first)
- fputc('\n', config->output);
+ print_metric_end(config, os);
}
}
-static int aggr_header_lens[] = {
- [AGGR_CORE] = 24,
- [AGGR_DIE] = 18,
- [AGGR_SOCKET] = 12,
- [AGGR_NONE] = 6,
- [AGGR_THREAD] = 24,
- [AGGR_NODE] = 6,
- [AGGR_GLOBAL] = 0,
-};
+static void print_metric_headers_std(struct perf_stat_config *config,
+ bool no_indent)
+{
+ fputc(' ', config->output);
-static const char *aggr_header_csv[] = {
- [AGGR_CORE] = "core,cpus,",
- [AGGR_DIE] = "die,cpus",
- [AGGR_SOCKET] = "socket,cpus",
- [AGGR_NONE] = "cpu,",
- [AGGR_THREAD] = "comm-pid,",
- [AGGR_NODE] = "node,",
- [AGGR_GLOBAL] = ""
-};
+ if (!no_indent) {
+ int len = aggr_header_lens[config->aggr_mode];
+
+ if (nr_cgroups || config->cgroup_list)
+ len += CGROUP_LEN + 1;
+
+ fprintf(config->output, "%*s", len, "");
+ }
+}
+
+static void print_metric_headers_csv(struct perf_stat_config *config,
+ bool no_indent __maybe_unused)
+{
+ if (config->interval)
+ fputs("time,", config->output);
+ if (!config->iostat_run)
+ fputs(aggr_header_csv[config->aggr_mode], config->output);
+}
+
+static void print_metric_headers_json(struct perf_stat_config *config __maybe_unused,
+ bool no_indent __maybe_unused)
+{
+}
static void print_metric_headers(struct perf_stat_config *config,
- struct evlist *evlist,
- const char *prefix, bool no_indent)
+ struct evlist *evlist, bool no_indent)
{
- struct perf_stat_output_ctx out;
struct evsel *counter;
struct outstate os = {
.fh = config->output
};
- bool first = true;
-
- if (config->json_output && !config->interval)
- fprintf(config->output, "{");
+ struct perf_stat_output_ctx out = {
+ .ctx = &os,
+ .print_metric = print_metric_header,
+ .new_line = new_line_metric,
+ .force_header = true,
+ };
- if (prefix && !config->json_output)
- fprintf(config->output, "%s", prefix);
+ if (config->json_output)
+ print_metric_headers_json(config, no_indent);
+ else if (config->csv_output)
+ print_metric_headers_csv(config, no_indent);
+ else
+ print_metric_headers_std(config, no_indent);
- if (!config->csv_output && !no_indent)
- fprintf(config->output, "%*s",
- aggr_header_lens[config->aggr_mode], "");
- if (config->csv_output) {
- if (config->interval)
- fputs("time,", config->output);
- if (!config->iostat_run)
- fputs(aggr_header_csv[config->aggr_mode], config->output);
- }
if (config->iostat_run)
iostat_print_header_prefix(config);
+ if (config->cgroup_list)
+ os.cgrp = evlist__first(evlist)->cgrp;
+
/* Print metrics headers only */
evlist__for_each_entry(evlist, counter) {
os.evsel = counter;
- out.ctx = &os;
- out.print_metric = print_metric_header;
- if (!first && config->json_output)
- fprintf(config->output, ", ");
- first = false;
- out.new_line = new_line_metric;
- out.force_header = true;
+
perf_stat__print_shadow_stats(config, counter, 0,
0,
&out,
&config->metric_events,
&rt_stat);
}
+
+ if (!config->json_output)
+ fputc('\n', config->output);
+}
+
+static void prepare_interval(struct perf_stat_config *config,
+ char *prefix, size_t len, struct timespec *ts)
+{
+ if (config->iostat_run)
+ return;
+
if (config->json_output)
- fprintf(config->output, "}");
- fputc('\n', config->output);
+ scnprintf(prefix, len, "\"interval\" : %lu.%09lu, ",
+ (unsigned long) ts->tv_sec, ts->tv_nsec);
+ else if (config->csv_output)
+ scnprintf(prefix, len, "%lu.%09lu%s",
+ (unsigned long) ts->tv_sec, ts->tv_nsec, config->csv_sep);
+ else
+ scnprintf(prefix, len, "%6lu.%09lu ",
+ (unsigned long) ts->tv_sec, ts->tv_nsec);
}
-static void print_interval(struct perf_stat_config *config,
- struct evlist *evlist,
- char *prefix, struct timespec *ts)
+static void print_header_interval_std(struct perf_stat_config *config,
+ struct target *_target __maybe_unused,
+ struct evlist *evlist,
+ int argc __maybe_unused,
+ const char **argv __maybe_unused)
{
- bool metric_only = config->metric_only;
- unsigned int unit_width = config->unit_width;
FILE *output = config->output;
- static int num_print_interval;
- if (config->interval_clear)
- puts(CONSOLE_CLEAR);
-
- if (!config->iostat_run && !config->json_output)
- sprintf(prefix, "%6lu.%09lu%s", (unsigned long) ts->tv_sec,
- ts->tv_nsec, config->csv_sep);
- if (!config->iostat_run && config->json_output && !config->metric_only)
- sprintf(prefix, "{\"interval\" : %lu.%09lu, ", (unsigned long)
- ts->tv_sec, ts->tv_nsec);
- if (!config->iostat_run && config->json_output && config->metric_only)
- sprintf(prefix, "{\"interval\" : %lu.%09lu}", (unsigned long)
- ts->tv_sec, ts->tv_nsec);
-
- if ((num_print_interval == 0 && !config->csv_output && !config->json_output)
- || config->interval_clear) {
- switch (config->aggr_mode) {
- case AGGR_NODE:
- fprintf(output, "# time node cpus");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- break;
- case AGGR_SOCKET:
- fprintf(output, "# time socket cpus");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- break;
- case AGGR_DIE:
- fprintf(output, "# time die cpus");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- break;
- case AGGR_CORE:
- fprintf(output, "# time core cpus");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- break;
- case AGGR_NONE:
- fprintf(output, "# time CPU ");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- break;
- case AGGR_THREAD:
- fprintf(output, "# time comm-pid");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- break;
- case AGGR_GLOBAL:
- default:
- if (!config->iostat_run) {
- fprintf(output, "# time");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- }
- case AGGR_UNSET:
- case AGGR_MAX:
- break;
- }
+ switch (config->aggr_mode) {
+ case AGGR_NODE:
+ case AGGR_SOCKET:
+ case AGGR_DIE:
+ case AGGR_CORE:
+ fprintf(output, "#%*s %-*s cpus",
+ INTERVAL_LEN - 1, "time",
+ aggr_header_lens[config->aggr_mode],
+ aggr_header_std[config->aggr_mode]);
+ break;
+ case AGGR_NONE:
+ fprintf(output, "#%*s %-*s",
+ INTERVAL_LEN - 1, "time",
+ aggr_header_lens[config->aggr_mode],
+ aggr_header_std[config->aggr_mode]);
+ break;
+ case AGGR_THREAD:
+ fprintf(output, "#%*s %*s-%-*s",
+ INTERVAL_LEN - 1, "time",
+ COMM_LEN, "comm", PID_LEN, "pid");
+ break;
+ case AGGR_GLOBAL:
+ default:
+ if (!config->iostat_run)
+ fprintf(output, "#%*s",
+ INTERVAL_LEN - 1, "time");
+ case AGGR_UNSET:
+ case AGGR_MAX:
+ break;
}
- if ((num_print_interval == 0 || config->interval_clear)
- && metric_only && !config->json_output)
- print_metric_headers(config, evlist, " ", true);
- if ((num_print_interval == 0 || config->interval_clear)
- && metric_only && config->json_output) {
- fprintf(output, "{");
- print_metric_headers(config, evlist, " ", true);
- }
- if (++num_print_interval == 25)
- num_print_interval = 0;
+ if (config->metric_only)
+ print_metric_headers(config, evlist, true);
+ else
+ fprintf(output, " %*s %*s events\n",
+ COUNTS_LEN, "counts", config->unit_width, "unit");
+}
+
+static void print_header_std(struct perf_stat_config *config,
+ struct target *_target, struct evlist *evlist,
+ int argc, const char **argv)
+{
+ FILE *output = config->output;
+ int i;
+
+ fprintf(output, "\n");
+ fprintf(output, " Performance counter stats for ");
+ if (_target->bpf_str)
+ fprintf(output, "\'BPF program(s) %s", _target->bpf_str);
+ else if (_target->system_wide)
+ fprintf(output, "\'system wide");
+ else if (_target->cpu_list)
+ fprintf(output, "\'CPU(s) %s", _target->cpu_list);
+ else if (!target__has_task(_target)) {
+ fprintf(output, "\'%s", argv ? argv[0] : "pipe");
+ for (i = 1; argv && (i < argc); i++)
+ fprintf(output, " %s", argv[i]);
+ } else if (_target->pid)
+ fprintf(output, "process id \'%s", _target->pid);
+ else
+ fprintf(output, "thread id \'%s", _target->tid);
+
+ fprintf(output, "\'");
+ if (config->run_count > 1)
+ fprintf(output, " (%d runs)", config->run_count);
+ fprintf(output, ":\n\n");
+
+ if (config->metric_only)
+ print_metric_headers(config, evlist, false);
+}
+
+static void print_header_csv(struct perf_stat_config *config,
+ struct target *_target __maybe_unused,
+ struct evlist *evlist,
+ int argc __maybe_unused,
+ const char **argv __maybe_unused)
+{
+ if (config->metric_only)
+ print_metric_headers(config, evlist, true);
+}
+static void print_header_json(struct perf_stat_config *config,
+ struct target *_target __maybe_unused,
+ struct evlist *evlist,
+ int argc __maybe_unused,
+ const char **argv __maybe_unused)
+{
+ if (config->metric_only)
+ print_metric_headers(config, evlist, true);
}
static void print_header(struct perf_stat_config *config,
struct target *_target,
+ struct evlist *evlist,
int argc, const char **argv)
{
- FILE *output = config->output;
- int i;
+ static int num_print_iv;
fflush(stdout);
- if (!config->csv_output && !config->json_output) {
- fprintf(output, "\n");
- fprintf(output, " Performance counter stats for ");
- if (_target->bpf_str)
- fprintf(output, "\'BPF program(s) %s", _target->bpf_str);
- else if (_target->system_wide)
- fprintf(output, "\'system wide");
- else if (_target->cpu_list)
- fprintf(output, "\'CPU(s) %s", _target->cpu_list);
- else if (!target__has_task(_target)) {
- fprintf(output, "\'%s", argv ? argv[0] : "pipe");
- for (i = 1; argv && (i < argc); i++)
- fprintf(output, " %s", argv[i]);
- } else if (_target->pid)
- fprintf(output, "process id \'%s", _target->pid);
- else
- fprintf(output, "thread id \'%s", _target->tid);
+ if (config->interval_clear)
+ puts(CONSOLE_CLEAR);
- fprintf(output, "\'");
- if (config->run_count > 1)
- fprintf(output, " (%d runs)", config->run_count);
- fprintf(output, ":\n\n");
+ if (num_print_iv == 0 || config->interval_clear) {
+ if (config->json_output)
+ print_header_json(config, _target, evlist, argc, argv);
+ else if (config->csv_output)
+ print_header_csv(config, _target, evlist, argc, argv);
+ else if (config->interval)
+ print_header_interval_std(config, _target, evlist, argc, argv);
+ else
+ print_header_std(config, _target, evlist, argc, argv);
}
+
+ if (num_print_iv++ == 25)
+ num_print_iv = 0;
}
static int get_precision(double num)
double avg = avg_stats(config->walltime_nsecs_stats) / NSEC_PER_SEC;
FILE *output = config->output;
+ if (config->interval || config->csv_output || config->json_output)
+ return;
+
if (!config->null_run)
fprintf(output, "\n");
fprintf(output, " %17.*f +- %.*f seconds time elapsed",
precision, avg, precision, sd);
- print_noise_pct(config, sd, avg);
+ print_noise_pct(config, sd, avg, /*before_metric=*/false);
}
fprintf(output, "\n\n");
"the same PMU. Try reorganizing the group.\n");
}
-static void print_percore_thread(struct perf_stat_config *config,
- struct evsel *counter, char *prefix)
-{
- int s;
- struct aggr_cpu_id s2, id;
- struct perf_cpu_map *cpus;
- bool first = true;
- int idx;
- struct perf_cpu cpu;
-
- cpus = evsel__cpus(counter);
- perf_cpu_map__for_each_cpu(cpu, idx, cpus) {
- s2 = config->aggr_get_id(config, cpu);
- for (s = 0; s < config->aggr_map->nr; s++) {
- id = config->aggr_map->map[s];
- if (aggr_cpu_id__equal(&s2, &id))
- break;
- }
-
- print_counter_aggrdata(config, counter, s,
- prefix, false,
- &first, cpu);
- }
-}
-
static void print_percore(struct perf_stat_config *config,
- struct evsel *counter, char *prefix)
+ struct evsel *counter, struct outstate *os)
{
bool metric_only = config->metric_only;
FILE *output = config->output;
- int s;
- bool first = true;
+ struct cpu_aggr_map *core_map;
+ int s, c, i;
if (!config->aggr_map || !config->aggr_get_id)
return;
if (config->percore_show_thread)
- return print_percore_thread(config, counter, prefix);
+ return print_counter(config, counter, os);
- for (s = 0; s < config->aggr_map->nr; s++) {
- if (prefix && metric_only)
- fprintf(output, "%s", prefix);
+ core_map = cpu_aggr_map__empty_new(config->aggr_map->nr);
+ if (core_map == NULL) {
+ fprintf(output, "Cannot allocate per-core aggr map for display\n");
+ return;
+ }
+
+ for (s = 0, c = 0; s < config->aggr_map->nr; s++) {
+ struct perf_cpu curr_cpu = config->aggr_map->map[s].cpu;
+ struct aggr_cpu_id core_id = aggr_cpu_id__core(curr_cpu, NULL);
+ bool found = false;
- print_counter_aggrdata(config, counter, s,
- prefix, metric_only,
- &first, (struct perf_cpu){ .cpu = -1 });
+ for (i = 0; i < c; i++) {
+ if (aggr_cpu_id__equal(&core_map->map[i], &core_id)) {
+ found = true;
+ break;
+ }
+ }
+ if (found)
+ continue;
+
+ print_counter_aggrdata(config, counter, s, os);
+
+ core_map->map[c++] = core_id;
}
+ free(core_map);
if (metric_only)
fputc('\n', output);
}
+static void print_cgroup_counter(struct perf_stat_config *config, struct evlist *evlist,
+ struct outstate *os)
+{
+ struct evsel *counter;
+
+ evlist__for_each_entry(evlist, counter) {
+ if (os->cgrp != counter->cgrp) {
+ if (os->cgrp != NULL)
+ print_metric_end(config, os);
+
+ os->cgrp = counter->cgrp;
+ print_metric_begin(config, evlist, os, /*aggr_idx=*/0);
+ }
+
+ print_counter(config, counter, os);
+ }
+ if (os->cgrp)
+ print_metric_end(config, os);
+}
+
void evlist__print_counters(struct evlist *evlist, struct perf_stat_config *config,
- struct target *_target, struct timespec *ts, int argc, const char **argv)
+ struct target *_target, struct timespec *ts,
+ int argc, const char **argv)
{
bool metric_only = config->metric_only;
int interval = config->interval;
struct evsel *counter;
- char buf[64], *prefix = NULL;
+ char buf[64];
+ struct outstate os = {
+ .fh = config->output,
+ .first = true,
+ };
if (config->iostat_run)
evlist->selected = evlist__first(evlist);
- if (interval)
- print_interval(config, evlist, prefix = buf, ts);
- else
- print_header(config, _target, argc, argv);
-
- if (metric_only) {
- static int num_print_iv;
-
- if (num_print_iv == 0 && !interval)
- print_metric_headers(config, evlist, prefix, false);
- if (num_print_iv++ == 25)
- num_print_iv = 0;
- if (config->aggr_mode == AGGR_GLOBAL && prefix && !config->iostat_run)
- fprintf(config->output, "%s", prefix);
-
- if (config->json_output && !config->metric_only)
- fprintf(config->output, "}");
+ if (interval) {
+ os.prefix = buf;
+ prepare_interval(config, buf, sizeof(buf), ts);
}
+ print_header(config, _target, evlist, argc, argv);
+
switch (config->aggr_mode) {
case AGGR_CORE:
case AGGR_DIE:
case AGGR_SOCKET:
case AGGR_NODE:
- print_aggr(config, evlist, prefix);
+ if (config->cgroup_list)
+ print_aggr_cgroup(config, evlist, &os);
+ else
+ print_aggr(config, evlist, &os);
break;
case AGGR_THREAD:
- evlist__for_each_entry(evlist, counter) {
- print_aggr_thread(config, _target, counter, prefix);
- }
- break;
case AGGR_GLOBAL:
- if (config->iostat_run)
- iostat_print_counters(evlist, config, ts, prefix = buf,
- print_counter_aggr);
- else {
+ if (config->iostat_run) {
+ iostat_print_counters(evlist, config, ts, buf,
+ (iostat_print_counter_t)print_counter, &os);
+ } else if (config->cgroup_list) {
+ print_cgroup_counter(config, evlist, &os);
+ } else {
+ print_metric_begin(config, evlist, &os, /*aggr_idx=*/0);
evlist__for_each_entry(evlist, counter) {
- print_counter_aggr(config, counter, prefix);
+ print_counter(config, counter, &os);
}
- if (metric_only)
- fputc('\n', config->output);
+ print_metric_end(config, &os);
}
break;
case AGGR_NONE:
if (metric_only)
- print_no_aggr_metric(config, evlist, prefix);
+ print_no_aggr_metric(config, evlist, &os);
else {
evlist__for_each_entry(evlist, counter) {
if (counter->percore)
- print_percore(config, counter, prefix);
+ print_percore(config, counter, &os);
else
- print_counter(config, counter, prefix);
+ print_counter(config, counter, &os);
}
}
break;
break;
}
- if (!interval && !config->csv_output && !config->json_output)
- print_footer(config);
+ print_footer(config);
fflush(config->output);
}
#include "units.h"
#include <linux/zalloc.h>
#include "iostat.h"
+#include "util/hashmap.h"
/*
* AGGR_GLOBAL: Use CPU 0
#include "evlist.h"
#include "evsel.h"
#include "thread_map.h"
-#ifdef HAVE_LIBBPF_SUPPORT
-#include <bpf/hashmap.h>
-#else
#include "util/hashmap.h"
-#endif
#include <linux/zalloc.h>
void update_stats(struct stats *stats, u64 val)
}
}
+static void evsel__reset_aggr_stats(struct evsel *evsel)
+{
+ struct perf_stat_evsel *ps = evsel->stats;
+ struct perf_stat_aggr *aggr = ps->aggr;
+
+ if (aggr)
+ memset(aggr, 0, sizeof(*aggr) * ps->nr_aggr);
+}
+
static void evsel__reset_stat_priv(struct evsel *evsel)
{
struct perf_stat_evsel *ps = evsel->stats;
init_stats(&ps->res_stats);
+ evsel__reset_aggr_stats(evsel);
+}
+
+static int evsel__alloc_aggr_stats(struct evsel *evsel, int nr_aggr)
+{
+ struct perf_stat_evsel *ps = evsel->stats;
+
+ if (ps == NULL)
+ return 0;
+
+ ps->nr_aggr = nr_aggr;
+ ps->aggr = calloc(nr_aggr, sizeof(*ps->aggr));
+ if (ps->aggr == NULL)
+ return -ENOMEM;
+
+ return 0;
+}
+
+int evlist__alloc_aggr_stats(struct evlist *evlist, int nr_aggr)
+{
+ struct evsel *evsel;
+
+ evlist__for_each_entry(evlist, evsel) {
+ if (evsel__alloc_aggr_stats(evsel, nr_aggr) < 0)
+ return -1;
+ }
+ return 0;
}
-static int evsel__alloc_stat_priv(struct evsel *evsel)
+static int evsel__alloc_stat_priv(struct evsel *evsel, int nr_aggr)
{
- evsel->stats = zalloc(sizeof(struct perf_stat_evsel));
- if (evsel->stats == NULL)
+ struct perf_stat_evsel *ps;
+
+ ps = zalloc(sizeof(*ps));
+ if (ps == NULL)
return -ENOMEM;
+
+ evsel->stats = ps;
+
+ if (nr_aggr && evsel__alloc_aggr_stats(evsel, nr_aggr) < 0) {
+ evsel->stats = NULL;
+ free(ps);
+ return -ENOMEM;
+ }
+
perf_stat_evsel_id_init(evsel);
evsel__reset_stat_priv(evsel);
return 0;
{
struct perf_stat_evsel *ps = evsel->stats;
- if (ps)
+ if (ps) {
+ zfree(&ps->aggr);
zfree(&ps->group_data);
+ }
zfree(&evsel->stats);
}
perf_counts__reset(evsel->prev_raw_counts);
}
-static int evsel__alloc_stats(struct evsel *evsel, bool alloc_raw)
+static int evsel__alloc_stats(struct evsel *evsel, int nr_aggr, bool alloc_raw)
{
- if (evsel__alloc_stat_priv(evsel) < 0 ||
+ if (evsel__alloc_stat_priv(evsel, nr_aggr) < 0 ||
evsel__alloc_counts(evsel) < 0 ||
(alloc_raw && evsel__alloc_prev_raw_counts(evsel) < 0))
return -ENOMEM;
return 0;
}
-int evlist__alloc_stats(struct evlist *evlist, bool alloc_raw)
+int evlist__alloc_stats(struct perf_stat_config *config,
+ struct evlist *evlist, bool alloc_raw)
{
struct evsel *evsel;
+ int nr_aggr = 0;
+
+ if (config && config->aggr_map)
+ nr_aggr = config->aggr_map->nr;
evlist__for_each_entry(evlist, evsel) {
- if (evsel__alloc_stats(evsel, alloc_raw))
+ if (evsel__alloc_stats(evsel, nr_aggr, alloc_raw))
goto out_free;
}
}
}
+void evlist__reset_aggr_stats(struct evlist *evlist)
+{
+ struct evsel *evsel;
+
+ evlist__for_each_entry(evlist, evsel)
+ evsel__reset_aggr_stats(evsel);
+}
+
void evlist__reset_prev_raw_counts(struct evlist *evlist)
{
struct evsel *evsel;
*perf_counts(evsel->prev_raw_counts, idx, thread);
}
}
-
- evsel->counts->aggr = evsel->prev_raw_counts->aggr;
}
void evlist__copy_prev_raw_counts(struct evlist *evlist)
evsel__copy_prev_raw_counts(evsel);
}
-void evlist__save_aggr_prev_raw_counts(struct evlist *evlist)
-{
- struct evsel *evsel;
-
- /*
- * To collect the overall statistics for interval mode,
- * we copy the counts from evsel->prev_raw_counts to
- * evsel->counts. The perf_stat_process_counter creates
- * aggr values from per cpu values, but the per cpu values
- * are 0 for AGGR_GLOBAL. So we use a trick that saves the
- * previous aggr value to the first member of perf_counts,
- * then aggr calculation in process_counter_values can work
- * correctly.
- */
- evlist__for_each_entry(evlist, evsel) {
- *perf_counts(evsel->prev_raw_counts, 0, 0) =
- evsel->prev_raw_counts->aggr;
- }
-}
-
static size_t pkg_id_hash(long __key, void *ctx __maybe_unused)
{
uint64_t *key = (uint64_t *) __key;
return ret;
}
+static bool evsel__count_has_error(struct evsel *evsel,
+ struct perf_counts_values *count,
+ struct perf_stat_config *config)
+{
+ /* the evsel was failed already */
+ if (evsel->err || evsel->counts->scaled == -1)
+ return true;
+
+ /* this is meaningful for CPU aggregation modes only */
+ if (config->aggr_mode == AGGR_GLOBAL)
+ return false;
+
+ /* it's considered ok when it actually ran */
+ if (count->ena != 0 && count->run != 0)
+ return false;
+
+ return true;
+}
+
static int
process_counter_values(struct perf_stat_config *config, struct evsel *evsel,
int cpu_map_idx, int thread,
struct perf_counts_values *count)
{
- struct perf_counts_values *aggr = &evsel->counts->aggr;
+ struct perf_stat_evsel *ps = evsel->stats;
static struct perf_counts_values zero;
bool skip = false;
if (skip)
count = &zero;
- switch (config->aggr_mode) {
- case AGGR_THREAD:
- case AGGR_CORE:
- case AGGR_DIE:
- case AGGR_SOCKET:
- case AGGR_NODE:
- case AGGR_NONE:
- if (!evsel->snapshot)
- evsel__compute_deltas(evsel, cpu_map_idx, thread, count);
- perf_counts_values__scale(count, config->scale, NULL);
- if ((config->aggr_mode == AGGR_NONE) && (!evsel->percore)) {
- perf_stat__update_shadow_stats(evsel, count->val,
- cpu_map_idx, &rt_stat);
- }
+ if (!evsel->snapshot)
+ evsel__compute_deltas(evsel, cpu_map_idx, thread, count);
+ perf_counts_values__scale(count, config->scale, NULL);
+
+ if (config->aggr_mode == AGGR_THREAD) {
+ struct perf_counts_values *aggr_counts = &ps->aggr[thread].counts;
+
+ /*
+ * Skip value 0 when enabling --per-thread globally,
+ * otherwise too many 0 output.
+ */
+ if (count->val == 0 && config->system_wide)
+ return 0;
+
+ ps->aggr[thread].nr++;
+
+ aggr_counts->val += count->val;
+ aggr_counts->ena += count->ena;
+ aggr_counts->run += count->run;
+ return 0;
+ }
- if (config->aggr_mode == AGGR_THREAD) {
- perf_stat__update_shadow_stats(evsel, count->val,
- thread, &rt_stat);
+ if (ps->aggr) {
+ struct perf_cpu cpu = perf_cpu_map__cpu(evsel->core.cpus, cpu_map_idx);
+ struct aggr_cpu_id aggr_id = config->aggr_get_id(config, cpu);
+ struct perf_stat_aggr *ps_aggr;
+ int i;
+
+ for (i = 0; i < ps->nr_aggr; i++) {
+ if (!aggr_cpu_id__equal(&aggr_id, &config->aggr_map->map[i]))
+ continue;
+
+ ps_aggr = &ps->aggr[i];
+ ps_aggr->nr++;
+
+ /*
+ * When any result is bad, make them all to give consistent output
+ * in interval mode. But per-task counters can have 0 enabled time
+ * when some tasks are idle.
+ */
+ if (evsel__count_has_error(evsel, count, config) && !ps_aggr->failed) {
+ ps_aggr->counts.val = 0;
+ ps_aggr->counts.ena = 0;
+ ps_aggr->counts.run = 0;
+ ps_aggr->failed = true;
+ }
+
+ if (!ps_aggr->failed) {
+ ps_aggr->counts.val += count->val;
+ ps_aggr->counts.ena += count->ena;
+ ps_aggr->counts.run += count->run;
+ }
+ break;
}
- break;
- case AGGR_GLOBAL:
- aggr->val += count->val;
- aggr->ena += count->ena;
- aggr->run += count->run;
- case AGGR_UNSET:
- case AGGR_MAX:
- default:
- break;
}
return 0;
int perf_stat_process_counter(struct perf_stat_config *config,
struct evsel *counter)
{
- struct perf_counts_values *aggr = &counter->counts->aggr;
struct perf_stat_evsel *ps = counter->stats;
- u64 *count = counter->counts->aggr.values;
+ u64 *count;
int ret;
- aggr->val = aggr->ena = aggr->run = 0;
-
if (counter->per_pkg)
evsel__zero_per_pkg(counter);
if (config->aggr_mode != AGGR_GLOBAL)
return 0;
- if (!counter->snapshot)
- evsel__compute_deltas(counter, -1, -1, aggr);
- perf_counts_values__scale(aggr, config->scale, &counter->counts->scaled);
-
+ /*
+ * GLOBAL aggregation mode only has a single aggr counts,
+ * so we can use ps->aggr[0] as the actual output.
+ */
+ count = ps->aggr[0].counts.values;
update_stats(&ps->res_stats, *count);
if (verbose > 0) {
evsel__name(counter), count[0], count[1], count[2]);
}
- /*
- * Save the full runtime - to allow normalization during printout:
- */
- perf_stat__update_shadow_stats(counter, *count, 0, &rt_stat);
+ return 0;
+}
+
+static int evsel__merge_aggr_counters(struct evsel *evsel, struct evsel *alias)
+{
+ struct perf_stat_evsel *ps_a = evsel->stats;
+ struct perf_stat_evsel *ps_b = alias->stats;
+ int i;
+
+ if (ps_a->aggr == NULL && ps_b->aggr == NULL)
+ return 0;
+
+ if (ps_a->nr_aggr != ps_b->nr_aggr) {
+ pr_err("Unmatched aggregation mode between aliases\n");
+ return -1;
+ }
+
+ for (i = 0; i < ps_a->nr_aggr; i++) {
+ struct perf_counts_values *aggr_counts_a = &ps_a->aggr[i].counts;
+ struct perf_counts_values *aggr_counts_b = &ps_b->aggr[i].counts;
+
+ /* NB: don't increase aggr.nr for aliases */
+
+ aggr_counts_a->val += aggr_counts_b->val;
+ aggr_counts_a->ena += aggr_counts_b->ena;
+ aggr_counts_a->run += aggr_counts_b->run;
+ }
return 0;
}
+/* events should have the same name, scale, unit, cgroup but on different PMUs */
+static bool evsel__is_alias(struct evsel *evsel_a, struct evsel *evsel_b)
+{
+ if (strcmp(evsel__name(evsel_a), evsel__name(evsel_b)))
+ return false;
+
+ if (evsel_a->scale != evsel_b->scale)
+ return false;
+
+ if (evsel_a->cgrp != evsel_b->cgrp)
+ return false;
+
+ if (strcmp(evsel_a->unit, evsel_b->unit))
+ return false;
+
+ if (evsel__is_clock(evsel_a) != evsel__is_clock(evsel_b))
+ return false;
+
+ return !!strcmp(evsel_a->pmu_name, evsel_b->pmu_name);
+}
+
+static void evsel__merge_aliases(struct evsel *evsel)
+{
+ struct evlist *evlist = evsel->evlist;
+ struct evsel *alias;
+
+ alias = list_prepare_entry(evsel, &(evlist->core.entries), core.node);
+ list_for_each_entry_continue(alias, &evlist->core.entries, core.node) {
+ /* Merge the same events on different PMUs. */
+ if (evsel__is_alias(evsel, alias)) {
+ evsel__merge_aggr_counters(evsel, alias);
+ alias->merged_stat = true;
+ }
+ }
+}
+
+static bool evsel__should_merge_hybrid(const struct evsel *evsel,
+ const struct perf_stat_config *config)
+{
+ return config->hybrid_merge && evsel__is_hybrid(evsel);
+}
+
+static void evsel__merge_stats(struct evsel *evsel, struct perf_stat_config *config)
+{
+ /* this evsel is already merged */
+ if (evsel->merged_stat)
+ return;
+
+ if (evsel->auto_merge_stats || evsel__should_merge_hybrid(evsel, config))
+ evsel__merge_aliases(evsel);
+}
+
+/* merge the same uncore and hybrid events if requested */
+void perf_stat_merge_counters(struct perf_stat_config *config, struct evlist *evlist)
+{
+ struct evsel *evsel;
+
+ if (config->no_merge)
+ return;
+
+ evlist__for_each_entry(evlist, evsel)
+ evsel__merge_stats(evsel, config);
+}
+
+static void evsel__update_percore_stats(struct evsel *evsel, struct aggr_cpu_id *core_id)
+{
+ struct perf_stat_evsel *ps = evsel->stats;
+ struct perf_counts_values counts = { 0, };
+ struct aggr_cpu_id id;
+ struct perf_cpu cpu;
+ int idx;
+
+ /* collect per-core counts */
+ perf_cpu_map__for_each_cpu(cpu, idx, evsel->core.cpus) {
+ struct perf_stat_aggr *aggr = &ps->aggr[idx];
+
+ id = aggr_cpu_id__core(cpu, NULL);
+ if (!aggr_cpu_id__equal(core_id, &id))
+ continue;
+
+ counts.val += aggr->counts.val;
+ counts.ena += aggr->counts.ena;
+ counts.run += aggr->counts.run;
+ }
+
+ /* update aggregated per-core counts for each CPU */
+ perf_cpu_map__for_each_cpu(cpu, idx, evsel->core.cpus) {
+ struct perf_stat_aggr *aggr = &ps->aggr[idx];
+
+ id = aggr_cpu_id__core(cpu, NULL);
+ if (!aggr_cpu_id__equal(core_id, &id))
+ continue;
+
+ aggr->counts.val = counts.val;
+ aggr->counts.ena = counts.ena;
+ aggr->counts.run = counts.run;
+
+ aggr->used = true;
+ }
+}
+
+/* we have an aggr_map for cpu, but want to aggregate the counters per-core */
+static void evsel__process_percore(struct evsel *evsel)
+{
+ struct perf_stat_evsel *ps = evsel->stats;
+ struct aggr_cpu_id core_id;
+ struct perf_cpu cpu;
+ int idx;
+
+ if (!evsel->percore)
+ return;
+
+ perf_cpu_map__for_each_cpu(cpu, idx, evsel->core.cpus) {
+ struct perf_stat_aggr *aggr = &ps->aggr[idx];
+
+ if (aggr->used)
+ continue;
+
+ core_id = aggr_cpu_id__core(cpu, NULL);
+ evsel__update_percore_stats(evsel, &core_id);
+ }
+}
+
+/* process cpu stats on per-core events */
+void perf_stat_process_percore(struct perf_stat_config *config, struct evlist *evlist)
+{
+ struct evsel *evsel;
+
+ if (config->aggr_mode != AGGR_NONE)
+ return;
+
+ evlist__for_each_entry(evlist, evsel)
+ evsel__process_percore(evsel);
+}
+
+static void evsel__update_shadow_stats(struct evsel *evsel)
+{
+ struct perf_stat_evsel *ps = evsel->stats;
+ int i;
+
+ if (ps->aggr == NULL)
+ return;
+
+ for (i = 0; i < ps->nr_aggr; i++) {
+ struct perf_counts_values *aggr_counts = &ps->aggr[i].counts;
+
+ perf_stat__update_shadow_stats(evsel, aggr_counts->val, i, &rt_stat);
+ }
+}
+
+void perf_stat_process_shadow_stats(struct perf_stat_config *config __maybe_unused,
+ struct evlist *evlist)
+{
+ struct evsel *evsel;
+
+ evlist__for_each_entry(evlist, evsel)
+ evsel__update_shadow_stats(evsel);
+}
int perf_event__process_stat_event(struct perf_session *session,
union perf_event *event)
#include <sys/resource.h>
#include "cpumap.h"
#include "rblist.h"
+#include "counts.h"
struct perf_cpu_map;
struct perf_stat_config;
PERF_STAT_EVSEL_ID__MAX,
};
+/* hold aggregated event info */
+struct perf_stat_aggr {
+ /* aggregated values */
+ struct perf_counts_values counts;
+ /* number of entries (CPUs) aggregated */
+ int nr;
+ /* whether any entry has failed to read/process event */
+ bool failed;
+ /* to mark this data is processed already */
+ bool used;
+};
+
+/* per-evsel event stats */
struct perf_stat_evsel {
+ /* used for repeated runs */
struct stats res_stats;
+ /* evsel id for quick check */
enum perf_stat_evsel_id id;
+ /* number of allocated 'aggr' */
+ int nr_aggr;
+ /* aggregated event values */
+ struct perf_stat_aggr *aggr;
+ /* used for group read */
u64 *group_data;
};
bool metric_no_group;
bool metric_no_merge;
bool stop_read_counter;
- bool quiet;
bool iostat_run;
char *user_requested_cpu_list;
bool system_wide;
struct evsel;
struct evlist;
-struct perf_aggr_thread_value {
- struct evsel *counter;
- struct aggr_cpu_id id;
- double uval;
- u64 val;
- u64 run;
- u64 ena;
-};
-
bool __perf_stat_evsel__is(struct evsel *evsel, enum perf_stat_evsel_id id);
#define perf_stat_evsel__is(evsel, id) \
struct runtime_stat *st);
void perf_stat__collect_metric_expr(struct evlist *);
-int evlist__alloc_stats(struct evlist *evlist, bool alloc_raw);
+int evlist__alloc_stats(struct perf_stat_config *config,
+ struct evlist *evlist, bool alloc_raw);
void evlist__free_stats(struct evlist *evlist);
void evlist__reset_stats(struct evlist *evlist);
void evlist__reset_prev_raw_counts(struct evlist *evlist);
void evlist__copy_prev_raw_counts(struct evlist *evlist);
void evlist__save_aggr_prev_raw_counts(struct evlist *evlist);
+int evlist__alloc_aggr_stats(struct evlist *evlist, int nr_aggr);
+void evlist__reset_aggr_stats(struct evlist *evlist);
+
int perf_stat_process_counter(struct perf_stat_config *config,
struct evsel *counter);
+void perf_stat_merge_counters(struct perf_stat_config *config, struct evlist *evlist);
+void perf_stat_process_percore(struct perf_stat_config *config, struct evlist *evlist);
+void perf_stat_process_shadow_stats(struct perf_stat_config *config, struct evlist *evlist);
+
struct perf_tool;
union perf_event;
struct perf_session;
(!used_opd && syms_ss->adjust_symbols)) {
GElf_Phdr phdr;
- if (elf_read_program_header(syms_ss->elf,
+ if (elf_read_program_header(runtime_ss->elf,
(u64)sym.st_value, &phdr)) {
pr_debug4("%s: failed to find program header for "
"symbol: %s st_value: %#" PRIx64 "\n",
s32 socket;
};
+void addr_location__put(struct addr_location *al);
+
int dso__load(struct dso *dso, struct map *map);
int dso__load_vmlinux(struct dso *dso, struct map *map,
const char *vmlinux, bool vmlinux_allocated);
return err;
}
+#ifdef HAVE_LIBTRACEEVENT
int perf_event__synthesize_tracing_data(struct perf_tool *tool, int fd, struct evlist *evlist,
perf_event__handler_t process)
{
return aligned_size;
}
+#endif
int perf_event__synthesize_build_id(struct perf_tool *tool, struct dso *pos, u16 misc,
perf_event__handler_t process, struct machine *machine)
len = pos->long_name_len + 1;
len = PERF_ALIGN(len, NAME_ALIGN);
memcpy(&ev.build_id.build_id, pos->bid.data, sizeof(pos->bid.data));
+ ev.build_id.size = pos->bid.size;
ev.build_id.header.type = PERF_RECORD_HEADER_BUILD_ID;
- ev.build_id.header.misc = misc;
+ ev.build_id.header.misc = misc | PERF_RECORD_MISC_BUILD_ID_SIZE;
ev.build_id.pid = machine->pid;
ev.build_id.header.size = sizeof(ev.build_id) + len;
memcpy(&ev.build_id.filename, pos->long_name, pos->long_name_len);
}
ret += err;
+#ifdef HAVE_LIBTRACEEVENT
if (have_tracepoints(&evlist->core.entries)) {
int fd = perf_data__fd(data);
}
ret += err;
}
+#else
+ (void)data;
+#endif
return ret;
}
void thread__free_stitch_list(struct thread *thread);
+void thread__resolve(struct thread *thread, struct addr_location *al,
+ struct perf_sample *sample);
+
#endif /* __PERF_THREAD_H */
#include "thread_map.h"
#include "debug.h"
#include "event.h"
+#include <internal/threadmap.h>
/* Skip "." and ".." directories */
static int filter(const struct dirent *dir)
#include <sys/types.h>
#include <stdio.h>
-#include <linux/refcount.h>
-#include <internal/threadmap.h>
#include <perf/threadmap.h>
struct perf_record_thread_map;
#include <api/fs/tracing_path.h>
#include "evsel.h"
#include "debug.h"
+#include "util.h"
#define VERSION "0.6"
#define MAX_EVENT_LENGTH 512
struct tracepoint_path *next;
};
-int bigendian(void)
-{
- unsigned char str[] = { 0x1, 0x2, 0x3, 0x4, 0x0, 0x0, 0x0, 0x0};
- unsigned int *ptr;
-
- ptr = (unsigned int *)(void *)str;
- return *ptr == 0x01020304;
-}
-
/* unfortunately, you can not stat debugfs or proc files for size */
static int record_file(const char *file, ssize_t hdr_sz)
{
/* ugh, handle big-endian hdr_size == 4 */
sizep = (char*)&size;
- if (bigendian())
+ if (host_is_bigendian())
sizep += sizeof(u64) - hdr_sz;
if (hdr_sz && pwrite(output_fd, sizep, hdr_sz, hdr_pos) < 0) {
return -1;
/* save endian */
- if (bigendian())
+ if (host_is_bigendian())
buf[0] = 1;
else
buf[0] = 0;
#include "trace-event.h"
#include <linux/ctype.h>
+#include <linux/kernel.h>
+#include <traceevent/event-parse.h>
static int get_common_field(struct scripting_context *context,
int *offset, int *size, const char *type)
#include <sys/stat.h>
#include <sys/wait.h>
#include <sys/mman.h>
+#include <traceevent/event-parse.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include "trace-event.h"
#include "debug.h"
+#include "util.h"
static int input_fd;
return -1;
}
file_bigendian = buf[0];
- host_bigendian = bigendian();
+ host_bigendian = host_is_bigendian() ? 1 : 0;
if (trace_event__init(tevent)) {
pr_debug("trace_event__init failed");
#include <stdlib.h>
#include <string.h>
#include <errno.h>
+#include <traceevent/event-parse.h>
#include "debug.h"
#include "trace-event.h"
-#include "event.h"
#include "evsel.h"
#include <linux/zalloc.h>
+#include "util/sample.h"
struct scripting_context *scripting_context;
// SPDX-License-Identifier: GPL-2.0
-
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#ifndef _PERF_UTIL_TRACE_EVENT_H
#define _PERF_UTIL_TRACE_EVENT_H
-#include <traceevent/event-parse.h>
-#include "parse-events.h"
+#include <stdbool.h>
+#include <stdio.h>
+#include <linux/types.h>
+struct evlist;
struct machine;
struct perf_sample;
union perf_event;
struct tep_plugin_list *plugin_list;
};
+typedef char *(tep_func_resolver_t)(void *priv,
+ unsigned long long *addrp, char **modp);
+
+bool have_tracepoints(struct list_head *evlist);
+
int trace_event__init(struct trace_event *t);
void trace_event__cleanup(struct trace_event *t);
int trace_event__register_resolver(struct machine *machine,
struct tep_event *trace_event__tp_format_id(int id);
-int bigendian(void);
-
void event_format__fprintf(struct tep_event *event,
int cpu, void *data, int size, FILE *fp);
/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef GIT_COMPAT_UTIL_H
-#define GIT_COMPAT_UTIL_H
+#ifndef __PERF_UTIL_H
+#define __PERF_UTIL_H
#define _BSD_SOURCE 1
/* glibc 2.20 deprecates _BSD_SOURCE in favour of _DEFAULT_SOURCE */
0; \
})
-#endif /* GIT_COMPAT_UTIL_H */
+static inline bool host_is_bigendian(void)
+{
+#ifdef __BYTE_ORDER__
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+ return false;
+#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+ return true;
+#else
+#error "Unrecognized __BYTE_ORDER__"
+#endif
+#else /* !__BYTE_ORDER__ */
+ unsigned char str[] = { 0x1, 0x2, 0x3, 0x4, 0x0, 0x0, 0x0, 0x0};
+ unsigned int *ptr;
+
+ ptr = (unsigned int *)(void *)str;
+ return *ptr == 0x01020304;
+#endif
+}
+
+#endif /* __PERF_UTIL_H */
// SPDX-License-Identifier: GPL-2.0+
/*
- * maple_tree.c: Userspace shim for maple tree test-suite
- * Copyright (c) 2018 Liam R. Howlett <Liam.Howlett@Oracle.com>
+ * maple_tree.c: Userspace testing for maple tree test-suite
+ * Copyright (c) 2018-2022 Oracle Corporation
+ * Author: Liam R. Howlett <Liam.Howlett@Oracle.com>
*
* Any tests that require internal knowledge of the tree or threads and other
* difficult to handle in kernel tests.
CONFIG_DYNAMIC_FTRACE=y
CONFIG_FPROBE=y
CONFIG_FTRACE_SYSCALLS=y
+CONFIG_FUNCTION_ERROR_INJECTION=y
CONFIG_FUNCTION_TRACER=y
CONFIG_GENEVE=y
CONFIG_IKCONFIG=y
#include <network_helpers.h>
#include <bpf/btf.h>
#include "bind4_prog.skel.h"
+#include "freplace_progmap.skel.h"
+#include "xdp_dummy.skel.h"
typedef int (*test_cb)(struct bpf_object *obj);
bind4_prog__destroy(skel);
}
+static void test_func_replace_progmap(void)
+{
+ struct bpf_cpumap_val value = { .qsize = 1 };
+ struct freplace_progmap *skel = NULL;
+ struct xdp_dummy *tgt_skel = NULL;
+ __u32 key = 0;
+ int err;
+
+ skel = freplace_progmap__open();
+ if (!ASSERT_OK_PTR(skel, "prog_open"))
+ return;
+
+ tgt_skel = xdp_dummy__open_and_load();
+ if (!ASSERT_OK_PTR(tgt_skel, "tgt_prog_load"))
+ goto out;
+
+ err = bpf_program__set_attach_target(skel->progs.xdp_cpumap_prog,
+ bpf_program__fd(tgt_skel->progs.xdp_dummy_prog),
+ "xdp_dummy_prog");
+ if (!ASSERT_OK(err, "set_attach_target"))
+ goto out;
+
+ err = freplace_progmap__load(skel);
+ if (!ASSERT_OK(err, "obj_load"))
+ goto out;
+
+ /* Prior to fixing the kernel, loading the PROG_TYPE_EXT 'redirect'
+ * program above will cause the map owner type of 'cpumap' to be set to
+ * PROG_TYPE_EXT. This in turn will cause the bpf_map_update_elem()
+ * below to fail, because the program we are inserting into the map is
+ * of PROG_TYPE_XDP. After fixing the kernel, the initial ownership will
+ * be correctly resolved to the *target* of the PROG_TYPE_EXT program
+ * (i.e., PROG_TYPE_XDP) and the map update will succeed.
+ */
+ value.bpf_prog.fd = bpf_program__fd(skel->progs.xdp_drop_prog);
+ err = bpf_map_update_elem(bpf_map__fd(skel->maps.cpu_map),
+ &key, &value, 0);
+ ASSERT_OK(err, "map_update");
+
+out:
+ xdp_dummy__destroy(tgt_skel);
+ freplace_progmap__destroy(skel);
+}
+
/* NOTE: affect other tests, must run in serial mode */
void serial_test_fexit_bpf2bpf(void)
{
test_func_replace_global_func();
if (test__start_subtest("fentry_to_cgroup_bpf"))
test_fentry_to_cgroup_bpf();
+ if (test__start_subtest("func_replace_progmap"))
+ test_func_replace_progmap();
}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+
+struct {
+ __uint(type, BPF_MAP_TYPE_CPUMAP);
+ __type(key, __u32);
+ __type(value, struct bpf_cpumap_val);
+ __uint(max_entries, 1);
+} cpu_map SEC(".maps");
+
+SEC("xdp/cpumap")
+int xdp_drop_prog(struct xdp_md *ctx)
+{
+ return XDP_DROP;
+}
+
+SEC("freplace")
+int xdp_cpumap_prog(struct xdp_md *ctx)
+{
+ return bpf_redirect_map(&cpu_map, 0, XDP_PASS);
+}
+
+char _license[] SEC("license") = "GPL";
SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
int task_untrusted_non_rcuptr(void *ctx)
{
- struct task_struct *task, *last_wakee;
+ struct task_struct *task, *group_leader;
task = bpf_get_current_task_btf();
bpf_rcu_read_lock();
- /* the pointer last_wakee marked as untrusted */
- last_wakee = task->real_parent->last_wakee;
- (void)bpf_task_storage_get(&map_a, last_wakee, 0, 0);
+ /* the pointer group_leader marked as untrusted */
+ group_leader = task->real_parent->group_leader;
+ (void)bpf_task_storage_get(&map_a, group_leader, 0, 0);
bpf_rcu_read_unlock();
return 0;
}
struct task_struct *acquired;
/* Can't invoke bpf_task_acquire() on a trusted pointer obtained from walking a struct. */
- acquired = bpf_task_acquire(task->last_wakee);
+ acquired = bpf_task_acquire(task->group_leader);
bpf_task_release(acquired);
return 0;
bond-lladdr-target.sh \
dev_addr_lists.sh \
mode-1-recovery-updelay.sh \
- mode-2-recovery-updelay.sh
+ mode-2-recovery-updelay.sh \
+ option_prio.sh
TEST_FILES := \
lag_lib.sh \
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Test bonding option prio
+#
+
+ALL_TESTS="
+ prio_arp_ip_target_test
+ prio_miimon_test
+"
+
+REQUIRE_MZ=no
+REQUIRE_JQ=no
+NUM_NETIFS=0
+lib_dir=$(dirname "$0")
+source "$lib_dir"/net_forwarding_lib.sh
+
+destroy()
+{
+ ip link del bond0 &>/dev/null
+ ip link del br0 &>/dev/null
+ ip link del veth0 &>/dev/null
+ ip link del veth1 &>/dev/null
+ ip link del veth2 &>/dev/null
+ ip netns del ns1 &>/dev/null
+ ip link del veth3 &>/dev/null
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ destroy
+}
+
+skip()
+{
+ local skip=1
+ ip link add name bond0 type bond mode 1 miimon 100 &>/dev/null
+ ip link add name veth0 type veth peer name veth0_p
+ ip link set veth0 master bond0
+
+ # check if iproute support prio option
+ ip link set dev veth0 type bond_slave prio 10
+ [[ $? -ne 0 ]] && skip=0
+
+ # check if bonding support prio option
+ ip -d link show veth0 | grep -q "prio 10"
+ [[ $? -ne 0 ]] && skip=0
+
+ ip link del bond0 &>/dev/null
+ ip link del veth0
+
+ return $skip
+}
+
+active_slave=""
+check_active_slave()
+{
+ local target_active_slave=$1
+ active_slave="$(cat /sys/class/net/bond0/bonding/active_slave)"
+ test "$active_slave" = "$target_active_slave"
+ check_err $? "Current active slave is $active_slave but not $target_active_slave"
+}
+
+
+# Test bonding prio option with mode=$mode monitor=$monitor
+# and primary_reselect=$primary_reselect
+prio_test()
+{
+ RET=0
+
+ local monitor=$1
+ local mode=$2
+ local primary_reselect=$3
+
+ local bond_ip4="192.169.1.2"
+ local peer_ip4="192.169.1.1"
+ local bond_ip6="2009:0a:0b::02"
+ local peer_ip6="2009:0a:0b::01"
+
+
+ # create veths
+ ip link add name veth0 type veth peer name veth0_p
+ ip link add name veth1 type veth peer name veth1_p
+ ip link add name veth2 type veth peer name veth2_p
+
+ # create bond
+ if [[ "$monitor" == "miimon" ]];then
+ ip link add name bond0 type bond mode $mode miimon 100 primary veth1 primary_reselect $primary_reselect
+ elif [[ "$monitor" == "arp_ip_target" ]];then
+ ip link add name bond0 type bond mode $mode arp_interval 1000 arp_ip_target $peer_ip4 primary veth1 primary_reselect $primary_reselect
+ elif [[ "$monitor" == "ns_ip6_target" ]];then
+ ip link add name bond0 type bond mode $mode arp_interval 1000 ns_ip6_target $peer_ip6 primary veth1 primary_reselect $primary_reselect
+ fi
+ ip link set bond0 up
+ ip link set veth0 master bond0
+ ip link set veth1 master bond0
+ ip link set veth2 master bond0
+ # check bonding member prio value
+ ip link set dev veth0 type bond_slave prio 0
+ ip link set dev veth1 type bond_slave prio 10
+ ip link set dev veth2 type bond_slave prio 11
+ ip -d link show veth0 | grep -q 'prio 0'
+ check_err $? "veth0 prio is not 0"
+ ip -d link show veth1 | grep -q 'prio 10'
+ check_err $? "veth0 prio is not 10"
+ ip -d link show veth2 | grep -q 'prio 11'
+ check_err $? "veth0 prio is not 11"
+
+ ip link set veth0 up
+ ip link set veth1 up
+ ip link set veth2 up
+ ip link set veth0_p up
+ ip link set veth1_p up
+ ip link set veth2_p up
+
+ # prepare ping target
+ ip link add name br0 type bridge
+ ip link set br0 up
+ ip link set veth0_p master br0
+ ip link set veth1_p master br0
+ ip link set veth2_p master br0
+ ip link add name veth3 type veth peer name veth3_p
+ ip netns add ns1
+ ip link set veth3_p master br0 up
+ ip link set veth3 netns ns1 up
+ ip netns exec ns1 ip addr add $peer_ip4/24 dev veth3
+ ip netns exec ns1 ip addr add $peer_ip6/64 dev veth3
+ ip addr add $bond_ip4/24 dev bond0
+ ip addr add $bond_ip6/64 dev bond0
+ sleep 5
+
+ ping $peer_ip4 -c5 -I bond0 &>/dev/null
+ check_err $? "ping failed 1."
+ ping6 $peer_ip6 -c5 -I bond0 &>/dev/null
+ check_err $? "ping6 failed 1."
+
+ # active salve should be the primary slave
+ check_active_slave veth1
+
+ # active slave should be the higher prio slave
+ ip link set $active_slave down
+ ping $peer_ip4 -c5 -I bond0 &>/dev/null
+ check_err $? "ping failed 2."
+ check_active_slave veth2
+
+ # when only 1 slave is up
+ ip link set $active_slave down
+ ping $peer_ip4 -c5 -I bond0 &>/dev/null
+ check_err $? "ping failed 3."
+ check_active_slave veth0
+
+ # when a higher prio slave change to up
+ ip link set veth2 up
+ ping $peer_ip4 -c5 -I bond0 &>/dev/null
+ check_err $? "ping failed 4."
+ case $primary_reselect in
+ "0")
+ check_active_slave "veth2"
+ ;;
+ "1")
+ check_active_slave "veth0"
+ ;;
+ "2")
+ check_active_slave "veth0"
+ ;;
+ esac
+ local pre_active_slave=$active_slave
+
+ # when the primary slave change to up
+ ip link set veth1 up
+ ping $peer_ip4 -c5 -I bond0 &>/dev/null
+ check_err $? "ping failed 5."
+ case $primary_reselect in
+ "0")
+ check_active_slave "veth1"
+ ;;
+ "1")
+ check_active_slave "$pre_active_slave"
+ ;;
+ "2")
+ check_active_slave "$pre_active_slave"
+ ip link set $active_slave down
+ ping $peer_ip4 -c5 -I bond0 &>/dev/null
+ check_err $? "ping failed 6."
+ check_active_slave "veth1"
+ ;;
+ esac
+
+ # Test changing bond salve prio
+ if [[ "$primary_reselect" == "0" ]];then
+ ip link set dev veth0 type bond_slave prio 1000000
+ ip link set dev veth1 type bond_slave prio 0
+ ip link set dev veth2 type bond_slave prio -50
+ ip -d link show veth0 | grep -q 'prio 1000000'
+ check_err $? "veth0 prio is not 1000000"
+ ip -d link show veth1 | grep -q 'prio 0'
+ check_err $? "veth1 prio is not 0"
+ ip -d link show veth2 | grep -q 'prio -50'
+ check_err $? "veth3 prio is not -50"
+ check_active_slave "veth1"
+
+ ip link set $active_slave down
+ ping $peer_ip4 -c5 -I bond0 &>/dev/null
+ check_err $? "ping failed 7."
+ check_active_slave "veth0"
+ fi
+
+ cleanup
+
+ log_test "prio_test" "Test bonding option 'prio' with mode=$mode monitor=$monitor and primary_reselect=$primary_reselect"
+}
+
+prio_miimon_test()
+{
+ local mode
+ local primary_reselect
+
+ for mode in 1 5 6; do
+ for primary_reselect in 0 1 2; do
+ prio_test "miimon" $mode $primary_reselect
+ done
+ done
+}
+
+prio_arp_ip_target_test()
+{
+ local primary_reselect
+
+ for primary_reselect in 0 1 2; do
+ prio_test "arp_ip_target" 1 $primary_reselect
+ done
+}
+
+if skip;then
+ log_test_skip "option_prio.sh" "Current iproute doesn't support 'prio'."
+ exit 0
+fi
+
+trap cleanup EXIT
+
+tests_run
+
+exit "$EXIT_STATUS"
check_reporter_info dummy healthy 3 3 10 true
- echo 8192> $DEBUGFS_DIR/health/binary_len
- check_fail $? "Failed set dummy reporter binary len to 8192"
+ echo 8192 > $DEBUGFS_DIR/health/binary_len
+ check_err $? "Failed set dummy reporter binary len to 8192"
local dump=$(devlink health dump show $DL_HANDLE reporter dummy -j)
check_err $? "Failed show dump of dummy reporter"
exit 1
fi
+check_netdev_down()
+{
+ state=$(cat /sys/class/net/${NETDEV}/flags)
+
+ if [ $((state & 1)) -ne 0 ]; then
+ echo "WARNING: unexpected interface UP, disable NetworkManager?"
+
+ ip link set dev $NETDEV down
+ fi
+}
+
init_test()
{
RET=0
RET=0
+ check_netdev_down
for trap_name in $(devlink_traps_get); do
devlink_trap_stats_idle_test $trap_name
check_err $? "Stats of trap $trap_name not idle when netdev down"
RET=0
+ check_netdev_down
for group_name in $(devlink_trap_groups_get); do
devlink_trap_group_stats_idle_test $group_name
check_err $? "Stats of trap group $group_name not idle when netdev down"
#!/bin/bash
-#
+# SPDX-License-Identifier: GPL-2.0-or-later OR copyleft-next-0.3.1
# Copyright (C) 2017 Luis R. Rodriguez <mcgrof@kernel.org>
#
-# This program is free software; you can redistribute it and/or modify it
-# under the terms of the GNU General Public License as published by the Free
-# Software Foundation; either version 2 of the License, or at your option any
-# later version; or, when distributed separately from the Linux kernel or
-# when incorporated into other software packages, subject to the following
-# license:
-#
-# This program is free software; you can redistribute it and/or modify it
-# under the terms of copyleft-next (version 0.3.1 or later) as published
-# at http://copyleft-next.org/.
-
# This is a stress test script for kmod, the kernel module loader. It uses
# test_kmod which exposes a series of knobs for the API for us so we can
# tweak each test in userspace rather than in kernelspace.
#include <sys/stat.h>
#include <sys/wait.h>
+#include "reg.h"
#include "utils.h"
#define THREADS 100 /* Max threads */
/* Prilvilege state DSCR access */
inline unsigned long get_dscr(void)
{
- unsigned long ret;
-
- asm volatile("mfspr %0,%1" : "=r" (ret) : "i" (SPRN_DSCR_PRIV));
-
- return ret;
+ return mfspr(SPRN_DSCR_PRIV);
}
inline void set_dscr(unsigned long val)
{
- asm volatile("mtspr %1,%0" : : "r" (val), "i" (SPRN_DSCR_PRIV));
+ mtspr(SPRN_DSCR_PRIV, val);
}
/* Problem state DSCR access */
inline unsigned long get_dscr_usr(void)
{
- unsigned long ret;
-
- asm volatile("mfspr %0,%1" : "=r" (ret) : "i" (SPRN_DSCR));
-
- return ret;
+ return mfspr(SPRN_DSCR);
}
inline void set_dscr_usr(unsigned long val)
{
- asm volatile("mtspr %1,%0" : : "r" (val), "i" (SPRN_DSCR));
+ mtspr(SPRN_DSCR, val);
}
/* Default DSCR access */
rc = read(fd, buf, sizeof(buf));
if (rc == -1) {
perror("read() failed");
+ close(fd);
return 1;
}
close(fd);
if (access(file, F_OK))
continue;
- if (check_cpu_dscr_default(file, val))
+ if (check_cpu_dscr_default(file, val)) {
+ closedir(sysfs);
return 1;
+ }
}
closedir(sysfs);
return 0;
#undef PKEY_DISABLE_EXECUTE
#define PKEY_DISABLE_EXECUTE 0x4
-/* Older versions of libc do not not define this */
+/* Older versions of libc do not define this */
#ifndef SEGV_PKUERR
#define SEGV_PKUERR 4
#endif
core = mmap(NULL, core_size, PROT_READ, MAP_PRIVATE, fd, 0);
if (core == (void *) -1) {
- perror("Error mmaping core file");
+ perror("Error mmapping core file");
ret = TEST_FAIL;
goto out;
}
goto out;
}
- ret = fread(core_pattern, 1, PATH_MAX, f);
+ ret = fread(core_pattern, 1, PATH_MAX - 1, f);
fclose(f);
if (!ret) {
perror("Error reading core_pattern file");
goto out;
}
+ core_pattern[ret] = '\0';
+
/* Check whether we can predict the name of the core file. */
if (!strcmp(core_pattern, "core") || !strcmp(core_pattern, "core.%p"))
*changed_ = false;
* Copyright (C) 2018 Michael Neuling, IBM Corporation.
*/
+#define _GNU_SOURCE
+
#include <unistd.h>
#include <assert.h>
+#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <sys/ioctl.h>
#include <sys/wait.h>
#include <sys/ptrace.h>
+#include <sys/resource.h>
#include <sys/sysinfo.h>
#include <asm/ptrace.h>
#include <elf.h>
static int perf_systemwide_event_open(int *fd, __u32 type, __u64 addr, __u64 len)
{
- int i = 0;
+ int i, ncpus, cpu, ret = 0;
+ struct rlimit rlim;
+ cpu_set_t *mask;
+ size_t size;
+
+ if (getrlimit(RLIMIT_NOFILE, &rlim)) {
+ perror("getrlimit");
+ return -1;
+ }
+ rlim.rlim_cur = 65536;
+ if (setrlimit(RLIMIT_NOFILE, &rlim)) {
+ perror("setrlimit");
+ return -1;
+ }
+
+ ncpus = get_nprocs_conf();
+ size = CPU_ALLOC_SIZE(ncpus);
+ mask = CPU_ALLOC(ncpus);
+ if (!mask) {
+ perror("malloc");
+ return -1;
+ }
+
+ CPU_ZERO_S(size, mask);
- /* Assume online processors are 0 to nprocs for simplisity */
- for (i = 0; i < nprocs; i++) {
- fd[i] = perf_cpu_event_open(i, type, addr, len);
+ if (sched_getaffinity(0, size, mask)) {
+ perror("sched_getaffinity");
+ ret = -1;
+ goto done;
+ }
+
+ for (i = 0, cpu = 0; i < nprocs && cpu < ncpus; cpu++) {
+ if (!CPU_ISSET_S(cpu, size, mask))
+ continue;
+ fd[i] = perf_cpu_event_open(cpu, type, addr, len);
if (fd[i] < 0) {
+ perror("perf_systemwide_event_open");
close_fds(fd, i);
- return fd[i];
+ ret = fd[i];
+ goto done;
}
+ i++;
}
- return 0;
+
+ if (i < nprocs) {
+ printf("Error: Number of online cpus reduced since start of test: %d < %d\n", i, nprocs);
+ close_fds(fd, i);
+ ret = -1;
+ }
+
+done:
+ CPU_FREE(mask);
+ return ret;
}
static inline bool breakpoint_test(int len)
int ret;
ret = perf_systemwide_event_open(fd1, HW_BREAKPOINT_RW, (__u64)&a, (__u64)sizeof(a));
- if (ret) {
- perror("perf_systemwide_event_open");
+ if (ret)
exit(EXIT_FAILURE);
- }
ret = perf_systemwide_event_open(fd2, HW_BREAKPOINT_RW, (__u64)&b, (__u64)sizeof(b));
if (ret) {
close_fds(fd1, nprocs);
- perror("perf_systemwide_event_open");
exit(EXIT_FAILURE);
}
int ret;
ret = perf_systemwide_event_open(fd1, HW_BREAKPOINT_RW, (__u64)&a, (__u64)sizeof(a));
- if (ret) {
- perror("perf_systemwide_event_open");
+ if (ret)
exit(EXIT_FAILURE);
- }
ret = perf_systemwide_event_open(fd2, HW_BREAKPOINT_RW, (__u64)&a, (__u64)sizeof(a));
if (ret) {
close_fds(fd1, nprocs);
- perror("perf_systemwide_event_open");
exit(EXIT_FAILURE);
}
int ret;
ret = perf_systemwide_event_open(fd1, HW_BREAKPOINT_W, (__u64)&a, (__u64)sizeof(a));
- if (ret) {
- perror("perf_systemwide_event_open");
+ if (ret)
exit(EXIT_FAILURE);
- }
ret = perf_systemwide_event_open(fd2, HW_BREAKPOINT_R, (__u64)&b, (__u64)sizeof(b));
if (ret) {
close_fds(fd1, nprocs);
- perror("perf_systemwide_event_open");
exit(EXIT_FAILURE);
}
int ret;
ret = perf_systemwide_event_open(fd1, HW_BREAKPOINT_W, (__u64)&a, (__u64)sizeof(a));
- if (ret) {
- perror("perf_systemwide_event_open");
+ if (ret)
exit(EXIT_FAILURE);
- }
ret = perf_systemwide_event_open(fd2, HW_BREAKPOINT_R, (__u64)&a, (__u64)sizeof(a));
if (ret) {
close_fds(fd1, nprocs);
- perror("perf_systemwide_event_open");
exit(EXIT_FAILURE);
}
#include <sys/syscall.h>
#include <linux/limits.h>
#include "ptrace.h"
+#include "reg.h"
#define SPRN_PVR 0x11F
#define PVR_8xx 0x00500000
int main(int argc, char **argv, char **envp)
{
- int pvr = 0;
- asm __volatile__ ("mfspr %0,%1" : "=r"(pvr) : "i"(SPRN_PVR));
- if (pvr == PVR_8xx)
- is_8xx = true;
+ is_8xx = mfspr(SPRN_PVR) == PVR_8xx;
return test_harness(ptrace_hwbreak, "ptrace-hwbreak");
}
/* Analyse TEXASR after TM failure */
inline unsigned long get_tfiar(void)
{
- unsigned long ret;
-
- asm volatile("mfspr %0,%1" : "=r" (ret) : "i" (SPRN_TFIAR));
- return ret;
+ return mfspr(SPRN_TFIAR);
}
void analyse_texasr(unsigned long texasr)
# for each chip+core combination
# todo - less fragile parsing
-egrep -o 'OCC: Chip [0-9a-f]+ Core [0-9a-f]' < /sys/firmware/opal/msglog |
+grep -E -o 'OCC: Chip [0-9a-f]+ Core [0-9a-f]' < /sys/firmware/opal/msglog |
while read chipcore; do
chip=$(echo "$chipcore"|awk '{print $3}')
core=$(echo "$chipcore"|awk '{print $5}')
#include <string.h>
#include <stdio.h>
#include <sys/utsname.h>
+#include "reg.h"
#include "utils.h"
#include "flush_utils.h"
init = 1;
}
- asm volatile("mtspr %1,%0" : : "r" (val), "i" (SPRN_DSCR));
+ mtspr(SPRN_DSCR, val);
}
#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later OR copyleft-next-0.3.1
# Copyright (C) 2017 Luis R. Rodriguez <mcgrof@kernel.org>
-#
-# This program is free software; you can redistribute it and/or modify it
-# under the terms of the GNU General Public License as published by the Free
-# Software Foundation; either version 2 of the License, or at your option any
-# later version; or, when distributed separately from the Linux kernel or
-# when incorporated into other software packages, subject to the following
-# license:
-#
-# This program is free software; you can redistribute it and/or modify it
-# under the terms of copyleft-next (version 0.3.1 or later) as published
-# at http://copyleft-next.org/.
# This performs a series tests against the proc sysctl interface.
* For more information, please refer to <http://unlicense.org/>
*/
-#define _BSD_SOURCE /* for endian.h */
+/* $(CROSS_COMPILE)cc -g -o aio_simple aio_simple.c -laio */
+
+#define _DEFAULT_SOURCE /* for endian.h */
#include <endian.h>
#include <errno.h>
#define BUF_LEN 8192
+/*
+ * cpu_to_le16/32 are used when initializing structures, a context where a
+ * function call is not allowed. To solve this, we code cpu_to_le16/32 in a way
+ * that allows them to be used when initializing structures.
+ */
+
+#if BYTE_ORDER == __LITTLE_ENDIAN
+#define cpu_to_le16(x) (x)
+#define cpu_to_le32(x) (x)
+#else
+#define cpu_to_le16(x) ((((x) >> 8) & 0xffu) | (((x) & 0xffu) << 8))
+#define cpu_to_le32(x) \
+ ((((x) & 0xff000000u) >> 24) | (((x) & 0x00ff0000u) >> 8) | \
+ (((x) & 0x0000ff00u) << 8) | (((x) & 0x000000ffu) << 24))
+#endif
+
/******************** Descriptors and Strings *******************************/
static const struct {
} __attribute__ ((__packed__)) fs_descs, hs_descs;
} __attribute__ ((__packed__)) descriptors = {
.header = {
- .magic = htole32(FUNCTIONFS_DESCRIPTORS_MAGIC_V2),
- .flags = htole32(FUNCTIONFS_HAS_FS_DESC |
+ .magic = cpu_to_le32(FUNCTIONFS_DESCRIPTORS_MAGIC_V2),
+ .flags = cpu_to_le32(FUNCTIONFS_HAS_FS_DESC |
FUNCTIONFS_HAS_HS_DESC),
- .length = htole32(sizeof(descriptors)),
+ .length = cpu_to_le32(sizeof(descriptors)),
},
- .fs_count = htole32(3),
+ .fs_count = cpu_to_le32(3),
.fs_descs = {
.intf = {
.bLength = sizeof(descriptors.fs_descs.intf),
.bmAttributes = USB_ENDPOINT_XFER_BULK,
},
},
- .hs_count = htole32(3),
+ .hs_count = cpu_to_le32(3),
.hs_descs = {
.intf = {
.bLength = sizeof(descriptors.hs_descs.intf),
.bDescriptorType = USB_DT_ENDPOINT,
.bEndpointAddress = 1 | USB_DIR_IN,
.bmAttributes = USB_ENDPOINT_XFER_BULK,
- .wMaxPacketSize = htole16(512),
+ .wMaxPacketSize = cpu_to_le16(512),
},
.bulk_source = {
.bLength = sizeof(descriptors.hs_descs.bulk_source),
.bDescriptorType = USB_DT_ENDPOINT,
.bEndpointAddress = 2 | USB_DIR_OUT,
.bmAttributes = USB_ENDPOINT_XFER_BULK,
- .wMaxPacketSize = htole16(512),
+ .wMaxPacketSize = cpu_to_le16(512),
},
},
};
} __attribute__ ((__packed__)) lang0;
} __attribute__ ((__packed__)) strings = {
.header = {
- .magic = htole32(FUNCTIONFS_STRINGS_MAGIC),
- .length = htole32(sizeof(strings)),
- .str_count = htole32(1),
- .lang_count = htole32(1),
+ .magic = cpu_to_le32(FUNCTIONFS_STRINGS_MAGIC),
+ .length = cpu_to_le32(sizeof(strings)),
+ .str_count = cpu_to_le32(1),
+ .lang_count = cpu_to_le32(1),
},
.lang0 = {
- htole16(0x0409), /* en-us */
+ cpu_to_le16(0x0409), /* en-us */
STR_INTERFACE,
},
};
projects / platform / kernel / linux-starfive.git / commitdiff